GUC flags
This isn't flagged with GUC_EXPLAIN:
enable_incremental_sort
Here's some more candidates:
shared_buffers - it seems unfortunate this is not included; actually, it seems
like maybe this should be always included - not just if it's set to a
non-default, but especially if it's left at the default..
I suppose it's more important for DML than SELECT.
temp_tablespaces isn't, but temp_buffers is;
autovacuum - if it's off, that can be the cause of the issue (same as force_parallel_mode, which has GUC_EXPLAIN);
max_worker_processes - isn't, but these are: max_parallel_workers, max_parallel_workers_per_gather;
track_io_timing - it can have high overhead;
session_preload_libraries, shared_preload_libraries, local_preload_libraries;
debug_assertions - it's be kind of nice to show this whenever it's true (even thought it's not "changed")
debug_discard_caches
lc_collate and lc_ctype ?
server_version_num - I asked about this in the past, but Tom thought it should
not be included, and I kind of agree, but I'm including it here for
completeness sake
This isn't marked GUC_NOT_IN_SAMPLE, like all other DEVELOPER_OPTIONS:
trace_recovery_messages
I'm not sure jit_tuple_deforming should be marked GUC_NOT_IN_SAMPLE.
I disable this one because it's slow for tables with many attributes.
Same for jit_expressions ?
bgwriter_lru_maxpages should have GUC_UNIT_BLOCKS
max_identifier_length should have BYTES (like log_parameter_max_length and
track_activity_query_size).
block_size and wal_block_size should say BYTES (like wal_segment_size)
And all three descriptions should say ".. in [prefix]bytes" (but see below).
Maybe these should have COMPAT_OPTIONS_PREVIOUS:
integer_datetimes
ssl_renegotiation_limit
autovacuum_freeze_max_age has a comment about pg_resetwal which is obsolete
since 74cf7d46a.
checkpoint_warning refers to "checkpoint segments", which is obsolete since
88e982302.
The attached does the least-disputable, lowest hanging fruit.
More ideas:
Maybe maintenance_io_concurrency should not be GUC_EXPLAIN. But it's used not
only by ANALYZE but also heap_index_delete_tuples.
Should these be GUC_RUNTIME_COMPUTED?
in_hot_standby, data_directory_mode
Since GUC_DISALLOW_IN_FILE effectively implies GUC_NOT_IN_SAMPLE in
src/backend/utils/misc/help_config.c:displayStruct(), many of the redundant
GUC_NOT_IN_SAMPLE could be removed.
I think several things with COMPAT_OPTIONS_PREVIOUS could have
GUC_NO_SHOW_ALL and/or GUC_NOT_IN_SAMPLE.
The GUC descriptions are a hodge podge of full sentences and telegrams.
There's no consistency whether the long description can be read independently
from the short description. For these GUCs, the short description reads more
like a "DETAIL" message:
|trace_recovery_messages, log_min_error_statement, log_min_messages, client_min_messages
|log_transaction_sample_rate, log_statement_sample_rate
|data_directory_mode, log_file_mode, unix_socket_permissions
|log_directory, log_destination, log_line_prefix
|unix_socket_group, default_tablespace, DateStyle, maintenance_work_mem, geqo_generations
For integer/real GUCs, the long description is being used just to describe the
"special" values:
|jit_inline_above_cost, jit_optimize_above_cost, jit_above_cost, log_startup_progress_interval,
|tcp_user_timeout, tcp_keepalives_interval, tcp_keepalives_idle, log_temp_files, old_snapshot_threshold,
|log_parameter_max_length_on_error, log_parameter_max_length, log_autovacuum_min_duration, log_min_duration_sample,
|idle_session_timeout, idle_in_transaction_session_timeout, lock_timeout,
|statement_timeout, shared_memory_size_in_huge_pages
Descriptions of some GUCs describe their default units, but other's don't.
The units are not very important for input, since a non-default unit can be
specified, like SET statement_timeout='1h'. It's important for output, and
SHOW already includes a unit, which may not be the default unit. So I think
the default units should be removed from the descriptions.
This cleanup is similar to GUC categories fixed in a55a98477.
Tom was of the impression that there's more loose ends on that thread.
/messages/by-id/16997-ff16127f6e0d1390@postgresql.org
--
Justin
Attachments:
0001-guc.c-fix-GUC-flags.patchtext/x-diff; charset=us-asciiDownload
From b52bbf317126dbd75a2f8e98ceec4f3beb66f572 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 28 Nov 2021 12:08:48 -0600
Subject: [PATCH 01/10] guc.c: fix GUC flags
---
src/backend/utils/misc/guc.c | 22 +++++++++++-----------
1 file changed, 11 insertions(+), 11 deletions(-)
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index e91d5a3cfd..a37ac8a287 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -1034,7 +1034,8 @@ static struct config_bool ConfigureNamesBool[] =
{
{"enable_incremental_sort", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of incremental sort steps."),
- NULL
+ NULL,
+ GUC_EXPLAIN
},
&enable_incremental_sort,
true,
@@ -3039,7 +3040,8 @@ static struct config_int ConfigureNamesInt[] =
{
{"bgwriter_lru_maxpages", PGC_SIGHUP, RESOURCES_BGWRITER,
gettext_noop("Background writer maximum number of LRU pages to flush per round."),
- NULL
+ NULL,
+ GUC_UNIT_BLOCKS
},
&bgwriter_lru_maxpages,
100, 0, INT_MAX / 2, /* Same upper limit as shared_buffers */
@@ -3188,7 +3190,7 @@ static struct config_int ConfigureNamesInt[] =
{"max_identifier_length", PGC_INTERNAL, PRESET_OPTIONS,
gettext_noop("Shows the maximum identifier length."),
NULL,
- GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+ GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_UNIT_BYTE
},
&max_identifier_length,
NAMEDATALEN - 1, NAMEDATALEN - 1, NAMEDATALEN - 1,
@@ -3199,7 +3201,7 @@ static struct config_int ConfigureNamesInt[] =
{"block_size", PGC_INTERNAL, PRESET_OPTIONS,
gettext_noop("Shows the size of a disk block."),
NULL,
- GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+ GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_UNIT_BYTE
},
&block_size,
BLCKSZ, BLCKSZ, BLCKSZ,
@@ -3221,7 +3223,7 @@ static struct config_int ConfigureNamesInt[] =
{"wal_block_size", PGC_INTERNAL, PRESET_OPTIONS,
gettext_noop("Shows the block size in the write ahead log."),
NULL,
- GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+ GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_UNIT_BYTE
},
&wal_block_size,
XLOG_BLCKSZ, XLOG_BLCKSZ, XLOG_BLCKSZ,
@@ -3298,10 +3300,7 @@ static struct config_int ConfigureNamesInt[] =
},
&autovacuum_freeze_max_age,
- /*
- * see pg_resetwal and vacuum_failsafe_age if you change the
- * upper-limit value.
- */
+ /* see vacuum_failsafe_age if you change the upper-limit value. */
200000000, 100000, 2000000000,
NULL, NULL, NULL
},
@@ -3403,7 +3402,7 @@ static struct config_int ConfigureNamesInt[] =
},
{
- {"ssl_renegotiation_limit", PGC_USERSET, CONN_AUTH_SSL,
+ {"ssl_renegotiation_limit", PGC_USERSET, COMPAT_OPTIONS_PREVIOUS,
gettext_noop("SSL renegotiation is no longer supported; this can only be 0."),
NULL,
GUC_NO_SHOW_ALL | GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE,
@@ -4837,7 +4836,8 @@ static struct config_enum ConfigureNamesEnum[] =
{"trace_recovery_messages", PGC_SIGHUP, DEVELOPER_OPTIONS,
gettext_noop("Enables logging of recovery-related debugging information."),
gettext_noop("Each level includes all the levels that follow it. The later"
- " the level, the fewer messages are sent.")
+ " the level, the fewer messages are sent."),
+ GUC_NOT_IN_SAMPLE,
},
&trace_recovery_messages,
--
2.17.0
0002-guc.c-fix-GUC-descriptions.patchtext/x-diff; charset=us-asciiDownload
From c1a322618816167dfc82b5dff676c16d60963a9f Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 28 Nov 2021 01:35:04 -0600
Subject: [PATCH 02/10] guc.c: fix GUC descriptions
---
src/backend/utils/misc/guc.c | 105 +++++++++++++++++------------------
1 file changed, 51 insertions(+), 54 deletions(-)
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index a37ac8a287..3abd370f9f 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -1224,7 +1224,7 @@ static struct config_bool ConfigureNamesBool[] =
},
{
{"ssl_passphrase_command_supports_reload", PGC_SIGHUP, CONN_AUTH_SSL,
- gettext_noop("Also use ssl_passphrase_command during server reload."),
+ gettext_noop("Controls whether ssl_passphrase_command is used during server reload."),
NULL
},
&ssl_passphrase_command_supports_reload,
@@ -1244,7 +1244,7 @@ static struct config_bool ConfigureNamesBool[] =
{"fsync", PGC_SIGHUP, WAL_SETTINGS,
gettext_noop("Forces synchronization of updates to disk."),
gettext_noop("The server will use the fsync() system call in several places to make "
- "sure that updates are physically written to disk. This insures "
+ "sure that updates are physically written to disk. This ensures "
"that a database cluster will recover to a consistent state after "
"an operating system or hardware crash.")
},
@@ -1283,9 +1283,9 @@ static struct config_bool ConfigureNamesBool[] =
},
{
{"ignore_invalid_pages", PGC_POSTMASTER, DEVELOPER_OPTIONS,
- gettext_noop("Continues recovery after an invalid pages failure."),
- gettext_noop("Detection of WAL records having references to "
- "invalid pages during recovery causes PostgreSQL to "
+ gettext_noop("Continues recovery after an invalid page failure."),
+ gettext_noop("Detection of WAL records which reference "
+ "an invalid page during recovery normally causes PostgreSQL to "
"raise a PANIC-level error, aborting the recovery. "
"Setting ignore_invalid_pages to true causes "
"the system to ignore invalid page references "
@@ -1304,10 +1304,10 @@ static struct config_bool ConfigureNamesBool[] =
{"full_page_writes", PGC_SIGHUP, WAL_SETTINGS,
gettext_noop("Writes full pages to WAL when first modified after a checkpoint."),
gettext_noop("A page write in process during an operating system crash might be "
- "only partially written to disk. During recovery, the row changes "
- "stored in WAL are not enough to recover. This option writes "
- "pages when first modified after a checkpoint to WAL so full recovery "
- "is possible.")
+ "only partially written to disk. Storing individual row changes "
+ "to WAL is not enough to allow recovery. To guarantee successful recovery, "
+ "this option writes entire pages to WAL following their first modification after each"
+ "checkpoint.")
},
&fullPageWrites,
true,
@@ -1364,7 +1364,7 @@ static struct config_bool ConfigureNamesBool[] =
},
{
{"log_disconnections", PGC_SU_BACKEND, LOGGING_WHAT,
- gettext_noop("Logs end of a session, including duration."),
+ gettext_noop("Logs end of sessions, including duration."),
NULL
},
&Log_disconnections,
@@ -1521,7 +1521,7 @@ static struct config_bool ConfigureNamesBool[] =
{
{"track_activities", PGC_SUSET, STATS_COLLECTOR,
gettext_noop("Collects information about executing commands."),
- gettext_noop("Enables the collection of information on the currently "
+ gettext_noop("Enables the collection of information about the currently "
"executing command of each session, along with "
"the time at which that command began execution.")
},
@@ -1747,7 +1747,7 @@ static struct config_bool ConfigureNamesBool[] =
{
{"array_nulls", PGC_USERSET, COMPAT_OPTIONS_PREVIOUS,
gettext_noop("Enable input of NULL elements in arrays."),
- gettext_noop("When turned on, unquoted NULL in an array input "
+ gettext_noop("When turned on, an unquoted NULL in an array input "
"value means a null value; "
"otherwise it is taken literally.")
},
@@ -1940,7 +1940,7 @@ static struct config_bool ConfigureNamesBool[] =
{
{"allow_system_table_mods", PGC_SUSET, DEVELOPER_OPTIONS,
- gettext_noop("Allows modifications of the structure of system tables."),
+ gettext_noop("Allows modifications to the structure of system tables."),
NULL,
GUC_NOT_IN_SAMPLE
},
@@ -2391,8 +2391,8 @@ static struct config_int ConfigureNamesInt[] =
"permission set. The parameter value is expected "
"to be a numeric mode specification in the form "
"accepted by the chmod and umask system calls. "
- "(To use the customary octal format the number must "
- "start with a 0 (zero).)")
+ "To use the customary octal format, the number must "
+ "start with a 0 (zero).")
},
&Unix_socket_permissions,
0777, 0000, 0777,
@@ -2405,8 +2405,8 @@ static struct config_int ConfigureNamesInt[] =
gettext_noop("The parameter value is expected "
"to be a numeric mode specification in the form "
"accepted by the chmod and umask system calls. "
- "(To use the customary octal format the number must "
- "start with a 0 (zero).)")
+ "To use the customary octal format, the number must "
+ "start with a 0 (zero).")
},
&Log_file_mode,
0600, 0000, 0777,
@@ -2417,10 +2417,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"data_directory_mode", PGC_INTERNAL, PRESET_OPTIONS,
gettext_noop("Shows the mode of the data directory."),
- gettext_noop("The parameter value is a numeric mode specification "
- "in the form accepted by the chmod and umask system "
- "calls. (To use the customary octal format the number "
- "must start with a 0 (zero).)"),
+ gettext_noop("The parameter value is shown in the customary octal format."),
GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_RUNTIME_COMPUTED
},
&data_directory_mode,
@@ -2482,7 +2479,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"temp_file_limit", PGC_SUSET, RESOURCES_DISK,
- gettext_noop("Limits the total size of all temporary files used by each process."),
+ gettext_noop("Limits the total size of all temporary files used by each server process."),
gettext_noop("-1 means no limit."),
GUC_UNIT_KB
},
@@ -2729,7 +2726,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"max_pred_locks_per_relation", PGC_SIGHUP, LOCK_MANAGEMENT,
gettext_noop("Sets the maximum number of predicate-locked pages and tuples per relation."),
- gettext_noop("If more than this total of pages and tuples in the same relation are locked "
+ gettext_noop("If more than this total number of pages and tuples in the same relation are locked "
"by a connection, those locks are replaced by a relation-level lock.")
},
&max_predicate_locks_per_relation,
@@ -2784,8 +2781,8 @@ static struct config_int ConfigureNamesInt[] =
{
{"min_wal_size", PGC_SIGHUP, WAL_CHECKPOINTS,
- gettext_noop("Sets the minimum size to shrink the WAL to."),
- NULL,
+ gettext_noop("Sets the minimum size of the WAL."),
+ gettext_noop("The WAL will not shrink below this size."),
GUC_UNIT_MB
},
&min_wal_size_mb,
@@ -2819,10 +2816,10 @@ static struct config_int ConfigureNamesInt[] =
{
{"checkpoint_warning", PGC_SIGHUP, WAL_CHECKPOINTS,
- gettext_noop("Enables warnings if checkpoint segments are filled more "
+ gettext_noop("Enables warnings if WAL segments are filled more "
"frequently than this."),
- gettext_noop("Write a message to the server log if checkpoints "
- "caused by the filling of checkpoint segment files happens more "
+ gettext_noop("A message is written to the server log if checkpoints "
+ "caused by the filling of WAL segment files happens more "
"frequently than this number of seconds. Zero turns off the warning."),
GUC_UNIT_S
},
@@ -2844,7 +2841,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"wal_buffers", PGC_POSTMASTER, WAL_SETTINGS,
- gettext_noop("Sets the number of disk-page buffers in shared memory for WAL."),
+ gettext_noop("Sets the number of WAL buffers to keep in shared memory."),
NULL,
GUC_UNIT_XBLOCKS
},
@@ -2945,7 +2942,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"commit_siblings", PGC_USERSET, WAL_SETTINGS,
- gettext_noop("Sets the minimum concurrent open transactions before performing "
+ gettext_noop("Sets the minimum number of concurrent open transactions before performing "
"commit_delay."),
NULL
},
@@ -3199,7 +3196,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"block_size", PGC_INTERNAL, PRESET_OPTIONS,
- gettext_noop("Shows the size of a disk block."),
+ gettext_noop("Shows the size of a disk block in bytes."),
NULL,
GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_UNIT_BYTE
},
@@ -3221,7 +3218,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"wal_block_size", PGC_INTERNAL, PRESET_OPTIONS,
- gettext_noop("Shows the block size in the write ahead log."),
+ gettext_noop("Shows the block size of the write ahead log in bytes."),
NULL,
GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_UNIT_BYTE
},
@@ -3450,7 +3447,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"min_parallel_table_scan_size", PGC_USERSET, QUERY_TUNING_COST,
gettext_noop("Sets the minimum amount of table data for a parallel scan."),
- gettext_noop("If the planner estimates that it will read a number of table pages too small to reach this limit, a parallel scan will not be considered."),
+ gettext_noop("If the planner estimates that it will read fewer than this number of table pages, a parallel scan will not be considered."),
GUC_UNIT_BLOCKS | GUC_EXPLAIN,
},
&min_parallel_table_scan_size,
@@ -3461,7 +3458,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"min_parallel_index_scan_size", PGC_USERSET, QUERY_TUNING_COST,
gettext_noop("Sets the minimum amount of index data for a parallel scan."),
- gettext_noop("If the planner estimates that it will read a number of index pages too small to reach this limit, a parallel scan will not be considered."),
+ gettext_noop("If the planner estimates that it will read fewer than this number of index pages, a parallel scan will not be considered."),
GUC_UNIT_BLOCKS | GUC_EXPLAIN,
},
&min_parallel_index_scan_size,
@@ -3561,7 +3558,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"client_connection_check_interval", PGC_USERSET, CONN_AUTH_SETTINGS,
- gettext_noop("Sets the time interval between checks for disconnection while running queries."),
+ gettext_noop("Sets the time interval between checks for client disconnection while running queries."),
NULL,
GUC_UNIT_MS
},
@@ -3822,7 +3819,7 @@ static struct config_real ConfigureNamesReal[] =
{
{"checkpoint_completion_target", PGC_SIGHUP, WAL_CHECKPOINTS,
- gettext_noop("Time spent flushing dirty buffers during checkpoint, as fraction of checkpoint interval."),
+ gettext_noop("Target fraction of checkpoint interval to spend writing dirty buffers to disk."),
NULL
},
&CheckPointCompletionTarget,
@@ -3842,7 +3839,7 @@ static struct config_real ConfigureNamesReal[] =
{
{"log_transaction_sample_rate", PGC_SUSET, LOGGING_WHEN,
- gettext_noop("Sets the fraction of transactions from which to log all statements."),
+ gettext_noop("Sets the fraction of transactions for which all statements are logged."),
gettext_noop("Use a value between 0.0 (never log) and 1.0 (log all "
"statements for all transactions).")
},
@@ -4043,7 +4040,7 @@ static struct config_string ConfigureNamesString[] =
{
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
- gettext_noop("Sets the default tablespace to create tables and indexes in."),
+ gettext_noop("Sets the default tablespace where tables and indexes are created."),
gettext_noop("An empty string selects the database's default tablespace."),
GUC_IS_NAME
},
@@ -4662,8 +4659,8 @@ static struct config_enum ConfigureNamesEnum[] =
{
{"client_min_messages", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the message levels that are sent to the client."),
- gettext_noop("Each level includes all the levels that follow it. The later"
- " the level, the fewer messages are sent.")
+ gettext_noop("Each level includes all the levels that follow it. Later"
+ " levels send fewer messages.")
},
&client_min_messages,
NOTICE, client_message_level_options,
@@ -4672,7 +4669,7 @@ static struct config_enum ConfigureNamesEnum[] =
{
{"compute_query_id", PGC_SUSET, STATS_MONITORING,
- gettext_noop("Compute query identifiers."),
+ gettext_noop("Enables in-core computation of a query identifier."),
NULL
},
&compute_query_id,
@@ -4682,9 +4679,9 @@ static struct config_enum ConfigureNamesEnum[] =
{
{"constraint_exclusion", PGC_USERSET, QUERY_TUNING_OTHER,
- gettext_noop("Enables the planner to use constraints to optimize queries."),
- gettext_noop("Table scans will be skipped if their constraints"
- " guarantee that no rows match the query."),
+ gettext_noop("Enables the planner's use of constraints to optimize queries."),
+ gettext_noop("Allows a table's scan to be skipped if its constraints"
+ " guarantee that none of its rows match the query."),
GUC_EXPLAIN
},
&constraint_exclusion,
@@ -4748,8 +4745,8 @@ static struct config_enum ConfigureNamesEnum[] =
{
{"log_min_messages", PGC_SUSET, LOGGING_WHEN,
gettext_noop("Sets the message levels that are logged."),
- gettext_noop("Each level includes all the levels that follow it. The later"
- " the level, the fewer messages are sent.")
+ gettext_noop("Each level includes all the levels that follow it. Later"
+ " levels log fewer messages.")
},
&log_min_messages,
WARNING, server_message_level_options,
@@ -4758,9 +4755,9 @@ static struct config_enum ConfigureNamesEnum[] =
{
{"log_min_error_statement", PGC_SUSET, LOGGING_WHEN,
- gettext_noop("Causes all statements generating error at or above this level to be logged."),
- gettext_noop("Each level includes all the levels that follow it. The later"
- " the level, the fewer messages are sent.")
+ gettext_noop("Causes statements generating messages at or above this level to be logged."),
+ gettext_noop("Each level includes all the levels that follow it. Later"
+ " levels log fewer messages.")
},
&log_min_error_statement,
ERROR, server_message_level_options,
@@ -4835,8 +4832,8 @@ static struct config_enum ConfigureNamesEnum[] =
{
{"trace_recovery_messages", PGC_SIGHUP, DEVELOPER_OPTIONS,
gettext_noop("Enables logging of recovery-related debugging information."),
- gettext_noop("Each level includes all the levels that follow it. The later"
- " the level, the fewer messages are sent."),
+ gettext_noop("Each level includes all the levels that follow it. Later"
+ " levels log fewer messages."),
GUC_NOT_IN_SAMPLE,
},
&trace_recovery_messages,
@@ -4932,7 +4929,7 @@ static struct config_enum ConfigureNamesEnum[] =
{
{"huge_pages", PGC_POSTMASTER, RESOURCES_MEM,
- gettext_noop("Use of huge pages on Linux or Windows."),
+ gettext_noop("Enable use of huge pages on Linux or Windows."),
NULL
},
&huge_pages,
@@ -4943,7 +4940,7 @@ static struct config_enum ConfigureNamesEnum[] =
{
{"force_parallel_mode", PGC_USERSET, DEVELOPER_OPTIONS,
gettext_noop("Forces use of parallel query facilities."),
- gettext_noop("If possible, run query using a parallel worker and with parallel restrictions."),
+ gettext_noop("If possible, run queries using a parallel worker and with parallel restrictions."),
GUC_NOT_IN_SAMPLE | GUC_EXPLAIN
},
&force_parallel_mode,
@@ -4963,7 +4960,7 @@ static struct config_enum ConfigureNamesEnum[] =
{
{"plan_cache_mode", PGC_USERSET, QUERY_TUNING_OTHER,
- gettext_noop("Controls the planner's selection of custom or generic plan."),
+ gettext_noop("Controls the planner's selection of custom or generic plans."),
gettext_noop("Prepared statements can have custom and generic plans, and the planner "
"will attempt to choose which is better. This can be set to override "
"the default behavior."),
--
2.17.0
0003-guc.c-do-not-mention-units.patchtext/x-diff; charset=us-asciiDownload
From 69bb61786295891ebbf918384aefea977215d869 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 28 Nov 2021 18:52:58 -0600
Subject: [PATCH 03/10] guc.c: do not mention units..
Since values can be specified in other units. The specified units are the
default when no unit is specified.
Note that PRESET options are an exception: they're used for output only,
so the description *should* include units...
---
src/backend/utils/misc/guc.c | 28 ++++++++++++++--------------
1 file changed, 14 insertions(+), 14 deletions(-)
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 3abd370f9f..96abcfaf90 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -2132,7 +2132,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"archive_timeout", PGC_SIGHUP, WAL_ARCHIVING,
gettext_noop("Forces a switch to the next WAL file if a "
- "new file has not been started within N seconds."),
+ "new file has not been started within this period of time."),
NULL,
GUC_UNIT_S
},
@@ -2142,7 +2142,7 @@ static struct config_int ConfigureNamesInt[] =
},
{
{"post_auth_delay", PGC_BACKEND, DEVELOPER_OPTIONS,
- gettext_noop("Waits N seconds on connection startup after authentication."),
+ gettext_noop("Waits this period of time on connection startup after authentication."),
gettext_noop("This allows attaching a debugger to the process."),
GUC_NOT_IN_SAMPLE | GUC_UNIT_S
},
@@ -2468,7 +2468,7 @@ static struct config_int ConfigureNamesInt[] =
*/
{
{"max_stack_depth", PGC_SUSET, RESOURCES_MEM,
- gettext_noop("Sets the maximum stack depth, in kilobytes."),
+ gettext_noop("Sets the maximum stack depth."),
NULL,
GUC_UNIT_KB
},
@@ -2759,7 +2759,7 @@ static struct config_int ConfigureNamesInt[] =
{
/* Not for general use */
{"pre_auth_delay", PGC_SIGHUP, DEVELOPER_OPTIONS,
- gettext_noop("Waits N seconds on connection startup before authentication."),
+ gettext_noop("Waits this period of time on connection startup before authentication."),
gettext_noop("This allows attaching a debugger to the process."),
GUC_NOT_IN_SAMPLE | GUC_UNIT_S
},
@@ -3003,7 +3003,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"log_parameter_max_length", PGC_SUSET, LOGGING_WHAT,
- gettext_noop("When logging statements, limit logged parameter values to first N bytes."),
+ gettext_noop("When logging statements, limit logged parameter values to this length."),
gettext_noop("-1 to print values in full."),
GUC_UNIT_BYTE
},
@@ -3014,7 +3014,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"log_parameter_max_length_on_error", PGC_USERSET, LOGGING_WHAT,
- gettext_noop("When reporting an error, limit logged parameter values to first N bytes."),
+ gettext_noop("When reporting an error, limit logged parameter values to this length."),
gettext_noop("-1 to print values in full."),
GUC_UNIT_BYTE
},
@@ -3141,7 +3141,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"log_rotation_age", PGC_SIGHUP, LOGGING_WHERE,
- gettext_noop("Automatic log file rotation will occur after N minutes."),
+ gettext_noop("Automatic log file rotation will occur after this period of time."),
NULL,
GUC_UNIT_MIN
},
@@ -3152,7 +3152,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"log_rotation_size", PGC_SIGHUP, LOGGING_WHERE,
- gettext_noop("Automatic log file rotation will occur after N kilobytes."),
+ gettext_noop("Automatic log file rotation will occur when the log exceeds this size."),
NULL,
GUC_UNIT_KB
},
@@ -3196,7 +3196,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"block_size", PGC_INTERNAL, PRESET_OPTIONS,
- gettext_noop("Shows the size of a disk block in bytes."),
+ gettext_noop("Shows the size of a disk block."),
NULL,
GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_UNIT_BYTE
},
@@ -3218,7 +3218,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"wal_block_size", PGC_INTERNAL, PRESET_OPTIONS,
- gettext_noop("Shows the block size of the write ahead log in bytes."),
+ gettext_noop("Shows the block size of the write ahead log."),
NULL,
GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_UNIT_BYTE
},
@@ -3480,7 +3480,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"log_temp_files", PGC_SUSET, LOGGING_WHAT,
- gettext_noop("Log the use of temporary files larger than this number of kilobytes."),
+ gettext_noop("Log the use of temporary files larger than this size."),
gettext_noop("Zero logs all files. The default is -1 (turning this feature off)."),
GUC_UNIT_KB
},
@@ -3491,7 +3491,7 @@ static struct config_int ConfigureNamesInt[] =
{
{"track_activity_query_size", PGC_POSTMASTER, STATS_COLLECTOR,
- gettext_noop("Sets the size reserved for pg_stat_activity.query, in bytes."),
+ gettext_noop("Sets the size reserved for pg_stat_activity.query."),
NULL,
GUC_UNIT_BYTE
},
@@ -3767,7 +3767,7 @@ static struct config_real ConfigureNamesReal[] =
{
{"vacuum_cost_delay", PGC_USERSET, RESOURCES_VACUUM_DELAY,
- gettext_noop("Vacuum cost delay in milliseconds."),
+ gettext_noop("Vacuum cost delay."),
NULL,
GUC_UNIT_MS
},
@@ -3778,7 +3778,7 @@ static struct config_real ConfigureNamesReal[] =
{
{"autovacuum_vacuum_cost_delay", PGC_SIGHUP, AUTOVACUUM,
- gettext_noop("Vacuum cost delay in milliseconds, for autovacuum."),
+ gettext_noop("Vacuum cost delay, for autovacuum."),
NULL,
GUC_UNIT_MS
},
--
2.17.0
On Sun, Nov 28, 2021 at 09:08:33PM -0600, Justin Pryzby wrote:
This isn't flagged with GUC_EXPLAIN:
enable_incremental_sort
Yeah, that's inconsistent.
This isn't marked GUC_NOT_IN_SAMPLE, like all other DEVELOPER_OPTIONS:
trace_recovery_messages
Indeed.
I'm not sure jit_tuple_deforming should be marked GUC_NOT_IN_SAMPLE.
I disable this one because it's slow for tables with many attributes.
Same for jit_expressions ?
That would be consistent. Both are not in postgresql.conf.sample.
bgwriter_lru_maxpages should have GUC_UNIT_BLOCKS
max_identifier_length should have BYTES (like log_parameter_max_length and
track_activity_query_size).block_size and wal_block_size should say BYTES (like wal_segment_size)
And all three descriptions should say ".. in [prefix]bytes" (but see below).
Okay for these.
Maybe these should have COMPAT_OPTIONS_PREVIOUS:
integer_datetimes
ssl_renegotiation_limit
Hmm. Okay as well for integer_datetimes.
autovacuum_freeze_max_age has a comment about pg_resetwal which is obsolete
since 74cf7d46a.checkpoint_warning refers to "checkpoint segments", which is obsolete since
88e982302.
That's part of 0002. That's a bit weird to use now, so I'd agree with
your suggestion to use "WAL segments" instead.
0001, to adjust the units, and 0003, to make the GUC descriptions less
unit-dependent, are good ideas.
- gettext_noop("Use of huge pages on Linux or Windows."),
+ gettext_noop("Enable use of huge pages on Linux or Windows."),
This should be "Enables use of".
{"compute_query_id", PGC_SUSET, STATS_MONITORING,
- gettext_noop("Compute query identifiers."),
+ gettext_noop("Enables in-core computation of a query identifier."),
This could just be "Computes"?
I am not completely sure that all the contents of 0002 are
improvements, but the suggestions done for huge_pages,
ssl_passphrase_command_supports_reload, checkpoint_warning and
commit_siblings seem fine to me.
--
Michael
On Mon, Nov 29, 2021 at 05:04:01PM +0900, Michael Paquier wrote:
0001, to adjust the units, and 0003, to make the GUC descriptions less
unit-dependent, are good ideas.
Actually, after sleeping on it and doing some digging in
codesearch.debian.org, changing the units of max_identifier_length,
block_size and wal_block_size could induce some breakages for anything
using a SHOW command, something that becomes particularly worse now
that SHOW is supported in replication connections, and it would force
clients to know and parse the units of a value. Perhaps I am being
too careful here, but that could harm a lot of users. It is worth
noting that I have found some driver code making use of pg_settings,
which would not be influenced by such a change, but it is unsafe to
assume that everybody does that.
The addition of GUC_EXPLAIN for enable_incremental_sort, the comment
fix for autovacuum_freeze_max_age, the use of COMPAT_OPTIONS_PREVIOUS
for ssl_renegotiation_limit and the addition of GUC_NOT_IN_SAMPLE for
trace_recovery_messages are fine, though.
I am not completely sure that all the contents of 0002 are
improvements, but the suggestions done for huge_pages,
ssl_passphrase_command_supports_reload, checkpoint_warning and
commit_siblings seem fine to me.
Hmm, I think the patched description of checkpoint_warning is not that
much an improvement compared to the current one. While the current
description uses the term "checkpoint segments", which is, I agree,
weird. The new one would lose the term "checkpoint", making the short
description of the parameter lose some of its context.
I have done a full review of the patch set, and applied the obvious
fixes/improvements as of be54551. Attached is an extra patch based on
the contents of the whole set sent upthread:
- Improvement of the description of checkpoint_segments.
- Reworded the description of all parameters using "N units", rather
than just switching to "this period of units". I have been using
something more generic.
Thoughts?
--
Michael
Attachments:
guc-descriptions.patchtext/x-diff; charset=us-asciiDownload
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 25ac4ca85b..927212b8ae 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -2131,8 +2131,8 @@ static struct config_int ConfigureNamesInt[] =
{
{
{"archive_timeout", PGC_SIGHUP, WAL_ARCHIVING,
- gettext_noop("Forces a switch to the next WAL file if a "
- "new file has not been started within N seconds."),
+ gettext_noop("Sets the amount of time to wait before forcing a "
+ "switch to the next WAL file."),
NULL,
GUC_UNIT_S
},
@@ -2142,7 +2142,8 @@ static struct config_int ConfigureNamesInt[] =
},
{
{"post_auth_delay", PGC_BACKEND, DEVELOPER_OPTIONS,
- gettext_noop("Waits N seconds on connection startup after authentication."),
+ gettext_noop("Sets the amount of seconds to wait on connection "
+ "startup after authentication."),
gettext_noop("This allows attaching a debugger to the process."),
GUC_NOT_IN_SAMPLE | GUC_UNIT_S
},
@@ -2762,7 +2763,8 @@ static struct config_int ConfigureNamesInt[] =
{
/* Not for general use */
{"pre_auth_delay", PGC_SIGHUP, DEVELOPER_OPTIONS,
- gettext_noop("Waits N seconds on connection startup before authentication."),
+ gettext_noop("Sets the amount of seconds to wait on connection "
+ "startup before authentication."),
gettext_noop("This allows attaching a debugger to the process."),
GUC_NOT_IN_SAMPLE | GUC_UNIT_S
},
@@ -2819,10 +2821,10 @@ static struct config_int ConfigureNamesInt[] =
{
{"checkpoint_warning", PGC_SIGHUP, WAL_CHECKPOINTS,
- gettext_noop("Enables warnings if checkpoint segments are filled more "
- "frequently than this."),
+ gettext_noop("Sets the maximum time before warning if checkpoints "
+ "triggered by WAL volume happen too frequently."),
gettext_noop("Write a message to the server log if checkpoints "
- "caused by the filling of checkpoint segment files happens more "
+ "caused by the filling of WAL segment files happens more "
"frequently than this number of seconds. Zero turns off the warning."),
GUC_UNIT_S
},
@@ -3006,7 +3008,8 @@ static struct config_int ConfigureNamesInt[] =
{
{"log_parameter_max_length", PGC_SUSET, LOGGING_WHAT,
- gettext_noop("When logging statements, limit logged parameter values to first N bytes."),
+ gettext_noop("Sets the maximum amount of data logged for bind "
+ "parameter values when logging statements."),
gettext_noop("-1 to print values in full."),
GUC_UNIT_BYTE
},
@@ -3017,7 +3020,8 @@ static struct config_int ConfigureNamesInt[] =
{
{"log_parameter_max_length_on_error", PGC_USERSET, LOGGING_WHAT,
- gettext_noop("When reporting an error, limit logged parameter values to first N bytes."),
+ gettext_noop("Sets the maximum amount of data logged for bind "
+ "parameter values when logging statements, on error."),
gettext_noop("-1 to print values in full."),
GUC_UNIT_BYTE
},
@@ -3143,7 +3147,8 @@ static struct config_int ConfigureNamesInt[] =
{
{"log_rotation_age", PGC_SIGHUP, LOGGING_WHERE,
- gettext_noop("Automatic log file rotation will occur after N minutes."),
+ gettext_noop("Sets the maximum amount of time to wait before "
+ "forcing log file rotation."),
NULL,
GUC_UNIT_MIN
},
@@ -3154,7 +3159,8 @@ static struct config_int ConfigureNamesInt[] =
{
{"log_rotation_size", PGC_SIGHUP, LOGGING_WHERE,
- gettext_noop("Automatic log file rotation will occur after N kilobytes."),
+ gettext_noop("Sets the maximum size of log file to reach before "
+ "forcing log file rotation."),
NULL,
GUC_UNIT_KB
},
On Tue, Nov 30, 2021 at 03:36:45PM +0900, Michael Paquier wrote:
- gettext_noop("Forces a switch to the next WAL file if a " - "new file has not been started within N seconds."), + gettext_noop("Sets the amount of time to wait before forcing a " + "switch to the next WAL file."),
..
- gettext_noop("Waits N seconds on connection startup after authentication."), + gettext_noop("Sets the amount of seconds to wait on connection " + "startup after authentication."),
"amount of time", like above.
- gettext_noop("Waits N seconds on connection startup before authentication."), + gettext_noop("Sets the amount of seconds to wait on connection " + "startup before authentication."),
same
{ {"checkpoint_warning", PGC_SIGHUP, WAL_CHECKPOINTS, - gettext_noop("Enables warnings if checkpoint segments are filled more " - "frequently than this."), + gettext_noop("Sets the maximum time before warning if checkpoints " + "triggered by WAL volume happen too frequently."), gettext_noop("Write a message to the server log if checkpoints " - "caused by the filling of checkpoint segment files happens more " + "caused by the filling of WAL segment files happens more "
It should say "happen" , since it's referring to "checkpoints".
That was a pre-existing issue.
{"log_parameter_max_length", PGC_SUSET, LOGGING_WHAT, - gettext_noop("When logging statements, limit logged parameter values to first N bytes."), + gettext_noop("Sets the maximum amount of data logged for bind " + "parameter values when logging statements."),
I think this one should actually say "in bytes" or at least say "maximum
length". It seems unlikely that someone is going to specify this in other
units, and it's confusing to everyone else to refer to "amount of data" instead
of "length in bytes".
{"log_parameter_max_length_on_error", PGC_USERSET, LOGGING_WHAT, - gettext_noop("When reporting an error, limit logged parameter values to first N bytes."), + gettext_noop("Sets the maximum amount of data logged for bind " + "parameter values when logging statements, on error."),
same
- gettext_noop("Automatic log file rotation will occur after N minutes."), + gettext_noop("Sets the maximum amount of time to wait before " + "forcing log file rotation."),
Should it say "maximum" ? Does that mean anything ?
--
Justin
On Wed, Dec 01, 2021 at 01:59:05AM -0600, Justin Pryzby wrote:
On Tue, Nov 30, 2021 at 03:36:45PM +0900, Michael Paquier wrote:
- gettext_noop("Waits N seconds on connection startup before authentication."), + gettext_noop("Sets the amount of seconds to wait on connection " + "startup before authentication."),same
Thanks. This makes things more consistent.
{ {"checkpoint_warning", PGC_SIGHUP, WAL_CHECKPOINTS, - gettext_noop("Enables warnings if checkpoint segments are filled more " - "frequently than this."), + gettext_noop("Sets the maximum time before warning if checkpoints " + "triggered by WAL volume happen too frequently."), gettext_noop("Write a message to the server log if checkpoints " - "caused by the filling of checkpoint segment files happens more " + "caused by the filling of WAL segment files happens more "It should say "happen" , since it's referring to "checkpoints".
That was a pre-existing issue.
Indeed.
{"log_parameter_max_length", PGC_SUSET, LOGGING_WHAT, - gettext_noop("When logging statements, limit logged parameter values to first N bytes."), + gettext_noop("Sets the maximum amount of data logged for bind " + "parameter values when logging statements."),I think this one should actually say "in bytes" or at least say "maximum
length". It seems unlikely that someone is going to specify this in other
units, and it's confusing to everyone else to refer to "amount of data" instead
of "length in bytes".
Okay. Do you like the updated version attached?
- gettext_noop("Automatic log file rotation will occur after N minutes."), + gettext_noop("Sets the maximum amount of time to wait before " + "forcing log file rotation."),Should it say "maximum" ? Does that mean anything ?
To be consistent with the rest of your suggestions, we could use here:
"Sets the amount of time to wait before forcing log file rotation"
Thanks,
--
Michael
Attachments:
guc-descriptions-v2.patchtext/x-diff; charset=us-asciiDownload
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 25ac4ca85b..003d649ff5 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -2131,8 +2131,8 @@ static struct config_int ConfigureNamesInt[] =
{
{
{"archive_timeout", PGC_SIGHUP, WAL_ARCHIVING,
- gettext_noop("Forces a switch to the next WAL file if a "
- "new file has not been started within N seconds."),
+ gettext_noop("Sets the amount of time to wait before forcing a "
+ "switch to the next WAL file."),
NULL,
GUC_UNIT_S
},
@@ -2142,7 +2142,8 @@ static struct config_int ConfigureNamesInt[] =
},
{
{"post_auth_delay", PGC_BACKEND, DEVELOPER_OPTIONS,
- gettext_noop("Waits N seconds on connection startup after authentication."),
+ gettext_noop("Sets the amount of time to wait on connection "
+ "startup after authentication."),
gettext_noop("This allows attaching a debugger to the process."),
GUC_NOT_IN_SAMPLE | GUC_UNIT_S
},
@@ -2762,7 +2763,8 @@ static struct config_int ConfigureNamesInt[] =
{
/* Not for general use */
{"pre_auth_delay", PGC_SIGHUP, DEVELOPER_OPTIONS,
- gettext_noop("Waits N seconds on connection startup before authentication."),
+ gettext_noop("Sets the amount of time to wait on connection "
+ "startup before authentication."),
gettext_noop("This allows attaching a debugger to the process."),
GUC_NOT_IN_SAMPLE | GUC_UNIT_S
},
@@ -2819,10 +2821,10 @@ static struct config_int ConfigureNamesInt[] =
{
{"checkpoint_warning", PGC_SIGHUP, WAL_CHECKPOINTS,
- gettext_noop("Enables warnings if checkpoint segments are filled more "
- "frequently than this."),
+ gettext_noop("Sets the maximum time before warning if checkpoints "
+ "triggered by WAL volume happen too frequently."),
gettext_noop("Write a message to the server log if checkpoints "
- "caused by the filling of checkpoint segment files happens more "
+ "caused by the filling of WAL segment files happen more "
"frequently than this number of seconds. Zero turns off the warning."),
GUC_UNIT_S
},
@@ -3006,7 +3008,8 @@ static struct config_int ConfigureNamesInt[] =
{
{"log_parameter_max_length", PGC_SUSET, LOGGING_WHAT,
- gettext_noop("When logging statements, limit logged parameter values to first N bytes."),
+ gettext_noop("Sets the maximum length in bytes of data logged for bind "
+ "parameter values when logging statements."),
gettext_noop("-1 to print values in full."),
GUC_UNIT_BYTE
},
@@ -3017,7 +3020,8 @@ static struct config_int ConfigureNamesInt[] =
{
{"log_parameter_max_length_on_error", PGC_USERSET, LOGGING_WHAT,
- gettext_noop("When reporting an error, limit logged parameter values to first N bytes."),
+ gettext_noop("Sets the maximum length in bytes of data logged for bind "
+ "parameter values when logging statements, on error."),
gettext_noop("-1 to print values in full."),
GUC_UNIT_BYTE
},
@@ -3143,7 +3147,8 @@ static struct config_int ConfigureNamesInt[] =
{
{"log_rotation_age", PGC_SIGHUP, LOGGING_WHERE,
- gettext_noop("Automatic log file rotation will occur after N minutes."),
+ gettext_noop("Sets the amount of time to wait before forcing "
+ "log file rotation."),
NULL,
GUC_UNIT_MIN
},
@@ -3154,7 +3159,8 @@ static struct config_int ConfigureNamesInt[] =
{
{"log_rotation_size", PGC_SIGHUP, LOGGING_WHERE,
- gettext_noop("Automatic log file rotation will occur after N kilobytes."),
+ gettext_noop("Sets the maximum size of log file to reach before "
+ "forcing log file rotation."),
NULL,
GUC_UNIT_KB
},
@@ -2142,7 +2142,8 @@ static struct config_int ConfigureNamesInt[] = {"post_auth_delay", PGC_BACKEND, DEVELOPER_OPTIONS, - gettext_noop("Waits N seconds on connection startup after authentication."), + gettext_noop("Sets the amount of time to wait on connection " + "startup after authentication."),
@@ -2762,7 +2763,8 @@ static struct config_int ConfigureNamesInt[] = {"pre_auth_delay", PGC_SIGHUP, DEVELOPER_OPTIONS, - gettext_noop("Waits N seconds on connection startup before authentication."), + gettext_noop("Sets the amount of time to wait on connection " + "startup before authentication."), gettext_noop("This allows attaching a debugger to the process."),
I wonder if these should say "Sets the amount of time to wait [before]
authentication during connection startup"
{"checkpoint_warning", PGC_SIGHUP, WAL_CHECKPOINTS, - gettext_noop("Enables warnings if checkpoint segments are filled more " - "frequently than this."), + gettext_noop("Sets the maximum time before warning if checkpoints " + "triggered by WAL volume happen too frequently."), gettext_noop("Write a message to the server log if checkpoints " - "caused by the filling of checkpoint segment files happens more " + "caused by the filling of WAL segment files happen more " "frequently than this number of seconds. Zero turns off the warning."),
Should this still say "seconds" ?
Or change it to "this amount of time"?
I'm not sure.
{"log_rotation_age", PGC_SIGHUP, LOGGING_WHERE, - gettext_noop("Automatic log file rotation will occur after N minutes."), + gettext_noop("Sets the amount of time to wait before forcing " + "log file rotation."), NULL, GUC_UNIT_MIN }, @@ -3154,7 +3159,8 @@ static struct config_int ConfigureNamesInt[] ={ {"log_rotation_size", PGC_SIGHUP, LOGGING_WHERE, - gettext_noop("Automatic log file rotation will occur after N kilobytes."), + gettext_noop("Sets the maximum size of log file to reach before " + "forcing log file rotation."),
Actually, I think that for log_rotation_size, it should not say "forcing".
"Sets the maximum size a log file can reach before being rotated"
BTW the EXPLAIN flag for enable_incremental_sort could be backpatched to v13.
--
Justin
On Wed, Dec 01, 2021 at 09:34:39PM -0600, Justin Pryzby wrote:
@@ -2762,7 +2763,8 @@ static struct config_int ConfigureNamesInt[] = {"pre_auth_delay", PGC_SIGHUP, DEVELOPER_OPTIONS, - gettext_noop("Waits N seconds on connection startup before authentication."), + gettext_noop("Sets the amount of time to wait on connection " + "startup before authentication."), gettext_noop("This allows attaching a debugger to the process."),I wonder if these should say "Sets the amount of time to wait [before]
authentication during connection startup"
Hmm. I don't see much a difference between both of wordings in this
context.
gettext_noop("Write a message to the server log if checkpoints " - "caused by the filling of checkpoint segment files happens more " + "caused by the filling of WAL segment files happen more " "frequently than this number of seconds. Zero turns off the warning."),Should this still say "seconds" ?
Or change it to "this amount of time"?
I'm not sure.
Either way would be fine by me, though I'd agree to be consistent and
use "this amount of time" here.
{"log_rotation_size", PGC_SIGHUP, LOGGING_WHERE, - gettext_noop("Automatic log file rotation will occur after N kilobytes."), + gettext_noop("Sets the maximum size of log file to reach before " + "forcing log file rotation."),Actually, I think that for log_rotation_size, it should not say "forcing".
"Sets the maximum size a log file can reach before being rotated"
Okay. Fine by me.
BTW the EXPLAIN flag for enable_incremental_sort could be backpatched to v13.
This could cause small diffs in EXPLAIN outputs, which could be
surprising. This is not worth taking any risks.
--
Michael
On Thu, Dec 02, 2021 at 02:11:38PM +0900, Michael Paquier wrote:
On Wed, Dec 01, 2021 at 09:34:39PM -0600, Justin Pryzby wrote:
@@ -2762,7 +2763,8 @@ static struct config_int ConfigureNamesInt[] = {"pre_auth_delay", PGC_SIGHUP, DEVELOPER_OPTIONS, - gettext_noop("Waits N seconds on connection startup before authentication."), + gettext_noop("Sets the amount of time to wait on connection " + "startup before authentication."), gettext_noop("This allows attaching a debugger to the process."),I wonder if these should say "Sets the amount of time to wait [before]
authentication during connection startup"Hmm. I don't see much a difference between both of wordings in this
context.
I find it easier to read "wait before authentication ..." than "wait ... before
authentication".
BTW the EXPLAIN flag for enable_incremental_sort could be backpatched to v13.
This could cause small diffs in EXPLAIN outputs, which could be
surprising. This is not worth taking any risks.
Only if one specifies explain(SETTINGS).
It's fine either way ;)
--
Justin
On Wed, Dec 01, 2021 at 11:17:34PM -0600, Justin Pryzby wrote:
I find it easier to read "wait before authentication ..." than "wait ... before
authentication".
I have a hard time seeing a strong difference here. At the end, I
have used what you suggested, adjusted the rest based on your set of
comments, and applied the patch.
--
Michael
On Fri, Dec 03, 2021 at 10:06:47AM +0900, Michael Paquier wrote:
On Wed, Dec 01, 2021 at 11:17:34PM -0600, Justin Pryzby wrote:
I find it easier to read "wait before authentication ..." than "wait ... before
authentication".I have a hard time seeing a strong difference here. At the end, I
have used what you suggested, adjusted the rest based on your set of
comments, and applied the patch.
Thanks. One more item. The check_guc script currently outputs 68 false
positives - even though it includes a list of 20 exceptions. This is not
useful.
$ (cd ./src/backend/utils/misc/; ./check_guc) |wc -l
68
With the attached:
$ (cd ./src/backend/utils/misc/; ./check_guc)
config_file seems to be missing from postgresql.conf.sample
That has a defacto exception for the "include" directive, which seems
reasonable.
This requires GNU awk. I'm not sure if that's a limitation of any
significance.
--
Justin
Attachments:
0001-check_guc-fix-absurd-number-of-false-positives.patchtext/x-diff; charset=us-asciiDownload
From 8d9130a25a59630063e644a884d354eb085faa4e Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 5 Dec 2021 23:22:04 -0600
Subject: [PATCH] check_guc: fix absurd number of false positives
Avoid false positives;
Avoid various assumptions;
Avoid list of exceptions;
Simplify shell/awk/sed/grep;
This requires GNU awk for RS as a regex.
---
src/backend/utils/misc/check_guc | 69 +++++---------------------------
1 file changed, 10 insertions(+), 59 deletions(-)
diff --git a/src/backend/utils/misc/check_guc b/src/backend/utils/misc/check_guc
index b171ef0e4f..323ca13191 100755
--- a/src/backend/utils/misc/check_guc
+++ b/src/backend/utils/misc/check_guc
@@ -1,78 +1,29 @@
-#!/bin/sh
+#! /bin/sh
+set -e
-## currently, this script makes a lot of assumptions:
+## this script makes some assumptions about the formatting of files it parses.
## in postgresql.conf.sample:
## 1) the valid config settings may be preceded by a '#', but NOT '# '
## (we use this to skip comments)
-## 2) the valid config settings will be followed immediately by ' ='
-## (at least one space preceding the '=')
-## in guc.c:
-## 3) the options have PGC_ on the same line as the option
-## 4) the options have '{' on the same line as the option
-
-## Problems
-## 1) Don't know what to do with TRANSACTION ISOLATION LEVEL
-
-## if an option is valid but shows up in only one file (guc.c but not
-## postgresql.conf.sample), it should be listed here so that it
-## can be ignored
-INTENTIONALLY_NOT_INCLUDED="debug_deadlocks in_hot_standby \
-is_superuser lc_collate lc_ctype lc_messages lc_monetary lc_numeric lc_time \
-pre_auth_delay role seed server_encoding server_version server_version_num \
-session_authorization trace_lock_oidmin trace_lock_table trace_locks trace_lwlocks \
-trace_notify trace_userlocks transaction_isolation transaction_read_only \
-zero_damaged_pages"
### What options are listed in postgresql.conf.sample, but don't appear
### in guc.c?
-# grab everything that looks like a setting and convert it to lower case
-SETTINGS=`grep ' =' postgresql.conf.sample |
-grep -v '^# ' | # strip comments
-sed -e 's/^#//' |
-awk '{print $1}'`
-
-SETTINGS=`echo "$SETTINGS" | tr 'A-Z' 'a-z'`
+# grab everything that looks like a setting
+SETTINGS=`sed '/^#[[:alnum:]]/!d; s/^#//; s/ =.*//; /^include/d' postgresql.conf.sample`
for i in $SETTINGS ; do
- hidden=0
## it sure would be nice to replace this with an sql "not in" statement
- ## it doesn't seem to make sense to have things in .sample and not in guc.c
-# for hidethis in $INTENTIONALLY_NOT_INCLUDED ; do
-# if [ "$hidethis" = "$i" ] ; then
-# hidden=1
-# fi
-# done
- if [ "$hidden" -eq 0 ] ; then
- grep -i '"'$i'"' guc.c > /dev/null
- if [ $? -ne 0 ] ; then
- echo "$i seems to be missing from guc.c";
- fi;
- fi
+ grep -i "\"$i\"" guc.c >/dev/null ||
+ echo "$i seems to be missing from guc.c";
done
### What options are listed in guc.c, but don't appear
### in postgresql.conf.sample?
# grab everything that looks like a setting and convert it to lower case
-
-SETTINGS=`grep '{.* PGC_' guc.c | awk '{print $1}' | \
- sed -e 's/{//g' -e 's/"//g' -e 's/,//'`
-
-SETTINGS=`echo "$SETTINGS" | tr 'A-Z' 'a-z'`
-
+SETTINGS=`gawk -F '[",]' 'BEGIN{RS="\n\t\\\\{\n"} /",[[:space:]]*PGC_.*.*gettext_noop/ && !/NOT_IN_SAMPLE/{print tolower($2)}' guc.c`
for i in $SETTINGS ; do
- hidden=0
- ## it sure would be nice to replace this with an sql "not in" statement
- for hidethis in $INTENTIONALLY_NOT_INCLUDED ; do
- if [ "$hidethis" = "$i" ] ; then
- hidden=1
- fi
- done
- if [ "$hidden" -eq 0 ] ; then
- grep -i '#'$i' ' postgresql.conf.sample > /dev/null
- if [ $? -ne 0 ] ; then
- echo "$i seems to be missing from postgresql.conf.sample";
- fi
- fi
+ grep "#$i " postgresql.conf.sample >/dev/null ||
+ echo "$i seems to be missing from postgresql.conf.sample";
done
--
2.17.0
On Sun, Dec 05, 2021 at 11:38:05PM -0600, Justin Pryzby wrote:
Thanks. One more item. The check_guc script currently outputs 68 false
positives - even though it includes a list of 20 exceptions. This is not
useful.
Indeed. Hmm. This script does a couple of things:
1) Check the format of the options defined in the various lists of
guc.c, which is something people format well, and pgindent also does
a part of this job.
2) Check that options in the hardcoded list of GUCs in
INTENTIONALLY_NOT_INCLUDED are not included in
postgresql.conf.sample
3) Check that nothing considered as a parameter in
postgresql.conf.sample is listed in guc.c.
Your patch removes 1) and 2), but keeps 3) to check for dead
parameter references in postgresql.conf.sample.
Is check_guc actually run on a periodic basis by somebody? Based on
the amount of false positives that has accumulated over the years, and
what `git grep` can already do for 3), it seems to me that we have
more arguments in favor of just removing it entirely.
--
Michael
On Mon, Dec 06, 2021 at 03:58:39PM +0900, Michael Paquier wrote:
On Sun, Dec 05, 2021 at 11:38:05PM -0600, Justin Pryzby wrote:
Thanks. One more item. The check_guc script currently outputs 68 false
positives - even though it includes a list of 20 exceptions. This is not
useful.Indeed. Hmm. This script does a couple of things:
1) Check the format of the options defined in the various lists of
guc.c, which is something people format well, and pgindent also does
a part of this job.
2) Check that options in the hardcoded list of GUCs in
INTENTIONALLY_NOT_INCLUDED are not included in
postgresql.conf.sample
3) Check that nothing considered as a parameter in
postgresql.conf.sample is listed in guc.c.Your patch removes 1) and 2), but keeps 3) to check for dead
parameter references in postgresql.conf.sample.
The script checks that guc.c and sample config are consistent.
I think your undertanding of INTENTIONALLY_NOT_INCLUDED is not right.
That's a list of stuff it "avoids reporting" as an suspected error, not an
additional list of stuff to checks. INTENTIONALLY_NOT_INCLUDED is a list of
stuff like NOT_IN_SAMPLE, which is better done by parsing /NOT_IN_SAMPLE/.
Is check_guc actually run on a periodic basis by somebody? Based on
the amount of false positives that has accumulated over the years, and
what `git grep` can already do for 3), it seems to me that we have
more arguments in favor of just removing it entirely.
I saw that Tom updated it within the last 12 months, which I took to mean that
it was still being maintained. But I'm okay with removing it.
--
Justin
On Mon, Dec 06, 2021 at 07:36:55AM -0600, Justin Pryzby wrote:
The script checks that guc.c and sample config are consistent.
I think your undertanding of INTENTIONALLY_NOT_INCLUDED is not right.
That's a list of stuff it "avoids reporting" as an suspected error, not an
additional list of stuff to checks. INTENTIONALLY_NOT_INCLUDED is a list of
stuff like NOT_IN_SAMPLE, which is better done by parsing /NOT_IN_SAMPLE/.
Indeed. I got that wrong, thanks for clarifying.
I saw that Tom updated it within the last 12 months, which I took to mean that
it was still being maintained. But I'm okay with removing it.
Yes, I saw that as of bf8a662. With 42 incorrect reports, I still see
more evidence with removing it. Before doing anything, let's wait for
and gather some opinions. I am adding Bruce (as the original author)
and Tom in CC as they are the ones who have updated this script the
most in the last ~15 years.
--
Michael
On 08.12.21 07:27, Michael Paquier wrote:
I saw that Tom updated it within the last 12 months, which I took to mean that
it was still being maintained. But I'm okay with removing it.Yes, I saw that as of bf8a662. With 42 incorrect reports, I still see
more evidence with removing it. Before doing anything, let's wait for
and gather some opinions. I am adding Bruce (as the original author)
and Tom in CC as they are the ones who have updated this script the
most in the last ~15 years.
I wasn't really aware of this script either. But I think it's a good
idea to have it. But only if it's run automatically as part of a test
suite run.
On Wed, Dec 08, 2021 at 01:23:51PM +0100, Peter Eisentraut wrote:
I wasn't really aware of this script either. But I think it's a good idea
to have it. But only if it's run automatically as part of a test suite run.
Okay. If we do that, I am wondering whether it would be better to
rewrite this script in perl then, so as there is no need to worry
about the compatibility of grep. And also, it would make sense to
return a non-zero exit code if an incompatibility is found for the
automation part.
--
Michael
On Thu, Dec 09, 2021 at 05:17:54PM +0900, Michael Paquier wrote:
On Wed, Dec 08, 2021 at 01:23:51PM +0100, Peter Eisentraut wrote:
I wasn't really aware of this script either. But I think it's a good idea
to have it. But only if it's run automatically as part of a test suite run.Okay. If we do that, I am wondering whether it would be better to
rewrite this script in perl then, so as there is no need to worry
about the compatibility of grep. And also, it would make sense to
return a non-zero exit code if an incompatibility is found for the
automation part.
One option is to expose the GUC flags in pg_settings, so this can all be done
in SQL regression tests.
Maybe the flags should be text strings, so it's a nicer user-facing interface.
But then the field would be pretty wide, even though we're only adding it for
regression tests. The only other alternative I can think of is to make a
sql-callable function like pg_get_guc_flags(text guc).
Attachments:
0001-check_guc-fix-absurd-number-of-false-positives.patchtext/x-diff; charset=us-asciiDownload
From 2cd8cdbe9f79d88cc920a6e093fa61780ea07a44 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 5 Dec 2021 23:22:04 -0600
Subject: [PATCH 1/2] check_guc: fix absurd number of false positives
Avoid false positives;
Avoid various assumptions;
Avoid list of exceptions;
Simplify shell/awk/sed/grep;
This requires GNU awk for RS as a regex.
---
src/backend/utils/misc/check_guc | 69 +++++---------------------------
1 file changed, 10 insertions(+), 59 deletions(-)
diff --git a/src/backend/utils/misc/check_guc b/src/backend/utils/misc/check_guc
index b171ef0e4f..323ca13191 100755
--- a/src/backend/utils/misc/check_guc
+++ b/src/backend/utils/misc/check_guc
@@ -1,78 +1,29 @@
-#!/bin/sh
+#! /bin/sh
+set -e
-## currently, this script makes a lot of assumptions:
+## this script makes some assumptions about the formatting of files it parses.
## in postgresql.conf.sample:
## 1) the valid config settings may be preceded by a '#', but NOT '# '
## (we use this to skip comments)
-## 2) the valid config settings will be followed immediately by ' ='
-## (at least one space preceding the '=')
-## in guc.c:
-## 3) the options have PGC_ on the same line as the option
-## 4) the options have '{' on the same line as the option
-
-## Problems
-## 1) Don't know what to do with TRANSACTION ISOLATION LEVEL
-
-## if an option is valid but shows up in only one file (guc.c but not
-## postgresql.conf.sample), it should be listed here so that it
-## can be ignored
-INTENTIONALLY_NOT_INCLUDED="debug_deadlocks in_hot_standby \
-is_superuser lc_collate lc_ctype lc_messages lc_monetary lc_numeric lc_time \
-pre_auth_delay role seed server_encoding server_version server_version_num \
-session_authorization trace_lock_oidmin trace_lock_table trace_locks trace_lwlocks \
-trace_notify trace_userlocks transaction_isolation transaction_read_only \
-zero_damaged_pages"
### What options are listed in postgresql.conf.sample, but don't appear
### in guc.c?
-# grab everything that looks like a setting and convert it to lower case
-SETTINGS=`grep ' =' postgresql.conf.sample |
-grep -v '^# ' | # strip comments
-sed -e 's/^#//' |
-awk '{print $1}'`
-
-SETTINGS=`echo "$SETTINGS" | tr 'A-Z' 'a-z'`
+# grab everything that looks like a setting
+SETTINGS=`sed '/^#[[:alnum:]]/!d; s/^#//; s/ =.*//; /^include/d' postgresql.conf.sample`
for i in $SETTINGS ; do
- hidden=0
## it sure would be nice to replace this with an sql "not in" statement
- ## it doesn't seem to make sense to have things in .sample and not in guc.c
-# for hidethis in $INTENTIONALLY_NOT_INCLUDED ; do
-# if [ "$hidethis" = "$i" ] ; then
-# hidden=1
-# fi
-# done
- if [ "$hidden" -eq 0 ] ; then
- grep -i '"'$i'"' guc.c > /dev/null
- if [ $? -ne 0 ] ; then
- echo "$i seems to be missing from guc.c";
- fi;
- fi
+ grep -i "\"$i\"" guc.c >/dev/null ||
+ echo "$i seems to be missing from guc.c";
done
### What options are listed in guc.c, but don't appear
### in postgresql.conf.sample?
# grab everything that looks like a setting and convert it to lower case
-
-SETTINGS=`grep '{.* PGC_' guc.c | awk '{print $1}' | \
- sed -e 's/{//g' -e 's/"//g' -e 's/,//'`
-
-SETTINGS=`echo "$SETTINGS" | tr 'A-Z' 'a-z'`
-
+SETTINGS=`gawk -F '[",]' 'BEGIN{RS="\n\t\\\\{\n"} /",[[:space:]]*PGC_.*.*gettext_noop/ && !/NOT_IN_SAMPLE/{print tolower($2)}' guc.c`
for i in $SETTINGS ; do
- hidden=0
- ## it sure would be nice to replace this with an sql "not in" statement
- for hidethis in $INTENTIONALLY_NOT_INCLUDED ; do
- if [ "$hidethis" = "$i" ] ; then
- hidden=1
- fi
- done
- if [ "$hidden" -eq 0 ] ; then
- grep -i '#'$i' ' postgresql.conf.sample > /dev/null
- if [ $? -ne 0 ] ; then
- echo "$i seems to be missing from postgresql.conf.sample";
- fi
- fi
+ grep "#$i " postgresql.conf.sample >/dev/null ||
+ echo "$i seems to be missing from postgresql.conf.sample";
done
--
2.17.0
0002-Expose-GUC-flags-in-pg_settings-retire-.-check_guc.patchtext/x-diff; charset=us-asciiDownload
From 838656dc9e88a51940a643f2d20970de8b5bfc9a Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Wed, 8 Dec 2021 12:06:35 -0600
Subject: [PATCH 2/2] Expose GUC flags in pg_settings; retire ./check_guc
---
src/backend/utils/misc/check_guc | 29 -------
src/backend/utils/misc/guc.c | 8 +-
src/include/catalog/pg_proc.dat | 6 +-
src/test/regress/expected/guc.out | 115 ++++++++++++++++++++++++++++
src/test/regress/expected/rules.out | 5 +-
src/test/regress/sql/guc.sql | 63 +++++++++++++++
6 files changed, 191 insertions(+), 35 deletions(-)
delete mode 100755 src/backend/utils/misc/check_guc
diff --git a/src/backend/utils/misc/check_guc b/src/backend/utils/misc/check_guc
deleted file mode 100755
index 323ca13191..0000000000
--- a/src/backend/utils/misc/check_guc
+++ /dev/null
@@ -1,29 +0,0 @@
-#! /bin/sh
-set -e
-
-## this script makes some assumptions about the formatting of files it parses.
-## in postgresql.conf.sample:
-## 1) the valid config settings may be preceded by a '#', but NOT '# '
-## (we use this to skip comments)
-
-### What options are listed in postgresql.conf.sample, but don't appear
-### in guc.c?
-
-# grab everything that looks like a setting
-SETTINGS=`sed '/^#[[:alnum:]]/!d; s/^#//; s/ =.*//; /^include/d' postgresql.conf.sample`
-
-for i in $SETTINGS ; do
- ## it sure would be nice to replace this with an sql "not in" statement
- grep -i "\"$i\"" guc.c >/dev/null ||
- echo "$i seems to be missing from guc.c";
-done
-
-### What options are listed in guc.c, but don't appear
-### in postgresql.conf.sample?
-
-# grab everything that looks like a setting and convert it to lower case
-SETTINGS=`gawk -F '[",]' 'BEGIN{RS="\n\t\\\\{\n"} /",[[:space:]]*PGC_.*.*gettext_noop/ && !/NOT_IN_SAMPLE/{print tolower($2)}' guc.c`
-for i in $SETTINGS ; do
- grep "#$i " postgresql.conf.sample >/dev/null ||
- echo "$i seems to be missing from postgresql.conf.sample";
-done
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index ee6a838b3a..fdadeda470 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9889,6 +9889,10 @@ GetConfigOptionByNum(int varnum, const char **values, bool *noshow)
}
values[16] = (conf->status & GUC_PENDING_RESTART) ? "t" : "f";
+
+ /* flags */
+ snprintf(buffer, sizeof(buffer), "%d", conf->flags);
+ values[17] = pstrdup(buffer);
}
/*
@@ -9944,7 +9948,7 @@ show_config_by_name_missing_ok(PG_FUNCTION_ARGS)
* show_all_settings - equiv to SHOW ALL command but implemented as
* a Table Function.
*/
-#define NUM_PG_SETTINGS_ATTS 17
+#define NUM_PG_SETTINGS_ATTS 18
Datum
show_all_settings(PG_FUNCTION_ARGS)
@@ -10006,6 +10010,8 @@ show_all_settings(PG_FUNCTION_ARGS)
INT4OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 17, "pending_restart",
BOOLOID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 18, "flags",
+ INT4OID, -1, 0);
/*
* Generate attribute metadata needed later to produce tuples from raw
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 79d787cd26..c2d3f35862 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6110,9 +6110,9 @@
{ oid => '2084', descr => 'SHOW ALL as a function',
proname => 'pg_show_all_settings', prorows => '1000', proretset => 't',
provolatile => 's', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,text,text,text,text,text,text,text,text,text,text,_text,text,text,text,int4,bool}',
- proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{name,setting,unit,category,short_desc,extra_desc,context,vartype,source,min_val,max_val,enumvals,boot_val,reset_val,sourcefile,sourceline,pending_restart}',
+ proallargtypes => '{text,text,text,text,text,text,text,text,text,text,text,_text,text,text,text,int4,bool,int4}',
+ proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{name,setting,unit,category,short_desc,extra_desc,context,vartype,source,min_val,max_val,enumvals,boot_val,reset_val,sourcefile,sourceline,pending_restart,flags}',
prosrc => 'show_all_settings' },
{ oid => '3329', descr => 'show config file settings',
proname => 'pg_show_all_file_settings', prorows => '1000', proretset => 't',
diff --git a/src/test/regress/expected/guc.out b/src/test/regress/expected/guc.out
index 2433038c3e..f99049cd55 100644
--- a/src/test/regress/expected/guc.out
+++ b/src/test/regress/expected/guc.out
@@ -832,3 +832,118 @@ ERROR: unrecognized configuration parameter "foo"
RESET plpgsql.extra_foo_warnings;
WARNING: unrecognized configuration parameter "plpgsql.extra_foo_warnings"
DETAIL: "plpgsql" is a reserved prefix.
+--
+-- Test GUC categories and flags.
+--
+-- test that GUCS are in postgresql.conf
+SELECT lower(name) FROM pg_settings WHERE (flags&32)=0 EXCEPT
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln~'^#?[[:alpha:]]'
+ORDER BY 1;
+ lower
+-----------------------------
+ config_file
+ plpgsql.check_asserts
+ plpgsql.extra_errors
+ plpgsql.extra_warnings
+ plpgsql.print_strict_params
+ plpgsql.variable_conflict
+(6 rows)
+
+-- test that lines in postgresql.conf that look like GUCs are GUCs
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln~'^#?[[:alpha:]]'
+EXCEPT SELECT lower(name) FROM pg_settings WHERE (flags&32)=0
+ORDER BY 1;
+ guc
+-------------------
+ include
+ include_dir
+ include_if_exists
+(3 rows)
+
+-- test that section:DEVELOPER GUCs are flagged GUC_NOT_IN_SAMPLE:
+SELECT * FROM (SELECT name, category, (flags&32) != 0 AS not_in_sample FROM pg_settings)x
+WHERE category='Developer Options' AND NOT not_in_sample
+ORDER BY 1;
+ name | category | not_in_sample
+------+----------+---------------
+(0 rows)
+
+-- Maybe the converse:
+SELECT * FROM (SELECT name, category, (flags&32) != 0 AS not_in_sample FROM pg_settings)x
+WHERE category NOT IN ('Developer Options', 'Preset Options') AND not_in_sample
+ORDER BY 1;
+ name | category | not_in_sample
+------------------------+-------------------------------------------------+---------------
+ application_name | Reporting and Logging / What to Log | t
+ transaction_deferrable | Client Connection Defaults / Statement Behavior | t
+ transaction_isolation | Client Connection Defaults / Statement Behavior | t
+ transaction_read_only | Client Connection Defaults / Statement Behavior | t
+(4 rows)
+
+-- Most Query Tuning GUCs are flagged as EXPLAIN:
+SELECT * FROM (SELECT name, category, (flags&1048576) != 0 AS guc_explain FROM pg_settings)x
+WHERE category~'^Query Tuning' AND NOT guc_explain
+ORDER BY 1;
+ name | category | guc_explain
+---------------------------+--------------------------------------+-------------
+ default_statistics_target | Query Tuning / Other Planner Options | f
+(1 row)
+
+-- Maybe the converse:
+SELECT * FROM (SELECT name, category, (flags&1048576) != 0 AS guc_explain FROM pg_settings)x
+WHERE guc_explain AND NOT category~'^Query Tuning|^Resource Usage'
+ORDER BY 1;
+ name | category | guc_explain
+---------------------+-------------------------------------------------+-------------
+ explain_regress | Developer Options | t
+ force_parallel_mode | Developer Options | t
+ search_path | Client Connection Defaults / Statement Behavior | t
+(3 rows)
+
+-- GUCs flagged RUNTIME are Preset
+SELECT * FROM (SELECT name, category, (flags&2097152) != 0 AS guc_computed FROM pg_settings)x
+WHERE guc_computed AND NOT category='Preset Options'
+ORDER BY 1;
+ name | category | guc_computed
+------+----------+--------------
+(0 rows)
+
+-- PRESET GUCs are flagged NOT_IN_SAMPLE
+SELECT * FROM (SELECT name, category, (flags&32) != 0 AS not_in_sample FROM pg_settings)x
+WHERE category='Preset Options' AND NOT not_in_sample
+ORDER BY 1;
+ name | category | not_in_sample
+------+----------+---------------
+(0 rows)
+
+-- NO_SHOW_ALL implies NO_RESET_ALL:
+SELECT * FROM (SELECT name, category, (flags&4)!=0 AS no_show_all, (flags&8)!=0 AS no_reset_all FROM pg_settings)x
+WHERE no_show_all AND NOT no_reset_all
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all
+------+----------+-------------+--------------
+(0 rows)
+
+-- Usually the converse:
+SELECT * FROM (SELECT name, category, (flags&4)!=0 AS no_show_all, (flags&8)!=0 AS no_reset_all FROM pg_settings)x
+WHERE NOT no_show_all AND no_reset_all
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all
+------------------------+-------------------------------------------------+-------------+--------------
+ transaction_deferrable | Client Connection Defaults / Statement Behavior | f | t
+ transaction_isolation | Client Connection Defaults / Statement Behavior | f | t
+ transaction_read_only | Client Connection Defaults / Statement Behavior | f | t
+(3 rows)
+
+-- NO_SHOW_ALL implies NOT_IN_SAMPLE:
+SELECT * FROM (SELECT name, category, (flags&4)!=0 AS no_show_all, (flags&32) != 0 AS not_in_sample FROM pg_settings)x
+WHERE no_show_all AND NOT not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | not_in_sample
+------+----------+-------------+---------------
+(0 rows)
+
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index b58b062b10..2db00f2a8d 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1724,8 +1724,9 @@ pg_settings| SELECT a.name,
a.reset_val,
a.sourcefile,
a.sourceline,
- a.pending_restart
- FROM pg_show_all_settings() a(name, setting, unit, category, short_desc, extra_desc, context, vartype, source, min_val, max_val, enumvals, boot_val, reset_val, sourcefile, sourceline, pending_restart);
+ a.pending_restart,
+ a.flags
+ FROM pg_show_all_settings() a(name, setting, unit, category, short_desc, extra_desc, context, vartype, source, min_val, max_val, enumvals, boot_val, reset_val, sourcefile, sourceline, pending_restart, flags);
pg_shadow| SELECT pg_authid.rolname AS usename,
pg_authid.oid AS usesysid,
pg_authid.rolcreatedb AS usecreatedb,
diff --git a/src/test/regress/sql/guc.sql b/src/test/regress/sql/guc.sql
index b57758ed27..0da67b9582 100644
--- a/src/test/regress/sql/guc.sql
+++ b/src/test/regress/sql/guc.sql
@@ -322,3 +322,66 @@ SHOW plpgsql.extra_foo_warnings; -- but the parameter is set
-- cleanup
RESET foo;
RESET plpgsql.extra_foo_warnings;
+
+--
+-- Test GUC categories and flags.
+--
+
+-- test that GUCS are in postgresql.conf
+SELECT lower(name) FROM pg_settings WHERE (flags&32)=0 EXCEPT
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln~'^#?[[:alpha:]]'
+ORDER BY 1;
+
+-- test that lines in postgresql.conf that look like GUCs are GUCs
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln~'^#?[[:alpha:]]'
+EXCEPT SELECT lower(name) FROM pg_settings WHERE (flags&32)=0
+ORDER BY 1;
+
+-- test that section:DEVELOPER GUCs are flagged GUC_NOT_IN_SAMPLE:
+SELECT * FROM (SELECT name, category, (flags&32) != 0 AS not_in_sample FROM pg_settings)x
+WHERE category='Developer Options' AND NOT not_in_sample
+ORDER BY 1;
+
+-- Maybe the converse:
+SELECT * FROM (SELECT name, category, (flags&32) != 0 AS not_in_sample FROM pg_settings)x
+WHERE category NOT IN ('Developer Options', 'Preset Options') AND not_in_sample
+ORDER BY 1;
+
+-- Most Query Tuning GUCs are flagged as EXPLAIN:
+SELECT * FROM (SELECT name, category, (flags&1048576) != 0 AS guc_explain FROM pg_settings)x
+WHERE category~'^Query Tuning' AND NOT guc_explain
+ORDER BY 1;
+
+-- Maybe the converse:
+SELECT * FROM (SELECT name, category, (flags&1048576) != 0 AS guc_explain FROM pg_settings)x
+WHERE guc_explain AND NOT category~'^Query Tuning|^Resource Usage'
+ORDER BY 1;
+
+-- GUCs flagged RUNTIME are Preset
+SELECT * FROM (SELECT name, category, (flags&2097152) != 0 AS guc_computed FROM pg_settings)x
+WHERE guc_computed AND NOT category='Preset Options'
+ORDER BY 1;
+
+-- PRESET GUCs are flagged NOT_IN_SAMPLE
+SELECT * FROM (SELECT name, category, (flags&32) != 0 AS not_in_sample FROM pg_settings)x
+WHERE category='Preset Options' AND NOT not_in_sample
+ORDER BY 1;
+
+-- NO_SHOW_ALL implies NO_RESET_ALL:
+SELECT * FROM (SELECT name, category, (flags&4)!=0 AS no_show_all, (flags&8)!=0 AS no_reset_all FROM pg_settings)x
+WHERE no_show_all AND NOT no_reset_all
+ORDER BY 1;
+
+-- Usually the converse:
+SELECT * FROM (SELECT name, category, (flags&4)!=0 AS no_show_all, (flags&8)!=0 AS no_reset_all FROM pg_settings)x
+WHERE NOT no_show_all AND no_reset_all
+ORDER BY 1;
+
+-- NO_SHOW_ALL implies NOT_IN_SAMPLE:
+SELECT * FROM (SELECT name, category, (flags&4)!=0 AS no_show_all, (flags&32) != 0 AS not_in_sample FROM pg_settings)x
+WHERE no_show_all AND NOT not_in_sample
+ORDER BY 1;
--
2.17.0
On Thu, Dec 09, 2021 at 09:53:23AM -0600, Justin Pryzby wrote:
On Thu, Dec 09, 2021 at 05:17:54PM +0900, Michael Paquier wrote:
On Wed, Dec 08, 2021 at 01:23:51PM +0100, Peter Eisentraut wrote:
I wasn't really aware of this script either. But I think it's a good idea
to have it. But only if it's run automatically as part of a test suite run.Okay. If we do that, I am wondering whether it would be better to
rewrite this script in perl then, so as there is no need to worry
about the compatibility of grep. And also, it would make sense to
return a non-zero exit code if an incompatibility is found for the
automation part.One option is to expose the GUC flags in pg_settings, so this can all be done
in SQL regression tests.Maybe the flags should be text strings, so it's a nicer user-facing interface.
But then the field would be pretty wide, even though we're only adding it for
regression tests. The only other alternative I can think of is to make a
sql-callable function like pg_get_guc_flags(text guc).
Fixed regression tests caused by another patches.
Attachments:
v2-0001-check_guc-fix-absurd-number-of-false-positives.patchtext/x-diff; charset=us-asciiDownload
From f18854a561d779c3d77689f9778dc175db1a2349 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 5 Dec 2021 23:22:04 -0600
Subject: [PATCH v2 1/2] check_guc: fix absurd number of false positives
Avoid false positives;
Avoid various assumptions;
Avoid list of exceptions;
Simplify shell/awk/sed/grep;
This requires GNU awk for RS as a regex.
---
src/backend/utils/misc/check_guc | 69 +++++---------------------------
1 file changed, 10 insertions(+), 59 deletions(-)
diff --git a/src/backend/utils/misc/check_guc b/src/backend/utils/misc/check_guc
index b171ef0e4f..323ca13191 100755
--- a/src/backend/utils/misc/check_guc
+++ b/src/backend/utils/misc/check_guc
@@ -1,78 +1,29 @@
-#!/bin/sh
+#! /bin/sh
+set -e
-## currently, this script makes a lot of assumptions:
+## this script makes some assumptions about the formatting of files it parses.
## in postgresql.conf.sample:
## 1) the valid config settings may be preceded by a '#', but NOT '# '
## (we use this to skip comments)
-## 2) the valid config settings will be followed immediately by ' ='
-## (at least one space preceding the '=')
-## in guc.c:
-## 3) the options have PGC_ on the same line as the option
-## 4) the options have '{' on the same line as the option
-
-## Problems
-## 1) Don't know what to do with TRANSACTION ISOLATION LEVEL
-
-## if an option is valid but shows up in only one file (guc.c but not
-## postgresql.conf.sample), it should be listed here so that it
-## can be ignored
-INTENTIONALLY_NOT_INCLUDED="debug_deadlocks in_hot_standby \
-is_superuser lc_collate lc_ctype lc_messages lc_monetary lc_numeric lc_time \
-pre_auth_delay role seed server_encoding server_version server_version_num \
-session_authorization trace_lock_oidmin trace_lock_table trace_locks trace_lwlocks \
-trace_notify trace_userlocks transaction_isolation transaction_read_only \
-zero_damaged_pages"
### What options are listed in postgresql.conf.sample, but don't appear
### in guc.c?
-# grab everything that looks like a setting and convert it to lower case
-SETTINGS=`grep ' =' postgresql.conf.sample |
-grep -v '^# ' | # strip comments
-sed -e 's/^#//' |
-awk '{print $1}'`
-
-SETTINGS=`echo "$SETTINGS" | tr 'A-Z' 'a-z'`
+# grab everything that looks like a setting
+SETTINGS=`sed '/^#[[:alnum:]]/!d; s/^#//; s/ =.*//; /^include/d' postgresql.conf.sample`
for i in $SETTINGS ; do
- hidden=0
## it sure would be nice to replace this with an sql "not in" statement
- ## it doesn't seem to make sense to have things in .sample and not in guc.c
-# for hidethis in $INTENTIONALLY_NOT_INCLUDED ; do
-# if [ "$hidethis" = "$i" ] ; then
-# hidden=1
-# fi
-# done
- if [ "$hidden" -eq 0 ] ; then
- grep -i '"'$i'"' guc.c > /dev/null
- if [ $? -ne 0 ] ; then
- echo "$i seems to be missing from guc.c";
- fi;
- fi
+ grep -i "\"$i\"" guc.c >/dev/null ||
+ echo "$i seems to be missing from guc.c";
done
### What options are listed in guc.c, but don't appear
### in postgresql.conf.sample?
# grab everything that looks like a setting and convert it to lower case
-
-SETTINGS=`grep '{.* PGC_' guc.c | awk '{print $1}' | \
- sed -e 's/{//g' -e 's/"//g' -e 's/,//'`
-
-SETTINGS=`echo "$SETTINGS" | tr 'A-Z' 'a-z'`
-
+SETTINGS=`gawk -F '[",]' 'BEGIN{RS="\n\t\\\\{\n"} /",[[:space:]]*PGC_.*.*gettext_noop/ && !/NOT_IN_SAMPLE/{print tolower($2)}' guc.c`
for i in $SETTINGS ; do
- hidden=0
- ## it sure would be nice to replace this with an sql "not in" statement
- for hidethis in $INTENTIONALLY_NOT_INCLUDED ; do
- if [ "$hidethis" = "$i" ] ; then
- hidden=1
- fi
- done
- if [ "$hidden" -eq 0 ] ; then
- grep -i '#'$i' ' postgresql.conf.sample > /dev/null
- if [ $? -ne 0 ] ; then
- echo "$i seems to be missing from postgresql.conf.sample";
- fi
- fi
+ grep "#$i " postgresql.conf.sample >/dev/null ||
+ echo "$i seems to be missing from postgresql.conf.sample";
done
--
2.17.0
v2-0002-Expose-GUC-flags-in-pg_settings-retire-.-check_gu.patchtext/x-diff; charset=us-asciiDownload
From ffaffb0350678c7c774aad0b05a7b49f301c08e6 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Wed, 8 Dec 2021 12:06:35 -0600
Subject: [PATCH v2 2/2] Expose GUC flags in pg_settings; retire ./check_guc
---
src/backend/utils/misc/check_guc | 29 -------
src/backend/utils/misc/guc.c | 8 +-
src/include/catalog/pg_proc.dat | 6 +-
src/test/regress/expected/guc.out | 114 ++++++++++++++++++++++++++++
src/test/regress/expected/rules.out | 5 +-
src/test/regress/sql/guc.sql | 63 +++++++++++++++
6 files changed, 190 insertions(+), 35 deletions(-)
delete mode 100755 src/backend/utils/misc/check_guc
diff --git a/src/backend/utils/misc/check_guc b/src/backend/utils/misc/check_guc
deleted file mode 100755
index 323ca13191..0000000000
--- a/src/backend/utils/misc/check_guc
+++ /dev/null
@@ -1,29 +0,0 @@
-#! /bin/sh
-set -e
-
-## this script makes some assumptions about the formatting of files it parses.
-## in postgresql.conf.sample:
-## 1) the valid config settings may be preceded by a '#', but NOT '# '
-## (we use this to skip comments)
-
-### What options are listed in postgresql.conf.sample, but don't appear
-### in guc.c?
-
-# grab everything that looks like a setting
-SETTINGS=`sed '/^#[[:alnum:]]/!d; s/^#//; s/ =.*//; /^include/d' postgresql.conf.sample`
-
-for i in $SETTINGS ; do
- ## it sure would be nice to replace this with an sql "not in" statement
- grep -i "\"$i\"" guc.c >/dev/null ||
- echo "$i seems to be missing from guc.c";
-done
-
-### What options are listed in guc.c, but don't appear
-### in postgresql.conf.sample?
-
-# grab everything that looks like a setting and convert it to lower case
-SETTINGS=`gawk -F '[",]' 'BEGIN{RS="\n\t\\\\{\n"} /",[[:space:]]*PGC_.*.*gettext_noop/ && !/NOT_IN_SAMPLE/{print tolower($2)}' guc.c`
-for i in $SETTINGS ; do
- grep "#$i " postgresql.conf.sample >/dev/null ||
- echo "$i seems to be missing from postgresql.conf.sample";
-done
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index f736e8d872..7d4e2e0374 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9889,6 +9889,10 @@ GetConfigOptionByNum(int varnum, const char **values, bool *noshow)
}
values[16] = (conf->status & GUC_PENDING_RESTART) ? "t" : "f";
+
+ /* flags */
+ snprintf(buffer, sizeof(buffer), "%d", conf->flags);
+ values[17] = pstrdup(buffer);
}
/*
@@ -9944,7 +9948,7 @@ show_config_by_name_missing_ok(PG_FUNCTION_ARGS)
* show_all_settings - equiv to SHOW ALL command but implemented as
* a Table Function.
*/
-#define NUM_PG_SETTINGS_ATTS 17
+#define NUM_PG_SETTINGS_ATTS 18
Datum
show_all_settings(PG_FUNCTION_ARGS)
@@ -10006,6 +10010,8 @@ show_all_settings(PG_FUNCTION_ARGS)
INT4OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 17, "pending_restart",
BOOLOID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 18, "flags",
+ INT4OID, -1, 0);
/*
* Generate attribute metadata needed later to produce tuples from raw
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4d992dc224..ee86d86905 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6092,9 +6092,9 @@
{ oid => '2084', descr => 'SHOW ALL as a function',
proname => 'pg_show_all_settings', prorows => '1000', proretset => 't',
provolatile => 's', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,text,text,text,text,text,text,text,text,text,text,_text,text,text,text,int4,bool}',
- proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{name,setting,unit,category,short_desc,extra_desc,context,vartype,source,min_val,max_val,enumvals,boot_val,reset_val,sourcefile,sourceline,pending_restart}',
+ proallargtypes => '{text,text,text,text,text,text,text,text,text,text,text,_text,text,text,text,int4,bool,int4}',
+ proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{name,setting,unit,category,short_desc,extra_desc,context,vartype,source,min_val,max_val,enumvals,boot_val,reset_val,sourcefile,sourceline,pending_restart,flags}',
prosrc => 'show_all_settings' },
{ oid => '3329', descr => 'show config file settings',
proname => 'pg_show_all_file_settings', prorows => '1000', proretset => 't',
diff --git a/src/test/regress/expected/guc.out b/src/test/regress/expected/guc.out
index 2433038c3e..681cf742d1 100644
--- a/src/test/regress/expected/guc.out
+++ b/src/test/regress/expected/guc.out
@@ -832,3 +832,117 @@ ERROR: unrecognized configuration parameter "foo"
RESET plpgsql.extra_foo_warnings;
WARNING: unrecognized configuration parameter "plpgsql.extra_foo_warnings"
DETAIL: "plpgsql" is a reserved prefix.
+--
+-- Test GUC categories and flags.
+--
+-- test that GUCS are in postgresql.conf
+SELECT lower(name) FROM pg_settings WHERE (flags&32)=0 EXCEPT
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln~'^#?[[:alpha:]]'
+ORDER BY 1;
+ lower
+-----------------------------
+ config_file
+ plpgsql.check_asserts
+ plpgsql.extra_errors
+ plpgsql.extra_warnings
+ plpgsql.print_strict_params
+ plpgsql.variable_conflict
+(6 rows)
+
+-- test that lines in postgresql.conf that look like GUCs are GUCs
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln~'^#?[[:alpha:]]'
+EXCEPT SELECT lower(name) FROM pg_settings WHERE (flags&32)=0
+ORDER BY 1;
+ guc
+-------------------
+ include
+ include_dir
+ include_if_exists
+(3 rows)
+
+-- test that section:DEVELOPER GUCs are flagged GUC_NOT_IN_SAMPLE:
+SELECT * FROM (SELECT name, category, (flags&32) != 0 AS not_in_sample FROM pg_settings)x
+WHERE category='Developer Options' AND NOT not_in_sample
+ORDER BY 1;
+ name | category | not_in_sample
+------+----------+---------------
+(0 rows)
+
+-- Maybe the converse:
+SELECT * FROM (SELECT name, category, (flags&32) != 0 AS not_in_sample FROM pg_settings)x
+WHERE category NOT IN ('Developer Options', 'Preset Options') AND not_in_sample
+ORDER BY 1;
+ name | category | not_in_sample
+------------------------+-------------------------------------------------+---------------
+ application_name | Reporting and Logging / What to Log | t
+ transaction_deferrable | Client Connection Defaults / Statement Behavior | t
+ transaction_isolation | Client Connection Defaults / Statement Behavior | t
+ transaction_read_only | Client Connection Defaults / Statement Behavior | t
+(4 rows)
+
+-- Most Query Tuning GUCs are flagged as EXPLAIN:
+SELECT * FROM (SELECT name, category, (flags&1048576) != 0 AS guc_explain FROM pg_settings)x
+WHERE category~'^Query Tuning' AND NOT guc_explain
+ORDER BY 1;
+ name | category | guc_explain
+---------------------------+--------------------------------------+-------------
+ default_statistics_target | Query Tuning / Other Planner Options | f
+(1 row)
+
+-- Maybe the converse:
+SELECT * FROM (SELECT name, category, (flags&1048576) != 0 AS guc_explain FROM pg_settings)x
+WHERE guc_explain AND NOT category~'^Query Tuning|^Resource Usage'
+ORDER BY 1;
+ name | category | guc_explain
+---------------------+-------------------------------------------------+-------------
+ force_parallel_mode | Developer Options | t
+ search_path | Client Connection Defaults / Statement Behavior | t
+(2 rows)
+
+-- GUCs flagged RUNTIME are Preset
+SELECT * FROM (SELECT name, category, (flags&2097152) != 0 AS guc_computed FROM pg_settings)x
+WHERE guc_computed AND NOT category='Preset Options'
+ORDER BY 1;
+ name | category | guc_computed
+------+----------+--------------
+(0 rows)
+
+-- PRESET GUCs are flagged NOT_IN_SAMPLE
+SELECT * FROM (SELECT name, category, (flags&32) != 0 AS not_in_sample FROM pg_settings)x
+WHERE category='Preset Options' AND NOT not_in_sample
+ORDER BY 1;
+ name | category | not_in_sample
+------+----------+---------------
+(0 rows)
+
+-- NO_SHOW_ALL implies NO_RESET_ALL:
+SELECT * FROM (SELECT name, category, (flags&4)!=0 AS no_show_all, (flags&8)!=0 AS no_reset_all FROM pg_settings)x
+WHERE no_show_all AND NOT no_reset_all
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all
+------+----------+-------------+--------------
+(0 rows)
+
+-- Usually the converse:
+SELECT * FROM (SELECT name, category, (flags&4)!=0 AS no_show_all, (flags&8)!=0 AS no_reset_all FROM pg_settings)x
+WHERE NOT no_show_all AND no_reset_all
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all
+------------------------+-------------------------------------------------+-------------+--------------
+ transaction_deferrable | Client Connection Defaults / Statement Behavior | f | t
+ transaction_isolation | Client Connection Defaults / Statement Behavior | f | t
+ transaction_read_only | Client Connection Defaults / Statement Behavior | f | t
+(3 rows)
+
+-- NO_SHOW_ALL implies NOT_IN_SAMPLE:
+SELECT * FROM (SELECT name, category, (flags&4)!=0 AS no_show_all, (flags&32) != 0 AS not_in_sample FROM pg_settings)x
+WHERE no_show_all AND NOT not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | not_in_sample
+------+----------+-------------+---------------
+(0 rows)
+
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index b58b062b10..2db00f2a8d 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1724,8 +1724,9 @@ pg_settings| SELECT a.name,
a.reset_val,
a.sourcefile,
a.sourceline,
- a.pending_restart
- FROM pg_show_all_settings() a(name, setting, unit, category, short_desc, extra_desc, context, vartype, source, min_val, max_val, enumvals, boot_val, reset_val, sourcefile, sourceline, pending_restart);
+ a.pending_restart,
+ a.flags
+ FROM pg_show_all_settings() a(name, setting, unit, category, short_desc, extra_desc, context, vartype, source, min_val, max_val, enumvals, boot_val, reset_val, sourcefile, sourceline, pending_restart, flags);
pg_shadow| SELECT pg_authid.rolname AS usename,
pg_authid.oid AS usesysid,
pg_authid.rolcreatedb AS usecreatedb,
diff --git a/src/test/regress/sql/guc.sql b/src/test/regress/sql/guc.sql
index b57758ed27..0da67b9582 100644
--- a/src/test/regress/sql/guc.sql
+++ b/src/test/regress/sql/guc.sql
@@ -322,3 +322,66 @@ SHOW plpgsql.extra_foo_warnings; -- but the parameter is set
-- cleanup
RESET foo;
RESET plpgsql.extra_foo_warnings;
+
+--
+-- Test GUC categories and flags.
+--
+
+-- test that GUCS are in postgresql.conf
+SELECT lower(name) FROM pg_settings WHERE (flags&32)=0 EXCEPT
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln~'^#?[[:alpha:]]'
+ORDER BY 1;
+
+-- test that lines in postgresql.conf that look like GUCs are GUCs
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln~'^#?[[:alpha:]]'
+EXCEPT SELECT lower(name) FROM pg_settings WHERE (flags&32)=0
+ORDER BY 1;
+
+-- test that section:DEVELOPER GUCs are flagged GUC_NOT_IN_SAMPLE:
+SELECT * FROM (SELECT name, category, (flags&32) != 0 AS not_in_sample FROM pg_settings)x
+WHERE category='Developer Options' AND NOT not_in_sample
+ORDER BY 1;
+
+-- Maybe the converse:
+SELECT * FROM (SELECT name, category, (flags&32) != 0 AS not_in_sample FROM pg_settings)x
+WHERE category NOT IN ('Developer Options', 'Preset Options') AND not_in_sample
+ORDER BY 1;
+
+-- Most Query Tuning GUCs are flagged as EXPLAIN:
+SELECT * FROM (SELECT name, category, (flags&1048576) != 0 AS guc_explain FROM pg_settings)x
+WHERE category~'^Query Tuning' AND NOT guc_explain
+ORDER BY 1;
+
+-- Maybe the converse:
+SELECT * FROM (SELECT name, category, (flags&1048576) != 0 AS guc_explain FROM pg_settings)x
+WHERE guc_explain AND NOT category~'^Query Tuning|^Resource Usage'
+ORDER BY 1;
+
+-- GUCs flagged RUNTIME are Preset
+SELECT * FROM (SELECT name, category, (flags&2097152) != 0 AS guc_computed FROM pg_settings)x
+WHERE guc_computed AND NOT category='Preset Options'
+ORDER BY 1;
+
+-- PRESET GUCs are flagged NOT_IN_SAMPLE
+SELECT * FROM (SELECT name, category, (flags&32) != 0 AS not_in_sample FROM pg_settings)x
+WHERE category='Preset Options' AND NOT not_in_sample
+ORDER BY 1;
+
+-- NO_SHOW_ALL implies NO_RESET_ALL:
+SELECT * FROM (SELECT name, category, (flags&4)!=0 AS no_show_all, (flags&8)!=0 AS no_reset_all FROM pg_settings)x
+WHERE no_show_all AND NOT no_reset_all
+ORDER BY 1;
+
+-- Usually the converse:
+SELECT * FROM (SELECT name, category, (flags&4)!=0 AS no_show_all, (flags&8)!=0 AS no_reset_all FROM pg_settings)x
+WHERE NOT no_show_all AND no_reset_all
+ORDER BY 1;
+
+-- NO_SHOW_ALL implies NOT_IN_SAMPLE:
+SELECT * FROM (SELECT name, category, (flags&4)!=0 AS no_show_all, (flags&32) != 0 AS not_in_sample FROM pg_settings)x
+WHERE no_show_all AND NOT not_in_sample
+ORDER BY 1;
--
2.17.0
On Thu, Dec 09, 2021 at 09:53:23AM -0600, Justin Pryzby wrote:
On Thu, Dec 09, 2021 at 05:17:54PM +0900, Michael Paquier wrote:
On Wed, Dec 08, 2021 at 01:23:51PM +0100, Peter Eisentraut wrote:
I wasn't really aware of this script either. But I think it's a good idea
to have it. But only if it's run automatically as part of a test suite run.Okay. If we do that, I am wondering whether it would be better to
rewrite this script in perl then, so as there is no need to worry
about the compatibility of grep. And also, it would make sense to
return a non-zero exit code if an incompatibility is found for the
automation part.One option is to expose the GUC flags in pg_settings, so this can all be done
in SQL regression tests.Maybe the flags should be text strings, so it's a nicer user-facing interface.
But then the field would be pretty wide, even though we're only adding it for
regression tests. The only other alternative I can think of is to make a
sql-callable function like pg_get_guc_flags(text guc).
Rebased on cab5b9ab2c066ba904f13de2681872dcda31e207.
And added 0003, which changes to instead exposes the flags as a function, to
avoid changing pg_settings and exposing internally-defined integer flags in
that somewhat prominent view.
--
Justin
Attachments:
v3-0001-check_guc-fix-absurd-number-of-false-positives.patchtext/x-diff; charset=us-asciiDownload
From 7f620514e85d19e5fb3b108c4de3f6f41f295114 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 5 Dec 2021 23:22:04 -0600
Subject: [PATCH v3 1/3] check_guc: fix absurd number of false positives
Avoid false positives;
Avoid various assumptions;
Avoid list of exceptions;
Simplify shell/awk/sed/grep;
This requires GNU awk for RS as a regex.
---
src/backend/utils/misc/check_guc | 69 +++++---------------------------
1 file changed, 10 insertions(+), 59 deletions(-)
diff --git a/src/backend/utils/misc/check_guc b/src/backend/utils/misc/check_guc
index b171ef0e4ff..323ca13191b 100755
--- a/src/backend/utils/misc/check_guc
+++ b/src/backend/utils/misc/check_guc
@@ -1,78 +1,29 @@
-#!/bin/sh
+#! /bin/sh
+set -e
-## currently, this script makes a lot of assumptions:
+## this script makes some assumptions about the formatting of files it parses.
## in postgresql.conf.sample:
## 1) the valid config settings may be preceded by a '#', but NOT '# '
## (we use this to skip comments)
-## 2) the valid config settings will be followed immediately by ' ='
-## (at least one space preceding the '=')
-## in guc.c:
-## 3) the options have PGC_ on the same line as the option
-## 4) the options have '{' on the same line as the option
-
-## Problems
-## 1) Don't know what to do with TRANSACTION ISOLATION LEVEL
-
-## if an option is valid but shows up in only one file (guc.c but not
-## postgresql.conf.sample), it should be listed here so that it
-## can be ignored
-INTENTIONALLY_NOT_INCLUDED="debug_deadlocks in_hot_standby \
-is_superuser lc_collate lc_ctype lc_messages lc_monetary lc_numeric lc_time \
-pre_auth_delay role seed server_encoding server_version server_version_num \
-session_authorization trace_lock_oidmin trace_lock_table trace_locks trace_lwlocks \
-trace_notify trace_userlocks transaction_isolation transaction_read_only \
-zero_damaged_pages"
### What options are listed in postgresql.conf.sample, but don't appear
### in guc.c?
-# grab everything that looks like a setting and convert it to lower case
-SETTINGS=`grep ' =' postgresql.conf.sample |
-grep -v '^# ' | # strip comments
-sed -e 's/^#//' |
-awk '{print $1}'`
-
-SETTINGS=`echo "$SETTINGS" | tr 'A-Z' 'a-z'`
+# grab everything that looks like a setting
+SETTINGS=`sed '/^#[[:alnum:]]/!d; s/^#//; s/ =.*//; /^include/d' postgresql.conf.sample`
for i in $SETTINGS ; do
- hidden=0
## it sure would be nice to replace this with an sql "not in" statement
- ## it doesn't seem to make sense to have things in .sample and not in guc.c
-# for hidethis in $INTENTIONALLY_NOT_INCLUDED ; do
-# if [ "$hidethis" = "$i" ] ; then
-# hidden=1
-# fi
-# done
- if [ "$hidden" -eq 0 ] ; then
- grep -i '"'$i'"' guc.c > /dev/null
- if [ $? -ne 0 ] ; then
- echo "$i seems to be missing from guc.c";
- fi;
- fi
+ grep -i "\"$i\"" guc.c >/dev/null ||
+ echo "$i seems to be missing from guc.c";
done
### What options are listed in guc.c, but don't appear
### in postgresql.conf.sample?
# grab everything that looks like a setting and convert it to lower case
-
-SETTINGS=`grep '{.* PGC_' guc.c | awk '{print $1}' | \
- sed -e 's/{//g' -e 's/"//g' -e 's/,//'`
-
-SETTINGS=`echo "$SETTINGS" | tr 'A-Z' 'a-z'`
-
+SETTINGS=`gawk -F '[",]' 'BEGIN{RS="\n\t\\\\{\n"} /",[[:space:]]*PGC_.*.*gettext_noop/ && !/NOT_IN_SAMPLE/{print tolower($2)}' guc.c`
for i in $SETTINGS ; do
- hidden=0
- ## it sure would be nice to replace this with an sql "not in" statement
- for hidethis in $INTENTIONALLY_NOT_INCLUDED ; do
- if [ "$hidethis" = "$i" ] ; then
- hidden=1
- fi
- done
- if [ "$hidden" -eq 0 ] ; then
- grep -i '#'$i' ' postgresql.conf.sample > /dev/null
- if [ $? -ne 0 ] ; then
- echo "$i seems to be missing from postgresql.conf.sample";
- fi
- fi
+ grep "#$i " postgresql.conf.sample >/dev/null ||
+ echo "$i seems to be missing from postgresql.conf.sample";
done
--
2.17.1
v3-0002-Expose-GUC-flags-in-pg_settings-retire-.-check_gu.patchtext/x-diff; charset=us-asciiDownload
From e567e0ea90adde3f13f0bfcd5cf769ca71f60e78 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Wed, 8 Dec 2021 12:06:35 -0600
Subject: [PATCH v3 2/3] Expose GUC flags in pg_settings; retire ./check_guc
---
src/backend/utils/misc/check_guc | 29 -------
src/backend/utils/misc/guc.c | 8 +-
src/include/catalog/pg_proc.dat | 6 +-
src/test/regress/expected/guc.out | 124 ++++++++++++++++++++++++++++
src/test/regress/expected/rules.out | 5 +-
src/test/regress/sql/guc.sql | 74 +++++++++++++++++
6 files changed, 211 insertions(+), 35 deletions(-)
delete mode 100755 src/backend/utils/misc/check_guc
diff --git a/src/backend/utils/misc/check_guc b/src/backend/utils/misc/check_guc
deleted file mode 100755
index 323ca13191b..00000000000
--- a/src/backend/utils/misc/check_guc
+++ /dev/null
@@ -1,29 +0,0 @@
-#! /bin/sh
-set -e
-
-## this script makes some assumptions about the formatting of files it parses.
-## in postgresql.conf.sample:
-## 1) the valid config settings may be preceded by a '#', but NOT '# '
-## (we use this to skip comments)
-
-### What options are listed in postgresql.conf.sample, but don't appear
-### in guc.c?
-
-# grab everything that looks like a setting
-SETTINGS=`sed '/^#[[:alnum:]]/!d; s/^#//; s/ =.*//; /^include/d' postgresql.conf.sample`
-
-for i in $SETTINGS ; do
- ## it sure would be nice to replace this with an sql "not in" statement
- grep -i "\"$i\"" guc.c >/dev/null ||
- echo "$i seems to be missing from guc.c";
-done
-
-### What options are listed in guc.c, but don't appear
-### in postgresql.conf.sample?
-
-# grab everything that looks like a setting and convert it to lower case
-SETTINGS=`gawk -F '[",]' 'BEGIN{RS="\n\t\\\\{\n"} /",[[:space:]]*PGC_.*.*gettext_noop/ && !/NOT_IN_SAMPLE/{print tolower($2)}' guc.c`
-for i in $SETTINGS ; do
- grep "#$i " postgresql.conf.sample >/dev/null ||
- echo "$i seems to be missing from postgresql.conf.sample";
-done
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index f9504d3aec4..822dfed2b6c 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9849,6 +9849,10 @@ GetConfigOptionByNum(int varnum, const char **values, bool *noshow)
}
values[16] = (conf->status & GUC_PENDING_RESTART) ? "t" : "f";
+
+ /* flags */
+ snprintf(buffer, sizeof(buffer), "%d", conf->flags);
+ values[17] = pstrdup(buffer);
}
/*
@@ -9904,7 +9908,7 @@ show_config_by_name_missing_ok(PG_FUNCTION_ARGS)
* show_all_settings - equiv to SHOW ALL command but implemented as
* a Table Function.
*/
-#define NUM_PG_SETTINGS_ATTS 17
+#define NUM_PG_SETTINGS_ATTS 18
Datum
show_all_settings(PG_FUNCTION_ARGS)
@@ -9966,6 +9970,8 @@ show_all_settings(PG_FUNCTION_ARGS)
INT4OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 17, "pending_restart",
BOOLOID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 18, "flags",
+ INT4OID, -1, 0);
/*
* Generate attribute metadata needed later to produce tuples from raw
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4d992dc2241..ee86d869057 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6092,9 +6092,9 @@
{ oid => '2084', descr => 'SHOW ALL as a function',
proname => 'pg_show_all_settings', prorows => '1000', proretset => 't',
provolatile => 's', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,text,text,text,text,text,text,text,text,text,text,_text,text,text,text,int4,bool}',
- proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{name,setting,unit,category,short_desc,extra_desc,context,vartype,source,min_val,max_val,enumvals,boot_val,reset_val,sourcefile,sourceline,pending_restart}',
+ proallargtypes => '{text,text,text,text,text,text,text,text,text,text,text,_text,text,text,text,int4,bool,int4}',
+ proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{name,setting,unit,category,short_desc,extra_desc,context,vartype,source,min_val,max_val,enumvals,boot_val,reset_val,sourcefile,sourceline,pending_restart,flags}',
prosrc => 'show_all_settings' },
{ oid => '3329', descr => 'show config file settings',
proname => 'pg_show_all_file_settings', prorows => '1000', proretset => 't',
diff --git a/src/test/regress/expected/guc.out b/src/test/regress/expected/guc.out
index 59da91ff04d..d0f3cda22c8 100644
--- a/src/test/regress/expected/guc.out
+++ b/src/test/regress/expected/guc.out
@@ -813,3 +813,127 @@ set default_with_oids to f;
-- Should not allow to set it to true.
set default_with_oids to t;
ERROR: tables declared WITH OIDS are not supported
+--
+-- Test GUC categories and flags.
+--
+begin;
+CREATE TEMP TABLE pg_settings AS SELECT name, category,
+ (flags&4) != 0 AS no_show_all,
+ (flags&8) != 0 AS no_reset_all,
+ (flags&32) != 0 AS not_in_sample,
+ (flags&1048576) != 0 AS guc_explain,
+ (flags&2097152) != 0 AS guc_computed
+ FROM pg_settings;
+-- test that GUCS are in postgresql.conf
+SELECT lower(name) FROM pg_settings WHERE NOT not_in_sample EXCEPT
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln ~ '^#?[[:alpha:]]'
+ORDER BY 1;
+ lower
+-----------------------------
+ config_file
+ plpgsql.check_asserts
+ plpgsql.extra_errors
+ plpgsql.extra_warnings
+ plpgsql.print_strict_params
+ plpgsql.variable_conflict
+(6 rows)
+
+-- test that lines in postgresql.conf that look like GUCs are GUCs
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln ~ '^#?[[:alpha:]]'
+EXCEPT SELECT lower(name) FROM pg_settings WHERE NOT not_in_sample
+ORDER BY 1;
+ guc
+-------------------
+ include
+ include_dir
+ include_if_exists
+(3 rows)
+
+-- test that section:DEVELOPER GUCs are flagged GUC_NOT_IN_SAMPLE:
+SELECT * FROM pg_settings
+WHERE category='Developer Options' AND NOT not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+-- Maybe the converse:
+SELECT * FROM pg_settings
+WHERE category NOT IN ('Developer Options', 'Preset Options') AND not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------------------------+-------------------------------------------------+-------------+--------------+---------------+-------------+--------------
+ application_name | Reporting and Logging / What to Log | f | f | t | f | f
+ transaction_deferrable | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+ transaction_isolation | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+ transaction_read_only | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+(4 rows)
+
+-- Most Query Tuning GUCs are flagged as EXPLAIN:
+SELECT * FROM pg_settings
+WHERE category ~ '^Query Tuning' AND NOT guc_explain
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+---------------------------+--------------------------------------+-------------+--------------+---------------+-------------+--------------
+ default_statistics_target | Query Tuning / Other Planner Options | f | f | f | f | f
+(1 row)
+
+-- Maybe the converse:
+SELECT * FROM pg_settings
+WHERE guc_explain AND NOT category ~ '^Query Tuning|^Resource Usage'
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+---------------------+-------------------------------------------------+-------------+--------------+---------------+-------------+--------------
+ force_parallel_mode | Developer Options | f | f | t | t | f
+ search_path | Client Connection Defaults / Statement Behavior | f | f | f | t | f
+(2 rows)
+
+-- GUCs flagged RUNTIME are Preset
+SELECT * FROM pg_settings
+WHERE guc_computed AND NOT category='Preset Options'
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+-- PRESET GUCs are flagged NOT_IN_SAMPLE
+SELECT * FROM pg_settings
+WHERE category='Preset Options' AND NOT not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+-- NO_SHOW_ALL implies NO_RESET_ALL:
+SELECT * FROM pg_settings
+WHERE no_show_all AND NOT no_reset_all
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+-- Usually the converse:
+SELECT * FROM pg_settings
+WHERE NOT no_show_all AND no_reset_all
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------------------------+-------------------------------------------------+-------------+--------------+---------------+-------------+--------------
+ transaction_deferrable | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+ transaction_isolation | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+ transaction_read_only | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+(3 rows)
+
+-- NO_SHOW_ALL implies NOT_IN_SAMPLE:
+SELECT * FROM pg_settings
+WHERE no_show_all AND NOT not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+DROP TABLE pg_settings;
+rollback;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index b58b062b10d..2db00f2a8df 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1724,8 +1724,9 @@ pg_settings| SELECT a.name,
a.reset_val,
a.sourcefile,
a.sourceline,
- a.pending_restart
- FROM pg_show_all_settings() a(name, setting, unit, category, short_desc, extra_desc, context, vartype, source, min_val, max_val, enumvals, boot_val, reset_val, sourcefile, sourceline, pending_restart);
+ a.pending_restart,
+ a.flags
+ FROM pg_show_all_settings() a(name, setting, unit, category, short_desc, extra_desc, context, vartype, source, min_val, max_val, enumvals, boot_val, reset_val, sourcefile, sourceline, pending_restart, flags);
pg_shadow| SELECT pg_authid.rolname AS usename,
pg_authid.oid AS usesysid,
pg_authid.rolcreatedb AS usecreatedb,
diff --git a/src/test/regress/sql/guc.sql b/src/test/regress/sql/guc.sql
index c39c11388d5..66eb28823b3 100644
--- a/src/test/regress/sql/guc.sql
+++ b/src/test/regress/sql/guc.sql
@@ -311,3 +311,77 @@ reset check_function_bodies;
set default_with_oids to f;
-- Should not allow to set it to true.
set default_with_oids to t;
+
+--
+-- Test GUC categories and flags.
+--
+begin;
+CREATE TEMP TABLE pg_settings AS SELECT name, category,
+ (flags&4) != 0 AS no_show_all,
+ (flags&8) != 0 AS no_reset_all,
+ (flags&32) != 0 AS not_in_sample,
+ (flags&1048576) != 0 AS guc_explain,
+ (flags&2097152) != 0 AS guc_computed
+ FROM pg_settings;
+
+-- test that GUCS are in postgresql.conf
+SELECT lower(name) FROM pg_settings WHERE NOT not_in_sample EXCEPT
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln ~ '^#?[[:alpha:]]'
+ORDER BY 1;
+
+-- test that lines in postgresql.conf that look like GUCs are GUCs
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln ~ '^#?[[:alpha:]]'
+EXCEPT SELECT lower(name) FROM pg_settings WHERE NOT not_in_sample
+ORDER BY 1;
+
+-- test that section:DEVELOPER GUCs are flagged GUC_NOT_IN_SAMPLE:
+SELECT * FROM pg_settings
+WHERE category='Developer Options' AND NOT not_in_sample
+ORDER BY 1;
+
+-- Maybe the converse:
+SELECT * FROM pg_settings
+WHERE category NOT IN ('Developer Options', 'Preset Options') AND not_in_sample
+ORDER BY 1;
+
+-- Most Query Tuning GUCs are flagged as EXPLAIN:
+SELECT * FROM pg_settings
+WHERE category ~ '^Query Tuning' AND NOT guc_explain
+ORDER BY 1;
+
+-- Maybe the converse:
+SELECT * FROM pg_settings
+WHERE guc_explain AND NOT category ~ '^Query Tuning|^Resource Usage'
+ORDER BY 1;
+
+-- GUCs flagged RUNTIME are Preset
+SELECT * FROM pg_settings
+WHERE guc_computed AND NOT category='Preset Options'
+ORDER BY 1;
+
+-- PRESET GUCs are flagged NOT_IN_SAMPLE
+SELECT * FROM pg_settings
+WHERE category='Preset Options' AND NOT not_in_sample
+ORDER BY 1;
+
+-- NO_SHOW_ALL implies NO_RESET_ALL:
+SELECT * FROM pg_settings
+WHERE no_show_all AND NOT no_reset_all
+ORDER BY 1;
+
+-- Usually the converse:
+SELECT * FROM pg_settings
+WHERE NOT no_show_all AND no_reset_all
+ORDER BY 1;
+
+-- NO_SHOW_ALL implies NOT_IN_SAMPLE:
+SELECT * FROM pg_settings
+WHERE no_show_all AND NOT not_in_sample
+ORDER BY 1;
+
+DROP TABLE pg_settings;
+rollback;
--
2.17.1
v3-0003-f-pg_get_guc_flags.patchtext/x-diff; charset=us-asciiDownload
From 1d5d95c9146f75f012f0b3df133f0e82859c0bde Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 26 Dec 2021 20:12:16 -0600
Subject: [PATCH v3 3/3] f!pg_get_guc_flags
---
src/backend/utils/misc/guc.c | 26 +++++++++++++++++++-------
src/include/catalog/pg_proc.dat | 10 +++++++---
src/test/regress/expected/guc.out | 2 +-
src/test/regress/expected/rules.out | 5 ++---
src/test/regress/sql/guc.sql | 2 +-
5 files changed, 30 insertions(+), 15 deletions(-)
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 822dfed2b6c..91ff2a18875 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9849,10 +9849,6 @@ GetConfigOptionByNum(int varnum, const char **values, bool *noshow)
}
values[16] = (conf->status & GUC_PENDING_RESTART) ? "t" : "f";
-
- /* flags */
- snprintf(buffer, sizeof(buffer), "%d", conf->flags);
- values[17] = pstrdup(buffer);
}
/*
@@ -9904,11 +9900,29 @@ show_config_by_name_missing_ok(PG_FUNCTION_ARGS)
PG_RETURN_TEXT_P(cstring_to_text(varval));
}
+/*
+ * Return the flags for the specified GUC, or NULL if it doesn't exist.
+ */
+Datum
+pg_get_guc_flags(PG_FUNCTION_ARGS)
+{
+ char *varname = TextDatumGetCString(PG_GETARG_DATUM(0));
+ struct config_generic *record;
+
+ record = find_option(varname, false, true, ERROR);
+
+ /* return NULL if no such variable */
+ if (record == NULL)
+ PG_RETURN_NULL();
+
+ PG_RETURN_INT32(record->flags);
+}
+
/*
* show_all_settings - equiv to SHOW ALL command but implemented as
* a Table Function.
*/
-#define NUM_PG_SETTINGS_ATTS 18
+#define NUM_PG_SETTINGS_ATTS 17
Datum
show_all_settings(PG_FUNCTION_ARGS)
@@ -9970,8 +9984,6 @@ show_all_settings(PG_FUNCTION_ARGS)
INT4OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 17, "pending_restart",
BOOLOID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 18, "flags",
- INT4OID, -1, 0);
/*
* Generate attribute metadata needed later to produce tuples from raw
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index ee86d869057..7fa701fe815 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6092,10 +6092,14 @@
{ oid => '2084', descr => 'SHOW ALL as a function',
proname => 'pg_show_all_settings', prorows => '1000', proretset => 't',
provolatile => 's', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,text,text,text,text,text,text,text,text,text,text,_text,text,text,text,int4,bool,int4}',
- proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{name,setting,unit,category,short_desc,extra_desc,context,vartype,source,min_val,max_val,enumvals,boot_val,reset_val,sourcefile,sourceline,pending_restart,flags}',
+ proallargtypes => '{text,text,text,text,text,text,text,text,text,text,text,_text,text,text,text,int4,bool}',
+ proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{name,setting,unit,category,short_desc,extra_desc,context,vartype,source,min_val,max_val,enumvals,boot_val,reset_val,sourcefile,sourceline,pending_restart}',
prosrc => 'show_all_settings' },
+{ oid => '6116', descr => 'return flags for specified GUC',
+ proname => 'pg_get_guc_flags', provolatile => 's',
+ prorettype => 'int4', proargtypes => 'text',
+ prosrc => 'pg_get_guc_flags' },
{ oid => '3329', descr => 'show config file settings',
proname => 'pg_show_all_file_settings', prorows => '1000', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/test/regress/expected/guc.out b/src/test/regress/expected/guc.out
index d0f3cda22c8..f7dd6c0b299 100644
--- a/src/test/regress/expected/guc.out
+++ b/src/test/regress/expected/guc.out
@@ -823,7 +823,7 @@ CREATE TEMP TABLE pg_settings AS SELECT name, category,
(flags&32) != 0 AS not_in_sample,
(flags&1048576) != 0 AS guc_explain,
(flags&2097152) != 0 AS guc_computed
- FROM pg_settings;
+ FROM pg_settings, pg_get_guc_flags(name) AS flags;
-- test that GUCS are in postgresql.conf
SELECT lower(name) FROM pg_settings WHERE NOT not_in_sample EXCEPT
SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 2db00f2a8df..b58b062b10d 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1724,9 +1724,8 @@ pg_settings| SELECT a.name,
a.reset_val,
a.sourcefile,
a.sourceline,
- a.pending_restart,
- a.flags
- FROM pg_show_all_settings() a(name, setting, unit, category, short_desc, extra_desc, context, vartype, source, min_val, max_val, enumvals, boot_val, reset_val, sourcefile, sourceline, pending_restart, flags);
+ a.pending_restart
+ FROM pg_show_all_settings() a(name, setting, unit, category, short_desc, extra_desc, context, vartype, source, min_val, max_val, enumvals, boot_val, reset_val, sourcefile, sourceline, pending_restart);
pg_shadow| SELECT pg_authid.rolname AS usename,
pg_authid.oid AS usesysid,
pg_authid.rolcreatedb AS usecreatedb,
diff --git a/src/test/regress/sql/guc.sql b/src/test/regress/sql/guc.sql
index 66eb28823b3..81eac9b8ed2 100644
--- a/src/test/regress/sql/guc.sql
+++ b/src/test/regress/sql/guc.sql
@@ -322,7 +322,7 @@ CREATE TEMP TABLE pg_settings AS SELECT name, category,
(flags&32) != 0 AS not_in_sample,
(flags&1048576) != 0 AS guc_explain,
(flags&2097152) != 0 AS guc_computed
- FROM pg_settings;
+ FROM pg_settings, pg_get_guc_flags(name) AS flags;
-- test that GUCS are in postgresql.conf
SELECT lower(name) FROM pg_settings WHERE NOT not_in_sample EXCEPT
--
2.17.1
At Tue, 28 Dec 2021 20:32:40 -0600, Justin Pryzby <pryzby@telsasoft.com> wrote in
On Thu, Dec 09, 2021 at 09:53:23AM -0600, Justin Pryzby wrote:
On Thu, Dec 09, 2021 at 05:17:54PM +0900, Michael Paquier wrote:
One option is to expose the GUC flags in pg_settings, so this can all be done
in SQL regression tests.Maybe the flags should be text strings, so it's a nicer user-facing interface.
But then the field would be pretty wide, even though we're only adding it for
regression tests. The only other alternative I can think of is to make a
sql-callable function like pg_get_guc_flags(text guc).Rebased on cab5b9ab2c066ba904f13de2681872dcda31e207.
And added 0003, which changes to instead exposes the flags as a function, to
avoid changing pg_settings and exposing internally-defined integer flags in
that somewhat prominent view.
Just an idea but couldn't we use flags in a series of one-letter flag
representations? It is more user-friendly than integers but shorter
than full-text representation.
+SELECT name, flags FROM pg_settings;
name | flags
------------------------+--------
application_name | ARsec
transaction_deferrable | Arsec
transaction_isolation | Arsec
transaction_read_only | Arsec
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
On Wed, Jan 05, 2022 at 11:47:57AM +0900, Kyotaro Horiguchi wrote:
At Tue, 28 Dec 2021 20:32:40 -0600, Justin Pryzby <pryzby@telsasoft.com> wrote in
On Thu, Dec 09, 2021 at 09:53:23AM -0600, Justin Pryzby wrote:
On Thu, Dec 09, 2021 at 05:17:54PM +0900, Michael Paquier wrote:
One option is to expose the GUC flags in pg_settings, so this can all be done
in SQL regression tests.Maybe the flags should be text strings, so it's a nicer user-facing interface.
But then the field would be pretty wide, even though we're only adding it for
regression tests. The only other alternative I can think of is to make a
sql-callable function like pg_get_guc_flags(text guc).Rebased on cab5b9ab2c066ba904f13de2681872dcda31e207.
And added 0003, which changes to instead exposes the flags as a function, to
avoid changing pg_settings and exposing internally-defined integer flags in
that somewhat prominent view.Just an idea but couldn't we use flags in a series of one-letter flag
representations? It is more user-friendly than integers but shorter
than full-text representation.+SELECT name, flags FROM pg_settings;
name | flags
------------------------+--------
application_name | ARsec
transaction_deferrable | Arsec
transaction_isolation | Arsec
transaction_read_only | Arsec
It's a good idea.
I suppose you intend that "A" means it's enabled and "a" means it's disabled ?
A => show all
R => reset all
S => not in sample
E => explain
C => computed
Which is enough to support the tests that I came up with:
+ (flags&4) != 0 AS no_show_all,
+ (flags&8) != 0 AS no_reset_all,
+ (flags&32) != 0 AS not_in_sample,
+ (flags&1048576) != 0 AS guc_explain,
+ (flags&2097152) != 0 AS guc_computed
However, I think if we add a field to pg_stat_activity, it would be in a
separate patch, expected to be independently useful.
1) expose GUC flags to pg_stat_activity;
2) rewrite check_guc as a sql regression test;
In that case, *all* the flags should be exposed. There's currently 20, which
means it may not work well after all - it's already too long, and could get
longer, and/or overflow the alphabet...
I think pg_get_guc_flags() may be best, but I'm interested to hear other
opinions.
--
Justin
On Tue, Jan 04, 2022 at 09:06:48PM -0600, Justin Pryzby wrote:
I think pg_get_guc_flags() may be best, but I'm interested to hear other
opinions.
My opinion on this matter is rather close to what you have here with
handling things through one extra attribute. But I don't see the
point of using an extra function where users would need to do a manual
mapping of the flag bits back to a a text representation of them. So
I would suggest to just add one text[] to pg_show_all_settings, with
values being the bit names themselves, without the prefix "GUC_", for
the ones we care most about. Sticking with one column for each one
would require a catversion bump all the time, which could be
cumbersome in the long run.
--
Michael
On Wed, Jan 05, 2022 at 02:17:11PM +0900, Michael Paquier wrote:
On Tue, Jan 04, 2022 at 09:06:48PM -0600, Justin Pryzby wrote:
I think pg_get_guc_flags() may be best, but I'm interested to hear other
opinions.My opinion on this matter is rather close to what you have here with
handling things through one extra attribute. But I don't see the
point of using an extra function where users would need to do a manual
mapping of the flag bits back to a a text representation of them.
If it were implemented as a function, this would be essentially internal and
left undocumented. Only exposed for the purpose of re-implementing check_guc.
I would suggest to just add one text[] to pg_show_all_settings
Good idea to use the backing function without updating the view.
pg_settings is currently defined with "SELECT *". Is it fine to enumerate a
list of columns instead ?
Attachments:
v4-0001-check_guc-fix-absurd-number-of-false-positives.patchtext/x-diff; charset=us-asciiDownload
From 713046d82668c4e189b78e9e274b272bccaaa480 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 5 Dec 2021 23:22:04 -0600
Subject: [PATCH v4 1/2] check_guc: fix absurd number of false positives
Avoid false positives;
Avoid various assumptions;
Avoid list of exceptions;
Simplify shell/awk/sed/grep;
This requires GNU awk for RS as a regex.
---
src/backend/utils/misc/check_guc | 69 +++++---------------------------
1 file changed, 10 insertions(+), 59 deletions(-)
diff --git a/src/backend/utils/misc/check_guc b/src/backend/utils/misc/check_guc
index b171ef0e4ff..323ca13191b 100755
--- a/src/backend/utils/misc/check_guc
+++ b/src/backend/utils/misc/check_guc
@@ -1,78 +1,29 @@
-#!/bin/sh
+#! /bin/sh
+set -e
-## currently, this script makes a lot of assumptions:
+## this script makes some assumptions about the formatting of files it parses.
## in postgresql.conf.sample:
## 1) the valid config settings may be preceded by a '#', but NOT '# '
## (we use this to skip comments)
-## 2) the valid config settings will be followed immediately by ' ='
-## (at least one space preceding the '=')
-## in guc.c:
-## 3) the options have PGC_ on the same line as the option
-## 4) the options have '{' on the same line as the option
-
-## Problems
-## 1) Don't know what to do with TRANSACTION ISOLATION LEVEL
-
-## if an option is valid but shows up in only one file (guc.c but not
-## postgresql.conf.sample), it should be listed here so that it
-## can be ignored
-INTENTIONALLY_NOT_INCLUDED="debug_deadlocks in_hot_standby \
-is_superuser lc_collate lc_ctype lc_messages lc_monetary lc_numeric lc_time \
-pre_auth_delay role seed server_encoding server_version server_version_num \
-session_authorization trace_lock_oidmin trace_lock_table trace_locks trace_lwlocks \
-trace_notify trace_userlocks transaction_isolation transaction_read_only \
-zero_damaged_pages"
### What options are listed in postgresql.conf.sample, but don't appear
### in guc.c?
-# grab everything that looks like a setting and convert it to lower case
-SETTINGS=`grep ' =' postgresql.conf.sample |
-grep -v '^# ' | # strip comments
-sed -e 's/^#//' |
-awk '{print $1}'`
-
-SETTINGS=`echo "$SETTINGS" | tr 'A-Z' 'a-z'`
+# grab everything that looks like a setting
+SETTINGS=`sed '/^#[[:alnum:]]/!d; s/^#//; s/ =.*//; /^include/d' postgresql.conf.sample`
for i in $SETTINGS ; do
- hidden=0
## it sure would be nice to replace this with an sql "not in" statement
- ## it doesn't seem to make sense to have things in .sample and not in guc.c
-# for hidethis in $INTENTIONALLY_NOT_INCLUDED ; do
-# if [ "$hidethis" = "$i" ] ; then
-# hidden=1
-# fi
-# done
- if [ "$hidden" -eq 0 ] ; then
- grep -i '"'$i'"' guc.c > /dev/null
- if [ $? -ne 0 ] ; then
- echo "$i seems to be missing from guc.c";
- fi;
- fi
+ grep -i "\"$i\"" guc.c >/dev/null ||
+ echo "$i seems to be missing from guc.c";
done
### What options are listed in guc.c, but don't appear
### in postgresql.conf.sample?
# grab everything that looks like a setting and convert it to lower case
-
-SETTINGS=`grep '{.* PGC_' guc.c | awk '{print $1}' | \
- sed -e 's/{//g' -e 's/"//g' -e 's/,//'`
-
-SETTINGS=`echo "$SETTINGS" | tr 'A-Z' 'a-z'`
-
+SETTINGS=`gawk -F '[",]' 'BEGIN{RS="\n\t\\\\{\n"} /",[[:space:]]*PGC_.*.*gettext_noop/ && !/NOT_IN_SAMPLE/{print tolower($2)}' guc.c`
for i in $SETTINGS ; do
- hidden=0
- ## it sure would be nice to replace this with an sql "not in" statement
- for hidethis in $INTENTIONALLY_NOT_INCLUDED ; do
- if [ "$hidethis" = "$i" ] ; then
- hidden=1
- fi
- done
- if [ "$hidden" -eq 0 ] ; then
- grep -i '#'$i' ' postgresql.conf.sample > /dev/null
- if [ $? -ne 0 ] ; then
- echo "$i seems to be missing from postgresql.conf.sample";
- fi
- fi
+ grep "#$i " postgresql.conf.sample >/dev/null ||
+ echo "$i seems to be missing from postgresql.conf.sample";
done
--
2.17.1
v4-0002-Expose-GUC-flags-in-SQL-function-retire-.-check_g.patchtext/x-diff; charset=us-asciiDownload
From d4773ee874ddc6aaa1238e4d1ae7ec6f8e20f36e Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Wed, 8 Dec 2021 12:06:35 -0600
Subject: [PATCH v4 2/2] Expose GUC flags in SQL function; retire ./check_guc
---
src/backend/catalog/system_views.sql | 19 ++++-
src/backend/utils/misc/check_guc | 29 -------
src/backend/utils/misc/guc.c | 37 +++++++-
src/include/catalog/pg_proc.dat | 6 +-
src/test/regress/expected/guc.out | 122 +++++++++++++++++++++++++++
src/test/regress/expected/rules.out | 2 +-
src/test/regress/sql/guc.sql | 72 ++++++++++++++++
7 files changed, 252 insertions(+), 35 deletions(-)
delete mode 100755 src/backend/utils/misc/check_guc
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 61b515cdb85..e2c01443cb6 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -580,7 +580,24 @@ FROM
JOIN pg_authid rol ON l.classoid = rol.tableoid AND l.objoid = rol.oid;
CREATE VIEW pg_settings AS
- SELECT * FROM pg_show_all_settings() AS A;
+ SELECT a.name,
+ a.setting,
+ a.unit,
+ a.category,
+ a.short_desc,
+ a.extra_desc,
+ a.context,
+ a.vartype,
+ a.source,
+ a.min_val,
+ a.max_val,
+ a.enumvals,
+ a.boot_val,
+ a.reset_val,
+ a.sourcefile,
+ a.sourceline,
+ a.pending_restart
+ FROM pg_show_all_settings() AS A;
CREATE RULE pg_settings_u AS
ON UPDATE TO pg_settings
diff --git a/src/backend/utils/misc/check_guc b/src/backend/utils/misc/check_guc
deleted file mode 100755
index 323ca13191b..00000000000
--- a/src/backend/utils/misc/check_guc
+++ /dev/null
@@ -1,29 +0,0 @@
-#! /bin/sh
-set -e
-
-## this script makes some assumptions about the formatting of files it parses.
-## in postgresql.conf.sample:
-## 1) the valid config settings may be preceded by a '#', but NOT '# '
-## (we use this to skip comments)
-
-### What options are listed in postgresql.conf.sample, but don't appear
-### in guc.c?
-
-# grab everything that looks like a setting
-SETTINGS=`sed '/^#[[:alnum:]]/!d; s/^#//; s/ =.*//; /^include/d' postgresql.conf.sample`
-
-for i in $SETTINGS ; do
- ## it sure would be nice to replace this with an sql "not in" statement
- grep -i "\"$i\"" guc.c >/dev/null ||
- echo "$i seems to be missing from guc.c";
-done
-
-### What options are listed in guc.c, but don't appear
-### in postgresql.conf.sample?
-
-# grab everything that looks like a setting and convert it to lower case
-SETTINGS=`gawk -F '[",]' 'BEGIN{RS="\n\t\\\\{\n"} /",[[:space:]]*PGC_.*.*gettext_noop/ && !/NOT_IN_SAMPLE/{print tolower($2)}' guc.c`
-for i in $SETTINGS ; do
- grep "#$i " postgresql.conf.sample >/dev/null ||
- echo "$i seems to be missing from postgresql.conf.sample";
-done
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index f9504d3aec4..9b6ea9279bd 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9622,6 +9622,36 @@ GetConfigOptionByName(const char *name, const char **varname, bool missing_ok)
return _ShowOption(record, true);
}
+/*
+ * Return an text[] array of the given flags (or a useful subset?) XXX
+ */
+static char *
+get_flags_array_text(int flags)
+{
+ StringInfoData ret;
+
+ initStringInfo(&ret);
+ appendStringInfoChar(&ret, '{');
+
+ if (flags & GUC_NO_SHOW_ALL)
+ appendStringInfo(&ret, "NO_SHOW_ALL,");
+ if (flags & GUC_NO_RESET_ALL)
+ appendStringInfo(&ret, "NO_RESET_ALL,");
+ if (flags & GUC_NOT_IN_SAMPLE)
+ appendStringInfo(&ret, "NOT_IN_SAMPLE,");
+ if (flags & GUC_EXPLAIN)
+ appendStringInfo(&ret, "EXPLAIN,");
+ if (flags & GUC_RUNTIME_COMPUTED)
+ appendStringInfo(&ret, "RUNTIME_COMPUTED,");
+
+ /* Remove trailing comma, if any */
+ if (ret.len > 1)
+ ret.data[--ret.len] = '\0';
+
+ appendStringInfoChar(&ret, '}');
+ return ret.data;
+}
+
/*
* Return GUC variable value by variable number; optionally return canonical
* form of name. Return value is palloc'd.
@@ -9849,6 +9879,9 @@ GetConfigOptionByNum(int varnum, const char **values, bool *noshow)
}
values[16] = (conf->status & GUC_PENDING_RESTART) ? "t" : "f";
+
+ /* flags */
+ values[17] = get_flags_array_text(conf->flags);
}
/*
@@ -9904,7 +9937,7 @@ show_config_by_name_missing_ok(PG_FUNCTION_ARGS)
* show_all_settings - equiv to SHOW ALL command but implemented as
* a Table Function.
*/
-#define NUM_PG_SETTINGS_ATTS 17
+#define NUM_PG_SETTINGS_ATTS 18
Datum
show_all_settings(PG_FUNCTION_ARGS)
@@ -9966,6 +9999,8 @@ show_all_settings(PG_FUNCTION_ARGS)
INT4OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 17, "pending_restart",
BOOLOID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 18, "flags",
+ TEXTARRAYOID, -1, 0);
/*
* Generate attribute metadata needed later to produce tuples from raw
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4d992dc2241..7e7e9a48fac 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6092,9 +6092,9 @@
{ oid => '2084', descr => 'SHOW ALL as a function',
proname => 'pg_show_all_settings', prorows => '1000', proretset => 't',
provolatile => 's', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,text,text,text,text,text,text,text,text,text,text,_text,text,text,text,int4,bool}',
- proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{name,setting,unit,category,short_desc,extra_desc,context,vartype,source,min_val,max_val,enumvals,boot_val,reset_val,sourcefile,sourceline,pending_restart}',
+ proallargtypes => '{text,text,text,text,text,text,text,text,text,text,text,_text,text,text,text,int4,bool,_text}',
+ proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{name,setting,unit,category,short_desc,extra_desc,context,vartype,source,min_val,max_val,enumvals,boot_val,reset_val,sourcefile,sourceline,pending_restart,flags}',
prosrc => 'show_all_settings' },
{ oid => '3329', descr => 'show config file settings',
proname => 'pg_show_all_file_settings', prorows => '1000', proretset => 't',
diff --git a/src/test/regress/expected/guc.out b/src/test/regress/expected/guc.out
index 59da91ff04d..d27b7c24747 100644
--- a/src/test/regress/expected/guc.out
+++ b/src/test/regress/expected/guc.out
@@ -813,3 +813,125 @@ set default_with_oids to f;
-- Should not allow to set it to true.
set default_with_oids to t;
ERROR: tables declared WITH OIDS are not supported
+--
+-- Test GUC categories and flags.
+--
+CREATE TABLE pg_settings_flags AS SELECT name, category,
+ 'NO_SHOW_ALL' =ANY(flags) AS no_show_all,
+ 'NO_RESET_ALL' =ANY(flags) AS no_reset_all,
+ 'NOT_IN_SAMPLE' =ANY(flags) AS not_in_sample,
+ 'EXPLAIN' =ANY(flags) AS guc_explain,
+ 'COMPUTED' =ANY(flags) AS guc_computed
+ FROM pg_show_all_settings();
+-- test that GUCS are in postgresql.conf
+SELECT lower(name) FROM pg_settings_flags WHERE NOT not_in_sample EXCEPT
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln ~ '^#?[[:alpha:]]'
+ORDER BY 1;
+ lower
+-----------------------------
+ config_file
+ plpgsql.check_asserts
+ plpgsql.extra_errors
+ plpgsql.extra_warnings
+ plpgsql.print_strict_params
+ plpgsql.variable_conflict
+(6 rows)
+
+-- test that lines in postgresql.conf that look like GUCs are GUCs
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln ~ '^#?[[:alpha:]]'
+EXCEPT SELECT lower(name) FROM pg_settings_flags WHERE NOT not_in_sample
+ORDER BY 1;
+ guc
+-------------------
+ include
+ include_dir
+ include_if_exists
+(3 rows)
+
+-- test that section:DEVELOPER GUCs are flagged GUC_NOT_IN_SAMPLE:
+SELECT * FROM pg_settings_flags
+WHERE category='Developer Options' AND NOT not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+-- Maybe the converse:
+SELECT * FROM pg_settings_flags
+WHERE category NOT IN ('Developer Options', 'Preset Options') AND not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------------------------+-------------------------------------------------+-------------+--------------+---------------+-------------+--------------
+ application_name | Reporting and Logging / What to Log | f | f | t | f | f
+ transaction_deferrable | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+ transaction_isolation | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+ transaction_read_only | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+(4 rows)
+
+-- Most Query Tuning GUCs are flagged as EXPLAIN:
+SELECT * FROM pg_settings_flags
+WHERE category ~ '^Query Tuning' AND NOT guc_explain
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+---------------------------+--------------------------------------+-------------+--------------+---------------+-------------+--------------
+ default_statistics_target | Query Tuning / Other Planner Options | f | f | f | f | f
+(1 row)
+
+-- Maybe the converse:
+SELECT * FROM pg_settings_flags
+WHERE guc_explain AND NOT category ~ '^Query Tuning|^Resource Usage'
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+---------------------+-------------------------------------------------+-------------+--------------+---------------+-------------+--------------
+ force_parallel_mode | Developer Options | f | f | t | t | f
+ search_path | Client Connection Defaults / Statement Behavior | f | f | f | t | f
+(2 rows)
+
+-- GUCs flagged RUNTIME are Preset
+SELECT * FROM pg_settings_flags
+WHERE guc_computed AND NOT category='Preset Options'
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+-- PRESET GUCs are flagged NOT_IN_SAMPLE
+SELECT * FROM pg_settings_flags
+WHERE category='Preset Options' AND NOT not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+-- NO_SHOW_ALL implies NO_RESET_ALL:
+SELECT * FROM pg_settings_flags
+WHERE no_show_all AND NOT no_reset_all
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+-- Usually the converse:
+SELECT * FROM pg_settings_flags
+WHERE NOT no_show_all AND no_reset_all
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------------------------+-------------------------------------------------+-------------+--------------+---------------+-------------+--------------
+ transaction_deferrable | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+ transaction_isolation | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+ transaction_read_only | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+(3 rows)
+
+-- NO_SHOW_ALL implies NOT_IN_SAMPLE:
+SELECT * FROM pg_settings_flags
+WHERE no_show_all AND NOT not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+DROP TABLE pg_settings_flags;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index b58b062b10d..71cbdc37233 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1725,7 +1725,7 @@ pg_settings| SELECT a.name,
a.sourcefile,
a.sourceline,
a.pending_restart
- FROM pg_show_all_settings() a(name, setting, unit, category, short_desc, extra_desc, context, vartype, source, min_val, max_val, enumvals, boot_val, reset_val, sourcefile, sourceline, pending_restart);
+ FROM pg_show_all_settings() a(name, setting, unit, category, short_desc, extra_desc, context, vartype, source, min_val, max_val, enumvals, boot_val, reset_val, sourcefile, sourceline, pending_restart, flags);
pg_shadow| SELECT pg_authid.rolname AS usename,
pg_authid.oid AS usesysid,
pg_authid.rolcreatedb AS usecreatedb,
diff --git a/src/test/regress/sql/guc.sql b/src/test/regress/sql/guc.sql
index c39c11388d5..72b4c367941 100644
--- a/src/test/regress/sql/guc.sql
+++ b/src/test/regress/sql/guc.sql
@@ -311,3 +311,75 @@ reset check_function_bodies;
set default_with_oids to f;
-- Should not allow to set it to true.
set default_with_oids to t;
+
+--
+-- Test GUC categories and flags.
+--
+CREATE TABLE pg_settings_flags AS SELECT name, category,
+ 'NO_SHOW_ALL' =ANY(flags) AS no_show_all,
+ 'NO_RESET_ALL' =ANY(flags) AS no_reset_all,
+ 'NOT_IN_SAMPLE' =ANY(flags) AS not_in_sample,
+ 'EXPLAIN' =ANY(flags) AS guc_explain,
+ 'COMPUTED' =ANY(flags) AS guc_computed
+ FROM pg_show_all_settings();
+
+-- test that GUCS are in postgresql.conf
+SELECT lower(name) FROM pg_settings_flags WHERE NOT not_in_sample EXCEPT
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln ~ '^#?[[:alpha:]]'
+ORDER BY 1;
+
+-- test that lines in postgresql.conf that look like GUCs are GUCs
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) = .*', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln ~ '^#?[[:alpha:]]'
+EXCEPT SELECT lower(name) FROM pg_settings_flags WHERE NOT not_in_sample
+ORDER BY 1;
+
+-- test that section:DEVELOPER GUCs are flagged GUC_NOT_IN_SAMPLE:
+SELECT * FROM pg_settings_flags
+WHERE category='Developer Options' AND NOT not_in_sample
+ORDER BY 1;
+
+-- Maybe the converse:
+SELECT * FROM pg_settings_flags
+WHERE category NOT IN ('Developer Options', 'Preset Options') AND not_in_sample
+ORDER BY 1;
+
+-- Most Query Tuning GUCs are flagged as EXPLAIN:
+SELECT * FROM pg_settings_flags
+WHERE category ~ '^Query Tuning' AND NOT guc_explain
+ORDER BY 1;
+
+-- Maybe the converse:
+SELECT * FROM pg_settings_flags
+WHERE guc_explain AND NOT category ~ '^Query Tuning|^Resource Usage'
+ORDER BY 1;
+
+-- GUCs flagged RUNTIME are Preset
+SELECT * FROM pg_settings_flags
+WHERE guc_computed AND NOT category='Preset Options'
+ORDER BY 1;
+
+-- PRESET GUCs are flagged NOT_IN_SAMPLE
+SELECT * FROM pg_settings_flags
+WHERE category='Preset Options' AND NOT not_in_sample
+ORDER BY 1;
+
+-- NO_SHOW_ALL implies NO_RESET_ALL:
+SELECT * FROM pg_settings_flags
+WHERE no_show_all AND NOT no_reset_all
+ORDER BY 1;
+
+-- Usually the converse:
+SELECT * FROM pg_settings_flags
+WHERE NOT no_show_all AND no_reset_all
+ORDER BY 1;
+
+-- NO_SHOW_ALL implies NOT_IN_SAMPLE:
+SELECT * FROM pg_settings_flags
+WHERE no_show_all AND NOT not_in_sample
+ORDER BY 1;
+
+DROP TABLE pg_settings_flags;
--
2.17.1
On Wed, Jan 05, 2022 at 05:55:17PM -0600, Justin Pryzby wrote:
pg_settings is currently defined with "SELECT *". Is it fine to enumerate a
list of columns instead ?
I'd like to think that this is a better practice when it comes
documenting the columns, but I don't see an actual need for this extra
complication here.
+ initStringInfo(&ret); + appendStringInfoChar(&ret, '{'); + + if (flags & GUC_NO_SHOW_ALL) + appendStringInfo(&ret, "NO_SHOW_ALL,"); + if (flags & GUC_NO_RESET_ALL) + appendStringInfo(&ret, "NO_RESET_ALL,"); + if (flags & GUC_NOT_IN_SAMPLE) + appendStringInfo(&ret, "NOT_IN_SAMPLE,"); + if (flags & GUC_EXPLAIN) + appendStringInfo(&ret, "EXPLAIN,"); + if (flags & GUC_RUNTIME_COMPUTED) + appendStringInfo(&ret, "RUNTIME_COMPUTED,"); + + /* Remove trailing comma, if any */ + if (ret.len > 1) + ret.data[--ret.len] = '\0';
The way of building the text array is incorrect here. See
heap_tuple_infomask_flags() in pageinspect as an example with all the
HEAP_* flags. I think that you should allocate an array of Datums,
use CStringGetTextDatum() to assign each array element, wrapping the
whole with construct_array() to build the final value for the
parameter tuple.
--
Michael
At Tue, 4 Jan 2022 21:06:48 -0600, Justin Pryzby <pryzby@telsasoft.com> wrote in
On Wed, Jan 05, 2022 at 11:47:57AM +0900, Kyotaro Horiguchi wrote:
At Tue, 28 Dec 2021 20:32:40 -0600, Justin Pryzby <pryzby@telsasoft.com> wrote in
In that case, *all* the flags should be exposed. There's currently 20, which
means it may not work well after all - it's already too long, and could get
longer, and/or overflow the alphabet...
Yeah, if we show all 20 properties, the string is too long as well as
all properties cannot have a sensible abbreviation character..
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
On Thu, Jan 06, 2022 at 02:19:08PM +0900, Michael Paquier wrote:
On Wed, Jan 05, 2022 at 05:55:17PM -0600, Justin Pryzby wrote:
pg_settings is currently defined with "SELECT *". Is it fine to enumerate a
list of columns instead ?I'd like to think that this is a better practice when it comes
documenting the columns, but I don't see an actual need for this extra
complication here.
The reason is to avoid showing the flags in the pg_settings view, which should
not be bloated just so we can retire check_guc.
+ initStringInfo(&ret); + appendStringInfoChar(&ret, '{'); + + if (flags & GUC_NO_SHOW_ALL) + appendStringInfo(&ret, "NO_SHOW_ALL,"); + if (flags & GUC_NO_RESET_ALL) + appendStringInfo(&ret, "NO_RESET_ALL,"); + if (flags & GUC_NOT_IN_SAMPLE) + appendStringInfo(&ret, "NOT_IN_SAMPLE,"); + if (flags & GUC_EXPLAIN) + appendStringInfo(&ret, "EXPLAIN,"); + if (flags & GUC_RUNTIME_COMPUTED) + appendStringInfo(&ret, "RUNTIME_COMPUTED,"); + + /* Remove trailing comma, if any */ + if (ret.len > 1) + ret.data[--ret.len] = '\0';The way of building the text array is incorrect here. See
heap_tuple_infomask_flags() in pageinspect as an example with all the
HEAP_* flags. I think that you should allocate an array of Datums,
use CStringGetTextDatum() to assign each array element, wrapping the
whole with construct_array() to build the final value for the
parameter tuple.
I actually did it that way last night ... however GetConfigOptionByNum() is
expecting it to return a text string, not an array.
--
Justin
On Wed, Jan 05, 2022 at 11:36:41PM -0600, Justin Pryzby wrote:
On Thu, Jan 06, 2022 at 02:19:08PM +0900, Michael Paquier wrote:
+ initStringInfo(&ret); + appendStringInfoChar(&ret, '{'); + + if (flags & GUC_NO_SHOW_ALL) + appendStringInfo(&ret, "NO_SHOW_ALL,"); + if (flags & GUC_NO_RESET_ALL) + appendStringInfo(&ret, "NO_RESET_ALL,"); + if (flags & GUC_NOT_IN_SAMPLE) + appendStringInfo(&ret, "NOT_IN_SAMPLE,"); + if (flags & GUC_EXPLAIN) + appendStringInfo(&ret, "EXPLAIN,"); + if (flags & GUC_RUNTIME_COMPUTED) + appendStringInfo(&ret, "RUNTIME_COMPUTED,"); + + /* Remove trailing comma, if any */ + if (ret.len > 1) + ret.data[--ret.len] = '\0';The way of building the text array is incorrect here. See
I think you'll find that this is how it's done elsewhere in postgres.
In the frontend, see appendPQExpBufferChar and appendPGArray and 3e6e86abc.
On the backend, see: git grep -F "'{'" |grep -w appendStringInfoChar
I updated the patch with a regex to accommodate GUCs without '=', as needed
since f47ed79cc8.
--
Justin
Attachments:
0002-Expose-GUC-flags-in-SQL-function-retire-.-check_guc.patchtext/x-diff; charset=us-asciiDownload
From f475b7eced95bb792c4509aa87373348b375742a Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Wed, 8 Dec 2021 12:06:35 -0600
Subject: [PATCH 2/2] Expose GUC flags in SQL function; retire ./check_guc
---
src/backend/catalog/system_views.sql | 19 ++++-
src/backend/utils/misc/check_guc | 29 -------
src/backend/utils/misc/guc.c | 37 +++++++-
src/include/catalog/pg_proc.dat | 6 +-
src/test/regress/expected/guc.out | 122 +++++++++++++++++++++++++++
src/test/regress/expected/rules.out | 2 +-
src/test/regress/sql/guc.sql | 72 ++++++++++++++++
7 files changed, 252 insertions(+), 35 deletions(-)
delete mode 100755 src/backend/utils/misc/check_guc
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 3cb69b1f87b..cdae0616bdd 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -582,7 +582,24 @@ FROM
JOIN pg_authid rol ON l.classoid = rol.tableoid AND l.objoid = rol.oid;
CREATE VIEW pg_settings AS
- SELECT * FROM pg_show_all_settings() AS A;
+ SELECT a.name,
+ a.setting,
+ a.unit,
+ a.category,
+ a.short_desc,
+ a.extra_desc,
+ a.context,
+ a.vartype,
+ a.source,
+ a.min_val,
+ a.max_val,
+ a.enumvals,
+ a.boot_val,
+ a.reset_val,
+ a.sourcefile,
+ a.sourceline,
+ a.pending_restart
+ FROM pg_show_all_settings() AS A;
CREATE RULE pg_settings_u AS
ON UPDATE TO pg_settings
diff --git a/src/backend/utils/misc/check_guc b/src/backend/utils/misc/check_guc
deleted file mode 100755
index 323ca13191b..00000000000
--- a/src/backend/utils/misc/check_guc
+++ /dev/null
@@ -1,29 +0,0 @@
-#! /bin/sh
-set -e
-
-## this script makes some assumptions about the formatting of files it parses.
-## in postgresql.conf.sample:
-## 1) the valid config settings may be preceded by a '#', but NOT '# '
-## (we use this to skip comments)
-
-### What options are listed in postgresql.conf.sample, but don't appear
-### in guc.c?
-
-# grab everything that looks like a setting
-SETTINGS=`sed '/^#[[:alnum:]]/!d; s/^#//; s/ =.*//; /^include/d' postgresql.conf.sample`
-
-for i in $SETTINGS ; do
- ## it sure would be nice to replace this with an sql "not in" statement
- grep -i "\"$i\"" guc.c >/dev/null ||
- echo "$i seems to be missing from guc.c";
-done
-
-### What options are listed in guc.c, but don't appear
-### in postgresql.conf.sample?
-
-# grab everything that looks like a setting and convert it to lower case
-SETTINGS=`gawk -F '[",]' 'BEGIN{RS="\n\t\\\\{\n"} /",[[:space:]]*PGC_.*.*gettext_noop/ && !/NOT_IN_SAMPLE/{print tolower($2)}' guc.c`
-for i in $SETTINGS ; do
- grep "#$i " postgresql.conf.sample >/dev/null ||
- echo "$i seems to be missing from postgresql.conf.sample";
-done
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 4c94f09c645..2cb84d68610 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9634,6 +9634,36 @@ GetConfigOptionByName(const char *name, const char **varname, bool missing_ok)
return _ShowOption(record, true);
}
+/*
+ * Return an text[] array of the given flags (or a useful subset?) XXX
+ */
+static char *
+get_flags_array_text(int flags)
+{
+ StringInfoData ret;
+
+ initStringInfo(&ret);
+ appendStringInfoChar(&ret, '{');
+
+ if (flags & GUC_NO_SHOW_ALL)
+ appendStringInfo(&ret, "NO_SHOW_ALL,");
+ if (flags & GUC_NO_RESET_ALL)
+ appendStringInfo(&ret, "NO_RESET_ALL,");
+ if (flags & GUC_NOT_IN_SAMPLE)
+ appendStringInfo(&ret, "NOT_IN_SAMPLE,");
+ if (flags & GUC_EXPLAIN)
+ appendStringInfo(&ret, "EXPLAIN,");
+ if (flags & GUC_RUNTIME_COMPUTED)
+ appendStringInfo(&ret, "RUNTIME_COMPUTED,");
+
+ /* Remove trailing comma, if any */
+ if (ret.len > 1)
+ ret.data[--ret.len] = '\0';
+
+ appendStringInfoChar(&ret, '}');
+ return ret.data;
+}
+
/*
* Return GUC variable value by variable number; optionally return canonical
* form of name. Return value is palloc'd.
@@ -9861,6 +9891,9 @@ GetConfigOptionByNum(int varnum, const char **values, bool *noshow)
}
values[16] = (conf->status & GUC_PENDING_RESTART) ? "t" : "f";
+
+ /* flags */
+ values[17] = get_flags_array_text(conf->flags);
}
/*
@@ -9916,7 +9949,7 @@ show_config_by_name_missing_ok(PG_FUNCTION_ARGS)
* show_all_settings - equiv to SHOW ALL command but implemented as
* a Table Function.
*/
-#define NUM_PG_SETTINGS_ATTS 17
+#define NUM_PG_SETTINGS_ATTS 18
Datum
show_all_settings(PG_FUNCTION_ARGS)
@@ -9978,6 +10011,8 @@ show_all_settings(PG_FUNCTION_ARGS)
INT4OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 17, "pending_restart",
BOOLOID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 18, "flags",
+ TEXTARRAYOID, -1, 0);
/*
* Generate attribute metadata needed later to produce tuples from raw
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 0859dc81cac..1d7f6b4d376 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6092,9 +6092,9 @@
{ oid => '2084', descr => 'SHOW ALL as a function',
proname => 'pg_show_all_settings', prorows => '1000', proretset => 't',
provolatile => 's', prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,text,text,text,text,text,text,text,text,text,text,_text,text,text,text,int4,bool}',
- proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{name,setting,unit,category,short_desc,extra_desc,context,vartype,source,min_val,max_val,enumvals,boot_val,reset_val,sourcefile,sourceline,pending_restart}',
+ proallargtypes => '{text,text,text,text,text,text,text,text,text,text,text,_text,text,text,text,int4,bool,_text}',
+ proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{name,setting,unit,category,short_desc,extra_desc,context,vartype,source,min_val,max_val,enumvals,boot_val,reset_val,sourcefile,sourceline,pending_restart,flags}',
prosrc => 'show_all_settings' },
{ oid => '3329', descr => 'show config file settings',
proname => 'pg_show_all_file_settings', prorows => '1000', proretset => 't',
diff --git a/src/test/regress/expected/guc.out b/src/test/regress/expected/guc.out
index 59da91ff04d..5f15b5680be 100644
--- a/src/test/regress/expected/guc.out
+++ b/src/test/regress/expected/guc.out
@@ -813,3 +813,125 @@ set default_with_oids to f;
-- Should not allow to set it to true.
set default_with_oids to t;
ERROR: tables declared WITH OIDS are not supported
+--
+-- Test GUC categories and flags.
+--
+CREATE TABLE pg_settings_flags AS SELECT name, category,
+ 'NO_SHOW_ALL' =ANY(flags) AS no_show_all,
+ 'NO_RESET_ALL' =ANY(flags) AS no_reset_all,
+ 'NOT_IN_SAMPLE' =ANY(flags) AS not_in_sample,
+ 'EXPLAIN' =ANY(flags) AS guc_explain,
+ 'COMPUTED' =ANY(flags) AS guc_computed
+ FROM pg_show_all_settings();
+-- test that GUCS are in postgresql.conf
+SELECT lower(name) FROM pg_settings_flags WHERE NOT not_in_sample EXCEPT
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) (= .*|[^ ]*$)', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln ~ '^#?[[:alpha:]]'
+ORDER BY 1;
+ lower
+-----------------------------
+ config_file
+ plpgsql.check_asserts
+ plpgsql.extra_errors
+ plpgsql.extra_warnings
+ plpgsql.print_strict_params
+ plpgsql.variable_conflict
+(6 rows)
+
+-- test that lines in postgresql.conf that look like GUCs are GUCs
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) (= .*|[^ ]*$)', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln ~ '^#?[[:alpha:]]'
+EXCEPT SELECT lower(name) FROM pg_settings_flags WHERE NOT not_in_sample
+ORDER BY 1;
+ guc
+-------------------
+ include
+ include_dir
+ include_if_exists
+(3 rows)
+
+-- test that section:DEVELOPER GUCs are flagged GUC_NOT_IN_SAMPLE:
+SELECT * FROM pg_settings_flags
+WHERE category='Developer Options' AND NOT not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+-- Maybe the converse:
+SELECT * FROM pg_settings_flags
+WHERE category NOT IN ('Developer Options', 'Preset Options') AND not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------------------------+-------------------------------------------------+-------------+--------------+---------------+-------------+--------------
+ application_name | Reporting and Logging / What to Log | f | f | t | f | f
+ transaction_deferrable | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+ transaction_isolation | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+ transaction_read_only | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+(4 rows)
+
+-- Most Query Tuning GUCs are flagged as EXPLAIN:
+SELECT * FROM pg_settings_flags
+WHERE category ~ '^Query Tuning' AND NOT guc_explain
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+---------------------------+--------------------------------------+-------------+--------------+---------------+-------------+--------------
+ default_statistics_target | Query Tuning / Other Planner Options | f | f | f | f | f
+(1 row)
+
+-- Maybe the converse:
+SELECT * FROM pg_settings_flags
+WHERE guc_explain AND NOT category ~ '^Query Tuning|^Resource Usage'
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+---------------------+-------------------------------------------------+-------------+--------------+---------------+-------------+--------------
+ force_parallel_mode | Developer Options | f | f | t | t | f
+ search_path | Client Connection Defaults / Statement Behavior | f | f | f | t | f
+(2 rows)
+
+-- GUCs flagged RUNTIME are Preset
+SELECT * FROM pg_settings_flags
+WHERE guc_computed AND NOT category='Preset Options'
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+-- PRESET GUCs are flagged NOT_IN_SAMPLE
+SELECT * FROM pg_settings_flags
+WHERE category='Preset Options' AND NOT not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+-- NO_SHOW_ALL implies NO_RESET_ALL:
+SELECT * FROM pg_settings_flags
+WHERE no_show_all AND NOT no_reset_all
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+-- Usually the converse:
+SELECT * FROM pg_settings_flags
+WHERE NOT no_show_all AND no_reset_all
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------------------------+-------------------------------------------------+-------------+--------------+---------------+-------------+--------------
+ transaction_deferrable | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+ transaction_isolation | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+ transaction_read_only | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+(3 rows)
+
+-- NO_SHOW_ALL implies NOT_IN_SAMPLE:
+SELECT * FROM pg_settings_flags
+WHERE no_show_all AND NOT not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+DROP TABLE pg_settings_flags;
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index d652f7b5fb4..833c1fab6c7 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1725,7 +1725,7 @@ pg_settings| SELECT a.name,
a.sourcefile,
a.sourceline,
a.pending_restart
- FROM pg_show_all_settings() a(name, setting, unit, category, short_desc, extra_desc, context, vartype, source, min_val, max_val, enumvals, boot_val, reset_val, sourcefile, sourceline, pending_restart);
+ FROM pg_show_all_settings() a(name, setting, unit, category, short_desc, extra_desc, context, vartype, source, min_val, max_val, enumvals, boot_val, reset_val, sourcefile, sourceline, pending_restart, flags);
pg_shadow| SELECT pg_authid.rolname AS usename,
pg_authid.oid AS usesysid,
pg_authid.rolcreatedb AS usecreatedb,
diff --git a/src/test/regress/sql/guc.sql b/src/test/regress/sql/guc.sql
index c39c11388d5..c3898ae690f 100644
--- a/src/test/regress/sql/guc.sql
+++ b/src/test/regress/sql/guc.sql
@@ -311,3 +311,75 @@ reset check_function_bodies;
set default_with_oids to f;
-- Should not allow to set it to true.
set default_with_oids to t;
+
+--
+-- Test GUC categories and flags.
+--
+CREATE TABLE pg_settings_flags AS SELECT name, category,
+ 'NO_SHOW_ALL' =ANY(flags) AS no_show_all,
+ 'NO_RESET_ALL' =ANY(flags) AS no_reset_all,
+ 'NOT_IN_SAMPLE' =ANY(flags) AS not_in_sample,
+ 'EXPLAIN' =ANY(flags) AS guc_explain,
+ 'COMPUTED' =ANY(flags) AS guc_computed
+ FROM pg_show_all_settings();
+
+-- test that GUCS are in postgresql.conf
+SELECT lower(name) FROM pg_settings_flags WHERE NOT not_in_sample EXCEPT
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) (= .*|[^ ]*$)', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln ~ '^#?[[:alpha:]]'
+ORDER BY 1;
+
+-- test that lines in postgresql.conf that look like GUCs are GUCs
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) (= .*|[^ ]*$)', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln ~ '^#?[[:alpha:]]'
+EXCEPT SELECT lower(name) FROM pg_settings_flags WHERE NOT not_in_sample
+ORDER BY 1;
+
+-- test that section:DEVELOPER GUCs are flagged GUC_NOT_IN_SAMPLE:
+SELECT * FROM pg_settings_flags
+WHERE category='Developer Options' AND NOT not_in_sample
+ORDER BY 1;
+
+-- Maybe the converse:
+SELECT * FROM pg_settings_flags
+WHERE category NOT IN ('Developer Options', 'Preset Options') AND not_in_sample
+ORDER BY 1;
+
+-- Most Query Tuning GUCs are flagged as EXPLAIN:
+SELECT * FROM pg_settings_flags
+WHERE category ~ '^Query Tuning' AND NOT guc_explain
+ORDER BY 1;
+
+-- Maybe the converse:
+SELECT * FROM pg_settings_flags
+WHERE guc_explain AND NOT category ~ '^Query Tuning|^Resource Usage'
+ORDER BY 1;
+
+-- GUCs flagged RUNTIME are Preset
+SELECT * FROM pg_settings_flags
+WHERE guc_computed AND NOT category='Preset Options'
+ORDER BY 1;
+
+-- PRESET GUCs are flagged NOT_IN_SAMPLE
+SELECT * FROM pg_settings_flags
+WHERE category='Preset Options' AND NOT not_in_sample
+ORDER BY 1;
+
+-- NO_SHOW_ALL implies NO_RESET_ALL:
+SELECT * FROM pg_settings_flags
+WHERE no_show_all AND NOT no_reset_all
+ORDER BY 1;
+
+-- Usually the converse:
+SELECT * FROM pg_settings_flags
+WHERE NOT no_show_all AND no_reset_all
+ORDER BY 1;
+
+-- NO_SHOW_ALL implies NOT_IN_SAMPLE:
+SELECT * FROM pg_settings_flags
+WHERE no_show_all AND NOT not_in_sample
+ORDER BY 1;
+
+DROP TABLE pg_settings_flags;
--
2.17.1
0001-check_guc-fix-absurd-number-of-false-positives.patchtext/x-diff; charset=us-asciiDownload
From eee345a1685a04eeb0fedf3ba9ad0fc6fe6e3d81 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 5 Dec 2021 23:22:04 -0600
Subject: [PATCH 1/2] check_guc: fix absurd number of false positives
Avoid false positives;
Avoid various assumptions;
Avoid list of exceptions;
Simplify shell/awk/sed/grep;
This requires GNU awk for RS as a regex.
---
src/backend/utils/misc/check_guc | 69 +++++---------------------------
1 file changed, 10 insertions(+), 59 deletions(-)
diff --git a/src/backend/utils/misc/check_guc b/src/backend/utils/misc/check_guc
index b171ef0e4ff..323ca13191b 100755
--- a/src/backend/utils/misc/check_guc
+++ b/src/backend/utils/misc/check_guc
@@ -1,78 +1,29 @@
-#!/bin/sh
+#! /bin/sh
+set -e
-## currently, this script makes a lot of assumptions:
+## this script makes some assumptions about the formatting of files it parses.
## in postgresql.conf.sample:
## 1) the valid config settings may be preceded by a '#', but NOT '# '
## (we use this to skip comments)
-## 2) the valid config settings will be followed immediately by ' ='
-## (at least one space preceding the '=')
-## in guc.c:
-## 3) the options have PGC_ on the same line as the option
-## 4) the options have '{' on the same line as the option
-
-## Problems
-## 1) Don't know what to do with TRANSACTION ISOLATION LEVEL
-
-## if an option is valid but shows up in only one file (guc.c but not
-## postgresql.conf.sample), it should be listed here so that it
-## can be ignored
-INTENTIONALLY_NOT_INCLUDED="debug_deadlocks in_hot_standby \
-is_superuser lc_collate lc_ctype lc_messages lc_monetary lc_numeric lc_time \
-pre_auth_delay role seed server_encoding server_version server_version_num \
-session_authorization trace_lock_oidmin trace_lock_table trace_locks trace_lwlocks \
-trace_notify trace_userlocks transaction_isolation transaction_read_only \
-zero_damaged_pages"
### What options are listed in postgresql.conf.sample, but don't appear
### in guc.c?
-# grab everything that looks like a setting and convert it to lower case
-SETTINGS=`grep ' =' postgresql.conf.sample |
-grep -v '^# ' | # strip comments
-sed -e 's/^#//' |
-awk '{print $1}'`
-
-SETTINGS=`echo "$SETTINGS" | tr 'A-Z' 'a-z'`
+# grab everything that looks like a setting
+SETTINGS=`sed '/^#[[:alnum:]]/!d; s/^#//; s/ =.*//; /^include/d' postgresql.conf.sample`
for i in $SETTINGS ; do
- hidden=0
## it sure would be nice to replace this with an sql "not in" statement
- ## it doesn't seem to make sense to have things in .sample and not in guc.c
-# for hidethis in $INTENTIONALLY_NOT_INCLUDED ; do
-# if [ "$hidethis" = "$i" ] ; then
-# hidden=1
-# fi
-# done
- if [ "$hidden" -eq 0 ] ; then
- grep -i '"'$i'"' guc.c > /dev/null
- if [ $? -ne 0 ] ; then
- echo "$i seems to be missing from guc.c";
- fi;
- fi
+ grep -i "\"$i\"" guc.c >/dev/null ||
+ echo "$i seems to be missing from guc.c";
done
### What options are listed in guc.c, but don't appear
### in postgresql.conf.sample?
# grab everything that looks like a setting and convert it to lower case
-
-SETTINGS=`grep '{.* PGC_' guc.c | awk '{print $1}' | \
- sed -e 's/{//g' -e 's/"//g' -e 's/,//'`
-
-SETTINGS=`echo "$SETTINGS" | tr 'A-Z' 'a-z'`
-
+SETTINGS=`gawk -F '[",]' 'BEGIN{RS="\n\t\\\\{\n"} /",[[:space:]]*PGC_.*.*gettext_noop/ && !/NOT_IN_SAMPLE/{print tolower($2)}' guc.c`
for i in $SETTINGS ; do
- hidden=0
- ## it sure would be nice to replace this with an sql "not in" statement
- for hidethis in $INTENTIONALLY_NOT_INCLUDED ; do
- if [ "$hidethis" = "$i" ] ; then
- hidden=1
- fi
- done
- if [ "$hidden" -eq 0 ] ; then
- grep -i '#'$i' ' postgresql.conf.sample > /dev/null
- if [ $? -ne 0 ] ; then
- echo "$i seems to be missing from postgresql.conf.sample";
- fi
- fi
+ grep "#$i " postgresql.conf.sample >/dev/null ||
+ echo "$i seems to be missing from postgresql.conf.sample";
done
--
2.17.1
On Mon, Jan 24, 2022 at 07:07:29PM -0600, Justin Pryzby wrote:
I think you'll find that this is how it's done elsewhere in postgres.
In the frontend, see appendPQExpBufferChar and appendPGArray and 3e6e86abc.
On the backend, see: git grep -F "'{'" |grep -w appendStringInfoChar
Yeah, I was not careful enough to look after the uses of TEXTARRAYOID,
and there is one in the same area, as of config_enum_get_options().
At least things are consistent this way.
I updated the patch with a regex to accommodate GUCs without '=', as needed
since f47ed79cc8.
Okay. While looking at your proposal, I was thinking that we had
better include the array with the flags by default in pg_settings, and
not just pg_show_all_settings().
+SELECT lower(name) FROM pg_settings_flags WHERE NOT not_in_sample EXCEPT
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) (= .*|[^ ]*$)', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'),
'\n') AS ln) conf
Tests reading postgresql.conf would break on instances started with a
custom config_file provided by a command line, no? You could change
the patch to use the value provided by the GUC, instead, but I am not
convinced that we need that at all, even if check_guc does so.
Regarding the tests, I am not sure if we need to be this much
extensive. We could take is slow, and I am also wondering if this
could not cause some issues with GUCs loaded via
shared_preload_libraries if we are too picky about the requirements,
as this could cause installcheck failures.
The following things have been issues recently, though, and they look
sensible enough to have checks for:
- GUC_NOT_IN_SAMPLE with developer options.
- Query-tuning parameters with GUC_EXPLAIN, and we'd better add some
comments in the test to explain why there are exceptions like
default_statistics_target.
- preset parameters marked as runtime-computed.
- NO_SHOW_ALL and NOT_IN_SAMPLE.
Thanks,
--
Michael
On 25.01.22 02:07, Justin Pryzby wrote:
+CREATE TABLE pg_settings_flags AS SELECT name, category, + 'NO_SHOW_ALL' =ANY(flags) AS no_show_all, + 'NO_RESET_ALL' =ANY(flags) AS no_reset_all, + 'NOT_IN_SAMPLE' =ANY(flags) AS not_in_sample, + 'EXPLAIN' =ANY(flags) AS guc_explain, + 'COMPUTED' =ANY(flags) AS guc_computed + FROM pg_show_all_settings();
Does this stuff have any value for users? I'm worried we are exposing a
bunch of stuff that is really just for internal purposes. Like, what
value does showing "not_in_sample" have? On the other hand,
"guc_explain" might be genuinely useful, since that is part of a
user-facing feature. (I don't like the "guc_*" naming though.)
Your patch doesn't contain a documentation change, so I don't know how
and to what extend this is supposed to be presented to users.
On Tue, Jan 25, 2022 at 11:47:14AM +0100, Peter Eisentraut wrote:
On 25.01.22 02:07, Justin Pryzby wrote:
+CREATE TABLE pg_settings_flags AS SELECT name, category, + 'NO_SHOW_ALL' =ANY(flags) AS no_show_all, + 'NO_RESET_ALL' =ANY(flags) AS no_reset_all, + 'NOT_IN_SAMPLE' =ANY(flags) AS not_in_sample, + 'EXPLAIN' =ANY(flags) AS guc_explain, + 'COMPUTED' =ANY(flags) AS guc_computed + FROM pg_show_all_settings();Does this stuff have any value for users? I'm worried we are exposing a
bunch of stuff that is really just for internal purposes. Like, what value
does showing "not_in_sample" have? On the other hand, "guc_explain" might
be genuinely useful, since that is part of a user-facing feature. (I don't
like the "guc_*" naming though.)Your patch doesn't contain a documentation change, so I don't know how and
to what extend this is supposed to be presented to users.
I want to avoid putting this in pg_settings.
The two options discussed so far are:
- to add an function to return the flags;
- to add the flags to pg_show_all_settings(), but not show it in pg_settings view;
I interpretted Michael's suggested as adding it to pg_get_all_settings(), but
*not* including it in the pg_settings view. Now it seems like I misunderstood,
and Michael wants to add it to the view.
But, even if we only handle the 5 flags we have an immediate use for, it makes
the user-facing view too "wide", just to accommodate this internal use.
If it were in the pg_settings view, I think it ought to have *all* the flags
(not just the flags that help us to retire ./check_guc). That's much too much.
--
Justin
On Tue, Jan 25, 2022 at 12:07:51PM -0600, Justin Pryzby wrote:
On Tue, Jan 25, 2022 at 11:47:14AM +0100, Peter Eisentraut wrote:
Does this stuff have any value for users? I'm worried we are exposing a
bunch of stuff that is really just for internal purposes. Like, what value
does showing "not_in_sample" have? On the other hand, "guc_explain" might
be genuinely useful, since that is part of a user-facing feature. (I don't
like the "guc_*" naming though.)
EXPLAIN is useful to know which parameter could be part of an explain
query, as that's not an information provided now, even if the category
provides a hint. COMPUTED is also useful for the purpose of postgres
-C in my opinion. I am reserved about the rest in terms of user
experience, but the other ones are useful to automate the checks
check_guc was doing, which is still the main goal of this patch if we
remove this script. And experience has proved lately that people
forget a lot to mark GUCs correctly.
I interpretted Michael's suggested as adding it to pg_get_all_settings(), but
*not* including it in the pg_settings view. Now it seems like I misunderstood,
and Michael wants to add it to the view.
Yeah, I meant to add that in the view, as it is already wide. I'd be
fine with a separate SQL function at the end, but putting that in
pg_show_all_settings() without considering pg_settings would not be
consistent. There is the argument that one could miss an update of
system_views.sql if adding more data to pg_show_all_settings(), even
if that's not really going to happen.
But, even if we only handle the 5 flags we have an immediate use for, it makes
the user-facing view too "wide", just to accommodate this internal use.
short_desc and extra_desc count for most of the bloat already, so that
would not change much, but I am fine to discard my point to not make
things worse.
--
Michael
On Wed, Jan 26, 2022 at 09:54:43AM +0900, Michael Paquier wrote:
On Tue, Jan 25, 2022 at 12:07:51PM -0600, Justin Pryzby wrote:
On Tue, Jan 25, 2022 at 11:47:14AM +0100, Peter Eisentraut wrote:
Does this stuff have any value for users? I'm worried we are exposing a
bunch of stuff that is really just for internal purposes. Like, what value
does showing "not_in_sample" have? On the other hand, "guc_explain" might
be genuinely useful, since that is part of a user-facing feature. (I don't
like the "guc_*" naming though.)EXPLAIN is useful to know which parameter could be part of an explain
query, as that's not an information provided now, even if the category
provides a hint. COMPUTED is also useful for the purpose of postgres
-C in my opinion.
It seems like an arbitrary and short-sighted policy to expose a handful of
flags in the view for the purpose of retiring ./check_guc, but not expose other
flags, because we thought we knew that no user could ever want them.
We should either expose all the flags, or should put them into an undocumented
function. Otherwise, how would we document the flags argument ? "Shows some
of the flags" ? An undocumented function avoids this issue.
Should I update the patch to put the function back ?
Should I also make the function expose all of the flags ?
--
Justin
On Tue, Jan 25, 2022 at 09:44:26PM -0600, Justin Pryzby wrote:
It seems like an arbitrary and short-sighted policy to expose a handful of
flags in the view for the purpose of retiring ./check_guc, but not expose other
flags, because we thought we knew that no user could ever want them.We should either expose all the flags, or should put them into an undocumented
function. Otherwise, how would we document the flags argument ? "Shows some
of the flags" ? An undocumented function avoids this issue.
My vote would be to have a documented function, with a minimal set of
the flags exposed and documented, with the option to expand that in
the future. COMPUTED and EXPLAIN are useful, and allow some of the
automated tests to happen. NOT_IN_SAMPLE and GUC_NO_SHOW_ALL are less
useful for the user, and are more developer oriented, but are useful
for the tests. So having these four seem like a good first cut.
--
Michael
On Wed, Jan 26, 2022 at 03:29:29PM +0900, Michael Paquier wrote:
On Tue, Jan 25, 2022 at 09:44:26PM -0600, Justin Pryzby wrote:
It seems like an arbitrary and short-sighted policy to expose a handful of
flags in the view for the purpose of retiring ./check_guc, but not expose other
flags, because we thought we knew that no user could ever want them.We should either expose all the flags, or should put them into an undocumented
function. Otherwise, how would we document the flags argument ? "Shows some
of the flags" ? An undocumented function avoids this issue.My vote would be to have a documented function, with a minimal set of
the flags exposed and documented, with the option to expand that in
the future. COMPUTED and EXPLAIN are useful, and allow some of the
automated tests to happen. NOT_IN_SAMPLE and GUC_NO_SHOW_ALL are less
useful for the user, and are more developer oriented, but are useful
for the tests. So having these four seem like a good first cut.
I implemented that (But my own preference would still be for an *undocumented*
function which returns whatever flags we find to be useful to include. Or
alternately, a documented function which exposes every flag).
+SELECT lower(name) FROM pg_settings_flags WHERE NOT not_in_sample EXCEPT +SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) (= .*|[^ ]*$)', '\1') AS guc +FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) confTests reading postgresql.conf would break on instances started with a
custom config_file provided by a command line, no?
Maybe you misunderstood - I'm not reading the file specified by
current_setting('config_file'). Rather, I'm reading
tmp_check/data/postgresql.conf, which is copied from the sample conf.
Do you see an issue with that ?
The regression tests are only intended run from a postgres source dir, and if
someone runs the from somewhere else, and they "fail", I think that's because
they violated their assumption, not because of a problem with the test.
I wondered if it should chomp off anything added by pg_regress --temp-regress.
However that's either going to be a valid guc (or else it would fail some other
test). Or an extention's guc (which this isn't testing), which has a dot, and
which this regex doesn't match, so doesn't cause false positives.
--
Justin
Attachments:
0001-check_guc-fix-absurd-number-of-false-positives.patchtext/x-diff; charset=us-asciiDownload
From d09cc770a3e307f55843942a850f7859c63d4c76 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sun, 5 Dec 2021 23:22:04 -0600
Subject: [PATCH 1/2] check_guc: fix absurd number of false positives
Avoid false positives;
Avoid various assumptions;
Avoid list of exceptions;
Simplify shell/awk/sed/grep;
This requires GNU awk for RS as a regex.
---
src/backend/utils/misc/check_guc | 69 +++++---------------------------
1 file changed, 10 insertions(+), 59 deletions(-)
diff --git a/src/backend/utils/misc/check_guc b/src/backend/utils/misc/check_guc
index b171ef0e4ff..323ca13191b 100755
--- a/src/backend/utils/misc/check_guc
+++ b/src/backend/utils/misc/check_guc
@@ -1,78 +1,29 @@
-#!/bin/sh
+#! /bin/sh
+set -e
-## currently, this script makes a lot of assumptions:
+## this script makes some assumptions about the formatting of files it parses.
## in postgresql.conf.sample:
## 1) the valid config settings may be preceded by a '#', but NOT '# '
## (we use this to skip comments)
-## 2) the valid config settings will be followed immediately by ' ='
-## (at least one space preceding the '=')
-## in guc.c:
-## 3) the options have PGC_ on the same line as the option
-## 4) the options have '{' on the same line as the option
-
-## Problems
-## 1) Don't know what to do with TRANSACTION ISOLATION LEVEL
-
-## if an option is valid but shows up in only one file (guc.c but not
-## postgresql.conf.sample), it should be listed here so that it
-## can be ignored
-INTENTIONALLY_NOT_INCLUDED="debug_deadlocks in_hot_standby \
-is_superuser lc_collate lc_ctype lc_messages lc_monetary lc_numeric lc_time \
-pre_auth_delay role seed server_encoding server_version server_version_num \
-session_authorization trace_lock_oidmin trace_lock_table trace_locks trace_lwlocks \
-trace_notify trace_userlocks transaction_isolation transaction_read_only \
-zero_damaged_pages"
### What options are listed in postgresql.conf.sample, but don't appear
### in guc.c?
-# grab everything that looks like a setting and convert it to lower case
-SETTINGS=`grep ' =' postgresql.conf.sample |
-grep -v '^# ' | # strip comments
-sed -e 's/^#//' |
-awk '{print $1}'`
-
-SETTINGS=`echo "$SETTINGS" | tr 'A-Z' 'a-z'`
+# grab everything that looks like a setting
+SETTINGS=`sed '/^#[[:alnum:]]/!d; s/^#//; s/ =.*//; /^include/d' postgresql.conf.sample`
for i in $SETTINGS ; do
- hidden=0
## it sure would be nice to replace this with an sql "not in" statement
- ## it doesn't seem to make sense to have things in .sample and not in guc.c
-# for hidethis in $INTENTIONALLY_NOT_INCLUDED ; do
-# if [ "$hidethis" = "$i" ] ; then
-# hidden=1
-# fi
-# done
- if [ "$hidden" -eq 0 ] ; then
- grep -i '"'$i'"' guc.c > /dev/null
- if [ $? -ne 0 ] ; then
- echo "$i seems to be missing from guc.c";
- fi;
- fi
+ grep -i "\"$i\"" guc.c >/dev/null ||
+ echo "$i seems to be missing from guc.c";
done
### What options are listed in guc.c, but don't appear
### in postgresql.conf.sample?
# grab everything that looks like a setting and convert it to lower case
-
-SETTINGS=`grep '{.* PGC_' guc.c | awk '{print $1}' | \
- sed -e 's/{//g' -e 's/"//g' -e 's/,//'`
-
-SETTINGS=`echo "$SETTINGS" | tr 'A-Z' 'a-z'`
-
+SETTINGS=`gawk -F '[",]' 'BEGIN{RS="\n\t\\\\{\n"} /",[[:space:]]*PGC_.*.*gettext_noop/ && !/NOT_IN_SAMPLE/{print tolower($2)}' guc.c`
for i in $SETTINGS ; do
- hidden=0
- ## it sure would be nice to replace this with an sql "not in" statement
- for hidethis in $INTENTIONALLY_NOT_INCLUDED ; do
- if [ "$hidethis" = "$i" ] ; then
- hidden=1
- fi
- done
- if [ "$hidden" -eq 0 ] ; then
- grep -i '#'$i' ' postgresql.conf.sample > /dev/null
- if [ $? -ne 0 ] ; then
- echo "$i seems to be missing from postgresql.conf.sample";
- fi
- fi
+ grep "#$i " postgresql.conf.sample >/dev/null ||
+ echo "$i seems to be missing from postgresql.conf.sample";
done
--
2.17.1
0002-Expose-GUC-flags-in-SQL-function-retire-.-check_guc.patchtext/x-diff; charset=us-asciiDownload
From ee2b323f329cc6ff1412bfbc5ad0b64c3584ff03 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Wed, 8 Dec 2021 12:06:35 -0600
Subject: [PATCH 2/2] Expose GUC flags in SQL function; retire ./check_guc
---
doc/src/sgml/func.sgml | 15 ++++
src/backend/utils/misc/check_guc | 29 -------
src/backend/utils/misc/guc.c | 37 +++++++++
src/include/catalog/pg_proc.dat | 6 ++
src/include/utils/guc.h | 3 +-
src/test/regress/expected/guc.out | 122 ++++++++++++++++++++++++++++++
src/test/regress/sql/guc.sql | 72 ++++++++++++++++++
7 files changed, 254 insertions(+), 30 deletions(-)
delete mode 100755 src/backend/utils/misc/check_guc
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 0ee6974f1c6..cbdbccb63d1 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -23596,6 +23596,21 @@ SELECT pg_type_is_visible('myschema.widget'::regtype);
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_get_get_flags</primary>
+ </indexterm>
+ <function>pg_get_get_flags</function> ( <parameter>guc</parameter> <type>text</type> )
+ <returnvalue>text[]</returnvalue>
+ </para>
+ <para>
+ Return an array of flags associated with the given GUC, or NULL if the
+ GUC does not exist. Not all flags are exposed; the set of flags which
+ are exposed is subject to change.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/utils/misc/check_guc b/src/backend/utils/misc/check_guc
deleted file mode 100755
index 323ca13191b..00000000000
--- a/src/backend/utils/misc/check_guc
+++ /dev/null
@@ -1,29 +0,0 @@
-#! /bin/sh
-set -e
-
-## this script makes some assumptions about the formatting of files it parses.
-## in postgresql.conf.sample:
-## 1) the valid config settings may be preceded by a '#', but NOT '# '
-## (we use this to skip comments)
-
-### What options are listed in postgresql.conf.sample, but don't appear
-### in guc.c?
-
-# grab everything that looks like a setting
-SETTINGS=`sed '/^#[[:alnum:]]/!d; s/^#//; s/ =.*//; /^include/d' postgresql.conf.sample`
-
-for i in $SETTINGS ; do
- ## it sure would be nice to replace this with an sql "not in" statement
- grep -i "\"$i\"" guc.c >/dev/null ||
- echo "$i seems to be missing from guc.c";
-done
-
-### What options are listed in guc.c, but don't appear
-### in postgresql.conf.sample?
-
-# grab everything that looks like a setting and convert it to lower case
-SETTINGS=`gawk -F '[",]' 'BEGIN{RS="\n\t\\\\{\n"} /",[[:space:]]*PGC_.*.*gettext_noop/ && !/NOT_IN_SAMPLE/{print tolower($2)}' guc.c`
-for i in $SETTINGS ; do
- grep "#$i " postgresql.conf.sample >/dev/null ||
- echo "$i seems to be missing from postgresql.conf.sample";
-done
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 4c94f09c645..d3b56b2007c 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9634,6 +9634,43 @@ GetConfigOptionByName(const char *name, const char **varname, bool missing_ok)
return _ShowOption(record, true);
}
+/*
+ * Return some flags for the specified GUC, or NULL if it doesn't exist.
+ */
+Datum
+pg_get_guc_flags(PG_FUNCTION_ARGS)
+{
+ char *varname = TextDatumGetCString(PG_GETARG_DATUM(0));
+ struct config_generic *record;
+ int cnt = 0;
+#define MAX_NUM_FLAGS 5
+ Datum flags[MAX_NUM_FLAGS];
+ ArrayType *a;
+
+ record = find_option(varname, false, true, ERROR);
+
+ /* return NULL if no such variable */
+ if (record == NULL)
+ PG_RETURN_NULL();
+
+ if (record->flags & GUC_NO_SHOW_ALL)
+ flags[cnt++] = CStringGetTextDatum("NO_SHOW_ALL");
+ if (record->flags & GUC_NO_RESET_ALL)
+ flags[cnt++] = CStringGetTextDatum("NO_RESET_ALL");
+ if (record->flags & GUC_NOT_IN_SAMPLE)
+ flags[cnt++] = CStringGetTextDatum("NOT_IN_SAMPLE");
+ if (record->flags & GUC_EXPLAIN)
+ flags[cnt++] = CStringGetTextDatum("EXPLAIN");
+ if (record->flags & GUC_RUNTIME_COMPUTED)
+ flags[cnt++] = CStringGetTextDatum("RUNTIME_COMPUTED");
+
+ Assert(cnt <= MAX_NUM_FLAGS);
+
+ /* Returns the record as Datum */
+ a = construct_array(flags, cnt, TEXTOID, -1, false, TYPALIGN_INT);
+ PG_RETURN_ARRAYTYPE_P(a);
+}
+
/*
* Return GUC variable value by variable number; optionally return canonical
* form of name. Return value is palloc'd.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 0859dc81cac..562c1d779cb 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6096,6 +6096,12 @@
proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{name,setting,unit,category,short_desc,extra_desc,context,vartype,source,min_val,max_val,enumvals,boot_val,reset_val,sourcefile,sourceline,pending_restart}',
prosrc => 'show_all_settings' },
+
+{ oid => '8921', descr => 'return flags for specified GUC',
+ proname => 'pg_get_guc_flags', provolatile => 's',
+ prorettype => '_text', proargtypes => 'text',
+ prosrc => 'pg_get_guc_flags' },
+
{ oid => '3329', descr => 'show config file settings',
proname => 'pg_show_all_file_settings', prorows => '1000', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h
index 6bb81707b09..1ac20f85ab3 100644
--- a/src/include/utils/guc.h
+++ b/src/include/utils/guc.h
@@ -198,8 +198,9 @@ typedef enum
#define GUC_QUALIFIER_SEPARATOR '.'
-/*
+/* --
* bit values in "flags" of a GUC variable
+ * Consider if any new flags should be exposed in pg_get_guc_flags().
*/
#define GUC_LIST_INPUT 0x0001 /* input can be list format */
#define GUC_LIST_QUOTE 0x0002 /* double-quote list elements */
diff --git a/src/test/regress/expected/guc.out b/src/test/regress/expected/guc.out
index 59da91ff04d..29173f3ce8e 100644
--- a/src/test/regress/expected/guc.out
+++ b/src/test/regress/expected/guc.out
@@ -813,3 +813,125 @@ set default_with_oids to f;
-- Should not allow to set it to true.
set default_with_oids to t;
ERROR: tables declared WITH OIDS are not supported
+--
+-- Test GUC categories and flags.
+--
+CREATE TABLE pg_settings_flags AS SELECT name, category,
+ 'NO_SHOW_ALL' =ANY(flags) AS no_show_all,
+ 'NO_RESET_ALL' =ANY(flags) AS no_reset_all,
+ 'NOT_IN_SAMPLE' =ANY(flags) AS not_in_sample,
+ 'EXPLAIN' =ANY(flags) AS guc_explain,
+ 'COMPUTED' =ANY(flags) AS guc_computed
+ FROM pg_show_all_settings() AS psas, pg_get_guc_flags(psas.name) AS flags;
+-- test that GUCS are in postgresql.conf
+SELECT lower(name) FROM pg_settings_flags WHERE NOT not_in_sample EXCEPT
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) (= .*|[^ ]*$)', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln ~ '^#?[[:alpha:]]'
+ORDER BY 1;
+ lower
+-----------------------------
+ config_file
+ plpgsql.check_asserts
+ plpgsql.extra_errors
+ plpgsql.extra_warnings
+ plpgsql.print_strict_params
+ plpgsql.variable_conflict
+(6 rows)
+
+-- test that lines in postgresql.conf that look like GUCs are GUCs
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) (= .*|[^ ]*$)', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln ~ '^#?[[:alpha:]]'
+EXCEPT SELECT lower(name) FROM pg_settings_flags WHERE NOT not_in_sample
+ORDER BY 1;
+ guc
+-------------------
+ include
+ include_dir
+ include_if_exists
+(3 rows)
+
+-- test that section:DEVELOPER GUCs are flagged GUC_NOT_IN_SAMPLE:
+SELECT * FROM pg_settings_flags
+WHERE category='Developer Options' AND NOT not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+-- Maybe the converse:
+SELECT * FROM pg_settings_flags
+WHERE category NOT IN ('Developer Options', 'Preset Options') AND not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------------------------+-------------------------------------------------+-------------+--------------+---------------+-------------+--------------
+ application_name | Reporting and Logging / What to Log | f | f | t | f | f
+ transaction_deferrable | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+ transaction_isolation | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+ transaction_read_only | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+(4 rows)
+
+-- Most Query Tuning GUCs are flagged as EXPLAIN:
+SELECT * FROM pg_settings_flags
+WHERE category ~ '^Query Tuning' AND NOT guc_explain
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+---------------------------+--------------------------------------+-------------+--------------+---------------+-------------+--------------
+ default_statistics_target | Query Tuning / Other Planner Options | f | f | f | f | f
+(1 row)
+
+-- Maybe the converse:
+SELECT * FROM pg_settings_flags
+WHERE guc_explain AND NOT category ~ '^Query Tuning|^Resource Usage'
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+---------------------+-------------------------------------------------+-------------+--------------+---------------+-------------+--------------
+ force_parallel_mode | Developer Options | f | f | t | t | f
+ search_path | Client Connection Defaults / Statement Behavior | f | f | f | t | f
+(2 rows)
+
+-- GUCs flagged RUNTIME are Preset
+SELECT * FROM pg_settings_flags
+WHERE guc_computed AND NOT category='Preset Options'
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+-- PRESET GUCs are flagged NOT_IN_SAMPLE
+SELECT * FROM pg_settings_flags
+WHERE category='Preset Options' AND NOT not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+-- NO_SHOW_ALL implies NO_RESET_ALL:
+SELECT * FROM pg_settings_flags
+WHERE no_show_all AND NOT no_reset_all
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+-- Usually the converse:
+SELECT * FROM pg_settings_flags
+WHERE NOT no_show_all AND no_reset_all
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------------------------+-------------------------------------------------+-------------+--------------+---------------+-------------+--------------
+ transaction_deferrable | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+ transaction_isolation | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+ transaction_read_only | Client Connection Defaults / Statement Behavior | f | t | t | f | f
+(3 rows)
+
+-- NO_SHOW_ALL implies NOT_IN_SAMPLE:
+SELECT * FROM pg_settings_flags
+WHERE no_show_all AND NOT not_in_sample
+ORDER BY 1;
+ name | category | no_show_all | no_reset_all | not_in_sample | guc_explain | guc_computed
+------+----------+-------------+--------------+---------------+-------------+--------------
+(0 rows)
+
+DROP TABLE pg_settings_flags;
diff --git a/src/test/regress/sql/guc.sql b/src/test/regress/sql/guc.sql
index c39c11388d5..47b6a233d5c 100644
--- a/src/test/regress/sql/guc.sql
+++ b/src/test/regress/sql/guc.sql
@@ -311,3 +311,75 @@ reset check_function_bodies;
set default_with_oids to f;
-- Should not allow to set it to true.
set default_with_oids to t;
+
+--
+-- Test GUC categories and flags.
+--
+CREATE TABLE pg_settings_flags AS SELECT name, category,
+ 'NO_SHOW_ALL' =ANY(flags) AS no_show_all,
+ 'NO_RESET_ALL' =ANY(flags) AS no_reset_all,
+ 'NOT_IN_SAMPLE' =ANY(flags) AS not_in_sample,
+ 'EXPLAIN' =ANY(flags) AS guc_explain,
+ 'COMPUTED' =ANY(flags) AS guc_computed
+ FROM pg_show_all_settings() AS psas, pg_get_guc_flags(psas.name) AS flags;
+
+-- test that GUCS are in postgresql.conf
+SELECT lower(name) FROM pg_settings_flags WHERE NOT not_in_sample EXCEPT
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) (= .*|[^ ]*$)', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln ~ '^#?[[:alpha:]]'
+ORDER BY 1;
+
+-- test that lines in postgresql.conf that look like GUCs are GUCs
+SELECT regexp_replace(ln, '^#?([_[:alpha:]]+) (= .*|[^ ]*$)', '\1') AS guc
+FROM (SELECT regexp_split_to_table(pg_read_file('postgresql.conf'), '\n') AS ln) conf
+WHERE ln ~ '^#?[[:alpha:]]'
+EXCEPT SELECT lower(name) FROM pg_settings_flags WHERE NOT not_in_sample
+ORDER BY 1;
+
+-- test that section:DEVELOPER GUCs are flagged GUC_NOT_IN_SAMPLE:
+SELECT * FROM pg_settings_flags
+WHERE category='Developer Options' AND NOT not_in_sample
+ORDER BY 1;
+
+-- Maybe the converse:
+SELECT * FROM pg_settings_flags
+WHERE category NOT IN ('Developer Options', 'Preset Options') AND not_in_sample
+ORDER BY 1;
+
+-- Most Query Tuning GUCs are flagged as EXPLAIN:
+SELECT * FROM pg_settings_flags
+WHERE category ~ '^Query Tuning' AND NOT guc_explain
+ORDER BY 1;
+
+-- Maybe the converse:
+SELECT * FROM pg_settings_flags
+WHERE guc_explain AND NOT category ~ '^Query Tuning|^Resource Usage'
+ORDER BY 1;
+
+-- GUCs flagged RUNTIME are Preset
+SELECT * FROM pg_settings_flags
+WHERE guc_computed AND NOT category='Preset Options'
+ORDER BY 1;
+
+-- PRESET GUCs are flagged NOT_IN_SAMPLE
+SELECT * FROM pg_settings_flags
+WHERE category='Preset Options' AND NOT not_in_sample
+ORDER BY 1;
+
+-- NO_SHOW_ALL implies NO_RESET_ALL:
+SELECT * FROM pg_settings_flags
+WHERE no_show_all AND NOT no_reset_all
+ORDER BY 1;
+
+-- Usually the converse:
+SELECT * FROM pg_settings_flags
+WHERE NOT no_show_all AND no_reset_all
+ORDER BY 1;
+
+-- NO_SHOW_ALL implies NOT_IN_SAMPLE:
+SELECT * FROM pg_settings_flags
+WHERE no_show_all AND NOT not_in_sample
+ORDER BY 1;
+
+DROP TABLE pg_settings_flags;
--
2.17.1
On Thu, Jan 27, 2022 at 10:36:21PM -0600, Justin Pryzby wrote:
Maybe you misunderstood - I'm not reading the file specified by
current_setting('config_file'). Rather, I'm reading
tmp_check/data/postgresql.conf, which is copied from the sample conf.
Do you see an issue with that ?
Yes, as of:
mv $PGDATA/postgresql.conf $PGDATA/popo.conf
pg_ctl start -D $PGDATA -o '-c config_file=popo.conf'
make installcheck
I have not checked, but I am pretty sure that a couple of
distributions out there would pass down a custom path for the
configuration file, while removing postgresql.conf from the data
directory to avoid any confusion because if one finds out that some
parameters are defined but not loaded. Your patch fails on that.
The regression tests are only intended run from a postgres source dir, and if
someone runs the from somewhere else, and they "fail", I think that's because
they violated their assumption, not because of a problem with the test.
The tests are able to work out on HEAD, I'd rather not break something
that has worked this way for years. Two other aspects that we may
want to worry about are include_dir and include if we were to add
tests for that, perhaps. This last part is not really a strong
requirement IMO, though.
I wondered if it should chomp off anything added by pg_regress --temp-regress.
However that's either going to be a valid guc (or else it would fail some other
test). Or an extention's guc (which this isn't testing), which has a dot, and
which this regex doesn't match, so doesn't cause false positives.
I am not sure about those parts, being reserved about the parts that
involve the format of postgresql.conf or any other configuration
parts, but we could tackle that after, if necessary.
For now, I have down a review of the patch, tweaking the docs, the
code and the test to take care of all the inconsistencies I could
find. This looks like a good first cut to be able to remove check_guc
(the attached removes it, but I think that we'd better treat that
independently of the actual feature proposed, for clarity).
--
Michael
Attachments:
v2-0001-Expose-GUC-flags-in-SQL-function-replacing-check_.patchtext/x-diff; charset=us-asciiDownload
From a87afcfb5757367aeef795505e9361d5a03facc7 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Sat, 29 Jan 2022 15:32:10 +0900
Subject: [PATCH v2] Expose GUC flags in SQL function, replacing check_guc
Note-to-self: CATVERSION bump!
---
src/include/catalog/pg_proc.dat | 6 +++
src/backend/utils/misc/check_guc | 78 -------------------------------
src/backend/utils/misc/guc.c | 39 ++++++++++++++++
src/test/regress/expected/guc.out | 71 ++++++++++++++++++++++++++++
src/test/regress/sql/guc.sql | 41 ++++++++++++++++
doc/src/sgml/func.sgml | 39 ++++++++++++++++
6 files changed, 196 insertions(+), 78 deletions(-)
delete mode 100755 src/backend/utils/misc/check_guc
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 0859dc81ca..3f927acb01 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6096,6 +6096,12 @@
proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
proargnames => '{name,setting,unit,category,short_desc,extra_desc,context,vartype,source,min_val,max_val,enumvals,boot_val,reset_val,sourcefile,sourceline,pending_restart}',
prosrc => 'show_all_settings' },
+
+{ oid => '8921', descr => 'return flags for specified GUC',
+ proname => 'pg_settings_get_flags', provolatile => 's',
+ prorettype => '_text', proargtypes => 'text',
+ prosrc => 'pg_settings_get_flags' },
+
{ oid => '3329', descr => 'show config file settings',
proname => 'pg_show_all_file_settings', prorows => '1000', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/backend/utils/misc/check_guc b/src/backend/utils/misc/check_guc
deleted file mode 100755
index b171ef0e4f..0000000000
--- a/src/backend/utils/misc/check_guc
+++ /dev/null
@@ -1,78 +0,0 @@
-#!/bin/sh
-
-## currently, this script makes a lot of assumptions:
-## in postgresql.conf.sample:
-## 1) the valid config settings may be preceded by a '#', but NOT '# '
-## (we use this to skip comments)
-## 2) the valid config settings will be followed immediately by ' ='
-## (at least one space preceding the '=')
-## in guc.c:
-## 3) the options have PGC_ on the same line as the option
-## 4) the options have '{' on the same line as the option
-
-## Problems
-## 1) Don't know what to do with TRANSACTION ISOLATION LEVEL
-
-## if an option is valid but shows up in only one file (guc.c but not
-## postgresql.conf.sample), it should be listed here so that it
-## can be ignored
-INTENTIONALLY_NOT_INCLUDED="debug_deadlocks in_hot_standby \
-is_superuser lc_collate lc_ctype lc_messages lc_monetary lc_numeric lc_time \
-pre_auth_delay role seed server_encoding server_version server_version_num \
-session_authorization trace_lock_oidmin trace_lock_table trace_locks trace_lwlocks \
-trace_notify trace_userlocks transaction_isolation transaction_read_only \
-zero_damaged_pages"
-
-### What options are listed in postgresql.conf.sample, but don't appear
-### in guc.c?
-
-# grab everything that looks like a setting and convert it to lower case
-SETTINGS=`grep ' =' postgresql.conf.sample |
-grep -v '^# ' | # strip comments
-sed -e 's/^#//' |
-awk '{print $1}'`
-
-SETTINGS=`echo "$SETTINGS" | tr 'A-Z' 'a-z'`
-
-for i in $SETTINGS ; do
- hidden=0
- ## it sure would be nice to replace this with an sql "not in" statement
- ## it doesn't seem to make sense to have things in .sample and not in guc.c
-# for hidethis in $INTENTIONALLY_NOT_INCLUDED ; do
-# if [ "$hidethis" = "$i" ] ; then
-# hidden=1
-# fi
-# done
- if [ "$hidden" -eq 0 ] ; then
- grep -i '"'$i'"' guc.c > /dev/null
- if [ $? -ne 0 ] ; then
- echo "$i seems to be missing from guc.c";
- fi;
- fi
-done
-
-### What options are listed in guc.c, but don't appear
-### in postgresql.conf.sample?
-
-# grab everything that looks like a setting and convert it to lower case
-
-SETTINGS=`grep '{.* PGC_' guc.c | awk '{print $1}' | \
- sed -e 's/{//g' -e 's/"//g' -e 's/,//'`
-
-SETTINGS=`echo "$SETTINGS" | tr 'A-Z' 'a-z'`
-
-for i in $SETTINGS ; do
- hidden=0
- ## it sure would be nice to replace this with an sql "not in" statement
- for hidethis in $INTENTIONALLY_NOT_INCLUDED ; do
- if [ "$hidethis" = "$i" ] ; then
- hidden=1
- fi
- done
- if [ "$hidden" -eq 0 ] ; then
- grep -i '#'$i' ' postgresql.conf.sample > /dev/null
- if [ $? -ne 0 ] ; then
- echo "$i seems to be missing from postgresql.conf.sample";
- fi
- fi
-done
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 4c94f09c64..b3fd42e0f1 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9634,6 +9634,45 @@ GetConfigOptionByName(const char *name, const char **varname, bool missing_ok)
return _ShowOption(record, true);
}
+/*
+ * Return some of the flags associated to the specified GUC in the shape of
+ * a text array, and NULL if it does not exist. An empty array is returned
+ * if the GUC exists without any meaningful flags to show.
+ */
+Datum
+pg_settings_get_flags(PG_FUNCTION_ARGS)
+{
+#define MAX_GUC_FLAGS 5
+ char *varname = TextDatumGetCString(PG_GETARG_DATUM(0));
+ struct config_generic *record;
+ int cnt = 0;
+ Datum flags[MAX_GUC_FLAGS];
+ ArrayType *a;
+
+ record = find_option(varname, false, true, ERROR);
+
+ /* return NULL if no such variable */
+ if (record == NULL)
+ PG_RETURN_NULL();
+
+ if (record->flags & GUC_EXPLAIN)
+ flags[cnt++] = CStringGetTextDatum("EXPLAIN");
+ if (record->flags & GUC_NO_RESET_ALL)
+ flags[cnt++] = CStringGetTextDatum("NO_RESET_ALL");
+ if (record->flags & GUC_NO_SHOW_ALL)
+ flags[cnt++] = CStringGetTextDatum("NO_SHOW_ALL");
+ if (record->flags & GUC_NOT_IN_SAMPLE)
+ flags[cnt++] = CStringGetTextDatum("NOT_IN_SAMPLE");
+ if (record->flags & GUC_RUNTIME_COMPUTED)
+ flags[cnt++] = CStringGetTextDatum("RUNTIME_COMPUTED");
+
+ Assert(cnt <= MAX_GUC_FLAGS);
+
+ /* Returns the record as Datum */
+ a = construct_array(flags, cnt, TEXTOID, -1, false, TYPALIGN_INT);
+ PG_RETURN_ARRAYTYPE_P(a);
+}
+
/*
* Return GUC variable value by variable number; optionally return canonical
* form of name. Return value is palloc'd.
diff --git a/src/test/regress/expected/guc.out b/src/test/regress/expected/guc.out
index 59da91ff04..5fe53c82e3 100644
--- a/src/test/regress/expected/guc.out
+++ b/src/test/regress/expected/guc.out
@@ -813,3 +813,74 @@ set default_with_oids to f;
-- Should not allow to set it to true.
set default_with_oids to t;
ERROR: tables declared WITH OIDS are not supported
+-- Test GUC categories and flag patterns
+CREATE TABLE pg_settings_flags AS SELECT name, category,
+ 'EXPLAIN' = ANY(flags) AS explain,
+ 'NO_RESET_ALL' = ANY(flags) AS no_reset_all,
+ 'NO_SHOW_ALL' = ANY(flags) AS no_show_all,
+ 'NOT_IN_SAMPLE' = ANY(flags) AS not_in_sample,
+ 'RUNTIME_COMPUTED' = ANY(flags) AS runtime_computed
+ FROM pg_show_all_settings() AS psas,
+ pg_settings_get_flags(psas.name) AS flags;
+-- Developer GUCs should be flagged with GUC_NOT_IN_SAMPLE:
+SELECT name FROM pg_settings_flags
+ WHERE category = 'Developer Options' AND NOT not_in_sample
+ ORDER BY 1;
+ name
+------
+(0 rows)
+
+-- Most query-tuning GUCs are flagged as valid for EXPLAIN.
+-- default_statistics_target is an exception.
+SELECT name FROM pg_settings_flags
+ WHERE category ~ '^Query Tuning' AND NOT explain
+ ORDER BY 1;
+ name
+---------------------------
+ default_statistics_target
+(1 row)
+
+-- Runtime-computed GUCs should be part of the preset category.
+SELECT name FROM pg_settings_flags
+ WHERE NOT category = 'Preset Options' AND runtime_computed
+ ORDER BY 1;
+ name
+------
+(0 rows)
+
+-- Preset GUCs are flagged as NOT_IN_SAMPLE.
+SELECT name FROM pg_settings_flags
+ WHERE category = 'Preset Options' AND NOT not_in_sample
+ ORDER BY 1;
+ name
+------
+(0 rows)
+
+-- NO_SHOW_ALL implies NO_RESET_ALL, and vice-versa.
+SELECT name FROM pg_settings_flags
+ WHERE no_show_all AND NOT no_reset_all
+ ORDER BY 1;
+ name
+------
+(0 rows)
+
+-- Three exceptions as of transaction_*
+SELECT name FROM pg_settings_flags
+ WHERE NOT no_show_all AND no_reset_all
+ ORDER BY 1;
+ name
+------------------------
+ transaction_deferrable
+ transaction_isolation
+ transaction_read_only
+(3 rows)
+
+-- NO_SHOW_ALL implies NOT_IN_SAMPLE:
+SELECT name FROM pg_settings_flags
+ WHERE no_show_all AND NOT not_in_sample
+ ORDER BY 1;
+ name
+------
+(0 rows)
+
+DROP TABLE pg_settings_flags;
diff --git a/src/test/regress/sql/guc.sql b/src/test/regress/sql/guc.sql
index c39c11388d..7a78b8f3be 100644
--- a/src/test/regress/sql/guc.sql
+++ b/src/test/regress/sql/guc.sql
@@ -311,3 +311,44 @@ reset check_function_bodies;
set default_with_oids to f;
-- Should not allow to set it to true.
set default_with_oids to t;
+
+-- Test GUC categories and flag patterns
+CREATE TABLE pg_settings_flags AS SELECT name, category,
+ 'EXPLAIN' = ANY(flags) AS explain,
+ 'NO_RESET_ALL' = ANY(flags) AS no_reset_all,
+ 'NO_SHOW_ALL' = ANY(flags) AS no_show_all,
+ 'NOT_IN_SAMPLE' = ANY(flags) AS not_in_sample,
+ 'RUNTIME_COMPUTED' = ANY(flags) AS runtime_computed
+ FROM pg_show_all_settings() AS psas,
+ pg_settings_get_flags(psas.name) AS flags;
+
+-- Developer GUCs should be flagged with GUC_NOT_IN_SAMPLE:
+SELECT name FROM pg_settings_flags
+ WHERE category = 'Developer Options' AND NOT not_in_sample
+ ORDER BY 1;
+-- Most query-tuning GUCs are flagged as valid for EXPLAIN.
+-- default_statistics_target is an exception.
+SELECT name FROM pg_settings_flags
+ WHERE category ~ '^Query Tuning' AND NOT explain
+ ORDER BY 1;
+-- Runtime-computed GUCs should be part of the preset category.
+SELECT name FROM pg_settings_flags
+ WHERE NOT category = 'Preset Options' AND runtime_computed
+ ORDER BY 1;
+-- Preset GUCs are flagged as NOT_IN_SAMPLE.
+SELECT name FROM pg_settings_flags
+ WHERE category = 'Preset Options' AND NOT not_in_sample
+ ORDER BY 1;
+-- NO_SHOW_ALL implies NO_RESET_ALL, and vice-versa.
+SELECT name FROM pg_settings_flags
+ WHERE no_show_all AND NOT no_reset_all
+ ORDER BY 1;
+-- Three exceptions as of transaction_*
+SELECT name FROM pg_settings_flags
+ WHERE NOT no_show_all AND no_reset_all
+ ORDER BY 1;
+-- NO_SHOW_ALL implies NOT_IN_SAMPLE:
+SELECT name FROM pg_settings_flags
+ WHERE no_show_all AND NOT not_in_sample
+ ORDER BY 1;
+DROP TABLE pg_settings_flags;
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 0ee6974f1c..0cf1ffbcb0 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -23596,6 +23596,45 @@ SELECT pg_type_is_visible('myschema.widget'::regtype);
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_get_get_flags</primary>
+ </indexterm>
+ <function>pg_get_get_flags</function> ( <parameter>guc</parameter> <type>text</type> )
+ <returnvalue>text[]</returnvalue>
+ </para>
+ <para>
+ Returns an array of the flags associated with the given GUC, or
+ <literal>NULL</literal> if the does not exist. The result is
+ an empty array if the GUC exists but there are no flags to show,
+ as supported by the list below.
+ The following flags are exposed (the most meaningful ones are
+ included):
+ <simplelist>
+ <member>
+ <literal>EXPLAIN</literal>, parameters included in
+ <command>EXPLAIN</command> commands.
+ </member>
+ <member>
+ <literal>NO_SHOW_ALL</literal>, parameters excluded from
+ <command>SHOW ALL</command> commands.
+ </member>
+ <member>
+ <literal>NO_RESET_ALL</literal>, parameters excluded from
+ <command>RESET ALL</command> commands.
+ </member>
+ <member>
+ <literal>NOT_IN_SAMPLE</literal>, parameters not included in
+ <filename>postgresql.conf</filename> by default.
+ </member>
+ <member>
+ <literal>RUNTIME_COMPUTED</literal>, runtime-computed parameters.
+ </member>
+ </simplelist>
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
--
2.34.1
On Sat, Jan 29, 2022 at 03:38:53PM +0900, Michael Paquier wrote:
+-- Three exceptions as of transaction_* +SELECT name FROM pg_settings_flags + WHERE NOT no_show_all AND no_reset_all + ORDER BY 1; + name +------------------------ + transaction_deferrable + transaction_isolation + transaction_read_only +(3 rows)
I think "as of" is not the right phrase here.
Maybe say: Exceptions are transaction_*
--- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -23596,6 +23596,45 @@ SELECT pg_type_is_visible('myschema.widget'::regtype); + <para> + Returns an array of the flags associated with the given GUC, or + <literal>NULL</literal> if the does not exist. The result is
I guess it should say "if the GUC does not exist".
+ an empty array if the GUC exists but there are no flags to show, + as supported by the list below.
I'd say "...but none of the GUC's flags are exposed by this function."
+ The following flags are exposed (the most meaningful ones are + included):
"The most meaningful ones are included" doesn't seem to add anything.
Maybe it'd be useful to say "(Only the most useful flags are exposed)"
+ <literal>EXPLAIN</literal>, parameters included in + <command>EXPLAIN</command> commands. + </member> + <member>
I think the description is wrong, or just copied from the others.
EXPLAIN is for GUCs which are shown in EXPLAIN(SETTINGS).
|EXPLAIN, parameters included in EXPLAIN commands.
|NO_SHOW_ALL, parameters excluded from SHOW ALL commands.
|NO_RESET_ALL, parameters excluded from RESET ALL commands.
|NOT_IN_SAMPLE, parameters not included in postgresql.conf by default.
|RUNTIME_COMPUTED, runtime-computed parameters.
Instead of a comma, these should use a colon, or something else?
--
Justin
On Sat, Jan 29, 2022 at 06:18:50PM -0600, Justin Pryzby wrote:
"The most meaningful ones are included" doesn't seem to add anything.
Maybe it'd be useful to say "(Only the most useful flags are exposed)"
Yes, I have used something like that.
I think the description is wrong, or just copied from the others.
EXPLAIN is for GUCs which are shown in EXPLAIN(SETTINGS).
Added some details here.
|EXPLAIN, parameters included in EXPLAIN commands.
|NO_SHOW_ALL, parameters excluded from SHOW ALL commands.
|NO_RESET_ALL, parameters excluded from RESET ALL commands.
|NOT_IN_SAMPLE, parameters not included in postgresql.conf by default.
|RUNTIME_COMPUTED, runtime-computed parameters.Instead of a comma, these should use a colon, or something else?
And switched to a colon here.
With all those doc fixes, applied after an extra round of review. So
this makes us rather covered with the checks on the flags.
Now, what do we do with the rest of check_guc that involve a direct
lookup at what's on disk. We have the following:
1) Check the format of the option lists in guc.c.
2) Check the format of postgresql.conf.sample:
-- Valid options preceded by a '#' character.
-- Valid options followed by ' =', with at least one space before the
equal sign.
3) Check that options not marked as NOT_IN_SAMPLE are in the sample
file.
I have never seen 1) as a problem, and pgindent takes care of that at
some degree. 2) is also mostly cosmetic, and committers are usually
careful when adding a new GUC. 3) would be the most interesting
piece, and would cover most cases if we consider that a default
installation just copies postgresql.conf.sample over, as proposed
upthread in 0002.
Now, 3) has also the problem that it would fail installcheck as one
can freely add a developer option in the configuration. We could
solve that by adding a check in a TAP test, by using pg_config
--sharedir to find where the sample file is located. I wonder if this
would be a problem for some distributions, though, so adding such a
dependency feels a bit scary even if it would mean that initdb is
patched.
As a whole, I'd like to think that we would not lose much if check_guc
is removed.
--
Michael
On Mon, Jan 31, 2022 at 02:17:41PM +0900, Michael Paquier wrote:
With all those doc fixes, applied after an extra round of review. So
this makes us rather covered with the checks on the flags.
Thanks
Now, what do we do with the rest of check_guc that involve a direct
lookup at what's on disk. We have the following:
1) Check the format of the option lists in guc.c.
2) Check the format of postgresql.conf.sample:
-- Valid options preceded by a '#' character.
-- Valid options followed by ' =', with at least one space before the
equal sign.
3) Check that options not marked as NOT_IN_SAMPLE are in the sample
file.I have never seen 1) as a problem, and pgindent takes care of that at
some degree. 2) is also mostly cosmetic, and committers are usually
careful when adding a new GUC. 3) would be the most interesting
piece, and would cover most cases if we consider that a default
installation just copies postgresql.conf.sample over, as proposed
upthread in 0002.Now, 3) has also the problem that it would fail installcheck as one
can freely add a developer option in the configuration. We could
I'm not clear on what things are required/prohibited to allow/expect
"installcheck" to pass. It's possible that postgresql.conf doesn't even exist
in the data dir, right ?
It's okay with me if the config_file-reading stuff isn't re-implemented.
--
Justin
On Mon, Jan 31, 2022 at 04:56:45PM -0600, Justin Pryzby wrote:
I'm not clear on what things are required/prohibited to allow/expect
"installcheck" to pass. It's possible that postgresql.conf doesn't even exist
in the data dir, right ?
There are no written instructions AFAIK, but I have as personal rule
to not break the tests in configurations where they worked
previously.
It's okay with me if the config_file-reading stuff isn't re-implemented.
Actually, I am thinking that we should implement it before retiring
completely check_guc, but not in the fashion you are suggesting. I
would be tempted to add something in the TAP tests as of
src/test/misc/, where we initialize an instance to get the information
about all the GUCs from SQL, and map that to the sample file located
at pg_config --sharedir. I actually have in my patch set for
pg_upgrade's TAP a perl routine that could be used for this purpose,
as of the following in Cluster.pm:
+=item $node->config_data($option)
+
+Grab some data from pg_config, with $option being the command switch
+used.
+
+=cut
+
+sub config_data
+{
+ my ($self, $option) = @_;
+ local %ENV = $self->_get_env();
+
+ my ($stdout, $stderr);
+ my $result =
+ IPC::Run::run [ $self->installed_command('pg_config'), $option
],
+ '>', \$stdout, '2>', \$stderr
+ or die "could not execute pg_config";
+ chomp($stdout);
+ $stdout =~ s/\r$//;
+
+ return $stdout;
+}
What do you think? (I was thinking about applying that separately
anyway, to lower the load of the pg_upgrade patch a bit.)
--
Michael
On Sun, Feb 06, 2022 at 02:09:45PM +0900, Michael Paquier wrote:
Actually, I am thinking that we should implement it before retiring
completely check_guc, but not in the fashion you are suggesting. I
would be tempted to add something in the TAP tests as of
src/test/misc/, where we initialize an instance to get the information
about all the GUCs from SQL, and map that to the sample file located
at pg_config --sharedir. I actually have in my patch set for
pg_upgrade's TAP a perl routine that could be used for this purpose,
as of the following in Cluster.pm:
I have been poking at that, and this is finishing to be pretty
elegant as of the attached. With this in place, we are able to
cross-check GUCs marked as NOT_IN_SAMPLE (or not) with the contents of
postgresql.conf.sample, so as check_guc could get retired without us
losing much.
I am planning to apply the Cluster.pm part of the patch separately,
for clarity, as I want this routine in place for some other patch.
--
Michael
Attachments:
0001-Add-TAP-test-to-automate-check_guc.patchtext/x-diff; charset=us-asciiDownload
From 5c3fbf6af5111927aa7a61025abfce87a9bd230e Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Mon, 7 Feb 2022 11:32:17 +0900
Subject: [PATCH] Add TAP test to automate check_guc
---
src/test/modules/test_misc/t/003_check_guc.pl | 77 +++++++++++++++++++
src/test/perl/PostgreSQL/Test/Cluster.pm | 25 ++++++
2 files changed, 102 insertions(+)
create mode 100644 src/test/modules/test_misc/t/003_check_guc.pl
diff --git a/src/test/modules/test_misc/t/003_check_guc.pl b/src/test/modules/test_misc/t/003_check_guc.pl
new file mode 100644
index 0000000000..cd64fb8b49
--- /dev/null
+++ b/src/test/modules/test_misc/t/003_check_guc.pl
@@ -0,0 +1,77 @@
+# Tests to cross-check the consistency of GUC parameters with
+# postgresql.conf.sample.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node = PostgreSQL::Test::Cluster->new('main');
+$node->init;
+$node->start;
+
+# Grab the names of all the parameters that can be listed in the
+# configuration sample file.
+my $all_params = $node->safe_psql(
+ 'postgres',
+ "SELECT name
+ FROM pg_settings
+ WHERE NOT 'NOT_IN_SAMPLE' = ANY (pg_settings_get_flags(name))
+ ORDER BY 1");
+# Note the lower-case conversion, for consistency.
+my @all_params_array = split("\n", lc($all_params));
+
+# Grab the names of all parameters marked as NOT_IN_SAMPLE.
+my $not_in_sample = $node->safe_psql(
+ 'postgres',
+ "SELECT name
+ FROM pg_settings
+ WHERE 'NOT_IN_SAMPLE' = ANY (pg_settings_get_flags(name))
+ ORDER BY 1");
+# Note the lower-case conversion, for consistency.
+my @not_in_sample_array = split("\n", lc($not_in_sample));
+
+# Find the location of postgresql.conf.sample, based on the information
+# provided by pg_config.
+my $sample_file =
+ $node->config_data('--sharedir') . '/postgresql.conf.sample';
+
+# Read the sample file line-by-line, checking its contents.
+my $num_tests = 0;
+open(my $contents, '<', $sample_file)
+ || die "Could not open $sample_file: $!";
+while (my $line = <$contents>)
+{
+ # Check if this line matches a GUC parameter.
+ if ($line =~ m/^#?([_[:alpha:]]+) (= .*|[^ ]*$)/)
+ {
+ # Lower-case conversion matters for some of the GUCs.
+ my $param_name = lc($1);
+
+ # Ignore some exceptions.
+ next if $param_name eq "include";
+ next if $param_name eq "include_dir";
+ next if $param_name eq "include_if_exists";
+
+ my $marked_all_params = grep(/^$param_name$/, @all_params_array);
+ my $marked_not_in_sample =
+ grep(/^$param_name$/, @not_in_sample_array);
+
+ # All parameters marked as NOT_IN_SAMPLE cannot be in
+ # the sample file.
+ is($marked_not_in_sample, 0,
+ "checked $param_name not marked with NOT_IN_SAMPLE");
+ # All parameters *not* marked as NOT_IN_SAMPLE can be
+ # in the sample file.
+ is($marked_all_params, 1,
+ "checked $param_name in postgresql.conf.sample");
+
+ # Update test count.
+ $num_tests += 2;
+ }
+}
+
+close $contents;
+
+plan tests => $num_tests;
diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm
index 265f3ae657..832d119f2e 100644
--- a/src/test/perl/PostgreSQL/Test/Cluster.pm
+++ b/src/test/perl/PostgreSQL/Test/Cluster.pm
@@ -327,6 +327,31 @@ sub install_path
=pod
+=item $node->config_data($option)
+
+Grab some data from pg_config, with $option being the option switch
+used to grab the wanted data.
+
+=cut
+
+sub config_data
+{
+ my ($self, $option) = @_;
+ local %ENV = $self->_get_env();
+
+ my ($stdout, $stderr);
+ my $result =
+ IPC::Run::run [ $self->installed_command('pg_config'), $option ],
+ '>', \$stdout, '2>', \$stderr
+ or die "could not execute pg_config";
+ chomp($stdout);
+ $stdout =~ s/\r$//;
+
+ return $stdout;
+}
+
+=pod
+
=item $node->info()
Return a string containing human-readable diagnostic information (paths, etc)
--
2.34.1
Thanks for working on it.
Your test is checking that stuff in sample.conf is actually a GUC and not
marked NOT_IN_SAMPLE. But those are both unlikely mistakes to make.
The important/interesting test is the opposite: that all GUCs are present in
the sample file. It's a lot easier for someone to forget to add a GUC to
sample.conf than it is for someone to accidentally add something that isn't a
GUC.
I'd first parse the GUC-like lines in the file, making a list of gucs_in_file
and then compare the two lists.
--
Justin
On Sun, Feb 06, 2022 at 09:04:14PM -0600, Justin Pryzby wrote:
Your test is checking that stuff in sample.conf is actually a GUC and not
marked NOT_IN_SAMPLE. But those are both unlikely mistakes to make.
Yeah, you are right. Still, I don't see any reason to not include both.
I'd first parse the GUC-like lines in the file, making a list of gucs_in_file
and then compare the two lists.
This is a good idea, and makes the tests faster because there is no
need to test each GUC separately. While testing a bit more, I got
recalled by the fact that config_file is not marked as NOT_IN_SAMPLE
and not in postgresql.conf.sample, so the new case you suggested was
failing.
What do you think about the updated version attached? I have applied
the addition of config_data() separately.
--
Michael
Attachments:
v2-0001-Add-TAP-test-to-automate-check_guc.patchtext/x-diff; charset=us-asciiDownload
From dd9f0cb34960e89dd1ea5c12ab03ee1bc6884e1f Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Tue, 8 Feb 2022 10:10:42 +0900
Subject: [PATCH v2] Add TAP test to automate check_guc
---
src/test/modules/test_misc/t/003_check_guc.pl | 105 ++++++++++++++++++
1 file changed, 105 insertions(+)
create mode 100644 src/test/modules/test_misc/t/003_check_guc.pl
diff --git a/src/test/modules/test_misc/t/003_check_guc.pl b/src/test/modules/test_misc/t/003_check_guc.pl
new file mode 100644
index 0000000000..01f954e09c
--- /dev/null
+++ b/src/test/modules/test_misc/t/003_check_guc.pl
@@ -0,0 +1,105 @@
+# Tests to cross-check the consistency of GUC parameters with
+# postgresql.conf.sample.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 3;
+
+my $node = PostgreSQL::Test::Cluster->new('main');
+$node->init;
+$node->start;
+
+# Grab the names of all the parameters that can be listed in the
+# configuration sample file. config_file is an exception. It is not
+# in postgresql.conf.sample but is part of the lists from guc.c.
+my $all_params = $node->safe_psql(
+ 'postgres',
+ "SELECT name
+ FROM pg_settings
+ WHERE NOT 'NOT_IN_SAMPLE' = ANY (pg_settings_get_flags(name)) AND
+ name <> 'config_file'
+ ORDER BY 1");
+# Note the lower-case conversion, for consistency.
+my @all_params_array = split("\n", lc($all_params));
+
+# Grab the names of all parameters marked as NOT_IN_SAMPLE.
+my $not_in_sample = $node->safe_psql(
+ 'postgres',
+ "SELECT name
+ FROM pg_settings
+ WHERE 'NOT_IN_SAMPLE' = ANY (pg_settings_get_flags(name))
+ ORDER BY 1");
+# Note the lower-case conversion, for consistency.
+my @not_in_sample_array = split("\n", lc($not_in_sample));
+
+# Find the location of postgresql.conf.sample, based on the information
+# provided by pg_config.
+my $sample_file =
+ $node->config_data('--sharedir') . '/postgresql.conf.sample';
+
+# List of all the GUCs found in the sample file.
+my @gucs_in_file;
+
+# Read the sample file line-by-line, checking its contents to build a list
+# of everything known as a GUC.
+my $num_tests = 0;
+open(my $contents, '<', $sample_file)
+ || die "Could not open $sample_file: $!";
+while (my $line = <$contents>)
+{
+ # Check if this line matches a GUC parameter.
+ if ($line =~ m/^#?([_[:alpha:]]+) (= .*|[^ ]*$)/)
+ {
+ # Lower-case conversion matters for some of the GUCs.
+ my $param_name = lc($1);
+
+ # Ignore some exceptions.
+ next if $param_name eq "include";
+ next if $param_name eq "include_dir";
+ next if $param_name eq "include_if_exists";
+
+ # Update the list of GUCs found in the sample file, for the
+ # follow-up tests.
+ push @gucs_in_file, $param_name;
+ }
+}
+
+close $contents;
+
+# Cross-check that all the GUCs found in the sample file match the ones
+# fetched above. This maps the arrays to a hash, making the creation of
+# each exclude and intersection list easier.
+my %gucs_in_file_hash = map { $_ => 1 } @gucs_in_file;
+my %all_params_hash = map { $_ => 1 } @all_params_array;
+my %not_in_sample_hash = map { $_ => 1 } @not_in_sample_array;
+
+my @missing_from_file = grep(!$gucs_in_file_hash{$_}, @all_params_array);
+is(scalar(@missing_from_file),
+ 0, "no parameters missing from postgresql.conf.sample");
+
+my @missing_from_list = grep(!$all_params_hash{$_}, @gucs_in_file);
+is(scalar(@missing_from_list), 0, "no parameters missing from guc.c");
+
+my @sample_intersect = grep($not_in_sample_hash{$_}, @gucs_in_file);
+is(scalar(@sample_intersect),
+ 0, "no parameters marked as NOT_IN_SAMPLE in postgresql.conf.sample");
+
+# These would log some information only on errors.
+foreach my $param (@missing_from_file)
+{
+ print("found GUC $param in guc.c, missing from postgresql.conf.sample\n");
+}
+foreach my $param (@missing_from_list)
+{
+ print(
+ "found GUC $param in postgresql.conf.sample, with incorrect info in guc.c\n"
+ );
+}
+foreach my $param (@sample_intersect)
+{
+ print(
+ "found GUC $param in postgresql.conf.sample, marked as NOT_IN_SAMPLE\n"
+ );
+}
--
2.34.1
On Tue, Feb 08, 2022 at 10:44:07AM +0900, Michael Paquier wrote:
What do you think about the updated version attached? I have applied
the addition of config_data() separately.
Looks fine
+ # Check if this line matches a GUC parameter. + if ($line =~ m/^#?([_[:alpha:]]+) (= .*|[^ ]*$)/)
I think this is the regex I wrote to handle either "name = value" or "name
value", which was needed between f47ed79cc..4d7c3e344. See skip_equals.
It's fine the way it is, but could also remove the 2nd half of the alternation
(|), since GUCs shouldn't be added to sample.conf without '='.
--
Justin
On Mon, Feb 07, 2022 at 09:07:28PM -0600, Justin Pryzby wrote:
I think this is the regex I wrote to handle either "name = value" or "name
value", which was needed between f47ed79cc..4d7c3e344. See skip_equals.
Yes, I took it from there, noticing that it was working just fine for
this purpose.
It's fine the way it is, but could also remove the 2nd half of the alternation
(|), since GUCs shouldn't be added to sample.conf without '='.
Makes sense. check_guc also checks after this pattern.
--
Michael