Removing wal_keep_segments as default configuration in PostgresNode.pm
Hi all,
Right now, PostgresNode.pm uses this set of parameters when initializing a node:
print $conf "\n# Added by PostgresNode.pm\n";
print $conf "fsync = off\n";
print $conf "restart_after_crash = off\n";
print $conf "log_line_prefix = '%m [%p] %q%a '\n";
print $conf "log_statement = all\n";
print $conf "wal_retrieve_retry_interval = '500ms'\n";
print $conf "port = $port\n";
When streaming is enabled, the following set is used:
print $conf "wal_level = replica\n"; # replace by logical here
is you need ;)
print $conf "max_wal_senders = 5\n";
print $conf "max_replication_slots = 5\n";
print $conf "max_wal_size = 128MB\n";
print $conf "shared_buffers = 1MB\n";
print $conf "wal_log_hints = on\n";
print $conf "hot_standby = on\n";
print $conf "max_connections = 10\n";
While all those settings are good defaults in my opinion, I have been
trapped by wal_keep_segments being present when designing a test. The
TAP test in question here was forcing WAL segments to be recycled with
two checkpoints and some forced segment switches to make a
disconnected standby sync back to a primary using some archives, but
then I got surprised that max_wal_size was not respected. Until I
noticed that keep_wal_segments was in play.
I tend to think that while all the other parameters make sense to
deploy instances that need few resources, wal_keep_segments may cause
up to 350MB of WAL segments to be kept in each pg_wal's instance,
while max_wal_size is set at 128MB. The only test in the code tree in
need of wal_keep_segments is actually pg_rewind, which enforces
checkpoints after the rewind to update the source's control file.
So, thoughts about the attached that reworks this portion of PostgresNode.pm?
Thanks,
--
Michael
Attachments:
tap-minimize-default.patchapplication/octet-stream; name=tap-minimize-default.patchDownload
diff --git a/src/bin/pg_rewind/RewindTest.pm b/src/bin/pg_rewind/RewindTest.pm
index 76ce295cef..e6041f38a5 100644
--- a/src/bin/pg_rewind/RewindTest.pm
+++ b/src/bin/pg_rewind/RewindTest.pm
@@ -119,6 +119,11 @@ sub setup_cluster
# Initialize master, data checksums are mandatory
$node_master = get_new_node('master' . ($extra_name ? "_${extra_name}" : ''));
$node_master->init(allows_streaming => 1);
+ # Set wal_keep_segments to prevent WAL segment recycling after enforced
+ # checkpoints in the tests.
+ $node_master->append_conf('postgresql.conf', qq(
+wal_keep_segments = 20
+));
}
sub start_master
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index edcac6fb9f..988942f547 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -434,7 +434,6 @@ sub init
}
print $conf "max_wal_senders = 5\n";
print $conf "max_replication_slots = 5\n";
- print $conf "wal_keep_segments = 20\n";
print $conf "max_wal_size = 128MB\n";
print $conf "shared_buffers = 1MB\n";
print $conf "wal_log_hints = on\n";
On 9/11/17 21:55, Michael Paquier wrote:
I tend to think that while all the other parameters make sense to
deploy instances that need few resources, wal_keep_segments may cause
up to 350MB of WAL segments to be kept in each pg_wal's instance,
while max_wal_size is set at 128MB. The only test in the code tree in
need of wal_keep_segments is actually pg_rewind, which enforces
checkpoints after the rewind to update the source's control file.So, thoughts about the attached that reworks this portion of PostgresNode.pm?
Committed.
Besides the resource usage, it would probably be bad if a
wal_keep_segments setting papered over problems with replication slots
for example.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Nov 2, 2017 at 4:47 PM, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
On 9/11/17 21:55, Michael Paquier wrote:
I tend to think that while all the other parameters make sense to
deploy instances that need few resources, wal_keep_segments may cause
up to 350MB of WAL segments to be kept in each pg_wal's instance,
while max_wal_size is set at 128MB. The only test in the code tree in
need of wal_keep_segments is actually pg_rewind, which enforces
checkpoints after the rewind to update the source's control file.So, thoughts about the attached that reworks this portion of PostgresNode.pm?
Committed.
Besides the resource usage, it would probably be bad if a
wal_keep_segments setting papered over problems with replication slots
for example.
Thanks! I almost forgot this patch.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers