pg_basebackup and pg_switch_wal()
Hi,
I read the doc concerning pg_basebackup
pg_basebackup cannot force the standby to switch to a new WAL file at the
end of backup.
When you are using -X none, if write activity on the primary is low,
pg_basebackup may need to wait a long time for the last WAL file required
for the backup to be switched and archived.
In this case, it may be useful to run pg_switch_wal on the primary in order
to trigger an immediate WAL file switch.
This is not the case also when the backup is made from the primary?
I didn't find any pg_switch_wal() call in the source code of the
pg_basebackup
~/GIT/postgres-18beta1/src/bin/pg_basebackup (master)$ grep
--include="*.c" "switch" *
astreamer_inject.c: switch (context)
pg_basebackup.c: switch (pg_check_dir(dirname))
pg_basebackup.c: switch (GetCopyDataByte(r, copybuf, &cursor))
pg_basebackup.c: switch (c)
pg_createsubscriber.c: pg_log_warning_hint("You can use
--enable-two-phase switch to enable two_phase.");
pg_createsubscriber.c: switch (c)
pg_createsubscriber.c: char *bad_switch = NULL;
pg_createsubscriber.c: bad_switch = "--database";
pg_createsubscriber.c: bad_switch = "--publication";
pg_createsubscriber.c: bad_switch = "--replication-slot";
pg_createsubscriber.c: bad_switch = "--subscription";
pg_createsubscriber.c: if (bad_switch)
pg_createsubscriber.c: pg_log_error("%s cannot be used
with -a/--all", bad_switch);
pg_receivewal.c: * timeline switch, xlogpos points to the beginning
of the segment because
pg_receivewal.c: pg_log_info("switched to timeline %u at
%X/%X",
pg_receivewal.c: switch (c)
pg_recvlogical.c: switch (c)
pg_recvlogical.c: switch (reason)
Thanks for your comment
Regards
Fabrice
Hello
This is not the case also when the backup is made from the primary?
On primary it is handled in do_pg_backup_stop function (called from perform_base_backup after basebackup was transferred), without explicitly called pg_switch_wal from pg_basebackup. pg_basebackup can change this behavior using options to the BASE_BACKUP replication command.
regards, Sergei
Thanks for answering Sergei.
ok in file postgres/src/backend/access/transam/xlog.c
function
void
do_pg_backup_stop(BackupState *state, bool waitforarchive)
there is
/*
* Force a switch to a new xlog segment file, so that the backup is
* valid as soon as archiver moves out the current segment file.
*/
RequestXLogSwitch(false);
But I do not understand this error with the barman wrapper:
Backup completed (start time: 2025-07-26 21:45:07.238223, elapsed time: 19
seconds)
Waiting for the WAL file 0000000300000B3F0000000C from server 'x_service'
(max: 600 seconds)
Processing xlog segments from streaming for x_service
0000000300000B3F0000000A
Processing xlog segments from streaming for x_service
0000000300000B3F0000000B
ERROR: The WAL file 0000000300000B3F0000000C has not been received in 600
seconds
Regards,
Fabrice
On Tue, Jul 29, 2025 at 12:15 PM Sergei Kornilov <sk@zsrv.org> wrote:
Show quoted text
Hello
This is not the case also when the backup is made from the primary?
On primary it is handled in do_pg_backup_stop function (called from
perform_base_backup after basebackup was transferred), without explicitly
called pg_switch_wal from pg_basebackup. pg_basebackup can change this
behavior using options to the BASE_BACKUP replication command.regards, Sergei
Hello Fabrice,
But I do not understand this error with the barman wrapper:
Backup completed (start time: 2025-07-26 21:45:07.238223, elapsed time:
19 seconds)
Waiting for the WAL file 0000000300000B3F0000000C from server 'x_service'
(max: 600 seconds)
Processing xlog segments from streaming for x_service
0000000300000B3F0000000A
Processing xlog segments from streaming for x_service
0000000300000B3F0000000B
ERROR: The WAL file 0000000300000B3F0000000C has not been received in 600
seconds
Are you by chance running PostgreSQL 17?
If you are using PostgreSQL 17, you are very likely encountering the issue
tracked here: https://github.com/EnterpriseDB/barman/issues/1041. In that
case,
a fix has already been merged and will be included in an upcoming release of
Barman.
As a quick note for the future, the Barman GitHub issues page is the
correct place
to report problems like this directly to the development team. In any case,
I understand
that you raised it here given the context of the original question about
pg_basebackup.
Hope that helps you.
Best regards,
Israel.
Thanks Israel for these informations. In fact, I thought at first of a
problem related to the version of pg_basebackup in version 17. I should
have posted first on the barman github.
Best regards,
Fabrice
On Wed, Jul 30, 2025 at 12:47 AM Israel Barth Rubio <barthisrael@gmail.com>
wrote:
Show quoted text
Hello Fabrice,
But I do not understand this error with the barman wrapper:
Backup completed (start time: 2025-07-26 21:45:07.238223, elapsed time:
19 seconds)
Waiting for the WAL file 0000000300000B3F0000000C from server
'x_service' (max: 600 seconds)
Processing xlog segments from streaming for x_service
0000000300000B3F0000000A
Processing xlog segments from streaming for x_service
0000000300000B3F0000000B
ERROR: The WAL file 0000000300000B3F0000000C has not been received in600 seconds
Are you by chance running PostgreSQL 17?
If you are using PostgreSQL 17, you are very likely encountering the issue
tracked here: https://github.com/EnterpriseDB/barman/issues/1041. In that
case,
a fix has already been merged and will be included in an upcoming release
of
Barman.As a quick note for the future, the Barman GitHub issues page is the
correct place
to report problems like this directly to the development team. In any
case, I understand
that you raised it here given the context of the original question about
pg_basebackup.Hope that helps you.
Best regards,
Israel.
To note effectively that the problem comes from a fix in version 17
PostgreSQL: Release Notes <https://www.postgresql.org/docs/release/17.0/>
Change file boundary handling of two WAL file name functions (Kyotaro
Horiguchi, Andres Freund, Bruce Momjian)
Regards,
Fabrice
On Wed, Jul 30, 2025 at 10:33 AM Fabrice Chapuis <fabrice636861@gmail.com>
wrote:
Show quoted text
Thanks Israel for these informations. In fact, I thought at first of a
problem related to the version of pg_basebackup in version 17. I should
have posted first on the barman github.Best regards,
Fabrice
On Wed, Jul 30, 2025 at 12:47 AM Israel Barth Rubio <barthisrael@gmail.com>
wrote:Hello Fabrice,
But I do not understand this error with the barman wrapper:
Backup completed (start time: 2025-07-26 21:45:07.238223, elapsed time:
19 seconds)
Waiting for the WAL file 0000000300000B3F0000000C from server
'x_service' (max: 600 seconds)
Processing xlog segments from streaming for x_service
0000000300000B3F0000000A
Processing xlog segments from streaming for x_service
0000000300000B3F0000000B
ERROR: The WAL file 0000000300000B3F0000000C has not been received in600 seconds
Are you by chance running PostgreSQL 17?
If you are using PostgreSQL 17, you are very likely encountering the issue
tracked here: https://github.com/EnterpriseDB/barman/issues/1041. In
that case,
a fix has already been merged and will be included in an upcoming release
of
Barman.As a quick note for the future, the Barman GitHub issues page is the
correct place
to report problems like this directly to the development team. In any
case, I understand
that you raised it here given the context of the original question about
pg_basebackup.Hope that helps you.
Best regards,
Israel.