PgBackRest Full backup and N/W reliability
List,
I am doing a full backup using PgBackRest from a production server to
Reposerver.
My connection is IPSec VPN over ILL ( 8 Mbps link) between the
Production DB Server and the remote RepoServer.
I understood the bottleneck of 8 Mbps between servers. (Server NICs 10Gbps
and switch)
Query : I have started the backup command and it is running (may go for
hours and days as link speed is minimal) .
If the link disconnected or Network error happens before completion of the
backup command Definitely the option is to reissue the backup command
again.
If so, does the backup process start again from scratch ? or it resumes
from where the backup process is stopped ?
If it starts from scratch I am afraid that I can''t complete the initial
full backup never :(
Or is there a work around if the network connectivity is lost in between
?
Any suggestions much appreciated
Thank you ,
Krishane
[root@dbtest pgbackrest]# sudo -u postgres pgbackrest --stanza=Repo
--log-level-console=info backup
2024-08-29 10:55:27.729 P00 INFO: backup command begin 2.52.1: --delta
--exec-id=523103-56943986 --log-level-console=info --log-level-file=debug
--pg1-host=10.15.0.202 --pg1-host-user=enterprisedb
--pg1-path=/data/edb/as16/data --pg-version-force=16 --process-max=5
--repo1-block --repo1-bundle --repo1-cipher-pass=<redacted>
--repo1-cipher-type=aes-256-cbc --repo1-path=/data/DB_BKUPS
--repo1-retention-diff=2 --repo1-retention-full=2 --stanza=Repo --start-fast
WARN: no prior backup exists, incr backup has been changed to full
2024-08-29 10:55:30.589 P00 INFO: execute non-exclusive backup start:
backup begins after the requested immediate checkpoint completes
2024-08-29 10:55:31.543 P00 INFO: backup start archive =
00000001000000850000004C, lsn = 85/4C0007F8
2024-08-29 10:55:31.543 P00 INFO: check archive for prior segment
00000001000000850000004B
ON Repo Server:
[root@dbtest backup]# date
Thursday 29 August 2024 10:58:08 AM IST
[root@dbtest backup]# du -h
165M ./Repo
165M
[root@dbtest backup]# date
Thursday 29 August 2024 11:37:03 AM IST
[root@dbtest backup]# du -h
1.9G ./Repo
1.9G
ON Production Server /data/edb/as16/data directory size is 500 GB
On Thu, Aug 29, 2024 at 2:21 AM KK CHN <kkchn.in@gmail.com> wrote:
I am doing a full backup using PgBackRest from a production server to
Reposerver.
...
If so, does the backup process start again from scratch ? or it
resumes from where the backup process is stopped ?
It resumes. You will see a message like this:
WARN: resumable backup 20240829-091727F of same type exists -- invalid
files will be removed then the backup will resume
Any suggestions much appreciated
Boost your process-max as high as you are able to speed up your backup time.
Cheers,
Greg
On Thu, Aug 29, 2024 at 6:54 PM Greg Sabino Mullane <htamfids@gmail.com>
wrote:
On Thu, Aug 29, 2024 at 2:21 AM KK CHN <kkchn.in@gmail.com> wrote:
I am doing a full backup using PgBackRest from a production server to
Reposerver....
If so, does the backup process start again from scratch ? or it
resumes from where the backup process is stopped ?It resumes. You will see a message like this:
WARN: resumable backup 20240829-091727F of same type exists -- invalid
files will be removed then the backup will resumeAny suggestions much appreciated
Boost your process-max as high as you are able to speed up your backup
time.
what will be the ideal process-max number to use ?
Once I update the process-max param in pgbackrest.conf , do I need to
stop and start the pgback rest ?
Or just editing the pgbackrest.conf on the fly will reflect the increased
process-max numbers advantage ?
Is there a limit for proces max. setting ?
Show quoted text
Cheers,
Greg
On Thu, Aug 29, 2024 at 12:08 PM KK CHN <kkchn.in@gmail.com> wrote:
On Thu, Aug 29, 2024 at 6:54 PM Greg Sabino Mullane <htamfids@gmail.com>
wrote:On Thu, Aug 29, 2024 at 2:21 AM KK CHN <kkchn.in@gmail.com> wrote:
I am doing a full backup using PgBackRest from a production server to
Reposerver....
If so, does the backup process start again from scratch ? or it
resumes from where the backup process is stopped ?It resumes. You will see a message like this:
WARN: resumable backup 20240829-091727F of same type exists -- invalid
files will be removed then the backup will resumeAny suggestions much appreciated
Boost your process-max as high as you are able to speed up your backup
time.what will be the ideal process-max number to use ?
We can't answer that. It depends on what else your PG and PgBackRest
servers are doing when "pgbackrest backup" runs.
Once I update the process-max param in pgbackrest.conf , do I need to
stop and start the pgback rest ?
Or just editing the pgbackrest.conf on the fly will reflect the increased
process-max numbers advantage ?
"pgbackrest backup" reads the config file when it starts.
Is there a limit for proces max. setting ?
I think you need managed DBA services.
--
Death to America, and butter sauce.
Iraq lobster!