PgBackRest client_loop: send disconnect: Connection reset

Started by KK CHNover 1 year ago3 messagesgeneral
Jump to latest
#1KK CHN
kkchn.in@gmail.com

List,

I am facing an Error : on a connection reset while PgBackRest is doing
backup process for DB server to Repo Server.

But the reissue of the pgbackrest backup command fails as follows...

"Unable to acquire lock on file '/tmp/pgbackrest/Repo-backup.lock':
Resource temporarily unavailable HINT: is another pgBackRest process
running?

After multiple attempts and during 8 to 10 minutes, the backup command
from RepoServer starts working again.. How to avoid /what is the exact
reason for "Resource temporarily unavailable"? HINT : Is another
PgBackRest process running ?

Please advise ..

On Connection reset then issuing backup command I get out puts as below.

######################################################################

[root@dbtest ~]# sudo -u postgres pgbackrest --stanza=Repo
--log-level-console=info backup
2024-08-29 16:38:13.922 P00 INFO: backup command begin 2.52.1: --delta
--exec-id=658924-527c843b --log-level-console=info --log-level-file=debug
--pg1-host=10.15.0.202 --pg1-host-user=enterprisedb
--pg1-path=/data/edb/as16/data --pg-version-force=16 --process-max=5
--repo1-block --repo1-bundle --repo1-cipher-pass=<redacted>
--repo1-cipher-type=aes-256-cbc --repo1-path=/data/DB_BKUPS
--repo1-retention-diff=2 --repo1-retention-full=2 --stanza=Repo --start-fast
WARN: no prior backup exists, incr backup has been changed to full
2024-08-29 16:38:16.745 P00 INFO: execute non-exclusive backup start:
backup begins after the requested immediate checkpoint completes
2024-08-29 16:38:18.003 P00 INFO: backup start archive =
00000001000000850000009A, lsn = 85/9A0000D8
2024-08-29 16:38:18.003 P00 INFO: check archive for prior segment
000000010000008500000099
WARN: resumable backup 20240829-105625F of same type exists -- invalid
files will be removed then the backup will resume
client_loop: send disconnect: Connection reset

[root@dbtest ~]# sudo -u postgres pgbackrest --stanza=Repo
--log-level-console=info backup
2024-08-29 18:45:06.630 P00 INFO: backup command begin 2.52.1: --delta
--exec-id=691718-6c37ff5c --log-level-console=info --log-level-file=debug
--pg1-host=10.15.0.202 --pg1-host-user=enterprisedb
--pg1-path=/data/edb/as16/data --pg-version-force=16 --process-max=5
--repo1-block --repo1-bundle --repo1-cipher-pass=<redacted>
--repo1-cipher-type=aes-256-cbc --repo1-path=/data/DB_BKUPS
--repo1-retention-diff=2 --repo1-retention-full=2 --stanza=Repo --start-fast
ERROR: [050]: unable to acquire lock on file
'/tmp/pgbackrest/Repo-backup.lock': Resource temporarily unavailable
HINT: is another pgBackRest process running?

2024-08-29 18:45:06.631 P00 INFO: backup command end: aborted with
exception [050]
[root@dbtest ~]# sudo -u postgres pgbackrest --stanza=Repo
--log-level-console=info backup
2024-08-29 18:45:14.531 P00 INFO: backup command begin 2.52.1: --delta
--exec-id=691730-06e11989 --log-level-console=info --log-level-file=debug
--pg1-host=10.15.0.202 --pg1-host-user=enterprisedb
--pg1-path=/data/edb/as16/data --pg-version-force=16 --process-max=5
--repo1-block --repo1-bundle --repo1-cipher-pass=<redacted>
--repo1-cipher-type=aes-256-cbc --repo1-path=/data/DB_BKUPS
--repo1-retention-diff=2 --repo1-retention-full=2 --stanza=Repo --start-fast
ERROR: [050]: unable to acquire lock on file
'/tmp/pgbackrest/Repo-backup.lock': Resource temporarily unavailable
HINT: is another pgBackRest process running?
2024-08-29 18:45:14.531 P00 INFO: backup command end: aborted with
exception [050]

###########################################################################
After 10 minutes or so I issued the command again as follows it starts
backing up as follows ?

2024-08-29 18:48:19.561 P00 INFO: backup command end: aborted with
exception [050]
[root@dbtest ~]# sudo -u postgres pgbackrest --stanza=Repo
--log-level-console=info backup
2024-08-29 18:52:33.834 P00 INFO: backup command begin 2.52.1: --delta
--exec-id=693517-4994fba1 --log-level-console=info --log-level-file=debug
--pg1-host=10.15.0.202 --pg1-host-user=enterprisedb
--pg1-path=/data/edb/as16/data --pg-version-force=16 --process-max=5
--repo1-block --repo1-bundle --repo1-cipher-pass=<redacted>
--repo1-cipher-type=aes-256-cbc --repo1-path=/data/DB_BKUPS_NGERSS
--repo1-retention-diff=2 --repo1-retention-full=2 --stanza=Repo --start-fast
WARN: no prior backup exists, incr backup has been changed to full
2024-08-29 18:52:36.813 P00 INFO: execute non-exclusive backup start:
backup begins after the requested immediate checkpoint completes
2024-08-29 18:52:37.788 P00 INFO: backup start archive =
000000010000008600000063, lsn = 86/63000028
2024-08-29 18:52:37.788 P00 INFO: check archive for prior segment
000000010000008600000062
WARN: resumable backup 20240829-105625F of same type exists -- invalid
files will be removed then the backup will resume

Thank you,
Krishane

#2Greg Sabino Mullane
greg@turnstep.com
In reply to: KK CHN (#1)
Re: PgBackRest client_loop: send disconnect: Connection reset

On Thu, Aug 29, 2024 at 9:31 AM KK CHN <kkchn.in@gmail.com> wrote:

"Unable to acquire lock on file '/tmp/pgbackrest/Repo-backup.lock'

When this happens, take a look inside this file. If there is another
pgbackrest process running, the pid will be inside that file. Kill that
process before trying to run another backup.

Cheers,
Greg

#3Ron
ronljohnsonjr@gmail.com
In reply to: Greg Sabino Mullane (#2)
Re: PgBackRest client_loop: send disconnect: Connection reset

On Thu, Aug 29, 2024 at 10:12 AM Greg Sabino Mullane <htamfids@gmail.com>
wrote:

On Thu, Aug 29, 2024 at 9:31 AM KK CHN <kkchn.in@gmail.com> wrote:

"Unable to acquire lock on file '/tmp/pgbackrest/Repo-backup.lock'

When this happens, take a look inside this file. If there is another
pgbackrest process running, the pid will be inside that file. Kill that
process before trying to run another backup.

Or don't run pgbackrest when another instance is running. This is what I
do:
pgrep -a pgbackrest | grep -q "pgbackrest backup" && { echo "another
pgbackrest back is running"; exit 1; }

(There's probably a better way to check that, but this works.)

--
Death to America, and butter sauce.
Iraq lobster!