pg_dump throwing segfault error during backup

Started by Cassandra Thriftover 1 year ago3 messagesgeneral
Jump to latest
#1Cassandra Thrift
cssndrthrift@gmail.com

Hello Team,

We are facing a segfault issue while initiating backup with pg_dump.
This is for Postgresql community version 12.5 and rhel 7.4 (PostgreSQL 12.5
on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat
4.8.5-39), 64-bit)

The below error is showing :

from /var/log/message

[root@USMS-PV-CRMDB2 log]# grep 'segfault' messages*
messages:Sep 24 00:15:56 USMS-PV-CRMDB2 kernel: postgres[1535]: segfault at
23c0000 ip 00007f272c91f61d sp 00007ffe22636718 error 6 in libc-2.17.so
[7f272c7d3000+1b8000]
messages:Sep 25 00:14:59 USMS-PV-CRMDB2 kernel: postgres[4245]: segfault at
23c0000 ip 00007f272c91f61d sp 00007ffe22636718 error 6 in libc-2.17.so
[7f272c7d3000+1b8000]
messages-20240923:Sep 21 00:07:13 USMS-PV-CRMDB2 kernel: postgres[14743]:
segfault at 23c0000 ip 00007f272c91f622 sp 00007ffe22636718 error 6 in
libc-2.17.so[7f272c7d3000+1b8000]
messages-20240923:Sep 22 00:06:53 USMS-PV-CRMDB2 kernel: postgres[392]:
segfault at 23c0000 ip 00007f272c91f61d sp 00007ffe22636718 error 6 in
libc-2.17.so[7f272c7d3000+1b8000]
messages-20240923:Sep 23 00:16:07 USMS-PV-CRMDB2 kernel: postgres[28843]:
segfault at 23c0000 ip 00007f272c91f622 sp 00007ffe22636718 error 6 in
libc-2.17.so[7f272c7d3000+1b8000]

From db log :

2024-09-26 00:47:05.312 +08 [9908] HINT: In a moment you should be able to
reconnect to the database and repeat your command.
2024-09-26 00:47:05.306 +08 [10870] WARNING: terminating connection
because of crash of another server process
2024-09-26 00:47:05.306 +08 [10870] DETAIL: The postmaster has commanded
this server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.

We fond the timing which shows database backup is getting invoked at the
time. So we had to stop backup (pg_backup)

we are manually taking table data currently.

Core dump also generated however RedHat refused to analyze this saying it
is third party product.

Kindly let us know if this is a bug for the version as it is too old and if
the core dump can be analyzed somehow as it is in binary.

Regards
Somraj Chakrabarty

#2Ron
ronljohnsonjr@gmail.com
In reply to: Cassandra Thrift (#1)
Re: pg_dump throwing segfault error during backup

On Wed, Oct 9, 2024 at 4:08 AM Cassandra Thrift <cssndrthrift@gmail.com>
wrote:

Hello Team,

We are facing a segfault issue while initiating backup with pg_dump.
This is for Postgresql community version 12.5 and rhel 7.4 (PostgreSQL
12.5 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat
4.8.5-39), 64-bit)

The below error is showing :

from /var/log/message

[root@USMS-PV-CRMDB2 log]# grep 'segfault' messages*
messages:Sep 24 00:15:56 USMS-PV-CRMDB2 kernel: postgres[1535]: segfault
at 23c0000 ip 00007f272c91f61d sp 00007ffe22636718 error 6 in libc-2.17.so
[7f272c7d3000+1b8000]
messages:Sep 25 00:14:59 USMS-PV-CRMDB2 kernel: postgres[4245]: segfault
at 23c0000 ip 00007f272c91f61d sp 00007ffe22636718 error 6 in libc-2.17.so
[7f272c7d3000+1b8000]
messages-20240923:Sep 21 00:07:13 USMS-PV-CRMDB2 kernel: postgres[14743]:
segfault at 23c0000 ip 00007f272c91f622 sp 00007ffe22636718 error 6 in
libc-2.17.so[7f272c7d3000+1b8000]
messages-20240923:Sep 22 00:06:53 USMS-PV-CRMDB2 kernel: postgres[392]:
segfault at 23c0000 ip 00007f272c91f61d sp 00007ffe22636718 error 6 in
libc-2.17.so[7f272c7d3000+1b8000]
messages-20240923:Sep 23 00:16:07 USMS-PV-CRMDB2 kernel: postgres[28843]:
segfault at 23c0000 ip 00007f272c91f622 sp 00007ffe22636718 error 6 in
libc-2.17.so[7f272c7d3000+1b8000]

From db log :

2024-09-26 00:47:05.312 +08 [9908] HINT: In a moment you should be able
to reconnect to the database and repeat your command.
2024-09-26 00:47:05.306 +08 [10870] WARNING: terminating connection
because of crash of another server process
2024-09-26 00:47:05.306 +08 [10870] DETAIL: The postmaster has commanded
this server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.

We fond the timing which shows database backup is getting invoked at the
time. So we had to stop backup (pg_backup)

we are manually taking table data currently.

Core dump also generated however RedHat refused to analyze this saying it
is third party product.

Kindly let us know if this is a bug for the version as it is too old and
if the core dump can be analyzed somehow as it is in binary.

PG 12.5 is missing *four years* of patches, and RHEL 7.4 is also far behind
the current 7.9.

https://www.postgresql.org/docs/release/ tells you what's been done to v12
in every patch release.

--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> crustacean!

#3Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Cassandra Thrift (#1)
Re: pg_dump throwing segfault error during backup

On 10/7/24 22:42, Cassandra Thrift wrote:

Hello Team,

We are facing a segfault issue while initiating backup with pg_dump.
This is for Postgresql community version 12.5 and rhel 7.4 (PostgreSQL
12.5 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red
Hat 4.8.5-39), 64-bit)

FYI, Postgres 12 goes EOL in one month.

What repo are you getting the Postgres package from, RH's or
community(PGDG).

If community see:

https://yum.postgresql.org/news/rhel7-end-of-life/

We fond the timing which shows database backup is getting invoked at the
time. So we had to stop backup (pg_backup)

I am not familiar with pg_backup, is that an actual program or our you
using it as alias for something else?

we are manually taking table data currently.

Show the exact command/process you are using to do above.

Core dump also generated however RedHat refused to analyze this saying
it is third party product.

Kindly let us know if this is a bug for the version as it is too old and
if the core dump can be analyzed somehow as it is in binary.

The most recent minor version of Postgres 12 is .20. It would not be a
bad idea to upgrade to that for immediate needs and then start testing
newer Postgres versions for a change in major version to get to a
version that will be community supported for some time.

Regards
Somraj Chakrabarty

--
Adrian Klaver
adrian.klaver@aklaver.com