Postgresql Crasching

On 1/25/19 10:21 AM, Márcio Antônio Sepp wrote:

Hi all,

We are experiencing the follow error in last few weeks:

Jan 25 16:03:51 FreeBSD11 kernel: pid 78639 (postgres), uid 770: exited on
signal 11 (core dumped)
Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-1] 2019-01-25 16:03:51.135 -02
[73403] WARNING: terminating connection because of crash of another server
process
Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-2] 2019-01-25 16:03:51.135 -02
[73403] DETAIL: The postmaster has commanded this server process to roll
back the current transacti
on and exit, because another server process exited abnormally and possibly
corrupted shared memory.
Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-3] 2019-01-25 16:03:51.135 -02
[73403] HINT: In a moment you should be able to reconnect to the database
and repeat your command.

What is the logs just before the above?

This is our test machine and some days ago we try to increase memory and
vcpu, but the problem persist.

Some information:
root@FreeBSD11:~ # uname -a
FreeBSD FreeBSD11.2 11.2-RELEASE-p4 FreeBSD 11.2-RELEASE-p4 #0: Thu Sep 27
08:16:24 UTC 2018
root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64

root@FreeBSD11:~ # swapinfo -h
Device 1K-blocks Used Avail Capacity
/dev/da0s1b 4194268 0B 4.0G 0%

root@FreeBSD11:~ # df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/da0s1a 141G 13G 117G 10% /
devfs 1.0K 1.0K 0B 100% /dev

# select version();
PostgreSQL 11.0 on amd64-portbld-freebsd11.2, compiled by FreeBSD clang
version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0), 64-bit

Any suggestion?

Tks a lot

--
Att
Márcio A. Sepp

--
Adrian Klaver
adrian.klaver@aklaver.com

=?iso-8859-1?Q?M=E1rcio_Ant=F4nio_Sepp?= <marcio@zyontecnologia.com.br> writes:

We are experiencing the follow error in last few weeks:
Jan 25 16:03:51 FreeBSD11 kernel: pid 78639 (postgres), uid 770: exited on
signal 11 (core dumped)

# select version();
PostgreSQL 11.0 on amd64-portbld-freebsd11.2, compiled by FreeBSD clang
version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0), 64-bit

First thing to try is updating to 11.1, just to see if problem is already
fixed. If not, find out what query is crashing (see postmaster log),
and if you can get a stack trace out of one of the core files, that'd
be pretty helpful too.

https://wiki.postgresql.org/wiki/Generating_a_stack_trace_of_a_PostgreSQL_backend

We might need to ask for a self-contained test case, if the stack
trace doesn't make it clear what's failing.

regards, tom lane

On 1/25/19 10:21 AM, Márcio Antônio Sepp wrote:

Hi all,

We are experiencing the follow error in last few weeks:

Jan 25 16:03:51 FreeBSD11 kernel: pid 78639 (postgres), uid 770: exited

on

signal 11 (core dumped)
Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-1] 2019-01-25 16:03:51.135

-02

[73403] WARNING: terminating connection because of crash of another

server

process
Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-2] 2019-01-25 16:03:51.135

-02

[73403] DETAIL: The postmaster has commanded this server process to roll
back the current transacti
on and exit, because another server process exited abnormally and

possibly

corrupted shared memory.
Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-3] 2019-01-25 16:03:51.135

-02

[73403] HINT: In a moment you should be able to reconnect to the

database

and repeat your command.

What is the logs just before the above?

Jan 25 16:01:55 FreeBSD11 postgres[71605]: [6-2] 2019-01-25 16:01:55.733 -02 [71605] DETAIL: Key (tdoemp, codemp, codfil, origem, sercai, numdoc, tdopes, codpes, datpar)=(1, 59
84305, 1, 55, 1, 160, 1, 5984305000107, 201901) is not present in table "e026pag_p2019".
Jan 25 16:01:55 FreeBSD11 postgres[71605]: [6-3] 2019-01-25 16:01:55.733 -02 [71605] STATEMENT: COMMIT
Jan 25 16:02:35 FreeBSD11 postgres[73403]: [6-1] 2019-01-25 16:02:35.184 -02 [73403] ERROR: insert or update on table "e026par_p2019" violates foreign key constraint "e026par_p
2019_tdoemp_fkey"
Jan 25 16:02:35 FreeBSD11 postgres[73403]: [6-2] 2019-01-25 16:02:35.184 -02 [73403] DETAIL: Key (tdoemp, codemp, codfil, origem, sercai, numdoc, tdopes, codpes, datpar)=(1, 59
84305, 1, 55, 1, 160, 1, 5984305000107, 201901) is not present in table "e026pag_p2019".
Jan 25 16:02:35 FreeBSD11 postgres[73403]: [6-3] 2019-01-25 16:02:35.184 -02 [73403] STATEMENT: COMMIT
Jan 25 16:03:51 FreeBSD11 kernel: pid 78639 (postgres), uid 770: exited on signal 11 (core dumped)

On 1/25/19 10:50 AM, Márcio Antônio Sepp wrote:

On 1/25/19 10:21 AM, Márcio Antônio Sepp wrote:

Hi all,

We are experiencing the follow error in last few weeks:

Jan 25 16:03:51 FreeBSD11 kernel: pid 78639 (postgres), uid 770: exited

on

signal 11 (core dumped)
Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-1] 2019-01-25 16:03:51.135

-02

[73403] WARNING: terminating connection because of crash of another

server

process
Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-2] 2019-01-25 16:03:51.135

-02

[73403] DETAIL: The postmaster has commanded this server process to roll
back the current transacti
on and exit, because another server process exited abnormally and

possibly

corrupted shared memory.
Jan 25 16:03:51 FreeBSD11 postgres[73403]: [7-3] 2019-01-25 16:03:51.135

-02

[73403] HINT: In a moment you should be able to reconnect to the

database

and repeat your command.

What is the logs just before the above?

Jan 25 16:01:55 FreeBSD11 postgres[71605]: [6-2] 2019-01-25 16:01:55.733 -02 [71605] DETAIL: Key (tdoemp, codemp, codfil, origem, sercai, numdoc, tdopes, codpes, datpar)=(1, 59
84305, 1, 55, 1, 160, 1, 5984305000107, 201901) is not present in table "e026pag_p2019".
Jan 25 16:01:55 FreeBSD11 postgres[71605]: [6-3] 2019-01-25 16:01:55.733 -02 [71605] STATEMENT: COMMIT
Jan 25 16:02:35 FreeBSD11 postgres[73403]: [6-1] 2019-01-25 16:02:35.184 -02 [73403] ERROR: insert or update on table "e026par_p2019" violates foreign key constraint "e026par_p
2019_tdoemp_fkey"
Jan 25 16:02:35 FreeBSD11 postgres[73403]: [6-2] 2019-01-25 16:02:35.184 -02 [73403] DETAIL: Key (tdoemp, codemp, codfil, origem, sercai, numdoc, tdopes, codpes, datpar)=(1, 59
84305, 1, 55, 1, 160, 1, 5984305000107, 201901) is not present in table "e026pag_p2019".
Jan 25 16:02:35 FreeBSD11 postgres[73403]: [6-3] 2019-01-25 16:02:35.184 -02 [73403] STATEMENT: COMMIT
Jan 25 16:03:51 FreeBSD11 kernel: pid 78639 (postgres), uid 770: exited on signal 11 (core dumped)

Hmm, a 1:16 between the query failing and the crash. Hard to say if
there is a cause and effect. I would follow Tom Lane's advice and see if
you can get a stack trace.

--
Adrian Klaver
adrian.klaver@aklaver.com