BUG #14261: Hanged randomly in hash_seq_search

Started by Amaury Decremeover 9 years ago4 messagesbugs
Jump to latest
#1Amaury Decreme
amdecreme@airfrance.fr

The following bug has been logged on the website:

Bug reference: 14261
Logged by: Amaury Decreme
Email address: amdecreme@airfrance.fr
PostgreSQL version: 9.4.1
Operating system: Red Hat Enterprise Linux Server release 6.6
Description:

Hello,

When I execute the following request around 10-50 times in a row, postgres
hangs randomly on the connection. New connections are still working.

The CPU peaks at 100% and the process needs to be SIGKILLed. SIGTERM is not
treated.

The request is:
SELECT * FROM MEASURE_FLAT_H
INNER JOIN (
SELECT max(dt) as dtmax FROM MEASURE_FLAT_H WHERE (ASSETS_NAME = ?) AND
(METRIC_NAME = ?)
) d ON (D.DTMAX = DT)
WHERE (ASSETS_NAME = ?) AND (METRIC_NAME = ?)

At this step, no lock is found in pg_locks.

The process can be found stuck in the hash_seq_search().

Call stack:
#0 0x000000000075e919 in hash_seq_search ()
#1 0x00007fd0af74fb2b in pgqs_entry_dealloc ()
from /tech/postgres/9.4.1/server/lib/pg_qualstats.so
#2 0x00007fd0af750a15 in pgqs_ExecutorEnd ()
from /tech/postgres/9.4.1/server/lib/pg_qualstats.so
#3 0x000000000055dade in PortalCleanup ()
#4 0x000000000077438a in PortalDrop ()
#5 0x000000000077472a in PreCommit_Portals ()
#6 0x00000000004b36ec in CommitTransaction ()
#7 0x00000000004b4ab5 in CommitTransactionCommand ()
#8 0x000000000067a269 in finish_xact_command ()
#9 0x000000000067d685 in PostgresMain ()
#10 0x000000000062c246 in PostmasterMain ()
#11 0x00000000005c4ab8 in main ()

Each time, I've seen the process looping infinitely from 75e910 to 75e927:
0x000000000075e910 <+96>: mov (%r9,%rcx,8),%rax
0x000000000075e914 <+100>: test %rax,%rax
0x000000000075e917 <+103>: jne 0x75e93c <hash_seq_search+140>
0x000000000075e919 <+105>: add $0x1,%edx
---Type <return> to continue, or q <return> to quit---
=> 0x000000000075e91c <+108>: cmp %edx,%esi
0x000000000075e91e <+110>: jb 0x75e958 <hash_seq_search+168>
0x000000000075e920 <+112>: add $0x1,%rcx
0x000000000075e924 <+116>: cmp %rcx,%r8
0x000000000075e927 <+119>: jg 0x75e910 <hash_seq_search+96>

(I know 9.4.1 is not the last minor release for 9.4 but I didn't find this
bug reported for the >=9.4 majors)

Do you have an idea of what is causing this strange issue ?

Best regards,

Amaury

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Amaury Decreme (#1)
Re: BUG #14261: Hanged randomly in hash_seq_search

amdecreme@airfrance.fr writes:

The process can be found stuck in the hash_seq_search().

Call stack:
#0 0x000000000075e919 in hash_seq_search ()
#1 0x00007fd0af74fb2b in pgqs_entry_dealloc ()
from /tech/postgres/9.4.1/server/lib/pg_qualstats.so
#2 0x00007fd0af750a15 in pgqs_ExecutorEnd ()
from /tech/postgres/9.4.1/server/lib/pg_qualstats.so
#3 0x000000000055dade in PortalCleanup ()
#4 0x000000000077438a in PortalDrop ()
#5 0x000000000077472a in PreCommit_Portals ()
#6 0x00000000004b36ec in CommitTransaction ()

I do not know what pg_qualstats.so is, but it's certainly not part of the
PG core distribution. I suggest asking its author(s) about known bugs
in their code ...

regards, tom lane

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#3Julien Rouhaud
rjuju123@gmail.com
In reply to: Tom Lane (#2)
Re: BUG #14261: Hanged randomly in hash_seq_search

On 19/07/2016 18:58, Tom Lane wrote:

amdecreme@airfrance.fr writes:

The process can be found stuck in the hash_seq_search().

Call stack:
#0 0x000000000075e919 in hash_seq_search ()
#1 0x00007fd0af74fb2b in pgqs_entry_dealloc ()
from /tech/postgres/9.4.1/server/lib/pg_qualstats.so
#2 0x00007fd0af750a15 in pgqs_ExecutorEnd ()
from /tech/postgres/9.4.1/server/lib/pg_qualstats.so
#3 0x000000000055dade in PortalCleanup ()
#4 0x000000000077438a in PortalDrop ()
#5 0x000000000077472a in PreCommit_Portals ()
#6 0x00000000004b36ec in CommitTransaction ()

I do not know what pg_qualstats.so is, but it's certainly not part of the
PG core distribution. I suggest asking its author(s) about known bugs
in their code ...

Culprit here, and unfortunately that's not a bug that've already been
reported.

Amaury, could you instead send a bug report on
https://github.com/dalibo/pg_qualstats/issues ?

--
Julien Rouhaud
http://dalibo.com - http://dalibo.org

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#4Amaury Decreme
amdecreme@airfrance.fr
In reply to: Julien Rouhaud (#3)
Re: BUG #14261: Hanged randomly in hash_seq_search

Thanks for your answers.

I'm going to fill this bug on pg_qualstats repo.

Best regards,

Amaury Decreme
Capacity Management team

Phone +33 4 97283060 / 70 83060
Mail amdecreme@airfrance.fr
Jabber amdecreme@jabber.airfrance.fr
(Home office on Friday: +33 6 63 99 83 13)
Air France KLM

De : Julien Rouhaud <julien.rouhaud@dalibo.com>

A : Tom Lane <tgl@sss.pgh.pa.us>, amdecreme@airfrance.fr

Cc : pgsql-bugs@postgresql.org, Ronan Dunklau <ronan.dunklau@dalibo.com>

Date: 19/07/2016 20:00

Objet : Re: [BUGS] BUG #14261: Hanged randomly in hash_seq_search

On 19/07/2016 18:58, Tom Lane wrote:

amdecreme@airfrance.fr writes:

The process can be found stuck in the hash_seq_search().

Call stack:
#0 0x000000000075e919 in hash_seq_search ()
#1 0x00007fd0af74fb2b in pgqs_entry_dealloc ()
from /tech/postgres/9.4.1/server/lib/pg_qualstats.so
#2 0x00007fd0af750a15 in pgqs_ExecutorEnd ()
from /tech/postgres/9.4.1/server/lib/pg_qualstats.so
#3 0x000000000055dade in PortalCleanup ()
#4 0x000000000077438a in PortalDrop ()
#5 0x000000000077472a in PreCommit_Portals ()
#6 0x00000000004b36ec in CommitTransaction ()

I do not know what pg_qualstats.so is, but it's certainly not part of the
PG core distribution. I suggest asking its author(s) about known bugs
in their code ...

Culprit here, and unfortunately that's not a bug that've already been
reported.

Amaury, could you instead send a bug report on
https://github.com/dalibo/pg_qualstats/issues ?

--
Julien Rouhaud
http://dalibo.com - http://dalibo.org

--
Accédez aux meilleurs tarifs Air France, gérez vos réservations et enregistrez-vous en ligne sur http://www.airfrance.com
Find best Air France fares, manage your reservations and check in online at http://www.airfrance.com Les données et renseignements contenus dans ce message peuvent être de nature confidentielle et soumis au secret professionnel et sont destinés à l'usage exclusif du destinataire dont les coordonnées figurent ci-dessus. Si vous recevez cette communication par erreur, nous vous demandons de ne pas la copier, l'utiliser ou la divulguer. Nous vous prions de notifier cette erreur à l'expéditeur et d'effacer immédiatement cette communication de votre système. Société Air France - Société anonyme au capital de 126 748 775 euros - RCS Bobigny (France) 420 495 178 - 45, rue de Paris, Tremblay-en-France, 95747 Roissy Charles de Gaulle CEDEX
The data and information contained in this message may be confidential and subject to professional secrecy and are intended for the exclusive use of the recipient at the address shown above. If you receive this message by mistake, we ask you not to copy, use or disclose it. Please notify this error to the sender immediately and delete this message from your system. Société Air France - Limited company with capital of 126,748,775 euros - Bobigny register of companies (France) 420 495 178 - 45, rue de Paris, Tremblay-en-France, 95747 Roissy Charles de Gaulle CEDEX Pensez à l'environnement avant d'imprimer ce message.
Think of the environment before printing this mail.

Attachments:

10802846.jpgimage/jpeg; name=10802846.jpgDownload
graycol.gifimage/gif; name=graycol.gifDownload
ecblank.gifimage/gif; name=ecblank.gifDownload