Many processes blocked at ProcArrayLock

On Tue, Dec 2, 2014 at 5:07 PM, Xiaoyulei <xiaoyulei@huawei.com> wrote:

Test configuration:
Hardware:
4P intel server, 60 core 120 hard thread.
Memory:512G
SSD:2.4T

PG:
max_connections = 160 # (change requires restart)
shared_buffers = 32GB
work_mem = 128MB
maintenance_work_mem = 32MB
bgwriter_delay = 100ms # 10-10000ms between rounds
bgwriter_lru_maxpages = 200 # 0-1000 max buffers written/round
bgwriter_lru_multiplier = 2.0 # 0-10.0 multipler on buffers scanned/round
wal_level = minimal # minimal, archive, or hot_standby
wal_buffers = 256MB # min 32kB, -1 sets based on shared_buffers
autovacuum = off
checkpoint_timeout=60min
checkpoint_segments = 1000
archive_mode = off
synchronous_commit = off
fsync = off
full_page_writes = off

We use tpcc and pgbench to test postgresql 9.4beat2 performance. And we found the tps/tpmc could not increase with the terminal increase. The detail information is in attachment.

Many processes is blocked, I dump the call stack, and found these processes is blocked at: ProcArrayLock. 60% processes is blocked in ProcArrayEndTransaction with ProcArrayLock EXCLUSIVE, 20% is in GetSnapshotData with ProcArrayLock SHARED. Others locks like XLogFlush and WALInsertLock are not very heavy.

Is there any way we solve this problem?

Providing complete backtraces showing in which code paths those
processes are blocked would help better in understand what may be
going on.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

I put all the stack in attachment.

Xiaoyulei wrote:

I put all the stack in attachment.

Not sure that this is really all that useful. At least I don't have the
patience to examine all this, and I'm not sure it contains the needed
info in the first place. If you were to ensure your build is using
-fno-omit-frame-pointer in cflags and then used "perf record -a -g"
while the test runs and "perf report -g" once it's finished, you'd get a
useful profile that would show who is acquiring the problematic lock and
why.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers