[w32] test_shm_mq test suite permanently burns connections slots

On Fri, Jul 25, 2014 at 3:25 PM, Noah Misch <noah@leadboat.com> wrote:

On a Windows or other EXEC_BACKEND build, the following eventually gets
failures because all, or all but one, max_connections slot is consumed:

for run in `seq 1 100`; do make -C contrib/test_shm_mq installcheck; done

When I use max_connections=40, it fails on the sixth iteration. Only the six
basic processes are actually running at that time.

The tests start 7 workers each time, so that makes sense: 7 * 5 < 40
but 7 * 6 > 40. What I'm not sure is why they are leaking connection
slots, and why they're only doing it in EXEC_BACKEND mode. A quick
code audit didn't uncover any obvious explanation, so I'll try to
reproduce and debug.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Mon, Jul 28, 2014 at 3:59 PM, Robert Haas <robertmhaas@gmail.com> wrote:

On Fri, Jul 25, 2014 at 3:25 PM, Noah Misch <noah@leadboat.com> wrote:

On a Windows or other EXEC_BACKEND build, the following eventually gets
failures because all, or all but one, max_connections slot is consumed:

for run in `seq 1 100`; do make -C contrib/test_shm_mq installcheck; done

When I use max_connections=40, it fails on the sixth iteration. Only the six
basic processes are actually running at that time.

The tests start 7 workers each time, so that makes sense: 7 * 5 < 40
but 7 * 6 > 40. What I'm not sure is why they are leaking connection
slots, and why they're only doing it in EXEC_BACKEND mode. A quick
code audit didn't uncover any obvious explanation, so I'll try to
reproduce and debug.

OK, I think I see the problem. In EXEC_BACKEND mode,
SubPostmasterMain() calls InitProcess() before IsBackgroundWorker has
been set. InitProcess() therefore pulls the PGPROC for the worker
from freeProcs rather than bgworkerFreeProcs. By exit time,
IsBackgroundWorker has been set, so the PGPROC gets put back on the
bgworkerFreeProcs list. Eventually there are no regular PGPROCs left;
they've all been moved to the bgworkerFreeProcs list.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Robert Haas wrote:

OK, I think I see the problem. In EXEC_BACKEND mode,
SubPostmasterMain() calls InitProcess() before IsBackgroundWorker has
been set. InitProcess() therefore pulls the PGPROC for the worker
from freeProcs rather than bgworkerFreeProcs. By exit time,
IsBackgroundWorker has been set, so the PGPROC gets put back on the
bgworkerFreeProcs list. Eventually there are no regular PGPROCs left;
they've all been moved to the bgworkerFreeProcs list.

Doh. I'm surprised -- I tested a worker that crashed over and over to
ensure PGPROCs were reused sanely. I guess I forgot to run it under
EXEC_BACKEND.

Are you fixing it?

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Mon, Jul 28, 2014 at 9:38 PM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:

Robert Haas wrote:

OK, I think I see the problem. In EXEC_BACKEND mode,
SubPostmasterMain() calls InitProcess() before IsBackgroundWorker has
been set. InitProcess() therefore pulls the PGPROC for the worker
from freeProcs rather than bgworkerFreeProcs. By exit time,
IsBackgroundWorker has been set, so the PGPROC gets put back on the
bgworkerFreeProcs list. Eventually there are no regular PGPROCs left;
they've all been moved to the bgworkerFreeProcs list.

Doh. I'm surprised -- I tested a worker that crashed over and over to
ensure PGPROCs were reused sanely. I guess I forgot to run it under
EXEC_BACKEND.

Are you fixing it?

Working on it now.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers