connection file descriptors created with identical number after process fork on mac
Hi All,
I'm writing some multi-process code in Python and trying to make sure I
open a new connection for each process. Here's the really cut down code:
"""
import os, time
import psycopg2
from multiprocessing import Pool
def init():
conn = psycopg2.connect("dbname=...host=localhost")
print os.getpid(), ' child fd:', conn.fileno()
if __name__=='__main__':
pool = Pool(initializer=init)
time.sleep(30)
"""
What's really surpising to me is the output on a mac:
$ python psycopg2_multiprocess.py
44276 child fd: 13
44277 child fd: 13
44278 child fd: 13
44279 child fd: 13
The getpid() output indicates that the connec() call is being made
inside a different process each time, yet the connection appears to
still be using the same fd.
conn.file() is basically (long int)PQsocket(self->pgconn);:
https://github.com/psycopg/psycopg2/blob/master/psycopg/connection_type.c#L898
Is there something I'm missing about file descriptors on Macs or is
something bad happening here?
Chris
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
Chris Withers <chris@simplistix.co.uk> writes:
I'm writing some multi-process code in Python and trying to make sure I
open a new connection for each process. Here's the really cut down code:
...
What's really surpising to me is the output on a mac:
$ python psycopg2_multiprocess.py
44276 child fd: 13
44277 child fd: 13
44278 child fd: 13
44279 child fd: 13
The getpid() output indicates that the connec() call is being made
inside a different process each time, yet the connection appears to
still be using the same fd.
FD numbers are process-local in all flavors of Unix. The above only
proves that all of these processes had FDs 0..12 open already, which
doesn't seem terribly surprising.
regards, tom lane
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
On 04/08/2016 00:20, Tom Lane wrote:
Chris Withers <chris@simplistix.co.uk> writes:
I'm writing some multi-process code in Python and trying to make sure I
open a new connection for each process. Here's the really cut down code:
...
What's really surpising to me is the output on a mac:$ python psycopg2_multiprocess.py
44276 child fd: 13
44277 child fd: 13
44278 child fd: 13
44279 child fd: 13The getpid() output indicates that the connec() call is being made
inside a different process each time, yet the connection appears to
still be using the same fd.FD numbers are process-local in all flavors of Unix. The above only
proves that all of these processes had FDs 0..12 open already, which
doesn't seem terribly surprising.
Thanks, that's certainly good news!
How can I convince myself, from the client side, that I really have got
a new connection and not somehow ended up with one that been passed on
as part of the fork?
cheers,
Chris
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
On 8/3/2016 4:25 PM, Chris Withers wrote:
How can I convince myself, from the client side, that I really have
got a new connection and not somehow ended up with one that been
passed on as part of the fork?
$ psql -tc "select pg_backend_pid();"
18635
$ psql -tc "select pg_backend_pid();"
18665
$ psql -tc "select pg_backend_pid();"
18727
now, operating system process ID's do get recycled eventually but not
very fast.
--
john r pierce, recycling bits in santa cruz
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general