There are many processes created and died frequently during PostgreSQl idle

Started by 高健almost 13 years ago4 messagesgeneral
Jump to latest
#1高健
luckyjackgao@gmail.com

Hello all:

I found that during postgresql running, there are so many processes being
created and then died.
I am interested in the reason.

Here is the detail:
I installed from postgresql-9.2.1.tar.bz2.
I put some debug code in fd.c 's PathNameOpenFile function:

fprintf(stderr,"+++While Calling PathNameOpenFile pid=%d.....",getpid();
fprintf(stderr,"VfdCache Address is: %p \n\n",VfdCache);

When it run, I open two termianal using psql(pid=22109 and pid=22114), then
I can got information like this:
[postgres@lex pgsql]$ ./bin/pg_ctl -D ./data start
server starting
[postgres@lex pgsql]$ LOG: database system was shut down at 2013-05-24
09:28:59 CST
LOG: autovacuum launcher started
+++While Calling PathNameOpenFile pid=22106.....VfdCache Address is:
0xbfb3320

LOG: database system is ready to accept connections
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is:
0xbfba400

+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is:
0xbfba400

+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is:
0xbfba400

+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is:
0xbfba400

+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is:
0xbfe6700

But when I grep process information using ps command, I can't find process
such like 22120,22127,22134,22142,22149,22156,22178.
And the output is growing longer and longer...

[root@lex ~]# ps -ef | grep post
root 4641 4621 0 08:28 pts/3 00:00:00 su - postgres
postgres 4642 4641 0 08:28 pts/3 00:00:00 -bash
root 4709 4688 0 08:28 pts/4 00:00:00 su - postgres
postgres 4710 4709 0 08:28 pts/4 00:00:00 -bash
root 4772 4751 0 08:29 pts/5 00:00:00 su - postgres
postgres 4773 4772 0 08:29 pts/5 00:00:00 -bash
postgres 22101 1 0 09:45 pts/3 00:00:00
/usr/local/pgsql/bin/postgres -D ./data
postgres 22103 22101 0 09:45 ? 00:00:00 postgres: checkpointer
process
postgres 22104 22101 0 09:45 ? 00:00:00 postgres: writer process

postgres 22105 22101 0 09:45 ? 00:00:00 postgres: wal writer
process
postgres 22106 22101 0 09:45 ? 00:00:00 postgres: autovacuum
launcher process
postgres 22107 22101 0 09:45 ? 00:00:00 postgres: stats collector
process
postgres 22108 4710 0 09:45 pts/4 00:00:00 ./psql
postgres 22109 22101 0 09:45 ? 00:00:00 postgres: postgres postgres
[local] idle
postgres 22113 4773 0 09:46 pts/5 00:00:00 ./psql
postgres 22114 22101 0 09:46 ? 00:00:00 postgres: postgres postgres
[local] idle
root 22187 6429 0 09:55 pts/6 00:00:00 grep post
[root@lex ~]#

So I think that some process is created and then quickly died . But would
somebody can kindly say what are the purpose of those process?
How about using one worker process and keep it be alive?

#2Jov
amutu@amutu.com
In reply to: 高健 (#1)
Re: There are many processes created and died frequently during PostgreSQl idle

the process you see may be autovacuum worker process,you can tune the
log_autovacuum_min_duration
to log the autovacuum activities and check if the pids match
http://www.postgresql.org/docs/9.2/static/runtime-config-autovacuum.html

2013/5/24 高健 <luckyjackgao@gmail.com>

Hello all:

I found that during postgresql running, there are so many processes
being created and then died.
I am interested in the reason.

Here is the detail:
I installed from postgresql-9.2.1.tar.bz2.
I put some debug code in fd.c 's PathNameOpenFile function:

fprintf(stderr,"+++While Calling PathNameOpenFile pid=%d.....",getpid();
fprintf(stderr,"VfdCache Address is: %p \n\n",VfdCache);

When it run, I open two termianal using psql(pid=22109 and pid=22114),
then I can got information like this:
[postgres@lex pgsql]$ ./bin/pg_ctl -D ./data start
server starting
[postgres@lex pgsql]$ LOG: database system was shut down at 2013-05-24
09:28:59 CST
LOG: autovacuum launcher started
+++While Calling PathNameOpenFile pid=22106.....VfdCache Address is:
0xbfb3320

LOG: database system is ready to accept connections
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is:
0xbfba400

+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is:
0xbfba400

+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is:
0xbfba400

+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is:
0xbfba400

+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is:
0xbfe6700

But when I grep process information using ps command, I can't find process
such like 22120,22127,22134,22142,22149,22156,22178.
And the output is growing longer and longer...

[root@lex ~]# ps -ef | grep post
root 4641 4621 0 08:28 pts/3 00:00:00 su - postgres
postgres 4642 4641 0 08:28 pts/3 00:00:00 -bash
root 4709 4688 0 08:28 pts/4 00:00:00 su - postgres
postgres 4710 4709 0 08:28 pts/4 00:00:00 -bash
root 4772 4751 0 08:29 pts/5 00:00:00 su - postgres
postgres 4773 4772 0 08:29 pts/5 00:00:00 -bash
postgres 22101 1 0 09:45 pts/3 00:00:00
/usr/local/pgsql/bin/postgres -D ./data
postgres 22103 22101 0 09:45 ? 00:00:00 postgres: checkpointer
process
postgres 22104 22101 0 09:45 ? 00:00:00 postgres: writer process

postgres 22105 22101 0 09:45 ? 00:00:00 postgres: wal writer
process
postgres 22106 22101 0 09:45 ? 00:00:00 postgres: autovacuum
launcher process
postgres 22107 22101 0 09:45 ? 00:00:00 postgres: stats collector
process
postgres 22108 4710 0 09:45 pts/4 00:00:00 ./psql
postgres 22109 22101 0 09:45 ? 00:00:00 postgres: postgres
postgres [local] idle
postgres 22113 4773 0 09:46 pts/5 00:00:00 ./psql
postgres 22114 22101 0 09:46 ? 00:00:00 postgres: postgres
postgres [local] idle
root 22187 6429 0 09:55 pts/6 00:00:00 grep post
[root@lex ~]#

So I think that some process is created and then quickly died . But would
somebody can kindly say what are the purpose of those process?
How about using one worker process and keep it be alive?

--
Jov
blog: http:amutu.com/blog <http://amutu.com/blog&gt;

#3高健
luckyjackgao@gmail.com
In reply to: Jov (#2)
Re: There are many processes created and died frequently during PostgreSQl idle

Hi:
Thanks for Jov's reply.
I traced it again, and found they are really for autovacuum.
I found that
some will call proc_exit() from within AutoVacLauncherMain function,
some will call proc_exit() from within AutoVacWorkerMain function.

But I wonder why not using only a few daemon , instead of making those
processes be short life.

2013/5/24 Jov <amutu@amutu.com>

Show quoted text

the process you see may be autovacuum worker process,you can tune the log_autovacuum_min_duration
to log the autovacuum activities and check if the pids match
http://www.postgresql.org/docs/9.2/static/runtime-config-autovacuum.html

2013/5/24 高健 <luckyjackgao@gmail.com>

Hello all:

I found that during postgresql running, there are so many processes
being created and then died.
I am interested in the reason.

Here is the detail:
I installed from postgresql-9.2.1.tar.bz2.
I put some debug code in fd.c 's PathNameOpenFile function:

fprintf(stderr,"+++While Calling PathNameOpenFile pid=%d.....",getpid();
fprintf(stderr,"VfdCache Address is: %p \n\n",VfdCache);

When it run, I open two termianal using psql(pid=22109 and pid=22114),
then I can got information like this:
[postgres@lex pgsql]$ ./bin/pg_ctl -D ./data start
server starting
[postgres@lex pgsql]$ LOG: database system was shut down at 2013-05-24
09:28:59 CST
LOG: autovacuum launcher started
+++While Calling PathNameOpenFile pid=22106.....VfdCache Address is:
0xbfb3320

LOG: database system is ready to accept connections
+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22109.....VfdCache Address is:
0xbfb2de0

+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is:
0xbfba400

+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is:
0xbfba400

+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is:
0xbfba400

+++While Calling PathNameOpenFile pid=22114.....VfdCache Address is:
0xbfba400

+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22120.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22127.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22134.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22142.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22149.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22156.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22165.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22171.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is:
0xbfe6700

+++While Calling PathNameOpenFile pid=22178.....VfdCache Address is:
0xbfe6700

But when I grep process information using ps command, I can't find
process such like 22120,22127,22134,22142,22149,22156,22178.
And the output is growing longer and longer...

[root@lex ~]# ps -ef | grep post
root 4641 4621 0 08:28 pts/3 00:00:00 su - postgres
postgres 4642 4641 0 08:28 pts/3 00:00:00 -bash
root 4709 4688 0 08:28 pts/4 00:00:00 su - postgres
postgres 4710 4709 0 08:28 pts/4 00:00:00 -bash
root 4772 4751 0 08:29 pts/5 00:00:00 su - postgres
postgres 4773 4772 0 08:29 pts/5 00:00:00 -bash
postgres 22101 1 0 09:45 pts/3 00:00:00
/usr/local/pgsql/bin/postgres -D ./data
postgres 22103 22101 0 09:45 ? 00:00:00 postgres: checkpointer
process
postgres 22104 22101 0 09:45 ? 00:00:00 postgres: writer process

postgres 22105 22101 0 09:45 ? 00:00:00 postgres: wal writer
process
postgres 22106 22101 0 09:45 ? 00:00:00 postgres: autovacuum
launcher process
postgres 22107 22101 0 09:45 ? 00:00:00 postgres: stats collector
process
postgres 22108 4710 0 09:45 pts/4 00:00:00 ./psql
postgres 22109 22101 0 09:45 ? 00:00:00 postgres: postgres
postgres [local] idle
postgres 22113 4773 0 09:46 pts/5 00:00:00 ./psql
postgres 22114 22101 0 09:46 ? 00:00:00 postgres: postgres
postgres [local] idle
root 22187 6429 0 09:55 pts/6 00:00:00 grep post
[root@lex ~]#

So I think that some process is created and then quickly died . But would
somebody can kindly say what are the purpose of those process?
How about using one worker process and keep it be alive?

--
Jov
blog: http:amutu.com/blog <http://amutu.com/blog&gt;

#4Amit Langote
Langote_Amit_f8@lab.ntt.co.jp
In reply to: 高健 (#3)
Re: There are many processes created and died frequently during PostgreSQl idle

On Mon, May 27, 2013 at 10:56 AM, 高健 <luckyjackgao@gmail.com> wrote:

Hi:
Thanks for Jov's reply.
I traced it again, and found they are really for autovacuum.
I found that
some will call proc_exit() from within AutoVacLauncherMain function,
some will call proc_exit() from within AutoVacWorkerMain function.

But I wonder why not using only a few daemon , instead of making those
processes be short life.

The short lived processes are autovacuum "worker" processes which need
to exit as soon as their work is done. There does exist a daemon
called autovacuum launcher, which periodically forks autovacuum worker
processes to perform vacuum activity. You might want to read more
about them here:

http://www.postgresql.org/docs/9.2/static/routine-vacuuming.html#AUTOVACUUM

--
Amit Langote

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general