Postmaster only takes 4-5% CPU
Hi all:
When I run a (big) query from pgsql, postmaster used to take almost 100%
of CPU. Now, with aparent no reason, Postmaster only takes 4-5% CPU and
the same query takes really longer to finish.
I have tried renicing -20 postmaster but it still only takes a small
percentage of CPU, while 90% is idle.
This is a "top" screen while running the query:
2:27pm up 5 min, 2 users, load average: 0,80, 0,26, 0,08
74 processes: 72 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 3,1% user, 2,1% system, 0,0% nice, 94,6% idle
Mem: 1031012K av, 246040K used, 784972K free, 0K shrd, 11496K
buff
Swap: 2040244K av, 0K used, 2040244K free 196916K
cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1165 postgres 0 -20 3476 3476 2912 D < 4,5 0,3 0:02 postmaster
268 root 15 0 0 0 0 SW 0,1 0,0 0:00 kjournald
Any idea of how can I get postmaster to eat as much CPU as possible?
Thanks a lot! Ruben.
Did not your postmaster wait for harddisk ?
Ruben wrote:
Show quoted text
Hi all:
When I run a (big) query from pgsql, postmaster used to take almost 100%
of CPU. Now, with aparent no reason, Postmaster only takes 4-5% CPU and
the same query takes really longer to finish.I have tried renicing -20 postmaster but it still only takes a small
percentage of CPU, while 90% is idle.This is a "top" screen while running the query:
2:27pm up 5 min, 2 users, load average: 0,80, 0,26, 0,08
74 processes: 72 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 3,1% user, 2,1% system, 0,0% nice, 94,6% idle
Mem: 1031012K av, 246040K used, 784972K free, 0K shrd, 11496K
buff
Swap: 2040244K av, 0K used, 2040244K free 196916K
cachedPID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1165 postgres 0 -20 3476 3476 2912 D < 4,5 0,3 0:02 postmaster
268 root 15 0 0 0 0 SW 0,1 0,0 0:00 kjournaldAny idea of how can I get postmaster to eat as much CPU as possible?
Thanks a lot! Ruben.
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
Hi Jan:
There is no disk activity to wait for.
Thanks.
Jan Poslusny wrote:
Show quoted text
Did not your postmaster wait for harddisk ?
Ruben wrote:
Hi all:
When I run a (big) query from pgsql, postmaster used to take almost
100% of CPU. Now, with aparent no reason, Postmaster only takes 4-5%
CPU and the same query takes really longer to finish.I have tried renicing -20 postmaster but it still only takes a small
percentage of CPU, while 90% is idle.This is a "top" screen while running the query:
2:27pm up 5 min, 2 users, load average: 0,80, 0,26, 0,08
74 processes: 72 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 3,1% user, 2,1% system, 0,0% nice, 94,6% idle
Mem: 1031012K av, 246040K used, 784972K free, 0K shrd,
11496K buff
Swap: 2040244K av, 0K used, 2040244K free
196916K cachedPID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1165 postgres 0 -20 3476 3476 2912 D < 4,5 0,3 0:02 postmaster
268 root 15 0 0 0 0 SW 0,1 0,0 0:00 kjournaldAny idea of how can I get postmaster to eat as much CPU as possible?
Thanks a lot! Ruben.
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
On 29 May 2003 at 13:24, Ruben wrote:
When I run a (big) query from pgsql, postmaster used to take almost 100%
of CPU. Now, with aparent no reason, Postmaster only takes 4-5% CPU and
the same query takes really longer to finish.
Check vmstat when query is running. Is it saturating disk BW? If yes, have you
vacuummed the table/database recently?
HTH
Bye
Shridhar
--
Slurm, n.: The slime that accumulates on the underside of a soap bar when it
sits in the dish too long. -- Rich Hall, "Sniglets"
Since there is 2.1% system usage, kjournald has cpu-usage and postgres
has the status 'D' that's probably the case.
Arjen
Show quoted text
Jan Poslusny wrote:
Did not your postmaster wait for harddisk ?
Ruben wrote:
Hi all:
When I run a (big) query from pgsql, postmaster used to take almost
100%
of CPU. Now, with aparent no reason, Postmaster only takes4-5% CPU and
the same query takes really longer to finish.
I have tried renicing -20 postmaster but it still only takes a small
percentage of CPU, while 90% is idle.This is a "top" screen while running the query:
2:27pm up 5 min, 2 users, load average: 0,80, 0,26, 0,08 74
processes: 72 sleeping, 2 running, 0 zombie, 0 stopped CPU states:
3,1% user, 2,1% system, 0,0% nice, 94,6% idle
Mem: 1031012K av, 246040K used, 784972K free, 0Kshrd, 11496K
buff
Swap: 2040244K av, 0K used, 2040244K free196916K
cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM
TIME COMMAND
1165 postgres 0 -20 3476 3476 2912 D < 4,5 0,3
0:02 postmaster
268 root 15 0 0 0 0 SW 0,1 0,0
0:00 kjournald
Any idea of how can I get postmaster to eat as much CPU as possible?
Thanks a lot! Ruben.
---------------------------(end of
broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster---------------------------(end of
broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to
majordomo@postgresql.org
On Thu, 29 May 2003, Ruben wrote:
Hi all:
When I run a (big) query from pgsql, postmaster used to take almost 100%
of CPU. Now, with aparent no reason, Postmaster only takes 4-5% CPU and
the same query takes really longer to finish.I have tried renicing -20 postmaster but it still only takes a small
percentage of CPU, while 90% is idle.This is a "top" screen while running the query:
2:27pm up 5 min, 2 users, load average: 0,80, 0,26, 0,08
74 processes: 72 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 3,1% user, 2,1% system, 0,0% nice, 94,6% idle
Mem: 1031012K av, 246040K used, 784972K free, 0K shrd, 11496K
buff
Swap: 2040244K av, 0K used, 2040244K free 196916K
cachedPID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1165 postgres 0 -20 3476 3476 2912 D < 4,5 0,3 0:02 postmaster
268 root 15 0 0 0 0 SW 0,1 0,0 0:00 kjournaldAny idea of how can I get postmaster to eat as much CPU as possible?
Looks like you're waiting on I/O. What's your disk subsystem
configuration?
Thanks for your answers Jan, Arjen, Shridar and Scott:
Postmaster is back taking lots of CPU when needed. I guess you were
right about Postgres waiting for some I/O activity. Anyway, I was not
able to see what process was responsible for this, since top showed
nothing remarkable.
This is a new top while running a VACUUM of the database:
8:51pm up 6:30, 3 users, load average: 0,45, 0,11, 0,04
93 processes: 89 sleeping, 4 running, 0 zombie, 0 stopped
CPU states: 0,1% user, 1,1% system, 0,0% nice, 98,6% idle
Mem: 1031012K av, 1019612K used, 11400K free, 0K shrd, 44896K buff
Swap: 2040244K av, 1716K used, 2038528K free 894008K cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1673 postgres 15 0 5120 5120 3196 R 0,9 0,4 0:02 postmaster
1683 root 15 0 1044 1044 840 R 0,1 0,1 0:00 top
1 root 15 0 476 476 424 S 0,0 0,0 0:04 init
2 root 15 0 0 0 0 SW 0,0 0,0 0:00 keventd
3 root 15 0 0 0 0 SW 0,0 0,0 0:00 kapmd
4 root 34 19 0 0 0 SWN 0,0 0,0 0:00
ksoftirqd_CPU0
5 root 15 0 0 0 0 SW 0,0 0,0 0:00 kswapd
6 root 15 0 0 0 0 SW 0,0 0,0 0:00 bdflush
7 root 15 0 0 0 0 SW 0,0 0,0 0:00 kupdated
8 root 25 0 0 0 0 SW 0,0 0,0 0:00 mdrecoveryd
12 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
66 root 15 0 0 0 0 SW 0,0 0,0 0:00 khubd
266 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
267 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
268 root 15 0 0 0 0 SW 0,0 0,0 0:05 kjournald
269 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
547 root 15 0 0 0 0 SW 0,0 0,0 0:00 eth0
Cheers, Ruben.
Jan Poslusny wrote:
Show quoted text
Did not your postmaster wait for harddisk ?
Ruben wrote:
Hi all:
When I run a (big) query from pgsql, postmaster used to take almost
100% of CPU. Now, with aparent no reason, Postmaster only takes 4-5%
CPU and the same query takes really longer to finish.I have tried renicing -20 postmaster but it still only takes a small
percentage of CPU, while 90% is idle.This is a "top" screen while running the query:
2:27pm up 5 min, 2 users, load average: 0,80, 0,26, 0,08
74 processes: 72 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 3,1% user, 2,1% system, 0,0% nice, 94,6% idle
Mem: 1031012K av, 246040K used, 784972K free, 0K shrd,
11496K buff
Swap: 2040244K av, 0K used, 2040244K free
196916K cachedPID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1165 postgres 0 -20 3476 3476 2912 D < 4,5 0,3 0:02 postmaster
268 root 15 0 0 0 0 SW 0,1 0,0 0:00 kjournaldAny idea of how can I get postmaster to eat as much CPU as possible?
Thanks a lot! Ruben.
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster
Under some circumstances, on my (somewhat) older 7.2 box running the 2.4.9
kernel, the kswapd and bdflush daemons will start working overtime when
there are accesses by windows users via samba to look through large
directories.
This makes the box QUITE slow and pokey, and all the tasks that are
running besides kswapd / bdflush are in D state mostly, while the swap
daemon eats 0 to 100% CPU depending on some change in state in samba I've
not bothered tracking down.
I understand a newer kernel might be a good thing for me, so eventually
I'll do that. For now I just put the Windows Samba people on another box.
They both go through the the same NAS, and it keeps the load off the main
server.
So, look for other processes that might be doing something like scanning a
directory of 40,000 files or some such.
On Thu, 29 May 2003, Ruben wrote:
Show quoted text
Thanks for your answers Jan, Arjen, Shridar and Scott:
Postmaster is back taking lots of CPU when needed. I guess you were
right about Postgres waiting for some I/O activity. Anyway, I was not
able to see what process was responsible for this, since top showed
nothing remarkable.This is a new top while running a VACUUM of the database:
8:51pm up 6:30, 3 users, load average: 0,45, 0,11, 0,04
93 processes: 89 sleeping, 4 running, 0 zombie, 0 stopped
CPU states: 0,1% user, 1,1% system, 0,0% nice, 98,6% idle
Mem: 1031012K av, 1019612K used, 11400K free, 0K shrd, 44896K buff
Swap: 2040244K av, 1716K used, 2038528K free 894008K cachedPID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1673 postgres 15 0 5120 5120 3196 R 0,9 0,4 0:02 postmaster
1683 root 15 0 1044 1044 840 R 0,1 0,1 0:00 top
1 root 15 0 476 476 424 S 0,0 0,0 0:04 init
2 root 15 0 0 0 0 SW 0,0 0,0 0:00 keventd
3 root 15 0 0 0 0 SW 0,0 0,0 0:00 kapmd
4 root 34 19 0 0 0 SWN 0,0 0,0 0:00
ksoftirqd_CPU0
5 root 15 0 0 0 0 SW 0,0 0,0 0:00 kswapd
6 root 15 0 0 0 0 SW 0,0 0,0 0:00 bdflush
7 root 15 0 0 0 0 SW 0,0 0,0 0:00 kupdated
8 root 25 0 0 0 0 SW 0,0 0,0 0:00 mdrecoveryd
12 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
66 root 15 0 0 0 0 SW 0,0 0,0 0:00 khubd
266 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
267 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
268 root 15 0 0 0 0 SW 0,0 0,0 0:05 kjournald
269 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
547 root 15 0 0 0 0 SW 0,0 0,0 0:00 eth0Cheers, Ruben.
Jan Poslusny wrote:
Did not your postmaster wait for harddisk ?
Ruben wrote:
Hi all:
When I run a (big) query from pgsql, postmaster used to take almost
100% of CPU. Now, with aparent no reason, Postmaster only takes 4-5%
CPU and the same query takes really longer to finish.I have tried renicing -20 postmaster but it still only takes a small
percentage of CPU, while 90% is idle.This is a "top" screen while running the query:
2:27pm up 5 min, 2 users, load average: 0,80, 0,26, 0,08
74 processes: 72 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 3,1% user, 2,1% system, 0,0% nice, 94,6% idle
Mem: 1031012K av, 246040K used, 784972K free, 0K shrd,
11496K buff
Swap: 2040244K av, 0K used, 2040244K free
196916K cachedPID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1165 postgres 0 -20 3476 3476 2912 D < 4,5 0,3 0:02 postmaster
268 root 15 0 0 0 0 SW 0,1 0,0 0:00 kjournaldAny idea of how can I get postmaster to eat as much CPU as possible?
Thanks a lot! Ruben.
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
Hi:
I'm sorry to get back with this, but I claimed victory too soon.
Postmaster is back (intermitently) taking just a small % of cpu while
system is almost 100% idle, as you can see in "top" below.
Any ideas of how can I fix this? Programs running queries are going too
slooooow.
Thanks!
11:21am up 7 days, 21:00, 6 users, load average: 1,10, 1,04, 0,86
106 processes: 102 sleeping, 4 running, 0 zombie, 0 stopped
CPU states: 0,5% user, 1,7% system, 0,0% nice, 97,6% idle
Mem: 1031012K av, 1019052K used, 11960K free, 0K shrd, 11340K buff
Swap: 2040244K av, 153828K used, 1886416K free 958512K cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
23057 postgres 15 0 3316 3136 2292 R 1,1 0,3 0:17 postmaster
23073 postgres 15 0 1096 1096 840 R 0,1 0,1 0:00 top
1 root 15 0 468 428 416 S 0,0 0,0 0:04 init
2 root 15 0 0 0 0 SW 0,0 0,0 0:00 keventd
3 root 15 0 0 0 0 SW 0,0 0,0 0:00 kapmd
4 root 34 19 0 0 0 SWN 0,0 0,0 0:00 ksoftirqd_C
5 root 15 0 0 0 0 SW 0,0 0,0 1:18 kswapd
6 root 15 0 0 0 0 SW 0,0 0,0 0:07 bdflush
7 root 15 0 0 0 0 SW 0,0 0,0 0:00 kupdated
8 root 25 0 0 0 0 SW 0,0 0,0 0:00 mdrecoveryd
12 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
66 root 15 0 0 0 0 SW 0,0 0,0 0:00 khubd
266 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
267 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
268 root 15 0 0 0 0 SW 0,0 0,0 0:29 kjournald
269 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
547 root 15 0 0 0 0 SW 0,0 0,0 0:00 eth0
scott.marlowe wrote:
Show quoted text
Under some circumstances, on my (somewhat) older 7.2 box running the 2.4.9
kernel, the kswapd and bdflush daemons will start working overtime when
there are accesses by windows users via samba to look through large
directories.This makes the box QUITE slow and pokey, and all the tasks that are
running besides kswapd / bdflush are in D state mostly, while the swap
daemon eats 0 to 100% CPU depending on some change in state in samba I've
not bothered tracking down.I understand a newer kernel might be a good thing for me, so eventually
I'll do that. For now I just put the Windows Samba people on another box.
They both go through the the same NAS, and it keeps the load off the main
server.So, look for other processes that might be doing something like scanning a
directory of 40,000 files or some such.On Thu, 29 May 2003, Ruben wrote:
Thanks for your answers Jan, Arjen, Shridar and Scott:
Postmaster is back taking lots of CPU when needed. I guess you were
right about Postgres waiting for some I/O activity. Anyway, I was not
able to see what process was responsible for this, since top showed
nothing remarkable.This is a new top while running a VACUUM of the database:
8:51pm up 6:30, 3 users, load average: 0,45, 0,11, 0,04
93 processes: 89 sleeping, 4 running, 0 zombie, 0 stopped
CPU states: 0,1% user, 1,1% system, 0,0% nice, 98,6% idle
Mem: 1031012K av, 1019612K used, 11400K free, 0K shrd, 44896K buff
Swap: 2040244K av, 1716K used, 2038528K free 894008K cachedPID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1673 postgres 15 0 5120 5120 3196 R 0,9 0,4 0:02 postmaster
1683 root 15 0 1044 1044 840 R 0,1 0,1 0:00 top
1 root 15 0 476 476 424 S 0,0 0,0 0:04 init
2 root 15 0 0 0 0 SW 0,0 0,0 0:00 keventd
3 root 15 0 0 0 0 SW 0,0 0,0 0:00 kapmd
4 root 34 19 0 0 0 SWN 0,0 0,0 0:00
ksoftirqd_CPU0
5 root 15 0 0 0 0 SW 0,0 0,0 0:00 kswapd
6 root 15 0 0 0 0 SW 0,0 0,0 0:00 bdflush
7 root 15 0 0 0 0 SW 0,0 0,0 0:00 kupdated
8 root 25 0 0 0 0 SW 0,0 0,0 0:00 mdrecoveryd
12 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
66 root 15 0 0 0 0 SW 0,0 0,0 0:00 khubd
266 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
267 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
268 root 15 0 0 0 0 SW 0,0 0,0 0:05 kjournald
269 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
547 root 15 0 0 0 0 SW 0,0 0,0 0:00 eth0Cheers, Ruben.
Jan Poslusny wrote:
Did not your postmaster wait for harddisk ?
Ruben wrote:
Hi all:
When I run a (big) query from pgsql, postmaster used to take almost
100% of CPU. Now, with aparent no reason, Postmaster only takes 4-5%
CPU and the same query takes really longer to finish.I have tried renicing -20 postmaster but it still only takes a small
percentage of CPU, while 90% is idle.This is a "top" screen while running the query:
2:27pm up 5 min, 2 users, load average: 0,80, 0,26, 0,08
74 processes: 72 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 3,1% user, 2,1% system, 0,0% nice, 94,6% idle
Mem: 1031012K av, 246040K used, 784972K free, 0K shrd,
11496K buff
Swap: 2040244K av, 0K used, 2040244K free
196916K cachedPID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1165 postgres 0 -20 3476 3476 2912 D < 4,5 0,3 0:02 postmaster
268 root 15 0 0 0 0 SW 0,1 0,0 0:00 kjournaldAny idea of how can I get postmaster to eat as much CPU as possible?
Thanks a lot! Ruben.
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
On Fri, 06 Jun 2003, Ruben wrote:
Hi:
I'm sorry to get back with this, but I claimed victory too soon.
Postmaster is back (intermitently) taking just a small % of cpu while
system is almost 100% idle, as you can see in "top" below.Any ideas of how can I fix this? Programs running queries are going too
slooooow.
11:21am up 7 days, 21:00, 6 users, load average: 1,10, 1,04, 0,86
106 processes: 102 sleeping, 4 running, 0 zombie, 0 stopped
CPU states: 0,5% user, 1,7% system, 0,0% nice, 97,6% idle
Mem: 1031012K av, 1019052K used, 11960K free, 0K shrd, 11340K buff
Swap: 2040244K av, 153828K used, 1886416K free 958512K cached
What does the output of 'vmstat 5 15' look like while you're running the
query? (that command will run for 75 seconds.)
I'm willing to bet your problem is I/O contention.
-j
--
Jamie Lawrence jal@jal.org
"Many and loud have been the voices crying that Aids was God's
way of punishing an unnatural practice. One day, perhaps, an
inoculation against HIV may be discovered. A bottle of champagne,
then, for whoever cites me evidence of one of those voices
crying that the breakthrough is God's way of telling us to
bugger each other."
- Matthew Parris
Thanks Jamie:
I'll send the output of vmstat next time this happens.
Ruben.
Jamie Lawrence wrote:
Show quoted text
On Fri, 06 Jun 2003, Ruben wrote:
Hi:
I'm sorry to get back with this, but I claimed victory too soon.
Postmaster is back (intermitently) taking just a small % of cpu while
system is almost 100% idle, as you can see in "top" below.Any ideas of how can I fix this? Programs running queries are going too
slooooow.11:21am up 7 days, 21:00, 6 users, load average: 1,10, 1,04, 0,86
106 processes: 102 sleeping, 4 running, 0 zombie, 0 stopped
CPU states: 0,5% user, 1,7% system, 0,0% nice, 97,6% idle
Mem: 1031012K av, 1019052K used, 11960K free, 0K shrd, 11340K buff
Swap: 2040244K av, 153828K used, 1886416K free 958512K cachedWhat does the output of 'vmstat 5 15' look like while you're running the
query? (that command will run for 75 seconds.)I'm willing to bet your problem is I/O contention.
-j