Postmaster only takes 4-5% CPU

Started by Rubenalmost 23 years ago11 messagesgeneral
Jump to latest
#1Ruben
ruben12@superguai.com

Hi all:

When I run a (big) query from pgsql, postmaster used to take almost 100%
of CPU. Now, with aparent no reason, Postmaster only takes 4-5% CPU and
the same query takes really longer to finish.

I have tried renicing -20 postmaster but it still only takes a small
percentage of CPU, while 90% is idle.

This is a "top" screen while running the query:

2:27pm up 5 min, 2 users, load average: 0,80, 0,26, 0,08
74 processes: 72 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 3,1% user, 2,1% system, 0,0% nice, 94,6% idle
Mem: 1031012K av, 246040K used, 784972K free, 0K shrd, 11496K
buff
Swap: 2040244K av, 0K used, 2040244K free 196916K
cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1165 postgres 0 -20 3476 3476 2912 D < 4,5 0,3 0:02 postmaster
268 root 15 0 0 0 0 SW 0,1 0,0 0:00 kjournald

Any idea of how can I get postmaster to eat as much CPU as possible?

Thanks a lot! Ruben.

#2Jan Poslusny
pajout@gingerall.cz
In reply to: Ruben (#1)
Re: Postmaster only takes 4-5% CPU

Did not your postmaster wait for harddisk ?

Ruben wrote:

Show quoted text

Hi all:

When I run a (big) query from pgsql, postmaster used to take almost 100%
of CPU. Now, with aparent no reason, Postmaster only takes 4-5% CPU and
the same query takes really longer to finish.

I have tried renicing -20 postmaster but it still only takes a small
percentage of CPU, while 90% is idle.

This is a "top" screen while running the query:

2:27pm up 5 min, 2 users, load average: 0,80, 0,26, 0,08
74 processes: 72 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 3,1% user, 2,1% system, 0,0% nice, 94,6% idle
Mem: 1031012K av, 246040K used, 784972K free, 0K shrd, 11496K
buff
Swap: 2040244K av, 0K used, 2040244K free 196916K
cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1165 postgres 0 -20 3476 3476 2912 D < 4,5 0,3 0:02 postmaster
268 root 15 0 0 0 0 SW 0,1 0,0 0:00 kjournald

Any idea of how can I get postmaster to eat as much CPU as possible?

Thanks a lot! Ruben.

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

#3Ruben
ruben12@superguai.com
In reply to: Ruben (#1)
Re: Postmaster only takes 4-5% CPU

Hi Jan:

There is no disk activity to wait for.

Thanks.

Jan Poslusny wrote:

Show quoted text

Did not your postmaster wait for harddisk ?

Ruben wrote:

Hi all:

When I run a (big) query from pgsql, postmaster used to take almost
100% of CPU. Now, with aparent no reason, Postmaster only takes 4-5%
CPU and the same query takes really longer to finish.

I have tried renicing -20 postmaster but it still only takes a small
percentage of CPU, while 90% is idle.

This is a "top" screen while running the query:

2:27pm up 5 min, 2 users, load average: 0,80, 0,26, 0,08
74 processes: 72 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 3,1% user, 2,1% system, 0,0% nice, 94,6% idle
Mem: 1031012K av, 246040K used, 784972K free, 0K shrd,
11496K buff
Swap: 2040244K av, 0K used, 2040244K free
196916K cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1165 postgres 0 -20 3476 3476 2912 D < 4,5 0,3 0:02 postmaster
268 root 15 0 0 0 0 SW 0,1 0,0 0:00 kjournald

Any idea of how can I get postmaster to eat as much CPU as possible?

Thanks a lot! Ruben.

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

#4Shridhar Daithankar
shridhar_daithankar@persistent.co.in
In reply to: Ruben (#1)
Re: Postmaster only takes 4-5% CPU

On 29 May 2003 at 13:24, Ruben wrote:

When I run a (big) query from pgsql, postmaster used to take almost 100%
of CPU. Now, with aparent no reason, Postmaster only takes 4-5% CPU and
the same query takes really longer to finish.

Check vmstat when query is running. Is it saturating disk BW? If yes, have you
vacuummed the table/database recently?

HTH
Bye
Shridhar

--
Slurm, n.: The slime that accumulates on the underside of a soap bar when it
sits in the dish too long. -- Rich Hall, "Sniglets"

#5Arjen van der Meijden
acm@tweakers.net
In reply to: Jan Poslusny (#2)
Re: Postmaster only takes 4-5% CPU

Since there is 2.1% system usage, kjournald has cpu-usage and postgres
has the status 'D' that's probably the case.

Arjen

Show quoted text

Jan Poslusny wrote:

Did not your postmaster wait for harddisk ?

Ruben wrote:

Hi all:

When I run a (big) query from pgsql, postmaster used to take almost
100%
of CPU. Now, with aparent no reason, Postmaster only takes

4-5% CPU and

the same query takes really longer to finish.

I have tried renicing -20 postmaster but it still only takes a small
percentage of CPU, while 90% is idle.

This is a "top" screen while running the query:

2:27pm up 5 min, 2 users, load average: 0,80, 0,26, 0,08 74
processes: 72 sleeping, 2 running, 0 zombie, 0 stopped CPU states:
3,1% user, 2,1% system, 0,0% nice, 94,6% idle
Mem: 1031012K av, 246040K used, 784972K free, 0K

shrd, 11496K

buff
Swap: 2040244K av, 0K used, 2040244K free

196916K

cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM

TIME COMMAND

1165 postgres 0 -20 3476 3476 2912 D < 4,5 0,3

0:02 postmaster

268 root 15 0 0 0 0 SW 0,1 0,0

0:00 kjournald

Any idea of how can I get postmaster to eat as much CPU as possible?

Thanks a lot! Ruben.

---------------------------(end of
broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

---------------------------(end of
broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to
majordomo@postgresql.org

#6scott.marlowe
scott.marlowe@ihs.com
In reply to: Ruben (#1)
Re: Postmaster only takes 4-5% CPU

On Thu, 29 May 2003, Ruben wrote:

Hi all:

When I run a (big) query from pgsql, postmaster used to take almost 100%
of CPU. Now, with aparent no reason, Postmaster only takes 4-5% CPU and
the same query takes really longer to finish.

I have tried renicing -20 postmaster but it still only takes a small
percentage of CPU, while 90% is idle.

This is a "top" screen while running the query:

2:27pm up 5 min, 2 users, load average: 0,80, 0,26, 0,08
74 processes: 72 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 3,1% user, 2,1% system, 0,0% nice, 94,6% idle
Mem: 1031012K av, 246040K used, 784972K free, 0K shrd, 11496K
buff
Swap: 2040244K av, 0K used, 2040244K free 196916K
cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1165 postgres 0 -20 3476 3476 2912 D < 4,5 0,3 0:02 postmaster
268 root 15 0 0 0 0 SW 0,1 0,0 0:00 kjournald

Any idea of how can I get postmaster to eat as much CPU as possible?

Looks like you're waiting on I/O. What's your disk subsystem
configuration?

#7Ruben
ruben12@superguai.com
In reply to: Ruben (#1)
Re: Postmaster only takes 4-5% CPU

Thanks for your answers Jan, Arjen, Shridar and Scott:

Postmaster is back taking lots of CPU when needed. I guess you were
right about Postgres waiting for some I/O activity. Anyway, I was not
able to see what process was responsible for this, since top showed
nothing remarkable.

This is a new top while running a VACUUM of the database:

8:51pm up 6:30, 3 users, load average: 0,45, 0,11, 0,04
93 processes: 89 sleeping, 4 running, 0 zombie, 0 stopped
CPU states: 0,1% user, 1,1% system, 0,0% nice, 98,6% idle
Mem: 1031012K av, 1019612K used, 11400K free, 0K shrd, 44896K buff
Swap: 2040244K av, 1716K used, 2038528K free 894008K cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1673 postgres 15 0 5120 5120 3196 R 0,9 0,4 0:02 postmaster
1683 root 15 0 1044 1044 840 R 0,1 0,1 0:00 top
1 root 15 0 476 476 424 S 0,0 0,0 0:04 init
2 root 15 0 0 0 0 SW 0,0 0,0 0:00 keventd
3 root 15 0 0 0 0 SW 0,0 0,0 0:00 kapmd
4 root 34 19 0 0 0 SWN 0,0 0,0 0:00
ksoftirqd_CPU0
5 root 15 0 0 0 0 SW 0,0 0,0 0:00 kswapd
6 root 15 0 0 0 0 SW 0,0 0,0 0:00 bdflush
7 root 15 0 0 0 0 SW 0,0 0,0 0:00 kupdated
8 root 25 0 0 0 0 SW 0,0 0,0 0:00 mdrecoveryd
12 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
66 root 15 0 0 0 0 SW 0,0 0,0 0:00 khubd
266 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
267 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
268 root 15 0 0 0 0 SW 0,0 0,0 0:05 kjournald
269 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
547 root 15 0 0 0 0 SW 0,0 0,0 0:00 eth0

Cheers, Ruben.

Jan Poslusny wrote:

Show quoted text

Did not your postmaster wait for harddisk ?

Ruben wrote:

Hi all:

When I run a (big) query from pgsql, postmaster used to take almost
100% of CPU. Now, with aparent no reason, Postmaster only takes 4-5%
CPU and the same query takes really longer to finish.

I have tried renicing -20 postmaster but it still only takes a small
percentage of CPU, while 90% is idle.

This is a "top" screen while running the query:

2:27pm up 5 min, 2 users, load average: 0,80, 0,26, 0,08
74 processes: 72 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 3,1% user, 2,1% system, 0,0% nice, 94,6% idle
Mem: 1031012K av, 246040K used, 784972K free, 0K shrd,
11496K buff
Swap: 2040244K av, 0K used, 2040244K free
196916K cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1165 postgres 0 -20 3476 3476 2912 D < 4,5 0,3 0:02 postmaster
268 root 15 0 0 0 0 SW 0,1 0,0 0:00 kjournald

Any idea of how can I get postmaster to eat as much CPU as possible?

Thanks a lot! Ruben.

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

#8scott.marlowe
scott.marlowe@ihs.com
In reply to: Ruben (#7)
Re: Postmaster only takes 4-5% CPU

Under some circumstances, on my (somewhat) older 7.2 box running the 2.4.9
kernel, the kswapd and bdflush daemons will start working overtime when
there are accesses by windows users via samba to look through large
directories.

This makes the box QUITE slow and pokey, and all the tasks that are
running besides kswapd / bdflush are in D state mostly, while the swap
daemon eats 0 to 100% CPU depending on some change in state in samba I've
not bothered tracking down.

I understand a newer kernel might be a good thing for me, so eventually
I'll do that. For now I just put the Windows Samba people on another box.
They both go through the the same NAS, and it keeps the load off the main
server.

So, look for other processes that might be doing something like scanning a
directory of 40,000 files or some such.

On Thu, 29 May 2003, Ruben wrote:

Show quoted text

Thanks for your answers Jan, Arjen, Shridar and Scott:

Postmaster is back taking lots of CPU when needed. I guess you were
right about Postgres waiting for some I/O activity. Anyway, I was not
able to see what process was responsible for this, since top showed
nothing remarkable.

This is a new top while running a VACUUM of the database:

8:51pm up 6:30, 3 users, load average: 0,45, 0,11, 0,04
93 processes: 89 sleeping, 4 running, 0 zombie, 0 stopped
CPU states: 0,1% user, 1,1% system, 0,0% nice, 98,6% idle
Mem: 1031012K av, 1019612K used, 11400K free, 0K shrd, 44896K buff
Swap: 2040244K av, 1716K used, 2038528K free 894008K cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1673 postgres 15 0 5120 5120 3196 R 0,9 0,4 0:02 postmaster
1683 root 15 0 1044 1044 840 R 0,1 0,1 0:00 top
1 root 15 0 476 476 424 S 0,0 0,0 0:04 init
2 root 15 0 0 0 0 SW 0,0 0,0 0:00 keventd
3 root 15 0 0 0 0 SW 0,0 0,0 0:00 kapmd
4 root 34 19 0 0 0 SWN 0,0 0,0 0:00
ksoftirqd_CPU0
5 root 15 0 0 0 0 SW 0,0 0,0 0:00 kswapd
6 root 15 0 0 0 0 SW 0,0 0,0 0:00 bdflush
7 root 15 0 0 0 0 SW 0,0 0,0 0:00 kupdated
8 root 25 0 0 0 0 SW 0,0 0,0 0:00 mdrecoveryd
12 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
66 root 15 0 0 0 0 SW 0,0 0,0 0:00 khubd
266 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
267 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
268 root 15 0 0 0 0 SW 0,0 0,0 0:05 kjournald
269 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
547 root 15 0 0 0 0 SW 0,0 0,0 0:00 eth0

Cheers, Ruben.

Jan Poslusny wrote:

Did not your postmaster wait for harddisk ?

Ruben wrote:

Hi all:

When I run a (big) query from pgsql, postmaster used to take almost
100% of CPU. Now, with aparent no reason, Postmaster only takes 4-5%
CPU and the same query takes really longer to finish.

I have tried renicing -20 postmaster but it still only takes a small
percentage of CPU, while 90% is idle.

This is a "top" screen while running the query:

2:27pm up 5 min, 2 users, load average: 0,80, 0,26, 0,08
74 processes: 72 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 3,1% user, 2,1% system, 0,0% nice, 94,6% idle
Mem: 1031012K av, 246040K used, 784972K free, 0K shrd,
11496K buff
Swap: 2040244K av, 0K used, 2040244K free
196916K cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1165 postgres 0 -20 3476 3476 2912 D < 4,5 0,3 0:02 postmaster
268 root 15 0 0 0 0 SW 0,1 0,0 0:00 kjournald

Any idea of how can I get postmaster to eat as much CPU as possible?

Thanks a lot! Ruben.

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

#9Ruben
ruben12@superguai.com
In reply to: scott.marlowe (#8)
Re: Postmaster only takes 4-5% CPU

Hi:

I'm sorry to get back with this, but I claimed victory too soon.
Postmaster is back (intermitently) taking just a small % of cpu while
system is almost 100% idle, as you can see in "top" below.

Any ideas of how can I fix this? Programs running queries are going too
slooooow.

Thanks!

11:21am up 7 days, 21:00, 6 users, load average: 1,10, 1,04, 0,86
106 processes: 102 sleeping, 4 running, 0 zombie, 0 stopped
CPU states: 0,5% user, 1,7% system, 0,0% nice, 97,6% idle
Mem: 1031012K av, 1019052K used, 11960K free, 0K shrd, 11340K buff
Swap: 2040244K av, 153828K used, 1886416K free 958512K cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
23057 postgres 15 0 3316 3136 2292 R 1,1 0,3 0:17 postmaster
23073 postgres 15 0 1096 1096 840 R 0,1 0,1 0:00 top
1 root 15 0 468 428 416 S 0,0 0,0 0:04 init
2 root 15 0 0 0 0 SW 0,0 0,0 0:00 keventd
3 root 15 0 0 0 0 SW 0,0 0,0 0:00 kapmd
4 root 34 19 0 0 0 SWN 0,0 0,0 0:00 ksoftirqd_C
5 root 15 0 0 0 0 SW 0,0 0,0 1:18 kswapd
6 root 15 0 0 0 0 SW 0,0 0,0 0:07 bdflush
7 root 15 0 0 0 0 SW 0,0 0,0 0:00 kupdated
8 root 25 0 0 0 0 SW 0,0 0,0 0:00 mdrecoveryd
12 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
66 root 15 0 0 0 0 SW 0,0 0,0 0:00 khubd
266 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
267 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
268 root 15 0 0 0 0 SW 0,0 0,0 0:29 kjournald
269 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
547 root 15 0 0 0 0 SW 0,0 0,0 0:00 eth0

scott.marlowe wrote:

Show quoted text

Under some circumstances, on my (somewhat) older 7.2 box running the 2.4.9
kernel, the kswapd and bdflush daemons will start working overtime when
there are accesses by windows users via samba to look through large
directories.

This makes the box QUITE slow and pokey, and all the tasks that are
running besides kswapd / bdflush are in D state mostly, while the swap
daemon eats 0 to 100% CPU depending on some change in state in samba I've
not bothered tracking down.

I understand a newer kernel might be a good thing for me, so eventually
I'll do that. For now I just put the Windows Samba people on another box.
They both go through the the same NAS, and it keeps the load off the main
server.

So, look for other processes that might be doing something like scanning a
directory of 40,000 files or some such.

On Thu, 29 May 2003, Ruben wrote:

Thanks for your answers Jan, Arjen, Shridar and Scott:

Postmaster is back taking lots of CPU when needed. I guess you were
right about Postgres waiting for some I/O activity. Anyway, I was not
able to see what process was responsible for this, since top showed
nothing remarkable.

This is a new top while running a VACUUM of the database:

8:51pm up 6:30, 3 users, load average: 0,45, 0,11, 0,04
93 processes: 89 sleeping, 4 running, 0 zombie, 0 stopped
CPU states: 0,1% user, 1,1% system, 0,0% nice, 98,6% idle
Mem: 1031012K av, 1019612K used, 11400K free, 0K shrd, 44896K buff
Swap: 2040244K av, 1716K used, 2038528K free 894008K cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1673 postgres 15 0 5120 5120 3196 R 0,9 0,4 0:02 postmaster
1683 root 15 0 1044 1044 840 R 0,1 0,1 0:00 top
1 root 15 0 476 476 424 S 0,0 0,0 0:04 init
2 root 15 0 0 0 0 SW 0,0 0,0 0:00 keventd
3 root 15 0 0 0 0 SW 0,0 0,0 0:00 kapmd
4 root 34 19 0 0 0 SWN 0,0 0,0 0:00
ksoftirqd_CPU0
5 root 15 0 0 0 0 SW 0,0 0,0 0:00 kswapd
6 root 15 0 0 0 0 SW 0,0 0,0 0:00 bdflush
7 root 15 0 0 0 0 SW 0,0 0,0 0:00 kupdated
8 root 25 0 0 0 0 SW 0,0 0,0 0:00 mdrecoveryd
12 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
66 root 15 0 0 0 0 SW 0,0 0,0 0:00 khubd
266 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
267 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
268 root 15 0 0 0 0 SW 0,0 0,0 0:05 kjournald
269 root 15 0 0 0 0 SW 0,0 0,0 0:00 kjournald
547 root 15 0 0 0 0 SW 0,0 0,0 0:00 eth0

Cheers, Ruben.

Jan Poslusny wrote:

Did not your postmaster wait for harddisk ?

Ruben wrote:

Hi all:

When I run a (big) query from pgsql, postmaster used to take almost
100% of CPU. Now, with aparent no reason, Postmaster only takes 4-5%
CPU and the same query takes really longer to finish.

I have tried renicing -20 postmaster but it still only takes a small
percentage of CPU, while 90% is idle.

This is a "top" screen while running the query:

2:27pm up 5 min, 2 users, load average: 0,80, 0,26, 0,08
74 processes: 72 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 3,1% user, 2,1% system, 0,0% nice, 94,6% idle
Mem: 1031012K av, 246040K used, 784972K free, 0K shrd,
11496K buff
Swap: 2040244K av, 0K used, 2040244K free
196916K cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1165 postgres 0 -20 3476 3476 2912 D < 4,5 0,3 0:02 postmaster
268 root 15 0 0 0 0 SW 0,1 0,0 0:00 kjournald

Any idea of how can I get postmaster to eat as much CPU as possible?

Thanks a lot! Ruben.

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

#10Jamie Lawrence
postgres@jal.org
In reply to: Ruben (#9)
Re: Postmaster only takes 4-5% CPU

On Fri, 06 Jun 2003, Ruben wrote:

Hi:

I'm sorry to get back with this, but I claimed victory too soon.
Postmaster is back (intermitently) taking just a small % of cpu while
system is almost 100% idle, as you can see in "top" below.

Any ideas of how can I fix this? Programs running queries are going too
slooooow.

11:21am up 7 days, 21:00, 6 users, load average: 1,10, 1,04, 0,86
106 processes: 102 sleeping, 4 running, 0 zombie, 0 stopped
CPU states: 0,5% user, 1,7% system, 0,0% nice, 97,6% idle
Mem: 1031012K av, 1019052K used, 11960K free, 0K shrd, 11340K buff
Swap: 2040244K av, 153828K used, 1886416K free 958512K cached

What does the output of 'vmstat 5 15' look like while you're running the
query? (that command will run for 75 seconds.)

I'm willing to bet your problem is I/O contention.

-j

--
Jamie Lawrence jal@jal.org
"Many and loud have been the voices crying that Aids was God's
way of punishing an unnatural practice. One day, perhaps, an
inoculation against HIV may be discovered. A bottle of champagne,
then, for whoever cites me evidence of one of those voices
crying that the breakthrough is God's way of telling us to
bugger each other."
- Matthew Parris

#11Ruben
ruben12@superguai.com
In reply to: scott.marlowe (#8)
Re: Postmaster only takes 4-5% CPU

Thanks Jamie:

I'll send the output of vmstat next time this happens.

Ruben.

Jamie Lawrence wrote:

Show quoted text

On Fri, 06 Jun 2003, Ruben wrote:

Hi:

I'm sorry to get back with this, but I claimed victory too soon.
Postmaster is back (intermitently) taking just a small % of cpu while
system is almost 100% idle, as you can see in "top" below.

Any ideas of how can I fix this? Programs running queries are going too
slooooow.

11:21am up 7 days, 21:00, 6 users, load average: 1,10, 1,04, 0,86
106 processes: 102 sleeping, 4 running, 0 zombie, 0 stopped
CPU states: 0,5% user, 1,7% system, 0,0% nice, 97,6% idle
Mem: 1031012K av, 1019052K used, 11960K free, 0K shrd, 11340K buff
Swap: 2040244K av, 153828K used, 1886416K free 958512K cached

What does the output of 'vmstat 5 15' look like while you're running the
query? (that command will run for 75 seconds.)

I'm willing to bet your problem is I/O contention.

-j