Re: Synchrnonized Scan test

Luke Lonergan wrote:

I've attached the results of a test with the first patch and the test. It
uses five simultaneous scans as before, but this time the table is 120GB on
a machine with 8GB of RAM. The data is stored on one non-raid disk, so a
single scan should take about 33 minutes.

The scans are started 5 minutes apart, so the last scan would end in about 3
hours if they were independent (they're not). 8.2 unmodified runs the test
in about 4 hours and 20 minutes. With the first patch, it runs in about 5
hours and 30 minutes.

The results show that the first patch actually slowed things down quite a
bit. I'll test the last patch you sent and see what we get.

I think this is worthwhile, you can definitely see a slowdown by having the
competing scans. The OS (Linux 2.6.18) is doing a pretty good job of
keeping the scans scheduled well, but we could still speed things up by
about 30% if we got syncscan working.

Thanks for running the tests.

I don't know exactly what's happening in your case, but it may have to
do with some unbalanced scheduling. What if we actually throttled each
"follower" to read no more than N pages before it task-switched? Might
that keep the scans closer together?

If you have any way of tracing what is actually happening, that would be
very helpful.

It would be nice to know:
(1) Are the scans diverging because some processes are running forward
with fast shared_buffers page hits, and maybe other processes
task-switch when they need to read() (even if it's in the OS buffer cache)?
(2) How much difference is there really between a shared_buffer cache
hit and an OS buffer cache hit?
(3) Does an OS buffer cache hit always cause a context switch? How often?
(4) Is read-ahead per-process (like Tom suggested), or is it system wide?

Regards,
Jeff Davis