Regression tests

Are there any regression tests or unit tests beyond 'make check', or
possibly benchmarks which not only measure performance but also verify
that the results are correct? I have patches which I want to test
under high load from multiple concurrent clients, so 'make check'
isn't enough. Google has tons of hits for articles and RDBMS reviews
that mention SuperSmack, but no hits for the actual software.

The whole point of make check is to check correctness, not performance.
It has concurrent loading as well.

Chris

Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:

The whole point of make check is to check correctness, not
performance.

I understand that.

It has concurrent loading as well.

It doesn't stress the system anywhere near enough to reveal bugs in,
say, the shared memory or semaphore code.

DES
--
Dag-Erling Smørgrav - des@des.no

Dag-Erling Sm�rgrav wrote:

It doesn't stress the system anywhere near enough to reveal bugs in,
say, the shared memory or semaphore code.

I agree -- I think we definitely need more tests for the concurrent
behavior of the system.

-Neil

On Wed, 2005-05-04 at 10:56 +1000, Neil Conway wrote:

Dag-Erling Smørgrav wrote:

It doesn't stress the system anywhere near enough to reveal bugs in,
say, the shared memory or semaphore code.

I agree -- I think we definitely need more tests for the concurrent
behavior of the system.

Yes, very much agree. Contributions in that area would be most welcome.

Best Regards, Simon Riggs

Neil Conway <neilc@samurai.com> writes:

Dag-Erling Smørgrav wrote:

It doesn't stress the system anywhere near enough to reveal bugs in,
say, the shared memory or semaphore code.

I agree -- I think we definitely need more tests for the concurrent
behavior of the system.

Quite, but in the meantime, a good benchmark should stress the system
enough to cause crashes, lockups or at least incorrect results if a
bug is introduced in the shared memory or semaphore code, and will
definitely reveal any slowdowns introduced by new code, so my question
is: where can I find a good benchmark for PostgreSQL? Note that I
don't care about comparing PostgreSQL to other RDBMSes; I just want to
a) test PostgreSQL under high concurrent load and b) if possible,
measure the performance impact of a patch.

DES
--
Dag-Erling Smørgrav - des@des.no

Quite, but in the meantime, a good benchmark should stress the system
enough to cause crashes, lockups or at least incorrect results if a
bug is introduced in the shared memory or semaphore code, and will
definitely reveal any slowdowns introduced by new code, so my question
is: where can I find a good benchmark for PostgreSQL? Note that I
don't care about comparing PostgreSQL to other RDBMSes; I just want to
a) test PostgreSQL under high concurrent load and b) if possible,
measure the performance impact of a patch.

You can use contrib/pgbench as a rather simplistic test, or you could
try OSDB (http://osdb.sourceforge.net/)

Chris

Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:

Quite, but in the meantime, a good benchmark should stress the system
enough to cause crashes, lockups or at least incorrect results if a
bug is introduced in the shared memory or semaphore code, and will
definitely reveal any slowdowns introduced by new code, so my question
is: where can I find a good benchmark for PostgreSQL? Note that I
don't care about comparing PostgreSQL to other RDBMSes; I just want to
a) test PostgreSQL under high concurrent load and b) if possible,
measure the performance impact of a patch.

You can use contrib/pgbench as a rather simplistic test, or you could
try OSDB (http://osdb.sourceforge.net/)

For something a little tougher, talk to Mark Wong about using OSDL's
testbed.

regards, tom lane

On Wednesday 04 May 2005 03:20, Christopher Kings-Lynne wrote:

Quite, but in the meantime, a good benchmark should stress the system
enough to cause crashes, lockups or at least incorrect results if a
bug is introduced in the shared memory or semaphore code, and will
definitely reveal any slowdowns introduced by new code, so my question
is: where can I find a good benchmark for PostgreSQL? Note that I
don't care about comparing PostgreSQL to other RDBMSes; I just want to
a) test PostgreSQL under high concurrent load and b) if possible,
measure the performance impact of a patch.

You can use contrib/pgbench as a rather simplistic test, or you could
try OSDB (http://osdb.sourceforge.net/)

Shouldn't we be getting some of this testing out of the work being done by
Mark Wong and the OSDN folks?

--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL