Yet another open-source benchmark

Started by Tom Lanealmost 23 years ago7 messages
#1Tom Lane
tgl@sss.pgh.pa.us

OSDL has just come out with a set of open-source database benchmarks:
http://www.osdl.org/projects/performance/

The bad news:
"This tool kit works with SAP DB open source database versions 7.3.0.23
or 7.3.0.25."

(In fact, they seem to think they are testing kernel performance, not
database performance, which strikes me as rather bizarre. But anyway.)

The good news:
"We are planning to port this test kit to other databases."

Perhaps someone around here should help out...

regards, tom lane

#2Justin Clift
justin@postgresql.org
In reply to: Tom Lane (#1)
Re: Yet another open-source benchmark

Tom Lane wrote:

OSDL has just come out with a set of open-source database benchmarks:
http://www.osdl.org/projects/performance/

The bad news:
"This tool kit works with SAP DB open source database versions 7.3.0.23
or 7.3.0.25."

(In fact, they seem to think they are testing kernel performance, not
database performance, which strikes me as rather bizarre. But anyway.)

The good news:
"We are planning to port this test kit to other databases."

Perhaps someone around here should help out...

Yep, this is the group that have hit a performance limit with SAPDB and
are 100% definitely looking to move it to PostgreSQL, *if* they can get
people to assist them.

:-)

Regards and best wishes,

Justin Clift

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

--
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
- Indira Gandhi

#3Noname
cbbrowne@cbbrowne.com
In reply to: Tom Lane (#1)
Re: Yet another open-source benchmark

OSDL has just come out with a set of open-source database benchmarks:
http://www.osdl.org/projects/performance/

The bad news:
"This tool kit works with SAP DB open source database versions 7.3.0.23
or 7.3.0.25."

(In fact, they seem to think they are testing kernel performance, not
database performance, which strikes me as rather bizarre. But anyway.)

That may be a terminology thing; the main SAP-DB process is called the
"kernel," and it's more than likely that the "SAP-DB Kernel" is the sense in
which the term is being used.

When they translate things from German, sometimes wordings change :-).
--
output = reverse("moc.enworbbc@" "enworbbc")
http://www.ntlug.org/~cbbrowne/linuxxian.html
Rules of the Evil Overlord #41. "Once my power is secure, I will
destroy all those pesky time-travel devices."
<http://www.eviloverlord.com/&gt;

#4Tommi Maekitalo
t.maekitalo@epgmbh.de
In reply to: Noname (#3)
Re: Yet another open-source benchmark

On the results page they list kernels like linux-2.4.18-1tier or
linux-2.4.19-rc2 or redhat-stock-2.4.7-10cmp. This sounds really like
linux-kernel-versions.

Am Montag, 3. März 2003 13:41 schrieb cbbrowne@cbbrowne.com:

OSDL has just come out with a set of open-source database benchmarks:
http://www.osdl.org/projects/performance/

The bad news:
"This tool kit works with SAP DB open source database versions 7.3.0.23
or 7.3.0.25."

(In fact, they seem to think they are testing kernel performance, not
database performance, which strikes me as rather bizarre. But anyway.)

That may be a terminology thing; the main SAP-DB process is called the
"kernel," and it's more than likely that the "SAP-DB Kernel" is the sense
in which the term is being used.

When they translate things from German, sometimes wordings change :-).
--
output = reverse("moc.enworbbc@" "enworbbc")
http://www.ntlug.org/~cbbrowne/linuxxian.html
Rules of the Evil Overlord #41. "Once my power is secure, I will
destroy all those pesky time-travel devices."
<http://www.eviloverlord.com/&gt;

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

--
Dr. Eckhardt + Partner GmbH
http://www.epgmbh.de

#5scott.marlowe
scott.marlowe@ihs.com
In reply to: Tommi Maekitalo (#4)
Re: Yet another open-source benchmark

On Mon, 3 Mar 2003, Tommi Maekitalo wrote:

On the results page they list kernels like linux-2.4.18-1tier or
linux-2.4.19-rc2 or redhat-stock-2.4.7-10cmp. This sounds really like
linux-kernel-versions.

Am Montag, 3. März 2003 13:41 schrieb cbbrowne@cbbrowne.com:

OSDL has just come out with a set of open-source database benchmarks:
http://www.osdl.org/projects/performance/

The bad news:
"This tool kit works with SAP DB open source database versions 7.3.0.23
or 7.3.0.25."

(In fact, they seem to think they are testing kernel performance, not
database performance, which strikes me as rather bizarre. But anyway.)

That may be a terminology thing; the main SAP-DB process is called the
"kernel," and it's more than likely that the "SAP-DB Kernel" is the sense
in which the term is being used.

When they translate things from German, sometimes wordings change :-).
--
output = reverse("moc.enworbbc@" "enworbbc")
http://www.ntlug.org/~cbbrowne/linuxxian.html
Rules of the Evil Overlord #41. "Once my power is secure, I will
destroy all those pesky time-travel devices."
<http://www.eviloverlord.com/&gt;

I think they are testing how tuning the linux kernel impacts the database
running on top, at least that's the feeling I got from the site.

#6Neil Conway
neilc@samurai.com
In reply to: Noname (#3)
Re: Yet another open-source benchmark

On Mon, 2003-03-03 at 07:41, cbbrowne@cbbrowne.com wrote:

(In fact, they seem to think they are testing kernel performance, not
database performance, which strikes me as rather bizarre. But anyway.)

That may be a terminology thing; the main SAP-DB process is called the
"kernel," and it's more than likely that the "SAP-DB Kernel" is the sense in
which the term is being used.

Actually, I believe the reason the benchmark was developed was to
provide a workload for optimizing high-end Linux kernel performance
(with the inference being that SAP-DB is pretty close to Oracle, Oracle
performance is important for enterprise deployment of Linux, and
therefore optimizing the kernel's handling of SAP-DB running TPC
benchmarks will tend to improve the kernel's performance running
Oracle/DB2/etc.) So when they mean "kernel", I think they really mean
"kernel".

That's not to say that the benchmark wouldn't be useful for doing other
stuff, like pure database benchmarks (as long as its a valid
implementation of TPC-C (or TPC-H, etc.), it should be fine...)

A research group at the university I attend (www.queensu.ca) expressed
some interested in a TPC-C implementation for PostgreSQL, so I was
planning to port the OSDL TPC-C implementation to PostgreSQL.
Unfortunately, I got sidetracked for a couple reasons: (1) lack of time
(2) increasing awareness of just how boring writing benchmark apps is
:-) (3) distaste for ODBC. While I'd like to get some time to do the
port in the future, that shouldn't stop anyone else from doing so in the
mean time :-)

Cheers,

Neil
--
Neil Conway <neilc@samurai.com> || PGP Key ID: DB3C29FC

#7Mark Wong
markw@osdl.org
In reply to: Neil Conway (#6)
Re: Yet another open-source benchmark

On Mon, 2003-03-03 at 12:29, Neil Conway wrote:

On Mon, 2003-03-03 at 07:41, cbbrowne@cbbrowne.com wrote:

(In fact, they seem to think they are testing kernel performance, not
database performance, which strikes me as rather bizarre. But anyway.)

That may be a terminology thing; the main SAP-DB process is called the
"kernel," and it's more than likely that the "SAP-DB Kernel" is the sense in
which the term is being used.

Actually, I believe the reason the benchmark was developed was to
provide a workload for optimizing high-end Linux kernel performance
(with the inference being that SAP-DB is pretty close to Oracle, Oracle
performance is important for enterprise deployment of Linux, and
therefore optimizing the kernel's handling of SAP-DB running TPC
benchmarks will tend to improve the kernel's performance running
Oracle/DB2/etc.) So when they mean "kernel", I think they really mean
"kernel".

Yeah, Neil more-or-less hit it on the nose. The SAP DB folks do refer
to their processes as kernel processes, but our focus is on the Linux
kernel and helping Linux gain more ground for the enterprise.

That's not to say that the benchmark wouldn't be useful for doing other
stuff, like pure database benchmarks (as long as its a valid
implementation of TPC-C (or TPC-H, etc.), it should be fine...)

A research group at the university I attend (www.queensu.ca) expressed
some interested in a TPC-C implementation for PostgreSQL, so I was
planning to port the OSDL TPC-C implementation to PostgreSQL.
Unfortunately, I got sidetracked for a couple reasons: (1) lack of time
(2) increasing awareness of just how boring writing benchmark apps is
:-) (3) distaste for ODBC. While I'd like to get some time to do the
port in the future, that shouldn't stop anyone else from doing so in the
mean time :-)

And we're prepared to aid any effort. :)

Cheers,

Neil
--
Neil Conway <neilc@samurai.com> || PGP Key ID: DB3C29FC

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

--
Mark Wong - - markw@osdl.org
Open Source Development Lab Inc - A non-profit corporation
15275 SW Koll Parkway - Suite H - Beaverton OR, 97006
(503)-626-2455 x 32 (office)
(503)-626-2436 (fax)
http://www.osdl.org/archive/markw/