Asking for feedback on Pgperffarm

Hi Yedil,

On Mon, Aug 08, 2022 at 02:50:17PM +0200, Yedil Serzhan wrote:

Dear hackers,

I'm Yedil. I'm working on the project "Postgres Performance Farm" during
Gsoc. Pgperffarm is a project like Postgres build farm but focuses on the
performance of the database. Now it has 2 types of benchmarks, pgbench and
tpc-h. The website is online here <http://140.211.168.145/&gt;, and the repo
is here <https://github.com/PGPerfFarm/pgperffarm_server&gt;.

I would like you to take a look at our website and, if possible, give some
feedback on, for example, what other data should be collected or what other
metrics could be used to compare performance.

Nice work!

We need to be careful with how results based on the TPC-H specification
are presented. It needs to be changed, but maybe not dramatically.
Something like "Fair use derivation of TPC-H". It needs to be clear
that it's not an official TPC-H result.

I think I've hinted at it in the #perffarm slack channel, that I think
it would be better if you leveraged one of the already existing TPC-H
derived kits. While I'm partial to dbt-3, because I'm trying to
maintain it and because it sounded like you were starting to do
something similar to that, I think you can save a good amount of effort
from reimplementing another kit from scratch.

Regards,
Mark

--
Mark Wong
EDB https://enterprisedb.com

Hi, Mark, really thank you for your feedback.

On Mon, Aug 8, 2022 at 7:06 PM Mark Wong <markwkm@gmail.com> wrote:

Hi Yedil,

On Mon, Aug 08, 2022 at 02:50:17PM +0200, Yedil Serzhan wrote:

Dear hackers,

I'm Yedil. I'm working on the project "Postgres Performance Farm" during
Gsoc. Pgperffarm is a project like Postgres build farm but focuses on the
performance of the database. Now it has 2 types of benchmarks, pgbench

and

tpc-h. The website is online here <http://140.211.168.145/&gt;, and the

repo

is here <https://github.com/PGPerfFarm/pgperffarm_server&gt;.

I would like you to take a look at our website and, if possible, give

some

feedback on, for example, what other data should be collected or what

other

metrics could be used to compare performance.

Nice work!

We need to be careful with how results based on the TPC-H specification
are presented. It needs to be changed, but maybe not dramatically.
Something like "Fair use derivation of TPC-H". It needs to be clear
that it's not an official TPC-H result.

I think I've hinted at it in the #perffarm slack channel, that I think
it would be better if you leveraged one of the already existing TPC-H
derived kits. While I'm partial to dbt-3, because I'm trying to
maintain it and because it sounded like you were starting to do
something similar to that, I think you can save a good amount of effort
from reimplementing another kit from scratch.

Regards,
Mark

It makes sense to put it as a "fair use derivation of TPC-H". I also used
the term "composite score" because of your previous feedback on it.

I'll also check out the dbt-3 tool and if the effort is worth it, and if
it's necessary, I'll try to switch to it.

These are very valuable feedback, thank you again.

Best,
Yedil