[PATCH] Refactor pgbench to make future improvements easier

Started by Hannu Krosingabout 2 months ago14 messageshackers
Jump to latest
#1Hannu Krosing
hannuk@google.com

Hi hackers,

`src/bin/pgbench/pgbench.c` has grown into a ~8,150-line monolithic source
file combining benchmark statistics, variable management, socket event
multiplexing, script/expression parsing and evaluation, meta-command
handlers, schema/data initialization, and the multi-threaded client state
machine.

This 7-patch series splits `src/bin/pgbench/pgbench.c` into focused,
self-contained compilation units while preserving exact behavior and all
existing tests:

1. `stats.c/h`: Extract `SimpleStats` and `StatsData` structures, latency
and retry accumulation, and formatting helpers.
2. `variable.c/h`: Extract `PgBenchValue`, `Variable`, and `Variables`
management, binary-search variable lookup, type coercion, and SQL query
variable substitution (`assignVariables` / `getQueryParams`).
3. `poller.c/h`: Extract the `socket_set` abstraction and platform-specific
`ppoll(2)` / `select(2)` socket multiplexing implementation.
4. `script.c/h`: Extract built-in script definitions, SQL and backslash
meta-command parsing, Bison/Flex expression parser callbacks, and
runtime expression evaluation (`evaluateExpr`, random distributions,
hashing, and `permute`).
5. `commands.c/h`: Extract runtime meta-command execution (`executeMetaCommand`,
`\set`, `\sleep`, `\if`/`\elif`/`\else`/`\endif`, `\shell`/`\setshell`),
`\gset`/`\aset` result processing, and pipeline command preparation.
6. `init.c/h`: Extract `-i` initialization routines (`runInitSteps`), DDL
table/index/partition creation, client-side and server-side data
generation, and vacuum helpers.
7. `engine.c/h`: Extract the client connection state machine
(`advanceConnectionState`), error/retry handling, worker thread loop
(`threadRun`), progress/log reporting, and final results reporting,
leaving `pgbench.c` (~978 lines) dedicated to CLI option parsing (`main`
and `usage`).

Each step builds cleanly with both Make and Meson and passes all 686 TAP
tests in `src/bin/pgbench`.

This modularization also prepares a clean boundary for extracting pgbench's
random number distributions, hashing, and permutation routines into
`src/common` (`pgbench_funcs.c/h`) and exposing them in SQL via
`contrib/pgbench`.

Hannu Krosing (7):
Refactor pgbench: extract statistics and latency tracking into
stats.c/h
Refactor pgbench: extract scoped variable store and value evaluation
into variable.c/h
Refactor pgbench: extract socket multiplexing and event poller into
poller.c/h
Refactor pgbench: extract script parsing and expression evaluation
into script.c/h
Refactor pgbench: extract meta-command execution and handlers into
commands.c/h
Refactor pgbench: extract initialization, schema, and data generators
into init.c/h
Refactor pgbench: extract execution engine, state machine, and thread
runner into engine.c/h

src/bin/pgbench/Makefile | 9 +-
src/bin/pgbench/commands.c | 737 ++++
src/bin/pgbench/commands.h | 70 +
src/bin/pgbench/engine.c | 2141 +++++++++++
src/bin/pgbench/engine.h | 161 +
src/bin/pgbench/init.c | 786 ++++
src/bin/pgbench/init.h | 66 +
src/bin/pgbench/meson.build | 7 +
src/bin/pgbench/pgbench.c | 7170 +----------------------------------
src/bin/pgbench/pgbench.h | 218 +-
src/bin/pgbench/poller.c | 196 +
src/bin/pgbench/poller.h | 48 +
src/bin/pgbench/script.c | 1973 ++++++++++
src/bin/pgbench/script.h | 253 ++
src/bin/pgbench/stats.c | 203 +
src/bin/pgbench/stats.h | 172 +
src/bin/pgbench/variable.c | 834 ++++
src/bin/pgbench/variable.h | 130 +
18 files changed, 7902 insertions(+), 7272 deletions(-)
create mode 100644 src/bin/pgbench/commands.c
create mode 100644 src/bin/pgbench/commands.h
create mode 100644 src/bin/pgbench/engine.c
create mode 100644 src/bin/pgbench/engine.h
create mode 100644 src/bin/pgbench/init.c
create mode 100644 src/bin/pgbench/init.h
create mode 100644 src/bin/pgbench/poller.c
create mode 100644 src/bin/pgbench/poller.h
create mode 100644 src/bin/pgbench/script.c
create mode 100644 src/bin/pgbench/script.h
create mode 100644 src/bin/pgbench/stats.c
create mode 100644 src/bin/pgbench/stats.h
create mode 100644 src/bin/pgbench/variable.c
create mode 100644 src/bin/pgbench/variable.h

--
2.56.0.rc1.315.gc6ed9934b7-goog

Attachments:

t254037_1
v2-0001-Refactor-pgbench-extract-statistics-and-latency-t.patchapplication/x-patch; name=v2-0001-Refactor-pgbench-extract-statistics-and-latency-t.patchDownload+385-329
v2-0004-Refactor-pgbench-extract-script-parsing-and-expre.patchapplication/x-patch; name=v2-0004-Refactor-pgbench-extract-script-parsing-and-expre.patchDownload+4353-4375
v2-0002-Refactor-pgbench-extract-scoped-variable-store-an.patchapplication/x-patch; name=v2-0002-Refactor-pgbench-extract-scoped-variable-store-an.patchDownload+970-700
v2-0007-Refactor-pgbench-extract-execution-engine-state-m.patchapplication/x-patch; name=v2-0007-Refactor-pgbench-extract-execution-engine-state-m.patchDownload+2395-2321
v2-0006-Refactor-pgbench-extract-initialization-schema-an.patchapplication/x-patch; name=v2-0006-Refactor-pgbench-extract-initialization-schema-an.patchDownload+859-817
v2-0003-Refactor-pgbench-extract-socket-multiplexing-and-.patchapplication/x-patch; name=v2-0003-Refactor-pgbench-extract-socket-multiplexing-and-.patchDownload+248-224
v2-0005-Refactor-pgbench-extract-meta-command-execution-a.patchapplication/x-patch; name=v2-0005-Refactor-pgbench-extract-meta-command-execution-a.patchDownload+818-638
#2Zsolt Parragi
zsolt.parragi@percona.com
In reply to: Hannu Krosing (#1)
Re: [PATCH] Refactor pgbench to make future improvements easier

Hello!

From the email explaining that this is a split, and the commit messages saying "extract" I expected a move-only patchset that strictly extracts part of the code into different files/modules. But that doesn't seem to be the case?

For example "executeMetaCommand" looks completely different in the end, chooseScript and other functions got signature changes, the number of comment lines got reduces by ~25%, a VariableScopeStack typedef is introduced that isn't used anywhere, etc.

I think for something like this to be easily reviewable, moves and logic changes should be strictly separate patches, not mixed together, and anything that's not a simple cut-and-paste into another file should be mentioned. In the current patchset, 0007 seems to be the closest to a pure move, but even that isn't just that. (When I am doing something similar, I usually follow a one refactoring - one commit/patch approach during the review, only squashing things together later)

I also checked the commit history of pgbench, it seems to get around ~4 backpatched commits per year, so that doesn't seem to be that bad through a refactoring.

#3Hannu Krosing
hannuk@google.com
In reply to: Zsolt Parragi (#2)
Re: [PATCH] Refactor pgbench to make future improvements easier

Hi Zsolt,

Thanks for reviewing this

Sorry for leaving more changes in than absolutely needed.

This was extracted back from a more invasive set of patches that

* extracts the various random distributions and other functions from
pgbench language and making them available as an in-database extension
https://commitfest.postgresql.org/patch/7225/

* adds a few more functions and language constructs to make it easier
to write TPC-* - like benchmarks

Would it make it easier for you to review if I reworked this set to
strictly move functions between source files, or are you still able to
review this as it is?

Best Regards

Hannu

Show quoted text

On Sun, Oct 4, 2026 at 12:20 AM Zsolt Parragi <zsolt.parragi@percona.com> wrote:

Hello!

From the email explaining that this is a split, and the commit
messages saying "extract" I expected a move-only patchset that
strictly extracts part of the code into different files/modules. But
that doesn't seem to be the case?

For example "executeMetaCommand" looks completely different in the
end, chooseScript and other functions got signature changes, the
number of comment lines got reduces by ~25%, a VariableScopeStack
typedef is introduced that isn't used anywhere, etc.

I think for something like this to be easily reviewable, moves and
logic changes should be strictly separate patches, not mixed together,
and anything that's not a simple cut-and-paste into another file
should be mentioned. In the current patchset, 0007 seems to be the
closest to a pure move, but even that isn't just that. (When I am
doing something similar, I usually follow a one refactoring - one
commit/patch approach during the review, only squashing things
together later)

I also checked the commit history of pgbench, it seems to get around
~4 backpatched commits per year, so that doesn't seem to be that bad
through a refactoring.

#4Hannu Krosing
hannuk@google.com
In reply to: Hannu Krosing (#3)
Re: [PATCH] Refactor pgbench to make future improvements easier

The prliminary gap analysis for what is needed in pgbench to support
TPC and CH-benCHmark style workloads is in
https://wiki.postgresql.org/wiki/Pgbench-for-tpc-like-benchmarks

Show quoted text

On Sun, Oct 4, 2026 at 11:17 AM Hannu Krosing <hannuk@google.com> wrote:

Hi Zsolt,

Thanks for reviewing this

Sorry for leaving more changes in than absolutely needed.

This was extracted back from a more invasive set of patches that

* extracts the various random distributions and other functions from
pgbench language and making them available as an in-database extension
https://commitfest.postgresql.org/patch/7225/

* adds a few more functions and language constructs to make it easier
to write TPC-* - like benchmarks

Would it make it easier for you to review if I reworked this set to
strictly move functions between source files, or are you still able to
review this as it is?

Best Regards

Hannu

On Sun, Oct 4, 2026 at 12:20 AM Zsolt Parragi <zsolt.parragi@percona.com> wrote:

Hello!

From the email explaining that this is a split, and the commit
messages saying "extract" I expected a move-only patchset that
strictly extracts part of the code into different files/modules. But
that doesn't seem to be the case?

For example "executeMetaCommand" looks completely different in the
end, chooseScript and other functions got signature changes, the
number of comment lines got reduces by ~25%, a VariableScopeStack
typedef is introduced that isn't used anywhere, etc.

I think for something like this to be easily reviewable, moves and
logic changes should be strictly separate patches, not mixed together,
and anything that's not a simple cut-and-paste into another file
should be mentioned. In the current patchset, 0007 seems to be the
closest to a pure move, but even that isn't just that. (When I am
doing something similar, I usually follow a one refactoring - one
commit/patch approach during the review, only squashing things
together later)

I also checked the commit history of pgbench, it seems to get around
~4 backpatched commits per year, so that doesn't seem to be that bad
through a refactoring.

#5Hannu Krosing
hannuk@google.com
In reply to: Hannu Krosing (#4)
Re: [PATCH] Refactor pgbench to make future improvements easier

From 2164409cb62f3dc3904822f56a8cc7cd3a4e4e18 Mon Sep 17 00:00:00 2001
From: Hannu Krosing <hannuk@google.com>
Date: Sun, 11 Oct 2026 13:26:28 +0000
Subject: [PATCH v4 00/12] Modularize pgbench: pure code-movement extraction
(0001-0007) followed by API/encapsulation changes (0008-0012)

Hi Zolt, hackers,

This v4 patch series restructures the modularization of
`src/bin/pgbench/pgbench.c`
(~8,080 lines on `master` at `173cf85a079`) into two strictly separated
phases on
branch `pgbenc-refactor-obly`:

1. **Patches 0001-0007 (Pure Code Movement + Strictly Required Linkage
Adjustments)**:
Extracts `stats.c/h`, `variable.c/h`, `poller.c/h`, `script.c/h`,
`commands.c/h`,
`init.c/h`, and `engine.c/h` from `pgbench.c`.
- Zero function bodies, expressions, statements, parameter lists, return
types,
or comments are modified in patches 0001-0007.
- All 136 function definitions in `master:src/bin/pgbench/pgbench.c` (129
unique
functions, plus 7 `#ifdef`-paired definitions for the 6 `poller.c`
functions and
`setalarm`) are preserved verbatim at commit 0007.
- The **only** changes to any function or variable declaration in 0001-0007
are the
C linkage adjustments strictly required for multi-file compilation:
a) Removing the `static` storage-class specifier from the **54 unique
functions**
(61 definitions counting `#ifdef` branches) and the shared global variables
that are defined in one `.c` file and referenced from another `.c`
translation
unit, and declaring `extern` prototypes for them in the module headers. In C
(ISO/IEC 9899:1999 6.2.2), `static` gives an identifier internal linkage
visible only within a single translation unit; once caller and callee live
in
separate `.c` files, external linkage is mandatory or linking fails with
undefined symbol errors (and `-Wall -Wunused-function` warns in the defining
translation unit).
b) Moving the two `static inline` timing helpers (`pg_time_now` and
`pg_time_now_lazy`) verbatim into `stats.h`. Under C99 6.7.4, an `inline`
function with internal linkage (`static inline`) must have its definition
visible in every translation unit that calls it (`script.c`, `commands.c`,
`init.c`, `engine.c`, and `pgbench.c`).
c) Keeping `static` on all **69 internal helper functions** that are only
ever
called within their own `.c` file.

2. **Patches 0008-0012 (Functional, Encapsulation, and API Changes)**:
Applies the functional/API improvements in separate commits after the pure
extraction:
- **0008 (stats.c/h)**: Adds `mergeStats()` to deduplicate per-thread
`StatsData`
accumulation in `printProgressReport()` and `main()`, decouples `stats.c`
from
the global `throttle_delay` and `failures_detailed` variables via explicit
parameters on `accumStats()` and `getResultString()`, and adds `const`
qualifiers
to `mergeSimpleStats()` and `printSimpleStats()`.
- **0009 (variable.c/h)**: Upgrades `Variables` into a lexical scope stack
(`VariableScope` / `VariableScopeStack`), adds lifecycle and scope
management
functions (`initVariables()`, `destroyVariables()`, `copyVariables()`,
`var_scope_push()`, `var_scope_pop()`, `var_put_value()`,
`var_get_value()`),
exports `lookupCreateVariable()`, `is_an_int()`, and `valueTypeName()`, adds
`const` qualifiers, and replaces the manual startup variable copy loop in
`main()` with `copyVariables()`.
- **0010 (poller.c/h)**: Moves the platform `#ifdef`s (`POLL_USING_PPOLL` /
`POLL_USING_SELECT`) and `struct socket_set` definitions from `poller.h`
into
`poller.c` so `socket_set` is an opaque type, adds
`socket_wait_method_name()`,
and adds the `PgBenchPoller` typedef and `poller_*` macro aliases.
- **0011 (script.c/h, engine.c/h)**: Changes `chooseScript(TState *thread)`
to
`chooseScript(pg_prng_state *random_state)` so `script.h` no longer depends
on
`TState`.
- **0012 (commands.c/h, engine.c)**: Introduces `CommandHandler` and
`CommandDescriptor` (`command_handlers[]`), decomposes
`executeMetaCommand()`
into `handle_cmd_*` functions, and extracts `skipConditionalCommands()` and
`processGSetResult()` from `engine.c` into `commands.c/h`.

Changes in v4:
- Rebased onto latest `master` (`173cf85a079`).
- Added `#include <unistd.h>` to `src/bin/pgbench/init.c` (commit 0006) for
`isatty()`,
fixing the `macOS - Meson` build.

Hannu Krosing (12):
Refactor pgbench: extract statistics structures and functions to
stats.c/h
Refactor pgbench: extract variable management and value types to
variable.c/h
Refactor pgbench: extract socket polling abstraction to poller.c/h
Refactor pgbench: extract script parsing, built-in scripts, and
expression evaluation to script.c/h
Refactor pgbench: extract meta-command execution and prepared
statement helpers to commands.c/h
Refactor pgbench: extract database initialization (-i) to init.c/h
Refactor pgbench: extract benchmark execution engine and client state
machine to engine.c/h
Refactor pgbench stats: add mergeStats() and decouple global flags
Refactor pgbench variables: introduce scoped variable store
Refactor pgbench poller: encapsulate socket_set and add poller API
aliases
Refactor pgbench script: decouple chooseScript() from TState
Refactor pgbench commands: introduce pluggable meta-command dispatch

src/bin/pgbench/Makefile | 7 +
src/bin/pgbench/commands.c | 734 ++++
src/bin/pgbench/commands.h | 70 +
src/bin/pgbench/engine.c | 2159 +++++++++++
src/bin/pgbench/engine.h | 159 +
src/bin/pgbench/init.c | 828 ++++
src/bin/pgbench/init.h | 51 +
src/bin/pgbench/meson.build | 7 +
src/bin/pgbench/pgbench.c | 7167 +----------------------------------
src/bin/pgbench/pgbench.h | 193 +-
src/bin/pgbench/poller.c | 242 ++
src/bin/pgbench/poller.h | 46 +
src/bin/pgbench/script.c | 1946 ++++++++++
src/bin/pgbench/script.h | 170 +
src/bin/pgbench/stats.c | 199 +
src/bin/pgbench/stats.h | 164 +
src/bin/pgbench/variable.c | 842 ++++
src/bin/pgbench/variable.h | 128 +
18 files changed, 7953 insertions(+), 7159 deletions(-)
create mode 100644 src/bin/pgbench/commands.c
create mode 100644 src/bin/pgbench/commands.h
create mode 100644 src/bin/pgbench/engine.c
create mode 100644 src/bin/pgbench/engine.h
create mode 100644 src/bin/pgbench/init.c
create mode 100644 src/bin/pgbench/init.h
create mode 100644 src/bin/pgbench/poller.c
create mode 100644 src/bin/pgbench/poller.h
create mode 100644 src/bin/pgbench/script.c
create mode 100644 src/bin/pgbench/script.h
create mode 100644 src/bin/pgbench/stats.c
create mode 100644 src/bin/pgbench/stats.h
create mode 100644 src/bin/pgbench/variable.c
create mode 100644 src/bin/pgbench/variable.h

base-commit: 173cf85a079516fdd2b09ed853702b05fbfcd15e
--
2.56.0.385.gd3acb90ef8-goog

On Sun, Oct 4, 2026 at 12:32 PM Hannu Krosing <hannuk@google.com> wrote:

Show quoted text

The prliminary gap analysis for what is needed in pgbench to support
TPC and CH-benCHmark style workloads is in
https://wiki.postgresql.org/wiki/Pgbench-for-tpc-like-benchmarks

On Sun, Oct 4, 2026 at 11:17 AM Hannu Krosing <hannuk@google.com> wrote:

Hi Zsolt,

Thanks for reviewing this

Sorry for leaving more changes in than absolutely needed.

This was extracted back from a more invasive set of patches that

* extracts the various random distributions and other functions from
pgbench language and making them available as an in-database extension
https://commitfest.postgresql.org/patch/7225/

* adds a few more functions and language constructs to make it easier
to write TPC-* - like benchmarks

Would it make it easier for you to review if I reworked this set to
strictly move functions between source files, or are you still able to
review this as it is?

Best Regards

Hannu

On Sun, Oct 4, 2026 at 12:20 AM Zsolt Parragi <zsolt.parragi@percona.com>

wrote:

Hello!

From the email explaining that this is a split, and the commit
messages saying "extract" I expected a move-only patchset that
strictly extracts part of the code into different files/modules. But
that doesn't seem to be the case?

For example "executeMetaCommand" looks completely different in the
end, chooseScript and other functions got signature changes, the
number of comment lines got reduces by ~25%, a VariableScopeStack
typedef is introduced that isn't used anywhere, etc.

I think for something like this to be easily reviewable, moves and
logic changes should be strictly separate patches, not mixed together,
and anything that's not a simple cut-and-paste into another file
should be mentioned. In the current patchset, 0007 seems to be the
closest to a pure move, but even that isn't just that. (When I am
doing something similar, I usually follow a one refactoring - one
commit/patch approach during the review, only squashing things
together later)

I also checked the commit history of pgbench, it seems to get around
~4 backpatched commits per year, so that doesn't seem to be that bad
through a refactoring.

Attachments:

t254037_5
v4-0003-Refactor-pgbench-extract-socket-polling-abstracti.patchapplication/x-patch; name=v4-0003-Refactor-pgbench-extract-socket-polling-abstracti.patchDownload+266-223
v4-0001-Refactor-pgbench-extract-statistics-structures-an.patchapplication/x-patch; name=v4-0001-Refactor-pgbench-extract-statistics-structures-an.patchDownload+353-297
v4-0002-Refactor-pgbench-extract-variable-management-and-.patchapplication/x-patch; name=v4-0002-Refactor-pgbench-extract-variable-management-and-.patchDownload+739-676
v4-0004-Refactor-pgbench-extract-script-parsing-built-in-.patchapplication/x-patch; name=v4-0004-Refactor-pgbench-extract-script-parsing-built-in-.patchDownload+4434-4357
v4-0005-Refactor-pgbench-extract-meta-command-execution-a.patchapplication/x-patch; name=v4-0005-Refactor-pgbench-extract-meta-command-execution-a.patchDownload+548-499
v4-0008-Refactor-pgbench-stats-add-mergeStats-and-decoupl.patchapplication/x-patch; name=v4-0008-Refactor-pgbench-stats-add-mergeStats-and-decoupl.patchDownload+35-41
v4-0010-Refactor-pgbench-poller-encapsulate-socket_set-an.patchapplication/x-patch; name=v4-0010-Refactor-pgbench-poller-encapsulate-socket_set-an.patchDownload+64-43
v4-0006-Refactor-pgbench-extract-database-initialization-.patchapplication/x-patch; name=v4-0006-Refactor-pgbench-extract-database-initialization-.patchDownload+887-821
v4-0009-Refactor-pgbench-variables-introduce-scoped-varia.patchapplication/x-patch; name=v4-0009-Refactor-pgbench-variables-introduce-scoped-varia.patchDownload+310-98
v4-0011-Refactor-pgbench-script-decouple-chooseScript-fro.patchapplication/x-patch; name=v4-0011-Refactor-pgbench-script-decouple-chooseScript-fro.patchDownload+4-9
v4-0012-Refactor-pgbench-commands-introduce-pluggable-met.patchapplication/x-patch; name=v4-0012-Refactor-pgbench-commands-introduce-pluggable-met.patchDownload+441-319
v4-0007-Refactor-pgbench-extract-benchmark-execution-engi.patchapplication/x-patch; name=v4-0007-Refactor-pgbench-extract-benchmark-execution-engi.patchDownload+2552-2484
#6Hannu Krosing
hannuk@google.com
In reply to: Hannu Krosing (#5)
Re: [PATCH] Refactor pgbench to make future improvements easier

[PATCH v5 00/22]
* Modular pgbench refactoring,
* contrib/pgbench extension,
* advanced benchmark capabilities,
* and standard workload suites (pgbench-b, pgbench-c, pgbench-e,
pgbench-h, pgbench-ds, pgbench-htap)

Hi hackers,

This patch series presents a comprehensive modernization of pgbench,
transforming
it from a monolithic single-script driver into a modern, modular, multi-workload
benchmarking engine with native support for complex OLTP, OLAP, and
HTAP benchmarks.

See https://wiki.postgresql.org/wiki/Pgbench-modernization-2026 for
more background and rationale.

The series is organized into five logical parts across 22 clean, self-contained
commits rebased cleanly on latest master:

================================================================================
Part 1: Modular pgbench Refactoring (Patches 0001 - 0012)
================================================================================
pgbench.c had grown to over 7,400 lines combining distinct responsibilities:
statistics accumulation, variable management, event polling, script parsing,
meta-command execution, database initialization (-i), and the multi-threaded
state machine.

Patches 0001-0007 extract these modules without altering behavior or
public APIs:
- stats.c / stats.h: StatsData, latency tally, rate-limiting logic.
- variable.c / variable.h: Scoped variable store and value evaluation.
- poller.c / poller.h: Socket set polling abstraction (select/poll).
- script.c / script.h: Script parsing, AST nodes, expression evaluator.
- commands.c / commands.h: Meta-command handlers and prepared
statement helpers.
- init.c / init.h: Database initialization (-i) logic and data generators.
- engine.c / engine.h: Multi-threaded client state machine and event loop.

Patches 0008-0012 decouple remaining global dependencies cleanly:
- stats: add mergeStats() and decouple global flags.
- variable: introduce scoped variable store (VariableSpace).
- poller: encapsulate socket_set and provide poller API wrappers.
- script: decouple chooseScript() from TState.
- commands: introduce pluggable meta-command dispatch table.

================================================================================
Part 2: Code Extraction & In-Database Functions (Patches 0013 - 0014)
================================================================================
Patch 0013 extracts pgbench's mathematical and pseudo-random
generators (Gaussian,
Exponential, Zipfian, Permute, MurmurHash64) into src/common/pgbench_funcs.c,
making them reusable across both client and server binaries.

Patch 0014 introduces contrib/pgbench, a server-side extension that
exposes these
pseudo-random generators and helper functions directly to SQL under
SCHEMA pgbench.
This allows high-performance parallel in-database synthetic data
generation using
standard SQL (e.g. generate_series()) without relying on slow client roundtrips.

================================================================================
Part 3: Advanced Types, Functions, Control Flow & Pacing (Patches 0015 - 0019)
================================================================================
Patch 0015 extends PgBenchValue with text, timestamp, and array types,
along with
array operations (array_append, array_length, array_get, array_generate), text
concatenation, and timestamp arithmetic.

Patch 0016 adds standard benchmark helper functions to pgbench expressions and
contrib/pgbench:
- nurand(A, s, x, y): Non-uniform random distribution generator.
- c_last(num): Syllable-based customer last name generator.
- random_astring(min, max), random_nstring(min, max): Alphanumeric
and numeric strings.
- format_string(fmt, ...), date_add(ts, amt, unit), int_to_date(int).

Patch 0017 adds script control flow and exception handling:
- Loops: \for <var> <start> .. <end> [BY <step>], \while <expr>,
\break, \continue.
- Exception handling: \try ... \catch [<sqlstate>] ... \endtry,
\expect_error <sqlstate>.

Patch 0018 adds result set metadata inspection and array column parsing:
- \gset_meta <prefix>: Extracts query result status, error code,
rows affected, and timing.
- \aset and array-aware \gset: Maps multi-row query results into
client-side array variables.

Patch 0019 introduces dynamic client pacing:
- Dynamic \sleep expressions (e.g. \sleep random_exponential(1, 1000, 10) ms).
- \keying_time and \think_time commands for realistic transaction
pacing per specification.

================================================================================
Part 4: Heterogeneous Groups, Latency Percentiles & Reporting (Patches
0020 - 0021)
================================================================================
Patch 0020 introduces heterogeneous client groups via --group:
- Run concurrent mixes of different scripts with distinct client
counts and rates
(e.g., --group name=oltp,clients=10,weight=100 --group
name=olap,clients=2,sequential=true).
- Supports sequential stream execution for multi-query analytical streams.

Patch 0021 implements High Dynamic Range (HDR) latency histogram tracking:
- Configurable percentiles (p50, p90, p95, p99, p99.9).
- SLA validation via --sla target=<metric>,threshold=<value>.
- Metric computation for tpmC and QphH@SF.
- Structured machine-readable JSON output via --json-report=<path>.

================================================================================
Part 5: Standard Workload Suites (Patch 0022)
================================================================================

<THIS WAS LEFT OUT FROM THIS EMAIL>

* Trademark & Naming Rationale:
These workloads are inspired by and modeled after industry-standard
specifications
(TPC-C, TPC-E, TPC-H, TPC-DS, and CH-benCHmark). To respect and
avoid infringing
upon Transaction Processing Performance Council (TPC) trademarks,
and to clearly
distinguish these community engineering tools from official
council-audited benchmarks,
they are explicitly named pgbench-b, pgbench-c, pgbench-e,
pgbench-h, pgbench-ds, and pgbench-htap.
Full background and guidelines are documented in
src/bin/pgbench/workloads/README.

* In-Database Parallel Loading:
Each workload includes an initialization script
(<workload>_init.sql) designed to
run in parallel across multiple worker sessions (-c N -j N -t 1 -D
num_workers=N).
Data generation is executed entirely within the database engine
using generate_series()
and contrib/pgbench SQL functions, achieving rapid parallel load
rates without external
file generation or ETL binaries.

Comprehensive documentation is provided in doc/src/sgml/ref/pgbench.sgml and
doc/src/sgml/pgbench-ext.sgml, and full end-to-end TAP coverage is added in
src/bin/pgbench/t/001_pgbench_with_server.pl (passing all tests across
all platforms).

Feedback, testing, and reviews are warmly welcomed!

Hannu Krosing (22):
Refactor pgbench: extract statistics structures and functions to
stats.c/h
Refactor pgbench: extract variable management and value types to
variable.c/h
Refactor pgbench: extract socket polling abstraction to poller.c/h
Refactor pgbench: extract script parsing, built-in scripts, and
expression evaluation to script.c/h
Refactor pgbench: extract meta-command execution and prepared
statement helpers to commands.c/h
Refactor pgbench: extract database initialization (-i) to init.c/h
Refactor pgbench: extract benchmark execution engine and client state
machine to engine.c/h
Refactor pgbench stats: add mergeStats() and decouple global flags
Refactor pgbench variables: introduce scoped variable store
Refactor pgbench poller: encapsulate socket_set and add poller API
aliases
Refactor pgbench script: decouple chooseScript() from TState
Refactor pgbench commands: introduce pluggable meta-command dispatch
Extract pgbench low-level functions into src/common
Add contrib/pgbench extension exposing benchmark functions in SCHEMA
pgbench
pgbench: Extend PgBenchValue with text, timestamp, and array types
pgbench: Add TPC-C NURand, syllable, string, date/timestamp, and array
functions
pgbench: Add script loops (\for, \while) and exception handling (\try,
\catch, \expect_error)
pgbench: Add \gset_meta and array column parsing in \gset and \aset
pgbench: Support dynamic \sleep expressions, \keying_time, and
\think_time
pgbench: Add heterogeneous client groups (--group) and sequential
streams
pgbench: Add HDR latency percentiles, --sla validation, tpmC/QphH
metrics, and --json-report
pgbench: Add pgbench-b, pgbench-c, pgbench-e, pgbench-h, pgbench-ds,
and pgbench-htap workload suites

---
Hannu

Attachments:

t254037_6
v5-0001-Refactor-pgbench-extract-statistics-structures-an.patchapplication/x-patch; name=v5-0001-Refactor-pgbench-extract-statistics-structures-an.patchDownload+353-297
v5-0003-Refactor-pgbench-extract-socket-polling-abstracti.patchapplication/x-patch; name=v5-0003-Refactor-pgbench-extract-socket-polling-abstracti.patchDownload+266-223
v5-0004-Refactor-pgbench-extract-script-parsing-built-in-.patchapplication/x-patch; name=v5-0004-Refactor-pgbench-extract-script-parsing-built-in-.patchDownload+4434-4357
v5-0002-Refactor-pgbench-extract-variable-management-and-.patchapplication/x-patch; name=v5-0002-Refactor-pgbench-extract-variable-management-and-.patchDownload+739-676
v5-0006-Refactor-pgbench-extract-database-initialization-.patchapplication/x-patch; name=v5-0006-Refactor-pgbench-extract-database-initialization-.patchDownload+887-821
v5-0005-Refactor-pgbench-extract-meta-command-execution-a.patchapplication/x-patch; name=v5-0005-Refactor-pgbench-extract-meta-command-execution-a.patchDownload+548-499
v5-0007-Refactor-pgbench-extract-benchmark-execution-engi.patchapplication/x-patch; name=v5-0007-Refactor-pgbench-extract-benchmark-execution-engi.patchDownload+2552-2484
v5-0008-Refactor-pgbench-stats-add-mergeStats-and-decoupl.patchapplication/x-patch; name=v5-0008-Refactor-pgbench-stats-add-mergeStats-and-decoupl.patchDownload+35-41
v5-0009-Refactor-pgbench-variables-introduce-scoped-varia.patchapplication/x-patch; name=v5-0009-Refactor-pgbench-variables-introduce-scoped-varia.patchDownload+310-98
v5-0011-Refactor-pgbench-script-decouple-chooseScript-fro.patchapplication/x-patch; name=v5-0011-Refactor-pgbench-script-decouple-chooseScript-fro.patchDownload+4-9
v5-0010-Refactor-pgbench-poller-encapsulate-socket_set-an.patchapplication/x-patch; name=v5-0010-Refactor-pgbench-poller-encapsulate-socket_set-an.patchDownload+64-43
v5-0012-Refactor-pgbench-commands-introduce-pluggable-met.patchapplication/x-patch; name=v5-0012-Refactor-pgbench-commands-introduce-pluggable-met.patchDownload+441-319
v5-0013-Extract-pgbench-low-level-functions-into-src-comm.patchapplication/x-patch; name=v5-0013-Extract-pgbench-low-level-functions-into-src-comm.patchDownload+388-327
v5-0014-Add-contrib-pgbench-extension-exposing-benchmark-.patchapplication/x-patch; name=v5-0014-Add-contrib-pgbench-extension-exposing-benchmark-.patchDownload+797-11
v5-0016-pgbench-Add-TPC-C-NURand-syllable-string-date-tim.patchapplication/x-patch; name=v5-0016-pgbench-Add-TPC-C-NURand-syllable-string-date-tim.patchDownload+2015-237
v5-0015-pgbench-Extend-PgBenchValue-with-text-timestamp-a.patchapplication/x-patch; name=v5-0015-pgbench-Extend-PgBenchValue-with-text-timestamp-a.patchDownload+1417-107
v5-0017-pgbench-Add-script-loops-for-while-and-exception-.patchapplication/x-patch; name=v5-0017-pgbench-Add-script-loops-for-while-and-exception-.patchDownload+1032-60
v5-0018-pgbench-Add-gset_meta-and-array-column-parsing-in.patchapplication/x-patch; name=v5-0018-pgbench-Add-gset_meta-and-array-column-parsing-in.patchDownload+421-9
v5-0019-pgbench-Support-dynamic-sleep-expressions-keying_.patchapplication/x-patch; name=v5-0019-pgbench-Support-dynamic-sleep-expressions-keying_.patchDownload+415-72
v5-0020-pgbench-Add-heterogeneous-client-groups-group-and.patchapplication/x-patch; name=v5-0020-pgbench-Add-heterogeneous-client-groups-group-and.patchDownload+539-35
v5-0021-pgbench-Add-HDR-latency-percentiles-sla-validatio.patchapplication/x-patch; name=v5-0021-pgbench-Add-HDR-latency-percentiles-sla-validatio.patchDownload+716-11
#7Hannu Krosing
hannuk@google.com
In reply to: Hannu Krosing (#6)
Re: [PATCH] Refactor pgbench to make future improvements easier

From b102b11350e0ab77499f9249b2fd27097865088c Mon Sep 17 00:00:00 2001
From: Hannu Krosing <hannuk@google.com>
Date: Wed, 21 Oct 2026 17:20:20 +0000
Subject: [PATCH v6 00/28] Modularize pgbench, extract shared PRNG/math
to src/common, add contrib/pgbench, TPC/HTAP/LDBC-Graph/VecBench
suites, JSON configs, --log-db telemetry, and WASM viewer

Hi hackers,

Attached is v6 of the patch series to modularize pgbench, extract its
core PRNG/hashing/distribution routines into src/common for reuse by a
new server-side contrib/pgbench extension, and extend pgbench with the
scripting primitives, concurrency/dataset pillars, declarative JSON/JSONC
configurations, database telemetry logging (--log-db), modular WebAssembly
viewer (src/bin/pgbench/viewer/), and bundled workload suites needed to run
complex multi-transaction OLTP, DSS, HTAP, Graph, and Vector workloads
natively without external middleware.

Changes in v6 (compared to v5):
-------------------------------
1. Rebased onto latest master (c00a97b646c).
2. Expanded the patch series from 22 patches (v5) to 28 patches (v6) with
six cohesive feature commits (Patches 0023-0028):
- Patch 0023 (Common Horizontal Pillars Tiers 1-3):
* Read-only mmap dataset loader (dataset.c/h) for binary vector files
(.fvecs, .bvecs, .ivecs) and columnar CSV/TSV files, plus native
PGBT_VECTOR and PGBT_BYTEA value types and \gset_col.
* Lock-free shared reservoirs (--reservoir, \reservoir_push,
\reservoir_sample) and causal consistency watermarks (\wait_until,
\advance_watermark) for cross-stream producer/consumer coordination.
* Non-blocking mid-script connection switching (\session_connect,
\session_disconnect), subroutine calls (\call), Recall@K / NDCG@K
telemetry (\record_recall), schedule lag SLAs, and deterministic
trace replay (--replay).
- Patch 0024 (LDBC-Inspired Graph Benchmark Suites):
* Correlated power-law graph generators (ldbc_correlated_edge,
ldbc_flashmob_ts), dependent write tracking (\begin_dependent_write,
\end_dependent_write), phased execution barriers (\phase_barrier),
and in-script result assertions (\validate_result).
* LDBC-inspired graph suites across three SQL execution models:
standard relational SQL (pgbench-graph), recursive CTEs with
UNION DISTINCT ON (pgbench-graph-sql-ud), and ISO SQL/PGQ
GRAPH_TABLE / CREATE PROPERTY GRAPH (pgbench-graph-pgq) covering
LSQB (Q1-Q9), SNB Interactive v1/v2 (IC1-IC13, IS1-IS7, IU1-IU8,
ID1-ID8), SNB Business Intelligence (Power -> \phase_barrier ->
Batch Update -> \phase_barrier -> Throughput), FinBench, and
Graphalytics, with BIGINT key alignment across vertex/edge tables.
- Patch 0025 (VecBench-Inspired Vector ANN Suite & Documentation):
* VecBench-inspired vector ANN benchmark suite (pgbench-vec Modes 1-6:
unfiltered search, pre-filtered search, post-filtered oversampling,
streaming insert, multi-vector late interaction, and continuous
aging/index degradation tracking Q1 vs Q4 recall), named per-stream
recall SLAs (--sla="total:<metric>:recall>=X"), --vector-base,
--ground-truth-table, and comprehensive user documentation in
src/bin/pgbench/workloads/README.
- Patch 0026 (Declarative JSON/JSONC Workload Configuration):
* Adds --config=<file> and --dump-config=<file> (config.c/h),
multi-stage initialization pipelines (init_steps[]), direct replay
of --json-report files via --config, and 23 ready-to-run
.pgbench.json workload definitions.
- Patch 0027 (Direct PostgreSQL Telemetry Logging & SQL Comparison SRFs):
* Adds --log-db, --log-db-unlogged[=stream|all|off],
--log-db-mode={stream|deferred}, --run-label, --log-db-schema, and
--log-db-index (logdb.c/h) to stream run metadata, HDR latency
percentiles, Recall@K metrics, SLA compliance, and UNLOGGED
interval/transaction samples into PostgreSQL via COPY, plus 5
overloaded SQL inspection/comparison SRFs (pgbench_show_run and
pgbench_compare_runs) and pgbench_viewer_payload_json().
- Patch 0028 (Modular bottle.py + Water.css + WebAssembly Viewer):
* Adds a standalone zero-dependency web telemetry viewer and two-stage
metatemplate SQL Data Explorer (src/bin/pgbench/viewer/) organized
into html/ (10 SimpleTemplate .html.stpl views), css/ (water.css +
viewer.css), wasm/ (439-byte pgbench_chart.wasm SVG charting kernel),
and src/ (C/WAT sources and builder), documented in viewer/README.md.
3. Cross-platform portability & CI fixes (Windows MSVC, MinGW, Meson):
- Added a WIN32 portable temporary file stream fallback
(open_temp_json_stream() / finish_temp_json_stream()) for
serializeConfigJsonToString() and serializeReportJsonToString() in
src/bin/pgbench/logdb.c since POSIX open_memstream() is unavailable on
Windows (MinGW / MSVC).
- Replaced GCC __attribute__((aligned(64))) struct field attributes in
src/bin/pgbench/dataset.h (PgBenchReservoir) and
src/bin/pgbench/dataset.c (PgBenchCausalClock) with explicit cache-line
padding arrays, and added MSVC (_MSC_VER) _Interlocked* and
_ReadWriteBarrier() shims in src/bin/pgbench/pgbench.h for GCC
__atomic_* builtins used in dataset.c, commands.c, engine.c, and
script.c.
- Updated POLL_USING_SELECT in src/bin/pgbench/poller.c to track
write_fds alongside read_fds so non-blocking PGRES_POLLING_WRITING
states wake immediately on Windows select().
- Updated dataset/vector/ground-truth spec parsers in dataset.c to
recognize Windows drive-letter colons (C:\...) and backslash path
separators.
- Pinned deterministic cluster seed 42 in pgbench-vec SQL workloads and
increased Phase 2 multi-script transaction sample count in
001_pgbench_with_server.pl for deterministic CI runs.

Architecture & Series Overview (28 Patches):
--------------------------------------------
The series is organized into ten logical parts across 28 self-contained
patches so each layer can be reviewed and tested independently:

Part 1: Modularize src/bin/pgbench (Patches 0001-0012)
Splits the monolithic ~8,000-line src/bin/pgbench/pgbench.c into focused
compilation units with clean headers and minimal global coupling:
- 0001, 0008: stats.c/h (SimpleStats, StatsData, LatencyHist, mergeStats)
- 0002, 0009: variable.c/h (PgBenchValue, Variable, scoped VariableSpace)
- 0003, 0010: poller.c/h (socket_set abstraction over poll/select)
- 0004, 0011: script.c/h (ParsedScript, expression evaluation, chooseScript)
- 0005, 0012: commands.c/h (meta-command execution & pluggable handler table)
- 0006: init.c/h (database initialization -i)
- 0007: engine.c/h (CState/TState client state machine & thread loop)

Part 2: Shared PRNG/Math Library & contrib/pgbench (Patches 0013-0014)
- 0013: Extracts pgbench's deterministic PRNG (pg_erand48 / Xoshiro),
64-bit FNV/Murmur hashing, uniform/exponential/gaussian/zipfian random
distributions, permute(), TPC-C NURand, syllable generation, random
strings, Pareto/lognormal distributions, and correlated graph/vector
generators into src/common/pgbench_funcs.c (and
src/include/common/pgbench_funcs.h), compiled into both libpgcommon.a
and libpgcommon_shlib.a.
- 0014: Introduces contrib/pgbench (CREATE EXTENSION pgbench), installing
SQL-callable C functions into SCHEMA pgbench so server-side data
generation (INSERT ... SELECT FROM generate_series(...)) produces
values using the exact same algorithms and distributions as the
pgbench client.

Part 3: Rich Scripting & Client State Machine Primitives (Patches 0015-0019)
- 0015: Extends PgBenchValue with text (PGBT_TEXT), timestamp
(PGBT_TIMESTAMP), and 1D array (PGBT_ARRAY) types, with automatic
SQL literal quoting and array subscripting (:arr[:i]).
- 0016: Adds built-in expression functions: nurand(), c_last(),
random_string(), random_alnum(), random_date(), random_timestamp(),
array_length(), array_append(), and format().
- 0017: Adds structured control-flow meta-commands (\for / \endfor,
\while / \endwhile) and block-scoped exception handling (\try /
\catch / \endtry with automatic SAVEPOINT / ROLLBACK TO SAVEPOINT
and serialization/deadlock retry support, plus \expect_error).
- 0018: Adds \gset_meta (capturing rowcount, sqlstate, and backend_pid)
and PostgreSQL array text parsing in \gset and \aset.
- 0019: Extends \sleep to evaluate arbitrary arithmetic expressions and
adds \keying_time and \think_time (log-normal user think times that
sleep without inflating active transaction latency).

Part 4: Multi-Group Orchestration, Telemetry & Standard Suites
(Patches 0020-0022)
- 0020: Adds heterogeneous client groups (--group name:clients=N,
threads=M,rate=R,scripts=...) and sequential execution streams for
concurrent mixed OLTP + DSS/OLAP workloads.
- 0021: Adds power-of-two sub-bucket HDR latency histograms (p50, p90,
p95, p99, p99.9, max), --sla threshold validation, composite throughput
metrics (tpmC, QphH), and machine-readable --json-report output.
- 0022: Adds six self-contained benchmark workload suites under
src/bin/pgbench/workloads/:
* pgbench-b (enhanced TPCB-like banking workload)
* pgbench-c (TPC-C-inspired wholesale supplier OLTP workload)
* pgbench-e (TPC-E-inspired brokerage OLTP workload)
* pgbench-h (TPC-H-inspired decision support workload)
* pgbench-ds (TPC-DS-inspired multi-channel retail DSS workload)
* pgbench-htap (hybrid concurrent OLTP + analytical OLAP workload)

Part 5: Common Horizontal Pillars Tiers 1-3 (Patch 0023)
- 0023: Implements in-memory mmap datasets (dataset.c/h) for zero-copy
loading of .fvecs/.bvecs/.ivecs and .csv/.tsv datasets, PGBT_VECTOR
and PGBT_BYTEA value types, \gset_col, lock-free shared reservoirs
(--reservoir, \reservoir_push, \reservoir_sample), causal watermarks
(\wait_until, \advance_watermark), non-blocking \session_connect /
\session_disconnect, \call / \return subroutines, Recall@K / NDCG@K
accuracy telemetry (\record_recall), schedule lag SLAs, and
deterministic trace replay (--replay), along with reference
documentation in doc/src/sgml/ref/pgbench.sgml and
doc/src/sgml/pgbench-ext.sgml.

Part 6: LDBC-Inspired Graph Benchmark Suites (Patch 0024)
- 0024: Implements correlated power-law graph generators
(ldbc_correlated_edge, ldbc_flashmob_ts), dependent write tracking
(\begin_dependent_write, \end_dependent_write), phased execution
barriers (\phase_barrier), result validation (\validate_result),
SGML reference documentation (doc/src/sgml/ref/pgbench.sgml and
doc/src/sgml/pgbench-ext.sgml), and the unified BIGINT-aligned
LDBC-inspired schema and query suites for LSQB (Q1-Q9), SNB
Interactive v1/v2, SNB BI, FinBench, and Graphalytics across standard
relational SQL (workloads/pgbench-graph/), recursive UNION DISTINCT ON
(workloads/pgbench-graph-sql-ud/), and native ISO SQL/PGQ GRAPH_TABLE
(workloads/pgbench-graph-pgq/).

Part 7: VecBench-Inspired Vector Benchmark Suite & Docs (Patch 0025)
- 0025: Implements the VecBench-inspired vector ANN benchmark suite
under workloads/pgbench-vec/ across 6 operational modes (unfiltered
search, pre-filtered search, post-filtered adaptive oversampling,
streaming ingestion, multi-vector MaxSim late interaction, and
continuous aging/index degradation comparing initial Q1 vs mature Q4
recall), plus --vector-base, --ground-truth-table, named per-stream
recall SLAs, and comprehensive user documentation and trademark/naming
guardrails in src/bin/pgbench/workloads/README.

Part 8: Declarative JSON/JSONC Configuration (Patch 0026)
- 0026: Adds --config=<file> and --dump-config=<file> (src/bin/pgbench/
config.c/h) using PostgreSQL's native src/common/jsonapi.c parser with
a C/C++ comment stripper for JSONC support, multi-stage init_steps,
direct replay of --json-report output files, and 23 ready-to-run
.pgbench.json workload definitions.

Part 9: Direct PostgreSQL Telemetry Logging & SQL Comparison SRFs (Patch 0027)
- 0027: Adds --log-db, --log-db-unlogged[=stream|all|off],
--log-db-mode={stream|deferred}, --run-label, --log-db-schema, and
--log-db-index (src/bin/pgbench/logdb.c/h) to stream run metadata,
per-group/per-script HDR percentiles, Recall@K metrics, SLA compliance,
and high-throughput UNLOGGED per-transaction/per-window samples into a
3-tier, 9-table PostgreSQL schema using COPY, plus 5 overloaded SQL
comparison SRFs (pgbench_show_run, pgbench_compare_runs) and
pgbench_viewer_payload_json().

Part 10: Modular WebAssembly Telemetry Viewer & SQL Data Explorer (Patch 0028)
- 0028: Adds a standalone zero-dependency bottle.py + Water.css +
439-byte WebAssembly SVG chart viewer and two-stage metatemplate SQL
Data Explorer (src/bin/pgbench/viewer/ organized into html/, css/,
wasm/, and src/, with documentation in viewer/README.md and
src/bin/pgbench/README).

Testing:
--------
- `make -C contrib/pgbench check` passes all SQL regression tests for
`SCHEMA pgbench` functions and generators.
- `make -C src/bin/pgbench check` passes all TAP tests in
`t/001_pgbench_with_server.pl` and `t/002_pgbench_no_server.pl`.

Hannu Krosing (28):
Refactor pgbench: extract statistics structures and functions to
stats.c/h
Refactor pgbench: extract variable management and value types to
variable.c/h
Refactor pgbench: extract socket polling abstraction to poller.c/h
Refactor pgbench: extract script parsing, built-in scripts, and
expression evaluation to script.c/h
Refactor pgbench: extract meta-command execution and prepared
statement helpers to commands.c/h
Refactor pgbench: extract database initialization (-i) to init.c/h
Refactor pgbench: extract benchmark execution engine and client state
machine to engine.c/h
Refactor pgbench stats: add mergeStats() and decouple global flags
Refactor pgbench variables: introduce scoped variable store
Refactor pgbench poller: encapsulate socket_set and add poller API
aliases
Refactor pgbench script: decouple chooseScript() from TState
Refactor pgbench commands: introduce pluggable meta-command dispatch
Extract pgbench low-level functions into src/common
Add contrib/pgbench extension exposing benchmark functions in SCHEMA
pgbench
pgbench: Extend PgBenchValue with text, timestamp, and array types
pgbench: Add TPC-C NURand, syllable, string, date/timestamp, and array
functions
pgbench: Add script loops (\for, \while) and exception handling (\try,
\catch, \expect_error)
pgbench: Add \gset_meta and array column parsing in \gset and \aset
pgbench: Support dynamic \sleep expressions, \keying_time, and
\think_time
pgbench: Add heterogeneous client groups (--group) and sequential
streams
pgbench: Add HDR latency percentiles, --sla validation, tpmC/QphH
metrics, and --json-report
pgbench: Add pgbench-b, pgbench-c, pgbench-e, pgbench-h, pgbench-ds,
and pgbench-htap workload suites
pgbench: add mmap datasets, vector/bytea types, reservoirs,
watermarks, \call, Recall@K, and --replay
pgbench: add LDBC-inspired graph suites (SQL, SQL_UD, SQL/PGQ),
generators, and barriers
pgbench: add VecBench-inspired vector ANN suite (Modes 1-6), recall
SLAs, and docs
pgbench: add --config and --dump-config JSON/JSONC workload
configuration support
pgbench: add --log-db PostgreSQL telemetry logging, UNLOGGED stream
tables, and SQL comparison SRFs
pgbench: add modular bottle.py + Water.css + WebAssembly telemetry
viewer

contrib/Makefile | 1 +
contrib/meson.build | 1 +
contrib/pgbench/.gitignore | 7 +
contrib/pgbench/Makefile | 29 +
contrib/pgbench/README | 34 +
contrib/pgbench/expected/pgbench.out | 306 +
contrib/pgbench/meson.build | 35 +
contrib/pgbench/pgbench--1.0.sql | 427 +
contrib/pgbench/pgbench.c | 1034 +++
contrib/pgbench/pgbench.control | 6 +
contrib/pgbench/sql/pgbench.sql | 111 +
doc/src/sgml/contrib.sgml | 2 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/pgbench-ext.sgml | 580 ++
doc/src/sgml/ref/pgbench.sgml | 1062 ++-
src/bin/pgbench/.gitignore | 1 +
src/bin/pgbench/Makefile | 13 +
src/bin/pgbench/README | 217 +
src/bin/pgbench/commands.c | 2057 +++++
src/bin/pgbench/commands.h | 87 +
src/bin/pgbench/config.c | 3052 +++++++
src/bin/pgbench/config.h | 55 +
src/bin/pgbench/dataset.c | 2894 ++++++
src/bin/pgbench/dataset.h | 159 +
src/bin/pgbench/engine.c | 4143 +++++++++
src/bin/pgbench/engine.h | 253 +
src/bin/pgbench/exprparse.y | 234 +-
src/bin/pgbench/exprscan.l | 18 +
src/bin/pgbench/init.c | 828 ++
src/bin/pgbench/init.h | 51 +
src/bin/pgbench/logdb.c | 2735 ++++++
src/bin/pgbench/logdb.h | 77 +
src/bin/pgbench/meson.build | 10 +
src/bin/pgbench/pgbench.c | 8042 ++---------------
src/bin/pgbench/pgbench.h | 470 +-
src/bin/pgbench/poller.c | 287 +
src/bin/pgbench/poller.h | 47 +
src/bin/pgbench/script.c | 4435 +++++++++
src/bin/pgbench/script.h | 198 +
src/bin/pgbench/stats.c | 753 ++
src/bin/pgbench/stats.h | 220 +
src/bin/pgbench/t/001_pgbench_with_server.pl | 1544 ++++
src/bin/pgbench/t/002_pgbench_no_server.pl | 557 ++
src/bin/pgbench/variable.c | 1772 ++++
src/bin/pgbench/variable.h | 177 +
src/bin/pgbench/viewer/README.md | 133 +
src/bin/pgbench/viewer/bottle.py | 3829 ++++++++
src/bin/pgbench/viewer/css/viewer.css | 202 +
src/bin/pgbench/viewer/css/water.css | 81 +
.../viewer/html/compare_matrix.html.stpl | 43 +
.../viewer/html/compare_pair.html.stpl | 49 +
.../viewer/html/explorer_body.html.stpl | 51 +
.../viewer/html/explorer_table_meta.html.stpl | 87 +
src/bin/pgbench/viewer/html/layout.html.stpl | 46 +
.../viewer/html/live_monitor.html.stpl | 100 +
.../pgbench/viewer/html/phase1_body.html.stpl | 30 +
.../viewer/html/run_selector.html.stpl | 120 +
.../pgbench/viewer/html/single_run.html.stpl | 146 +
.../viewer/html/wasm_chart_box.html.stpl | 5 +
.../viewer/html/wasm_chart_module.html.stpl | 246 +
src/bin/pgbench/viewer/pgbench_viewer.py | 704 ++
src/bin/pgbench/viewer/src/build_wasm.py | 216 +
src/bin/pgbench/viewer/src/pgbench_chart.c | 66 +
src/bin/pgbench/viewer/src/pgbench_chart.wat | 105 +
.../pgbench/viewer/wasm/pgbench_chart.wasm | Bin 0 -> 439 bytes
src/bin/pgbench/workloads/README | 511 ++
.../pgbench-b/pgbench_b.pgbench.json | 37 +
.../pgbench-b/pgbench_b_constraints.sql | 17 +
.../workloads/pgbench-b/pgbench_b_init.sql | 32 +
.../workloads/pgbench-b/pgbench_b_schema.sql | 100 +
.../pgbench-b/pgbench_b_select_only.sql | 9 +
.../pgbench-b/pgbench_b_simple_update.sql | 14 +
.../workloads/pgbench-b/pgbench_b_tx.sql | 14 +
.../pgbench-c/pgbench_c.pgbench.json | 53 +
.../pgbench-c/pgbench_c_constraints.sql | 11 +
.../pgbench-c/pgbench_c_delivery.sql | 15 +
.../workloads/pgbench-c/pgbench_c_init.sql | 206 +
.../pgbench-c/pgbench_c_new_order.sql | 41 +
.../pgbench-c/pgbench_c_order_status.sql | 19 +
.../workloads/pgbench-c/pgbench_c_payment.sql | 31 +
.../workloads/pgbench-c/pgbench_c_schema.sql | 470 +
.../pgbench-c/pgbench_c_stock_level.sql | 11 +
.../pgbench-ds/pgbench_ds.pgbench.json | 63 +
.../pgbench-ds/pgbench_ds_constraints.sql | 22 +
.../workloads/pgbench-ds/pgbench_ds_init.sql | 306 +
.../pgbench-ds/pgbench_ds_maintenance.sql | 35 +
.../pgbench-ds/pgbench_ds_q_cross_channel.sql | 61 +
.../pgbench-ds/pgbench_ds_q_inventory.sql | 32 +
.../pgbench-ds/pgbench_ds_q_promotion.sql | 31 +
.../pgbench-ds/pgbench_ds_q_store_sales.sql | 47 +
.../pgbench-ds/pgbench_ds_schema.sql | 214 +
.../pgbench-e/pgbench_e.pgbench.json | 57 +
.../pgbench-e/pgbench_e_broker_volume.sql | 8 +
.../pgbench-e/pgbench_e_constraints.sql | 20 +
.../pgbench-e/pgbench_e_customer_position.sql | 19 +
.../workloads/pgbench-e/pgbench_e_init.sql | 202 +
.../pgbench-e/pgbench_e_market_watch.sql | 12 +
.../workloads/pgbench-e/pgbench_e_schema.sql | 509 ++
.../pgbench-e/pgbench_e_trade_order.sql | 24 +
.../pgbench-e/pgbench_e_trade_result.sql | 14 +
.../pgbench-e/pgbench_e_trade_status.sql | 14 +
.../pgbench-graph-pgq/finbench.pgbench.json | 37 +
.../graphalytics.pgbench.json | 37 +
.../pgbench-graph-pgq/lsqb.pgbench.json | 37 +
.../pgbench_graph_pgq_1hop.sql | 40 +
.../pgbench_graph_pgq_2hop.sql | 38 +
.../pgbench_graph_pgq_add_edge.sql | 14 +
.../pgbench_graph_pgq_constraints.sql | 151 +
.../pgbench_graph_pgq_ddl.sql | 93 +
.../pgbench_graph_pgq_init.sql | 203 +
.../pgbench_graph_pgq_schema.sql | 273 +
.../pgbench_ldbc_pgq_finbench.sql | 178 +
.../pgbench_ldbc_pgq_graphalytics.sql | 186 +
.../pgbench_ldbc_pgq_lsqb.sql | 138 +
.../pgbench_ldbc_pgq_snb_bi.sql | 255 +
.../pgbench_ldbc_pgq_snb_ic.sql | 151 +
.../pgbench_ldbc_pgq_snb_id.sql | 83 +
.../pgbench_ldbc_pgq_snb_is.sql | 141 +
.../pgbench_ldbc_pgq_snb_iu.sql | 87 +
.../pgbench-graph-pgq/snb_bi.pgbench.json | 37 +
.../snb_interactive.pgbench.json | 66 +
.../finbench.pgbench.json | 37 +
.../graphalytics.pgbench.json | 37 +
.../pgbench-graph-sql-ud/lsqb.pgbench.json | 37 +
.../pgbench_graph_sql_ud_1hop.sql | 28 +
.../pgbench_graph_sql_ud_2hop.sql | 39 +
.../pgbench_graph_sql_ud_add_edge.sql | 14 +
.../pgbench_graph_sql_ud_constraints.sql | 151 +
.../pgbench_graph_sql_ud_init.sql | 203 +
.../pgbench_graph_sql_ud_schema.sql | 181 +
.../pgbench_ldbc_sql_ud_finbench.sql | 135 +
.../pgbench_ldbc_sql_ud_graphalytics.sql | 152 +
.../pgbench_ldbc_sql_ud_lsqb.sql | 109 +
.../pgbench_ldbc_sql_ud_snb_bi.sql | 219 +
.../pgbench_ldbc_sql_ud_snb_ic.sql | 125 +
.../pgbench_ldbc_sql_ud_snb_id.sql | 76 +
.../pgbench_ldbc_sql_ud_snb_is.sql | 91 +
.../pgbench_ldbc_sql_ud_snb_iu.sql | 87 +
.../pgbench-graph-sql-ud/snb_bi.pgbench.json | 37 +
.../snb_interactive.pgbench.json | 66 +
.../pgbench-graph/finbench.pgbench.json | 37 +
.../pgbench-graph/graphalytics.pgbench.json | 37 +
.../workloads/pgbench-graph/lsqb.pgbench.json | 37 +
.../pgbench-graph/pgbench_graph_1hop.sql | 28 +
.../pgbench-graph/pgbench_graph_2hop.sql | 39 +
.../pgbench-graph/pgbench_graph_add_edge.sql | 14 +
.../pgbench_graph_constraints.sql | 151 +
.../pgbench-graph/pgbench_graph_init.sql | 203 +
.../pgbench-graph/pgbench_graph_pgq_ddl.sql | 93 +
.../pgbench-graph/pgbench_graph_schema.sql | 181 +
.../pgbench-graph/pgbench_ldbc_finbench.sql | 146 +
.../pgbench_ldbc_graphalytics.sql | 190 +
.../pgbench-graph/pgbench_ldbc_lsqb.sql | 108 +
.../pgbench-graph/pgbench_ldbc_snb_bi.sql | 225 +
.../pgbench-graph/pgbench_ldbc_snb_ic.sql | 130 +
.../pgbench-graph/pgbench_ldbc_snb_id.sql | 76 +
.../pgbench-graph/pgbench_ldbc_snb_is.sql | 91 +
.../pgbench-graph/pgbench_ldbc_snb_iu.sql | 87 +
.../pgbench_ldbc_sql_ud_finbench.sql | 135 +
.../pgbench_ldbc_sql_ud_graphalytics.sql | 152 +
.../pgbench_ldbc_sql_ud_lsqb.sql | 109 +
.../pgbench_ldbc_sql_ud_snb_bi.sql | 219 +
.../pgbench_ldbc_sql_ud_snb_ic.sql | 125 +
.../pgbench_ldbc_sql_ud_snb_id.sql | 76 +
.../pgbench_ldbc_sql_ud_snb_is.sql | 91 +
.../pgbench_ldbc_sql_ud_snb_iu.sql | 87 +
.../pgbench-graph/snb_bi.pgbench.json | 37 +
.../snb_interactive.pgbench.json | 66 +
.../pgbench-h/pgbench_h.pgbench.json | 52 +
.../pgbench-h/pgbench_h_constraints.sql | 20 +
.../workloads/pgbench-h/pgbench_h_init.sql | 187 +
.../workloads/pgbench-h/pgbench_h_q1.sql | 28 +
.../workloads/pgbench-h/pgbench_h_q10.sql | 40 +
.../workloads/pgbench-h/pgbench_h_q12.sql | 36 +
.../workloads/pgbench-h/pgbench_h_q14.sql | 21 +
.../workloads/pgbench-h/pgbench_h_q19.sql | 46 +
.../workloads/pgbench-h/pgbench_h_q3.sql | 32 +
.../workloads/pgbench-h/pgbench_h_q5.sql | 34 +
.../workloads/pgbench-h/pgbench_h_q6.sql | 21 +
.../workloads/pgbench-h/pgbench_h_queries.sql | 37 +
.../workloads/pgbench-h/pgbench_h_refresh.sql | 54 +
.../workloads/pgbench-h/pgbench_h_schema.sql | 105 +
.../pgbench-htap/pgbench_htap.pgbench.json | 70 +
.../pgbench-htap/pgbench_htap_constraints.sql | 15 +
.../pgbench-htap/pgbench_htap_init.sql | 233 +
.../pgbench-htap/pgbench_htap_olap_q1.sql | 19 +
.../pgbench-htap/pgbench_htap_olap_q18.sql | 36 +
.../pgbench-htap/pgbench_htap_olap_q3.sql | 34 +
.../pgbench-htap/pgbench_htap_olap_q5.sql | 39 +
.../pgbench-htap/pgbench_htap_olap_q6.sql | 10 +
.../pgbench-htap/pgbench_htap_oltp.sql | 46 +
.../pgbench-htap/pgbench_htap_schema.sql | 396 +
.../pgbench-vec/pgbench_vec_aging.sql | 79 +
.../pgbench-vec/pgbench_vec_constraints.sql | 26 +
.../pgbench-vec/pgbench_vec_filtered.sql | 19 +
.../pgbench-vec/pgbench_vec_gt_gen.sql | 51 +
.../pgbench-vec/pgbench_vec_init.sql | 34 +
.../pgbench-vec/pgbench_vec_insert.sql | 22 +
.../pgbench-vec/pgbench_vec_multivec.sql | 39 +
.../pgbench-vec/pgbench_vec_postfiltered.sql | 36 +
.../pgbench-vec/pgbench_vec_schema.sql | 91 +
.../pgbench-vec/pgbench_vec_search.sql | 28 +
.../pgbench-vec/vec_aging.pgbench.json | 45 +
.../pgbench-vec/vec_hybrid.pgbench.json | 64 +
src/common/Makefile | 1 +
src/common/meson.build | 1 +
src/common/pgbench_funcs.c | 1687 ++++
src/include/common/pgbench_funcs.h | 117 +
208 files changed, 52414 insertions(+), 7138 deletions(-)
create mode 100644 contrib/pgbench/.gitignore
create mode 100644 contrib/pgbench/Makefile
create mode 100644 contrib/pgbench/README
create mode 100644 contrib/pgbench/expected/pgbench.out
create mode 100644 contrib/pgbench/meson.build
create mode 100644 contrib/pgbench/pgbench--1.0.sql
create mode 100644 contrib/pgbench/pgbench.c
create mode 100644 contrib/pgbench/pgbench.control
create mode 100644 contrib/pgbench/sql/pgbench.sql
create mode 100644 doc/src/sgml/pgbench-ext.sgml
create mode 100644 src/bin/pgbench/README
create mode 100644 src/bin/pgbench/commands.c
create mode 100644 src/bin/pgbench/commands.h
create mode 100644 src/bin/pgbench/config.c
create mode 100644 src/bin/pgbench/config.h
create mode 100644 src/bin/pgbench/dataset.c
create mode 100644 src/bin/pgbench/dataset.h
create mode 100644 src/bin/pgbench/engine.c
create mode 100644 src/bin/pgbench/engine.h
create mode 100644 src/bin/pgbench/init.c
create mode 100644 src/bin/pgbench/init.h
create mode 100644 src/bin/pgbench/logdb.c
create mode 100644 src/bin/pgbench/logdb.h
create mode 100644 src/bin/pgbench/poller.c
create mode 100644 src/bin/pgbench/poller.h
create mode 100644 src/bin/pgbench/script.c
create mode 100644 src/bin/pgbench/script.h
create mode 100644 src/bin/pgbench/stats.c
create mode 100644 src/bin/pgbench/stats.h
create mode 100644 src/bin/pgbench/variable.c
create mode 100644 src/bin/pgbench/variable.h
create mode 100644 src/bin/pgbench/viewer/README.md
create mode 100644 src/bin/pgbench/viewer/bottle.py
create mode 100644 src/bin/pgbench/viewer/css/viewer.css
create mode 100644 src/bin/pgbench/viewer/css/water.css
create mode 100644 src/bin/pgbench/viewer/html/compare_matrix.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/compare_pair.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/explorer_body.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/explorer_table_meta.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/layout.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/live_monitor.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/phase1_body.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/run_selector.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/single_run.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/wasm_chart_box.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/wasm_chart_module.html.stpl
create mode 100755 src/bin/pgbench/viewer/pgbench_viewer.py
create mode 100644 src/bin/pgbench/viewer/src/build_wasm.py
create mode 100644 src/bin/pgbench/viewer/src/pgbench_chart.c
create mode 100644 src/bin/pgbench/viewer/src/pgbench_chart.wat
create mode 100644 src/bin/pgbench/viewer/wasm/pgbench_chart.wasm
create mode 100644 src/bin/pgbench/workloads/README
create mode 100644 src/bin/pgbench/workloads/pgbench-b/pgbench_b.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-b/pgbench_b_constraints.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-b/pgbench_b_init.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-b/pgbench_b_schema.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-b/pgbench_b_select_only.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-b/pgbench_b_simple_update.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-b/pgbench_b_tx.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-c/pgbench_c_constraints.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_delivery.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_init.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_new_order.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-c/pgbench_c_order_status.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_payment.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_schema.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-c/pgbench_c_stock_level.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-ds/pgbench_ds.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_constraints.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_init.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_maintenance.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_q_cross_channel.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_q_inventory.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_q_promotion.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_q_store_sales.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_schema.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-e/pgbench_e.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-e/pgbench_e_broker_volume.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-e/pgbench_e_constraints.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-e/pgbench_e_customer_position.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-e/pgbench_e_init.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-e/pgbench_e_market_watch.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-e/pgbench_e_schema.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-e/pgbench_e_trade_order.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-e/pgbench_e_trade_result.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-e/pgbench_e_trade_status.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/finbench.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/graphalytics.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/lsqb.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_1hop.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_2hop.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_add_edge.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_constraints.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_ddl.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_init.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_schema.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_finbench.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_graphalytics.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_lsqb.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_snb_bi.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_snb_ic.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_snb_id.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_snb_is.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_snb_iu.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/snb_bi.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/snb_interactive.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/finbench.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/graphalytics.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/lsqb.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_1hop.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_2hop.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_add_edge.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_constraints.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_init.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_schema.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_finbench.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_graphalytics.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_lsqb.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_snb_bi.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_snb_ic.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_snb_id.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_snb_is.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_snb_iu.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/snb_bi.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/snb_interactive.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/finbench.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/graphalytics.pgbench.json
create mode 100644 src/bin/pgbench/workloads/pgbench-graph/lsqb.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_1hop.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_2hop.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_add_edge.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_constraints.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_init.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_pgq_ddl.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_schema.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_finbench.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_graphalytics.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_lsqb.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_snb_bi.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_snb_ic.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_snb_id.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_snb_is.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_snb_iu.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_finbench.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_graphalytics.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_lsqb.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_snb_bi.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_snb_ic.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_snb_id.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_snb_is.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_snb_iu.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-graph/snb_bi.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/snb_interactive.pgbench.json
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-h/pgbench_h_constraints.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_init.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q1.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q10.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q12.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q14.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q19.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q3.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q5.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q6.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_queries.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_refresh.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_schema.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-htap/pgbench_htap.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_constraints.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_init.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_olap_q1.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_olap_q18.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_olap_q3.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_olap_q5.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_olap_q6.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_oltp.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_schema.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_aging.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_constraints.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_filtered.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_gt_gen.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_init.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_insert.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_multivec.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_postfiltered.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_schema.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_search.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-vec/vec_aging.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-vec/vec_hybrid.pgbench.json
create mode 100644 src/common/pgbench_funcs.c
create mode 100644 src/include/common/pgbench_funcs.h

--
2.56.0

Show quoted text

On Sun, Oct 18, 2026 at 5:16 PM Hannu Krosing <hannuk@google.com> wrote:

[PATCH v5 00/22]
* Modular pgbench refactoring,
* contrib/pgbench extension,
* advanced benchmark capabilities,
* and standard workload suites (pgbench-b, pgbench-c, pgbench-e,
pgbench-h, pgbench-ds, pgbench-htap)

Hi hackers,

This patch series presents a comprehensive modernization of pgbench,
transforming
it from a monolithic single-script driver into a modern, modular, multi-workload
benchmarking engine with native support for complex OLTP, OLAP, and
HTAP benchmarks.

See https://wiki.postgresql.org/wiki/Pgbench-modernization-2026 for
more background and rationale.

The series is organized into five logical parts across 22 clean, self-contained
commits rebased cleanly on latest master:

================================================================================
Part 1: Modular pgbench Refactoring (Patches 0001 - 0012)
================================================================================
pgbench.c had grown to over 7,400 lines combining distinct responsibilities:
statistics accumulation, variable management, event polling, script parsing,
meta-command execution, database initialization (-i), and the multi-threaded
state machine.

Patches 0001-0007 extract these modules without altering behavior or
public APIs:
- stats.c / stats.h: StatsData, latency tally, rate-limiting logic.
- variable.c / variable.h: Scoped variable store and value evaluation.
- poller.c / poller.h: Socket set polling abstraction (select/poll).
- script.c / script.h: Script parsing, AST nodes, expression evaluator.
- commands.c / commands.h: Meta-command handlers and prepared
statement helpers.
- init.c / init.h: Database initialization (-i) logic and data generators.
- engine.c / engine.h: Multi-threaded client state machine and event loop.

Patches 0008-0012 decouple remaining global dependencies cleanly:
- stats: add mergeStats() and decouple global flags.
- variable: introduce scoped variable store (VariableSpace).
- poller: encapsulate socket_set and provide poller API wrappers.
- script: decouple chooseScript() from TState.
- commands: introduce pluggable meta-command dispatch table.

================================================================================
Part 2: Code Extraction & In-Database Functions (Patches 0013 - 0014)
================================================================================
Patch 0013 extracts pgbench's mathematical and pseudo-random
generators (Gaussian,
Exponential, Zipfian, Permute, MurmurHash64) into src/common/pgbench_funcs.c,
making them reusable across both client and server binaries.

Patch 0014 introduces contrib/pgbench, a server-side extension that
exposes these
pseudo-random generators and helper functions directly to SQL under
SCHEMA pgbench.
This allows high-performance parallel in-database synthetic data
generation using
standard SQL (e.g. generate_series()) without relying on slow client roundtrips.

================================================================================
Part 3: Advanced Types, Functions, Control Flow & Pacing (Patches 0015 - 0019)
================================================================================
Patch 0015 extends PgBenchValue with text, timestamp, and array types,
along with
array operations (array_append, array_length, array_get, array_generate), text
concatenation, and timestamp arithmetic.

Patch 0016 adds standard benchmark helper functions to pgbench expressions and
contrib/pgbench:
- nurand(A, s, x, y): Non-uniform random distribution generator.
- c_last(num): Syllable-based customer last name generator.
- random_astring(min, max), random_nstring(min, max): Alphanumeric
and numeric strings.
- format_string(fmt, ...), date_add(ts, amt, unit), int_to_date(int).

Patch 0017 adds script control flow and exception handling:
- Loops: \for <var> <start> .. <end> [BY <step>], \while <expr>,
\break, \continue.
- Exception handling: \try ... \catch [<sqlstate>] ... \endtry,
\expect_error <sqlstate>.

Patch 0018 adds result set metadata inspection and array column parsing:
- \gset_meta <prefix>: Extracts query result status, error code,
rows affected, and timing.
- \aset and array-aware \gset: Maps multi-row query results into
client-side array variables.

Patch 0019 introduces dynamic client pacing:
- Dynamic \sleep expressions (e.g. \sleep random_exponential(1, 1000, 10) ms).
- \keying_time and \think_time commands for realistic transaction
pacing per specification.

================================================================================
Part 4: Heterogeneous Groups, Latency Percentiles & Reporting (Patches
0020 - 0021)
================================================================================
Patch 0020 introduces heterogeneous client groups via --group:
- Run concurrent mixes of different scripts with distinct client
counts and rates
(e.g., --group name=oltp,clients=10,weight=100 --group
name=olap,clients=2,sequential=true).
- Supports sequential stream execution for multi-query analytical streams.

Patch 0021 implements High Dynamic Range (HDR) latency histogram tracking:
- Configurable percentiles (p50, p90, p95, p99, p99.9).
- SLA validation via --sla target=<metric>,threshold=<value>.
- Metric computation for tpmC and QphH@SF.
- Structured machine-readable JSON output via --json-report=<path>.

================================================================================
Part 5: Standard Workload Suites (Patch 0022)
================================================================================

<THIS WAS LEFT OUT FROM THIS EMAIL>

* Trademark & Naming Rationale:
These workloads are inspired by and modeled after industry-standard
specifications
(TPC-C, TPC-E, TPC-H, TPC-DS, and CH-benCHmark). To respect and
avoid infringing
upon Transaction Processing Performance Council (TPC) trademarks,
and to clearly
distinguish these community engineering tools from official
council-audited benchmarks,
they are explicitly named pgbench-b, pgbench-c, pgbench-e,
pgbench-h, pgbench-ds, and pgbench-htap.
Full background and guidelines are documented in
src/bin/pgbench/workloads/README.

* In-Database Parallel Loading:
Each workload includes an initialization script
(<workload>_init.sql) designed to
run in parallel across multiple worker sessions (-c N -j N -t 1 -D
num_workers=N).
Data generation is executed entirely within the database engine
using generate_series()
and contrib/pgbench SQL functions, achieving rapid parallel load
rates without external
file generation or ETL binaries.

Comprehensive documentation is provided in doc/src/sgml/ref/pgbench.sgml and
doc/src/sgml/pgbench-ext.sgml, and full end-to-end TAP coverage is added in
src/bin/pgbench/t/001_pgbench_with_server.pl (passing all tests across
all platforms).

Feedback, testing, and reviews are warmly welcomed!

Hannu Krosing (22):
Refactor pgbench: extract statistics structures and functions to
stats.c/h
Refactor pgbench: extract variable management and value types to
variable.c/h
Refactor pgbench: extract socket polling abstraction to poller.c/h
Refactor pgbench: extract script parsing, built-in scripts, and
expression evaluation to script.c/h
Refactor pgbench: extract meta-command execution and prepared
statement helpers to commands.c/h
Refactor pgbench: extract database initialization (-i) to init.c/h
Refactor pgbench: extract benchmark execution engine and client state
machine to engine.c/h
Refactor pgbench stats: add mergeStats() and decouple global flags
Refactor pgbench variables: introduce scoped variable store
Refactor pgbench poller: encapsulate socket_set and add poller API
aliases
Refactor pgbench script: decouple chooseScript() from TState
Refactor pgbench commands: introduce pluggable meta-command dispatch
Extract pgbench low-level functions into src/common
Add contrib/pgbench extension exposing benchmark functions in SCHEMA
pgbench
pgbench: Extend PgBenchValue with text, timestamp, and array types
pgbench: Add TPC-C NURand, syllable, string, date/timestamp, and array
functions
pgbench: Add script loops (\for, \while) and exception handling (\try,
\catch, \expect_error)
pgbench: Add \gset_meta and array column parsing in \gset and \aset
pgbench: Support dynamic \sleep expressions, \keying_time, and
\think_time
pgbench: Add heterogeneous client groups (--group) and sequential
streams
pgbench: Add HDR latency percentiles, --sla validation, tpmC/QphH
metrics, and --json-report
pgbench: Add pgbench-b, pgbench-c, pgbench-e, pgbench-h, pgbench-ds,
and pgbench-htap workload suites

---
Hannu

Attachments:

t254037_7
v6-0001-Refactor-pgbench-extract-statistics-structures-an.patchapplication/x-patch; name=v6-0001-Refactor-pgbench-extract-statistics-structures-an.patchDownload+353-297
v6-0004-Refactor-pgbench-extract-script-parsing-built-in-.patchapplication/x-patch; name=v6-0004-Refactor-pgbench-extract-script-parsing-built-in-.patchDownload+4434-4357
v6-0003-Refactor-pgbench-extract-socket-polling-abstracti.patchapplication/x-patch; name=v6-0003-Refactor-pgbench-extract-socket-polling-abstracti.patchDownload+266-223
v6-0002-Refactor-pgbench-extract-variable-management-and-.patchapplication/x-patch; name=v6-0002-Refactor-pgbench-extract-variable-management-and-.patchDownload+739-676
v6-0005-Refactor-pgbench-extract-meta-command-execution-a.patchapplication/x-patch; name=v6-0005-Refactor-pgbench-extract-meta-command-execution-a.patchDownload+548-499
v6-0008-Refactor-pgbench-stats-add-mergeStats-and-decoupl.patchapplication/x-patch; name=v6-0008-Refactor-pgbench-stats-add-mergeStats-and-decoupl.patchDownload+35-41
v6-0006-Refactor-pgbench-extract-database-initialization-.patchapplication/x-patch; name=v6-0006-Refactor-pgbench-extract-database-initialization-.patchDownload+887-821
v6-0009-Refactor-pgbench-variables-introduce-scoped-varia.patchapplication/x-patch; name=v6-0009-Refactor-pgbench-variables-introduce-scoped-varia.patchDownload+310-98
v6-0010-Refactor-pgbench-poller-encapsulate-socket_set-an.patchapplication/x-patch; name=v6-0010-Refactor-pgbench-poller-encapsulate-socket_set-an.patchDownload+64-43
v6-0011-Refactor-pgbench-script-decouple-chooseScript-fro.patchapplication/x-patch; name=v6-0011-Refactor-pgbench-script-decouple-chooseScript-fro.patchDownload+4-9
v6-0007-Refactor-pgbench-extract-benchmark-execution-engi.patchapplication/x-patch; name=v6-0007-Refactor-pgbench-extract-benchmark-execution-engi.patchDownload+2552-2484
v6-0012-Refactor-pgbench-commands-introduce-pluggable-met.patchapplication/x-patch; name=v6-0012-Refactor-pgbench-commands-introduce-pluggable-met.patchDownload+441-319
v6-0013-Extract-pgbench-low-level-functions-into-src-comm.patchapplication/x-patch; name=v6-0013-Extract-pgbench-low-level-functions-into-src-comm.patchDownload+388-327
v6-0017-pgbench-Add-script-loops-for-while-and-exception-.patchapplication/x-patch; name=v6-0017-pgbench-Add-script-loops-for-while-and-exception-.patchDownload+1032-60
v6-0015-pgbench-Extend-PgBenchValue-with-text-timestamp-a.patchapplication/x-patch; name=v6-0015-pgbench-Extend-PgBenchValue-with-text-timestamp-a.patchDownload+1417-107
v6-0016-pgbench-Add-TPC-C-NURand-syllable-string-date-tim.patchapplication/x-patch; name=v6-0016-pgbench-Add-TPC-C-NURand-syllable-string-date-tim.patchDownload+2015-237
v6-0014-Add-contrib-pgbench-extension-exposing-benchmark-.patchapplication/x-patch; name=v6-0014-Add-contrib-pgbench-extension-exposing-benchmark-.patchDownload+804-11
v6-0018-pgbench-Add-gset_meta-and-array-column-parsing-in.patchapplication/x-patch; name=v6-0018-pgbench-Add-gset_meta-and-array-column-parsing-in.patchDownload+421-9
v6-0019-pgbench-Support-dynamic-sleep-expressions-keying_.patchapplication/x-patch; name=v6-0019-pgbench-Support-dynamic-sleep-expressions-keying_.patchDownload+415-72
v6-0021-pgbench-Add-HDR-latency-percentiles-sla-validatio.patchapplication/x-patch; name=v6-0021-pgbench-Add-HDR-latency-percentiles-sla-validatio.patchDownload+716-11
v6-0020-pgbench-Add-heterogeneous-client-groups-group-and.patchapplication/x-patch; name=v6-0020-pgbench-Add-heterogeneous-client-groups-group-and.patchDownload+539-35
v6-0023-pgbench-add-mmap-datasets-vector-bytea-types-rese.patchapplication/x-patch; name=v6-0023-pgbench-add-mmap-datasets-vector-bytea-types-rese.patchDownload+9424-298
v6-0024-pgbench-add-LDBC-inspired-graph-suites-SQL-SQL_UD.patchapplication/x-patch; name=v6-0024-pgbench-add-LDBC-inspired-graph-suites-SQL-SQL_UD.patchDownload+8129-120
v6-0022-pgbench-Add-pgbench-b-pgbench-c-pgbench-e-pgbench.patchapplication/x-patch; name=v6-0022-pgbench-Add-pgbench-b-pgbench-c-pgbench-e-pgbench.patchDownload+4661-4
v6-0025-pgbench-add-VecBench-inspired-vector-ANN-suite-Mo.patchapplication/x-patch; name=v6-0025-pgbench-add-VecBench-inspired-vector-ANN-suite-Mo.patchDownload+1347-87
v6-0026-pgbench-add-config-and-dump-config-JSON-JSONC-wor.patchapplication/x-patch; name=v6-0026-pgbench-add-config-and-dump-config-JSON-JSONC-wor.patchDownload+4935-74
v6-0028-pgbench-add-modular-bottle.py-Water.css-WebAssemb.patchapplication/x-patch; name=v6-0028-pgbench-add-modular-bottle.py-Water.css-WebAssemb.patchDownload+6274-3
v6-0027-pgbench-add-log-db-PostgreSQL-telemetry-logging-U.patchapplication/x-patch; name=v6-0027-pgbench-add-log-db-PostgreSQL-telemetry-logging-U.patchDownload+4029-649
#8Koshi Shibagaki (Fujitsu)
shibagaki.koshi@fujitsu.com
In reply to: Hannu Krosing (#7)
Re: [PATCH] Refactor pgbench to make future improvements easier

Hi Hannu,

Thank you for patches.
I'm interested in the pgbench refactoring.

The current series includes code movement, changes to internal
interfaces, and new features. To make review easier, could we focus
this thread on moving the existing code into separate modules while
preserving existing behavior?

For v6, patches 0001-0007 seem like a suitable scope for that review.
I have left 0008-0012 out of this scope because they go beyond code
movement and include changes to internal interfaces and structures.
Those changes could be discussed separately, along with the new
features.

I'd like to help review the code movement in 0001-0007.

Regards,
Koshi Shibagaki
FUJITSU LIMITED
https://www.fujitsu.com/

________________________________________
差出人: Hannu Krosing <hannuk@google.com>
送信日時: 2026年10月22日 2:43
宛先: Zsolt Parragi
CC: pgsql-hackers@lists.postgresql.org
件名: Re: [PATCH] Refactor pgbench to make future improvements easier

From b102b11350e0ab77499f9249b2fd27097865088c Mon Sep 17 00:00:00 2001
From: Hannu Krosing <hannuk@google.com>
Date: Wed, 21 Oct 2026 17:20:20 +0000
Subject: [PATCH v6 00/28] Modularize pgbench, extract shared PRNG/math
to src/common, add contrib/pgbench, TPC/HTAP/LDBC-Graph/VecBench
suites, JSON configs, --log-db telemetry, and WASM viewer

Hi hackers,

Attached is v6 of the patch series to modularize pgbench, extract its
core PRNG/hashing/distribution routines into src/common for reuse by a
new server-side contrib/pgbench extension, and extend pgbench with the
scripting primitives, concurrency/dataset pillars, declarative JSON/JSONC
configurations, database telemetry logging (--log-db), modular WebAssembly
viewer (src/bin/pgbench/viewer/), and bundled workload suites needed to run
complex multi-transaction OLTP, DSS, HTAP, Graph, and Vector workloads
natively without external middleware.

Changes in v6 (compared to v5):
-------------------------------
1. Rebased onto latest master (c00a97b646c).
2. Expanded the patch series from 22 patches (v5) to 28 patches (v6) with
six cohesive feature commits (Patches 0023-0028):
- Patch 0023 (Common Horizontal Pillars Tiers 1-3):
* Read-only mmap dataset loader (dataset.c/h) for binary vector files
(.fvecs, .bvecs, .ivecs) and columnar CSV/TSV files, plus native
PGBT_VECTOR and PGBT_BYTEA value types and \gset_col.
* Lock-free shared reservoirs (--reservoir, \reservoir_push,
\reservoir_sample) and causal consistency watermarks (\wait_until,
\advance_watermark) for cross-stream producer/consumer coordination.
* Non-blocking mid-script connection switching (\session_connect,
\session_disconnect), subroutine calls (\call), Recall@K / NDCG@K
telemetry (\record_recall), schedule lag SLAs, and deterministic
trace replay (--replay).
- Patch 0024 (LDBC-Inspired Graph Benchmark Suites):
* Correlated power-law graph generators (ldbc_correlated_edge,
ldbc_flashmob_ts), dependent write tracking (\begin_dependent_write,
\end_dependent_write), phased execution barriers (\phase_barrier),
and in-script result assertions (\validate_result).
* LDBC-inspired graph suites across three SQL execution models:
standard relational SQL (pgbench-graph), recursive CTEs with
UNION DISTINCT ON (pgbench-graph-sql-ud), and ISO SQL/PGQ
GRAPH_TABLE / CREATE PROPERTY GRAPH (pgbench-graph-pgq) covering
LSQB (Q1-Q9), SNB Interactive v1/v2 (IC1-IC13, IS1-IS7, IU1-IU8,
ID1-ID8), SNB Business Intelligence (Power -> \phase_barrier ->
Batch Update -> \phase_barrier -> Throughput), FinBench, and
Graphalytics, with BIGINT key alignment across vertex/edge tables.
- Patch 0025 (VecBench-Inspired Vector ANN Suite & Documentation):
* VecBench-inspired vector ANN benchmark suite (pgbench-vec Modes 1-6:
unfiltered search, pre-filtered search, post-filtered oversampling,
streaming insert, multi-vector late interaction, and continuous
aging/index degradation tracking Q1 vs Q4 recall), named per-stream
recall SLAs (--sla="total:<metric>:recall>=X"), --vector-base,
--ground-truth-table, and comprehensive user documentation in
src/bin/pgbench/workloads/README.
- Patch 0026 (Declarative JSON/JSONC Workload Configuration):
* Adds --config=<file> and --dump-config=<file> (config.c/h),
multi-stage initialization pipelines (init_steps[]), direct replay
of --json-report files via --config, and 23 ready-to-run
.pgbench.json workload definitions.
- Patch 0027 (Direct PostgreSQL Telemetry Logging & SQL Comparison SRFs):
* Adds --log-db, --log-db-unlogged[=stream|all|off],
--log-db-mode={stream|deferred}, --run-label, --log-db-schema, and
--log-db-index (logdb.c/h) to stream run metadata, HDR latency
percentiles, Recall@K metrics, SLA compliance, and UNLOGGED
interval/transaction samples into PostgreSQL via COPY, plus 5
overloaded SQL inspection/comparison SRFs (pgbench_show_run and
pgbench_compare_runs) and pgbench_viewer_payload_json().
- Patch 0028 (Modular bottle.py + Water.css + WebAssembly Viewer):
* Adds a standalone zero-dependency web telemetry viewer and two-stage
metatemplate SQL Data Explorer (src/bin/pgbench/viewer/) organized
into html/ (10 SimpleTemplate .html.stpl views), css/ (water.css +
viewer.css), wasm/ (439-byte pgbench_chart.wasm SVG charting kernel),
and src/ (C/WAT sources and builder), documented in viewer/README.md.
3. Cross-platform portability & CI fixes (Windows MSVC, MinGW, Meson):
- Added a WIN32 portable temporary file stream fallback
(open_temp_json_stream() / finish_temp_json_stream()) for
serializeConfigJsonToString() and serializeReportJsonToString() in
src/bin/pgbench/logdb.c since POSIX open_memstream() is unavailable on
Windows (MinGW / MSVC).
- Replaced GCC __attribute__((aligned(64))) struct field attributes in
src/bin/pgbench/dataset.h (PgBenchReservoir) and
src/bin/pgbench/dataset.c (PgBenchCausalClock) with explicit cache-line
padding arrays, and added MSVC (_MSC_VER) _Interlocked* and
_ReadWriteBarrier() shims in src/bin/pgbench/pgbench.h for GCC
__atomic_* builtins used in dataset.c, commands.c, engine.c, and
script.c.
- Updated POLL_USING_SELECT in src/bin/pgbench/poller.c to track
write_fds alongside read_fds so non-blocking PGRES_POLLING_WRITING
states wake immediately on Windows select().
- Updated dataset/vector/ground-truth spec parsers in dataset.c to
recognize Windows drive-letter colons (C:\...) and backslash path
separators.
- Pinned deterministic cluster seed 42 in pgbench-vec SQL workloads and
increased Phase 2 multi-script transaction sample count in
001_pgbench_with_server.pl for deterministic CI runs.

Architecture & Series Overview (28 Patches):
--------------------------------------------
The series is organized into ten logical parts across 28 self-contained
patches so each layer can be reviewed and tested independently:

Part 1: Modularize src/bin/pgbench (Patches 0001-0012)
Splits the monolithic ~8,000-line src/bin/pgbench/pgbench.c into focused
compilation units with clean headers and minimal global coupling:
- 0001, 0008: stats.c/h (SimpleStats, StatsData, LatencyHist, mergeStats)
- 0002, 0009: variable.c/h (PgBenchValue, Variable, scoped VariableSpace)
- 0003, 0010: poller.c/h (socket_set abstraction over poll/select)
- 0004, 0011: script.c/h (ParsedScript, expression evaluation, chooseScript)
- 0005, 0012: commands.c/h (meta-command execution & pluggable handler table)
- 0006: init.c/h (database initialization -i)
- 0007: engine.c/h (CState/TState client state machine & thread loop)

Part 2: Shared PRNG/Math Library & contrib/pgbench (Patches 0013-0014)
- 0013: Extracts pgbench's deterministic PRNG (pg_erand48 / Xoshiro),
64-bit FNV/Murmur hashing, uniform/exponential/gaussian/zipfian random
distributions, permute(), TPC-C NURand, syllable generation, random
strings, Pareto/lognormal distributions, and correlated graph/vector
generators into src/common/pgbench_funcs.c (and
src/include/common/pgbench_funcs.h), compiled into both libpgcommon.a
and libpgcommon_shlib.a.
- 0014: Introduces contrib/pgbench (CREATE EXTENSION pgbench), installing
SQL-callable C functions into SCHEMA pgbench so server-side data
generation (INSERT ... SELECT FROM generate_series(...)) produces
values using the exact same algorithms and distributions as the
pgbench client.

Part 3: Rich Scripting & Client State Machine Primitives (Patches 0015-0019)
- 0015: Extends PgBenchValue with text (PGBT_TEXT), timestamp
(PGBT_TIMESTAMP), and 1D array (PGBT_ARRAY) types, with automatic
SQL literal quoting and array subscripting (:arr[:i]).
- 0016: Adds built-in expression functions: nurand(), c_last(),
random_string(), random_alnum(), random_date(), random_timestamp(),
array_length(), array_append(), and format().
- 0017: Adds structured control-flow meta-commands (\for / \endfor,
\while / \endwhile) and block-scoped exception handling (\try /
\catch / \endtry with automatic SAVEPOINT / ROLLBACK TO SAVEPOINT
and serialization/deadlock retry support, plus \expect_error).
- 0018: Adds \gset_meta (capturing rowcount, sqlstate, and backend_pid)
and PostgreSQL array text parsing in \gset and \aset.
- 0019: Extends \sleep to evaluate arbitrary arithmetic expressions and
adds \keying_time and \think_time (log-normal user think times that
sleep without inflating active transaction latency).

Part 4: Multi-Group Orchestration, Telemetry & Standard Suites
(Patches 0020-0022)
- 0020: Adds heterogeneous client groups (--group name:clients=N,
threads=M,rate=R,scripts=...) and sequential execution streams for
concurrent mixed OLTP + DSS/OLAP workloads.
- 0021: Adds power-of-two sub-bucket HDR latency histograms (p50, p90,
p95, p99, p99.9, max), --sla threshold validation, composite throughput
metrics (tpmC, QphH), and machine-readable --json-report output.
- 0022: Adds six self-contained benchmark workload suites under
src/bin/pgbench/workloads/:
* pgbench-b (enhanced TPCB-like banking workload)
* pgbench-c (TPC-C-inspired wholesale supplier OLTP workload)
* pgbench-e (TPC-E-inspired brokerage OLTP workload)
* pgbench-h (TPC-H-inspired decision support workload)
* pgbench-ds (TPC-DS-inspired multi-channel retail DSS workload)
* pgbench-htap (hybrid concurrent OLTP + analytical OLAP workload)

Part 5: Common Horizontal Pillars Tiers 1-3 (Patch 0023)
- 0023: Implements in-memory mmap datasets (dataset.c/h) for zero-copy
loading of .fvecs/.bvecs/.ivecs and .csv/.tsv datasets, PGBT_VECTOR
and PGBT_BYTEA value types, \gset_col, lock-free shared reservoirs
(--reservoir, \reservoir_push, \reservoir_sample), causal watermarks
(\wait_until, \advance_watermark), non-blocking \session_connect /
\session_disconnect, \call / \return subroutines, Recall@K / NDCG@K
accuracy telemetry (\record_recall), schedule lag SLAs, and
deterministic trace replay (--replay), along with reference
documentation in doc/src/sgml/ref/pgbench.sgml and
doc/src/sgml/pgbench-ext.sgml.

Part 6: LDBC-Inspired Graph Benchmark Suites (Patch 0024)
- 0024: Implements correlated power-law graph generators
(ldbc_correlated_edge, ldbc_flashmob_ts), dependent write tracking
(\begin_dependent_write, \end_dependent_write), phased execution
barriers (\phase_barrier), result validation (\validate_result),
SGML reference documentation (doc/src/sgml/ref/pgbench.sgml and
doc/src/sgml/pgbench-ext.sgml), and the unified BIGINT-aligned
LDBC-inspired schema and query suites for LSQB (Q1-Q9), SNB
Interactive v1/v2, SNB BI, FinBench, and Graphalytics across standard
relational SQL (workloads/pgbench-graph/), recursive UNION DISTINCT ON
(workloads/pgbench-graph-sql-ud/), and native ISO SQL/PGQ GRAPH_TABLE
(workloads/pgbench-graph-pgq/).

Part 7: VecBench-Inspired Vector Benchmark Suite & Docs (Patch 0025)
- 0025: Implements the VecBench-inspired vector ANN benchmark suite
under workloads/pgbench-vec/ across 6 operational modes (unfiltered
search, pre-filtered search, post-filtered adaptive oversampling,
streaming ingestion, multi-vector MaxSim late interaction, and
continuous aging/index degradation comparing initial Q1 vs mature Q4
recall), plus --vector-base, --ground-truth-table, named per-stream
recall SLAs, and comprehensive user documentation and trademark/naming
guardrails in src/bin/pgbench/workloads/README.

Part 8: Declarative JSON/JSONC Configuration (Patch 0026)
- 0026: Adds --config=<file> and --dump-config=<file> (src/bin/pgbench/
config.c/h) using PostgreSQL's native src/common/jsonapi.c parser with
a C/C++ comment stripper for JSONC support, multi-stage init_steps,
direct replay of --json-report output files, and 23 ready-to-run
.pgbench.json workload definitions.

Part 9: Direct PostgreSQL Telemetry Logging & SQL Comparison SRFs (Patch 0027)
- 0027: Adds --log-db, --log-db-unlogged[=stream|all|off],
--log-db-mode={stream|deferred}, --run-label, --log-db-schema, and
--log-db-index (src/bin/pgbench/logdb.c/h) to stream run metadata,
per-group/per-script HDR percentiles, Recall@K metrics, SLA compliance,
and high-throughput UNLOGGED per-transaction/per-window samples into a
3-tier, 9-table PostgreSQL schema using COPY, plus 5 overloaded SQL
comparison SRFs (pgbench_show_run, pgbench_compare_runs) and
pgbench_viewer_payload_json().

Part 10: Modular WebAssembly Telemetry Viewer & SQL Data Explorer (Patch 0028)
- 0028: Adds a standalone zero-dependency bottle.py + Water.css +
439-byte WebAssembly SVG chart viewer and two-stage metatemplate SQL
Data Explorer (src/bin/pgbench/viewer/ organized into html/, css/,
wasm/, and src/, with documentation in viewer/README.md and
src/bin/pgbench/README).

Testing:
--------
- `make -C contrib/pgbench check` passes all SQL regression tests for
`SCHEMA pgbench` functions and generators.
- `make -C src/bin/pgbench check` passes all TAP tests in
`t/001_pgbench_with_server.pl` and `t/002_pgbench_no_server.pl`.

Hannu Krosing (28):
Refactor pgbench: extract statistics structures and functions to
stats.c/h
Refactor pgbench: extract variable management and value types to
variable.c/h
Refactor pgbench: extract socket polling abstraction to poller.c/h
Refactor pgbench: extract script parsing, built-in scripts, and
expression evaluation to script.c/h
Refactor pgbench: extract meta-command execution and prepared
statement helpers to commands.c/h
Refactor pgbench: extract database initialization (-i) to init.c/h
Refactor pgbench: extract benchmark execution engine and client state
machine to engine.c/h
Refactor pgbench stats: add mergeStats() and decouple global flags
Refactor pgbench variables: introduce scoped variable store
Refactor pgbench poller: encapsulate socket_set and add poller API
aliases
Refactor pgbench script: decouple chooseScript() from TState
Refactor pgbench commands: introduce pluggable meta-command dispatch
Extract pgbench low-level functions into src/common
Add contrib/pgbench extension exposing benchmark functions in SCHEMA
pgbench
pgbench: Extend PgBenchValue with text, timestamp, and array types
pgbench: Add TPC-C NURand, syllable, string, date/timestamp, and array
functions
pgbench: Add script loops (\for, \while) and exception handling (\try,
\catch, \expect_error)
pgbench: Add \gset_meta and array column parsing in \gset and \aset
pgbench: Support dynamic \sleep expressions, \keying_time, and
\think_time
pgbench: Add heterogeneous client groups (--group) and sequential
streams
pgbench: Add HDR latency percentiles, --sla validation, tpmC/QphH
metrics, and --json-report
pgbench: Add pgbench-b, pgbench-c, pgbench-e, pgbench-h, pgbench-ds,
and pgbench-htap workload suites
pgbench: add mmap datasets, vector/bytea types, reservoirs,
watermarks, \call, Recall@K, and --replay
pgbench: add LDBC-inspired graph suites (SQL, SQL_UD, SQL/PGQ),
generators, and barriers
pgbench: add VecBench-inspired vector ANN suite (Modes 1-6), recall
SLAs, and docs
pgbench: add --config and --dump-config JSON/JSONC workload
configuration support
pgbench: add --log-db PostgreSQL telemetry logging, UNLOGGED stream
tables, and SQL comparison SRFs
pgbench: add modular bottle.py + Water.css + WebAssembly telemetry
viewer

contrib/Makefile | 1 +
contrib/meson.build | 1 +
contrib/pgbench/.gitignore | 7 +
contrib/pgbench/Makefile | 29 +
contrib/pgbench/README | 34 +
contrib/pgbench/expected/pgbench.out | 306 +
contrib/pgbench/meson.build | 35 +
contrib/pgbench/pgbench--1.0.sql | 427 +
contrib/pgbench/pgbench.c | 1034 +++
contrib/pgbench/pgbench.control | 6 +
contrib/pgbench/sql/pgbench.sql | 111 +
doc/src/sgml/contrib.sgml | 2 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/pgbench-ext.sgml | 580 ++
doc/src/sgml/ref/pgbench.sgml | 1062 ++-
src/bin/pgbench/.gitignore | 1 +
src/bin/pgbench/Makefile | 13 +
src/bin/pgbench/README | 217 +
src/bin/pgbench/commands.c | 2057 +++++
src/bin/pgbench/commands.h | 87 +
src/bin/pgbench/config.c | 3052 +++++++
src/bin/pgbench/config.h | 55 +
src/bin/pgbench/dataset.c | 2894 ++++++
src/bin/pgbench/dataset.h | 159 +
src/bin/pgbench/engine.c | 4143 +++++++++
src/bin/pgbench/engine.h | 253 +
src/bin/pgbench/exprparse.y | 234 +-
src/bin/pgbench/exprscan.l | 18 +
src/bin/pgbench/init.c | 828 ++
src/bin/pgbench/init.h | 51 +
src/bin/pgbench/logdb.c | 2735 ++++++
src/bin/pgbench/logdb.h | 77 +
src/bin/pgbench/meson.build | 10 +
src/bin/pgbench/pgbench.c | 8042 ++---------------
src/bin/pgbench/pgbench.h | 470 +-
src/bin/pgbench/poller.c | 287 +
src/bin/pgbench/poller.h | 47 +
src/bin/pgbench/script.c | 4435 +++++++++
src/bin/pgbench/script.h | 198 +
src/bin/pgbench/stats.c | 753 ++
src/bin/pgbench/stats.h | 220 +
src/bin/pgbench/t/001_pgbench_with_server.pl | 1544 ++++
src/bin/pgbench/t/002_pgbench_no_server.pl | 557 ++
src/bin/pgbench/variable.c | 1772 ++++
src/bin/pgbench/variable.h | 177 +
src/bin/pgbench/viewer/README.md | 133 +
src/bin/pgbench/viewer/bottle.py | 3829 ++++++++
src/bin/pgbench/viewer/css/viewer.css | 202 +
src/bin/pgbench/viewer/css/water.css | 81 +
.../viewer/html/compare_matrix.html.stpl | 43 +
.../viewer/html/compare_pair.html.stpl | 49 +
.../viewer/html/explorer_body.html.stpl | 51 +
.../viewer/html/explorer_table_meta.html.stpl | 87 +
src/bin/pgbench/viewer/html/layout.html.stpl | 46 +
.../viewer/html/live_monitor.html.stpl | 100 +
.../pgbench/viewer/html/phase1_body.html.stpl | 30 +
.../viewer/html/run_selector.html.stpl | 120 +
.../pgbench/viewer/html/single_run.html.stpl | 146 +
.../viewer/html/wasm_chart_box.html.stpl | 5 +
.../viewer/html/wasm_chart_module.html.stpl | 246 +
src/bin/pgbench/viewer/pgbench_viewer.py | 704 ++
src/bin/pgbench/viewer/src/build_wasm.py | 216 +
src/bin/pgbench/viewer/src/pgbench_chart.c | 66 +
src/bin/pgbench/viewer/src/pgbench_chart.wat | 105 +
.../pgbench/viewer/wasm/pgbench_chart.wasm | Bin 0 -> 439 bytes
src/bin/pgbench/workloads/README | 511 ++
.../pgbench-b/pgbench_b.pgbench.json | 37 +
.../pgbench-b/pgbench_b_constraints.sql | 17 +
.../workloads/pgbench-b/pgbench_b_init.sql | 32 +
.../workloads/pgbench-b/pgbench_b_schema.sql | 100 +
.../pgbench-b/pgbench_b_select_only.sql | 9 +
.../pgbench-b/pgbench_b_simple_update.sql | 14 +
.../workloads/pgbench-b/pgbench_b_tx.sql | 14 +
.../pgbench-c/pgbench_c.pgbench.json | 53 +
.../pgbench-c/pgbench_c_constraints.sql | 11 +
.../pgbench-c/pgbench_c_delivery.sql | 15 +
.../workloads/pgbench-c/pgbench_c_init.sql | 206 +
.../pgbench-c/pgbench_c_new_order.sql | 41 +
.../pgbench-c/pgbench_c_order_status.sql | 19 +
.../workloads/pgbench-c/pgbench_c_payment.sql | 31 +
.../workloads/pgbench-c/pgbench_c_schema.sql | 470 +
.../pgbench-c/pgbench_c_stock_level.sql | 11 +
.../pgbench-ds/pgbench_ds.pgbench.json | 63 +
.../pgbench-ds/pgbench_ds_constraints.sql | 22 +
.../workloads/pgbench-ds/pgbench_ds_init.sql | 306 +
.../pgbench-ds/pgbench_ds_maintenance.sql | 35 +
.../pgbench-ds/pgbench_ds_q_cross_channel.sql | 61 +
.../pgbench-ds/pgbench_ds_q_inventory.sql | 32 +
.../pgbench-ds/pgbench_ds_q_promotion.sql | 31 +
.../pgbench-ds/pgbench_ds_q_store_sales.sql | 47 +
.../pgbench-ds/pgbench_ds_schema.sql | 214 +
.../pgbench-e/pgbench_e.pgbench.json | 57 +
.../pgbench-e/pgbench_e_broker_volume.sql | 8 +
.../pgbench-e/pgbench_e_constraints.sql | 20 +
.../pgbench-e/pgbench_e_customer_position.sql | 19 +
.../workloads/pgbench-e/pgbench_e_init.sql | 202 +
.../pgbench-e/pgbench_e_market_watch.sql | 12 +
.../workloads/pgbench-e/pgbench_e_schema.sql | 509 ++
.../pgbench-e/pgbench_e_trade_order.sql | 24 +
.../pgbench-e/pgbench_e_trade_result.sql | 14 +
.../pgbench-e/pgbench_e_trade_status.sql | 14 +
.../pgbench-graph-pgq/finbench.pgbench.json | 37 +
.../graphalytics.pgbench.json | 37 +
.../pgbench-graph-pgq/lsqb.pgbench.json | 37 +
.../pgbench_graph_pgq_1hop.sql | 40 +
.../pgbench_graph_pgq_2hop.sql | 38 +
.../pgbench_graph_pgq_add_edge.sql | 14 +
.../pgbench_graph_pgq_constraints.sql | 151 +
.../pgbench_graph_pgq_ddl.sql | 93 +
.../pgbench_graph_pgq_init.sql | 203 +
.../pgbench_graph_pgq_schema.sql | 273 +
.../pgbench_ldbc_pgq_finbench.sql | 178 +
.../pgbench_ldbc_pgq_graphalytics.sql | 186 +
.../pgbench_ldbc_pgq_lsqb.sql | 138 +
.../pgbench_ldbc_pgq_snb_bi.sql | 255 +
.../pgbench_ldbc_pgq_snb_ic.sql | 151 +
.../pgbench_ldbc_pgq_snb_id.sql | 83 +
.../pgbench_ldbc_pgq_snb_is.sql | 141 +
.../pgbench_ldbc_pgq_snb_iu.sql | 87 +
.../pgbench-graph-pgq/snb_bi.pgbench.json | 37 +
.../snb_interactive.pgbench.json | 66 +
.../finbench.pgbench.json | 37 +
.../graphalytics.pgbench.json | 37 +
.../pgbench-graph-sql-ud/lsqb.pgbench.json | 37 +
.../pgbench_graph_sql_ud_1hop.sql | 28 +
.../pgbench_graph_sql_ud_2hop.sql | 39 +
.../pgbench_graph_sql_ud_add_edge.sql | 14 +
.../pgbench_graph_sql_ud_constraints.sql | 151 +
.../pgbench_graph_sql_ud_init.sql | 203 +
.../pgbench_graph_sql_ud_schema.sql | 181 +
.../pgbench_ldbc_sql_ud_finbench.sql | 135 +
.../pgbench_ldbc_sql_ud_graphalytics.sql | 152 +
.../pgbench_ldbc_sql_ud_lsqb.sql | 109 +
.../pgbench_ldbc_sql_ud_snb_bi.sql | 219 +
.../pgbench_ldbc_sql_ud_snb_ic.sql | 125 +
.../pgbench_ldbc_sql_ud_snb_id.sql | 76 +
.../pgbench_ldbc_sql_ud_snb_is.sql | 91 +
.../pgbench_ldbc_sql_ud_snb_iu.sql | 87 +
.../pgbench-graph-sql-ud/snb_bi.pgbench.json | 37 +
.../snb_interactive.pgbench.json | 66 +
.../pgbench-graph/finbench.pgbench.json | 37 +
.../pgbench-graph/graphalytics.pgbench.json | 37 +
.../workloads/pgbench-graph/lsqb.pgbench.json | 37 +
.../pgbench-graph/pgbench_graph_1hop.sql | 28 +
.../pgbench-graph/pgbench_graph_2hop.sql | 39 +
.../pgbench-graph/pgbench_graph_add_edge.sql | 14 +
.../pgbench_graph_constraints.sql | 151 +
.../pgbench-graph/pgbench_graph_init.sql | 203 +
.../pgbench-graph/pgbench_graph_pgq_ddl.sql | 93 +
.../pgbench-graph/pgbench_graph_schema.sql | 181 +
.../pgbench-graph/pgbench_ldbc_finbench.sql | 146 +
.../pgbench_ldbc_graphalytics.sql | 190 +
.../pgbench-graph/pgbench_ldbc_lsqb.sql | 108 +
.../pgbench-graph/pgbench_ldbc_snb_bi.sql | 225 +
.../pgbench-graph/pgbench_ldbc_snb_ic.sql | 130 +
.../pgbench-graph/pgbench_ldbc_snb_id.sql | 76 +
.../pgbench-graph/pgbench_ldbc_snb_is.sql | 91 +
.../pgbench-graph/pgbench_ldbc_snb_iu.sql | 87 +
.../pgbench_ldbc_sql_ud_finbench.sql | 135 +
.../pgbench_ldbc_sql_ud_graphalytics.sql | 152 +
.../pgbench_ldbc_sql_ud_lsqb.sql | 109 +
.../pgbench_ldbc_sql_ud_snb_bi.sql | 219 +
.../pgbench_ldbc_sql_ud_snb_ic.sql | 125 +
.../pgbench_ldbc_sql_ud_snb_id.sql | 76 +
.../pgbench_ldbc_sql_ud_snb_is.sql | 91 +
.../pgbench_ldbc_sql_ud_snb_iu.sql | 87 +
.../pgbench-graph/snb_bi.pgbench.json | 37 +
.../snb_interactive.pgbench.json | 66 +
.../pgbench-h/pgbench_h.pgbench.json | 52 +
.../pgbench-h/pgbench_h_constraints.sql | 20 +
.../workloads/pgbench-h/pgbench_h_init.sql | 187 +
.../workloads/pgbench-h/pgbench_h_q1.sql | 28 +
.../workloads/pgbench-h/pgbench_h_q10.sql | 40 +
.../workloads/pgbench-h/pgbench_h_q12.sql | 36 +
.../workloads/pgbench-h/pgbench_h_q14.sql | 21 +
.../workloads/pgbench-h/pgbench_h_q19.sql | 46 +
.../workloads/pgbench-h/pgbench_h_q3.sql | 32 +
.../workloads/pgbench-h/pgbench_h_q5.sql | 34 +
.../workloads/pgbench-h/pgbench_h_q6.sql | 21 +
.../workloads/pgbench-h/pgbench_h_queries.sql | 37 +
.../workloads/pgbench-h/pgbench_h_refresh.sql | 54 +
.../workloads/pgbench-h/pgbench_h_schema.sql | 105 +
.../pgbench-htap/pgbench_htap.pgbench.json | 70 +
.../pgbench-htap/pgbench_htap_constraints.sql | 15 +
.../pgbench-htap/pgbench_htap_init.sql | 233 +
.../pgbench-htap/pgbench_htap_olap_q1.sql | 19 +
.../pgbench-htap/pgbench_htap_olap_q18.sql | 36 +
.../pgbench-htap/pgbench_htap_olap_q3.sql | 34 +
.../pgbench-htap/pgbench_htap_olap_q5.sql | 39 +
.../pgbench-htap/pgbench_htap_olap_q6.sql | 10 +
.../pgbench-htap/pgbench_htap_oltp.sql | 46 +
.../pgbench-htap/pgbench_htap_schema.sql | 396 +
.../pgbench-vec/pgbench_vec_aging.sql | 79 +
.../pgbench-vec/pgbench_vec_constraints.sql | 26 +
.../pgbench-vec/pgbench_vec_filtered.sql | 19 +
.../pgbench-vec/pgbench_vec_gt_gen.sql | 51 +
.../pgbench-vec/pgbench_vec_init.sql | 34 +
.../pgbench-vec/pgbench_vec_insert.sql | 22 +
.../pgbench-vec/pgbench_vec_multivec.sql | 39 +
.../pgbench-vec/pgbench_vec_postfiltered.sql | 36 +
.../pgbench-vec/pgbench_vec_schema.sql | 91 +
.../pgbench-vec/pgbench_vec_search.sql | 28 +
.../pgbench-vec/vec_aging.pgbench.json | 45 +
.../pgbench-vec/vec_hybrid.pgbench.json | 64 +
src/common/Makefile | 1 +
src/common/meson.build | 1 +
src/common/pgbench_funcs.c | 1687 ++++
src/include/common/pgbench_funcs.h | 117 +
208 files changed, 52414 insertions(+), 7138 deletions(-)
create mode 100644 contrib/pgbench/.gitignore
create mode 100644 contrib/pgbench/Makefile
create mode 100644 contrib/pgbench/README
create mode 100644 contrib/pgbench/expected/pgbench.out
create mode 100644 contrib/pgbench/meson.build
create mode 100644 contrib/pgbench/pgbench--1.0.sql
create mode 100644 contrib/pgbench/pgbench.c
create mode 100644 contrib/pgbench/pgbench.control
create mode 100644 contrib/pgbench/sql/pgbench.sql
create mode 100644 doc/src/sgml/pgbench-ext.sgml
create mode 100644 src/bin/pgbench/README
create mode 100644 src/bin/pgbench/commands.c
create mode 100644 src/bin/pgbench/commands.h
create mode 100644 src/bin/pgbench/config.c
create mode 100644 src/bin/pgbench/config.h
create mode 100644 src/bin/pgbench/dataset.c
create mode 100644 src/bin/pgbench/dataset.h
create mode 100644 src/bin/pgbench/engine.c
create mode 100644 src/bin/pgbench/engine.h
create mode 100644 src/bin/pgbench/init.c
create mode 100644 src/bin/pgbench/init.h
create mode 100644 src/bin/pgbench/logdb.c
create mode 100644 src/bin/pgbench/logdb.h
create mode 100644 src/bin/pgbench/poller.c
create mode 100644 src/bin/pgbench/poller.h
create mode 100644 src/bin/pgbench/script.c
create mode 100644 src/bin/pgbench/script.h
create mode 100644 src/bin/pgbench/stats.c
create mode 100644 src/bin/pgbench/stats.h
create mode 100644 src/bin/pgbench/variable.c
create mode 100644 src/bin/pgbench/variable.h
create mode 100644 src/bin/pgbench/viewer/README.md
create mode 100644 src/bin/pgbench/viewer/bottle.py
create mode 100644 src/bin/pgbench/viewer/css/viewer.css
create mode 100644 src/bin/pgbench/viewer/css/water.css
create mode 100644 src/bin/pgbench/viewer/html/compare_matrix.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/compare_pair.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/explorer_body.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/explorer_table_meta.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/layout.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/live_monitor.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/phase1_body.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/run_selector.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/single_run.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/wasm_chart_box.html.stpl
create mode 100644 src/bin/pgbench/viewer/html/wasm_chart_module.html.stpl
create mode 100755 src/bin/pgbench/viewer/pgbench_viewer.py
create mode 100644 src/bin/pgbench/viewer/src/build_wasm.py
create mode 100644 src/bin/pgbench/viewer/src/pgbench_chart.c
create mode 100644 src/bin/pgbench/viewer/src/pgbench_chart.wat
create mode 100644 src/bin/pgbench/viewer/wasm/pgbench_chart.wasm
create mode 100644 src/bin/pgbench/workloads/README
create mode 100644 src/bin/pgbench/workloads/pgbench-b/pgbench_b.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-b/pgbench_b_constraints.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-b/pgbench_b_init.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-b/pgbench_b_schema.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-b/pgbench_b_select_only.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-b/pgbench_b_simple_update.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-b/pgbench_b_tx.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-c/pgbench_c_constraints.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_delivery.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_init.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_new_order.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-c/pgbench_c_order_status.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_payment.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_schema.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-c/pgbench_c_stock_level.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-ds/pgbench_ds.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_constraints.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_init.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_maintenance.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_q_cross_channel.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_q_inventory.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_q_promotion.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_q_store_sales.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_schema.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-e/pgbench_e.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-e/pgbench_e_broker_volume.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-e/pgbench_e_constraints.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-e/pgbench_e_customer_position.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-e/pgbench_e_init.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-e/pgbench_e_market_watch.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-e/pgbench_e_schema.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-e/pgbench_e_trade_order.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-e/pgbench_e_trade_result.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-e/pgbench_e_trade_status.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/finbench.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/graphalytics.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/lsqb.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_1hop.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_2hop.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_add_edge.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_constraints.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_ddl.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_init.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_schema.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_finbench.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_graphalytics.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_lsqb.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_snb_bi.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_snb_ic.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_snb_id.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_snb_is.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_snb_iu.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/snb_bi.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-pgq/snb_interactive.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/finbench.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/graphalytics.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/lsqb.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_1hop.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_2hop.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_add_edge.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_constraints.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_init.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_schema.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_finbench.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_graphalytics.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_lsqb.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_snb_bi.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_snb_ic.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_snb_id.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_snb_is.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_snb_iu.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/snb_bi.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph-sql-ud/snb_interactive.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/finbench.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/graphalytics.pgbench.json
create mode 100644 src/bin/pgbench/workloads/pgbench-graph/lsqb.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_1hop.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_2hop.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_add_edge.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_constraints.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_init.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_pgq_ddl.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_schema.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_finbench.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_graphalytics.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_lsqb.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_snb_bi.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_snb_ic.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_snb_id.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_snb_is.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_snb_iu.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_finbench.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_graphalytics.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_lsqb.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_snb_bi.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_snb_ic.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_snb_id.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_snb_is.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_snb_iu.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-graph/snb_bi.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-graph/snb_interactive.pgbench.json
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-h/pgbench_h_constraints.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_init.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q1.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q10.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q12.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q14.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q19.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q3.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q5.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q6.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_queries.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_refresh.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_schema.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-htap/pgbench_htap.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_constraints.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_init.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_olap_q1.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_olap_q18.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_olap_q3.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_olap_q5.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_olap_q6.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_oltp.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_schema.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_aging.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_constraints.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_filtered.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_gt_gen.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_init.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_insert.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_multivec.sql
create mode 100644
src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_postfiltered.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_schema.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_search.sql
create mode 100644 src/bin/pgbench/workloads/pgbench-vec/vec_aging.pgbench.json
create mode 100644
src/bin/pgbench/workloads/pgbench-vec/vec_hybrid.pgbench.json
create mode 100644 src/common/pgbench_funcs.c
create mode 100644 src/include/common/pgbench_funcs.h

--
2.56.0

Show quoted text

On Sun, Oct 18, 2026 at 5:16 PM Hannu Krosing <hannuk@google.com> wrote:

[PATCH v5 00/22]
* Modular pgbench refactoring,
* contrib/pgbench extension,
* advanced benchmark capabilities,
* and standard workload suites (pgbench-b, pgbench-c, pgbench-e,
pgbench-h, pgbench-ds, pgbench-htap)

Hi hackers,

This patch series presents a comprehensive modernization of pgbench,
transforming
it from a monolithic single-script driver into a modern, modular, multi-workload
benchmarking engine with native support for complex OLTP, OLAP, and
HTAP benchmarks.

See https://wiki.postgresql.org/wiki/Pgbench-modernization-2026 for
more background and rationale.

The series is organized into five logical parts across 22 clean, self-contained
commits rebased cleanly on latest master:

================================================================================
Part 1: Modular pgbench Refactoring (Patches 0001 - 0012)
================================================================================
pgbench.c had grown to over 7,400 lines combining distinct responsibilities:
statistics accumulation, variable management, event polling, script parsing,
meta-command execution, database initialization (-i), and the multi-threaded
state machine.

Patches 0001-0007 extract these modules without altering behavior or
public APIs:
- stats.c / stats.h: StatsData, latency tally, rate-limiting logic.
- variable.c / variable.h: Scoped variable store and value evaluation.
- poller.c / poller.h: Socket set polling abstraction (select/poll).
- script.c / script.h: Script parsing, AST nodes, expression evaluator.
- commands.c / commands.h: Meta-command handlers and prepared
statement helpers.
- init.c / init.h: Database initialization (-i) logic and data generators.
- engine.c / engine.h: Multi-threaded client state machine and event loop.

Patches 0008-0012 decouple remaining global dependencies cleanly:
- stats: add mergeStats() and decouple global flags.
- variable: introduce scoped variable store (VariableSpace).
- poller: encapsulate socket_set and provide poller API wrappers.
- script: decouple chooseScript() from TState.
- commands: introduce pluggable meta-command dispatch table.

================================================================================
Part 2: Code Extraction & In-Database Functions (Patches 0013 - 0014)
================================================================================
Patch 0013 extracts pgbench's mathematical and pseudo-random
generators (Gaussian,
Exponential, Zipfian, Permute, MurmurHash64) into src/common/pgbench_funcs.c,
making them reusable across both client and server binaries.

Patch 0014 introduces contrib/pgbench, a server-side extension that
exposes these
pseudo-random generators and helper functions directly to SQL under
SCHEMA pgbench.
This allows high-performance parallel in-database synthetic data
generation using
standard SQL (e.g. generate_series()) without relying on slow client roundtrips.

================================================================================
Part 3: Advanced Types, Functions, Control Flow & Pacing (Patches 0015 - 0019)
================================================================================
Patch 0015 extends PgBenchValue with text, timestamp, and array types,
along with
array operations (array_append, array_length, array_get, array_generate), text
concatenation, and timestamp arithmetic.

Patch 0016 adds standard benchmark helper functions to pgbench expressions and
contrib/pgbench:
- nurand(A, s, x, y): Non-uniform random distribution generator.
- c_last(num): Syllable-based customer last name generator.
- random_astring(min, max), random_nstring(min, max): Alphanumeric
and numeric strings.
- format_string(fmt, ...), date_add(ts, amt, unit), int_to_date(int).

Patch 0017 adds script control flow and exception handling:
- Loops: \for <var> <start> .. <end> [BY <step>], \while <expr>,
\break, \continue.
- Exception handling: \try ... \catch [<sqlstate>] ... \endtry,
\expect_error <sqlstate>.

Patch 0018 adds result set metadata inspection and array column parsing:
- \gset_meta <prefix>: Extracts query result status, error code,
rows affected, and timing.
- \aset and array-aware \gset: Maps multi-row query results into
client-side array variables.

Patch 0019 introduces dynamic client pacing:
- Dynamic \sleep expressions (e.g. \sleep random_exponential(1, 1000, 10) ms).
- \keying_time and \think_time commands for realistic transaction
pacing per specification.

================================================================================
Part 4: Heterogeneous Groups, Latency Percentiles & Reporting (Patches
0020 - 0021)
================================================================================
Patch 0020 introduces heterogeneous client groups via --group:
- Run concurrent mixes of different scripts with distinct client
counts and rates
(e.g., --group name=oltp,clients=10,weight=100 --group
name=olap,clients=2,sequential=true).
- Supports sequential stream execution for multi-query analytical streams.

Patch 0021 implements High Dynamic Range (HDR) latency histogram tracking:
- Configurable percentiles (p50, p90, p95, p99, p99.9).
- SLA validation via --sla target=<metric>,threshold=<value>.
- Metric computation for tpmC and QphH@SF.
- Structured machine-readable JSON output via --json-report=<path>.

================================================================================
Part 5: Standard Workload Suites (Patch 0022)
================================================================================

<THIS WAS LEFT OUT FROM THIS EMAIL>

* Trademark & Naming Rationale:
These workloads are inspired by and modeled after industry-standard
specifications
(TPC-C, TPC-E, TPC-H, TPC-DS, and CH-benCHmark). To respect and
avoid infringing
upon Transaction Processing Performance Council (TPC) trademarks,
and to clearly
distinguish these community engineering tools from official
council-audited benchmarks,
they are explicitly named pgbench-b, pgbench-c, pgbench-e,
pgbench-h, pgbench-ds, and pgbench-htap.
Full background and guidelines are documented in
src/bin/pgbench/workloads/README.

* In-Database Parallel Loading:
Each workload includes an initialization script
(<workload>_init.sql) designed to
run in parallel across multiple worker sessions (-c N -j N -t 1 -D
num_workers=N).
Data generation is executed entirely within the database engine
using generate_series()
and contrib/pgbench SQL functions, achieving rapid parallel load
rates without external
file generation or ETL binaries.

Comprehensive documentation is provided in doc/src/sgml/ref/pgbench.sgml and
doc/src/sgml/pgbench-ext.sgml, and full end-to-end TAP coverage is added in
src/bin/pgbench/t/001_pgbench_with_server.pl (passing all tests across
all platforms).

Feedback, testing, and reviews are warmly welcomed!

Hannu Krosing (22):
Refactor pgbench: extract statistics structures and functions to
stats.c/h
Refactor pgbench: extract variable management and value types to
variable.c/h
Refactor pgbench: extract socket polling abstraction to poller.c/h
Refactor pgbench: extract script parsing, built-in scripts, and
expression evaluation to script.c/h
Refactor pgbench: extract meta-command execution and prepared
statement helpers to commands.c/h
Refactor pgbench: extract database initialization (-i) to init.c/h
Refactor pgbench: extract benchmark execution engine and client state
machine to engine.c/h
Refactor pgbench stats: add mergeStats() and decouple global flags
Refactor pgbench variables: introduce scoped variable store
Refactor pgbench poller: encapsulate socket_set and add poller API
aliases
Refactor pgbench script: decouple chooseScript() from TState
Refactor pgbench commands: introduce pluggable meta-command dispatch
Extract pgbench low-level functions into src/common
Add contrib/pgbench extension exposing benchmark functions in SCHEMA
pgbench
pgbench: Extend PgBenchValue with text, timestamp, and array types
pgbench: Add TPC-C NURand, syllable, string, date/timestamp, and array
functions
pgbench: Add script loops (\for, \while) and exception handling (\try,
\catch, \expect_error)
pgbench: Add \gset_meta and array column parsing in \gset and \aset
pgbench: Support dynamic \sleep expressions, \keying_time, and
\think_time
pgbench: Add heterogeneous client groups (--group) and sequential
streams
pgbench: Add HDR latency percentiles, --sla validation, tpmC/QphH
metrics, and --json-report
pgbench: Add pgbench-b, pgbench-c, pgbench-e, pgbench-h, pgbench-ds,
and pgbench-htap workload suites

---
Hannu

#9Hannu Krosing
hannuk@google.com
In reply to: Koshi Shibagaki (Fujitsu) (#8)
Re: [PATCH] Refactor pgbench to make future improvements easier

On Fri, Oct 23, 2026 at 12:28 PM Koshi Shibagaki (Fujitsu)
<shibagaki.koshi@fujitsu.com> wrote:

Hi Hannu,

Thank you for patches.
I'm interested in the pgbench refactoring.

Thans you

The current series includes code movement, changes to internal
interfaces, and new features. To make review easier, could we focus
this thread on moving the existing code into separate modules while
preserving existing behavior?

The reason they are all together is that I did not want to post six or
seven separate patch sets,of increasing length, each next one
including all the previous ones to make CI happy.

And yes, I was expecting the review to take place in logical patches,
as described in the email.

For v6, patches 0001-0007 seem like a suitable scope for that review.

yes, these should be just pure code movement with only strictly needed
additions to make this movement possible.

I have left 0008-0012 out of this scope because they go beyond code
movement and include changes to internal interfaces and structures.

Agreed, better leave thes out for now.

Those changes could be discussed separately, along with the new
features.

The logical next patch after the pure file splitting in series would
be the one extracting functions into their own file and making them
also available as an extension.

But let's discuss that once the somple file-level refactoring is done

I'd like to help review the code movement in 0001-0007.

Yes please, absolutely.

Very much appreciated!

-----
Hannu

#10Zsolt Parragi
zsolt.parragi@percona.com
In reply to: Hannu Krosing (#9)
Re: [PATCH] Refactor pgbench to make future improvements easier

On Fri, 23 Oct 2026, Koshi Shibagaki (Fujitsu) <shibagaki.koshi@fujitsu.com> wrote:

For v6, patches 0001-0007 seem like a suitable scope for that review.
I have left 0008-0012 out of this scope because they go beyond code
movement and include changes to internal interfaces and structures.
Those changes could be discussed separately, along with the new
features.

+1

As I promised at pgconfeu, I tried to take another look at 1-7 later to see if they are properly code-movement commits.

But generally I would suggest the same (and what was also suggested in the other thread[1]/messages/by-id/atwDmsIb9TYfg9uH@paquier.xyz): put the idea, "overall PoC" part on github or another git hosting, resending the the entire patchset every time doesn't help.

A simple link and summary would be enough in this thread for anybody interested in it to take a look at it.

I am sure people could argue about just the first 7 patches, where to put which function and such, and constantly rebasing and resending 27 patches doesn't seem to be that useful to me. It also seems unlikely for somebody to commit this in one go, so from this perspective, keeping a single commitfest entry for everything is also wrong.

That could also help you to focus on the quality of the first patches.

One thing I immediately noticed is that the summary of 0004 is strange:

src/bin/pgbench/Makefile | 1 +
src/bin/pgbench/meson.build | 1 +
src/bin/pgbench/pgbench.c | 6419 +++++++++++------------------------
src/bin/pgbench/pgbench.h | 253 +-
src/bin/pgbench/script.c | 1946 +++++++++++
src/bin/pgbench/script.h | 170 +
6 files changed, 4434 insertions(+), 4356 deletions(-)

If we are only moving code out from pgbench.c, why do we see thousands of added lines to it? So far I only took a quick glance at it, but seems like it still mixes things together: internal reorganization (such as commandFailed is now at a different place in the file), and moving code out of it.

The reorganization part isn't mentioned anywhere in the commit message:

Move the script/command data structures (MetaCommand, QueryMode, Command,
ParsedScript, BuiltinScript), PRNG/distribution/hashing/permutation
functions, expression evaluator, script selector, and SQL/backslash
script parser out of pgbench.c into script.c and script.h. Also move the
shared client/thread state types (ConnectionStateEnum, CState, TStatus,
TState) into pgbench.h so both pgbench.c and script.c can access them.

I would probably argue that this part of the really long commit message is already two things and should be two or more separate patches to reduce the size (script.c itself is 2000 lines), and it clearly doesn't mention "reorganizing pgbench.c" anywhere.

And it shouldn't, I still think one patchfile should focus on a single well defined change, that can be described in a few sentences.

Alternatively, since we are talking about simple reorganization, maybe it shouldn't even start with patches? Maybe a table (CSV?) of what functions to move where, and/or some text based diagram format like mermaid to visualize the architecture/structure you want to get, if you want to talk about the long term vision?

A properly written pitch/design document could be an easier way for people to argue about it, and then it could help solve the question of how something this large can get committed?

[1]: /messages/by-id/atwDmsIb9TYfg9uH@paquier.xyz

#11Hannu Krosing
hannuk@google.com
In reply to: Zsolt Parragi (#10)
Re: [PATCH] Refactor pgbench to make future improvements easier

Thanks for detailed explanation.

I will start with a high level reorganizing and refactoring plan and a
diagram illustrating what I want to achieve.

And maybe even before that a more detailed description of the current
state to make the plan and the end goals easier to to discuss.

And until we have the ability in commitfest to link patches I will
provide instructions on how to get the actual pghackers-style patches
from my github if it gets to that point.

------------
Hannu

Show quoted text

On Sat, Oct 24, 2026 at 6:36 AM Zsolt Parragi <zsolt.parragi@percona.com> wrote:

On Fri, 23 Oct 2026, Koshi Shibagaki (Fujitsu)
<shibagaki.koshi@fujitsu.com> wrote:

For v6, patches 0001-0007 seem like a suitable scope for that review.
I have left 0008-0012 out of this scope because they go beyond code
movement and include changes to internal interfaces and structures.
Those changes could be discussed separately, along with the new
features.

+1

As I promised at pgconfeu, I tried to take another look at 1-7 later
to see if they are properly code-movement commits.

But generally I would suggest the same (and what was also suggested in
the other thread[1]): put the idea, "overall PoC" part on github or
another git hosting, resending the the entire patchset every time
doesn't help.

A simple link and summary would be enough in this thread for anybody
interested in it to take a look at it.

I am sure people could argue about just the first 7 patches, where to
put which function and such, and constantly rebasing and resending 27
patches doesn't seem to be that useful to me. It also seems unlikely
for somebody to commit this in one go, so from this perspective,
keeping a single commitfest entry for everything is also wrong.

That could also help you to focus on the quality of the first patches.

One thing I immediately noticed is that the summary of 0004 is strange:

src/bin/pgbench/Makefile | 1 +
src/bin/pgbench/meson.build | 1 +
src/bin/pgbench/pgbench.c | 6419 +++++++++++------------------------
src/bin/pgbench/pgbench.h | 253 +-
src/bin/pgbench/script.c | 1946 +++++++++++
src/bin/pgbench/script.h | 170 +
6 files changed, 4434 insertions(+), 4356 deletions(-)

If we are only moving code out from pgbench.c, why do we see thousands
of added lines to it? So far I only took a quick glance at it, but
seems like it still mixes things together: internal reorganization
(such as commandFailed is now at a different place in the file), and
moving code out of it.

The reorganization part isn't mentioned anywhere in the commit message:

Move the script/command data structures (MetaCommand, QueryMode, Command,
ParsedScript, BuiltinScript), PRNG/distribution/hashing/permutation
functions, expression evaluator, script selector, and SQL/backslash
script parser out of pgbench.c into script.c and script.h. Also move the
shared client/thread state types (ConnectionStateEnum, CState, TStatus,
TState) into pgbench.h so both pgbench.c and script.c can access them.

I would probably argue that this part of the really long commit
message is already two things and should be two or more separate
patches to reduce the size (script.c itself is 2000 lines), and it
clearly doesn't mention "reorganizing pgbench.c" anywhere.

And it shouldn't, I still think one patchfile should focus on a single
well defined change, that can be described in a few sentences.

Alternatively, since we are talking about simple reorganization, maybe
it shouldn't even start with patches? Maybe a table (CSV?) of what
functions to move where, and/or some text based diagram format like
mermaid to visualize the architecture/structure you want to get, if
you want to talk about the long term vision?

A properly written pitch/design document could be an easier way for
people to argue about it, and then it could help solve the question of
how something this large can get committed?

[1]: /messages/by-id/atwDmsIb9TYfg9uH@paquier.xyz

#12Hannu Krosing
hannuk@google.com
In reply to: Zsolt Parragi (#10)
Re: [PATCH] Refactor pgbench to make future improvements easier

Hi Zsolt

I have now extracted only the pure mo
Subject: [PATCH v7 00/10]

Hi Zsolt, hackers,

I have now moved the fast-paced do-everything development into a
separate github repository.

But as I still think refactoring the single-file pgbench.c into
functional modules is useful for anyone wanting to improve pgbench, so
here is a patch set to do just that.

I also split it into a few more modules than the original patch set.

I plan to follow up with patches to extract the user-callable
functions and make them available in both pgbench app and inside
PostgreSQL as the pgbench extension.

Following the discussion on splitting pgbench.c into focused modules before
layering any API, encapsulation, or feature changes on top, this v7 series
is scoped strictly to pure mechanical code and data movement across 10
focused patches (0001-0010):

0001: stats.c / stats.h (statistics structures, timing
helpers, accumulators)
0002: variable.c / variable.h (PgBenchValue, client variables, type coercion)
0003: poller.c / poller.h (ppoll(2) / select(2) socket-set abstraction)
0004: random.c / random.h (PRNG state, distributions,
FNV-1a/Murmur2, permute)
0005: expr.c / expr.h (recursive expression AST & built-in
function evaluator)
0006: script.c / script.h (SQL/backslash script parser &
built-in TPC-B scripts)
0007: commands.c / commands.h (shell runner, prepared helpers,
\sleep, meta-commands)
0008: init.c / init.h (database initialization -i DDL & data
population)
0009: engine.c / engine.h (connections,
advanceConnectionState(), threadRun(), alarms)
0010: report.c / report.h (doLog(), processXactStats(),
progress, printResults())

After patch 0010, src/bin/pgbench/pgbench.c is reduced from 8,069 lines to
931 lines containing only CLI option globals, usage(), and main().

Strict Mechanical Movement Invariants (0001-0010):
--------------------------------------------------
1. Zero changes to function bodies or signatures:
Every one of the 136 function definitions in master:src/bin/pgbench/pgbench.c
(129 unique functions plus 7 #ifdef-paired definitions for the 6 poller.c
functions and setalarm) has an identical body, parameter list, return type,
and const qualification at every commit 0001 through 0010.

2. Minimal linkage adjustments required by C99 for multi-TU compilation:
- The `static` specifier is removed ONLY on the 66 unique functions (73
definitions counting #ifdef pairs) and shared global variables that are
defined in one .c file and referenced from another .c translation unit,
with matching `extern` declarations in the corresponding module .h file.
- All 57 single-translation-unit helper definitions (plus the 2 `static
inline` timing helpers pg_time_now() and pg_time_now_lazy() placed
verbatim in stats.h) retain their `static` linkage.
- The 4 functions that were already non-static in master (strtoint64,
strtodouble, syntax_error, and main) keep their existing external linkage.

3. Monotonic, zero-reordering diffs in pgbench.c:
No remaining code in pgbench.c is reordered at any step, so each patch's
diff against pgbench.c consists solely of `#include` updates, `static`
removals on cross-TU symbols, and contiguous block deletions.

Changes in v7:
--------------
- Rebased onto master (77705d1e5a6).
- Isolated the pure mechanical code-movement extraction into its own
standalone 10-patch series (0001-0010), with zero functional, signature,
const-qualification, or encapsulation changes mixed in.
- Split the former monolithic script/expression/random extraction patch into
three smaller, single-responsibility patches:
* 0004 (random.c/h): PRNG state, uniform/exponential/Gaussian/Poisson/Zipfian
distributions, FNV-1a and Murmur2 hashes, and permute().
* 0005 (expr.c/h): recursive expression AST and built-in function evaluator.
* 0006 (script.c/h): SQL and backslash meta-command script parsing and
built-in script definitions.
- Split reporting and transaction logging out of the engine extraction patch
into patch 0010 (report.c/h: doLog(), processXactStats(),
printProgressReport(), printVersion(), and printResults()).

Show quoted text

On Sat, Oct 24, 2026 at 6:36 AM Zsolt Parragi <zsolt.parragi@percona.com> wrote:

On Fri, 23 Oct 2026, Koshi Shibagaki (Fujitsu)
<shibagaki.koshi@fujitsu.com> wrote:

For v6, patches 0001-0007 seem like a suitable scope for that review.
I have left 0008-0012 out of this scope because they go beyond code
movement and include changes to internal interfaces and structures.
Those changes could be discussed separately, along with the new
features.

+1

As I promised at pgconfeu, I tried to take another look at 1-7 later
to see if they are properly code-movement commits.

But generally I would suggest the same (and what was also suggested in
the other thread[1]): put the idea, "overall PoC" part on github or
another git hosting, resending the the entire patchset every time
doesn't help.

A simple link and summary would be enough in this thread for anybody
interested in it to take a look at it.

I am sure people could argue about just the first 7 patches, where to
put which function and such, and constantly rebasing and resending 27
patches doesn't seem to be that useful to me. It also seems unlikely
for somebody to commit this in one go, so from this perspective,
keeping a single commitfest entry for everything is also wrong.

That could also help you to focus on the quality of the first patches.

One thing I immediately noticed is that the summary of 0004 is strange:

src/bin/pgbench/Makefile | 1 +
src/bin/pgbench/meson.build | 1 +
src/bin/pgbench/pgbench.c | 6419 +++++++++++------------------------
src/bin/pgbench/pgbench.h | 253 +-
src/bin/pgbench/script.c | 1946 +++++++++++
src/bin/pgbench/script.h | 170 +
6 files changed, 4434 insertions(+), 4356 deletions(-)

If we are only moving code out from pgbench.c, why do we see thousands
of added lines to it? So far I only took a quick glance at it, but
seems like it still mixes things together: internal reorganization
(such as commandFailed is now at a different place in the file), and
moving code out of it.

The reorganization part isn't mentioned anywhere in the commit message:

Move the script/command data structures (MetaCommand, QueryMode, Command,
ParsedScript, BuiltinScript), PRNG/distribution/hashing/permutation
functions, expression evaluator, script selector, and SQL/backslash
script parser out of pgbench.c into script.c and script.h. Also move the
shared client/thread state types (ConnectionStateEnum, CState, TStatus,
TState) into pgbench.h so both pgbench.c and script.c can access them.

I would probably argue that this part of the really long commit
message is already two things and should be two or more separate
patches to reduce the size (script.c itself is 2000 lines), and it
clearly doesn't mention "reorganizing pgbench.c" anywhere.

And it shouldn't, I still think one patchfile should focus on a single
well defined change, that can be described in a few sentences.

Alternatively, since we are talking about simple reorganization, maybe
it shouldn't even start with patches? Maybe a table (CSV?) of what
functions to move where, and/or some text based diagram format like
mermaid to visualize the architecture/structure you want to get, if
you want to talk about the long term vision?

A properly written pitch/design document could be an easier way for
people to argue about it, and then it could help solve the question of
how something this large can get committed?

[1]: /messages/by-id/atwDmsIb9TYfg9uH@paquier.xyz

Attachments:

t254037_12
v7-0001-Refactor-pgbench-extract-statistics-structures-an.patchapplication/x-patch; name=v7-0001-Refactor-pgbench-extract-statistics-structures-an.patchDownload+355-305
v7-0002-Refactor-pgbench-extract-variable-management-and-.patchapplication/x-patch; name=v7-0002-Refactor-pgbench-extract-variable-management-and-.patchDownload+739-682
v7-0003-Refactor-pgbench-extract-socket-polling-abstracti.patchapplication/x-patch; name=v7-0003-Refactor-pgbench-extract-socket-polling-abstracti.patchDownload+267-230
v7-0004-Refactor-pgbench-extract-PRNG-distributions-hashi.patchapplication/x-patch; name=v7-0004-Refactor-pgbench-extract-PRNG-distributions-hashi.patchDownload+432-376
v7-0005-Refactor-pgbench-extract-expression-and-built-in-.patchapplication/x-patch; name=v7-0005-Refactor-pgbench-extract-expression-and-built-in-.patchDownload+965-917
v7-0006-Refactor-pgbench-extract-script-parsing-and-built.patchapplication/x-patch; name=v7-0006-Refactor-pgbench-extract-script-parsing-and-built.patchDownload+1139-1078
v7-0007-Refactor-pgbench-extract-meta-command-execution-a.patchapplication/x-patch; name=v7-0007-Refactor-pgbench-extract-meta-command-execution-a.patchDownload+551-500
v7-0008-Refactor-pgbench-extract-database-initialization-.patchapplication/x-patch; name=v7-0008-Refactor-pgbench-extract-database-initialization-.patchDownload+892-830
v7-0009-Refactor-pgbench-extract-benchmark-execution-engi.patchapplication/x-patch; name=v7-0009-Refactor-pgbench-extract-benchmark-execution-engi.patchDownload+1903-1821
v7-0010-Refactor-pgbench-extract-transaction-logging-prog.patchapplication/x-patch; name=v7-0010-Refactor-pgbench-extract-transaction-logging-prog.patchDownload+598-545
#13Dmitry Dolgov
9erthalion6@gmail.com
In reply to: Hannu Krosing (#12)
Re: [PATCH] Refactor pgbench to make future improvements easier

On Sun, Nov 01, 2026 at 06:27:35PM +0100, Hannu Krosing wrote:

But as I still think refactoring the single-file pgbench.c into
functional modules is useful for anyone wanting to improve pgbench, so
here is a patch set to do just that.

I've got stumbled upon this thread and wanted to look in details out of
pure curiosity and without any previous context. Agree with other
commenters regarding the state of this proposal. On top of that the
motivation for splitting pgbench.c seems to be a bit weak to me -- I
don't think a huge source code file is good in any sense, but we've got
some number of those and splitting them only for the sake of making
modules smaller feels a bit pointless.

How about this -- if you feel strong about this patch series, maybe
let's pick up a single feature out of those you want to propose, isolate
any refactoring needed explicitely to support it and go with the feature
as a motivation? I'm not talking about the original patch set from the
first message in this thread, but something much smaller and digestible.

#14Koshi Shibagaki (Fujitsu)
shibagaki.koshi@fujitsu.com
In reply to: Dmitry Dolgov (#13)
Re: [PATCH] Refactor pgbench to make future improvements easier

Hi

How about this -- if you feel strong about this patch series, maybe
let's pick up a single feature out of those you want to propose, isolate
any refactoring needed explicitely to support it and go with the feature
as a motivation? I'm not talking about the original patch set from the
first message in this thread, but something much smaller and digestible.

I agree with Dmitry's suggestion to pick one feature and limit the refactoring
to what is needed to support it.

Previously I suggested focusing on code movement, but a concrete use case
would help us judge the module boundaries and the scope of the changes.

Alternatively, since we are talking about simple reorganization, maybe
it shouldn't even start with patches? Maybe a table (CSV?) of what
functions to move where, and/or some text based diagram format like
mermaid to visualize the architecture/structure you want to get, if
you want to talk about the long term vision?

I also think Zsolt's suggestion of using a table or diagram could help us discuss
the proposed structure for that feature before working on the patches.

IIUC, the earlier proposal [1]/messages/by-id/CAMT0RQQ1Y47QSeLkVfF5SfanJMpDU+2QBPjDLXaBkB7PMVQXOQ@mail.gmail.com includes extracting shared implementations of
distribution generators, permutation functions, and hash functions, and exposing
them through a server-side extension.

For example, how about starting with the distribution generators and limiting
the scope to those? Generating skewed test data in SQL using pgbench’s distribution
algorithms would give us a concrete reason for that refactoring.

[1]: /messages/by-id/CAMT0RQQ1Y47QSeLkVfF5SfanJMpDU+2QBPjDLXaBkB7PMVQXOQ@mail.gmail.com

Koshi Shibagaki
FUJITSU LIMITED
https://www.fujitsu.com/