Material node doesn't show "Maximum Storage" for parallel workers
Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.
This thread has been committed, so CI has stopped here. Anything below is the last result it produced.
You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:
docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t254085psql -h localhost -U postgresBuilt from patchset v1 (message #1), October 12, 2026 at 07:35 AM.
Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:
git clone --branch t254085_1 https://github.com/hackorum-dev/postgres.gitIn a checkout you already have, add the fork once:
git remote add hackorum https://github.com/hackorum-dev/postgres.gitthen, for this patchset and every later one:
git fetch hackorum t254085_1 && git checkout t254085_1Patchset v1 (message #1) is on t254085_1
(This addresses a deficiency in Material node's "Maximum Storage"
output, as highlighted in [1]/messages/by-id/CAHza6qcJUOe4DnSYcKU-VX+7Y8JGbXTyuWj53epskGsOcbzeeA@mail.gmail.com, which proposes adding similar telemetry
to FunctionScan)
1eff8279d "Maximum Storage" for Material nodes, but I failed to think
about getting these details from parallel workers. That's probably not
the end of the world, but it does have the effect of the storage
details going missing when using debug_parallel_query = regress;
Here's a test case I borrowed from select_parallel.sql:
explain analyze
select * from tenk1 t1, tenk2 t2 where t1.two > t2.two; -- correctly
shows Maximum Storage
set debug_parallel_query = regress;
explain analyze
select * from tenk1 t1, tenk2 t2 where t1.two > t2.two; -- Maximum
Storage is missing.
The query uses explain without analyze in select_parallel.sql, so we
don't get any variations from buildfarm members running
debug_parallel_query = regress.
The attached patch aims to fix this. It seemed mostly boilerplate
stuff, so I got Claude Code to write it, and it did so by following
what Memoize does. I only manually adjusted the tuplestore_get_stats()
API afterwards to resolve the issue with transferring the pointer to
the string const into shared memory, plus manual review.
You can test the true parallel query output by running the following setup:
set debug_parallel_query = off;
set parallel_setup_cost=0;
set parallel_tuple_cost=0;
set min_parallel_table_scan_size=0;
set max_parallel_workers_per_gather=4;
set parallel_leader_participation = off;
alter table tenk1 set (parallel_workers = 4);
alter table tenk2 set (parallel_workers = 0);
Doing that, running the query outputs:
-> Materialize ...
Buffers: shared hit=1380
Worker 0: Storage: Memory Maximum Storage: 2849kB
Worker 1: Storage: Memory Maximum Storage: 2849kB
Worker 2: Storage: Memory Maximum Storage: 2849kB
Worker 3: Storage: Memory Maximum Storage: 2849kB
1eff8279d was added in v18, and you could argue this patch is a bug
fix for the missing information in debug_parallel_query = regress, but
also a new feature to show Material telemetry for parallel workers.
Per how debug_parallel_query works, it's really the same thing.
I think this should be master-only. No backpatch.
Any disagreements?
David
[1]: /messages/by-id/CAHza6qcJUOe4DnSYcKU-VX+7Y8JGbXTyuWj53epskGsOcbzeeA@mail.gmail.com
Hi David,
Thank you for the patch!
I reviewed v1. I confirmed the following, so LGTM.
- With debug_parallel_query = regress, the Storage line, which didn't
appear without the patch, is now shown.
- When run in parallel, the Storage lines for Workers 0 to 3 are shown,
and with parallel_leader_participation = on, the leader's line is
shown as well.
1eff8279d was added in v18, and you could argue this patch is a bug
fix for the missing information in debug_parallel_query = regress, but
also a new feature to show Material telemetry for parallel workers.
Per how debug_parallel_query works, it's really the same thing.I think this should be master-only. No backpatch.
I agree with not backpatching this.
While reviewing, I noticed that ExecEndMemoize() overwrites the worker's
slot with memcpy(), so when the Gather is rescanned, only the stats from
the last execution remain. I plan to post a patch in a separate thread
to accumulate the stats instead.
Regards,
Tatsuya Kawata
On Oct 7, 2026, at 09:11, David Rowley <dgrowleyml@gmail.com> wrote:
(This addresses a deficiency in Material node's "Maximum Storage"
output, as highlighted in [1], which proposes adding similar telemetry
to FunctionScan)1eff8279d "Maximum Storage" for Material nodes, but I failed to think
about getting these details from parallel workers. That's probably not
the end of the world, but it does have the effect of the storage
details going missing when using debug_parallel_query = regress;Here's a test case I borrowed from select_parallel.sql:
explain analyze
select * from tenk1 t1, tenk2 t2 where t1.two > t2.two; -- correctly
shows Maximum Storageset debug_parallel_query = regress;
explain analyze
select * from tenk1 t1, tenk2 t2 where t1.two > t2.two; -- Maximum
Storage is missing.The query uses explain without analyze in select_parallel.sql, so we
don't get any variations from buildfarm members running
debug_parallel_query = regress.The attached patch aims to fix this. It seemed mostly boilerplate
stuff, so I got Claude Code to write it, and it did so by following
what Memoize does. I only manually adjusted the tuplestore_get_stats()
API afterwards to resolve the issue with transferring the pointer to
the string const into shared memory, plus manual review.You can test the true parallel query output by running the following setup:
set debug_parallel_query = off;
set parallel_setup_cost=0;
set parallel_tuple_cost=0;
set min_parallel_table_scan_size=0;
set max_parallel_workers_per_gather=4;
set parallel_leader_participation = off;
alter table tenk1 set (parallel_workers = 4);
alter table tenk2 set (parallel_workers = 0);Doing that, running the query outputs:
-> Materialize ...
Buffers: shared hit=1380
Worker 0: Storage: Memory Maximum Storage: 2849kB
Worker 1: Storage: Memory Maximum Storage: 2849kB
Worker 2: Storage: Memory Maximum Storage: 2849kB
Worker 3: Storage: Memory Maximum Storage: 2849kB
I think the patch is quite clean and looks very good.
I have a small concern with the output, though it’s not an issue introduced by this patch.
The output is easily read as “used 2849kB of memory”. When Storage shows “Disk”, it would be read as “used NNNkB of disk space”. However, those two metrics are independent: “Storage: Disk” only means “ever used disk”, and “Maximum Storage” means the peak usage, which may come from memory or disk.
I’m not requesting a change to this output, just sharing my thoughts for your reference.
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
On Mon, 12 Oct 2026 at 18:17, Tatsuya Kawata
<kawatatatsuya0913@gmail.com> wrote:
Thank you for the patch!
I reviewed v1. I confirmed the following, so LGTM.
Thanks. I pushed this as two separate patches. The 1st adjusts the
tuplestore API, the 2nd adds the Material related EXPLAIN output.
David