Assert failure in parallel query due to SubPlan double visit for hash join
Hi,
Whilst stress testing the next version of the stack-based
instrumentation patch [0]/messages/by-id/CAP53PkzdBK8VJ1fS4AZ481LgMN8f9mJiC39ZRHqkFUSYq6KWmg@mail.gmail.com, I ran into what appears to be an ancient
bug in how we walk plan state for SubPlan nodes in parallel query
setup and instrumentation handling.
Specifically, the planstate_tree_walker callbacks in execParallel.c
assume that a given PlanState is only walked once, but that doesn't
hold true for hash joins since (I think, not tested) Postgres 10. We
create two separate references to the same subplan PlanState, one via
hashclauses and the other via hashkeys, each having their own
SubPlanState, but going to the same subplan PlanState. And
planstate_walk_subplans() descends into the subtree once per
reference.
I believe this is an issue since b8d7f053c5c (Postgres 10, CCing
Andres as the author of that one) changed how ExecInitHashJoin calls
ExecInitExpr, creating the second SubPlanState. More digging also
found the very old a64846f3ada that talked about a seemingly related
problem, but that must have happened before parallel query was in the
picture.
We can see this cause an assertion failure on master with the
following reproducer:
CREATE TABLE big (a int, b int);
INSERT INTO big SELECT g, g FROM generate_series(1, 10000) g;
CREATE TABLE small (x int PRIMARY KEY, y int);
INSERT INTO small SELECT g, g FROM generate_series(1, 1000) g;
CREATE TABLE other (k int PRIMARY KEY, f int);
INSERT INTO other SELECT g, g FROM generate_series(1, 1000) g;
ANALYZE big, small, other;
ALTER TABLE other SET (parallel_workers = 0);
SET parallel_setup_cost = 0;
SET parallel_tuple_cost = 0;
SET min_parallel_table_scan_size = 0;
SET enable_nestloop = off;
SET enable_mergejoin = off;
SET enable_parallel_hash = off;
SET parallel_leader_participation = off; -- to show loop count more clearly
EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF)
SELECT COUNT(*) FROM big b JOIN small s ON s.x =
CASE WHEN (b.a, b.b) NOT IN (SELECT o.k, o.f FROM other o ORDER BY o.f)
THEN 1 ELSE 2 END;
TRAP: failed Assert("toc->toc_entry[i].key != key"), File:
"shm_toc.c", Line: 190
Besides the assertion failure it also causes problems for
instrumentation. In this example we get an incorrect loop count (the
subplan is really only run in the two workers, so the loops=16 below
should be 4), and the Sort node is missing the Sort Method reporting:
QUERY PLAN
--------------------------------------------------------------
Finalize Aggregate (actual rows=1.00 loops=1)
-> Gather (actual rows=2.00 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Partial Aggregate (actual rows=1.00 loops=2)
-> Hash Join (actual rows=5000.00 loops=2)
Hash Cond: (CASE WHEN (NOT (ANY ((b.a = (hashed
SubPlan any_1).col1) AND (b.b = (hashed SubPlan any_1).col2)))) THEN 1
ELSE 2 END = s.x)
-> Parallel Seq Scan on big b (actual
rows=5000.00 loops=2)
-> Hash (actual rows=1000.00 loops=2)
Buckets: 1024 Batches: 1 Memory Usage: 44kB
-> Seq Scan on small s (actual
rows=1000.00 loops=2)
SubPlan any_1
-> Sort (actual rows=1000.00 loops=16)
Sort Key: o.f
-> Seq Scan on other o (actual
rows=1000.00 loops=16)
The assertion failure happens because ExecParallelInitializeDSM visits
that Sort twice and tries to shm_toc_insert() the same plan_node_id
key. The instrumentation issue happens because (1) for loops, we
accumulate the worker instrumentation twice in
ExecParallelReportInstrumentation and then again in
ExecParallelRetrieveInstrumentation for the nodes underneath the
shared SubPlan, (2) for "Sort Method" the leader and workers end up
pointing at different (duplicate) SharedSortInfo slots and the leader
looks at the empty one.
To fix this, the least convoluted way I could think of is to invent a
new mechanism to collect a deduplicated list of plan states, and then
call the existing logic on each list member. This also allows some
refactoring to remove logic for counting how many nodes we've visited.
With the attached patch, the instrumentation now shows correctly, and
the added regression test no longer fails with an assertion failure.
...
SubPlan any_1
-> Sort (actual rows=1000.00 loops=4)
Sort Key: o.f
Worker 0: Sort Method: quicksort Memory: 48kB
Worker 1: Sort Method: quicksort Memory: 48kB
-> Seq Scan on other o (actual
rows=1000.00 loops=4)
We could alternatively deal with this in the existing walkers, but
then we'd have to modify the walker context in four different places
to carry the identities of already visited plan states. I also assume
that changing how ExecInitHashJoin works is off the table, but I admit
that logic goes a bit over my head.
Note that technically the ExecShutdownNode_walker suffers from the
same problem, but it doesn't appear to be a practical problem with
in-core code today, and it's a bit more complicated to fix with the
current nested InstrStartNode assumptions (that I intend to fix
separately in the stack-based instrumentation series).
Thoughts?
Thanks,
Lukas
[0]: /messages/by-id/CAP53PkzdBK8VJ1fS4AZ481LgMN8f9mJiC39ZRHqkFUSYq6KWmg@mail.gmail.com
--
Lukas Fittl
Hi Lukas,
The assertion failure happens because ExecParallelInitializeDSM visits
that Sort twice and tries to shm_toc_insert() the same plan_node_id
key.
I confirmed this also on an assert build.
The instrumentation issue happens because (1) for loops, we
accumulate the worker instrumentation twice in
ExecParallelReportInstrumentation and then again in
ExecParallelRetrieveInstrumentation for the nodes underneath the
shared SubPlan, (2) for "Sort Method" the leader and workers end up
pointing at different (duplicate) SharedSortInfo slots and the leader
looks at the empty one.
In a non-assert build, this kind of case is definitely not easy to spot,
unless someone is really scrutinizing these parts of the plan.
Your patch looks fine to me. I just had a few small comments:
1/ Maybe rename planstate_tree_collect() to something like
planstate_tree_collect_distinct(). The important part here is that each
distinct PlanState appears in the returned list only once, so the name should
probably say that.
2/ Some execParallel.c comments still read as if the functions recurse. For
example ExecParallelRetrieveInstrumentation(),
ExecParallelReportInstrumentation(),
and ExecParallelInitializeWorker() now only deal with a single
PlanState at a time.
--
Sami Imseih
Amazon Web Services (AWS)
Hi Lukas!
Whilst stress testing the next version of the stack-based
instrumentation patch [0], I ran into what appears to be an ancient
bug in how we walk plan state for SubPlan nodes in parallel query
setup and instrumentation handling.
Good find.
Specifically, the planstate_tree_walker callbacks in execParallel.c
assume that a given PlanState is only walked once, but that doesn't
hold true for hash joins since (I think, not tested) Postgres 10. We
create two separate references to the same subplan PlanState, one via
hashclauses and the other via hashkeys, each having their own
SubPlanState, but going to the same subplan PlanState. And
planstate_walk_subplans() descends into the subtree once per
reference.
Yes. I've confirmed this and put some details below.
I believe this is an issue since b8d7f053c5c (Postgres 10, CCing
Andres as the author of that one) changed how ExecInitHashJoin calls
ExecInitExpr, creating the second SubPlanState. More digging also
found the very old a64846f3ada that talked about a seemingly related
problem, but that must have happened before parallel query was in the
picture.
(gdb) p *planstate
$17 = {
type = T_HashJoinState,
plan = 0x5a9ef8df0040,
state = 0x5a9ef8ddfed0,
...
(gdb) p ((SubPlanState *)planstate->subPlan->elements[0])->planstate
$15 = (PlanState *) 0x5a9ef8de01e0
(gdb) p ((SubPlanState *)planstate->subPlan->elements[1])->planstate
$16 = (PlanState *) 0x5a9ef8de01e0
This happens in ExecInitSubPlanExpr() which adds the sort PlanState
twice to the subPlan list.
We can see this cause an assertion failure on master with the
following reproducer:CREATE TABLE big (a int, b int);
INSERT INTO big SELECT g, g FROM generate_series(1, 10000) g;
CREATE TABLE small (x int PRIMARY KEY, y int);
INSERT INTO small SELECT g, g FROM generate_series(1, 1000) g;
CREATE TABLE other (k int PRIMARY KEY, f int);
INSERT INTO other SELECT g, g FROM generate_series(1, 1000) g;
ANALYZE big, small, other;
ALTER TABLE other SET (parallel_workers = 0);SET parallel_setup_cost = 0;
SET parallel_tuple_cost = 0;
SET min_parallel_table_scan_size = 0;
SET enable_nestloop = off;
SET enable_mergejoin = off;
SET enable_parallel_hash = off;
SET parallel_leader_participation = off; -- to show loop count more clearlyEXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF)
SELECT COUNT(*) FROM big b JOIN small s ON s.x =
CASE WHEN (b.a, b.b) NOT IN (SELECT o.k, o.f FROM other o ORDER BY o.f)
THEN 1 ELSE 2 END;TRAP: failed Assert("toc->toc_entry[i].key != key"), File:
"shm_toc.c", Line: 190
I can reproduce the assertion.
Besides the assertion failure it also causes problems for
instrumentation. In this example we get an incorrect loop count (the
subplan is really only run in the two workers, so the loops=16 below
should be 4), and the Sort node is missing the Sort Method reporting:QUERY PLAN
--------------------------------------------------------------
Finalize Aggregate (actual rows=1.00 loops=1)
-> Gather (actual rows=2.00 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Partial Aggregate (actual rows=1.00 loops=2)
-> Hash Join (actual rows=5000.00 loops=2)
Hash Cond: (CASE WHEN (NOT (ANY ((b.a = (hashed
SubPlan any_1).col1) AND (b.b = (hashed SubPlan any_1).col2)))) THEN 1
ELSE 2 END = s.x)
-> Parallel Seq Scan on big b (actual
rows=5000.00 loops=2)
-> Hash (actual rows=1000.00 loops=2)
Buckets: 1024 Batches: 1 Memory Usage: 44kB
-> Seq Scan on small s (actual
rows=1000.00 loops=2)
SubPlan any_1
-> Sort (actual rows=1000.00 loops=16)
Sort Key: o.f
-> Seq Scan on other o (actual
rows=1000.00 loops=16)The assertion failure happens because ExecParallelInitializeDSM visits
that Sort twice and tries to shm_toc_insert() the same plan_node_id
key. The instrumentation issue happens because (1) for loops, we
accumulate the worker instrumentation twice in
ExecParallelReportInstrumentation and then again in
ExecParallelRetrieveInstrumentation for the nodes underneath the
shared SubPlan, (2) for "Sort Method" the leader and workers end up
pointing at different (duplicate) SharedSortInfo slots and the leader
looks at the empty one.To fix this, the least convoluted way I could think of is to invent a
new mechanism to collect a deduplicated list of plan states, and then
call the existing logic on each list member. This also allows some
refactoring to remove logic for counting how many nodes we've visited.
While I like generally like your fix and the resulting code
simplifications, I'm wondering if it's the right fix.
It seems to me that the PlanState graph structure is broken for the case
you encountered. While it's generally possible that a PlanState is
referenced more than once (e.g. multiple CteScanStates reference the
same CteScanState::cteplanstate), it's weird that the same PlanState
references another PlanState more than once.
Beyond that, I'm wondering how much other code is broken or can break if
we don't change this structurally in the graph. You found already
another one in ExecShutdownNode().
A simple alternative fix would to ensure inExecInitSubPlanExpr() that
the subPlan list is always unique.
With the attached patch, the instrumentation now shows correctly, and
the added regression test no longer fails with an assertion failure....
SubPlan any_1
-> Sort (actual rows=1000.00 loops=4)
Sort Key: o.f
Worker 0: Sort Method: quicksort Memory: 48kB
Worker 1: Sort Method: quicksort Memory: 48kB
-> Seq Scan on other o (actual
rows=1000.00 loops=4)We could alternatively deal with this in the existing walkers, but
then we'd have to modify the walker context in four different places
to carry the identities of already visited plan states.
Really? Can't you check based on state you have already decide if it's
already initialized and if so bail? For example in ExecSortInitializeDSM():
if (node->shared_info != NULL)
return;
I also assume
that changing how ExecInitHashJoin works is off the table, but I admit
that logic goes a bit over my head.
It's these two calls that cause the sort PlanState to be added twice:
hjstate->hj_OuterHash =
ExecBuildHash32Expr(hjstate->js.ps.ps_ResultTupleDesc,
hjstate->js.ps.resultops,
outer_hashfuncid,
node->hashcollations,
node->hashkeys,
hash_strict,
&hjstate->js.ps,
0);
and
hjstate->hashclauses =
ExecInitQual(node->hashclauses, (PlanState *) hjstate);
Both, the hash clauses and the qual, reference the sub-plan. How would
you avoid this? Also, it's too easy for other plan nodes to do the same.
Hence, i think it's better to fix it structurally.
Note that technically the ExecShutdownNode_walker suffers from the
same problem, but it doesn't appear to be a practical problem with
in-core code today, and it's a bit more complicated to fix with the
current nested InstrStartNode assumptions (that I intend to fix
separately in the stack-based instrumentation series).
Can you give some more details why it's not a problem in-core and what
the problem with InstrStart/StopNode() is?
Any node sitting below the Sort node, which doesn't like having its
shutdown function called twice, might cause problems. Especially,
CustomScanStates that we don't have full control over could be problematic.
Seems to me that it would be good to fix it all in one go. But maybe I'm
not seeing the complications clearly enough.
--
David Geier
Hi Sami,
On Wed, Oct 28, 2026 at 5:17 PM Sami Imseih <samimseih.pg@gmail.com> wrote:
In a non-assert build, this kind of case is definitely not easy to spot,
unless someone is really scrutinizing these parts of the plan.
Yeah, agreed - its also a somewhat unlikely plan to have, so I don't
think this is seen much in practice, and if it is, people won't
realize its wrong.
Your patch looks fine to me. I just had a few small comments:
Thanks for reviewing!
1/ Maybe rename planstate_tree_collect() to something like
planstate_tree_collect_distinct(). The important part here is that each
distinct PlanState appears in the returned list only once, so the name should
probably say that.
Yeah, good point on naming - if we stay with this structure I'll adjust.
2/ Some execParallel.c comments still read as if the functions recurse. For
example ExecParallelRetrieveInstrumentation(),
ExecParallelReportInstrumentation(),
and ExecParallelInitializeWorker() now only deal with a single
PlanState at a time.
Good point - there are a few that clearly need updating (e.g.
ExecParallelRetrieveInstrumentation jumps out), will revise.
Thanks,
Lukas
--
Lukas Fittl
Hi David,
On Thu, Oct 29, 2026 at 1:26 AM David Geier <geidav.pg@gmail.com> wrote:
I believe this is an issue since b8d7f053c5c (Postgres 10, CCing
Andres as the author of that one) changed how ExecInitHashJoin calls
ExecInitExpr, creating the second SubPlanState. More digging also
found the very old a64846f3ada that talked about a seemingly related
problem, but that must have happened before parallel query was in the
picture.(gdb) p *planstate
$17 = {
type = T_HashJoinState,
plan = 0x5a9ef8df0040,
state = 0x5a9ef8ddfed0,
...
(gdb) p ((SubPlanState *)planstate->subPlan->elements[0])->planstate
$15 = (PlanState *) 0x5a9ef8de01e0
(gdb) p ((SubPlanState *)planstate->subPlan->elements[1])->planstate
$16 = (PlanState *) 0x5a9ef8de01e0This happens in ExecInitSubPlanExpr() which adds the sort PlanState
twice to the subPlan list.
Yep, exactly - thanks for adding those details.
To fix this, the least convoluted way I could think of is to invent a
new mechanism to collect a deduplicated list of plan states, and then
call the existing logic on each list member. This also allows some
refactoring to remove logic for counting how many nodes we've visited.While I like generally like your fix and the resulting code
simplifications, I'm wondering if it's the right fix.It seems to me that the PlanState graph structure is broken for the case
you encountered. While it's generally possible that a PlanState is
referenced more than once (e.g. multiple CteScanStates reference the
same CteScanState::cteplanstate), it's weird that the same PlanState
references another PlanState more than once.
Right - I agree.
Beyond that, I'm wondering how much other code is broken or can break if
we don't change this structurally in the graph. You found already
another one in ExecShutdownNode().A simple alternative fix would to ensure inExecInitSubPlanExpr() that
the subPlan list is always unique.
Yeah, that would be elegant, but I don't think it works. The problem
is that we can also have two different nodes reference the same
PlanState, like in this example with the same SubPlan being referenced
by different nodes underneath an Append (courtesy of Claude, FWIW):
create table t2s as
select i % 1000 as thousand, i % 2000 as twothousand
from generate_series(0, 999) i;
alter table t2s set (parallel_workers = 0);
-- Hash-partitioned table. The WHERE clause below is copied to every
-- partition scan, each with its own SubPlanState for the same subplan.
create table tp (unique1 int, hundred int) partition by hash (unique1);
create table tp0 partition of tp for values with (modulus 3, remainder 0);
create table tp1 partition of tp for values with (modulus 3, remainder 1);
create table tp2 partition of tp for values with (modulus 3, remainder 2);
insert into tp select i, i % 100 from generate_series(0, 299999) i;
vacuum analyze t2s;
vacuum analyze tp;
-- Make a parallel plan attractive on a small test table.
set parallel_leader_participation = off;
set parallel_setup_cost = 0;
set parallel_tuple_cost = 0;
set min_parallel_table_scan_size = 0;
set max_parallel_workers_per_gather = 2;
explain (analyze, costs off, timing off, summary off, buffers off)
select count(*)
from tp
where hundred = case when 5 not in
(select thousand from t2s order by twothousand limit 50)
then 1 else 2 end;
On master this shows in a non-assert enabled build (note the missing
Sort Method in the SubPlan):
Finalize Aggregate
-> Gather
Workers Planned: 2
-> Partial Aggregate
-> Parallel Append
-> Parallel Seq Scan on tp0 tp_1
Filter: (hundred = CASE WHEN (NOT (ANY (5
= (hashed SubPlan any_1).col1))) THEN 1 ELSE 2 END)
SubPlan any_1
-> Limit
-> Sort
Sort Key: t2s.twothousand
-> Seq Scan on t2s
-> Parallel Seq Scan on tp1 tp_2
Filter: (hundred = CASE WHEN (NOT (ANY (5
= (hashed SubPlan any_1).col1))) THEN 1 ELSE 2 END)
-> Parallel Seq Scan on tp2 tp_3
Filter: (hundred = CASE WHEN (NOT (ANY (5
= (hashed SubPlan any_1).col1))) THEN 1 ELSE 2 END)
With the attached patch, the instrumentation now shows correctly, and
the added regression test no longer fails with an assertion failure....
SubPlan any_1
-> Sort (actual rows=1000.00 loops=4)
Sort Key: o.f
Worker 0: Sort Method: quicksort Memory: 48kB
Worker 1: Sort Method: quicksort Memory: 48kB
-> Seq Scan on other o (actual
rows=1000.00 loops=4)We could alternatively deal with this in the existing walkers, but
then we'd have to modify the walker context in four different places
to carry the identities of already visited plan states.Really? Can't you check based on state you have already decide if it's
already initialized and if so bail? For example in ExecSortInitializeDSM():if (node->shared_info != NULL)
return;
I suppose, but that becomes a bit tedious since we're reinventing the
wheel on how to avoid duplicates in a customized way for each walker.
I also assume
that changing how ExecInitHashJoin works is off the table, but I admit
that logic goes a bit over my head.It's these two calls that cause the sort PlanState to be added twice:
hjstate->hj_OuterHash =
ExecBuildHash32Expr(hjstate->js.ps.ps_ResultTupleDesc,
hjstate->js.ps.resultops,
outer_hashfuncid,
node->hashcollations,
node->hashkeys,
hash_strict,
&hjstate->js.ps,
0);and
hjstate->hashclauses =
ExecInitQual(node->hashclauses, (PlanState *) hjstate);Both, the hash clauses and the qual, reference the sub-plan. How would
you avoid this? Also, it's too easy for other plan nodes to do the same.
Hence, i think it's better to fix it structurally.
Yeah, that's fair - I was mainly thinking "I don't see if there is a
way to solve it at that level, maybe someone else (Andres?) does", but
it seemed unlikely to me too.
Note that technically the ExecShutdownNode_walker suffers from the
same problem, but it doesn't appear to be a practical problem with
in-core code today, and it's a bit more complicated to fix with the
current nested InstrStartNode assumptions (that I intend to fix
separately in the stack-based instrumentation series).Can you give some more details why it's not a problem in-core and what
the problem with InstrStart/StopNode() is?
The problem is that InstrStart/StopNode expect to be called in a
nested manner, so that if shutdown activity occurs in a node, the
parent node also gets its instrumentation updated. If we were to
process each plan state one by one in a list, that nested aspect gets
lost, and parent nodes would no longer carry the instrumentation
information (we'd have to invent a mechanism to add it back, which is
a bunch of code). This gets a lot easier in a world where we are more
explicit about accumulation of instrumentation data (as is the case in
the stack-based instrumentation patch series), but otherwise seems a
bit overkill.
Any node sitting below the Sort node, which doesn't like having its
shutdown function called twice, might cause problems. Especially,
CustomScanStates that we don't have full control over could be problematic.
Yeah, that's a fair point - custom scans could certainly get
unexpected failures here.
Thanks,
Lukas
--
Lukas Fittl
Yeah, that would be elegant, but I don't think it works. The problem
is that we can also have two different nodes reference the same
PlanState, like in this example with the same SubPlan being referenced
by different nodes underneath an Append (courtesy of Claude, FWIW):create table t2s as
select i % 1000 as thousand, i % 2000 as twothousand
from generate_series(0, 999) i;
alter table t2s set (parallel_workers = 0);-- Hash-partitioned table. The WHERE clause below is copied to every
-- partition scan, each with its own SubPlanState for the same subplan.
create table tp (unique1 int, hundred int) partition by hash (unique1);
create table tp0 partition of tp for values with (modulus 3, remainder 0);
create table tp1 partition of tp for values with (modulus 3, remainder 1);
create table tp2 partition of tp for values with (modulus 3, remainder 2);
insert into tp select i, i % 100 from generate_series(0, 299999) i;vacuum analyze t2s;
vacuum analyze tp;-- Make a parallel plan attractive on a small test table.
set parallel_leader_participation = off;
set parallel_setup_cost = 0;
set parallel_tuple_cost = 0;
set min_parallel_table_scan_size = 0;
set max_parallel_workers_per_gather = 2;explain (analyze, costs off, timing off, summary off, buffers off)
select count(*)
from tp
where hundred = case when 5 not in
(select thousand from t2s order by twothousand limit 50)
then 1 else 2 end;On master this shows in a non-assert enabled build (note the missing
Sort Method in the SubPlan):Finalize Aggregate
-> Gather
Workers Planned: 2
-> Partial Aggregate
-> Parallel Append
-> Parallel Seq Scan on tp0 tp_1
Filter: (hundred = CASE WHEN (NOT (ANY (5
= (hashed SubPlan any_1).col1))) THEN 1 ELSE 2 END)
SubPlan any_1
-> Limit
-> Sort
Sort Key: t2s.twothousand
-> Seq Scan on t2s
-> Parallel Seq Scan on tp1 tp_2
Filter: (hundred = CASE WHEN (NOT (ANY (5
= (hashed SubPlan any_1).col1))) THEN 1 ELSE 2 END)
-> Parallel Seq Scan on tp2 tp_3
Filter: (hundred = CASE WHEN (NOT (ANY (5
= (hashed SubPlan any_1).col1))) THEN 1 ELSE 2 END)
Just realized that for CTEs it's different. The CTE is only stored in
initPlan and planstate_tree_walker() doesn't descend into
CTEScanState::cteplanstate. So every CTEScanState is only visited once.
But the example you found is a good one. Each PlanState only has a
single reference but in total there's more than a single reference to
the same PlanState.
So it seems like we cannot avoid this case by being smart about what we
store per node. In that case I would prefer if planstate_tree_walker()
would take care not further descending.
Do you know if there's any code that relies on nodes being visited
multiple times? If not, I would change planstate_tree_walker() such that
it visits every node only once. If there's such code, I would make
planstate_tree_walker() have two modes.
We could use Bitmapset on the plan_node_id to check if a node has been
visited already. That should be pretty cheap. Only question is how to
pass this through the recursion levels in a nice way.
Really? Can't you check based on state you have already decide if it's
already initialized and if so bail? For example in ExecSortInitializeDSM():if (node->shared_info != NULL)
return;I suppose, but that becomes a bit tedious since we're reinventing the
wheel on how to avoid duplicates in a customized way for each walker.
I would think in almost all code it's pretty easy to check for "is
initialized" but it makes the code hard to read and it's easy to forget
without having such corner-case tests in place.
I would prefer a solution that avoids the problem in the first place.
Can you give some more details why it's not a problem in-core and what
the problem with InstrStart/StopNode() is?The problem is that InstrStart/StopNode expect to be called in a
nested manner, so that if shutdown activity occurs in a node, the
parent node also gets its instrumentation updated. If we were to
process each plan state one by one in a list, that nested aspect gets
lost, and parent nodes would no longer carry the instrumentation
information (we'd have to invent a mechanism to add it back, which is
a bunch of code). This gets a lot easier in a world where we are more
explicit about accumulation of instrumentation data (as is the case in
the stack-based instrumentation patch series), but otherwise seems a
bit overkill.
That would naturally work with a solution that doesn't visit already
visited children, does it?
Any node sitting below the Sort node, which doesn't like having its
shutdown function called twice, might cause problems. Especially,
CustomScanStates that we don't have full control over could be problematic.Yeah, that's a fair point - custom scans could certainly get
unexpected failures here.
OK.
--
David Geier
On Mon, Nov 2, 2026 at 12:18 PM David Geier <geidav.pg@gmail.com> wrote:
But the example you found is a good one. Each PlanState only has a
single reference but in total there's more than a single reference to
the same PlanState.So it seems like we cannot avoid this case by being smart about what we
store per node. In that case I would prefer if planstate_tree_walker()
would take care not further descending.
Thanks for confirming - I do agree that if we can make
planstate_tree_walker do this consistently that would be good - I'm
mainly not sure how awkward we make the API with that.
Do you know if there's any code that relies on nodes being visited
multiple times? If not, I would change planstate_tree_walker() such that
it visits every node only once. If there's such code, I would make
planstate_tree_walker() have two modes.
From my read I don't think any of the callers expect it to visit the
same node multiple times, but if we go this way we should review one
more time.
We could use Bitmapset on the plan_node_id to check if a node has been
visited already. That should be pretty cheap. Only question is how to
pass this through the recursion levels in a nice way.
Yeah, exactly - I think there are two ways I can think of to solve it:
1) Make context not be generic, but rather a common base context
struct (with the Bitmapset) that is embedded as the first field in the
other context structs
2) Pass an extra argument to the walker functions and callback that
contains the Bitmapset
Preferences on these / can you think of a different way?
Really? Can't you check based on state you have already decide if it's
already initialized and if so bail? For example in ExecSortInitializeDSM():if (node->shared_info != NULL)
return;I suppose, but that becomes a bit tedious since we're reinventing the
wheel on how to avoid duplicates in a customized way for each walker.I would think in almost all code it's pretty easy to check for "is
initialized" but it makes the code hard to read and it's easy to forget
without having such corner-case tests in place.I would prefer a solution that avoids the problem in the first place.
Ack.
Can you give some more details why it's not a problem in-core and what
the problem with InstrStart/StopNode() is?The problem is that InstrStart/StopNode expect to be called in a
nested manner, so that if shutdown activity occurs in a node, the
parent node also gets its instrumentation updated. If we were to
process each plan state one by one in a list, that nested aspect gets
lost, and parent nodes would no longer carry the instrumentation
information (we'd have to invent a mechanism to add it back, which is
a bunch of code). This gets a lot easier in a world where we are more
explicit about accumulation of instrumentation data (as is the case in
the stack-based instrumentation patch series), but otherwise seems a
bit overkill.That would naturally work with a solution that doesn't visit already
visited children, does it?
Yes, it would. For me it was mostly an argument to not invent a custom
way of solving it there.
Thanks,
Lukas
--
Lukas Fittl
Thanks for confirming - I do agree that if we can make
planstate_tree_walker do this consistently that would be good - I'm
mainly not sure how awkward we make the API with that.
I think it won't be too bad. There are also only very fw call sites for
planstate_tree_walker().
Do you know if there's any code that relies on nodes being visited
multiple times? If not, I would change planstate_tree_walker() such that
it visits every node only once. If there's such code, I would make
planstate_tree_walker() have two modes.From my read I don't think any of the callers expect it to visit the
same node multiple times, but if we go this way we should review one
more time.
- ExplainPreScanNode() is fine either way.
- ExecShutdownNode_walker() should be fixed as discussed.
- The remaining calls are all from execParallel.c and should also be
fixed, shouldn't they?
These are the only call sites I can see. Based on that we could change
planstate_tree_walker() to visit every node only once.
We could use Bitmapset on the plan_node_id to check if a node has been
visited already. That should be pretty cheap. Only question is how to
pass this through the recursion levels in a nice way.Yeah, exactly - I think there are two ways I can think of to solve it:
1) Make context not be generic, but rather a common base context
struct (with the Bitmapset) that is embedded as the first field in the
other context structs
That sounds good. Maybe it's enough to require that the first member of
any context is a Bitmapset. Bitmapset is a node so we could assert on
that as well.
2) Pass an extra argument to the walker functions and callback that
contains the Bitmapset
That requires then always two state variables at the call sites. I
prefer (1).
Preferences on these / can you think of a different way?
An alternative to the Bitmapset would be storing traversal state inside
struct PlanState. The advantage of that is performance. A "has been
visited" check would be as cheap as comparing two variables.
A visited flag has the disadvantage that it needs to be reset after the
traversal has completed. What usually works quite nicely is a "traversal
generation" counter that is incremented prior to each traversal. A node
is considered visited if that node's counter is equal to the current
traversal generation (which means it has been incremented already during
the current traversal). The counter could be as small as 1 bit. We can
rely on wrap around to be different for each two successive traversals.
I'm leaning towards the less intrusive Bitmapset variant. The
is-member-of checks should be fast enough and likely be negligible
compared to all the cache misses when traversing the graph. We can
profile that.
The problem is that InstrStart/StopNode expect to be called in a
nested manner, so that if shutdown activity occurs in a node, the
parent node also gets its instrumentation updated. If we were to
process each plan state one by one in a list, that nested aspect gets
lost, and parent nodes would no longer carry the instrumentation
information (we'd have to invent a mechanism to add it back, which is
a bunch of code). This gets a lot easier in a world where we are more
explicit about accumulation of instrumentation data (as is the case in
the stack-based instrumentation patch series), but otherwise seems a
bit overkill.That would naturally work with a solution that doesn't visit already
visited children, does it?Yes, it would. For me it was mostly an argument to not invent a custom
way of solving it there.
Cool!
--
David Geier