BUG #19924: unrecognized node type in create_plan_recurse: UNION, LIMIT, json_populate_recordset, inheritance

Started by PG Bug reporting form2 days ago6 messagesbugs
Jump to latest
#1PG Bug reporting form
noreply@postgresql.org

The following bug has been logged on the website:

Bug reference: 19924
Logged by: Zheng Hacker
Email address: hackerzheng666@gmail.com
PostgreSQL version: 19.0
Operating system: Linux x86_64, Ubuntu 22.04.5, gcc 11.4.0, cassert
Description:

Tested on git master (20devel) d94964f14f39936d69c6b8d3fc5020f0fdae1617 of
2026-11-20 and on an older unreleased master build. The version selected in
this form is the closest option available; I did not test 19.0 or any
released branch.

A UNION (with duplicate elimination) of two identical queries, each with
json_populate_recordset() in the select list over a join that involves a
table with an inheritance child, together with a LIMIT, fails in the planner
with an internal error whose node tag is a garbage number.

Fresh cluster, default settings:

CREATE TABLE t1 (c0 int4, c1 int8);
CREATE TABLE t3 (c0 int4, c1 interval);
CREATE TABLE ch4 () INHERITS (t1);
(SELECT json_populate_recordset(CAST('2147483647' AS oid), CAST('[]' AS
json), FALSE) AS c0
FROM t3 AS a0 INNER JOIN t1 AS a1 USING (c0))
UNION
(SELECT json_populate_recordset(CAST('2147483647' AS oid), CAST('[]' AS
json), FALSE) AS c0
FROM t3 AS a0 INNER JOIN t1 AS a1 USING (c0))
LIMIT 2;

ERROR: unrecognized node type: 3301232

With \set VERBOSITY verbose on master d94964f:
ERROR: XX000: unrecognized node type: 3301232
LOCATION: create_plan_recurse, createplan.c:547

The number is the same on every run and for the variants below.

What I ran, all on the same statement list (no tables are populated, all
tables are empty):
- git master d94964f (built from source, cassert): 3 of 3 fresh clusters
give the error above.
- An older, unreleased master build (cassert): 3 of 3 fresh clusters give
the same error.
- On master, each of these removals makes the error disappear (one run each,
same cluster): replacing UNION with UNION ALL; removing LIMIT; using only
one branch with LIMIT; INTERSECT instead of UNION; dropping the inheritance
child ch4 before the query. Re-creating ch4 brings the error back.
- The function matters: with generate_series(1,2), unnest(ARRAY[1,2]) or a
plain column a0.c0 in place of json_populate_recordset(...) there is no
error (one run each on master). With json_populate_recordset(NULL::t1,
CAST('[]' AS json)) the same error with the same number 3301232 appears.

Found by a statement-sequence fuzzer and reduced to the statements above.

What I did not do: I did not trace where the Path node is freed or
overwritten, I did not test a build without --enable-cassert (the error
comes from elog, so I expect it there too, but I have not run it), and I did
not test released branches.

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: PG Bug reporting form (#1)
Re: BUG #19924: unrecognized node type in create_plan_recurse: UNION, LIMIT, json_populate_recordset, inheritance

PG Bug reporting form <noreply@postgresql.org> writes:

A UNION (with duplicate elimination) of two identical queries, each with
json_populate_recordset() in the select list over a join that involves a
table with an inheritance child, together with a LIMIT, fails in the planner
with an internal error whose node tag is a garbage number.

Yeah, that's pretty clearly a pfree'd Path node. I tried
git-bisecting this, and got an unhelpful-looking result:

8300d3ad4aa73dc6beec8dca7d9362dbc21c9a83 is the first bad commit
commit 8300d3ad4aa73dc6beec8dca7d9362dbc21c9a83 (HEAD)
Author: Robert Haas <rhaas@postgresql.org>
Date: Mon Mar 9 08:16:30 2026 -0400

Consider startup cost as a figure of merit for partial paths.

I suspect what this is telling us is that the bug predates that,
but by chance this example didn't fail earlier because our cost
heuristics didn't lead us to choose the broken path.

Valgrind doesn't find anything useful either: it is able to
report use of an uninitialized value, but it doesn't track
where the lost Path was allocated or freed.

Not sure about a productive way to narrow this down :-(

regards, tom lane

#3Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#2)
Re: BUG #19924: unrecognized node type in create_plan_recurse: UNION, LIMIT, json_populate_recordset, inheritance

I wrote:

Not sure about a productive way to narrow this down :-(

I made a little further progress with gdb, and determined that
the missing path is (unsurprisingly at this point) a partial
path, for which we still have a Gather path, and that path
is getting chosen. This makes the failure go away:

diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 3af494f269d..49803a325d6 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -863,7 +863,6 @@ add_partial_path(RelOptInfo *parent_rel, Path *new_path)
 		{
 			parent_rel->partial_pathlist =
 				foreach_delete_current(parent_rel->partial_pathlist, p1);
-			pfree(old_path);
 		}
 		else
 		{

but of course that's likely to be an unacceptably memory-hungry
solution.

What I think must be happening here is that we are building
Gather paths before we have finished letting the add_partial_path
tournament run. This is a Bad Thing just from the point of view
of wasted cycles, never mind dangling pointers. We should wait
till all the partial paths are made for a rel, and only then
build Gather paths atop the surviving partial paths.

regards, tom lane

#4Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#3)
Re: BUG #19924: unrecognized node type in create_plan_recurse: UNION, LIMIT, json_populate_recordset, inheritance

Ah, I identified the problem and a possible fix. The root culprit is
in grouping_planner():

/*
* Generate partial paths for final_rel, too, if outer query levels might
* be able to make use of them.
*/
if (final_rel->consider_parallel && root->query_level > 1 &&
!limit_needed(parse))
{
Assert(!parse->rowMarks && parse->commandType == CMD_SELECT);
foreach(lc, current_rel->partial_pathlist)
{
Path *partial_path = (Path *) lfirst(lc);

add_partial_path(final_rel, partial_path);
}
}

In this loop, calls of add_partial_path after the first one are
capable of removing partial paths from final_rel, and then it deletes
those despite their probably being referenced in Gather paths that
have already been copied into final_rel, not to mention that they're
still referenced within current_rel. So that's where the dangling
pointers come from. This behavior requires a few preconditions that
might themselves be bugs:

* It's not very clear how a set of Paths that all survived the
add_partial_path tournament for current_rel wouldn't all survive
the new tournament for final_rel. I suspect there is an
order-of-presentation bug in add_partial_path's path comparisons,
but didn't go digging for it. (I did check that final_rel's
partial_pathlist is empty at the start of this loop; but if it ever
isn't, that would offer a non-buggy way to reach the problem.)

* It's even less clear how the parent Gather of a partial path
that didn't survive this tournament would manage to win out at
the final_rel level and be chosen to become the Plan. This suggests
bug-grade inconsistencies between the costing comparisons of Gather
paths and partial paths.

I'm not planning to look into those issues myself, but probably
someone should. Whether they are bugs or no, I think we need some
kind of data structure consistency guard here.

After some thought I have an idea: grouping_planner is cheating to
some extent here by inserting Paths belonging to a lower-level rel
into final_rel. At the very least that leaves them with the wrong
parent pointer. We've been aware of that fact for awhile, and it
hasn't bit us yet, so I don't propose to do anything to change it
(yet). What I do suggest is that add_partial_path could check for
a mismatched Path.parent pointer and cowardly refuse to pfree a
Path that was originally made for some other parent rel. The
attached test patch logs the four possible cases, and what I see
in a run of the main regression tests is

$ grep 'freeing dominated old path' postmaster.log | wc
7785 77850 638474
$ grep 'conservatively keeping shared old' postmaster.log | wc
0 0 0
$ grep 'freeing dominated new path' postmaster.log | wc
23871 238710 1957478
$ grep 'conservatively keeping shared new' postmaster.log | wc
0 0 0

The submitted test case hits each of 'freeing dominated old path',
'conservatively keeping shared old', and 'freeing dominated new path'
twice, while not hitting 'conservatively keeping shared new' at all.
(Nonetheless I think we'd better check at both pfree sites.)

These numbers demonstrate that the problem is quite rare and thus
that adding this defense won't cost us much planner memory.

Comments?

regards, tom lane

Attachments:

t259559_4
test-not-freeing-shared-partial-paths.patchtext/x-diff; charset=us-ascii; name=test-not-freeing-shared-partial-paths.patchDownload+14-2
#5Richard Guo
guofenglinux@gmail.com
In reply to: Tom Lane (#4)
Re: BUG #19924: unrecognized node type in create_plan_recurse: UNION, LIMIT, json_populate_recordset, inheritance

On Sat, Nov 21, 2026 at 7:48 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Ah, I identified the problem and a possible fix. The root culprit is
in grouping_planner():

I haven't had a close look at this bug, but I feel that this is the
same bug as [1]/messages/by-id/CAM2+6=XaYtMGwh27O7oDTfAw7kV4VdUVbxC2RVzWtKz_ysekKw@mail.gmail.com, where Jeevan proposed the same parent check. His
reproducer fails on v18 too, so it predates 8300d3ad4.

What I do suggest is that add_partial_path could check for
a mismatched Path.parent pointer and cowardly refuse to pfree a
Path that was originally made for some other parent rel.

+1. I think add_path() should get the same check, as in Jeevan's
patch. create_ordered_paths() passes input_rel's sorted paths to
add_path() unchanged, which breaks the same rule, though it doesn't
crash today.

[1]: /messages/by-id/CAM2+6=XaYtMGwh27O7oDTfAw7kV4VdUVbxC2RVzWtKz_ysekKw@mail.gmail.com

- Richard

#6Tom Lane
tgl@sss.pgh.pa.us
In reply to: Richard Guo (#5)
Re: BUG #19924: unrecognized node type in create_plan_recurse: UNION, LIMIT, json_populate_recordset, inheritance

Richard Guo <guofenglinux@gmail.com> writes:

On Sat, Nov 21, 2026 at 7:48 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Ah, I identified the problem and a possible fix. The root culprit is
in grouping_planner():

I haven't had a close look at this bug, but I feel that this is the
same bug as [1], where Jeevan proposed the same parent check. His
reproducer fails on v18 too, so it predates 8300d3ad4.

Ah, I'd lost track of that thread, but I think you're entirely right
that it's the same bug. The apparent branch-to-branch behavioral
variations are not related to the bug mechanism, but to changing
choices about which path to prefer. Notably, it seems like the
reason 8300d3ad4 made this example fail is precisely that it allowed
startup cost to be considered for partial paths, and thus changed
which partial paths survive the add_partial_path tournament ...
but not which Gather paths survive the add_path tournament.

I concur with the comments on the other thread that reference-counting
Paths, while it's the right thing in principle, is far too invasive
to be attractive (much less back-patchable). This parent-pointer
check idea is surely a hack by comparison, but if it solves all known
cases, let's do it.

I'm unsure that a regression test for this will actually be very
useful; all the known examples seem pretty narrow/brittle and
hence unlikely to be meaningful for more than a version or two.

regards, tom lane