pgsql: Ignore PlaceHolderVars when looking up statistics

Started by Richard Guo11 months ago5 messageshackerscommitters
Jump to latest
#1Richard Guo
rguo@postgresql.org
hackerscommitters

Ignore PlaceHolderVars when looking up statistics

When looking up statistical data about an expression, we failed to
look through PlaceHolderVar nodes, treating them as opaque. This
could prevent us from matching an expression to base columns, index
expressions, or extended statistics, as examine_variable() relies on
strict structural matching.

As a result, queries involving PlaceHolderVar nodes often fell back to
default selectivity estimates, potentially leading to poor plan
choices.

This patch updates examine_variable() to strip PlaceHolderVars before
analysis. This is safe during estimation because PlaceHolderVars are
transparent for the purpose of statistics lookup: they do not alter
the value distribution of the underlying expression.

To minimize performance overhead on this hot path, a lightweight
walker first checks for the presence of PlaceHolderVars. The more
expensive mutator is invoked only when necessary.

There is one ensuing plan change in the regression tests, which is
expected and demonstrates the fix: the rowcount estimate becomes much
more accurate with this patch.

Back-patch to v18. Although this issue exists before that, changes in
this version made it common enough to notice. Given the lack of field
reports for older versions, I am not back-patching further.

Reported-by: Haowu Ge <gehaowu@bitmoe.com>
Author: Richard Guo <guofenglinux@gmail.com>
Discussion: /messages/by-id/62af586c-c270-44f3-9c5e-02c81d537e3d.gehaowu@bitmoe.com
Backpatch-through: 18

Branch
------
REL_18_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/7e9f852a79fe19d4d0f18aabc32a620797fb676e

Modified Files
--------------
src/backend/utils/adt/selfuncs.c | 93 +++++++++++++++++++++++++++++++++-----
src/test/regress/expected/join.out | 27 ++++++++++-
src/test/regress/sql/join.sql | 10 ++++
3 files changed, 116 insertions(+), 14 deletions(-)

#2Robert Haas
robertmhaas@gmail.com
In reply to: Richard Guo (#1)
hackerscommitters
Re: pgsql: Ignore PlaceHolderVars when looking up statistics

On Sun, Dec 28, 2025 at 9:45 PM Richard Guo <rguo@postgresql.org> wrote:

Ignore PlaceHolderVars when looking up statistics

Back-patch to v18. Although this issue exists before that, changes in
this version made it common enough to notice. Given the lack of field
reports for older versions, I am not back-patching further.

Generally we don't back-patch fixes that could change plans, because
it tends to produce user complaints. It's maybe more justifiable in
this case because v18 is quite new and you didn't back-patch to older
releases, but are we sure that it's warranted even there?

--
Robert Haas
EDB: http://www.enterprisedb.com

#3Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#2)
hackerscommitters
Re: pgsql: Ignore PlaceHolderVars when looking up statistics

Robert Haas <robertmhaas@gmail.com> writes:

On Sun, Dec 28, 2025 at 9:45 PM Richard Guo <rguo@postgresql.org> wrote:

Back-patch to v18. Although this issue exists before that, changes in
this version made it common enough to notice. Given the lack of field
reports for older versions, I am not back-patching further.

Generally we don't back-patch fixes that could change plans, because
it tends to produce user complaints. It's maybe more justifiable in
this case because v18 is quite new and you didn't back-patch to older
releases, but are we sure that it's warranted even there?

It's a regression if we don't. See nearby complaint at

/messages/by-id/b75866aa-bb54-456b-8f88-6b5bc52064ca@app.fastmail.com

That case was correctly handled in v17 and before.

regards, tom lane

#4Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#3)
hackerscommitters
Re: pgsql: Ignore PlaceHolderVars when looking up statistics

On Fri, Jan 2, 2026 at 4:20 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

It's a regression if we don't. See nearby complaint at

/messages/by-id/b75866aa-bb54-456b-8f88-6b5bc52064ca@app.fastmail.com

That case was correctly handled in v17 and before.

Oh, OK. Probably makes sense then, since v18 is new.

--
Robert Haas
EDB: http://www.enterprisedb.com

#5Richard Guo
guofenglinux@gmail.com
In reply to: Tom Lane (#3)
hackerscommitters
Re: pgsql: Ignore PlaceHolderVars when looking up statistics

On Sat, Jan 3, 2026 at 6:20 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

Generally we don't back-patch fixes that could change plans, because
it tends to produce user complaints. It's maybe more justifiable in
this case because v18 is quite new and you didn't back-patch to older
releases, but are we sure that it's warranted even there?

It's a regression if we don't.

That's right. I was actually of two minds for a while regarding the
back-patch, but in the end decided to proceed. You can find the
discussions and the reasoning behind this decision in the link below.

/messages/by-id/CAMbWs49h5F66KZwLxaeXoLwHe_9jAB7Eu44UmJhhQpLA38tKhw@mail.gmail.com

- Richard