BUG #19693: JSON_VALUE/JSON_QUERY PASSING a toasted text value reads the toast pointer instead of the text
Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.
This thread has been committed, so CI has stopped here. Anything below is the last result it produced.
You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:
docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t253840psql -h localhost -U postgresBuilt from patchset v3 (message #3), September 19, 2026 at 03:46 AM.
Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:
git clone --branch t253840_3 https://github.com/hackorum-dev/postgres.gitIn a checkout you already have, add the fork once:
git remote add hackorum https://github.com/hackorum-dev/postgres.gitthen, for this patchset and every later one:
git fetch hackorum t253840_3 && git checkout t253840_3Patchset v3 (message #3) is on t253840_3
The following bug has been logged on the website:
Bug reference: 19693
Logged by: Chaitanya Choudhary
Email address: chaitanyyachoudhary@gmail.com
PostgreSQL version: 18.6
Operating system: macOS 26 (aarch64), Homebrew build of 18.6
Description:
When a text or varchar column value is passed to a SQL/JSON query function
through PASSING and the value is stored out of line (TOAST), the jsonpath
variable does not contain the text. It contains a few bytes of the toast
pointer.
Steps to reproduce:
CREATE TABLE t (c text);
ALTER TABLE t ALTER COLUMN c SET STORAGE EXTERNAL;
INSERT INTO t VALUES (repeat('x', 10000));
SELECT length(c) AS stored,
length(JSON_VALUE('{}', '$x' PASSING c AS x)) AS via_passing
FROM t;
Result:
stored | via_passing
--------+-------------
10000 | 3
Expected: 10000 in both columns.
The 3-character result is not part of the stored value:
SELECT JSON_VALUE('{}', '$x' PASSING c AS x) = c FROM t; -- f
SELECT left(JSON_VALUE('{}', '$x' PASSING c AS x), 20) FROM t; --
\x12\x14'
Forcing a detoast before PASSING gives the right answer:
SELECT length(JSON_VALUE('{}', '$x' PASSING (c || '') AS x)) FROM t; --
10000
A short value, which is stored inline, also works. JSON_QUERY and
JSON_EXISTS are affected the same way; for example
SELECT JSON_EXISTS('{}', '$x ? (@ starts with "xxxxxxxxxx")' PASSING c AS
x) FROM t;
returns false for a value of ten thousand x's.
Cause:
In src/backend/utils/adt/jsonpath_exec.c, JsonItemFromDatum() handles
TEXTOID and VARCHAROID by reading the datum directly:
case TEXTOID:
case VARCHAROID:
res->type = jbvString;
res->val.string.val = VARDATA_ANY(val);
res->val.string.len = VARSIZE_ANY_EXHDR(val);
break;
The datum is never detoasted, so for an out-of-line value the macros read
the toast pointer's own bytes. The value comes from the PASSING argument
via GetJsonPathVar() -> JsonItemFromDatum(var->value, ...) and nothing on
that path detoasts it either. The other varlena cases in this function
(JSONB, and the datetime types through their output paths) go through code
that detoasts.
Fix: detoast the datum in that case, for example
text *txt = DatumGetTextPP(val);
res->val.string.val = VARDATA_ANY(txt);
res->val.string.len = VARSIZE_ANY_EXHDR(txt);
The same code is present on REL_18_STABLE and master as of 2026-09-17.
On Fri, Sep 18, 2026 at 02:39:08AM +0000, PG Bug reporting form wrote:
The datum is never detoasted, so for an out-of-line value the macros read
the toast pointer's own bytes. The value comes from the PASSING argument
via GetJsonPathVar() -> JsonItemFromDatum(var->value, ...) and nothing on
that path detoasts it either. The other varlena cases in this function
(JSONB, and the datetime types through their output paths) go through code
that detoasts.Fix: detoast the datum in that case, for example
text *txt = DatumGetTextPP(val);
res->val.string.val = VARDATA_ANY(txt);
res->val.string.len = VARSIZE_ANY_EXHDR(txt);The same code is present on REL_18_STABLE and master as of 2026-09-17.
At quick glance, your take seems right and that looks like an
oversight. Would you like to write a patch?
--
Michael
Hi,
I can reproduce this on master. It is not limited to EXTERNAL storage.
With the default storage an inline compressed value hits it too, so any
large text column passed with PASSING gives a wrong result.
Chaitanya, I hope you don't mind that I wrote it up. The fix is the one
you suggested.
Thanks,
Shihao
Attachments:
t253840_3v1-0001-Fix-PASSING-of-toasted-text-values-in-JSON-query-.patchapplication/octet-stream; name=v1-0001-Fix-PASSING-of-toasted-text-values-in-JSON-query-.patchDownload+9-5
v1-0002-Add-tests-for-PASSING-of-toasted-text-values.patchapplication/octet-stream; name=v1-0002-Add-tests-for-PASSING-of-toasted-text-values.patchDownload+14-1
On Fri, Sep 18, 2026 at 11:34:37PM -0400, shihao zhong wrote:
I can reproduce this on master. It is not limited to EXTERNAL storage.
With the default storage an inline compressed value hits it too, so any
large text column passed with PASSING gives a wrong result.
Yep, that sounds pretty much right, but it also feels like the tests
could be extended a bit. Will adjust a few things and fix down to
v17.
Thanks for the report.
--
Michael