Improved regular expression error message for backrefs
Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.
You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:
docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t44714psql -h localhost -U postgresBuilt from patchset v1 (message #1), July 27, 2026 at 04:19 PM.
Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:
git clone --branch t44714_1 https://github.com/hackorum-dev/postgres.gitIn a checkout you already have, add the fork once:
git remote add hackorum https://github.com/hackorum-dev/postgres.gitthen, for this patchset and every later one:
git fetch hackorum t44714_1 && git checkout t44714_1Patchset v1 (message #1) is on t44714_1
Hackers,
Please find attached an improvement to the error messages given for invalid backreference usage:
select 'xyz' ~ '(.)(.)\3';
ERROR: invalid regular expression: invalid backreference number
select 'xyz' ~ '(.)(.)(?=\2)';
-ERROR: invalid regular expression: invalid backreference number
+ERROR: invalid regular expression: backreference in lookaround assertionThe first regexp is invalid because only two capture groups exist, so \3 doesn't refer to anything. The second regexp is rejected because the regular expression system does not support backreferences within lookaround assertions. (See the docs, section 9.7.3.6. Limits And Compatibility.) It is flat wrong to say the backreference number is invalid. There is a perfectly valid capture that \2 refers to.
The patch defines a new error code REG_ENOBREF in regex/regex.h right next to REG_ESUBREG from which it is split out, rather than at the end of the list. Is there a project preference to add it at the end? Certainly, that would give a shorter git diff.
Are there dependencies on the current error messages which prevent such changes?
Mark Dilger <mark.dilger@enterprisedb.com> writes:
The patch defines a new error code REG_ENOBREF in regex/regex.h right next to REG_ESUBREG from which it is split out, rather than at the end of the list. Is there a project preference to add it at the end? Certainly, that would give a shorter git diff.
Are there dependencies on the current error messages which prevent such changes?
Yeah: the POSIX standard says what the error codes from regcomp() are.
POSIX defines
REG_ESUBREG
Number in \digit invalid or in error.
which does seem to cover this case, so what I'd argue is that we should
improve the "invalid backreference number" text rather than invent
a nonstandard error code. Maybe about like "backreference number does
not exist or cannot be referenced from here"?
(Admittedly, there's not a huge reason why src/backend/regex/ needs to
stay compliant with the POSIX API today. But I still have ambitions to
split that out as a free-standing library someday, as Henry Spencer had
originally planned to do. So I'd rather stick to the API spec.)
It might be worth checking what text is attributed to this error code
by PCRE and other implementations of the POSIX spec.
regards, tom lane
On Aug 22, 2021, at 7:47 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Yeah: the POSIX standard says what the error codes from regcomp() are.
I'm not sure how to interpret them. The language "The implementation may define additional macros or constants using names beginning with REG_" at the bottom of the docs might imply that one can add to the list.
POSIX defines
REG_ESUBREG
Number in \digit invalid or in error.which does seem to cover this case,
Hmm. The number is neither invalid nor in error. The only thing arguing in favor of using this code is that the error message contains the word "backreference":
"REG_ESUBREG", "invalid backreference number"
which gives the reader a clue that the problem has something to do with a backreference in the pattern. But the POSIX wording "Number in \digit invalid or in error." doesn't even have that advantage. We seem to be using the wrong return code. I would think a more generic
REG_BADPAT
Invalid regular expression.
would be the correct code, though arguably far less informative.
so what I'd argue is that we should
improve the "invalid backreference number" text rather than invent
a nonstandard error code. Maybe about like "backreference number does
not exist or cannot be referenced from here"?
Assuming we leave the error codes alone, how about, "backreference number invalid or cannot be referenced from here"?
(Admittedly, there's not a huge reason why src/backend/regex/ needs to
stay compliant with the POSIX API today. But I still have ambitions to
split that out as a free-standing library someday, as Henry Spencer had
originally planned to do. So I'd rather stick to the API spec.)
That's fine. Something else might kill that ambition, but this quibble over error messages isn't nearly important enough to do so.
It might be worth checking what text is attributed to this error code
by PCRE and other implementations of the POSIX spec.
Reading the docs at pcre.org, it appears that capture groups are allowed in look-around assertions. Our engine doesn't do that, instead treating all groups within assertions as non-capturing. I don't see anything about whether backreferences are allowed within pcre assertions, but I know that perl itself does allow them. So maybe the error text used by other implementations is irrelevant?
—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company