JSON/SQL: jsonpath: incomprehensible error message

Started by Erik Rijkersover 3 years ago8 messages

er@xs4all.nl

over 3 years ago

JSON/SQL jsonpath

For example, a jsonpath string with deliberate typo 'like_regexp'
(instead of 'like_regex'):

select js
from (values (jsonb '{}')) as f(js)
where js @? '$ ? (@ like_regexp "^xxx")';

ERROR: syntax error, unexpected IDENT_P at or near " " of jsonpath input
LINE 1: ...s from (values (jsonb '{}')) as f(js) where js @? '$ ? (@ li...
^

Both 'IDENT_P' and 'at or near " "' seem pretty useless.

Perhaps some improvement can be thought of?

Similar messages in release 14 seem to use 'invalid token', which is better:

select js
from (values (jsonb '{"a":"b"}')) as f(js)
where js @? '$ ? (@.a .= "b")';
ERROR: syntax error, unexpected invalid token at or near "=" of
jsonpath input

thanks,
Erik Rijkers

Andrew Dunstan

andrew@dunslane.net

over 3 years ago

In reply to: Erik Rijkers (#1)

1 attachment(s)

Re: JSON/SQL: jsonpath: incomprehensible error message

On 2022-06-26 Su 11:44, Erik Rijkers wrote:

JSON/SQL jsonpath

For example, a jsonpath string with deliberate typo 'like_regexp'
(instead of 'like_regex'):

select js
from (values (jsonb '{}')) as f(js)
where js @? '$ ? (@ like_regexp "^xxx")';

ERROR: syntax error, unexpected IDENT_P at or near " " of jsonpath input
LINE 1: ...s from (values (jsonb '{}')) as f(js) where js @? '$ ? (@
li...
^

Both 'IDENT_P' and 'at or near " "' seem pretty useless.

Perhaps some improvement can be thought of?

Similar messages in release 14 seem to use 'invalid token', which is
better:

select js
from (values (jsonb '{"a":"b"}')) as f(js)
where js @? '$ ? (@.a .= "b")';
ERROR: syntax error, unexpected invalid token at or near "=" of
jsonpath input

Yeah :-(

This apparently goes back to the original jsonpath commit 72b6460336e.
There are similar error messages in the back branch regression tests:

andrew@ub20:pgl $ grep -r IDENT_P pg_*/src/test/regress/expected/
pg_12/src/test/regress/expected/jsonpath.out:ERROR: syntax error, unexpected IDENT_P at end of jsonpath input
pg_13/src/test/regress/expected/jsonpath.out:ERROR: syntax error, unexpected IDENT_P at end of jsonpath input
pg_14/src/test/regress/expected/jsonpath.out:ERROR: syntax error, unexpected IDENT_P at end of jsonpath input

For some reason the parser contains a '%error-verbose' directive, unlike
all our other bison parsers. Removing that fixes it, as in this patch.
I'm a bit inclined to say we should backpatch the removal of the
directive, but I guess a lack of complaints suggests it's not a huge issue.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Attachments:

jsonpath-parser-fix.patchtext/x-patch; charset=UTF-8; name=jsonpath-parser-fix.patchDownload

diff --git a/src/backend/utils/adt/jsonpath_gram.y b/src/backend/utils/adt/jsonpath_gram.y
index 06d4c8c229..57f6beb27b 100644
--- a/src/backend/utils/adt/jsonpath_gram.y
+++ b/src/backend/utils/adt/jsonpath_gram.y
@@ -74,7 +74,6 @@ static JsonPathParseItem *makeItemLikeRegex(JsonPathParseItem *expr,
 %pure-parser
 %expect 0
 %name-prefix="jsonpath_yy"
-%error-verbose
 %parse-param {JsonPathParseResult **result}
 
 %union
diff --git a/src/test/regress/expected/jsonb_sqljson.out b/src/test/regress/expected/jsonb_sqljson.out
index ec7dc50593..e2f7df50a8 100644
--- a/src/test/regress/expected/jsonb_sqljson.out
+++ b/src/test/regress/expected/jsonb_sqljson.out
@@ -2083,7 +2083,7 @@ SELECT JSON_QUERY(jsonb '{"a": 123}', '$' || '.' || 'a' WITH WRAPPER);
 
 -- Should fail (invalid path)
 SELECT JSON_QUERY(jsonb '{"a": 123}', 'error' || ' ' || 'error');
-ERROR:  syntax error, unexpected IDENT_P at or near " " of jsonpath input
+ERROR:  syntax error at or near " " of jsonpath input
 -- Should fail (not supported)
 SELECT * FROM JSON_TABLE(jsonb '{"a": 123}', '$' || '.' || 'a' COLUMNS (foo int));
 ERROR:  only string constants supported in JSON_TABLE path specification

Amit Kapila

amit.kapila16@gmail.com

over 3 years ago

In reply to: Andrew Dunstan (#2)

Re: JSON/SQL: jsonpath: incomprehensible error message

On Mon, Jun 27, 2022 at 8:46 PM Andrew Dunstan <andrew@dunslane.net> wrote:

On 2022-06-26 Su 11:44, Erik Rijkers wrote:

JSON/SQL jsonpath

For example, a jsonpath string with deliberate typo 'like_regexp'
(instead of 'like_regex'):

select js
from (values (jsonb '{}')) as f(js)
where js @? '$ ? (@ like_regexp "^xxx")';

ERROR: syntax error, unexpected IDENT_P at or near " " of jsonpath input
LINE 1: ...s from (values (jsonb '{}')) as f(js) where js @? '$ ? (@
li...
^

Both 'IDENT_P' and 'at or near " "' seem pretty useless.

Perhaps some improvement can be thought of?

Similar messages in release 14 seem to use 'invalid token', which is
better:

select js
from (values (jsonb '{"a":"b"}')) as f(js)
where js @? '$ ? (@.a .= "b")';
ERROR: syntax error, unexpected invalid token at or near "=" of
jsonpath input

Yeah :-(

This apparently goes back to the original jsonpath commit 72b6460336e.
There are similar error messages in the back branch regression tests:

andrew@ub20:pgl $ grep -r IDENT_P pg_*/src/test/regress/expected/
pg_12/src/test/regress/expected/jsonpath.out:ERROR: syntax error, unexpected IDENT_P at end of jsonpath input
pg_13/src/test/regress/expected/jsonpath.out:ERROR: syntax error, unexpected IDENT_P at end of jsonpath input
pg_14/src/test/regress/expected/jsonpath.out:ERROR: syntax error, unexpected IDENT_P at end of jsonpath input

For some reason the parser contains a '%error-verbose' directive, unlike
all our other bison parsers. Removing that fixes it, as in this patch.
I'm a bit inclined to say we should backpatch the removal of the
directive,

I guess it is okay to backpatch unless we think some user will be
dependent on such a message or there could be other side effects of
removing this. One thing that is not clear to me is why OP sees an
acceptable message (ERROR: syntax error, unexpected invalid token at
or near "=" of jsonpath input) for a similar query in 14?

--
With Regards,
Amit Kapila.

Erik Rijkers

er@xs4all.nl

over 3 years ago

In reply to: Amit Kapila (#3)

Re: JSON/SQL: jsonpath: incomprehensible error message

Op 29-06-2022 om 15:00 schreef Amit Kapila:

On Mon, Jun 27, 2022 at 8:46 PM Andrew Dunstan <andrew@dunslane.net> wrote:

On 2022-06-26 Su 11:44, Erik Rijkers wrote:

JSON/SQL jsonpath

For example, a jsonpath string with deliberate typo 'like_regexp'
(instead of 'like_regex'):

select js
from (values (jsonb '{}')) as f(js)
where js @? '$ ? (@ like_regexp "^xxx")';

ERROR: syntax error, unexpected IDENT_P at or near " " of jsonpath input
LINE 1: ...s from (values (jsonb '{}')) as f(js) where js @? '$ ? (@
li...

Both 'IDENT_P' and 'at or near " "' seem pretty useless.

removing this. One thing that is not clear to me is why OP sees an
acceptable message (ERROR: syntax error, unexpected invalid token at
or near "=" of jsonpath input) for a similar query in 14?

To mention that was perhaps unwise of me because The IDENT_P (or more
generally, *_P) messages can be provoked on 14 too. I just thought
'invalid token' might be a better message because 'token' gives a more
direct association with 'errors during parsing' which I assume is the
case here.

IDENT_P or ANY_P convey exactly nothing.

Erik

Alexander Korotkov

aekorotkov@gmail.com

over 3 years ago

In reply to: Erik Rijkers (#4)

Re: JSON/SQL: jsonpath: incomprehensible error message

On Wed, Jun 29, 2022 at 4:28 PM Erik Rijkers <er@xs4all.nl> wrote:

Op 29-06-2022 om 15:00 schreef Amit Kapila:

On Mon, Jun 27, 2022 at 8:46 PM Andrew Dunstan <andrew@dunslane.net> wrote:

On 2022-06-26 Su 11:44, Erik Rijkers wrote:

JSON/SQL jsonpath

For example, a jsonpath string with deliberate typo 'like_regexp'
(instead of 'like_regex'):

select js
from (values (jsonb '{}')) as f(js)
where js @? '$ ? (@ like_regexp "^xxx")';

ERROR: syntax error, unexpected IDENT_P at or near " " of jsonpath input
LINE 1: ...s from (values (jsonb '{}')) as f(js) where js @? '$ ? (@
li...

Both 'IDENT_P' and 'at or near " "' seem pretty useless.

removing this. One thing that is not clear to me is why OP sees an
acceptable message (ERROR: syntax error, unexpected invalid token at
or near "=" of jsonpath input) for a similar query in 14?

To mention that was perhaps unwise of me because The IDENT_P (or more
generally, *_P) messages can be provoked on 14 too. I just thought
'invalid token' might be a better message because 'token' gives a more
direct association with 'errors during parsing' which I assume is the
case here.

IDENT_P or ANY_P convey exactly nothing.

------
Regards,
Alexander Korotkov

Andrew Dunstan

andrew@dunslane.net

over 3 years ago

In reply to: Alexander Korotkov (#5)

Re: JSON/SQL: jsonpath: incomprehensible error message

On 2022-06-29 We 10:58, Alexander Korotkov wrote:

On Wed, Jun 29, 2022 at 4:28 PM Erik Rijkers <er@xs4all.nl> wrote:

Op 29-06-2022 om 15:00 schreef Amit Kapila:

On Mon, Jun 27, 2022 at 8:46 PM Andrew Dunstan <andrew@dunslane.net> wrote:

On 2022-06-26 Su 11:44, Erik Rijkers wrote:

JSON/SQL jsonpath

For example, a jsonpath string with deliberate typo 'like_regexp'
(instead of 'like_regex'):

select js
from (values (jsonb '{}')) as f(js)
where js @? '$ ? (@ like_regexp "^xxx")';

ERROR: syntax error, unexpected IDENT_P at or near " " of jsonpath input
LINE 1: ...s from (values (jsonb '{}')) as f(js) where js @? '$ ? (@
li...

Both 'IDENT_P' and 'at or near " "' seem pretty useless.

removing this. One thing that is not clear to me is why OP sees an
acceptable message (ERROR: syntax error, unexpected invalid token at
or near "=" of jsonpath input) for a similar query in 14?

To mention that was perhaps unwise of me because The IDENT_P (or more
generally, *_P) messages can be provoked on 14 too. I just thought
'invalid token' might be a better message because 'token' gives a more
direct association with 'errors during parsing' which I assume is the
case here.

IDENT_P or ANY_P convey exactly nothing.

+1

I agree, but I don't think "invalid token" is all that much better. I
think the right fix is just to get rid of the parser setting that causes
production of these additions to the error message, and make it just
like all the other bison parsers we have. Then the problem just disappears.

It's a very slight change of behaviour, but I agree with Amit that we
can backpatch it. I will do so shortly unless there's an objection.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Amit Kapila

amit.kapila16@gmail.com

over 3 years ago

In reply to: Erik Rijkers (#4)

Re: JSON/SQL: jsonpath: incomprehensible error message

On Wed, Jun 29, 2022 at 6:58 PM Erik Rijkers <er@xs4all.nl> wrote:

Op 29-06-2022 om 15:00 schreef Amit Kapila:

On Mon, Jun 27, 2022 at 8:46 PM Andrew Dunstan <andrew@dunslane.net> wrote:

On 2022-06-26 Su 11:44, Erik Rijkers wrote:

JSON/SQL jsonpath

For example, a jsonpath string with deliberate typo 'like_regexp'
(instead of 'like_regex'):

select js
from (values (jsonb '{}')) as f(js)
where js @? '$ ? (@ like_regexp "^xxx")';

ERROR: syntax error, unexpected IDENT_P at or near " " of jsonpath input
LINE 1: ...s from (values (jsonb '{}')) as f(js) where js @? '$ ? (@
li...

Both 'IDENT_P' and 'at or near " "' seem pretty useless.

removing this. One thing that is not clear to me is why OP sees an
acceptable message (ERROR: syntax error, unexpected invalid token at
or near "=" of jsonpath input) for a similar query in 14?

To mention that was perhaps unwise of me because The IDENT_P (or more
generally, *_P) messages can be provoked on 14 too.

Okay, then I think it is better to backpatch this fix.

--
With Regards,
Amit Kapila.

Andrew Dunstan

andrew@dunslane.net

over 3 years ago

In reply to: Amit Kapila (#7)

Re: JSON/SQL: jsonpath: incomprehensible error message

On 2022-06-30 Th 04:19, Amit Kapila wrote:

On Wed, Jun 29, 2022 at 6:58 PM Erik Rijkers <er@xs4all.nl> wrote:

Op 29-06-2022 om 15:00 schreef Amit Kapila:

On Mon, Jun 27, 2022 at 8:46 PM Andrew Dunstan <andrew@dunslane.net> wrote:

On 2022-06-26 Su 11:44, Erik Rijkers wrote:

JSON/SQL jsonpath

For example, a jsonpath string with deliberate typo 'like_regexp'
(instead of 'like_regex'):

select js
from (values (jsonb '{}')) as f(js)
where js @? '$ ? (@ like_regexp "^xxx")';

ERROR: syntax error, unexpected IDENT_P at or near " " of jsonpath input
LINE 1: ...s from (values (jsonb '{}')) as f(js) where js @? '$ ? (@
li...

Both 'IDENT_P' and 'at or near " "' seem pretty useless.

removing this. One thing that is not clear to me is why OP sees an
acceptable message (ERROR: syntax error, unexpected invalid token at
or near "=" of jsonpath input) for a similar query in 14?

To mention that was perhaps unwise of me because The IDENT_P (or more
generally, *_P) messages can be provoked on 14 too.

Okay, then I think it is better to backpatch this fix.

Done.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com