​jsonb @@ jsonpath operator doc: ​Only the first item of the result is taken into account

On 01/04/2023 08:02 CEST jian he <jian.universality@gmail.com> wrote:

Hi,
https://www.postgresql.org/docs/current/functions-json.html

jsonb @@ jsonpath → boolean
Returns the result of a JSON path predicate check for the specified JSON
value. Only the first item of the result is taken into account. If the
result is not Boolean, then NULL is returned.
'{"a":[1,2,3,4,5]}'::jsonb @@ '$.a[*] > 2' → t

select jsonb_path_query('{"a":[1,2,3,4,5]}', '$.a[*]');
return

jsonb_path_query
------------------
 1
 2
 3
 4
 5
(5 rows)

I don't understand:"Only the first item of the result is taken into account.".

Here,JSON path predicate check for the specified JSON valuereturn true, some
return false. (1 > 2 is false, 2 > 2 is false).

The result is true if any array element matches the predicate because predicates
are evaluated on sequences. The documentation for executePredicate in
src/backend/utils/adt/jsonpath_exec.c explains it:

Predicates have existence semantics, because their operands are item
sequences. Pairs of items from the left and right operand's sequences are
checked. TRUE returned only if any pair satisfying the condition is found.
In strict mode, even if the desired pair has already been found, all pairs
still need to be examined to check the absence of errors. If any error
occurs, UNKNOWN (analogous to SQL NULL) is returned.

https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/utils/adt/jsonpath_exec.c;h=b561f0e7e803f0e5a546ad118a47f625225b9708;hb=HEAD#l1461

Difference between using a predicate as path expression vs filter expression:

=# select jsonb_path_query('{"a":[1,2,3,4,5]}', '$.a[*] > 2');
jsonb_path_query
------------------
true
(1 row)

=# select jsonb_path_query('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ > 2)');
jsonb_path_query
------------------
3
4
5
(3 rows)

If you want the predicate result for each element, you must apply the predicate
to the rows returned from jsonb_path_query:

=# select elem, elem::float > 2 as pred from jsonb_path_query('{"a":[1,2,3,4,5]}', '$.a[*]') elem;
elem | pred
------+------
1 | f
2 | f
3 | t
4 | t
5 | t
(5 rows)

--
Erik

Erik Wienhold <ewie@ewie.name> writes:

The result is true if any array element matches the predicate because predicates
are evaluated on sequences. The documentation for executePredicate in
src/backend/utils/adt/jsonpath_exec.c explains it:

I think the OP is correct that this is undocumented at the user level.
Want to propose some doc text?

regards, tom lane

On 4/2/23 06:21, Erik Wienhold wrote:

On 01/04/2023 08:02 CEST jian he <jian.universality@gmail.com> wrote:

Hi,
https://www.postgresql.org/docs/current/functions-json.html

jsonb @@ jsonpath → boolean
Returns the result of a JSON path predicate check for the specified JSON
value. Only the first item of the result is taken into account. If the
result is not Boolean, then NULL is returned.
'{"a":[1,2,3,4,5]}'::jsonb @@ '$.a[*] > 2' → t

select jsonb_path_query('{"a":[1,2,3,4,5]}', '$.a[*]');
return

jsonb_path_query
------------------
 1
 2
 3
 4
 5
(5 rows)

I don't understand:"Only the first item of the result is taken into account.".

Here,JSON path predicate check for the specified JSON valuereturn true, some
return false. (1 > 2 is false, 2 > 2 is false).

The result is true if any array element matches the predicate because predicates
are evaluated on sequences. The documentation for executePredicate in
src/backend/utils/adt/jsonpath_exec.c explains it:

Predicates have existence semantics, because their operands are item
sequences. Pairs of items from the left and right operand's sequences are
checked. TRUE returned only if any pair satisfying the condition is found.
In strict mode, even if the desired pair has already been found, all pairs
still need to be examined to check the absence of errors. If any error
occurs, UNKNOWN (analogous to SQL NULL) is returned.

https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/utils/adt/jsonpath_exec.c;h=b561f0e7e803f0e5a546ad118a47f625225b9708;hb=HEAD#l1461

Difference between using a predicate as path expression vs filter expression:

=# select jsonb_path_query('{"a":[1,2,3,4,5]}', '$.a[*] > 2');
jsonb_path_query
------------------
true
(1 row)

=# select jsonb_path_query('{"a":[1,2,3,4,5]}', '$.a[*] ? (@ > 2)');
jsonb_path_query
------------------
3
4
5
(3 rows)

If you want the predicate result for each element, you must apply the predicate
to the rows returned from jsonb_path_query:

=# select elem, elem::float > 2 as pred from jsonb_path_query('{"a":[1,2,3,4,5]}', '$.a[*]') elem;
elem | pred
------+------
1 | f
2 | f
3 | t
4 | t
5 | t
(5 rows)

That is a long way from:

jsonb @@ jsonpath → boolean

Returns the result of a JSON path predicate check for the specified JSON
value. Only the first item of the result is taken into account. If the
result is not Boolean, then NULL is returned.

--
Erik

--
Adrian Klaver
adrian.klaver@aklaver.com

On 02/04/2023 17:40 CEST Adrian Klaver <adrian.klaver@aklaver.com> wrote:

That is a long way from:

jsonb @@ jsonpath → boolean

Returns the result of a JSON path predicate check for the specified JSON
value. Only the first item of the result is taken into account. If the
result is not Boolean, then NULL is returned.

What do you mean? I responded to the OP's question. It's not a suggestion
to update the docs. Obviously it's quite a mouthful and needs to be boiled
down for the docs. Any suggestions?

--
Erik

Erik Wienhold <ewie@ewie.name> writes:

What do you mean? I responded to the OP's question. It's not a suggestion
to update the docs. Obviously it's quite a mouthful and needs to be boiled
down for the docs. Any suggestions?

At least for the example at hand, I think a good part of the confusion
stems from the undocumented way that the boolean test operator works.
As far as I can find, the only mention of that is in Section 9.16.2 [1]https://www.postgresql.org/docs/devel/functions-json.html#FUNCTIONS-SQLJSON-PATH,
which says (just before 9.16.2.1):

PostgreSQL's implementation of the SQL/JSON path language has the
following deviations from the SQL/JSON standard:

A path expression can be a Boolean predicate, although the
SQL/JSON standard allows predicates only in filters. This is
necessary for implementation of the @@ operator. For example, the
following jsonpath expression is valid in PostgreSQL:

$.track.segments[*].HR < 70

That is a complete documentation fail if you ask me, because it doesn't
explain what the construct actually does, nor is there anyplace else
where you could find that out, not even the SQL standard.

So I'd start by fleshing that out, and then see what to do with the
description of the @@ operator (and the equivalent jsonb_path_match
function). Some more examples for those might help.

regards, tom lane

[1]: https://www.postgresql.org/docs/devel/functions-json.html#FUNCTIONS-SQLJSON-PATH

On 4/3/23 08:11, Erik Wienhold wrote:

On 02/04/2023 17:40 CEST Adrian Klaver <adrian.klaver@aklaver.com> wrote:

That is a long way from:

jsonb @@ jsonpath → boolean

Returns the result of a JSON path predicate check for the specified JSON
value. Only the first item of the result is taken into account. If the
result is not Boolean, then NULL is returned.

What do you mean? I responded to the OP's question. It's not a suggestion
to update the docs. Obviously it's quite a mouthful and needs to be boiled
down for the docs. Any suggestions?

For me I don't see how:

Predicates have existence semantics, because their operands are item
sequences. Pairs of items from the left and right operand's sequences
are checked. TRUE returned only if any pair satisfying the condition is
found. In strict mode, even if the desired pair has already been found,
all pairs still need to be examined to check the absence of errors. If
any error occurs, UNKNOWN (analogous to SQL NULL) is returned.

resolves to :

Only the first item of the result is taken into account.

In other words reconciling "TRUE returned only if any pair satisfying
the condition is found." and "...first item of the result..."

--
Erik

--
Adrian Klaver
adrian.klaver@aklaver.com

On 03/04/2023 17:36 CEST Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 4/3/23 08:11, Erik Wienhold wrote:

On 02/04/2023 17:40 CEST Adrian Klaver <adrian.klaver@aklaver.com> wrote:

That is a long way from:

jsonb @@ jsonpath → boolean

Returns the result of a JSON path predicate check for the specified JSON
value. Only the first item of the result is taken into account. If the
result is not Boolean, then NULL is returned.

What do you mean? I responded to the OP's question. It's not a suggestion
to update the docs. Obviously it's quite a mouthful and needs to be boiled
down for the docs. Any suggestions?

For me I don't see how:

Predicates have existence semantics, because their operands are item
sequences. Pairs of items from the left and right operand's sequences
are checked. TRUE returned only if any pair satisfying the condition is
found. In strict mode, even if the desired pair has already been found,
all pairs still need to be examined to check the absence of errors. If
any error occurs, UNKNOWN (analogous to SQL NULL) is returned.

resolves to :

Only the first item of the result is taken into account.

In other words reconciling "TRUE returned only if any pair satisfying
the condition is found." and "...first item of the result..."

I see.

Thinking about it now, I believe that "first item of the result" is redundant
(and causing the OP's confusion) because the path predicate produces only a
single item: true, false, or null. That's what I wanted to show with the first
two jsonb_path_query examples in my initial response, where the second example
returns multiple items.

I think the gist of @@ and json_path_match is:

"Returns true if any JSON value at the given path matches the predicate.
Returns NULL when not a path predicate or comparing different types."

--
Erik

On 4/3/23 09:21, Erik Wienhold wrote:

On 03/04/2023 17:36 CEST Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 4/3/23 08:11, Erik Wienhold wrote:

On 02/04/2023 17:40 CEST Adrian Klaver <adrian.klaver@aklaver.com> wrote:

That is a long way from:

jsonb @@ jsonpath → boolean

Returns the result of a JSON path predicate check for the specified JSON
value. Only the first item of the result is taken into account. If the
result is not Boolean, then NULL is returned.

What do you mean? I responded to the OP's question. It's not a suggestion
to update the docs. Obviously it's quite a mouthful and needs to be boiled
down for the docs. Any suggestions?

For me I don't see how:

Predicates have existence semantics, because their operands are item
sequences. Pairs of items from the left and right operand's sequences
are checked. TRUE returned only if any pair satisfying the condition is
found. In strict mode, even if the desired pair has already been found,
all pairs still need to be examined to check the absence of errors. If
any error occurs, UNKNOWN (analogous to SQL NULL) is returned.

resolves to :

Only the first item of the result is taken into account.

In other words reconciling "TRUE returned only if any pair satisfying
the condition is found." and "...first item of the result..."

I see.

Thinking about it now, I believe that "first item of the result" is redundant
(and causing the OP's confusion) because the path predicate produces only a
single item: true, false, or null. That's what I wanted to show with the first
two jsonb_path_query examples in my initial response, where the second example
returns multiple items.

I think the gist of @@ and json_path_match is:

"Returns true if any JSON value at the given path matches the predicate.
Returns NULL when not a path predicate or comparing different types."

So basically a variation of jsonb @? jsonpath that returns NULL instead
of false when confused:

select '{"a":[1,2,3,4,5]}'::jsonb @? '$.a[*] ? (@ == "test")' ;
?column?
----------
f

select '{"a":[1,2,3,4,5]}'::jsonb @@ '$.a[*] == "test"' ;
?column?
----------
NULL

Otherwise it does the same thing:

select '{"a":[1,2,3,4,5]}'::jsonb @? '$.a[*] ? (@ > 2)' ;
?column?
----------
t

select '{"a":[1,2,3,4,5]}'::jsonb @@ '$.a[*] > 2' ;
?column?
----------
t

--
Erik

--
Adrian Klaver
adrian.klaver@aklaver.com

On 03/04/2023 18:37 CEST Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 4/3/23 09:21, Erik Wienhold wrote:

On 03/04/2023 17:36 CEST Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 4/3/23 08:11, Erik Wienhold wrote:

On 02/04/2023 17:40 CEST Adrian Klaver <adrian.klaver@aklaver.com> wrote:

That is a long way from:

jsonb @@ jsonpath → boolean

Returns the result of a JSON path predicate check for the specified JSON
value. Only the first item of the result is taken into account. If the
result is not Boolean, then NULL is returned.

What do you mean? I responded to the OP's question. It's not a suggestion
to update the docs. Obviously it's quite a mouthful and needs to be boiled
down for the docs. Any suggestions?

For me I don't see how:

Predicates have existence semantics, because their operands are item
sequences. Pairs of items from the left and right operand's sequences
are checked. TRUE returned only if any pair satisfying the condition is
found. In strict mode, even if the desired pair has already been found,
all pairs still need to be examined to check the absence of errors. If
any error occurs, UNKNOWN (analogous to SQL NULL) is returned.

resolves to :

Only the first item of the result is taken into account.

In other words reconciling "TRUE returned only if any pair satisfying
the condition is found." and "...first item of the result..."

I see.

Thinking about it now, I believe that "first item of the result" is redundant
(and causing the OP's confusion) because the path predicate produces only a
single item: true, false, or null. That's what I wanted to show with the first
two jsonb_path_query examples in my initial response, where the second example
returns multiple items.

I think the gist of @@ and json_path_match is:

"Returns true if any JSON value at the given path matches the predicate.
Returns NULL when not a path predicate or comparing different types."

So basically a variation of jsonb @? jsonpath that returns NULL instead
of false when confused:

select '{"a":[1,2,3,4,5]}'::jsonb @? '$.a[*] ? (@ == "test")' ;
?column?
----------
f

The filter expression does not match any values because predicate '@ == "test"'
returns unknown. This follows SQL's three-valued logic.

" ? (condition)

[...] The result of that step is filtered to include only those items that
satisfy the provided condition. SQL/JSON defines three-valued logic, so the
condition can be true, false, or unknown. The unknown value plays the same role
as SQL NULL and can be tested for with the is unknown predicate. Further path
evaluation steps use only those items for which the filter expression returned
true." https://www.postgresql.org/docs/current/functions-json.html

select '{"a":[1,2,3,4,5]}'::jsonb @@ '$.a[*] == "test"' ;
?column?
----------
NULL

In this case @@ returns null because the predicate returns unknown for all array
elements. It gets interesting in strict mode.

Lax mode (default) with an array element of matching type found by the predicate:

select '{"a":[1,2,3,4,5,"test"]}'::jsonb @@ '$.a[*] == "test"';
?column?
----------
t
(1 row)

In strict mode the unknown result for the first array element causes the
predicate evaluation to short-circuit and return unknown right away instead
of testing the remaining elements:

select '{"a":[1,2,3,4,5,"test"]}'::jsonb @@ 'strict $.a[*] == "test"';
?column?
----------
NULL
(1 row)

Otherwise it does the same thing:

select '{"a":[1,2,3,4,5]}'::jsonb @? '$.a[*] ? (@ > 2)' ;
?column?
----------
t

select '{"a":[1,2,3,4,5]}'::jsonb @@ '$.a[*] > 2' ;
?column?
----------
t

Yes, if the LHS and RHS types match. The equivalence is also supported by
documentation in src/backend/utils/adt/jsonb_gin.c:

The operators support, among the others, "jsonb @? jsonpath" and
"jsonb @@ jsonpath". Expressions containing these operators are easily
expressed through each other.

jb @? 'path' <=> jb @@ 'EXISTS(path)'
jb @@ 'expr' <=> jb @? '$ ? (expr)'

https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/utils/adt/jsonb_gin.c;h=e941439d7493365f8954c791f0e2368c080189b8;hb=HEAD#l15

--
Erik

On 4/3/23 12:00, Erik Wienhold wrote:

On 03/04/2023 18:37 CEST Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 4/3/23 09:21, Erik Wienhold wrote:

On 03/04/2023 17:36 CEST Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 4/3/23 08:11, Erik Wienhold wrote:

On 02/04/2023 17:40 CEST Adrian Klaver <adrian.klaver@aklaver.com> wrote:

That is a long way from:

jsonb @@ jsonpath → boolean

Returns the result of a JSON path predicate check for the specified JSON
value. Only the first item of the result is taken into account. If the
result is not Boolean, then NULL is returned.

What do you mean? I responded to the OP's question. It's not a suggestion
to update the docs. Obviously it's quite a mouthful and needs to be boiled
down for the docs. Any suggestions?

For me I don't see how:

Predicates have existence semantics, because their operands are item
sequences. Pairs of items from the left and right operand's sequences
are checked. TRUE returned only if any pair satisfying the condition is
found. In strict mode, even if the desired pair has already been found,
all pairs still need to be examined to check the absence of errors. If
any error occurs, UNKNOWN (analogous to SQL NULL) is returned.

resolves to :

Only the first item of the result is taken into account.

In other words reconciling "TRUE returned only if any pair satisfying
the condition is found." and "...first item of the result..."

I see.

Thinking about it now, I believe that "first item of the result" is redundant
(and causing the OP's confusion) because the path predicate produces only a
single item: true, false, or null. That's what I wanted to show with the first
two jsonb_path_query examples in my initial response, where the second example
returns multiple items.

I think the gist of @@ and json_path_match is:

"Returns true if any JSON value at the given path matches the predicate.
Returns NULL when not a path predicate or comparing different types."

So basically a variation of jsonb @? jsonpath that returns NULL instead
of false when confused:

select '{"a":[1,2,3,4,5]}'::jsonb @? '$.a[*] ? (@ == "test")' ;
?column?
----------
f

The filter expression does not match any values because predicate '@ == "test"'
returns unknown. This follows SQL's three-valued logic.

" ? (condition)

[...] The result of that step is filtered to include only those items that
satisfy the provided condition. SQL/JSON defines three-valued logic, so the
condition can be true, false, or unknown. The unknown value plays the same role
as SQL NULL and can be tested for with the is unknown predicate. Further path
evaluation steps use only those items for which the filter expression returned
true." https://www.postgresql.org/docs/current/functions-json.html

select '{"a":[1,2,3,4,5]}'::jsonb @@ '$.a[*] == "test"' ;
?column?
----------
NULL

In this case @@ returns null because the predicate returns unknown for all array
elements. It gets interesting in strict mode.

Lax mode (default) with an array element of matching type found by the predicate:

select '{"a":[1,2,3,4,5,"test"]}'::jsonb @@ '$.a[*] == "test"';
?column?
----------
t
(1 row)

In strict mode the unknown result for the first array element causes the
predicate evaluation to short-circuit and return unknown right away instead
of testing the remaining elements:

select '{"a":[1,2,3,4,5,"test"]}'::jsonb @@ 'strict $.a[*] == "test"';
?column?
----------
NULL
(1 row)

So from here:

https://www.postgresql.org/docs/current/functions-json.html

"
Note

The jsonpath operators @? and @@ suppress the following errors: missing
object field or array element, unexpected JSON item type, datetime and
numeric errors. The jsonpath-related functions described below can also
be told to suppress these types of errors. This behavior might be
helpful when searching JSON document collections of varying structure.
"

Needs to be updated to indicate that the above is true when in the
default lax mode, but changes if strict mode is specified.

Otherwise it does the same thing:

select '{"a":[1,2,3,4,5]}'::jsonb @? '$.a[*] ? (@ > 2)' ;
?column?
----------
t

select '{"a":[1,2,3,4,5]}'::jsonb @@ '$.a[*] > 2' ;
?column?
----------
t

Yes, if the LHS and RHS types match. The equivalence is also supported by
documentation in src/backend/utils/adt/jsonb_gin.c:

The operators support, among the others, "jsonb @? jsonpath" and
"jsonb @@ jsonpath". Expressions containing these operators are easily
expressed through each other.

jb @? 'path' <=> jb @@ 'EXISTS(path)'
jb @@ 'expr' <=> jb @? '$ ? (expr)'

https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/utils/adt/jsonb_gin.c;h=e941439d7493365f8954c791f0e2368c080189b8;hb=HEAD#l15

--
Erik

--
Adrian Klaver
adrian.klaver@aklaver.com

On Tue, Apr 4, 2023 at 12:22 AM Erik Wienhold <ewie@ewie.name> wrote:

On 03/04/2023 17:36 CEST Adrian Klaver <adrian.klaver@aklaver.com>

wrote:

On 4/3/23 08:11, Erik Wienhold wrote:

On 02/04/2023 17:40 CEST Adrian Klaver <adrian.klaver@aklaver.com>

wrote:

That is a long way from:

jsonb @@ jsonpath → boolean

Returns the result of a JSON path predicate check for the specified

JSON

value. Only the first item of the result is taken into account. If the
result is not Boolean, then NULL is returned.

What do you mean? I responded to the OP's question. It's not a

suggestion

to update the docs. Obviously it's quite a mouthful and needs to be

boiled

down for the docs. Any suggestions?

For me I don't see how:

Predicates have existence semantics, because their operands are item
sequences. Pairs of items from the left and right operand's sequences
are checked. TRUE returned only if any pair satisfying the condition is
found. In strict mode, even if the desired pair has already been found,
all pairs still need to be examined to check the absence of errors. If
any error occurs, UNKNOWN (analogous to SQL NULL) is returned.

resolves to :

Only the first item of the result is taken into account.

In other words reconciling "TRUE returned only if any pair satisfying
the condition is found." and "...first item of the result..."

I see.

Thinking about it now, I believe that "first item of the result" is
redundant
(and causing the OP's confusion) because the path predicate produces only a
single item: true, false, or null. That's what I wanted to show with the
first
two jsonb_path_query examples in my initial response, where the second
example
returns multiple items.

I think the gist of @@ and json_path_match is:

"Returns true if any JSON value at the given path matches the predicate.
Returns NULL when not a path predicate or comparing different types."

--
Erik

"Returns true if any JSON value at the given path matches the predicate.

Returns NULL when not a path predicate or comparing different types."

in first sentence, should we add something "otherwise return false." ?
also, should it be "Return true"? (since only one value returned)?

On 04/04/2023 03:50 CEST jian he <jian.universality@gmail.com> wrote:

"Returns true if any JSON value at the given path matches the predicate.
Returns NULL when not a path predicate or comparing different types."

in first sentence, should we add something "otherwise return false." ?

I omitted the "otherwise false" part because of the corner cases which I did
not want to gloss over. But the corner cases also apply if the predicate
matches some value, depending on strict mode, as I noticed later and wrote in
my previous message.

Suggestion:

"Returns true if any JSON value at the given path matches the predicate,
otherwise returns false. Unless the predicate compares different types
(depending on strict mode) or the jsonpath is not a path predicate, in
which case NULL is returned."

I guess it's best to document the corner cases in detail in the notes section
as Adrian pointed out and have the function doc refer to the notes.

also, should it be "Return true"? (since only one value returned)?

The third-person singular "returns" is correct in this case. It does not refer
to the number of returned values.

--
Erik

On 4/4/23 02:01, Erik Wienhold wrote:

On 04/04/2023 03:50 CEST jian he <jian.universality@gmail.com> wrote:

"Returns true if any JSON value at the given path matches the predicate.
Returns NULL when not a path predicate or comparing different types."

in first sentence, should we add something "otherwise return false." ?

I omitted the "otherwise false" part because of the corner cases which I did
not want to gloss over. But the corner cases also apply if the predicate
matches some value, depending on strict mode, as I noticed later and wrote in
my previous message.

Suggestion:

"Returns true if any JSON value at the given path matches the predicate,
otherwise returns false. Unless the predicate compares different types
(depending on strict mode) or the jsonpath is not a path predicate, in
which case NULL is returned."

That I understand. Thanks.

I guess it's best to document the corner cases in detail in the notes section
as Adrian pointed out and have the function doc refer to the notes.

also, should it be "Return true"? (since only one value returned)?

The third-person singular "returns" is correct in this case. It does not refer
to the number of returned values.

--
Erik

--
Adrian Klaver
adrian.klaver@aklaver.com