eval_const_expresisions and ScalarArrayOpExpr

Heikki Linnakangas <hlinnaka@iki.fi> writes:

Eval_const_expressions() doesn't know about ScalarArrayOpExpr.
...
That seems like an oversight. I guess that scenario doesn't happen very
often in practice, but there's no reason not to do it when it does.
Patch attached.

Yeah, I think it's a lack-of-round-tuits situation. Your patch reminds
me of a more ambitious attempt I made awhile back to reduce the amount
of duplicative boilerplate in eval_const_expressions. I think I wrote
it during last year's beta period, stashed it because I didn't feel like
arguing for applying it right then, and then forgot about it. Blowing
the dust off, it's attached. It fixes ArrayRef and RowExpr as well as
ScalarArrayOpExpr, with a net growth of only 20 lines (largely comments).

On a side-note, I find it a bit awkward that ScalarArrayOpExpr uses a
2-element List to hold the scalar and array arguments. Wouldn't it be
much more straightforward to have explicit "Expr *scalararg" and "Expr
*arrayarg" fields?

I think it's modeled on OpExpr, which also uses a list though you could
argue for separate leftarg and rightarg fields instead.

regards, tom lane

On 05/11/2017 06:21 PM, Tom Lane wrote:

Heikki Linnakangas <hlinnaka@iki.fi> writes:

Eval_const_expressions() doesn't know about ScalarArrayOpExpr.
...
That seems like an oversight. I guess that scenario doesn't happen very
often in practice, but there's no reason not to do it when it does.
Patch attached.

Yeah, I think it's a lack-of-round-tuits situation. Your patch reminds
me of a more ambitious attempt I made awhile back to reduce the amount
of duplicative boilerplate in eval_const_expressions. I think I wrote
it during last year's beta period, stashed it because I didn't feel like
arguing for applying it right then, and then forgot about it.

Hmph, now we're almost in beta period again. :-(.

Blowing the dust off, it's attached. It fixes ArrayRef and RowExpr as
well as ScalarArrayOpExpr, with a net growth of only 20 lines
(largely comments).

Nice!

On a side-note, I find it a bit awkward that ScalarArrayOpExpr uses a
2-element List to hold the scalar and array arguments. Wouldn't it be
much more straightforward to have explicit "Expr *scalararg" and "Expr
*arrayarg" fields?

I think it's modeled on OpExpr, which also uses a list though you could
argue for separate leftarg and rightarg fields instead.

Yeah, I think that would be better for OpExpr, too. (For an unary
operator, rightarg would be unused.)

- Heikki

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Heikki Linnakangas <hlinnaka@iki.fi> writes:

On 05/11/2017 06:21 PM, Tom Lane wrote:

On a side-note, I find it a bit awkward that ScalarArrayOpExpr uses a
2-element List to hold the scalar and array arguments. Wouldn't it be
much more straightforward to have explicit "Expr *scalararg" and "Expr
*arrayarg" fields?

I think it's modeled on OpExpr, which also uses a list though you could
argue for separate leftarg and rightarg fields instead.

Yeah, I think that would be better for OpExpr, too. (For an unary
operator, rightarg would be unused.)

I should think leftarg is the one that would go unused, for a normal
prefix unary operator.

But it seems like changing this would be way more invasive than it's
really worth. We'd save a couple of List cells per operator, which
is certainly something, but I'm afraid you'd be touching a heck of
a lot of code. And there are a nontrivial number of places that
share argument-processing with FuncExprs, which such a change would
break.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Heikki Linnakangas <hlinnaka@iki.fi> writes:

On 05/11/2017 06:21 PM, Tom Lane wrote:

Yeah, I think it's a lack-of-round-tuits situation. Your patch reminds
me of a more ambitious attempt I made awhile back to reduce the amount
of duplicative boilerplate in eval_const_expressions. I think I wrote
it during last year's beta period, stashed it because I didn't feel like
arguing for applying it right then, and then forgot about it.

Hmph, now we're almost in beta period again. :-(.

Actually, I now remember that I wrote that while at Salesforce (because
they'd run into the problem that constant ArrayRef expressions weren't
simplified). So that means it's been languishing in a git branch for
*two* years. I'm kind of inclined to push it before it gets forgotten
again ;-)

Looking at the patch, it still seems solid, but I remember that one
thing I was concerned about was whether the more generalized code
was any slower. Not sure about a good test case to measure that
though --- const-simplification isn't a large chunk of most workloads.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, May 12, 2017 at 12:04 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Heikki Linnakangas <hlinnaka@iki.fi> writes:

On 05/11/2017 06:21 PM, Tom Lane wrote:

Yeah, I think it's a lack-of-round-tuits situation. Your patch reminds
me of a more ambitious attempt I made awhile back to reduce the amount
of duplicative boilerplate in eval_const_expressions. I think I wrote
it during last year's beta period, stashed it because I didn't feel like
arguing for applying it right then, and then forgot about it.

Hmph, now we're almost in beta period again. :-(.

Actually, I now remember that I wrote that while at Salesforce (because
they'd run into the problem that constant ArrayRef expressions weren't
simplified). So that means it's been languishing in a git branch for
*two* years. I'm kind of inclined to push it before it gets forgotten
again ;-)

You will probably not be surprised to hear that I think this is a
feature and thus subject to the feature freeze currently in effect.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Robert Haas <robertmhaas@gmail.com> writes:

On Fri, May 12, 2017 at 12:04 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Actually, I now remember that I wrote that while at Salesforce (because
they'd run into the problem that constant ArrayRef expressions weren't
simplified). So that means it's been languishing in a git branch for
*two* years. I'm kind of inclined to push it before it gets forgotten
again ;-)

You will probably not be surprised to hear that I think this is a
feature and thus subject to the feature freeze currently in effect.

[ shrug ] Sure. I'll do what I should have done then, which is stick
it into the next CF list.

If you intend to also object to my proposed get_attstatsslot refactoring,
please do that promptly. That one I think is bug-proofing.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, May 12, 2017 at 12:55 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Fri, May 12, 2017 at 12:04 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Actually, I now remember that I wrote that while at Salesforce (because
they'd run into the problem that constant ArrayRef expressions weren't
simplified). So that means it's been languishing in a git branch for
*two* years. I'm kind of inclined to push it before it gets forgotten
again ;-)

You will probably not be surprised to hear that I think this is a
feature and thus subject to the feature freeze currently in effect.

[ shrug ] Sure. I'll do what I should have done then, which is stick
it into the next CF list.

Thanks.

If you intend to also object to my proposed get_attstatsslot refactoring,
please do that promptly. That one I think is bug-proofing.

I wasn't planning to express an opinion on that one.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

I wrote:

Looking at the patch, it still seems solid, but I remember that one
thing I was concerned about was whether the more generalized code
was any slower. Not sure about a good test case to measure that
though --- const-simplification isn't a large chunk of most workloads.

This patch no longer applies cleanly on HEAD, so here's a rebased version
(no substantive changes). As before, I think the most useful review task
would be to quantify whether it makes planning noticeably slower.

regards, tom lane

I wrote:

This patch no longer applies cleanly on HEAD, so here's a rebased version
(no substantive changes). As before, I think the most useful review task
would be to quantify whether it makes planning noticeably slower.

Rebased again (over the arrays-of-domains patch).

regards, tom lane

On Mon, Oct 2, 2017 at 7:52 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I wrote:

This patch no longer applies cleanly on HEAD, so here's a rebased version
(no substantive changes). As before, I think the most useful review task
would be to quantify whether it makes planning noticeably slower.

Rebased again (over the arrays-of-domains patch).

Nobody has showed up with the courage to review this patch. So moved
to next CF. The patch still applies cleanly.
--
Michael

On 12 September 2017 at 15:29, Tom Lane <tgl@sss.pgh.pa.us> wrote:

This patch no longer applies cleanly on HEAD, so here's a rebased version
(no substantive changes). As before, I think the most useful review task
would be to quantify whether it makes planning noticeably slower.

I tried to experiment a bit with this patch, hope it may be helpful.
Basically
I've made some number of pgbench runs over an instance with/without the
patch,
assuming, that since the only difference would be in the planner, this will
show performance impact from the patch. Also, I've manually collected some
statistics for "Planning time" from explain analyze (I suppose it's also a
valid
test). As a test target I've used queries like `select * from test where 1
in
(1, 2)` (as a side note - an involved table was empty) for different number
of
elements in an array, and for situations when this condition is true/false.

So, for:

pgbench -c 50 -s 100 -j 50 -n -t 1000 -f const_eval.sql

I've got this data (avg latency):

type w/ patch w/o patch

short array 5.203 5.564

long array 9.293 9.667

Just out of curiosity I've also made the same test for constant arrays
instead
of integers, but in this case numbers for with and without the patch were
almost identical.

For explain analyze (avg planning time):

type w/ patch w/o patch

short array 0.214 0.226

long array 0.374 0.320

I'm not sure why for the long array case with the patch I have smaller
latency
(I thought, that more complexity should have negative impact on the
performance), but a bit longer planning time. But at the end it looks that
the difference is not that big.

Speaking about the code, everything looks fine. As for me, some variables
could
be named in a more self-explanatory way (e.g `ece_evaluate_expr`, where
`ece`
is `eval_const_expresisions`, or `saop`, which is `ScalarArrayOp`), but it's
minor.

I wonder if it makes sense in this patch also deal with the following
situation?

explain analyze select * from test where 1 in (select 1);
QUERY PLAN
-----------------------------------------------------------------------------------------------
Result (cost=0.02..35.52 rows=2550 width=4) (actual time=0.036..0.036
rows=0 loops=1)
One-Time Filter: (hashed SubPlan 1)
-> Seq Scan on test (cost=0.02..35.52 rows=2550 width=4) (actual
time=0.009..0.009 rows=0 loops=1)
SubPlan 1
-> Result (cost=0.00..0.01 rows=1 width=4) (actual time=0.002..0.002
rows=1 loops=1)
Planning time: 0.381 ms
Execution time: 0.360 ms

It looks quite similar from a user perspective (although since a subplan is
involved, I assume it may be quite different in terms of implementation).

Dmitry Dolgov <9erthalion6@gmail.com> writes:

I tried to experiment a bit with this patch, hope it may be helpful.

Thanks for reviewing!

I took your idea of just running pgbench results, and adapted it to these
test cases:

select * from test where 1 = 1 or 1 = 2;

select * from test where 1 = 1 and 1 = 2;

select * from test where 1 in (1, 2);

select * from test where id in (1,2);

where the test table is just created with
create table test (id int);
and contains no data.

For me, the first, second, and fourth cases are within 1% of the
same speed with or without the patch; some a bit slower, some a
bit faster, but really all of those results are barely above the
noise floor. The third case is distinctly faster (~8%) with patch;
but since we're emitting a better plan, with no runtime WHERE test,
that's probably more about reduced executor overhead than anything else.

I did find one case where the patch makes things significantly slower:

select * from test where true is true
and true is true
and true is true
and true is true
... (100 times altogether)

That's a full 25% slower with the patch. Investigation confirms
that the extra cost can be blamed on using evaluate_expr() to
simplify BooleanTest nodes, in place of the specialized logic
that's there now. I don't feel bad about the other places where
evaluate_expr() is used: either they were like that before, or
the case wasn't const-simplified at all, so that even with some
overhead we can be pretty sure life is better with the patch.
But there's room to argue that BooleanTest occurs often enough that
it's worth having bespoke logic to simplify it, if that logic isn't too
complicated which it really isn't. So I'm inclined to undo the part
of the patch that turns BooleanTest processing into a generic case,
as per attached updated patch. Otherwise I think we can figure that
performance isn't a problem.

Speaking about the code, everything looks fine. As for me, some
variables could be named in a more self-explanatory way (e.g
`ece_evaluate_expr`, where `ece` is `eval_const_expresisions`, or
`saop`, which is `ScalarArrayOp`), but it's minor.

I'm disinclined to change the usage of "saop" here --- that's already
in use in lots of code touching ScalarArrayOp, so even if we had a better
idea, this patch isn't the place to change it. I'd be fine with changing
the "ece_" prefix, but I don't want to spell out eval_const_expressions
altogether; do you have a different abbreviation suggestion?

I wonder if it makes sense in this patch also deal with the following
situation?

explain analyze select * from test where 1 in (select 1);

No; lots of people depend on the fact that a sub-select is an optimization
fence in that way. If we were going to change that it'd need to be
carefully thought through, and I don't think it should happen as a side
effect of what's mostly a refactoring patch. The code changes wouldn't
be anywhere near this patch, either ...

regards, tom lane

On 2 January 2018 at 20:52, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I did find one case where the patch makes things significantly slower:

select * from test where true is true
and true is true
and true is true
and true is true
... (100 times altogether)

Yes, indeed, I never tested this kind of conditions. I can confirm, this
query
is significantly slower with the previous version of the patch, but the
current
version solves this.

I'm disinclined to change the usage of "saop" here --- that's already
in use in lots of code touching ScalarArrayOp

Oh, ok - I see now, thank you. So, I think no one would object if I'll mark
this patch as ready for committer.

Dmitry Dolgov <9erthalion6@gmail.com> writes:

Oh, ok - I see now, thank you. So, I think no one would object if I'll mark
this patch as ready for committer.

Pushed, thanks for reviewing!

regards, tom lane