Re: Optimizer Doesn't Push Down Where Expressions on Rollups
In your case, the WHERE clauses would get pushed down into the subquery
for both queries, with/without the ROLLUP. But since the subquery uses
grouping/grouping sets, the WHERE clauses would be put in HAVING of the
subquery.
Then when we plan for the subquery, we will decide whether a HAVING
clause can be transfered into WHERE. Usually we do not do that if there
are any nonempty grouping sets. Because if any referenced column isn't
present in all the grouping sets, moving such a clause into WHERE would
potentially change the results.
Thanks
Richard
Richard Guo <guofenglinux@gmail.com> writes:
In your case, the WHERE clauses would get pushed down into the subquery
for both queries, with/without the ROLLUP. But since the subquery uses
grouping/grouping sets, the WHERE clauses would be put in HAVING of the
subquery.
Right, we do successfully push the clauses into HAVING of the subquery.
Then when we plan for the subquery, we will decide whether a HAVING
clause can be transfered into WHERE. Usually we do not do that if there
are any nonempty grouping sets. Because if any referenced column isn't
present in all the grouping sets, moving such a clause into WHERE would
potentially change the results.
Yeah. I think that it might be safe if the proposed clause can
be proven strict for (some subset of?) the grouping columns, because
that would eliminate the rollup grouping sets where those columns
come out NULL because they aren't being grouped on. (This could then
also factor into throwing away those grouping sets, perhaps.)
Anyway, this is not a bug; it's a proposed planner improvement, and
not a trivial one.
regards, tom lane
Just in case it wasn’t obvious from the example, I’m talking about only cases where the all groups in the grouping set share a subset of columns in common and those are the columns being conditioned in the WHERE clause.
I get it if it’s not something actionable, but it is a bummer to see query time explode when going from one grouping set to two grouping sets. :/
Cheers, and I’ve been a big fan of y’alls work for going on two decades now. Thank you!
Show quoted text
On Mar 11, 2020, at 7:06 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Richard Guo <guofenglinux@gmail.com> writes:
In your case, the WHERE clauses would get pushed down into the subquery
for both queries, with/without the ROLLUP. But since the subquery uses
grouping/grouping sets, the WHERE clauses would be put in HAVING of the
subquery.Right, we do successfully push the clauses into HAVING of the subquery.
Then when we plan for the subquery, we will decide whether a HAVING
clause can be transfered into WHERE. Usually we do not do that if there
are any nonempty grouping sets. Because if any referenced column isn't
present in all the grouping sets, moving such a clause into WHERE would
potentially change the results.Yeah. I think that it might be safe if the proposed clause can
be proven strict for (some subset of?) the grouping columns, because
that would eliminate the rollup grouping sets where those columns
come out NULL because they aren't being grouped on. (This could then
also factor into throwing away those grouping sets, perhaps.)Anyway, this is not a bug; it's a proposed planner improvement, and
not a trivial one.regards, tom lane
On Wed, Mar 11, 2020 at 10:06 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Richard Guo <guofenglinux@gmail.com> writes:
In your case, the WHERE clauses would get pushed down into the subquery
for both queries, with/without the ROLLUP. But since the subquery uses
grouping/grouping sets, the WHERE clauses would be put in HAVING of the
subquery.Right, we do successfully push the clauses into HAVING of the subquery.
Then when we plan for the subquery, we will decide whether a HAVING
clause can be transfered into WHERE. Usually we do not do that if there
are any nonempty grouping sets. Because if any referenced column isn't
present in all the grouping sets, moving such a clause into WHERE would
potentially change the results.Yeah. I think that it might be safe if the proposed clause can
be proven strict for (some subset of?) the grouping columns, because
that would eliminate the rollup grouping sets where those columns
come out NULL because they aren't being grouped on. (This could then
also factor into throwing away those grouping sets, perhaps.)
This seems correct to me. If we can prove the HAVING clause is strict
for some grouping columns, then we can throw away the grouping sets that
do not contain these grouping columns, since their results would be
eliminated by this HAVING clause. After that we can move this HAVING
clause to WHERE. I'm thinking about this example:
select c1, c2, sum(c4) from t group by
grouping sets ((c1, c2), (c2, c3), (c1, c4)) having c2 = 2;
select c1, c2, sum(c4) from t group by
grouping sets ((c1, c2), (c2, c3)) having c2 = 2;
select c1, c2, sum(c4) from t where c2 = 2 group by
grouping sets ((c1, c2), (c2, c3));
For non-strict HAVING clause, if its referenced columns are present in
all the grouping sets, I think we should also be able to move it to
WHERE.
Thanks
Richard
Hi,
(cc'ing -hackers)
We used to push down clauses from HAVING to WHERE when grouping sets are
used in 61444bfb and then reverted it in a6897efa because of wrong
results issue. As now there are people suffering from performance issue
as described in [1]/messages/by-id/17F738BE-8D45-422C-BAD0-ACA3090BF46D@gmail.com, I'm wondering if we should give it another try.
[1]: /messages/by-id/17F738BE-8D45-422C-BAD0-ACA3090BF46D@gmail.com
/messages/by-id/17F738BE-8D45-422C-BAD0-ACA3090BF46D@gmail.com
Thanks
Richard
On Thu, Mar 12, 2020 at 6:22 PM Richard Guo <guofenglinux@gmail.com> wrote:
Show quoted text
On Wed, Mar 11, 2020 at 10:06 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Richard Guo <guofenglinux@gmail.com> writes:
In your case, the WHERE clauses would get pushed down into the subquery
for both queries, with/without the ROLLUP. But since the subquery uses
grouping/grouping sets, the WHERE clauses would be put in HAVING of the
subquery.Right, we do successfully push the clauses into HAVING of the subquery.
Then when we plan for the subquery, we will decide whether a HAVING
clause can be transfered into WHERE. Usually we do not do that if there
are any nonempty grouping sets. Because if any referenced column isn't
present in all the grouping sets, moving such a clause into WHERE would
potentially change the results.Yeah. I think that it might be safe if the proposed clause can
be proven strict for (some subset of?) the grouping columns, because
that would eliminate the rollup grouping sets where those columns
come out NULL because they aren't being grouped on. (This could then
also factor into throwing away those grouping sets, perhaps.)This seems correct to me. If we can prove the HAVING clause is strict
for some grouping columns, then we can throw away the grouping sets that
do not contain these grouping columns, since their results would be
eliminated by this HAVING clause. After that we can move this HAVING
clause to WHERE. I'm thinking about this example:select c1, c2, sum(c4) from t group by
grouping sets ((c1, c2), (c2, c3), (c1, c4)) having c2 = 2;select c1, c2, sum(c4) from t group by
grouping sets ((c1, c2), (c2, c3)) having c2 = 2;select c1, c2, sum(c4) from t where c2 = 2 group by
grouping sets ((c1, c2), (c2, c3));For non-strict HAVING clause, if its referenced columns are present in
all the grouping sets, I think we should also be able to move it to
WHERE.Thanks
Richard