Parallel Index-only scan
Hello all,
This is to propose a patch for enabling parallel index-only scans. With the
patch of parallel index scan around [1]/messages/by-id/CAOGQiiOneen9WEppO6V_myKpQ97CrjBQJ0Pv7ri0rxmMYvLcTg@mail.gmail.com -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/, adding the mechanism for
parallelising index-only scans makes sense. Without this mechanism for the
queries preferring index-only scans, it is likely that at higher scale
parallel index or parallel seq scan serve as cheaper alternative to
index-only scans and then query performance might suffer because of the
added processing of index or seq scans.
Performance
-----------------
Consider the performance of a simple query on TPC-H schema,
explain analyse select count(*) from lineitem where l_shipdate < date
'1995-01-03';
Without parallel index-only scan, parallel seq scan got picked and it took
around 23 seconds for the query to execute,
Finalize Aggregate (cost=651586.63..651586.64 rows=1 width=8) (actual
time=22853.872..22853.873 rows=1 loops=1)
-> Gather (cost=651586.21..651586.62 rows=4 width=8) (actual
time=22853.684..22853.864 rows=5 loops=1)
Workers Planned: 4
Workers Launched: 4
-> Partial Aggregate (cost=650586.21..650586.22 rows=1 width=8)
(actual time=22850.489..22850.489 rows=1 loops=5)
-> Parallel Seq Scan on lineitem (cost=0.00..618021.73
rows=13025795 width=0) (actual time=0.035..20553.495 rows=10342437 loops=5)
Filter: (l_shipdate < '1995-01-03'::date)
Rows Removed by Filter: 13656485
Planning time: 0.225 ms
Execution time: 22855.196 ms
However, with parallel index-only scan, it took only 8.5 seconds,
Finalize Aggregate (cost=568883.69..568883.70 rows=1 width=8) (actual
time=8548.993..8548.993 rows=1 loops=1)
-> Gather (cost=568883.27..568883.68 rows=4 width=8) (actual
time=8548.789..8548.976 rows=5 loops=1)
Workers Planned: 4
Workers Launched: 4
-> Partial Aggregate (cost=567883.27..567883.28 rows=1 width=8)
(actual time=8541.929..8541.929 rows=1 loops=5)
-> Parallel Index Only Scan using idx_l_shipdate on
lineitem (cost=0.57..535318.78 rows=13025795 width=0) (actual
time=0.113..5866.729 rows=10342437 loops=5)
Index Cond: (l_shipdate < '1995-01-03'::date)
Heap Fetches: 0
Planning time: 0.266 ms
Execution time: 8569.735 ms
The effect of parallel index-only scan can be seen more in some more
complex queries where parallelism is enabled till top of the tree, e.g,
following query takes 118 ms on head,
explain analyse select sum(l_extendedprice * l_discount) as revenue,
avg(o_orderkey) from orders, lineitem where o_orderkey < 10000 and
o_orderkey = l_orderkey group by l_shipmode order by revenue
Sort (cost=24396.44..24396.45 rows=7 width=75) (actual
time=118.823..118.825 rows=7 loops=1)
Sort Key: (sum((lineitem.l_extendedprice * lineitem.l_discount)))
Sort Method: quicksort Memory: 25kB
-> HashAggregate (cost=24396.23..24396.34 rows=7 width=75) (actual
time=118.749..118.786 rows=7 loops=1)
Group Key: lineitem.l_shipmode
-> Nested Loop (cost=1.13..24293.11 rows=10312 width=27)
(actual time=0.096..73.198 rows=9965 loops=1)
-> Index Only Scan using orders_pkey on orders
(cost=0.56..46.48 rows=2578 width=4) (actual time=0.072..1.663 rows=2503
loops=1)
Index Cond: (o_orderkey < 10000)
Heap Fetches: 0
-> Index Scan using idx_lineitem_orderkey on lineitem
(cost=0.57..6.45 rows=296 width=31) (actual time=0.018..0.023 rows=4
loops=2503)
Index Cond: (l_orderkey = orders.o_orderkey)
Planning time: 1.062 ms
Execution time: 118.977 ms
With parallel index-only scan, the performance improves to 40 ms,
Sort (cost=7191.33..7191.35 rows=7 width=75) (actual time=40.475..40.476
rows=7 loops=1)
Sort Key: (sum((lineitem.l_extendedprice * lineitem.l_discount)))
Sort Method: quicksort Memory: 25kB
-> Finalize GroupAggregate (cost=7190.78..7191.23 rows=7 width=75)
(actual time=40.168..40.451 rows=7 loops=1)
Group Key: lineitem.l_shipmode
-> Sort (cost=7190.78..7190.85 rows=28 width=75) (actual
time=40.105..40.127 rows=35 loops=1)
Sort Key: lineitem.l_shipmode
Sort Method: quicksort Memory: 29kB
-> Gather (cost=7187.22..7190.11 rows=28 width=75)
(actual time=39.344..39.983 rows=35 loops=1)
Workers Planned: 4
Workers Launched: 4
-> Partial HashAggregate (cost=6187.22..6187.31
rows=7 width=75) (actual time=25.981..26.011 rows=7 loops=5)
Group Key: lineitem.l_shipmode
-> Nested Loop (cost=1.13..6084.10 rows=10312
width=27) (actual time=0.139..16.352 rows=1993 loops=5)
-> Parallel Index Only Scan using
orders_pkey on orders (cost=0.56..27.14 rows=644 width=4) (actual
time=0.082..0.366 rows=501 loops=5)
Index Cond: (o_orderkey < 10000)
Heap Fetches: 0
-> Index Scan using
idx_lineitem_orderkey on lineitem (cost=0.57..6.45 rows=296 width=31)
(actual time=0.020..0.025 rows=4 loops=2503)
Index Cond: (l_orderkey =
orders.o_orderkey)
Planning time: 1.170 ms
Execution time: 40.898 ms
Test configuration and machine detail:
--------------------------------------------------
TPC-H: scale facto : 20
work_mem : 64MB
shared buffer : 1GB
max_parallel_workers_per_gather : 4
Machine : IBM POWER, 4 socket
machine with 512 GB RAM.
random_page_cost = seq_page_cost = 0.1
We kept random and seq page cost same since warm cache environment was
ensured, thus, keeping a value of random_page_cost that is higher than
seq_page_cost doesn't makes much sense in this setup.
Also, some additional indexes were build, for lineitem table l_shipdate,
l_shipmode, and l_returnflag, on orders table index is added for o_comment
and o_orderdate individually.
The point to note here is that with parallel index-only scan, not only the
heap fetches are saved but also the aggregates/sort can be pushed down to
workers and thus the total time of query can be improved. Clearly, the
performance of queries improved significantly with this new operator and
considering the changes required after parallel index scan patches is less
if evaluated against the improvement in performance it offers.
Attached file:
------------------
1. parallel_index_only_v1.patch
This patch is to be applied over parallel index scan [1]/messages/by-id/CAOGQiiOneen9WEppO6V_myKpQ97CrjBQJ0Pv7ri0rxmMYvLcTg@mail.gmail.com -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/.
* Thanks to my colleagues Dilip Kumar and Amit Kapila for their support and
feedback.
[1]: /messages/by-id/CAOGQiiOneen9WEppO6V_myKpQ97CrjBQJ0Pv7ri0rxmMYvLcTg@mail.gmail.com -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/
/messages/by-id/CAOGQiiOneen9WEppO6V_myKpQ97CrjBQJ0Pv7ri0rxmMYvLcTg@mail.gmail.com
--
Regards,
Rafia Sabih
EnterpriseDB: http://www.enterprisedb.com/
Attachments:
parallel_index_only_v1.patchapplication/octet-stream; name=parallel_index_only_v1.patchDownload
diff --git a/src/backend/access/transam/parallel.c b/src/backend/access/transam/parallel.c
index 59dc394..34c569e 100644
--- a/src/backend/access/transam/parallel.c
+++ b/src/backend/access/transam/parallel.c
@@ -36,6 +36,8 @@
#include "utils/resowner.h"
#include "utils/snapmgr.h"
+#include "optimizer/cost.h"
+
/*
* We don't want to waste a lot of memory on an error queue which, most of
@@ -946,7 +948,11 @@ ParallelWorkerMain(Datum main_arg)
char *asnapspace;
char *tstatespace;
StringInfoData msgbuf;
-
+ int i =0;
+ while (i)
+ {
+ sleep(10);
+ }
/* Set flag to indicate that we're initializing a parallel worker. */
InitializingParallelWorker = true;
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index c55b7c4..08329b0 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -198,6 +198,11 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
ExecIndexScanEstimate((IndexScanState *) planstate,
e->pcxt);
break;
+ case T_IndexOnlyScanState:
+ ExecIndexOnlyScanEstimate((IndexOnlyScanState *) planstate,
+ e->pcxt);
+ break;
+
case T_ForeignScanState:
ExecForeignScanEstimate((ForeignScanState *) planstate,
e->pcxt);
@@ -254,6 +259,11 @@ ExecParallelInitializeDSM(PlanState *planstate,
ExecIndexScanInitializeDSM((IndexScanState *) planstate,
d->pcxt);
break;
+ case T_IndexOnlyScanState:
+ ExecIndexOnlyScanInitializeDSM((IndexOnlyScanState *) planstate,
+ d->pcxt);
+ break;
+
case T_ForeignScanState:
ExecForeignScanInitializeDSM((ForeignScanState *) planstate,
d->pcxt);
@@ -701,6 +711,10 @@ ExecParallelInitializeWorker(PlanState *planstate, shm_toc *toc)
case T_IndexScanState:
ExecIndexScanInitializeWorker((IndexScanState *) planstate, toc);
break;
+ case T_IndexOnlyScanState:
+ ExecIndexOnlyScanInitializeWorker((IndexOnlyScanState *) planstate, toc);
+ break;
+
case T_ForeignScanState:
ExecForeignScanInitializeWorker((ForeignScanState *) planstate,
toc);
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 45566bd..8257f43 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -276,7 +276,17 @@ ExecIndexOnlyScan(IndexOnlyScanState *node)
*/
void
ExecReScanIndexOnlyScan(IndexOnlyScanState *node)
-{
+{
+ bool reset_parallel_scan = true;
+
+ /*
+ * if we are here to just update the scan keys, then don't reset parallel
+ * scan
+ */
+ if (node->ioss_NumRuntimeKeys != 0 && !node->ioss_RuntimeKeysReady)
+ reset_parallel_scan = false;
+
+
/*
* If we are doing runtime key calculations (ie, any of the index key
* values weren't simple Consts), compute the new key values. But first,
@@ -296,10 +306,16 @@ ExecReScanIndexOnlyScan(IndexOnlyScanState *node)
node->ioss_RuntimeKeysReady = true;
/* reset index scan */
- index_rescan(node->ioss_ScanDesc,
- node->ioss_ScanKeys, node->ioss_NumScanKeys,
- node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+ if (node->ioss_ScanDesc)
+ {
+
+ index_rescan(node->ioss_ScanDesc,
+ node->ioss_ScanKeys, node->ioss_NumScanKeys,
+ node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+ if (reset_parallel_scan)
+ index_parallelrescan(node->ioss_ScanDesc);
+ }
ExecScanReScan(&node->ss);
}
@@ -534,33 +550,99 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
{
indexstate->ioss_RuntimeContext = NULL;
}
-
- /*
- * Initialize scan descriptor.
- */
- indexstate->ioss_ScanDesc = index_beginscan(currentRelation,
+ if (!node->scan.plan.parallel_aware)
+ {
+ /*
+ * Initialize scan descriptor.
+ */
+ indexstate->ioss_ScanDesc = index_beginscan(currentRelation,
indexstate->ioss_RelationDesc,
estate->es_snapshot, NULL,
indexstate->ioss_NumScanKeys,
indexstate->ioss_NumOrderByKeys, false);
- /* Set it up for index-only scan */
- indexstate->ioss_ScanDesc->xs_want_itup = true;
- indexstate->ioss_VMBuffer = InvalidBuffer;
+ /* Set it up for index-only scan */
+ indexstate->ioss_ScanDesc->xs_want_itup = true;
+ indexstate->ioss_VMBuffer = InvalidBuffer;
- /*
- * If no run-time keys to calculate, go ahead and pass the scankeys to the
- * index AM.
- */
- if (indexstate->ioss_NumRuntimeKeys == 0)
- index_rescan(indexstate->ioss_ScanDesc,
+ /*
+ * If no run-time keys to calculate, go ahead and pass the scankeys to the
+ * index AM.
+ */
+ if (indexstate->ioss_NumRuntimeKeys == 0)
+ index_rescan(indexstate->ioss_ScanDesc,
indexstate->ioss_ScanKeys,
indexstate->ioss_NumScanKeys,
indexstate->ioss_OrderByKeys,
indexstate->ioss_NumOrderByKeys);
-
+ }
/*
* all done.
*/
return indexstate;
}
+void
+ExecIndexOnlyScanEstimate(IndexOnlyScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+
+ node->ioss_PscanLen = index_parallelscan_estimate(node->ioss_RelationDesc,
+ estate->es_snapshot);
+ shm_toc_estimate_chunk(&pcxt->estimator, node->ioss_PscanLen);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+void
+ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+ ParallelIndexScanDesc piscan;
+
+ piscan = shm_toc_allocate(pcxt->toc, node->ioss_PscanLen);
+ index_parallelscan_initialize(node->ss.ss_currentRelation,
+ node->ioss_RelationDesc,
+ estate->es_snapshot,
+ piscan);
+ shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, piscan);
+ node->ioss_ScanDesc =
+ index_beginscan_parallel(node->ss.ss_currentRelation,
+ node->ioss_RelationDesc,
+ node->ioss_NumScanKeys,
+ node->ioss_NumOrderByKeys,
+ piscan);
+ node->ioss_ScanDesc->xs_want_itup = true;
+
+ /*
+ * If no run-time keys to calculate, go ahead and pass the scankeys to the
+ * index AM.
+ */
+ if (node->ioss_NumRuntimeKeys == 0)
+ index_rescan(node->ioss_ScanDesc,
+ node->ioss_ScanKeys, node->ioss_NumScanKeys,
+ node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+}
+void
+ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node, shm_toc *toc)
+{
+ ParallelIndexScanDesc piscan;
+
+ piscan = shm_toc_lookup(toc, node->ss.ps.plan->plan_node_id);
+ node->ioss_ScanDesc =
+ index_beginscan_parallel(node->ss.ss_currentRelation,
+ node->ioss_RelationDesc,
+ node->ioss_NumScanKeys,
+ node->ioss_NumOrderByKeys,
+ piscan);
+ node->ioss_ScanDesc->xs_want_itup = true;
+
+ /*
+ * If no run-time keys to calculate, go ahead and pass the scankeys to the
+ * index AM.
+ */
+ if (node->ioss_NumRuntimeKeys == 0)
+ index_rescan(node->ioss_ScanDesc,
+ node->ioss_ScanKeys, node->ioss_NumScanKeys,
+ node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+}
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 54a8a2f..5d963f5 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -1098,8 +1098,8 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
* If appropriate, consider parallel index scan. We don't allow
* parallel index scan for bitmap scans.
*/
- if (index->amcanparallel &&
- !index_only_scan &&
+ if ((index->amcanparallel ||
+ index_only_scan) &&
rel->consider_parallel &&
outer_relids == NULL &&
scantype != ST_BITMAPSCAN)
diff --git a/src/include/executor/nodeIndexonlyscan.h b/src/include/executor/nodeIndexonlyscan.h
index d63d194..143d8cd 100644
--- a/src/include/executor/nodeIndexonlyscan.h
+++ b/src/include/executor/nodeIndexonlyscan.h
@@ -14,6 +14,7 @@
#ifndef NODEINDEXONLYSCAN_H
#define NODEINDEXONLYSCAN_H
+#include "access/parallel.h"
#include "nodes/execnodes.h"
extern IndexOnlyScanState *ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags);
@@ -23,4 +24,8 @@ extern void ExecIndexOnlyMarkPos(IndexOnlyScanState *node);
extern void ExecIndexOnlyRestrPos(IndexOnlyScanState *node);
extern void ExecReScanIndexOnlyScan(IndexOnlyScanState *node);
+extern void ExecIndexOnlyScanEstimate(IndexOnlyScanState *node, ParallelContext *pcxt);
+extern void ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node, ParallelContext *pcxt);
+extern void ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node, shm_toc *toc);
+
#endif /* NODEINDEXONLYSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 17d712a..b8f22da 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1397,6 +1397,8 @@ typedef struct IndexOnlyScanState
IndexScanDesc ioss_ScanDesc;
Buffer ioss_VMBuffer;
long ioss_HeapFetches;
+ /* This is needed for parallel index scan */
+ Size ioss_PscanLen;
} IndexOnlyScanState;
/* ----------------
Please find the attached file for the latest version of patch.
parallel_index_only_v2.patch
<http://postgresql.nabble.com/file/n5936010/parallel_index_only_v2.patch>
--
View this message in context: http://postgresql.nabble.com/Parallel-Index-only-scan-tp5934352p5936010.html
Sent from the PostgreSQL - hackers mailing list archive at Nabble.com.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Dec 23, 2016 at 12:04 AM, rafia.sabih
<rafia.sabih@enterprisedb.com> wrote:
Please find the attached file for the latest version of patch.
parallel_index_only_v2.patch
<http://postgresql.nabble.com/file/n5936010/parallel_index_only_v2.patch>
We want to have the patch actually attached, not just stored on nabble.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Robert Haas wrote:
On Fri, Dec 23, 2016 at 12:04 AM, rafia.sabih
<rafia.sabih@enterprisedb.com> wrote:Please find the attached file for the latest version of patch.
parallel_index_only_v2.patch
<http://postgresql.nabble.com/file/n5936010/parallel_index_only_v2.patch>We want to have the patch actually attached, not just stored on nabble.
I think the problem is that Nabble removes the attachment before
resending the email to the list. The OP should be posting directly
instead of via Nabble.
--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes:
On Fri, Dec 23, 2016 at 12:04 AM, rafia.sabih
<rafia.sabih@enterprisedb.com> wrote:Please find the attached file for the latest version of patch.
parallel_index_only_v2.patch
<http://postgresql.nabble.com/file/n5936010/parallel_index_only_v2.patch>
We want to have the patch actually attached, not just stored on nabble.
Or in words of one syllable: please do not use nabble to post to the
community mailing lists. Their habit of stripping attachments is not
the only evil thing about them (although it's the only one I remember
details of ATM).
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Dec 23, 2016 at 3:03 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Or in words of one syllable: please do not use nabble to post to the
community mailing lists.
Many of those words have two syllables, and one has four.
Anyhow, I think three castigating emails from committers in a span of
62 minutes is probably enough for the OP to get the point, unless
someone else REALLY feels the need to pile on.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Extremely sorry for the inconvenience caused, please find the attached file
for the latest version of the patch.
On Sat, Dec 24, 2016 at 1:41 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Dec 23, 2016 at 3:03 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Or in words of one syllable: please do not use nabble to post to the
community mailing lists.Many of those words have two syllables, and one has four.
Anyhow, I think three castigating emails from committers in a span of
62 minutes is probably enough for the OP to get the point, unless
someone else REALLY feels the need to pile on.--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Regards,
Rafia Sabih
EnterpriseDB: http://www.enterprisedb.com/
Attachments:
parallel_index_only_v2.patchapplication/octet-stream; name=parallel_index_only_v2.patchDownload
diff --git a/src/backend/access/nbtree/nbtsearch.c b/src/backend/access/nbtree/nbtsearch.c
index baa69fc..79eaf79 100644
--- a/src/backend/access/nbtree/nbtsearch.c
+++ b/src/backend/access/nbtree/nbtsearch.c
@@ -1394,6 +1394,9 @@ _bt_steppage(IndexScanDesc scan, ScanDirection dir)
blkno = _bt_parallel_seize(scan, &status);
if (status == false)
{
+ /* release the previous buffer, if pinned */
+ BTScanPosUnpinIfPinned(so->currPos);
+
BTScanPosInvalidate(so->currPos);
return false;
}
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index c55b7c4..08329b0 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -198,6 +198,11 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
ExecIndexScanEstimate((IndexScanState *) planstate,
e->pcxt);
break;
+ case T_IndexOnlyScanState:
+ ExecIndexOnlyScanEstimate((IndexOnlyScanState *) planstate,
+ e->pcxt);
+ break;
+
case T_ForeignScanState:
ExecForeignScanEstimate((ForeignScanState *) planstate,
e->pcxt);
@@ -254,6 +259,11 @@ ExecParallelInitializeDSM(PlanState *planstate,
ExecIndexScanInitializeDSM((IndexScanState *) planstate,
d->pcxt);
break;
+ case T_IndexOnlyScanState:
+ ExecIndexOnlyScanInitializeDSM((IndexOnlyScanState *) planstate,
+ d->pcxt);
+ break;
+
case T_ForeignScanState:
ExecForeignScanInitializeDSM((ForeignScanState *) planstate,
d->pcxt);
@@ -701,6 +711,10 @@ ExecParallelInitializeWorker(PlanState *planstate, shm_toc *toc)
case T_IndexScanState:
ExecIndexScanInitializeWorker((IndexScanState *) planstate, toc);
break;
+ case T_IndexOnlyScanState:
+ ExecIndexOnlyScanInitializeWorker((IndexOnlyScanState *) planstate, toc);
+ break;
+
case T_ForeignScanState:
ExecForeignScanInitializeWorker((ForeignScanState *) planstate,
toc);
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 45566bd..78304d1 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -21,6 +21,9 @@
* ExecEndIndexOnlyScan releases all storage.
* ExecIndexOnlyMarkPos marks scan position.
* ExecIndexOnlyRestrPos restores scan position.
+ * ExecIndexOnlyScanEstimate estimates DSM space needed for parallel index-only scan
+ * ExecIndexOnlyScanInitializeDSM initialize DSM for parallel index-only scan
+ * ExecIndexOnlyScanInitializeWorker attach to DSM info in parallel worker
*/
#include "postgres.h"
@@ -277,6 +280,16 @@ ExecIndexOnlyScan(IndexOnlyScanState *node)
void
ExecReScanIndexOnlyScan(IndexOnlyScanState *node)
{
+ bool reset_parallel_scan = true;
+
+ /*
+ * if we are here to just update the scan keys, then don't reset parallel
+ * scan
+ */
+ if (node->ioss_NumRuntimeKeys != 0 && !node->ioss_RuntimeKeysReady)
+ reset_parallel_scan = false;
+
+
/*
* If we are doing runtime key calculations (ie, any of the index key
* values weren't simple Consts), compute the new key values. But first,
@@ -296,10 +309,16 @@ ExecReScanIndexOnlyScan(IndexOnlyScanState *node)
node->ioss_RuntimeKeysReady = true;
/* reset index scan */
- index_rescan(node->ioss_ScanDesc,
- node->ioss_ScanKeys, node->ioss_NumScanKeys,
- node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+ if (node->ioss_ScanDesc)
+ {
+
+ index_rescan(node->ioss_ScanDesc,
+ node->ioss_ScanKeys, node->ioss_NumScanKeys,
+ node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+ if (reset_parallel_scan)
+ index_parallelrescan(node->ioss_ScanDesc);
+ }
ExecScanReScan(&node->ss);
}
@@ -534,33 +553,99 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
{
indexstate->ioss_RuntimeContext = NULL;
}
-
- /*
- * Initialize scan descriptor.
- */
- indexstate->ioss_ScanDesc = index_beginscan(currentRelation,
+ if (!node->scan.plan.parallel_aware)
+ {
+ /*
+ * Initialize scan descriptor.
+ */
+ indexstate->ioss_ScanDesc = index_beginscan(currentRelation,
indexstate->ioss_RelationDesc,
estate->es_snapshot, NULL,
indexstate->ioss_NumScanKeys,
indexstate->ioss_NumOrderByKeys, false);
- /* Set it up for index-only scan */
- indexstate->ioss_ScanDesc->xs_want_itup = true;
- indexstate->ioss_VMBuffer = InvalidBuffer;
+ /* Set it up for index-only scan */
+ indexstate->ioss_ScanDesc->xs_want_itup = true;
+ indexstate->ioss_VMBuffer = InvalidBuffer;
- /*
- * If no run-time keys to calculate, go ahead and pass the scankeys to the
- * index AM.
- */
- if (indexstate->ioss_NumRuntimeKeys == 0)
- index_rescan(indexstate->ioss_ScanDesc,
+ /*
+ * If no run-time keys to calculate, go ahead and pass the scankeys to the
+ * index AM.
+ */
+ if (indexstate->ioss_NumRuntimeKeys == 0)
+ index_rescan(indexstate->ioss_ScanDesc,
indexstate->ioss_ScanKeys,
indexstate->ioss_NumScanKeys,
indexstate->ioss_OrderByKeys,
indexstate->ioss_NumOrderByKeys);
-
+ }
/*
* all done.
*/
return indexstate;
}
+void
+ExecIndexOnlyScanEstimate(IndexOnlyScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+
+ node->ioss_PscanLen = index_parallelscan_estimate(node->ioss_RelationDesc,
+ estate->es_snapshot);
+ shm_toc_estimate_chunk(&pcxt->estimator, node->ioss_PscanLen);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+void
+ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+ ParallelIndexScanDesc piscan;
+
+ piscan = shm_toc_allocate(pcxt->toc, node->ioss_PscanLen);
+ index_parallelscan_initialize(node->ss.ss_currentRelation,
+ node->ioss_RelationDesc,
+ estate->es_snapshot,
+ piscan);
+ shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, piscan);
+ node->ioss_ScanDesc =
+ index_beginscan_parallel(node->ss.ss_currentRelation,
+ node->ioss_RelationDesc,
+ node->ioss_NumScanKeys,
+ node->ioss_NumOrderByKeys,
+ piscan);
+ node->ioss_ScanDesc->xs_want_itup = true;
+
+ /*
+ * If no run-time keys to calculate, go ahead and pass the scankeys to the
+ * index AM.
+ */
+ if (node->ioss_NumRuntimeKeys == 0)
+ index_rescan(node->ioss_ScanDesc,
+ node->ioss_ScanKeys, node->ioss_NumScanKeys,
+ node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+}
+void
+ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node, shm_toc *toc)
+{
+ ParallelIndexScanDesc piscan;
+
+ piscan = shm_toc_lookup(toc, node->ss.ps.plan->plan_node_id);
+ node->ioss_ScanDesc =
+ index_beginscan_parallel(node->ss.ss_currentRelation,
+ node->ioss_RelationDesc,
+ node->ioss_NumScanKeys,
+ node->ioss_NumOrderByKeys,
+ piscan);
+ node->ioss_ScanDesc->xs_want_itup = true;
+
+ /*
+ * If no run-time keys to calculate, go ahead and pass the scankeys to the
+ * index AM.
+ */
+ if (node->ioss_NumRuntimeKeys == 0)
+ index_rescan(node->ioss_ScanDesc,
+ node->ioss_ScanKeys, node->ioss_NumScanKeys,
+ node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+}
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 54a8a2f..5d963f5 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -1098,8 +1098,8 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
* If appropriate, consider parallel index scan. We don't allow
* parallel index scan for bitmap scans.
*/
- if (index->amcanparallel &&
- !index_only_scan &&
+ if ((index->amcanparallel ||
+ index_only_scan) &&
rel->consider_parallel &&
outer_relids == NULL &&
scantype != ST_BITMAPSCAN)
diff --git a/src/include/executor/nodeIndexonlyscan.h b/src/include/executor/nodeIndexonlyscan.h
index d63d194..143d8cd 100644
--- a/src/include/executor/nodeIndexonlyscan.h
+++ b/src/include/executor/nodeIndexonlyscan.h
@@ -14,6 +14,7 @@
#ifndef NODEINDEXONLYSCAN_H
#define NODEINDEXONLYSCAN_H
+#include "access/parallel.h"
#include "nodes/execnodes.h"
extern IndexOnlyScanState *ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags);
@@ -23,4 +24,8 @@ extern void ExecIndexOnlyMarkPos(IndexOnlyScanState *node);
extern void ExecIndexOnlyRestrPos(IndexOnlyScanState *node);
extern void ExecReScanIndexOnlyScan(IndexOnlyScanState *node);
+extern void ExecIndexOnlyScanEstimate(IndexOnlyScanState *node, ParallelContext *pcxt);
+extern void ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node, ParallelContext *pcxt);
+extern void ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node, shm_toc *toc);
+
#endif /* NODEINDEXONLYSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 17d712a..b8f22da 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1397,6 +1397,8 @@ typedef struct IndexOnlyScanState
IndexScanDesc ioss_ScanDesc;
Buffer ioss_VMBuffer;
long ioss_HeapFetches;
+ /* This is needed for parallel index scan */
+ Size ioss_PscanLen;
} IndexOnlyScanState;
/* ----------------
Rebased patch of parallel-index only scan based on the latest version of
parallel index scan [1]/messages/by-id/CAA4eK1LiNi7_Z1%25 2BPCV4y06o_v%3DZdZ1UThE%2BW9JhthX4B8uifnA%40mail.gmail.com is attached.
[1]: /messages/by-id/CAA4eK1LiNi7_Z1%25 2BPCV4y06o_v%3DZdZ1UThE%2BW9JhthX4B8uifnA%40mail.gmail.com
2BPCV4y06o_v%3DZdZ1UThE%2BW9JhthX4B8uifnA%40mail.gmail.com
On Sat, Dec 24, 2016 at 7:55 PM, Rafia Sabih <rafia.sabih@enterprisedb.com>
wrote:
Extremely sorry for the inconvenience caused, please find the attached
file for the latest version of the patch.On Sat, Dec 24, 2016 at 1:41 AM, Robert Haas <robertmhaas@gmail.com>
wrote:On Fri, Dec 23, 2016 at 3:03 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Or in words of one syllable: please do not use nabble to post to the
community mailing lists.Many of those words have two syllables, and one has four.
Anyhow, I think three castigating emails from committers in a span of
62 minutes is probably enough for the OP to get the point, unless
someone else REALLY feels the need to pile on.--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company--
Regards,
Rafia Sabih
EnterpriseDB: http://www.enterprisedb.com/
--
Regards,
Rafia Sabih
EnterpriseDB: http://www.enterprisedb.com/
Attachments:
parallel_index_only_v3.patchapplication/octet-stream; name=parallel_index_only_v3.patchDownload
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 3238cc7..a640c4b 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -29,6 +29,7 @@
#include "executor/nodeForeignscan.h"
#include "executor/nodeSeqscan.h"
#include "executor/nodeIndexscan.h"
+#include "executor/nodeIndexonlyscan.h"
#include "executor/tqueue.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/planmain.h"
@@ -198,6 +199,11 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
ExecIndexScanEstimate((IndexScanState *) planstate,
e->pcxt);
break;
+ case T_IndexOnlyScanState:
+ ExecIndexOnlyScanEstimate((IndexOnlyScanState *) planstate,
+ e->pcxt);
+ break;
+
case T_ForeignScanState:
ExecForeignScanEstimate((ForeignScanState *) planstate,
e->pcxt);
@@ -254,6 +260,11 @@ ExecParallelInitializeDSM(PlanState *planstate,
ExecIndexScanInitializeDSM((IndexScanState *) planstate,
d->pcxt);
break;
+ case T_IndexOnlyScanState:
+ ExecIndexOnlyScanInitializeDSM((IndexOnlyScanState *) planstate,
+ d->pcxt);
+ break;
+
case T_ForeignScanState:
ExecForeignScanInitializeDSM((ForeignScanState *) planstate,
d->pcxt);
@@ -701,6 +712,10 @@ ExecParallelInitializeWorker(PlanState *planstate, shm_toc *toc)
case T_IndexScanState:
ExecIndexScanInitializeWorker((IndexScanState *) planstate, toc);
break;
+ case T_IndexOnlyScanState:
+ ExecIndexOnlyScanInitializeWorker((IndexOnlyScanState *) planstate, toc);
+ break;
+
case T_ForeignScanState:
ExecForeignScanInitializeWorker((ForeignScanState *) planstate,
toc);
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 45566bd..78304d1 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -21,6 +21,9 @@
* ExecEndIndexOnlyScan releases all storage.
* ExecIndexOnlyMarkPos marks scan position.
* ExecIndexOnlyRestrPos restores scan position.
+ * ExecIndexOnlyScanEstimate estimates DSM space needed for parallel index-only scan
+ * ExecIndexOnlyScanInitializeDSM initialize DSM for parallel index-only scan
+ * ExecIndexOnlyScanInitializeWorker attach to DSM info in parallel worker
*/
#include "postgres.h"
@@ -277,6 +280,16 @@ ExecIndexOnlyScan(IndexOnlyScanState *node)
void
ExecReScanIndexOnlyScan(IndexOnlyScanState *node)
{
+ bool reset_parallel_scan = true;
+
+ /*
+ * if we are here to just update the scan keys, then don't reset parallel
+ * scan
+ */
+ if (node->ioss_NumRuntimeKeys != 0 && !node->ioss_RuntimeKeysReady)
+ reset_parallel_scan = false;
+
+
/*
* If we are doing runtime key calculations (ie, any of the index key
* values weren't simple Consts), compute the new key values. But first,
@@ -296,10 +309,16 @@ ExecReScanIndexOnlyScan(IndexOnlyScanState *node)
node->ioss_RuntimeKeysReady = true;
/* reset index scan */
- index_rescan(node->ioss_ScanDesc,
- node->ioss_ScanKeys, node->ioss_NumScanKeys,
- node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+ if (node->ioss_ScanDesc)
+ {
+
+ index_rescan(node->ioss_ScanDesc,
+ node->ioss_ScanKeys, node->ioss_NumScanKeys,
+ node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+ if (reset_parallel_scan)
+ index_parallelrescan(node->ioss_ScanDesc);
+ }
ExecScanReScan(&node->ss);
}
@@ -534,33 +553,99 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
{
indexstate->ioss_RuntimeContext = NULL;
}
-
- /*
- * Initialize scan descriptor.
- */
- indexstate->ioss_ScanDesc = index_beginscan(currentRelation,
+ if (!node->scan.plan.parallel_aware)
+ {
+ /*
+ * Initialize scan descriptor.
+ */
+ indexstate->ioss_ScanDesc = index_beginscan(currentRelation,
indexstate->ioss_RelationDesc,
estate->es_snapshot, NULL,
indexstate->ioss_NumScanKeys,
indexstate->ioss_NumOrderByKeys, false);
- /* Set it up for index-only scan */
- indexstate->ioss_ScanDesc->xs_want_itup = true;
- indexstate->ioss_VMBuffer = InvalidBuffer;
+ /* Set it up for index-only scan */
+ indexstate->ioss_ScanDesc->xs_want_itup = true;
+ indexstate->ioss_VMBuffer = InvalidBuffer;
- /*
- * If no run-time keys to calculate, go ahead and pass the scankeys to the
- * index AM.
- */
- if (indexstate->ioss_NumRuntimeKeys == 0)
- index_rescan(indexstate->ioss_ScanDesc,
+ /*
+ * If no run-time keys to calculate, go ahead and pass the scankeys to the
+ * index AM.
+ */
+ if (indexstate->ioss_NumRuntimeKeys == 0)
+ index_rescan(indexstate->ioss_ScanDesc,
indexstate->ioss_ScanKeys,
indexstate->ioss_NumScanKeys,
indexstate->ioss_OrderByKeys,
indexstate->ioss_NumOrderByKeys);
-
+ }
/*
* all done.
*/
return indexstate;
}
+void
+ExecIndexOnlyScanEstimate(IndexOnlyScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+
+ node->ioss_PscanLen = index_parallelscan_estimate(node->ioss_RelationDesc,
+ estate->es_snapshot);
+ shm_toc_estimate_chunk(&pcxt->estimator, node->ioss_PscanLen);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+void
+ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+ ParallelIndexScanDesc piscan;
+
+ piscan = shm_toc_allocate(pcxt->toc, node->ioss_PscanLen);
+ index_parallelscan_initialize(node->ss.ss_currentRelation,
+ node->ioss_RelationDesc,
+ estate->es_snapshot,
+ piscan);
+ shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, piscan);
+ node->ioss_ScanDesc =
+ index_beginscan_parallel(node->ss.ss_currentRelation,
+ node->ioss_RelationDesc,
+ node->ioss_NumScanKeys,
+ node->ioss_NumOrderByKeys,
+ piscan);
+ node->ioss_ScanDesc->xs_want_itup = true;
+
+ /*
+ * If no run-time keys to calculate, go ahead and pass the scankeys to the
+ * index AM.
+ */
+ if (node->ioss_NumRuntimeKeys == 0)
+ index_rescan(node->ioss_ScanDesc,
+ node->ioss_ScanKeys, node->ioss_NumScanKeys,
+ node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+}
+void
+ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node, shm_toc *toc)
+{
+ ParallelIndexScanDesc piscan;
+
+ piscan = shm_toc_lookup(toc, node->ss.ps.plan->plan_node_id);
+ node->ioss_ScanDesc =
+ index_beginscan_parallel(node->ss.ss_currentRelation,
+ node->ioss_RelationDesc,
+ node->ioss_NumScanKeys,
+ node->ioss_NumOrderByKeys,
+ piscan);
+ node->ioss_ScanDesc->xs_want_itup = true;
+
+ /*
+ * If no run-time keys to calculate, go ahead and pass the scankeys to the
+ * index AM.
+ */
+ if (node->ioss_NumRuntimeKeys == 0)
+ index_rescan(node->ioss_ScanDesc,
+ node->ioss_ScanKeys, node->ioss_NumScanKeys,
+ node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+}
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 54a8a2f..5d963f5 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -1098,8 +1098,8 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
* If appropriate, consider parallel index scan. We don't allow
* parallel index scan for bitmap scans.
*/
- if (index->amcanparallel &&
- !index_only_scan &&
+ if ((index->amcanparallel ||
+ index_only_scan) &&
rel->consider_parallel &&
outer_relids == NULL &&
scantype != ST_BITMAPSCAN)
diff --git a/src/include/executor/nodeIndexonlyscan.h b/src/include/executor/nodeIndexonlyscan.h
index d63d194..143d8cd 100644
--- a/src/include/executor/nodeIndexonlyscan.h
+++ b/src/include/executor/nodeIndexonlyscan.h
@@ -14,6 +14,7 @@
#ifndef NODEINDEXONLYSCAN_H
#define NODEINDEXONLYSCAN_H
+#include "access/parallel.h"
#include "nodes/execnodes.h"
extern IndexOnlyScanState *ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags);
@@ -23,4 +24,8 @@ extern void ExecIndexOnlyMarkPos(IndexOnlyScanState *node);
extern void ExecIndexOnlyRestrPos(IndexOnlyScanState *node);
extern void ExecReScanIndexOnlyScan(IndexOnlyScanState *node);
+extern void ExecIndexOnlyScanEstimate(IndexOnlyScanState *node, ParallelContext *pcxt);
+extern void ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node, ParallelContext *pcxt);
+extern void ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node, shm_toc *toc);
+
#endif /* NODEINDEXONLYSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 4013986..a364cf0 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1397,6 +1397,8 @@ typedef struct IndexOnlyScanState
IndexScanDesc ioss_ScanDesc;
Buffer ioss_VMBuffer;
long ioss_HeapFetches;
+ /* This is needed for parallel index scan */
+ Size ioss_PscanLen;
} IndexOnlyScanState;
/* ----------------
On Wed, Dec 28, 2016 at 12:18 PM, Rafia Sabih
<rafia.sabih@enterprisedb.com> wrote:
Rebased patch of parallel-index only scan based on the latest version of
parallel index scan [1] is attached.[1]
/messages/by-id/CAA4eK1LiNi7_Z1+PCV4y06o_v=ZdZ1UThE+W9JhthX4B8uifnA@mail.gmail.com
The link for the latest version is wrong, the correct link is:
/messages/by-id/CAA4eK1KthrAvNjmB2cWuUHz+p3ZTTtbD7o2KUw49PC8HK4r1Wg@mail.gmail.com
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hello,
On applying the patch on latest master branch and running regression tests
following failure occurs.
I applied it on latest parallel index scan patches as given in the link
above.
***
/home/rahila/postgres/postgres/src/test/regress/expected/select_parallel.out
2017-01-03 14:06:29.122022780 +0530
---
/home/rahila/postgres/postgres/src/test/regress/results/select_parallel.out
2017-01-12 14:35:56.652712622 +0530
***************
*** 92,103 ****
explain (costs off)
select sum(parallel_restricted(unique1)) from tenk1
group by(parallel_restricted(unique1));
! QUERY PLAN
! ----------------------------------------------------
HashAggregate
Group Key: parallel_restricted(unique1)
! -> Index Only Scan using tenk1_unique1 on tenk1
! (3 rows)
set force_parallel_mode=1;
explain (costs off)
--- 92,105 ----
explain (costs off)
select sum(parallel_restricted(unique1)) from tenk1
group by(parallel_restricted(unique1));
! QUERY PLAN
! -------------------------------------------------------------------
HashAggregate
Group Key: parallel_restricted(unique1)
! -> Gather
! Workers Planned: 4
! -> Parallel Index Only Scan using tenk1_unique1 on tenk1
! (5 rows)
IIUC, parallel operation being performed here is fine as parallel
restricted function occurs above Gather node
and just the expected output needs to be changed.
Thank you,
Rahila Syed
Attachments:
regression.diffsapplication/octet-stream; name=regression.diffsDownload
*** /home/rahila/postgres/postgres/src/test/regress/expected/select_parallel.out 2017-01-03 14:06:29.122022780 +0530
--- /home/rahila/postgres/postgres/src/test/regress/results/select_parallel.out 2017-01-12 14:35:56.652712622 +0530
***************
*** 92,103 ****
explain (costs off)
select sum(parallel_restricted(unique1)) from tenk1
group by(parallel_restricted(unique1));
! QUERY PLAN
! ----------------------------------------------------
HashAggregate
Group Key: parallel_restricted(unique1)
! -> Index Only Scan using tenk1_unique1 on tenk1
! (3 rows)
set force_parallel_mode=1;
explain (costs off)
--- 92,105 ----
explain (costs off)
select sum(parallel_restricted(unique1)) from tenk1
group by(parallel_restricted(unique1));
! QUERY PLAN
! -------------------------------------------------------------------
HashAggregate
Group Key: parallel_restricted(unique1)
! -> Gather
! Workers Planned: 4
! -> Parallel Index Only Scan using tenk1_unique1 on tenk1
! (5 rows)
set force_parallel_mode=1;
explain (costs off)
======================================================================
On Thu, Jan 12, 2017 at 5:39 PM, Rahila Syed <rahilasyed90@gmail.com> wrote:
Hello,
On applying the patch on latest master branch and running regression tests
following failure occurs.
I applied it on latest parallel index scan patches as given in the link
above.***
/home/rahila/postgres/postgres/src/test/regress/expected/select_parallel.out
2017-01-03 14:06:29.122022780 +0530
---
/home/rahila/postgres/postgres/src/test/regress/results/select_parallel.out
2017-01-12 14:35:56.652712622 +0530
***************
*** 92,103 ****
explain (costs off)
select sum(parallel_restricted(unique1)) from tenk1
group by(parallel_restricted(unique1));
! QUERY PLAN
! ----------------------------------------------------
HashAggregate
Group Key: parallel_restricted(unique1)
! -> Index Only Scan using tenk1_unique1 on tenk1
! (3 rows)set force_parallel_mode=1; explain (costs off) --- 92,105 ---- explain (costs off) select sum(parallel_restricted(unique1)) from tenk1 group by(parallel_restricted(unique1)); ! QUERY PLAN ! ------------------------------------------------------------------- HashAggregate Group Key: parallel_restricted(unique1) ! -> Gather ! Workers Planned: 4 ! -> Parallel Index Only Scan using tenk1_unique1 on tenk1 ! (5 rows)IIUC, parallel operation being performed here is fine as parallel restricted
function occurs above Gather node
and just the expected output needs to be changed.
True, fixed it, please find the attached file for the latest version.
Thank you,
Rahila Syed
Thanks a lot for the review Rahila.
--
Regards,
Rafia Sabih
EnterpriseDB: http://www.enterprisedb.com/
Attachments:
parallel_index_only_v4.patchapplication/octet-stream; name=parallel_index_only_v4.patchDownload
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index a474970..0d00d2b 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -29,6 +29,7 @@
#include "executor/nodeForeignscan.h"
#include "executor/nodeSeqscan.h"
#include "executor/nodeIndexscan.h"
+#include "executor/nodeIndexonlyscan.h"
#include "executor/tqueue.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/planmain.h"
@@ -200,6 +201,11 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
ExecIndexScanEstimate((IndexScanState *) planstate,
e->pcxt);
break;
+ case T_IndexOnlyScanState:
+ ExecIndexOnlyScanEstimate((IndexOnlyScanState *) planstate,
+ e->pcxt);
+ break;
+
case T_ForeignScanState:
ExecForeignScanEstimate((ForeignScanState *) planstate,
e->pcxt);
@@ -256,6 +262,11 @@ ExecParallelInitializeDSM(PlanState *planstate,
ExecIndexScanInitializeDSM((IndexScanState *) planstate,
d->pcxt);
break;
+ case T_IndexOnlyScanState:
+ ExecIndexOnlyScanInitializeDSM((IndexOnlyScanState *) planstate,
+ d->pcxt);
+ break;
+
case T_ForeignScanState:
ExecForeignScanInitializeDSM((ForeignScanState *) planstate,
d->pcxt);
@@ -735,6 +746,10 @@ ExecParallelInitializeWorker(PlanState *planstate, shm_toc *toc)
case T_IndexScanState:
ExecIndexScanInitializeWorker((IndexScanState *) planstate, toc);
break;
+ case T_IndexOnlyScanState:
+ ExecIndexOnlyScanInitializeWorker((IndexOnlyScanState *) planstate, toc);
+ break;
+
case T_ForeignScanState:
ExecForeignScanInitializeWorker((ForeignScanState *) planstate,
toc);
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 55c27a8..15a73fd 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -21,6 +21,9 @@
* ExecEndIndexOnlyScan releases all storage.
* ExecIndexOnlyMarkPos marks scan position.
* ExecIndexOnlyRestrPos restores scan position.
+ * ExecIndexOnlyScanEstimate estimates DSM space needed for parallel index-only scan
+ * ExecIndexOnlyScanInitializeDSM initialize DSM for parallel index-only scan
+ * ExecIndexOnlyScanInitializeWorker attach to DSM info in parallel worker
*/
#include "postgres.h"
@@ -277,6 +280,16 @@ ExecIndexOnlyScan(IndexOnlyScanState *node)
void
ExecReScanIndexOnlyScan(IndexOnlyScanState *node)
{
+ bool reset_parallel_scan = true;
+
+ /*
+ * if we are here to just update the scan keys, then don't reset parallel
+ * scan
+ */
+ if (node->ioss_NumRuntimeKeys != 0 && !node->ioss_RuntimeKeysReady)
+ reset_parallel_scan = false;
+
+
/*
* If we are doing runtime key calculations (ie, any of the index key
* values weren't simple Consts), compute the new key values. But first,
@@ -296,10 +309,16 @@ ExecReScanIndexOnlyScan(IndexOnlyScanState *node)
node->ioss_RuntimeKeysReady = true;
/* reset index scan */
- index_rescan(node->ioss_ScanDesc,
- node->ioss_ScanKeys, node->ioss_NumScanKeys,
- node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+ if (node->ioss_ScanDesc)
+ {
+
+ index_rescan(node->ioss_ScanDesc,
+ node->ioss_ScanKeys, node->ioss_NumScanKeys,
+ node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+ if (reset_parallel_scan)
+ index_parallelrescan(node->ioss_ScanDesc);
+ }
ExecScanReScan(&node->ss);
}
@@ -534,33 +553,99 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
{
indexstate->ioss_RuntimeContext = NULL;
}
-
- /*
- * Initialize scan descriptor.
- */
- indexstate->ioss_ScanDesc = index_beginscan(currentRelation,
+ if (!node->scan.plan.parallel_aware)
+ {
+ /*
+ * Initialize scan descriptor.
+ */
+ indexstate->ioss_ScanDesc = index_beginscan(currentRelation,
indexstate->ioss_RelationDesc,
estate->es_snapshot, NULL,
indexstate->ioss_NumScanKeys,
indexstate->ioss_NumOrderByKeys, false);
- /* Set it up for index-only scan */
- indexstate->ioss_ScanDesc->xs_want_itup = true;
- indexstate->ioss_VMBuffer = InvalidBuffer;
+ /* Set it up for index-only scan */
+ indexstate->ioss_ScanDesc->xs_want_itup = true;
+ indexstate->ioss_VMBuffer = InvalidBuffer;
- /*
- * If no run-time keys to calculate, go ahead and pass the scankeys to the
- * index AM.
- */
- if (indexstate->ioss_NumRuntimeKeys == 0)
- index_rescan(indexstate->ioss_ScanDesc,
+ /*
+ * If no run-time keys to calculate, go ahead and pass the scankeys to the
+ * index AM.
+ */
+ if (indexstate->ioss_NumRuntimeKeys == 0)
+ index_rescan(indexstate->ioss_ScanDesc,
indexstate->ioss_ScanKeys,
indexstate->ioss_NumScanKeys,
indexstate->ioss_OrderByKeys,
indexstate->ioss_NumOrderByKeys);
-
+ }
/*
* all done.
*/
return indexstate;
}
+void
+ExecIndexOnlyScanEstimate(IndexOnlyScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+
+ node->ioss_PscanLen = index_parallelscan_estimate(node->ioss_RelationDesc,
+ estate->es_snapshot);
+ shm_toc_estimate_chunk(&pcxt->estimator, node->ioss_PscanLen);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+void
+ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+ ParallelIndexScanDesc piscan;
+
+ piscan = shm_toc_allocate(pcxt->toc, node->ioss_PscanLen);
+ index_parallelscan_initialize(node->ss.ss_currentRelation,
+ node->ioss_RelationDesc,
+ estate->es_snapshot,
+ piscan);
+ shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, piscan);
+ node->ioss_ScanDesc =
+ index_beginscan_parallel(node->ss.ss_currentRelation,
+ node->ioss_RelationDesc,
+ node->ioss_NumScanKeys,
+ node->ioss_NumOrderByKeys,
+ piscan);
+ node->ioss_ScanDesc->xs_want_itup = true;
+
+ /*
+ * If no run-time keys to calculate, go ahead and pass the scankeys to the
+ * index AM.
+ */
+ if (node->ioss_NumRuntimeKeys == 0)
+ index_rescan(node->ioss_ScanDesc,
+ node->ioss_ScanKeys, node->ioss_NumScanKeys,
+ node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+}
+void
+ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node, shm_toc *toc)
+{
+ ParallelIndexScanDesc piscan;
+
+ piscan = shm_toc_lookup(toc, node->ss.ps.plan->plan_node_id);
+ node->ioss_ScanDesc =
+ index_beginscan_parallel(node->ss.ss_currentRelation,
+ node->ioss_RelationDesc,
+ node->ioss_NumScanKeys,
+ node->ioss_NumOrderByKeys,
+ piscan);
+ node->ioss_ScanDesc->xs_want_itup = true;
+
+ /*
+ * If no run-time keys to calculate, go ahead and pass the scankeys to the
+ * index AM.
+ */
+ if (node->ioss_NumRuntimeKeys == 0)
+ index_rescan(node->ioss_ScanDesc,
+ node->ioss_ScanKeys, node->ioss_NumScanKeys,
+ node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+}
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 5380b87..59fb561 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -1098,8 +1098,8 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
* If appropriate, consider parallel index scan. We don't allow
* parallel index scan for bitmap scans.
*/
- if (index->amcanparallel &&
- !index_only_scan &&
+ if ((index->amcanparallel ||
+ index_only_scan) &&
rel->consider_parallel &&
outer_relids == NULL &&
scantype != ST_BITMAPSCAN)
diff --git a/src/include/executor/nodeIndexonlyscan.h b/src/include/executor/nodeIndexonlyscan.h
index 4018af2..524e13f 100644
--- a/src/include/executor/nodeIndexonlyscan.h
+++ b/src/include/executor/nodeIndexonlyscan.h
@@ -14,6 +14,7 @@
#ifndef NODEINDEXONLYSCAN_H
#define NODEINDEXONLYSCAN_H
+#include "access/parallel.h"
#include "nodes/execnodes.h"
extern IndexOnlyScanState *ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags);
@@ -23,4 +24,8 @@ extern void ExecIndexOnlyMarkPos(IndexOnlyScanState *node);
extern void ExecIndexOnlyRestrPos(IndexOnlyScanState *node);
extern void ExecReScanIndexOnlyScan(IndexOnlyScanState *node);
+extern void ExecIndexOnlyScanEstimate(IndexOnlyScanState *node, ParallelContext *pcxt);
+extern void ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node, ParallelContext *pcxt);
+extern void ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node, shm_toc *toc);
+
#endif /* NODEINDEXONLYSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 58ec9d2..cb441f2 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1418,6 +1418,8 @@ typedef struct IndexOnlyScanState
IndexScanDesc ioss_ScanDesc;
Buffer ioss_VMBuffer;
long ioss_HeapFetches;
+ /* This is needed for parallel index scan */
+ Size ioss_PscanLen;
} IndexOnlyScanState;
/* ----------------
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 18e21b7..741e2bd 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -92,12 +92,14 @@ explain (costs off)
explain (costs off)
select sum(parallel_restricted(unique1)) from tenk1
group by(parallel_restricted(unique1));
- QUERY PLAN
-----------------------------------------------------
+ QUERY PLAN
+-------------------------------------------------------------------
HashAggregate
Group Key: parallel_restricted(unique1)
- -> Index Only Scan using tenk1_unique1 on tenk1
-(3 rows)
+ -> Gather
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on tenk1
+(5 rows)
set force_parallel_mode=1;
explain (costs off)
On Fri, Jan 13, 2017 at 2:19 PM, Rafia Sabih
<rafia.sabih@enterprisedb.com> wrote:
On Thu, Jan 12, 2017 at 5:39 PM, Rahila Syed <rahilasyed90@gmail.com> wrote:
Hello,
On applying the patch on latest master branch and running regression tests
following failure occurs.
I applied it on latest parallel index scan patches as given in the link
above.***
/home/rahila/postgres/postgres/src/test/regress/expected/select_parallel.out
2017-01-03 14:06:29.122022780 +0530
---
/home/rahila/postgres/postgres/src/test/regress/results/select_parallel.out
2017-01-12 14:35:56.652712622 +0530
***************
*** 92,103 ****
explain (costs off)
select sum(parallel_restricted(unique1)) from tenk1
group by(parallel_restricted(unique1));
! QUERY PLAN
! ----------------------------------------------------
HashAggregate
Group Key: parallel_restricted(unique1)
! -> Index Only Scan using tenk1_unique1 on tenk1
! (3 rows)set force_parallel_mode=1; explain (costs off) --- 92,105 ---- explain (costs off) select sum(parallel_restricted(unique1)) from tenk1 group by(parallel_restricted(unique1)); ! QUERY PLAN ! ------------------------------------------------------------------- HashAggregate Group Key: parallel_restricted(unique1) ! -> Gather ! Workers Planned: 4 ! -> Parallel Index Only Scan using tenk1_unique1 on tenk1 ! (5 rows)IIUC, parallel operation being performed here is fine as parallel restricted
function occurs above Gather node
and just the expected output needs to be changed.True, fixed it, please find the attached file for the latest version.
Please find the attached file rebased patch of parallel index-only
scan on the latest Parallel index-scan patch [1]/messages/by-id/CAA4eK1L-gb0Fum3mQN4c5PWJXNE7xs7pzwMDWsrDYLucKqvJ2A@mail.gmail.com -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/.
[1]: /messages/by-id/CAA4eK1L-gb0Fum3mQN4c5PWJXNE7xs7pzwMDWsrDYLucKqvJ2A@mail.gmail.com -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/
--
Regards,
Rafia Sabih
EnterpriseDB: http://www.enterprisedb.com/
Attachments:
parallel_index_only_v5.patchapplication/octet-stream; name=parallel_index_only_v5.patchDownload
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index cb91cc3..ea670ed 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -29,6 +29,7 @@
#include "executor/nodeForeignscan.h"
#include "executor/nodeSeqscan.h"
#include "executor/nodeIndexscan.h"
+#include "executor/nodeIndexonlyscan.h"
#include "executor/tqueue.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/planmain.h"
@@ -202,6 +203,10 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
ExecIndexScanEstimate((IndexScanState *) planstate,
e->pcxt);
break;
+ case T_IndexOnlyScanState:
+ ExecIndexOnlyScanEstimate((IndexOnlyScanState *) planstate,
+ e->pcxt);
+ break;
case T_ForeignScanState:
ExecForeignScanEstimate((ForeignScanState *) planstate,
e->pcxt);
@@ -258,6 +263,10 @@ ExecParallelInitializeDSM(PlanState *planstate,
ExecIndexScanInitializeDSM((IndexScanState *) planstate,
d->pcxt);
break;
+ case T_IndexOnlyScanState:
+ ExecIndexOnlyScanInitializeDSM((IndexOnlyScanState *) planstate,
+ d->pcxt);
+ break;
case T_ForeignScanState:
ExecForeignScanInitializeDSM((ForeignScanState *) planstate,
d->pcxt);
@@ -737,6 +746,9 @@ ExecParallelInitializeWorker(PlanState *planstate, shm_toc *toc)
case T_IndexScanState:
ExecIndexScanInitializeWorker((IndexScanState *) planstate, toc);
break;
+ case T_IndexOnlyScanState:
+ ExecIndexOnlyScanInitializeWorker((IndexOnlyScanState *) planstate, toc);
+ break;
case T_ForeignScanState:
ExecForeignScanInitializeWorker((ForeignScanState *) planstate,
toc);
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index ddef3a4..9e65d06 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -21,6 +21,11 @@
* ExecEndIndexOnlyScan releases all storage.
* ExecIndexOnlyMarkPos marks scan position.
* ExecIndexOnlyRestrPos restores scan position.
+ * ExecIndexOnlyScanEstimate estimates DSM space needed for
+ * parallel index-only scan
+ * ExecIndexOnlyScanInitializeDSM initialize DSM for parallel
+ * index-only scan
+ * ExecIndexOnlyScanInitializeWorker attach to DSM info in parallel worker
*/
#include "postgres.h"
@@ -277,6 +282,15 @@ ExecIndexOnlyScan(IndexOnlyScanState *node)
void
ExecReScanIndexOnlyScan(IndexOnlyScanState *node)
{
+ bool reset_parallel_scan = true;
+
+ /*
+ * if we are here to just update the scan keys, then don't reset parallel
+ * scan
+ */
+ if (node->ioss_NumRuntimeKeys != 0 && !node->ioss_RuntimeKeysReady)
+ reset_parallel_scan = false;
+
/*
* If we are doing runtime key calculations (ie, any of the index key
* values weren't simple Consts), compute the new key values. But first,
@@ -296,10 +310,15 @@ ExecReScanIndexOnlyScan(IndexOnlyScanState *node)
node->ioss_RuntimeKeysReady = true;
/* reset index scan */
- index_rescan(node->ioss_ScanDesc,
- node->ioss_ScanKeys, node->ioss_NumScanKeys,
- node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+ if (node->ioss_ScanDesc)
+ {
+ index_rescan(node->ioss_ScanDesc,
+ node->ioss_ScanKeys, node->ioss_NumScanKeys,
+ node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+ if (reset_parallel_scan)
+ index_parallelrescan(node->ioss_ScanDesc);
+ }
ExecScanReScan(&node->ss);
}
@@ -538,29 +557,121 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
/*
* Initialize scan descriptor.
*/
- indexstate->ioss_ScanDesc = index_beginscan(currentRelation,
+ if (!node->scan.plan.parallel_aware)
+ {
+ indexstate->ioss_ScanDesc = index_beginscan(currentRelation,
indexstate->ioss_RelationDesc,
estate->es_snapshot,
indexstate->ioss_NumScanKeys,
indexstate->ioss_NumOrderByKeys);
- /* Set it up for index-only scan */
- indexstate->ioss_ScanDesc->xs_want_itup = true;
- indexstate->ioss_VMBuffer = InvalidBuffer;
- /*
- * If no run-time keys to calculate, go ahead and pass the scankeys to the
- * index AM.
- */
- if (indexstate->ioss_NumRuntimeKeys == 0)
- index_rescan(indexstate->ioss_ScanDesc,
+ /* Set it up for index-only scan */
+ indexstate->ioss_ScanDesc->xs_want_itup = true;
+ indexstate->ioss_VMBuffer = InvalidBuffer;
+ /*
+ * If no run-time keys to calculate, go ahead and pass the scankeys to the
+ * index AM.
+ */
+ if (indexstate->ioss_NumRuntimeKeys == 0)
+ index_rescan(indexstate->ioss_ScanDesc,
indexstate->ioss_ScanKeys,
indexstate->ioss_NumScanKeys,
indexstate->ioss_OrderByKeys,
indexstate->ioss_NumOrderByKeys);
-
+ }
/*
* all done.
*/
return indexstate;
+ }
+
+/* ----------------------------------------------------------------
+ * Parallel Index-only Scan Support
+ * ----------------------------------------------------------------
+ */
+
+/* ----------------------------------------------------------------
+ * ExecIndexOnlyScanEstimate
+ *
+ * estimates the space required to serialize index-only scan node.
+ * ----------------------------------------------------------------
+ */
+void
+ExecIndexOnlyScanEstimate(IndexOnlyScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+
+ node->ioss_PscanLen = index_parallelscan_estimate(node->ioss_RelationDesc,
+ estate->es_snapshot);
+ shm_toc_estimate_chunk(&pcxt->estimator, node->ioss_PscanLen);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+/* ----------------------------------------------------------------
+ * ExecIndexOnlyScanInitializeDSM
+ *
+ * Set up a parallel index-only scan descriptor.
+ * ----------------------------------------------------------------
+ */
+void
+ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+ ParallelIndexScanDesc piscan;
+
+ piscan = shm_toc_allocate(pcxt->toc, node->ioss_PscanLen);
+ index_parallelscan_initialize(node->ss.ss_currentRelation,
+ node->ioss_RelationDesc,
+ estate->es_snapshot,
+ piscan);
+ shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, piscan);
+ node->ioss_ScanDesc =
+ index_beginscan_parallel(node->ss.ss_currentRelation,
+ node->ioss_RelationDesc,
+ node->ioss_NumScanKeys,
+ node->ioss_NumOrderByKeys,
+ piscan);
+ node->ioss_ScanDesc->xs_want_itup = true;
+
+ /*
+ * If no run-time keys to calculate, go ahead and pass the scankeys to the
+ * index AM.
+ */
+ if (node->ioss_NumRuntimeKeys == 0)
+ index_rescan(node->ioss_ScanDesc,
+ node->ioss_ScanKeys, node->ioss_NumScanKeys,
+ node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
+}
+
+/* ----------------------------------------------------------------
+ * ExecIndexOnlyScanInitializeWorker
+ *
+ * Copy relevant information from TOC into planstate.
+ * ----------------------------------------------------------------
+ */
+void
+ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node, shm_toc *toc)
+{
+ ParallelIndexScanDesc piscan;
+
+ piscan = shm_toc_lookup(toc, node->ss.ps.plan->plan_node_id);
+ node->ioss_ScanDesc =
+ index_beginscan_parallel(node->ss.ss_currentRelation,
+ node->ioss_RelationDesc,
+ node->ioss_NumScanKeys,
+ node->ioss_NumOrderByKeys,
+ piscan);
+ node->ioss_ScanDesc->xs_want_itup = true;
+
+ /*
+ * If no run-time keys to calculate, go ahead and pass the scankeys to the
+ * index AM.
+ */
+ if (node->ioss_NumRuntimeKeys == 0)
+ index_rescan(node->ioss_ScanDesc,
+ node->ioss_ScanKeys, node->ioss_NumScanKeys,
+ node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
}
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 1905917..6231774 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -1050,8 +1050,8 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
* If appropriate, consider parallel index scan. We don't allow
* parallel index scan for bitmap scans.
*/
- if (index->amcanparallel &&
- !index_only_scan &&
+ if ((index->amcanparallel ||
+ index_only_scan) &&
rel->consider_parallel &&
outer_relids == NULL &&
scantype != ST_BITMAPSCAN)
diff --git a/src/include/executor/nodeIndexonlyscan.h b/src/include/executor/nodeIndexonlyscan.h
index 4018af2..5bb44c2 100644
--- a/src/include/executor/nodeIndexonlyscan.h
+++ b/src/include/executor/nodeIndexonlyscan.h
@@ -15,6 +15,7 @@
#define NODEINDEXONLYSCAN_H
#include "nodes/execnodes.h"
+#include "access/parallel.h"
extern IndexOnlyScanState *ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags);
extern TupleTableSlot *ExecIndexOnlyScan(IndexOnlyScanState *node);
@@ -23,4 +24,9 @@ extern void ExecIndexOnlyMarkPos(IndexOnlyScanState *node);
extern void ExecIndexOnlyRestrPos(IndexOnlyScanState *node);
extern void ExecReScanIndexOnlyScan(IndexOnlyScanState *node);
+/* Support functions for parallel index-only scans */
+extern void ExecIndexOnlyScanEstimate(IndexOnlyScanState *node, ParallelContext *pcxt);
+extern void ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node, ParallelContext *pcxt);
+extern void ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node, shm_toc *toc);
+
#endif /* NODEINDEXONLYSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index b565f28..02a519c 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1418,6 +1418,7 @@ typedef struct IndexScanState
* ScanDesc index scan descriptor
* VMBuffer buffer in use for visibility map testing, if any
* HeapFetches number of tuples we were forced to fetch from heap
+ * ioss_PscanLen This is needed for parallel index scan
* ----------------
*/
typedef struct IndexOnlyScanState
@@ -1436,6 +1437,7 @@ typedef struct IndexOnlyScanState
IndexScanDesc ioss_ScanDesc;
Buffer ioss_VMBuffer;
long ioss_HeapFetches;
+ Size ioss_PscanLen; /* This is needed for parallel index scan */
} IndexOnlyScanState;
/* ----------------
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 140fb3c..c919e59 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -92,12 +92,14 @@ explain (costs off)
explain (costs off)
select sum(parallel_restricted(unique1)) from tenk1
group by(parallel_restricted(unique1));
- QUERY PLAN
-----------------------------------------------------
+ QUERY PLAN
+-------------------------------------------------------------------
HashAggregate
Group Key: parallel_restricted(unique1)
- -> Index Only Scan using tenk1_unique1 on tenk1
-(3 rows)
+ -> Gather
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on tenk1
+(5 rows)
-- test parallel index scans.
set enable_seqscan to off;
On 01/19/2017 05:37 PM, Rafia Sabih wrote:
Please find the attached file rebased patch of parallel index-only
scan on the latest Parallel index-scan patch [1].
We did some testing of this feature and written few testcases. PFA the
sql script(along with the expected .out files)
In addition we have generated the LCOV (code coverage) report and
compared the files which are changed for this.
You can see the numbers for "with_patch" V/s "with_patch+TestCases"
(.pdf file is attached)
--
regards,tushar
EnterpriseDB https://www.enterprisedb.com/
The Enterprise PostgreSQL Company
Attachments:
pios.pdfapplication/pdf; name=pios.pdfDownload
%PDF-1.4
% ����
4
0
obj
<<
/Type
/Catalog
/Names
<<
/JavaScript
3
0
R
>>
/PageLabels
<<
/Nums
[
0
<<
/S
/D
/St
1
>>
]
>>
/Outlines
2
0
R
/Pages
1
0
R
>>
endobj
5
0
obj
<<
/Creator
(�� G o o g l e S h e e t s)
/Title
()
>>
endobj
6
0
obj
<<
/Type
/Page
/Parent
1
0
R
/MediaBox
[
0
0
792
612
]
/Contents
7
0
R
/Resources
8
0
R
/Annots
10
0
R
/Group
<<
/S
/Transparency
/CS
/DeviceRGB
>>
>>
endobj
7
0
obj
<<
/Filter
/FlateDecode
/Length
9
0
R
>>
stream
x��\]�����b�&�����CP$ErC���[8�N���G��q�Ix�&-��_�Rg(�]���q�<{�3G�p���y�^������0������~��+���r����Z���_P�@�����xC�����P�v�v?������;��\2^�H���ri����*���$e�d���lLE� vc�g���4���0@�f8����{NC��^�3��fY�S���`"�wQO��A��u_�������������u_Pr
t�;8P���������cv��9$E�#�����n�����R��z�:5��o��T��l�Q�S����!��5��5�H�
��mo��|)zqp9��#V�l��X������n|v?�>��������?1�Q����7~c�����%C�:�X
M�����y4�5~\��Y[�f���&���a���ph�[*�`|
�����]Wa����V0*���"`2]�� ��F�4IAh�+���3`�4J�f@
�uDAg���
W���X0��K3 �:���� ���.kq�r�F��H�nR ��*������4�"�'{W������f ����p���=\m�q���m�8��?�
����'pX��'� �%2u���1`� �����4���G�N^o��
�a� ���AI���DI��*�"�=�l*#Q%!�zZ�����x��!~�dS94 9���(�������&8@Dy�S�K�&��{d�O�t��N����jS��������0r)i*"b��Aa @���7�9�`���L]'(
��ph ��D
d@IbQl�F���mK����34�������n�OE�U:h� ��i,�R� �3[�����!C�
DB�H�(Yp�F����G����$4���-A����i���J����B��0���M���L ��I
�fA�C 98��!�����qA����"�H�������f�pLY��E�iH�A{��=�%��$\�<�X�t���Q�Enc��,��E'��6��N��DA�A@�$h��6��NP�!Dy��3�8D2�ddA�-SF%��kl��9����8t(����]���hy\��\�BY���&