heapgettup() with NoMovementScanDirection unused in core?

Started by Melanie Plagemanalmost 3 years ago17 messages
#1Melanie Plageman
melanieplageman@gmail.com

Hi,

David Rowley and I were discussing how to test the
NoMovementScanDirection case for heapgettup() and heapgettup_pagemode()
in [1]/messages/by-id/CAAKRu_ZyiXwWS1WXSZneoy+sjoH_+F5UhO-1uFhyi-u0d6z+fA@mail.gmail.com (since there is not currently coverage). We are actually
wondering if it is dead code (in core).

This is a link to the code in question on github in [2]https://github.com/postgres/postgres/blob/master/src/backend/access/heap/heapam.c#L656 (side note: is
there a more evergreen way to do this that doesn't involve pasting a
hundred lines of code into this email? You need quite a few lines of
context for it to be clear what code I am talking about.)

standard_ExecutorRun() doesn't run ExecutePlan() if scan direction is no
movement.

if (!ScanDirectionIsNoMovement(direction))
{
...
ExecutePlan(estate,
queryDesc->planstate,
}

And other users of heapgettup() through table_scan_getnextslot() and the
like all seem to pass ForwardScanDirection as the direction.

A skilled code archaeologist brought our attention to adbfab119b308a
which appears to remove the only users in the core codebase calling
heapgettup() and heapgettup_pagemode() with NoMovementScanDirection (and
those users were not themselves used).

Perhaps we can remove support for NoMovementScanDirection with
heapgettup()? Unless someone knows of a good use case for a table AM to
do this?

- Melanie

[1]: /messages/by-id/CAAKRu_ZyiXwWS1WXSZneoy+sjoH_+F5UhO-1uFhyi-u0d6z+fA@mail.gmail.com
[2]: https://github.com/postgres/postgres/blob/master/src/backend/access/heap/heapam.c#L656

#2David Rowley
dgrowleyml@gmail.com
In reply to: Melanie Plageman (#1)
Re: heapgettup() with NoMovementScanDirection unused in core?

On Wed, 25 Jan 2023 at 13:55, Melanie Plageman
<melanieplageman@gmail.com> wrote:

David Rowley and I were discussing how to test the
NoMovementScanDirection case for heapgettup() and heapgettup_pagemode()
in [1] (since there is not currently coverage). We are actually
wondering if it is dead code (in core).

Yeah, so I see nothing in core that can cause heapgettup() or
heapgettup_pagemode() to be called with NoMovementScanDirection. I
imagine one possible way to hit it might be in an extension where
someone's written their own ExecutorRun_hook that does not have the
same NoMovementScanDirection check that standard_ExecutorRun() has.

So far my thoughts are that we should just rip it out and see if
anyone complains. If they complain loudly enough, then it's easy
enough to put it back without any compatibility issues. However, if
it's for the ExecutorRun_hook reason, then they'll likely be better to
add the same NoMovementScanDirection as we have in
standard_ExecutorRun().

I'm just not keen on refactoring the code without the means to test
that the new code actually works.

Does anyone know of any reason why we shouldn't ditch the nomovement
code in heapgettup/heapgettup_pagemode?

Maybe we'd also want to Assert that the direction is either forwards
or backwards in table_scan_getnextslot and
table_scan_getnextslot_tidrange. (I see heap_getnextslot_tidrange()
does not have any handling for NoMovementScanDirection, so if this is
not dead code, that probably needs to be fixed)

David

#3Tom Lane
tgl@sss.pgh.pa.us
In reply to: David Rowley (#2)
Re: heapgettup() with NoMovementScanDirection unused in core?

David Rowley <dgrowleyml@gmail.com> writes:

Does anyone know of any reason why we shouldn't ditch the nomovement
code in heapgettup/heapgettup_pagemode?

AFAICS, the remaining actual use-case for NoMovementScanDirection
is that defined by ExecutorRun:

* If direction is NoMovementScanDirection then nothing is done
* except to start up/shut down the destination. Otherwise,
* we retrieve up to 'count' tuples in the specified direction.
*
* Note: count = 0 is interpreted as no portal limit, i.e., run to
* completion.

We must have the NoMovementScanDirection option because count = 0
does not mean "do nothing", and I noted at least two call sites
that require it.

The heapgettup definition is thus not only unreachable, but confusingly
inconsistent with this meaning.

I wonder if we couldn't also get rid of this confusingly-inconsistent
alternative usage in the planner:

* 'indexscandir' is one of:
* ForwardScanDirection: forward scan of an ordered index
* BackwardScanDirection: backward scan of an ordered index
* NoMovementScanDirection: scan of an unordered index, or don't care
* (The executor doesn't care whether it gets ForwardScanDirection or
* NoMovementScanDirection for an indexscan, but the planner wants to
* distinguish ordered from unordered indexes for building pathkeys.)

While that comment's claim is plausible, I think it's been wrong for
years. AFAICS indxpath.c makes pathkeys before it ever does this:

index_is_ordered ?
ForwardScanDirection :
NoMovementScanDirection,

and nothing depends on that later either. So I think we could
simplify this to something like "indexscandir is either
ForwardScanDirection or BackwardScanDirection. (Unordered
index types need not support BackwardScanDirection.)"

regards, tom lane

#4Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#3)
Re: heapgettup() with NoMovementScanDirection unused in core?

Hi,

On 2023-01-25 10:02:28 -0500, Tom Lane wrote:

David Rowley <dgrowleyml@gmail.com> writes:

Does anyone know of any reason why we shouldn't ditch the nomovement
code in heapgettup/heapgettup_pagemode?

+1

Because I dug it up yesterday. There used to be callers of heap* with
NoMovement. But they were unused themselves:
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=adbfab119b308a7e0e6b1305de9be222cfd5c85b

* If direction is NoMovementScanDirection then nothing is done
* except to start up/shut down the destination. Otherwise,
* we retrieve up to 'count' tuples in the specified direction.
*
* Note: count = 0 is interpreted as no portal limit, i.e., run to
* completion.

We must have the NoMovementScanDirection option because count = 0
does not mean "do nothing", and I noted at least two call sites
that require it.

I wonder if we'd be better off removing NoMovementScanDirection, and using
count == (uint64)-1 for what NoMovementScanDirection is currently used for as
an ExecutorRun parameter. Seems less confusing to me - right now we have two
parameters with non-obbvious meanings and interactions.

I wonder if we couldn't also get rid of this confusingly-inconsistent
alternative usage in the planner:

* 'indexscandir' is one of:
* ForwardScanDirection: forward scan of an ordered index
* BackwardScanDirection: backward scan of an ordered index
* NoMovementScanDirection: scan of an unordered index, or don't care
* (The executor doesn't care whether it gets ForwardScanDirection or
* NoMovementScanDirection for an indexscan, but the planner wants to
* distinguish ordered from unordered indexes for building pathkeys.)

+1

Certainly seems confusing to me.

Greetings,

Andres Freund

#5Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#4)
Re: heapgettup() with NoMovementScanDirection unused in core?

Andres Freund <andres@anarazel.de> writes:

On 2023-01-25 10:02:28 -0500, Tom Lane wrote:

We must have the NoMovementScanDirection option because count = 0
does not mean "do nothing", and I noted at least two call sites
that require it.

I wonder if we'd be better off removing NoMovementScanDirection, and using
count == (uint64)-1 for what NoMovementScanDirection is currently used for as
an ExecutorRun parameter. Seems less confusing to me - right now we have two
parameters with non-obbvious meanings and interactions.

I'm down on that because it seems just about certain to break extensions
that call the executor, and it isn't adding enough clarity (IMHO) to
justify that.

regards, tom lane

#6Melanie Plageman
melanieplageman@gmail.com
In reply to: Tom Lane (#3)
1 attachment(s)
Re: heapgettup() with NoMovementScanDirection unused in core?

Hi,

I have written the patch to remove the unreachable code in
heapgettup_pagemode]().

On Wed, Jan 25, 2023 at 10:02 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:

I wonder if we couldn't also get rid of this confusingly-inconsistent
alternative usage in the planner:

* 'indexscandir' is one of:
* ForwardScanDirection: forward scan of an ordered index
* BackwardScanDirection: backward scan of an ordered index
* NoMovementScanDirection: scan of an unordered index, or don't care
* (The executor doesn't care whether it gets ForwardScanDirection or
* NoMovementScanDirection for an indexscan, but the planner wants to
* distinguish ordered from unordered indexes for building pathkeys.)

While that comment's claim is plausible, I think it's been wrong for
years. AFAICS indxpath.c makes pathkeys before it ever does this:

index_is_ordered ?
ForwardScanDirection :
NoMovementScanDirection,

and nothing depends on that later either. So I think we could
simplify this to something like "indexscandir is either
ForwardScanDirection or BackwardScanDirection. (Unordered
index types need not support BackwardScanDirection.)"

I also did what I *think* Tom is suggesting here -- make index scan's
scan direction always forward or backward...

Maybe the set should be two patches...dunno.

- Melanie

Attachments:

v1-0001-Remove-NoMovementScanDirection-inconsistencies.patchtext/x-patch; charset=US-ASCII; name=v1-0001-Remove-NoMovementScanDirection-inconsistencies.patchDownload
From be8119af72499aec03e59200c9db44f8520ccebf Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 27 Jan 2023 14:02:03 -0500
Subject: [PATCH v1] Remove NoMovementScanDirection inconsistencies

Remove use of NoMovementScanDirection for index scans. Unordered indexes
will now always use ForwardScanDirection.

Remove unreachable code in heapgettup() and heapgettup_pagemode()
handling NoMovementScanDirection.

Add assertions in case table AMs try to use scan directions other than
forward and backward.

Discussion: https://www.postgresql.org/message-id/flat/CAAKRu_bvkhka0CZQun28KTqhuUh5ZqY%3D_T8QEqZqOL02rpi2bw%40mail.gmail.com
---
 src/backend/access/heap/heapam.c         | 73 ++++--------------------
 src/backend/commands/explain.c           |  3 -
 src/backend/executor/nodeIndexonlyscan.c | 17 +++---
 src/backend/executor/nodeIndexscan.c     | 17 +++---
 src/backend/optimizer/path/indxpath.c    |  8 +--
 src/backend/optimizer/util/pathnode.c    |  5 +-
 src/include/access/tableam.h             |  6 ++
 src/test/regress/expected/hash_index.out | 27 +++++++++
 src/test/regress/sql/hash_index.sql      | 23 ++++++++
 9 files changed, 87 insertions(+), 92 deletions(-)

diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e6024a980b..be0555f5c4 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -523,6 +523,9 @@ heapgettup(HeapScanDesc scan,
 	int			linesleft;
 	ItemId		lpp;
 
+	Assert(dir == ForwardScanDirection ||
+		   dir == BackwardScanDirection);
+
 	/*
 	 * calculate next starting lineoff, given scan direction
 	 */
@@ -583,7 +586,7 @@ heapgettup(HeapScanDesc scan,
 
 		linesleft = lines - lineoff + 1;
 	}
-	else if (backward)
+	else
 	{
 		/* backward parallel scan not supported */
 		Assert(scan->rs_base.rs_parallel == NULL);
@@ -653,34 +656,6 @@ heapgettup(HeapScanDesc scan,
 
 		linesleft = lineoff;
 	}
-	else
-	{
-		/*
-		 * ``no movement'' scan direction: refetch prior tuple
-		 */
-		if (!scan->rs_inited)
-		{
-			Assert(!BufferIsValid(scan->rs_cbuf));
-			tuple->t_data = NULL;
-			return;
-		}
-
-		block = ItemPointerGetBlockNumber(&(tuple->t_self));
-		if (block != scan->rs_cblock)
-			heapgetpage((TableScanDesc) scan, block);
-
-		/* Since the tuple was previously fetched, needn't lock page here */
-		page = BufferGetPage(scan->rs_cbuf);
-		TestForOldSnapshot(snapshot, scan->rs_base.rs_rd, page);
-		lineoff = ItemPointerGetOffsetNumber(&(tuple->t_self));
-		lpp = PageGetItemId(page, lineoff);
-		Assert(ItemIdIsNormal(lpp));
-
-		tuple->t_data = (HeapTupleHeader) PageGetItem(page, lpp);
-		tuple->t_len = ItemIdGetLength(lpp);
-
-		return;
-	}
 
 	/*
 	 * advance the scan until we find a qualifying tuple or run out of stuff
@@ -861,6 +836,9 @@ heapgettup_pagemode(HeapScanDesc scan,
 	int			linesleft;
 	ItemId		lpp;
 
+	Assert(dir == ForwardScanDirection ||
+		   dir == BackwardScanDirection);
+
 	/*
 	 * calculate next starting lineindex, given scan direction
 	 */
@@ -918,7 +896,7 @@ heapgettup_pagemode(HeapScanDesc scan,
 
 		linesleft = lines - lineindex;
 	}
-	else if (backward)
+	else
 	{
 		/* backward parallel scan not supported */
 		Assert(scan->rs_base.rs_parallel == NULL);
@@ -978,38 +956,6 @@ heapgettup_pagemode(HeapScanDesc scan,
 
 		linesleft = lineindex + 1;
 	}
-	else
-	{
-		/*
-		 * ``no movement'' scan direction: refetch prior tuple
-		 */
-		if (!scan->rs_inited)
-		{
-			Assert(!BufferIsValid(scan->rs_cbuf));
-			tuple->t_data = NULL;
-			return;
-		}
-
-		block = ItemPointerGetBlockNumber(&(tuple->t_self));
-		if (block != scan->rs_cblock)
-			heapgetpage((TableScanDesc) scan, block);
-
-		/* Since the tuple was previously fetched, needn't lock page here */
-		page = BufferGetPage(scan->rs_cbuf);
-		TestForOldSnapshot(scan->rs_base.rs_snapshot, scan->rs_base.rs_rd, page);
-		lineoff = ItemPointerGetOffsetNumber(&(tuple->t_self));
-		lpp = PageGetItemId(page, lineoff);
-		Assert(ItemIdIsNormal(lpp));
-
-		tuple->t_data = (HeapTupleHeader) PageGetItem(page, lpp);
-		tuple->t_len = ItemIdGetLength(lpp);
-
-		/* check that rs_cindex is in sync */
-		Assert(scan->rs_cindex < scan->rs_ntuples);
-		Assert(lineoff == scan->rs_vistuples[scan->rs_cindex]);
-
-		return;
-	}
 
 	/*
 	 * advance the scan until we find a qualifying tuple or run out of stuff
@@ -1299,6 +1245,9 @@ heap_getnext(TableScanDesc sscan, ScanDirection direction)
 {
 	HeapScanDesc scan = (HeapScanDesc) sscan;
 
+	Assert(direction == ForwardScanDirection ||
+		   direction == BackwardScanDirection);
+
 	/*
 	 * This is still widely used directly, without going through table AM, so
 	 * add a safety check.  It's possible we should, at a later point,
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index a0311ce9dc..d38311fa7e 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -3746,9 +3746,6 @@ ExplainIndexScanDetails(Oid indexid, ScanDirection indexorderdir,
 			case BackwardScanDirection:
 				scandir = "Backward";
 				break;
-			case NoMovementScanDirection:
-				scandir = "NoMovement";
-				break;
 			case ForwardScanDirection:
 				scandir = "Forward";
 				break;
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 8c7da9ee60..de7c71890a 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -70,15 +70,7 @@ IndexOnlyNext(IndexOnlyScanState *node)
 	 * extract necessary information from index scan node
 	 */
 	estate = node->ss.ps.state;
-	direction = estate->es_direction;
-	/* flip direction if this is an overall backward scan */
-	if (ScanDirectionIsBackward(((IndexOnlyScan *) node->ss.ps.plan)->indexorderdir))
-	{
-		if (ScanDirectionIsForward(direction))
-			direction = BackwardScanDirection;
-		else if (ScanDirectionIsBackward(direction))
-			direction = ForwardScanDirection;
-	}
+	direction = estate->es_direction * ((IndexOnlyScan *) node->ss.ps.plan)->indexorderdir;
 	scandesc = node->ioss_ScanDesc;
 	econtext = node->ss.ps.ps_ExprContext;
 	slot = node->ss.ss_ScanTupleSlot;
@@ -503,6 +495,13 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags)
 	indexstate->ss.ps.state = estate;
 	indexstate->ss.ps.ExecProcNode = ExecIndexOnlyScan;
 
+	/*
+	 * Only forward and backward are valid scan directions. Unordered indexes
+	 * will always have ForwardScanDirection.
+	 */
+	Assert(node->indexorderdir == ForwardScanDirection ||
+		   node->indexorderdir == BackwardScanDirection);
+
 	/*
 	 * Miscellaneous initialization
 	 *
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index f1ced9ff0f..403ace9c90 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -90,15 +90,7 @@ IndexNext(IndexScanState *node)
 	 * extract necessary information from index scan node
 	 */
 	estate = node->ss.ps.state;
-	direction = estate->es_direction;
-	/* flip direction if this is an overall backward scan */
-	if (ScanDirectionIsBackward(((IndexScan *) node->ss.ps.plan)->indexorderdir))
-	{
-		if (ScanDirectionIsForward(direction))
-			direction = BackwardScanDirection;
-		else if (ScanDirectionIsBackward(direction))
-			direction = ForwardScanDirection;
-	}
+	direction = estate->es_direction * ((IndexScan *) node->ss.ps.plan)->indexorderdir;
 	scandesc = node->iss_ScanDesc;
 	econtext = node->ss.ps.ps_ExprContext;
 	slot = node->ss.ss_ScanTupleSlot;
@@ -916,6 +908,13 @@ ExecInitIndexScan(IndexScan *node, EState *estate, int eflags)
 	indexstate->ss.ps.state = estate;
 	indexstate->ss.ps.ExecProcNode = ExecIndexScan;
 
+	/*
+	 * Only forward and backward are valid scan directions. Unordered indexes
+	 * will always have ForwardScanDirection.
+	 */
+	Assert(node->indexorderdir == ForwardScanDirection ||
+		   node->indexorderdir == BackwardScanDirection);
+
 	/*
 	 * Miscellaneous initialization
 	 *
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index e13c8f1914..751fa2edfe 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -1015,9 +1015,7 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
 								  orderbyclauses,
 								  orderbyclausecols,
 								  useful_pathkeys,
-								  index_is_ordered ?
-								  ForwardScanDirection :
-								  NoMovementScanDirection,
+								  ForwardScanDirection,
 								  index_only_scan,
 								  outer_relids,
 								  loop_count,
@@ -1037,9 +1035,7 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
 									  orderbyclauses,
 									  orderbyclausecols,
 									  useful_pathkeys,
-									  index_is_ordered ?
-									  ForwardScanDirection :
-									  NoMovementScanDirection,
+									  ForwardScanDirection,
 									  index_only_scan,
 									  outer_relids,
 									  loop_count,
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 4478036bb6..dd5236682f 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -982,9 +982,8 @@ create_samplescan_path(PlannerInfo *root, RelOptInfo *rel, Relids required_outer
  * 'indexorderbycols' is an integer list of index column numbers (zero based)
  *			the ordering operators can be used with.
  * 'pathkeys' describes the ordering of the path.
- * 'indexscandir' is ForwardScanDirection or BackwardScanDirection
- *			for an ordered index, or NoMovementScanDirection for
- *			an unordered index.
+ * 'indexscandir' is either ForwardScanDirection or BackwardScanDirection.
+ * 			Unordered index types need not support BackwardScanDirection.
  * 'indexonly' is true if an index-only scan is wanted.
  * 'required_outer' is the set of outer relids for a parameterized path.
  * 'loop_count' is the number of repetitions of the indexscan to factor into
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 3fb184717f..28abe819b0 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -1033,6 +1033,9 @@ extern void table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot);
 static inline bool
 table_scan_getnextslot(TableScanDesc sscan, ScanDirection direction, TupleTableSlot *slot)
 {
+	Assert(direction == ForwardScanDirection ||
+		   direction == BackwardScanDirection);
+
 	slot->tts_tableOid = RelationGetRelid(sscan->rs_rd);
 
 	/*
@@ -1096,6 +1099,9 @@ static inline bool
 table_scan_getnextslot_tidrange(TableScanDesc sscan, ScanDirection direction,
 								TupleTableSlot *slot)
 {
+	Assert(direction == ForwardScanDirection ||
+		   direction == BackwardScanDirection);
+
 	/* Ensure table_beginscan_tidrange() was used. */
 	Assert((sscan->rs_flags & SO_TYPE_TIDRANGESCAN) != 0);
 
diff --git a/src/test/regress/expected/hash_index.out b/src/test/regress/expected/hash_index.out
index a2036a1597..fde395a7b6 100644
--- a/src/test/regress/expected/hash_index.out
+++ b/src/test/regress/expected/hash_index.out
@@ -40,6 +40,33 @@ CREATE INDEX hash_name_index ON hash_name_heap USING hash (random name_ops);
 CREATE INDEX hash_txt_index ON hash_txt_heap USING hash (random text_ops);
 CREATE INDEX hash_f8_index ON hash_f8_heap USING hash (random float8_ops)
   WITH (fillfactor=60);
+CREATE OR REPLACE FUNCTION scan_dir(query text, scan_type text)
+RETURNS TEXT LANGUAGE plpgsql
+AS
+$$
+DECLARE
+  plan json;
+  node json;
+BEGIN
+  FOR plan IN
+    EXECUTE 'EXPLAIN (FORMAT ''json'') ' || query
+  LOOP
+    node := json_extract_path(plan, '0', 'Plan');
+    IF node->>'Node Type' = scan_type THEN
+      RETURN node->>'Scan Direction';
+    END IF;
+  END LOOP;
+  RETURN NULL;
+END;
+$$;
+-- Hash Indexes are unordered and thus only forward scan direction makes sense.
+SELECT 'Forward' = scan_dir(
+   $$ SELECT random FROM hash_i4_heap WHERE random = 1345971420; $$, 'Index Scan');
+ ?column? 
+----------
+ t
+(1 row)
+
 --
 -- Also try building functional, expressional, and partial indexes on
 -- tables that already contain data.
diff --git a/src/test/regress/sql/hash_index.sql b/src/test/regress/sql/hash_index.sql
index 527024f710..8a96f53612 100644
--- a/src/test/regress/sql/hash_index.sql
+++ b/src/test/regress/sql/hash_index.sql
@@ -53,6 +53,29 @@ CREATE INDEX hash_txt_index ON hash_txt_heap USING hash (random text_ops);
 CREATE INDEX hash_f8_index ON hash_f8_heap USING hash (random float8_ops)
   WITH (fillfactor=60);
 
+CREATE OR REPLACE FUNCTION scan_dir(query text, scan_type text)
+RETURNS TEXT LANGUAGE plpgsql
+AS
+$$
+DECLARE
+  plan json;
+  node json;
+BEGIN
+  FOR plan IN
+    EXECUTE 'EXPLAIN (FORMAT ''json'') ' || query
+  LOOP
+    node := json_extract_path(plan, '0', 'Plan');
+    IF node->>'Node Type' = scan_type THEN
+      RETURN node->>'Scan Direction';
+    END IF;
+  END LOOP;
+  RETURN NULL;
+END;
+$$;
+
+-- Hash Indexes are unordered and thus only forward scan direction makes sense.
+SELECT 'Forward' = scan_dir(
+   $$ SELECT random FROM hash_i4_heap WHERE random = 1345971420; $$, 'Index Scan');
 --
 -- Also try building functional, expressional, and partial indexes on
 -- tables that already contain data.
-- 
2.37.2

#7Tom Lane
tgl@sss.pgh.pa.us
In reply to: Melanie Plageman (#6)
Re: heapgettup() with NoMovementScanDirection unused in core?

Melanie Plageman <melanieplageman@gmail.com> writes:

I have written the patch to remove the unreachable code in
heapgettup_pagemode]().

A few thoughts ...

1. Do we really need quite so many Asserts? I'd kind of lean
to just having one, at some higher level of the executor.

2. I'm not sure if we want to do this:

-	direction = estate->es_direction;
-	/* flip direction if this is an overall backward scan */
-	if (ScanDirectionIsBackward(((IndexScan *) node->ss.ps.plan)->indexorderdir))
-	{
-		if (ScanDirectionIsForward(direction))
-			direction = BackwardScanDirection;
-		else if (ScanDirectionIsBackward(direction))
-			direction = ForwardScanDirection;
-	}
+	direction = estate->es_direction * ((IndexScan *) node->ss.ps.plan)->indexorderdir;

AFAIR, there is noplace today that depends on the exact encoding
of ForwardScanDirection and BackwardScanDirection, and I'm not
sure that we want to introduce such a dependency. If we do it
at least deserves a comment here, and you probably ought to adjust
the wishy-washy comment in sdir.h as well. Taking out the existing
comment explaining what this code is doing is not an improvement
either.

3. You didn't update the header comment for heapgettup, nor the
one in pathnodes.h for IndexPath.indexscandir.

4. I don't think the proposed test case is worth the cycles.

regards, tom lane

#8Melanie Plageman
melanieplageman@gmail.com
In reply to: Tom Lane (#7)
Re: heapgettup() with NoMovementScanDirection unused in core?

On Fri, Jan 27, 2023 at 05:05:16PM -0500, Tom Lane wrote:

Melanie Plageman <melanieplageman@gmail.com> writes:

I have written the patch to remove the unreachable code in
heapgettup_pagemode]().

A few thoughts ...

1. Do we really need quite so many Asserts? I'd kind of lean
to just having one, at some higher level of the executor.

Yes, perhaps I was a bit overzealous putting them in functions called
for every tuple.

I'm not sure where in the executor would make the most sense.
ExecInitSeqScan() comes to mind, but I'm not sure that covers all of the
desired cases.

2. I'm not sure if we want to do this:

-	direction = estate->es_direction;
-	/* flip direction if this is an overall backward scan */
-	if (ScanDirectionIsBackward(((IndexScan *) node->ss.ps.plan)->indexorderdir))
-	{
-		if (ScanDirectionIsForward(direction))
-			direction = BackwardScanDirection;
-		else if (ScanDirectionIsBackward(direction))
-			direction = ForwardScanDirection;
-	}
+	direction = estate->es_direction * ((IndexScan *) node->ss.ps.plan)->indexorderdir;

AFAIR, there is noplace today that depends on the exact encoding
of ForwardScanDirection and BackwardScanDirection, and I'm not
sure that we want to introduce such a dependency. If we do it
at least deserves a comment here, and you probably ought to adjust
the wishy-washy comment in sdir.h as well. Taking out the existing
comment explaining what this code is doing is not an improvement
either.

I think you mean the enum value when you say encoding? I actually
started using the ScanDirection value in the refactor of heapgettup()
and heapgettup_pagemode() which I proposed here [1]/messages/by-id/CAAKRu_ZJg_N7zHtWP+JoSY_hrce4+GKioL137Y2c2En-kuXQ7g@mail.gmail.com. Why wouldn't we
want to introduce such a dependency?

3. You didn't update the header comment for heapgettup, nor the
one in pathnodes.h for IndexPath.indexscandir.

Oops -- thanks for catching those!

4. I don't think the proposed test case is worth the cycles.

Just the one I wrote or any test case?

- Melanie

[1]: /messages/by-id/CAAKRu_ZJg_N7zHtWP+JoSY_hrce4+GKioL137Y2c2En-kuXQ7g@mail.gmail.com

#9Tom Lane
tgl@sss.pgh.pa.us
In reply to: Melanie Plageman (#8)
Re: heapgettup() with NoMovementScanDirection unused in core?

Melanie Plageman <melanieplageman@gmail.com> writes:

On Fri, Jan 27, 2023 at 05:05:16PM -0500, Tom Lane wrote:

AFAIR, there is noplace today that depends on the exact encoding
of ForwardScanDirection and BackwardScanDirection, and I'm not
sure that we want to introduce such a dependency.

I think you mean the enum value when you say encoding? I actually
started using the ScanDirection value in the refactor of heapgettup()
and heapgettup_pagemode() which I proposed here [1]. Why wouldn't we
want to introduce such a dependency?

It's just that in general, depending on the numeric values of an
enum isn't a great coding practice.

After thinking about it for awhile, I'd be happier if we added
something like this to sdir.h, and then used it rather than
directly depending on multiplication:

/*
* Determine the net effect of two direction specifications.
* This relies on having ForwardScanDirection = +1, BackwardScanDirection = -1,
* and will probably not do what you want if applied to any other values.
*/
#define CombineScanDirections(a, b) ((a) * (b))

The main thing this'd buy us is being able to grep for uses of the
trick. If it's written as just multiplication, good luck being
able to find what's depending on that, should you ever need to.

4. I don't think the proposed test case is worth the cycles.

Just the one I wrote or any test case?

I think that all this code is quite well-tested already, so I'm
not sure what's the point of adding another test for it.

regards, tom lane

#10David Rowley
dgrowleyml@gmail.com
In reply to: Tom Lane (#9)
Re: heapgettup() with NoMovementScanDirection unused in core?

On Sat, 28 Jan 2023 at 12:15, Tom Lane <tgl@sss.pgh.pa.us> wrote:

/*
* Determine the net effect of two direction specifications.
* This relies on having ForwardScanDirection = +1, BackwardScanDirection = -1,
* and will probably not do what you want if applied to any other values.
*/
#define CombineScanDirections(a, b) ((a) * (b))

The main thing this'd buy us is being able to grep for uses of the
trick. If it's written as just multiplication, good luck being
able to find what's depending on that, should you ever need to.

Yeah, I think the multiplication macro is a good way of doing it.
Having the definition of it close to the ScanDirection enum's
definition is likely a very good idea so that anyone adjusting the
enum values is more likely to notice that it'll cause an issue. A
small note on the enum declaration about the -1, +1 values being
exploited in various places might be a good idea too. I see v6-0006 in
[1]: /messages/by-id/CAAKRu_ZyiXwWS1WXSZneoy+sjoH_+F5UhO-1uFhyi-u0d6z+fA@mail.gmail.com

My personal preference would have been to call it
ScanDirectionCombine, so the naming is more aligned to the 4 other
macro names that start with ScanDirection in sdir.h, but I'm not going
to fuss over it.

David

[1]: /messages/by-id/CAAKRu_ZyiXwWS1WXSZneoy+sjoH_+F5UhO-1uFhyi-u0d6z+fA@mail.gmail.com

David

#11Tom Lane
tgl@sss.pgh.pa.us
In reply to: David Rowley (#10)
Re: heapgettup() with NoMovementScanDirection unused in core?

David Rowley <dgrowleyml@gmail.com> writes:

My personal preference would have been to call it
ScanDirectionCombine, so the naming is more aligned to the 4 other
macro names that start with ScanDirection in sdir.h, but I'm not going
to fuss over it.

No objection to that.

regards, tom lane

#12Melanie Plageman
melanieplageman@gmail.com
In reply to: David Rowley (#10)
1 attachment(s)
Re: heapgettup() with NoMovementScanDirection unused in core?

v2 attached

On Fri, Jan 27, 2023 at 6:28 PM David Rowley <dgrowleyml@gmail.com> wrote:

On Sat, 28 Jan 2023 at 12:15, Tom Lane <tgl@sss.pgh.pa.us> wrote:

/*
* Determine the net effect of two direction specifications.
* This relies on having ForwardScanDirection = +1, BackwardScanDirection = -1,
* and will probably not do what you want if applied to any other values.
*/
#define CombineScanDirections(a, b) ((a) * (b))

The main thing this'd buy us is being able to grep for uses of the
trick. If it's written as just multiplication, good luck being
able to find what's depending on that, should you ever need to.

Yeah, I think the multiplication macro is a good way of doing it.
Having the definition of it close to the ScanDirection enum's
definition is likely a very good idea so that anyone adjusting the
enum values is more likely to notice that it'll cause an issue. A
small note on the enum declaration about the -1, +1 values being
exploited in various places might be a good idea too. I see v6-0006 in
[1] further exploits this, so that's further reason to document that.

My personal preference would have been to call it
ScanDirectionCombine, so the naming is more aligned to the 4 other
macro names that start with ScanDirection in sdir.h, but I'm not going
to fuss over it.

I've gone with this macro name.
I've also updated comments Tom mentioned and removed the test.

As for the asserts, I was at a bit of a loss as to where to put an
assert which will make it clear that heapgettup() and
heapgettup_pagemode() do not handle NoMovementScanDirection but was
at a higher level of the executor. Do we not have to accommodate the
direction changing from tuple to tuple? If we don't expect the plan node
direction to change during execution, then why recalculate
estate->es_direction for each invocation of Index/SeqNext()?

As such, in this version I've put the asserts in heapgettup() and
heapgettup_pagemode().

I also realized that it doesn't really make sense to assert about the
index scan direction in ExecInitIndexOnlyScan() and ExecInitIndexScan()
-- so I've moved the assertion to planner when we make the index plan
from the path. I'm not sure if it is needed.

- Melanie

Attachments:

v2-0001-remove-nomovement-scandir.patchtext/x-patch; charset=US-ASCII; name=v2-0001-remove-nomovement-scandir.patchDownload
From d6aae3a73864aaaf84b4dc00ff299c2e8b4a5729 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 30 Jan 2023 09:49:54 -0500
Subject: [PATCH v2] remove nomovement scandir

---
 src/backend/access/heap/heapam.c         | 71 ++----------------------
 src/backend/commands/explain.c           |  3 -
 src/backend/executor/nodeIndexonlyscan.c | 11 +---
 src/backend/executor/nodeIndexscan.c     | 11 +---
 src/backend/optimizer/path/indxpath.c    |  8 +--
 src/backend/optimizer/plan/createplan.c  | 14 +++++
 src/backend/optimizer/util/pathnode.c    |  5 +-
 src/include/access/sdir.h                | 11 +++-
 src/include/nodes/pathnodes.h            |  6 +-
 9 files changed, 41 insertions(+), 99 deletions(-)

diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e6024a980b..10aaeb14aa 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -490,9 +490,6 @@ heapgetpage(TableScanDesc sscan, BlockNumber block)
  *		tuple as indicated by "dir"; return the next tuple in scan->rs_ctup,
  *		or set scan->rs_ctup.t_data = NULL if no more tuples.
  *
- * dir == NoMovementScanDirection means "re-fetch the tuple indicated
- * by scan->rs_ctup".
- *
  * Note: the reason nkeys/key are passed separately, even though they are
  * kept in the scan descriptor, is that the caller may not want us to check
  * the scankeys.
@@ -523,6 +520,8 @@ heapgettup(HeapScanDesc scan,
 	int			linesleft;
 	ItemId		lpp;
 
+	Assert(dir != NoMovementScanDirection);
+
 	/*
 	 * calculate next starting lineoff, given scan direction
 	 */
@@ -583,7 +582,7 @@ heapgettup(HeapScanDesc scan,
 
 		linesleft = lines - lineoff + 1;
 	}
-	else if (backward)
+	else
 	{
 		/* backward parallel scan not supported */
 		Assert(scan->rs_base.rs_parallel == NULL);
@@ -653,34 +652,6 @@ heapgettup(HeapScanDesc scan,
 
 		linesleft = lineoff;
 	}
-	else
-	{
-		/*
-		 * ``no movement'' scan direction: refetch prior tuple
-		 */
-		if (!scan->rs_inited)
-		{
-			Assert(!BufferIsValid(scan->rs_cbuf));
-			tuple->t_data = NULL;
-			return;
-		}
-
-		block = ItemPointerGetBlockNumber(&(tuple->t_self));
-		if (block != scan->rs_cblock)
-			heapgetpage((TableScanDesc) scan, block);
-
-		/* Since the tuple was previously fetched, needn't lock page here */
-		page = BufferGetPage(scan->rs_cbuf);
-		TestForOldSnapshot(snapshot, scan->rs_base.rs_rd, page);
-		lineoff = ItemPointerGetOffsetNumber(&(tuple->t_self));
-		lpp = PageGetItemId(page, lineoff);
-		Assert(ItemIdIsNormal(lpp));
-
-		tuple->t_data = (HeapTupleHeader) PageGetItem(page, lpp);
-		tuple->t_len = ItemIdGetLength(lpp);
-
-		return;
-	}
 
 	/*
 	 * advance the scan until we find a qualifying tuple or run out of stuff
@@ -861,6 +832,8 @@ heapgettup_pagemode(HeapScanDesc scan,
 	int			linesleft;
 	ItemId		lpp;
 
+	Assert(dir != NoMovementScanDirection);
+
 	/*
 	 * calculate next starting lineindex, given scan direction
 	 */
@@ -918,7 +891,7 @@ heapgettup_pagemode(HeapScanDesc scan,
 
 		linesleft = lines - lineindex;
 	}
-	else if (backward)
+	else
 	{
 		/* backward parallel scan not supported */
 		Assert(scan->rs_base.rs_parallel == NULL);
@@ -978,38 +951,6 @@ heapgettup_pagemode(HeapScanDesc scan,
 
 		linesleft = lineindex + 1;
 	}
-	else
-	{
-		/*
-		 * ``no movement'' scan direction: refetch prior tuple
-		 */
-		if (!scan->rs_inited)
-		{
-			Assert(!BufferIsValid(scan->rs_cbuf));
-			tuple->t_data = NULL;
-			return;
-		}
-
-		block = ItemPointerGetBlockNumber(&(tuple->t_self));
-		if (block != scan->rs_cblock)
-			heapgetpage((TableScanDesc) scan, block);
-
-		/* Since the tuple was previously fetched, needn't lock page here */
-		page = BufferGetPage(scan->rs_cbuf);
-		TestForOldSnapshot(scan->rs_base.rs_snapshot, scan->rs_base.rs_rd, page);
-		lineoff = ItemPointerGetOffsetNumber(&(tuple->t_self));
-		lpp = PageGetItemId(page, lineoff);
-		Assert(ItemIdIsNormal(lpp));
-
-		tuple->t_data = (HeapTupleHeader) PageGetItem(page, lpp);
-		tuple->t_len = ItemIdGetLength(lpp);
-
-		/* check that rs_cindex is in sync */
-		Assert(scan->rs_cindex < scan->rs_ntuples);
-		Assert(lineoff == scan->rs_vistuples[scan->rs_cindex]);
-
-		return;
-	}
 
 	/*
 	 * advance the scan until we find a qualifying tuple or run out of stuff
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index a0311ce9dc..d38311fa7e 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -3746,9 +3746,6 @@ ExplainIndexScanDetails(Oid indexid, ScanDirection indexorderdir,
 			case BackwardScanDirection:
 				scandir = "Backward";
 				break;
-			case NoMovementScanDirection:
-				scandir = "NoMovement";
-				break;
 			case ForwardScanDirection:
 				scandir = "Forward";
 				break;
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 8c7da9ee60..e75b3bb8e1 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -70,15 +70,10 @@ IndexOnlyNext(IndexOnlyScanState *node)
 	 * extract necessary information from index scan node
 	 */
 	estate = node->ss.ps.state;
-	direction = estate->es_direction;
+
 	/* flip direction if this is an overall backward scan */
-	if (ScanDirectionIsBackward(((IndexOnlyScan *) node->ss.ps.plan)->indexorderdir))
-	{
-		if (ScanDirectionIsForward(direction))
-			direction = BackwardScanDirection;
-		else if (ScanDirectionIsBackward(direction))
-			direction = ForwardScanDirection;
-	}
+	direction = ScanDirectionCombine(estate->es_direction,
+									 ((IndexOnlyScan *) node->ss.ps.plan)->indexorderdir);
 	scandesc = node->ioss_ScanDesc;
 	econtext = node->ss.ps.ps_ExprContext;
 	slot = node->ss.ss_ScanTupleSlot;
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index f1ced9ff0f..1c1c558b22 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -90,15 +90,10 @@ IndexNext(IndexScanState *node)
 	 * extract necessary information from index scan node
 	 */
 	estate = node->ss.ps.state;
-	direction = estate->es_direction;
+
 	/* flip direction if this is an overall backward scan */
-	if (ScanDirectionIsBackward(((IndexScan *) node->ss.ps.plan)->indexorderdir))
-	{
-		if (ScanDirectionIsForward(direction))
-			direction = BackwardScanDirection;
-		else if (ScanDirectionIsBackward(direction))
-			direction = ForwardScanDirection;
-	}
+	direction = ScanDirectionCombine(estate->es_direction,
+									 ((IndexScan *) node->ss.ps.plan)->indexorderdir);
 	scandesc = node->iss_ScanDesc;
 	econtext = node->ss.ps.ps_ExprContext;
 	slot = node->ss.ss_ScanTupleSlot;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index e13c8f1914..751fa2edfe 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -1015,9 +1015,7 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
 								  orderbyclauses,
 								  orderbyclausecols,
 								  useful_pathkeys,
-								  index_is_ordered ?
-								  ForwardScanDirection :
-								  NoMovementScanDirection,
+								  ForwardScanDirection,
 								  index_only_scan,
 								  outer_relids,
 								  loop_count,
@@ -1037,9 +1035,7 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
 									  orderbyclauses,
 									  orderbyclausecols,
 									  useful_pathkeys,
-									  index_is_ordered ?
-									  ForwardScanDirection :
-									  NoMovementScanDirection,
+									  ForwardScanDirection,
 									  index_only_scan,
 									  outer_relids,
 									  loop_count,
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index cd68942af0..1302d25e54 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -5512,6 +5512,13 @@ make_indexscan(List *qptlist,
 	IndexScan  *node = makeNode(IndexScan);
 	Plan	   *plan = &node->scan.plan;
 
+	/*
+	 * Only forward and backward index scan directions are meaningful to the
+	 * executor. Unordered indexes will always have ForwardScanDirection.
+	 */
+	Assert(indexscandir == ForwardScanDirection ||
+		   indexscandir == BackwardScanDirection);
+
 	plan->targetlist = qptlist;
 	plan->qual = qpqual;
 	plan->lefttree = NULL;
@@ -5542,6 +5549,13 @@ make_indexonlyscan(List *qptlist,
 	IndexOnlyScan *node = makeNode(IndexOnlyScan);
 	Plan	   *plan = &node->scan.plan;
 
+	/*
+	 * Only forward and backward index scan directions are meaningful to the
+	 * executor. Unordered indexes will always have ForwardScanDirection.
+	 */
+	Assert(indexscandir == ForwardScanDirection ||
+		   indexscandir == BackwardScanDirection);
+
 	plan->targetlist = qptlist;
 	plan->qual = qpqual;
 	plan->lefttree = NULL;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 4478036bb6..dd5236682f 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -982,9 +982,8 @@ create_samplescan_path(PlannerInfo *root, RelOptInfo *rel, Relids required_outer
  * 'indexorderbycols' is an integer list of index column numbers (zero based)
  *			the ordering operators can be used with.
  * 'pathkeys' describes the ordering of the path.
- * 'indexscandir' is ForwardScanDirection or BackwardScanDirection
- *			for an ordered index, or NoMovementScanDirection for
- *			an unordered index.
+ * 'indexscandir' is either ForwardScanDirection or BackwardScanDirection.
+ * 			Unordered index types need not support BackwardScanDirection.
  * 'indexonly' is true if an index-only scan is wanted.
  * 'required_outer' is the set of outer relids for a parameterized path.
  * 'loop_count' is the number of repetitions of the indexscan to factor into
diff --git a/src/include/access/sdir.h b/src/include/access/sdir.h
index 16cb06c709..bd79d977ec 100644
--- a/src/include/access/sdir.h
+++ b/src/include/access/sdir.h
@@ -16,8 +16,8 @@
 
 
 /*
- * ScanDirection was an int8 for no apparent reason. I kept the original
- * values because I'm not sure if I'll break anything otherwise.  -ay 2/95
+ * These enum values were originally int8 values. Using -1, 0, and 1 as their
+ * values conveniently mirrors their semantic value when used during execution.
  */
 typedef enum ScanDirection
 {
@@ -26,6 +26,13 @@ typedef enum ScanDirection
 	ForwardScanDirection = 1
 } ScanDirection;
 
+/*
+ * Determine the net effect of two direction specifications.
+ * This relies on having ForwardScanDirection = +1, BackwardScanDirection = -1,
+ * and will probably not do what you want if applied to any other values.
+ */
+#define ScanDirectionCombine(a, b)  ((a) * (b))
+
 /*
  * ScanDirectionIsValid
  *		True iff scan direction is valid.
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 2d1d8f4bcd..cdb2a06409 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1611,10 +1611,8 @@ typedef struct Path
  * 'indexscandir' is one of:
  *		ForwardScanDirection: forward scan of an ordered index
  *		BackwardScanDirection: backward scan of an ordered index
- *		NoMovementScanDirection: scan of an unordered index, or don't care
- * (The executor doesn't care whether it gets ForwardScanDirection or
- * NoMovementScanDirection for an indexscan, but the planner wants to
- * distinguish ordered from unordered indexes for building pathkeys.)
+ * NoMovementScanDirection for indexscandir is meaningless for the executor, so
+ * unordered indexes will always set ForwardScanDirection.
  *
  * 'indextotalcost' and 'indexselectivity' are saved in the IndexPath so that
  * we need not recompute them when considering using the same index in a
-- 
2.37.2

#13David Rowley
dgrowleyml@gmail.com
In reply to: Melanie Plageman (#12)
Re: heapgettup() with NoMovementScanDirection unused in core?

On Tue, 31 Jan 2023 at 09:57, Melanie Plageman
<melanieplageman@gmail.com> wrote:

As for the asserts, I was at a bit of a loss as to where to put an
assert which will make it clear that heapgettup() and
heapgettup_pagemode() do not handle NoMovementScanDirection but was
at a higher level of the executor.

My thoughts were that we might want to put them
table_scan_getnextslot() and table_scan_getnextslot_tidrange(). My
rationale for that was that it makes it more clear to table AM devs
that they don't need to handle NoMovementScanDirection.

Do we not have to accommodate the
direction changing from tuple to tuple? If we don't expect the plan node
direction to change during execution, then why recalculate
estate->es_direction for each invocation of Index/SeqNext()?

Yeah, this needs to be handled. FETCH can fetch forwards or backwards
from a cursor. The code you have looks fine to me.

As such, in this version I've put the asserts in heapgettup() and
heapgettup_pagemode().

I also realized that it doesn't really make sense to assert about the
index scan direction in ExecInitIndexOnlyScan() and ExecInitIndexScan()
-- so I've moved the assertion to planner when we make the index plan
from the path. I'm not sure if it is needed.

That's probably slightly better.

The only thing I really have on this is my thoughts on the Asserts
going in tableam.h plus the following comment:

/*
* These enum values were originally int8 values. Using -1, 0, and 1 as their
* values conveniently mirrors their semantic value when used during execution.
*/

I don't really see any reason to keep the historical note here. I
think something like the following might be better:

/*
* Defines the direction for scanning a table or an index. Scans are never
* invoked using NoMovementScanDirectionScans. For convenience, we use the
* values -1 and 1 for backward and forward scans. This allows us to perform
* a few mathematical tricks such as what is done in ScanDirectionCombine.
*/

Also, a nitpick around the inconsistency with the Asserts. In
make_indexscan() and make_indexonlyscan() you're checking you're
getting a forward and backward value, but in heapgettup() and
heapgettup_pagemode() you're checking you don't get
NoMovementScanDirection. I think the != NoMovementScanDirection is
fine for both cases.

Both can be easily fixed, so no need to submit another patch as far as
I'm concerned.

I'll leave this til tomorrow in case Tom wants to have another look too.

David

#14Melanie Plageman
melanieplageman@gmail.com
In reply to: David Rowley (#13)
Re: heapgettup() with NoMovementScanDirection unused in core?

On Tue, Jan 31, 2023 at 11:46:05PM +1300, David Rowley wrote:

On Tue, 31 Jan 2023 at 09:57, Melanie Plageman
<melanieplageman@gmail.com> wrote:

As for the asserts, I was at a bit of a loss as to where to put an
assert which will make it clear that heapgettup() and
heapgettup_pagemode() do not handle NoMovementScanDirection but was
at a higher level of the executor.

My thoughts were that we might want to put them
table_scan_getnextslot() and table_scan_getnextslot_tidrange(). My
rationale for that was that it makes it more clear to table AM devs
that they don't need to handle NoMovementScanDirection.

I previously had the asserts here, but I thought perhaps we shouldn't
restrict table AMs from using NoMovementScanDirection in whatever way
they'd like. We care about protecting heapgettup() and
heapgettup_pagemode(). We could put a comment in the table AM API about
NoMovementScanDirection not necessarily making sense for a next() type
function and informing table AMs that they need not support it.

Do we not have to accommodate the
direction changing from tuple to tuple? If we don't expect the plan node
direction to change during execution, then why recalculate
estate->es_direction for each invocation of Index/SeqNext()?

Yeah, this needs to be handled. FETCH can fetch forwards or backwards
from a cursor. The code you have looks fine to me.

As such, in this version I've put the asserts in heapgettup() and
heapgettup_pagemode().

I also realized that it doesn't really make sense to assert about the
index scan direction in ExecInitIndexOnlyScan() and ExecInitIndexScan()
-- so I've moved the assertion to planner when we make the index plan
from the path. I'm not sure if it is needed.

That's probably slightly better.

The only thing I really have on this is my thoughts on the Asserts
going in tableam.h plus the following comment:

/*
* These enum values were originally int8 values. Using -1, 0, and 1 as their
* values conveniently mirrors their semantic value when used during execution.
*/

I don't really see any reason to keep the historical note here. I
think something like the following might be better:

/*
* Defines the direction for scanning a table or an index. Scans are never
* invoked using NoMovementScanDirectionScans. For convenience, we use the
* values -1 and 1 for backward and forward scans. This allows us to perform
* a few mathematical tricks such as what is done in ScanDirectionCombine.
*/

This comment looks good to me.

Also, a nitpick around the inconsistency with the Asserts. In
make_indexscan() and make_indexonlyscan() you're checking you're
getting a forward and backward value, but in heapgettup() and
heapgettup_pagemode() you're checking you don't get
NoMovementScanDirection. I think the != NoMovementScanDirection is
fine for both cases.

Yes, I thought about it being weird that they are different. Perhaps we
should check in both places that it is forward or backward. In
heapgettup[_pagemode()] there is if/else -- so if the assert is only for
NoMovementScanDirection and a new scan direction is added, it would fall
through to the else.

In planner, it is not that we are not "handling" NoMovementScanDirection
(like in heapgettup) but rather that we are only passing Forward and
Backward scan directions when creating the path nodes, so the Assert
would be mainly to remind the developer that if they are creating a plan
with a different scan direction that they should be intentional about
it.

So, I would favor having both asserts check that the direction is one of
forward or backward.

Both can be easily fixed, so no need to submit another patch as far as
I'm concerned.

I realized I forgot a commit message in the second version. Patch v1 has
one.

- Melanie

#15David Rowley
dgrowleyml@gmail.com
In reply to: Melanie Plageman (#14)
Re: heapgettup() with NoMovementScanDirection unused in core?

On Wed, 1 Feb 2023 at 03:02, Melanie Plageman <melanieplageman@gmail.com> wrote:

On Tue, Jan 31, 2023 at 11:46:05PM +1300, David Rowley wrote:

My thoughts were that we might want to put them
table_scan_getnextslot() and table_scan_getnextslot_tidrange(). My
rationale for that was that it makes it more clear to table AM devs
that they don't need to handle NoMovementScanDirection.

I previously had the asserts here, but I thought perhaps we shouldn't
restrict table AMs from using NoMovementScanDirection in whatever way
they'd like. We care about protecting heapgettup() and
heapgettup_pagemode(). We could put a comment in the table AM API about
NoMovementScanDirection not necessarily making sense for a next() type
function and informing table AMs that they need not support it.

hmm, but the recent discovery is that we'll never call ExecutePlan()
with NoMovementScanDirection, so what exactly is going to call
table_scan_getnextslot() and table_scan_getnextslot_tidrange() with
NoMovementScanDirection?

So, I would favor having both asserts check that the direction is one of
forward or backward.

That sounds fine to me.

David

#16Tom Lane
tgl@sss.pgh.pa.us
In reply to: David Rowley (#15)
Re: heapgettup() with NoMovementScanDirection unused in core?

David Rowley <dgrowleyml@gmail.com> writes:

On Wed, 1 Feb 2023 at 03:02, Melanie Plageman <melanieplageman@gmail.com> wrote:

I previously had the asserts here, but I thought perhaps we shouldn't
restrict table AMs from using NoMovementScanDirection in whatever way
they'd like. We care about protecting heapgettup() and
heapgettup_pagemode(). We could put a comment in the table AM API about
NoMovementScanDirection not necessarily making sense for a next() type
function and informing table AMs that they need not support it.

hmm, but the recent discovery is that we'll never call ExecutePlan()
with NoMovementScanDirection, so what exactly is going to call
table_scan_getnextslot() and table_scan_getnextslot_tidrange() with
NoMovementScanDirection?

Yeah. This is not an AM-local API.

regards, tom lane

#17David Rowley
dgrowleyml@gmail.com
In reply to: Melanie Plageman (#14)
Re: heapgettup() with NoMovementScanDirection unused in core?

On Wed, 1 Feb 2023 at 03:02, Melanie Plageman <melanieplageman@gmail.com> wrote:

On Tue, Jan 31, 2023 at 11:46:05PM +1300, David Rowley wrote:

Both can be easily fixed, so no need to submit another patch as far as
I'm concerned.

I realized I forgot a commit message in the second version. Patch v1 has
one.

I made a couple of other adjustments to the Asserts and comments and
pushed the result.

On further looking, I felt the Assert was better off done in
create_indexscan_plan() rather than make_index[only]scan(). I also put
the asserts in tableam.h and removed the heapam.c ones. The rest was
just comment adjustments.

David