GSoC 2017: weekly progress reports (week 4) and patch for hash index

Started by Shubham Baraiover 8 years ago29 messages
#1Shubham Barai
shubhambaraiss@gmail.com
1 attachment(s)

Project: Explicitly support predicate locks in index AMs besides b-tree

Hi,

During this week, I continued my work on predicate locking in the hash
index and created a patch for it. As I have written in my proposal for the
hash index, every scan operation acquires a predicate lock on the primary
page of the bucket.
And hash indexes are used for equality comparison. So, this can still
generate false positives when a scan operation and an insert operation are
trying to access the same bucket but are looking for different tuples.
Let's see that with an example.

setup:

create table hash_tbl(id int4, p integer);

create index hash_pointidx on hash_tbl using hash(p);

insert into hash_tbl (id, p)
select g, g*2 from generate_series(1, 10000000) g;

read operation:
select * from hash_tbl where p=78988658;

insert operation:
insert into hash_tbl values(99999999, 546789888);

If these two hash keys (78988658 and 546789888) mapped to the same bucket,
this will result in false serialization failure.
Please correct me if this assumption about false positives is wrong.

In summary, I have done following things in this week.

1) found appropriate places in the hash index AM to insert calls to
existing functions (PredicateLockPage(), CheckForSerializableConflictIn()
...etc)

2) created tests to check serialization failures and false positives

3) tested my patch on the current head

Regards,

Shubham

<https://mailtrack.io/&gt; Sent with Mailtrack
<https://mailtrack.io/install?source=signature&amp;lang=en&amp;referral=shubhambaraiss@gmail.com&amp;idSignature=22&gt;

Attachments:

Predicate-Locking-in-hash-index_3.patchapplication/octet-stream; name=Predicate-Locking-in-hash-index_3.patchDownload
From 3e1bc6deb4a238dd0e5a56806b2f85490ff4b70e Mon Sep 17 00:00:00 2001
From: shubhambaraiss <you@example.com>
Date: Sun, 25 Jun 2017 04:44:41 +0530
Subject: [PATCH] Predicate Locking in hash index

---
 src/backend/access/hash/hash.c                   |   2 +-
 src/backend/access/hash/hashinsert.c             |   3 +
 src/backend/access/hash/hashpage.c               |   9 +
 src/backend/access/hash/hashsearch.c             |   2 +
 src/backend/storage/lmgr/README-SSI              |   4 +
 src/test/isolation/expected/predicate-hash-2.out | 321 +++++++++++++++++++++
 src/test/isolation/expected/predicate-hash.out   | 339 +++++++++++++++++++++++
 src/test/isolation/isolation_schedule            |   2 +
 src/test/isolation/specs/predicate-hash-2.spec   |  44 +++
 src/test/isolation/specs/predicate-hash.spec     |  44 +++
 10 files changed, 769 insertions(+), 1 deletion(-)
 create mode 100644 src/test/isolation/expected/predicate-hash-2.out
 create mode 100644 src/test/isolation/expected/predicate-hash.out
 create mode 100644 src/test/isolation/specs/predicate-hash-2.spec
 create mode 100644 src/test/isolation/specs/predicate-hash.spec

diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index d89c192..fa4d4d6 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -68,7 +68,7 @@ hashhandler(PG_FUNCTION_ARGS)
 	amroutine->amsearchnulls = false;
 	amroutine->amstorage = false;
 	amroutine->amclusterable = false;
-	amroutine->ampredlocks = false;
+	amroutine->ampredlocks = true;
 	amroutine->amcanparallel = false;
 	amroutine->amkeytype = INT4OID;
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index dc08db9..cb7fba5 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -17,6 +17,7 @@
 
 #include "access/hash.h"
 #include "access/hash_xlog.h"
+#include "storage/predicate.h"
 #include "access/heapam.h"
 #include "miscadmin.h"
 #include "utils/rel.h"
@@ -88,6 +89,8 @@ restart_insert:
 										  &usedmetap);
 	Assert(usedmetap != NULL);
 
+	CheckForSerializableConflictIn(rel, NULL, buf);
+
 	/* remember the primary bucket buffer to release the pin on it at end. */
 	bucket_buf = buf;
 
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index 1cb18a7..2a950e1 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -33,6 +33,7 @@
 #include "miscadmin.h"
 #include "storage/lmgr.h"
 #include "storage/smgr.h"
+#include "storage/predicate.h"
 
 
 static bool _hash_alloc_buckets(Relation rel, BlockNumber firstblock,
@@ -947,6 +948,10 @@ restart_expand:
 					  buf_oblkno, buf_nblkno, NULL,
 					  maxbucket, highmask, lowmask);
 
+	PredicateLockPageSplit(rel,
+				BufferGetBlockNumber(buf_oblkno),
+				BufferGetBlockNumber(buf_nblkno));
+
 	/* all done, now release the locks and pins on primary buckets. */
 	_hash_relbuf(rel, buf_oblkno);
 	_hash_relbuf(rel, buf_nblkno);
@@ -1425,6 +1430,10 @@ _hash_finish_split(Relation rel, Buffer metabuf, Buffer obuf, Bucket obucket,
 					  nbucket, obuf, bucket_nbuf, tidhtab,
 					  maxbucket, highmask, lowmask);
 
+	PredicateLockPageSplit(rel,
+				BufferGetBlockNumber(obuf),
+				BufferGetBlockNumber(bucket_nbuf));
+
 	_hash_relbuf(rel, bucket_nbuf);
 	LockBuffer(obuf, BUFFER_LOCK_UNLOCK);
 	hash_destroy(tidhtab);
diff --git a/src/backend/access/hash/hashsearch.c b/src/backend/access/hash/hashsearch.c
index 3e461ad..3aef09d 100644
--- a/src/backend/access/hash/hashsearch.c
+++ b/src/backend/access/hash/hashsearch.c
@@ -16,6 +16,7 @@
 
 #include "access/hash.h"
 #include "access/relscan.h"
+#include "storage/predicate.h"
 #include "miscadmin.h"
 #include "pgstat.h"
 #include "utils/rel.h"
@@ -284,6 +285,7 @@ _hash_first(IndexScanDesc scan, ScanDirection dir)
 	so->hashso_sk_hash = hashkey;
 
 	buf = _hash_getbucketbuf_from_hashkey(rel, hashkey, HASH_READ, NULL);
+	PredicateLockPage(rel, BufferGetBlockNumber(buf), scan->xs_snapshot);
 	page = BufferGetPage(buf);
 	TestForOldSnapshot(scan->xs_snapshot, rel, page);
 	opaque = (HashPageOpaque) PageGetSpecialPointer(page);
diff --git a/src/backend/storage/lmgr/README-SSI b/src/backend/storage/lmgr/README-SSI
index a9dc01f..6e9cbba 100644
--- a/src/backend/storage/lmgr/README-SSI
+++ b/src/backend/storage/lmgr/README-SSI
@@ -379,6 +379,10 @@ level during a GiST search. An index insert at the leaf level can
 then be trusted to ripple up to all levels and locations where
 conflicting predicate locks may exist.
 
+    * Hash index searches acquire predicate locks on the primary
+page of a bucket. During a bucket split, a predicate lock is copied from
+the primary page of an old bucket to the primary page of a new bucket.
+
     * The effects of page splits, overflows, consolidations, and
 removals must be carefully reviewed to ensure that predicate locks
 aren't "lost" during those operations, or kept with pages which could
diff --git a/src/test/isolation/expected/predicate-hash-2.out b/src/test/isolation/expected/predicate-hash-2.out
new file mode 100644
index 0000000..2b08278
--- /dev/null
+++ b/src/test/isolation/expected/predicate-hash-2.out
@@ -0,0 +1,321 @@
+Parsed test spec with 2 sessions
+
+starting permutation: rxy1 wx1 c1 rxy2 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step c1: commit;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step c2: commit;
+
+starting permutation: rxy1 wx1 rxy2 c1 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step c2: commit;
+
+starting permutation: rxy1 wx1 rxy2 wy2 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy1 wx1 rxy2 wy2 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy1 rxy2 wx1 c1 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step c2: commit;
+
+starting permutation: rxy1 rxy2 wx1 wy2 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy1 rxy2 wx1 wy2 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy1 rxy2 wy2 wx1 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy1 rxy2 wy2 wx1 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy1 rxy2 wy2 c2 wx1 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step c1: commit;
+
+starting permutation: rxy2 rxy1 wx1 c1 wy2 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step c2: commit;
+
+starting permutation: rxy2 rxy1 wx1 wy2 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy2 rxy1 wx1 wy2 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy2 rxy1 wy2 wx1 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy2 rxy1 wy2 wx1 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy2 rxy1 wy2 c2 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step c1: commit;
+
+starting permutation: rxy2 wy2 rxy1 wx1 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy2 wy2 rxy1 wx1 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy2 wy2 rxy1 c2 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step c1: commit;
+
+starting permutation: rxy2 wy2 c2 rxy1 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g;
+step c2: commit;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g;
+step c1: commit;
diff --git a/src/test/isolation/expected/predicate-hash.out b/src/test/isolation/expected/predicate-hash.out
new file mode 100644
index 0000000..20c0874
--- /dev/null
+++ b/src/test/isolation/expected/predicate-hash.out
@@ -0,0 +1,339 @@
+Parsed test spec with 2 sessions
+
+starting permutation: rxy1 wx1 c1 rxy2 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step c1: commit;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+600            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step c2: commit;
+
+starting permutation: rxy1 wx1 rxy2 c1 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy1 wx1 rxy2 wy2 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 wx1 rxy2 wy2 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wx1 c1 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy1 rxy2 wx1 wy2 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wx1 wy2 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 wx1 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 wx1 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 c2 wx1 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy2 rxy1 wx1 c1 wy2 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy2 rxy1 wx1 wy2 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wx1 wy2 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 wx1 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 wx1 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 c2 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy2 wy2 rxy1 wx1 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 wy2 rxy1 wx1 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 wy2 rxy1 c2 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy2 wy2 c2 rxy1 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g;
+step c2: commit;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+400            
+step wx1: insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g;
+step c1: commit;
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 32c965b..284b87a 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -62,3 +62,5 @@ test: sequence-ddl
 test: async-notify
 test: vacuum-reltuples
 test: timeouts
+test: predicate-hash
+test: predicate-hash-2
diff --git a/src/test/isolation/specs/predicate-hash-2.spec b/src/test/isolation/specs/predicate-hash-2.spec
new file mode 100644
index 0000000..51c4af6
--- /dev/null
+++ b/src/test/isolation/specs/predicate-hash-2.spec
@@ -0,0 +1,44 @@
+# Test for predicate locking in hash index
+#
+# Test to check false positives
+#
+# Queries are written in such a way that an index scan(from one transaction) and an index insert(from another transaction) will try to access different buckets of the index.
+
+
+setup
+{
+ create table hash_tbl as select g*10+i id, g*10 p
+ from  generate_series(1,4)g, generate_series(1,10)i;
+ create index hash_pointidx on hash_tbl using hash(p);
+}
+
+teardown
+{
+ DROP TABLE hash_tbl;
+}
+
+
+session "s1"
+setup		{ 
+		  BEGIN ISOLATION LEVEL SERIALIZABLE;
+		  set enable_seqscan=off;
+		  set enable_bitmapscan=off;
+		  set enable_indexonlyscan=on;
+		}
+step "rxy1"	{ select sum(p) from hash_tbl where p=20; }
+step "wx1"	{ insert into hash_tbl (id, p)
+                  select g, 50 from generate_series(51, 60) g; }
+step "c1"	{ commit; }
+
+
+session "s2"
+setup		{ 
+		  BEGIN ISOLATION LEVEL SERIALIZABLE;
+		  set enable_seqscan=off;
+		  set enable_bitmapscan=off;
+		  set enable_indexonlyscan=on;
+		}
+step "rxy2"	{ select sum(p) from hash_tbl where p=30; }
+step "wy2"	{ insert into hash_tbl (id, p)
+                  select g, 60 from generate_series(61, 70) g; }
+step "c2"	{ commit; }
diff --git a/src/test/isolation/specs/predicate-hash.spec b/src/test/isolation/specs/predicate-hash.spec
new file mode 100644
index 0000000..7af4b3f
--- /dev/null
+++ b/src/test/isolation/specs/predicate-hash.spec
@@ -0,0 +1,44 @@
+# Test for predicate locking in hash index
+#
+# Test to verify serialization failures
+#
+# Queries are written in such a way that an index scan(from one transaction) and an index insert(from another transaction) will try to access the same bucket of the index.
+
+
+setup
+{
+ create table hash_tbl as select g*10+i id, g*10 p
+ from  generate_series(1,4)g, generate_series(1,10)i;
+ create index hash_pointidx on hash_tbl using hash(p);
+}
+
+teardown
+{
+ DROP TABLE hash_tbl;
+}
+
+
+session "s1"
+setup		{ 
+		  BEGIN ISOLATION LEVEL SERIALIZABLE;
+		  set enable_seqscan=off;
+		  set enable_bitmapscan=off;
+		  set enable_indexonlyscan=on;
+		}
+step "rxy1"	{ select sum(p) from hash_tbl where p=20; }
+step "wx1"	{ insert into hash_tbl (id, p)
+                  select g, 30 from generate_series(51, 60) g; }
+step "c1"	{ commit; }
+
+
+session "s2"
+setup		{ 
+		  BEGIN ISOLATION LEVEL SERIALIZABLE;
+		  set enable_seqscan=off;
+		  set enable_bitmapscan=off;
+		  set enable_indexonlyscan=on;
+		}
+step "rxy2"	{ select sum(p) from hash_tbl where p=30; }
+step "wy2"	{ insert into hash_tbl (id, p)
+                  select g, 20 from generate_series(61, 70) g; }
+step "c2"	{ commit; }
-- 
1.9.1

#2Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Shubham Barai (#1)
Re: GSoC 2017: weekly progress reports (week 4) and patch for hash index

Hi Shubham,

On Tue, Jun 27, 2017 at 9:21 PM, Shubham Barai <shubhambaraiss@gmail.com> wrote:

If these two hash keys (78988658 and 546789888) mapped to the same bucket, this will result in false serialization failure.
Please correct me if this assumption about false positives is wrong.

I wonder if there is an opportunity to use computed hash values
directly in predicate lock tags, rather than doing it on the basis of
buckets. Perhaps I'm missing something important about the way that
locks need to escalate that would prevent that from working.

3) tested my patch on the current head

This no longer applies, but it's in "Needs review" status in the
Commitfest. Could you please post a rebased version?

--
Thomas Munro
http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Shubham Barai
shubhambaraiss@gmail.com
In reply to: Thomas Munro (#2)
1 attachment(s)
Re: GSoC 2017: weekly progress reports (week 4) and patch for hash index

Hi Thomas,

I have attached the rebased version of patch here.

Kind Regards,
Shubham

On 8 September 2017 at 06:37, Thomas Munro <thomas.munro@enterprisedb.com>
wrote:

Show quoted text

Hi Shubham,

On Tue, Jun 27, 2017 at 9:21 PM, Shubham Barai <shubhambaraiss@gmail.com>
wrote:

If these two hash keys (78988658 and 546789888) mapped to the same

bucket, this will result in false serialization failure.

Please correct me if this assumption about false positives is wrong.

I wonder if there is an opportunity to use computed hash values
directly in predicate lock tags, rather than doing it on the basis of
buckets. Perhaps I'm missing something important about the way that
locks need to escalate that would prevent that from working.

3) tested my patch on the current head

This no longer applies, but it's in "Needs review" status in the
Commitfest. Could you please post a rebased version?

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

Predicate-locking-in-hash-index_4.patchapplication/octet-stream; name=Predicate-locking-in-hash-index_4.patchDownload
From 8ccfad5af8291d5279bf0dffe240856143d5993c Mon Sep 17 00:00:00 2001
From: shubhambaraiss <you@example.com>
Date: Mon, 25 Sep 2017 17:31:52 +0530
Subject: [PATCH] Predicate locking in hash index

---
 src/backend/access/hash/hash.c                   |   2 +-
 src/backend/access/hash/hashinsert.c             |   3 +
 src/backend/access/hash/hashpage.c               |   9 +
 src/backend/access/hash/hashsearch.c             |   2 +
 src/backend/storage/lmgr/README-SSI              |   4 +
 src/test/isolation/expected/predicate-hash-2.out | 321 +++++++++++++++++++++
 src/test/isolation/expected/predicate-hash.out   | 339 +++++++++++++++++++++++
 src/test/isolation/isolation_schedule            |   2 +
 src/test/isolation/specs/predicate-hash-2.spec   |  51 ++++
 src/test/isolation/specs/predicate-hash.spec     |  51 ++++
 10 files changed, 783 insertions(+), 1 deletion(-)
 create mode 100644 src/test/isolation/expected/predicate-hash-2.out
 create mode 100644 src/test/isolation/expected/predicate-hash.out
 create mode 100644 src/test/isolation/specs/predicate-hash-2.spec
 create mode 100644 src/test/isolation/specs/predicate-hash.spec

diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index d89c192..fa4d4d6 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -68,7 +68,7 @@ hashhandler(PG_FUNCTION_ARGS)
 	amroutine->amsearchnulls = false;
 	amroutine->amstorage = false;
 	amroutine->amclusterable = false;
-	amroutine->ampredlocks = false;
+	amroutine->ampredlocks = true;
 	amroutine->amcanparallel = false;
 	amroutine->amkeytype = INT4OID;
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index dc08db9..c8005eb 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -22,6 +22,7 @@
 #include "utils/rel.h"
 #include "storage/lwlock.h"
 #include "storage/buf_internals.h"
+#include "storage/predicate.h"
 
 static void _hash_vacuum_one_page(Relation rel, Buffer metabuf, Buffer buf,
 					  RelFileNode hnode);
@@ -88,6 +89,8 @@ restart_insert:
 										  &usedmetap);
 	Assert(usedmetap != NULL);
 
+	CheckForSerializableConflictIn(rel, NULL, buf);
+
 	/* remember the primary bucket buffer to release the pin on it at end. */
 	bucket_buf = buf;
 
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index 1cb18a7..2a950e1 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -33,6 +33,7 @@
 #include "miscadmin.h"
 #include "storage/lmgr.h"
 #include "storage/smgr.h"
+#include "storage/predicate.h"
 
 
 static bool _hash_alloc_buckets(Relation rel, BlockNumber firstblock,
@@ -947,6 +948,10 @@ restart_expand:
 					  buf_oblkno, buf_nblkno, NULL,
 					  maxbucket, highmask, lowmask);
 
+	PredicateLockPageSplit(rel,
+				BufferGetBlockNumber(buf_oblkno),
+				BufferGetBlockNumber(buf_nblkno));
+
 	/* all done, now release the locks and pins on primary buckets. */
 	_hash_relbuf(rel, buf_oblkno);
 	_hash_relbuf(rel, buf_nblkno);
@@ -1425,6 +1430,10 @@ _hash_finish_split(Relation rel, Buffer metabuf, Buffer obuf, Bucket obucket,
 					  nbucket, obuf, bucket_nbuf, tidhtab,
 					  maxbucket, highmask, lowmask);
 
+	PredicateLockPageSplit(rel,
+				BufferGetBlockNumber(obuf),
+				BufferGetBlockNumber(bucket_nbuf));
+
 	_hash_relbuf(rel, bucket_nbuf);
 	LockBuffer(obuf, BUFFER_LOCK_UNLOCK);
 	hash_destroy(tidhtab);
diff --git a/src/backend/access/hash/hashsearch.c b/src/backend/access/hash/hashsearch.c
index 3e461ad..dfde237 100644
--- a/src/backend/access/hash/hashsearch.c
+++ b/src/backend/access/hash/hashsearch.c
@@ -19,6 +19,7 @@
 #include "miscadmin.h"
 #include "pgstat.h"
 #include "utils/rel.h"
+#include "storage/predicate.h"
 
 
 /*
@@ -284,6 +285,7 @@ _hash_first(IndexScanDesc scan, ScanDirection dir)
 	so->hashso_sk_hash = hashkey;
 
 	buf = _hash_getbucketbuf_from_hashkey(rel, hashkey, HASH_READ, NULL);
+	PredicateLockPage(rel, BufferGetBlockNumber(buf), scan->xs_snapshot);
 	page = BufferGetPage(buf);
 	TestForOldSnapshot(scan->xs_snapshot, rel, page);
 	opaque = (HashPageOpaque) PageGetSpecialPointer(page);
diff --git a/src/backend/storage/lmgr/README-SSI b/src/backend/storage/lmgr/README-SSI
index a9dc01f..6e9cbba 100644
--- a/src/backend/storage/lmgr/README-SSI
+++ b/src/backend/storage/lmgr/README-SSI
@@ -379,6 +379,10 @@ level during a GiST search. An index insert at the leaf level can
 then be trusted to ripple up to all levels and locations where
 conflicting predicate locks may exist.
 
+    * Hash index searches acquire predicate locks on the primary
+page of a bucket. During a bucket split, a predicate lock is copied from
+the primary page of an old bucket to the primary page of a new bucket.
+
     * The effects of page splits, overflows, consolidations, and
 removals must be carefully reviewed to ensure that predicate locks
 aren't "lost" during those operations, or kept with pages which could
diff --git a/src/test/isolation/expected/predicate-hash-2.out b/src/test/isolation/expected/predicate-hash-2.out
new file mode 100644
index 0000000..78312d0
--- /dev/null
+++ b/src/test/isolation/expected/predicate-hash-2.out
@@ -0,0 +1,321 @@
+Parsed test spec with 2 sessions
+
+starting permutation: rxy1 wx1 c1 rxy2 wy2 c2
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy1 wx1 rxy2 c1 wy2 c2
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step c1: commit;
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy1 wx1 rxy2 wy2 c1 c2
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy1 wx1 rxy2 wy2 c2 c1
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy1 rxy2 wx1 c1 wy2 c2
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy1 rxy2 wx1 wy2 c1 c2
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy1 rxy2 wx1 wy2 c2 c1
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy1 rxy2 wy2 wx1 c1 c2
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy1 rxy2 wy2 wx1 c2 c1
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy1 rxy2 wy2 c2 wx1 c1
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy2 rxy1 wx1 c1 wy2 c2
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy2 rxy1 wx1 wy2 c1 c2
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy2 rxy1 wx1 wy2 c2 c1
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy2 rxy1 wy2 wx1 c1 c2
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy2 rxy1 wy2 wx1 c2 c1
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy2 rxy1 wy2 c2 wx1 c1
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy2 wy2 rxy1 wx1 c1 c2
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy2 wy2 rxy1 wx1 c2 c1
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy2 wy2 rxy1 c2 wx1 c1
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step c2: commit;
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy2 wy2 c2 rxy1 wx1 c1
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
diff --git a/src/test/isolation/expected/predicate-hash.out b/src/test/isolation/expected/predicate-hash.out
new file mode 100644
index 0000000..272cb2e
--- /dev/null
+++ b/src/test/isolation/expected/predicate-hash.out
@@ -0,0 +1,339 @@
+Parsed test spec with 2 sessions
+
+starting permutation: rxy1 wx1 c1 rxy2 wy2 c2
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+600            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy1 wx1 rxy2 c1 wy2 c2
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step c1: commit;
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy1 wx1 rxy2 wy2 c1 c2
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 wx1 rxy2 wy2 c2 c1
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wx1 c1 wy2 c2
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy1 rxy2 wx1 wy2 c1 c2
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wx1 wy2 c2 c1
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 wx1 c1 c2
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 wx1 c2 c1
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 c2 wx1 c1
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy2 rxy1 wx1 c1 wy2 c2
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy2 rxy1 wx1 wy2 c1 c2
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wx1 wy2 c2 c1
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 wx1 c1 c2
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 wx1 c2 c1
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 c2 wx1 c1
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy2 wy2 rxy1 wx1 c1 c2
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 wy2 rxy1 wx1 c2 c1
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 wy2 rxy1 c2 wx1 c1
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+200            
+step c2: commit;
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy2 wy2 c2 rxy1 wx1 c1
+step rxy2: select sum(p) from hash_point_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step rxy1: select sum(p) from hash_point_tbl where p=20;
+sum            
+
+400            
+step wx1: insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 32c965b..284b87a 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -62,3 +62,5 @@ test: sequence-ddl
 test: async-notify
 test: vacuum-reltuples
 test: timeouts
+test: predicate-hash
+test: predicate-hash-2
diff --git a/src/test/isolation/specs/predicate-hash-2.spec b/src/test/isolation/specs/predicate-hash-2.spec
new file mode 100644
index 0000000..09d424d
--- /dev/null
+++ b/src/test/isolation/specs/predicate-hash-2.spec
@@ -0,0 +1,51 @@
+# Test for predicate locking in hash index
+#
+# Test to check false positives
+#
+# Queries are written in such a way that an index scan(from one transaction) and an index insert(from another transaction) will try to access different buckets of the index.
+
+
+setup
+{
+ create table hash_point_tbl(id int4, p integer);
+ create index hash_pointidx on hash_point_tbl using hash(p);
+ insert into hash_point_tbl (id, p)
+ select g, 10 from generate_series(1, 10) g;
+ insert into hash_point_tbl (id, p)
+ select g, 20 from generate_series(11, 20) g;
+ insert into hash_point_tbl (id, p)
+ select g, 30 from generate_series(21, 30) g;
+ insert into hash_point_tbl (id, p)
+ select g, 40 from generate_series(31, 40) g;
+}
+
+teardown
+{
+ DROP TABLE hash_point_tbl;
+}
+
+
+session "s1"
+setup		{
+		  BEGIN ISOLATION LEVEL SERIALIZABLE;
+		  set enable_seqscan=off;
+		  set enable_bitmapscan=off;
+		  set enable_indexonlyscan=on;
+		}
+step "rxy1"	{ select sum(p) from hash_point_tbl where p=20; }
+step "wx1"	{ insert into hash_point_tbl (id, p)
+                  select g, 50 from generate_series(41, 50) g; }
+step "c1"	{ commit; }
+
+
+session "s2"
+setup		{
+		  BEGIN ISOLATION LEVEL SERIALIZABLE;
+		  set enable_seqscan=off;
+		  set enable_bitmapscan=off;
+		  set enable_indexonlyscan=on;
+		}
+step "rxy2"	{ select sum(p) from hash_point_tbl where p=30; }
+step "wy2"	{ insert into hash_point_tbl (id, p)
+                  select g, 60 from generate_series(51, 60) g; }
+step "c2"	{ commit; }
diff --git a/src/test/isolation/specs/predicate-hash.spec b/src/test/isolation/specs/predicate-hash.spec
new file mode 100644
index 0000000..5bb2e52
--- /dev/null
+++ b/src/test/isolation/specs/predicate-hash.spec
@@ -0,0 +1,51 @@
+# Test for predicate locking in hash index
+#
+# Test to verify serialization failures
+#
+# Queries are written in such a way that an index scan(from one transaction) and an index insert(from another transaction) will try to access the same bucket of the index.
+
+
+setup
+{
+ create table hash_point_tbl(id int4, p integer);
+ create index hash_pointidx on hash_point_tbl using hash(p);
+ insert into hash_point_tbl (id, p)
+ select g, 10 from generate_series(1, 10) g;
+ insert into hash_point_tbl (id, p)
+ select g, 20 from generate_series(11, 20) g;
+ insert into hash_point_tbl (id, p)
+ select g, 30 from generate_series(21, 30) g;
+ insert into hash_point_tbl (id, p)
+ select g, 40 from generate_series(31, 40) g;
+}
+
+teardown
+{
+ DROP TABLE hash_point_tbl;
+}
+
+
+session "s1"
+setup		{
+		  BEGIN ISOLATION LEVEL SERIALIZABLE;
+		  set enable_seqscan=off;
+		  set enable_bitmapscan=off;
+		  set enable_indexonlyscan=on;
+		}
+step "rxy1"	{ select sum(p) from hash_point_tbl where p=20; }
+step "wx1"	{ insert into hash_point_tbl (id, p)
+                  select g, 30 from generate_series(41, 50) g; }
+step "c1"	{ commit; }
+
+
+session "s2"
+setup		{
+		  BEGIN ISOLATION LEVEL SERIALIZABLE;
+		  set enable_seqscan=off;
+		  set enable_bitmapscan=off;
+		  set enable_indexonlyscan=on;
+		}
+step "rxy2"	{ select sum(p) from hash_point_tbl where p=30; }
+step "wy2"	{ insert into hash_point_tbl (id, p)
+                  select g, 20 from generate_series(51, 60) g; }
+step "c2"	{ commit; }
-- 
1.9.1

#4Alexander Korotkov
a.korotkov@postgrespro.ru
In reply to: Thomas Munro (#2)
Re: GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Fri, Sep 8, 2017 at 4:07 AM, Thomas Munro <thomas.munro@enterprisedb.com>
wrote:

Hi Shubham,

On Tue, Jun 27, 2017 at 9:21 PM, Shubham Barai <shubhambaraiss@gmail.com>
wrote:

If these two hash keys (78988658 and 546789888) mapped to the same

bucket, this will result in false serialization failure.

Please correct me if this assumption about false positives is wrong.

I wonder if there is an opportunity to use computed hash values
directly in predicate lock tags, rather than doing it on the basis of
buckets. Perhaps I'm missing something important about the way that
locks need to escalate that would prevent that from working.

+1,
Very nice idea! Locking hash values directly seems to be superior over
locking hash index pages.
Shubham, do you have any comment on this?

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

#5Michael Paquier
michael.paquier@gmail.com
In reply to: Shubham Barai (#3)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Mon, Sep 25, 2017 at 10:34 PM, Shubham Barai
<shubhambaraiss@gmail.com> wrote:

I have attached the rebased version of patch here.

The patch does not apply and there has been no reviews as well. In
consequence, I am moving it to next CF with "waiting on author" as
status. Please provide a rebased patch.
--
Michael

#6Stephen Frost
sfrost@snowman.net
In reply to: Michael Paquier (#5)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

Greeting Shubham, all,

* Michael Paquier (michael.paquier@gmail.com) wrote:

On Mon, Sep 25, 2017 at 10:34 PM, Shubham Barai
<shubhambaraiss@gmail.com> wrote:

I have attached the rebased version of patch here.

The patch does not apply and there has been no reviews as well. In
consequence, I am moving it to next CF with "waiting on author" as
status. Please provide a rebased patch.

Shubham, would you mind providing an updated patch which applies
cleanly, so we can change this to Needs Review and hopefully get someone
looking at it? Also, it would be good to respond to Alexander's as to
if it would work or not (and perhaps updating the patch accordingly).
Otherwise, I'm afriad this patch may end up just getting bumped to the
next CF or ending up as 'returned with feedback'. Would be great to get
this up to a point where it could be committed.

Thanks!

Stephen

#7Shubham Barai
shubhambaraiss@gmail.com
In reply to: Stephen Frost (#6)
1 attachment(s)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On 15 January 2018 at 08:03, Stephen Frost <sfrost@snowman.net> wrote:

Greeting Shubham, all,

* Michael Paquier (michael.paquier@gmail.com) wrote:

On Mon, Sep 25, 2017 at 10:34 PM, Shubham Barai
<shubhambaraiss@gmail.com> wrote:

I have attached the rebased version of patch here.

The patch does not apply and there has been no reviews as well. In
consequence, I am moving it to next CF with "waiting on author" as
status. Please provide a rebased patch.

Shubham, would you mind providing an updated patch which applies
cleanly, so we can change this to Needs Review and hopefully get someone
looking at it? Also, it would be good to respond to Alexander's as to
if it would work or not (and perhaps updating the patch accordingly).
Otherwise, I'm afriad this patch may end up just getting bumped to the
next CF or ending up as 'returned with feedback'. Would be great to get
this up to a point where it could be committed.

Hi Stephen,

The new approach was suggested after completion of GSoC. So, I didn't get
enough time to implement this approach. Also, I was constantly updating my
patches for gist and gin index based on reviews.

Here, I am attaching the rebased version of the patch (which is based on an
old approch:
acquiring a predicate lock on primary page of a bucket)

Kind Regards,
Shubham

Attachments:

Predicate-Locking-in-hash-index_v5.patchapplication/octet-stream; name=Predicate-Locking-in-hash-index_v5.patchDownload
From 8c869531aff0175be95cca205db032a6e72b7f13 Mon Sep 17 00:00:00 2001
From: shubhambaraiss <you@example.com>
Date: Mon, 15 Jan 2018 19:36:25 +0530
Subject: [PATCH] Predicate Locking in hash index

---
 src/backend/access/hash/hash.c                 |   2 +-
 src/backend/access/hash/hashinsert.c           |   3 +
 src/backend/access/hash/hashpage.c             |  10 +
 src/backend/access/hash/hashsearch.c           |   2 +
 src/backend/storage/lmgr/README-SSI            |   5 +
 src/test/isolation/expected/predicate-hash.out | 659 +++++++++++++++++++++++++
 src/test/isolation/isolation_schedule          |   1 +
 src/test/isolation/specs/predicate-hash.spec   | 122 +++++
 8 files changed, 803 insertions(+), 1 deletion(-)
 create mode 100644 src/test/isolation/expected/predicate-hash.out
 create mode 100644 src/test/isolation/specs/predicate-hash.spec

diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 718e2be..2932649 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -68,7 +68,7 @@ hashhandler(PG_FUNCTION_ARGS)
 	amroutine->amsearchnulls = false;
 	amroutine->amstorage = false;
 	amroutine->amclusterable = false;
-	amroutine->ampredlocks = false;
+	amroutine->ampredlocks = true;
 	amroutine->amcanparallel = false;
 	amroutine->amkeytype = INT4OID;
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index f668dcf..ade49d8 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -22,6 +22,7 @@
 #include "utils/rel.h"
 #include "storage/lwlock.h"
 #include "storage/buf_internals.h"
+#include "storage/predicate.h"
 
 static void _hash_vacuum_one_page(Relation rel, Buffer metabuf, Buffer buf,
 					  RelFileNode hnode);
@@ -88,6 +89,8 @@ restart_insert:
 										  &usedmetap);
 	Assert(usedmetap != NULL);
 
+	CheckForSerializableConflictIn(rel, NULL, buf);
+
 	/* remember the primary bucket buffer to release the pin on it at end. */
 	bucket_buf = buf;
 
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index e3c8721..fe2b22d 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -33,6 +33,7 @@
 #include "miscadmin.h"
 #include "storage/lmgr.h"
 #include "storage/smgr.h"
+#include "storage/predicate.h"
 
 
 static bool _hash_alloc_buckets(Relation rel, BlockNumber firstblock,
@@ -957,6 +958,10 @@ restart_expand:
 					  buf_oblkno, buf_nblkno, NULL,
 					  maxbucket, highmask, lowmask);
 
+	PredicateLockPageSplit(rel,
+						   BufferGetBlockNumber(buf_oblkno),
+						   BufferGetBlockNumber(buf_nblkno));
+
 	/* all done, now release the pins on primary buckets. */
 	_hash_dropbuf(rel, buf_oblkno);
 	_hash_dropbuf(rel, buf_nblkno);
@@ -1459,6 +1464,11 @@ _hash_finish_split(Relation rel, Buffer metabuf, Buffer obuf, Bucket obucket,
 					  nbucket, obuf, bucket_nbuf, tidhtab,
 					  maxbucket, highmask, lowmask);
 
+	PredicateLockPageSplit(rel,
+						   BufferGetBlockNumber(obuf),
+						   BufferGetBlockNumber(bucket_nbuf));
+
+
 	_hash_dropbuf(rel, bucket_nbuf);
 	hash_destroy(tidhtab);
 }
diff --git a/src/backend/access/hash/hashsearch.c b/src/backend/access/hash/hashsearch.c
index c692c5b..da95fd8 100644
--- a/src/backend/access/hash/hashsearch.c
+++ b/src/backend/access/hash/hashsearch.c
@@ -19,6 +19,7 @@
 #include "miscadmin.h"
 #include "pgstat.h"
 #include "utils/rel.h"
+#include "storage/predicate.h"
 
 static bool _hash_readpage(IndexScanDesc scan, Buffer *bufP,
 			   ScanDirection dir);
@@ -347,6 +348,7 @@ _hash_first(IndexScanDesc scan, ScanDirection dir)
 	so->hashso_sk_hash = hashkey;
 
 	buf = _hash_getbucketbuf_from_hashkey(rel, hashkey, HASH_READ, NULL);
+	PredicateLockPage(rel, BufferGetBlockNumber(buf), scan->xs_snapshot);
 	page = BufferGetPage(buf);
 	TestForOldSnapshot(scan->xs_snapshot, rel, page);
 	opaque = (HashPageOpaque) PageGetSpecialPointer(page);
diff --git a/src/backend/storage/lmgr/README-SSI b/src/backend/storage/lmgr/README-SSI
index a9dc01f..739efec 100644
--- a/src/backend/storage/lmgr/README-SSI
+++ b/src/backend/storage/lmgr/README-SSI
@@ -379,6 +379,11 @@ level during a GiST search. An index insert at the leaf level can
 then be trusted to ripple up to all levels and locations where
 conflicting predicate locks may exist.
 
+    * Hash index searches acquire predicate locks on the primary
+page of a bucket. During a bucket split, a predicate lock is copied from
+the primary page of an old bucket to the primary page of a new bucket.
+
+
     * The effects of page splits, overflows, consolidations, and
 removals must be carefully reviewed to ensure that predicate locks
 aren't "lost" during those operations, or kept with pages which could
diff --git a/src/test/isolation/expected/predicate-hash.out b/src/test/isolation/expected/predicate-hash.out
new file mode 100644
index 0000000..53e500f
--- /dev/null
+++ b/src/test/isolation/expected/predicate-hash.out
@@ -0,0 +1,659 @@
+Parsed test spec with 2 sessions
+
+starting permutation: rxy1 wx1 c1 rxy2 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+600            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy2 wy2 c2 rxy1 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+400            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy3 wx3 c1 rxy4 wy4 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy4 wy4 c2 rxy3 wx3 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy1 wx1 rxy2 c1 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy1 wx1 rxy2 wy2 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 wx1 rxy2 wy2 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wx1 c1 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy1 rxy2 wx1 wy2 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wx1 wy2 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 wx1 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 wx1 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 c2 wx1 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy2 rxy1 wx1 c1 wy2 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy2 rxy1 wx1 wy2 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wx1 wy2 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 wx1 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 wx1 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 c2 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy2 wy2 rxy1 wx1 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 wy2 rxy1 wx1 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 wy2 rxy1 c2 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy3 wx3 rxy4 c1 wy4 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step c1: commit;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy3 wx3 rxy4 wy4 c1 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy3 wx3 rxy4 wy4 c2 c1
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy3 rxy4 wx3 c1 wy4 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy3 rxy4 wx3 wy4 c1 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy3 rxy4 wx3 wy4 c2 c1
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy3 rxy4 wy4 wx3 c1 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy3 rxy4 wy4 wx3 c2 c1
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy3 rxy4 wy4 c2 wx3 c1
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy4 rxy3 wx3 c1 wy4 c2
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy4 rxy3 wx3 wy4 c1 c2
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy4 rxy3 wx3 wy4 c2 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy4 rxy3 wy4 wx3 c1 c2
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy4 rxy3 wy4 wx3 c2 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy4 rxy3 wy4 c2 wx3 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy4 wy4 rxy3 wx3 c1 c2
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy4 wy4 rxy3 wx3 c2 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy4 wy4 rxy3 c2 wx3 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step c2: commit;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index befe676..d1adaa1 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -65,3 +65,4 @@ test: async-notify
 test: vacuum-reltuples
 test: timeouts
 test: vacuum-concurrent-drop
+test: predicate-hash
diff --git a/src/test/isolation/specs/predicate-hash.spec b/src/test/isolation/specs/predicate-hash.spec
new file mode 100644
index 0000000..8c83cb3
--- /dev/null
+++ b/src/test/isolation/specs/predicate-hash.spec
@@ -0,0 +1,122 @@
+# Test for page level predicate locking in hash index
+#
+# Test to verify serialization failures and to check reduced false positives
+#
+# To verify serialization failures, queries and permutations are written in such
+# a way that an index scan  (from one transaction) and an index insert (from
+# another transaction) will try to access the same bucket of the index
+# whereas to check reduced false positives, they will try to access different
+# buckets of the index.
+
+setup
+{
+ create table hash_tbl(id int4, p integer);
+ create index hash_idx on hash_tbl using hash(p);
+ insert into hash_tbl (id, p)
+ select g, 10 from generate_series(1, 10) g;
+ insert into hash_tbl (id, p)
+ select g, 20 from generate_series(11, 20) g;
+ insert into hash_tbl (id, p)
+ select g, 30 from generate_series(21, 30) g;
+ insert into hash_tbl (id, p)
+ select g, 40 from generate_series(31, 40) g;
+}
+
+teardown
+{
+ drop table hash_tbl;
+}
+
+session "s1"
+setup		
+{
+ begin isolation level serializable;
+ set enable_seqscan=off;
+ set enable_bitmapscan=off;
+ set enable_indexonlyscan=on;
+}
+step "rxy1"	{ select sum(p) from hash_tbl where p=20; }
+step "wx1"	{ insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g; }
+step "rxy3"	{ select sum(p) from hash_tbl where p=20; }
+step "wx3"	{ insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g; }
+step "c1"	{ commit; }
+
+
+session "s2"
+setup		
+{
+ begin isolation level serializable;
+ set enable_seqscan=off;
+ set enable_bitmapscan=off;
+ set enable_indexonlyscan=on;
+}
+step "rxy2"	{ select sum(p) from hash_tbl where p=30; }
+step "wy2"	{ insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g; }
+step "rxy4"	{ select sum(p) from hash_tbl where p=30; }
+step "wy4"	{ insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g; }
+step "c2"	{ commit; }
+
+
+# An index scan (from one transaction) and an index insert (from another
+# transaction) try to access the same bucket of the index but one transaction
+# commits before other transaction begins so no r-w conflict.
+
+permutation "rxy1" "wx1" "c1" "rxy2" "wy2" "c2"
+permutation "rxy2" "wy2" "c2" "rxy1" "wx1" "c1"
+
+# An index scan (from one transaction) and an index insert (from another
+# transaction) try to access different buckets of the index and also one
+# transaction commits before other transaction begins, so no r-w conflict.
+
+permutation "rxy3" "wx3" "c1" "rxy4" "wy4" "c2"
+permutation "rxy4" "wy4" "c2" "rxy3" "wx3" "c1"
+
+
+# An index scan (from one transaction) and an index insert (from another
+# transaction) try to access the same bucket of the index and one transaction
+# begins before other transaction commits so there is a r-w conflict.
+
+permutation "rxy1" "wx1" "rxy2" "c1" "wy2" "c2"
+permutation "rxy1" "wx1" "rxy2" "wy2" "c1" "c2"
+permutation "rxy1" "wx1" "rxy2" "wy2" "c2" "c1"
+permutation "rxy1" "rxy2" "wx1" "c1" "wy2" "c2"
+permutation "rxy1" "rxy2" "wx1" "wy2" "c1" "c2"
+permutation "rxy1" "rxy2" "wx1" "wy2" "c2" "c1"
+permutation "rxy1" "rxy2" "wy2" "wx1" "c1" "c2"
+permutation "rxy1" "rxy2" "wy2" "wx1" "c2" "c1"
+permutation "rxy1" "rxy2" "wy2" "c2" "wx1" "c1"
+permutation "rxy2" "rxy1" "wx1" "c1" "wy2" "c2"
+permutation "rxy2" "rxy1" "wx1" "wy2" "c1" "c2"
+permutation "rxy2" "rxy1" "wx1" "wy2" "c2" "c1"
+permutation "rxy2" "rxy1" "wy2" "wx1" "c1" "c2"
+permutation "rxy2" "rxy1" "wy2" "wx1" "c2" "c1"
+permutation "rxy2" "rxy1" "wy2" "c2" "wx1" "c1"
+permutation "rxy2" "wy2" "rxy1" "wx1" "c1" "c2"
+permutation "rxy2" "wy2" "rxy1" "wx1" "c2" "c1"
+permutation "rxy2" "wy2" "rxy1" "c2" "wx1" "c1"
+
+# An index scan (from one transaction) and an index insert (from another 
+# transaction) try to access different buckets of the index so no r-w conflict.
+
+permutation "rxy3" "wx3" "rxy4" "c1" "wy4" "c2"
+permutation "rxy3" "wx3" "rxy4" "wy4" "c1" "c2"
+permutation "rxy3" "wx3" "rxy4" "wy4" "c2" "c1"
+permutation "rxy3" "rxy4" "wx3" "c1" "wy4" "c2"
+permutation "rxy3" "rxy4" "wx3" "wy4" "c1" "c2"
+permutation "rxy3" "rxy4" "wx3" "wy4" "c2" "c1"
+permutation "rxy3" "rxy4" "wy4" "wx3" "c1" "c2"
+permutation "rxy3" "rxy4" "wy4" "wx3" "c2" "c1"
+permutation "rxy3" "rxy4" "wy4" "c2" "wx3" "c1"
+permutation "rxy4" "rxy3" "wx3" "c1" "wy4" "c2"
+permutation "rxy4" "rxy3" "wx3" "wy4" "c1" "c2"
+permutation "rxy4" "rxy3" "wx3" "wy4" "c2" "c1"
+permutation "rxy4" "rxy3" "wy4" "wx3" "c1" "c2"
+permutation "rxy4" "rxy3" "wy4" "wx3" "c2" "c1"
+permutation "rxy4" "rxy3" "wy4" "c2" "wx3" "c1"
+permutation "rxy4" "wy4" "rxy3" "wx3" "c1" "c2"
+permutation "rxy4" "wy4" "rxy3" "wx3" "c2" "c1"
+permutation "rxy4" "wy4" "rxy3" "c2" "wx3" "c1"
-- 
1.9.1

#8Amit Kapila
amit.kapila16@gmail.com
In reply to: Alexander Korotkov (#4)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Fri, Sep 29, 2017 at 8:20 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:

On Fri, Sep 8, 2017 at 4:07 AM, Thomas Munro <thomas.munro@enterprisedb.com>
wrote:

Hi Shubham,

On Tue, Jun 27, 2017 at 9:21 PM, Shubham Barai <shubhambaraiss@gmail.com>
wrote:

If these two hash keys (78988658 and 546789888) mapped to the same
bucket, this will result in false serialization failure.
Please correct me if this assumption about false positives is wrong.

I wonder if there is an opportunity to use computed hash values
directly in predicate lock tags, rather than doing it on the basis of
buckets. Perhaps I'm missing something important about the way that
locks need to escalate that would prevent that from working.

+1,
Very nice idea! Locking hash values directly seems to be superior over
locking hash index pages.
Shubham, do you have any comment on this?

As Shubham seems to be running out of time, I thought of helping him
by looking into the above-suggested idea. I think one way to lock a
particular hash value is we can use TID of heap tuple associated with
each index entry (index entry for the hash index will be hash value).
However, do we really need it for implementing predicate locking for
hash indexes? If we look at the "Index AM implementations" section of
README-SSI, it doesn't seem to be required. Basically, if we look at
the strategy of predicate locks in btree [1]* B-tree index searches acquire predicate locks only on the index *leaf* pages needed to lock the appropriate index range. If, however, a search discovers that no root page has yet been created, a predicate lock on the index relation is required., it seems to me locking
at page level for hash index seems to be a right direction as similar
to btree, the corresponding heap tuple read will be locked.

What do you think?

[1]: * B-tree index searches acquire predicate locks only on the index *leaf* pages needed to lock the appropriate index range. If, however, a search discovers that no root page has yet been created, a predicate lock on the index relation is required.
* B-tree index searches acquire predicate locks only on the index
*leaf* pages needed to lock the appropriate index range. If, however,
a search discovers that no root page has yet been created, a predicate
lock on the index relation is required.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

#9Alexander Korotkov
a.korotkov@postgrespro.ru
In reply to: Amit Kapila (#8)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Sat, Jan 20, 2018 at 4:24 PM, Amit Kapila <amit.kapila16@gmail.com>
wrote:

On Fri, Sep 29, 2017 at 8:20 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:

On Fri, Sep 8, 2017 at 4:07 AM, Thomas Munro <

thomas.munro@enterprisedb.com>

wrote:

Hi Shubham,

On Tue, Jun 27, 2017 at 9:21 PM, Shubham Barai <

shubhambaraiss@gmail.com>

wrote:

If these two hash keys (78988658 and 546789888) mapped to the same
bucket, this will result in false serialization failure.
Please correct me if this assumption about false positives is wrong.

I wonder if there is an opportunity to use computed hash values
directly in predicate lock tags, rather than doing it on the basis of
buckets. Perhaps I'm missing something important about the way that
locks need to escalate that would prevent that from working.

+1,
Very nice idea! Locking hash values directly seems to be superior over
locking hash index pages.
Shubham, do you have any comment on this?

As Shubham seems to be running out of time, I thought of helping him
by looking into the above-suggested idea. I think one way to lock a
particular hash value is we can use TID of heap tuple associated with
each index entry (index entry for the hash index will be hash value).

Sorry, I didn't get what do you particularly mean. If locking either TID of
associated heap tuple or TID of hash index tuple, then what will we lock
in the case when nothing found? Even if we found nothing, we have
to place some lock according to search key in order to detect cases when
somebody has inserted the row which we might see according to that search
key.

However, do we really need it for implementing predicate locking for
hash indexes? If we look at the "Index AM implementations" section of
README-SSI, it doesn't seem to be required. Basically, if we look at
the strategy of predicate locks in btree [1], it seems to me locking
at page level for hash index seems to be a right direction as similar
to btree, the corresponding heap tuple read will be locked.

Btree uses leaf-pages locking because it supports both range searches
and exact value searches. And it needs to detect overlaps between
these two kinds of searches. Therefore, btree locks leaf-pages in both
cases. Hash index case is different. Hash index doesn't support and
isn't going to support range searches. Assuming, that hash index
supports only exact value searches, locking hash values would be
superior over page locking (unless I'm missing something), because
it would provide better locality of locks.

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

#10Amit Kapila
amit.kapila16@gmail.com
In reply to: Alexander Korotkov (#9)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Thu, Jan 25, 2018 at 7:29 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:

On Sat, Jan 20, 2018 at 4:24 PM, Amit Kapila <amit.kapila16@gmail.com>
wrote:

On Fri, Sep 29, 2017 at 8:20 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:

+1,
Very nice idea! Locking hash values directly seems to be superior over
locking hash index pages.
Shubham, do you have any comment on this?

As Shubham seems to be running out of time, I thought of helping him
by looking into the above-suggested idea. I think one way to lock a
particular hash value is we can use TID of heap tuple associated with
each index entry (index entry for the hash index will be hash value).

Sorry, I didn't get what do you particularly mean. If locking either TID of
associated heap tuple or TID of hash index tuple, then what will we lock
in the case when nothing found? Even if we found nothing, we have
to place some lock according to search key in order to detect cases when
somebody has inserted the row which we might see according to that search
key.

Okay, but if you use hash value as lock tag (which is possible) how
will we deal with things like page split? I think even if use
blocknumber/page/bucketnumber corresponding to the hash value along
with hash value in lock tag, then also it doesn't appear to work. I
think using page level locks for index makes sense, especially because
it will be convinient to deal with page splits. Also, as predicate
locks stay in-memory, so creating too many such locks doesn't sound
like a nice strategy even though we have a way to upgrade it to next
level (page) as that has a separate cost to it.

However, do we really need it for implementing predicate locking for
hash indexes? If we look at the "Index AM implementations" section of
README-SSI, it doesn't seem to be required. Basically, if we look at
the strategy of predicate locks in btree [1], it seems to me locking
at page level for hash index seems to be a right direction as similar
to btree, the corresponding heap tuple read will be locked.

Btree uses leaf-pages locking because it supports both range searches
and exact value searches. And it needs to detect overlaps between
these two kinds of searches. Therefore, btree locks leaf-pages in both
cases.

Also, probably using page level locks makes it easier to deal index
operations like page split.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

#11Amit Kapila
amit.kapila16@gmail.com
In reply to: Amit Kapila (#10)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Sun, Jan 28, 2018 at 7:28 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:

On Thu, Jan 25, 2018 at 7:29 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:

As Shubham seems to be running out of time, I thought of helping him
by looking into the above-suggested idea. I think one way to lock a
particular hash value is we can use TID of heap tuple associated with
each index entry (index entry for the hash index will be hash value).

Sorry, I didn't get what do you particularly mean. If locking either TID of
associated heap tuple or TID of hash index tuple, then what will we lock
in the case when nothing found? Even if we found nothing, we have
to place some lock according to search key in order to detect cases when
somebody has inserted the row which we might see according to that search
key.

Okay, but if you use hash value as lock tag (which is possible) how
will we deal with things like page split? I think even if use
blocknumber/page/bucketnumber corresponding to the hash value along
with hash value in lock tag, then also it doesn't appear to work. I
think using page level locks for index makes sense, especially because
it will be convinient to deal with page splits.

What I intend to say here is that we already have a mechanism like
PredicateLockPageSplit() which can deal with predicate locks during
page split if we operate at page level. However, if we want to go for
has value locking technique, it can be quite complex and expensive to
make it work as we have to search all the locks that belong to the
bucket being split and then move them for the new bucket.

Alexander/Thomas, do you have better ideas to make it work, otherwise,
I think we can proceed to review the Shubham's current approach/patch?

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

#12Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Amit Kapila (#11)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Tue, Feb 13, 2018 at 4:28 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:

On Sun, Jan 28, 2018 at 7:28 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:

On Thu, Jan 25, 2018 at 7:29 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:

As Shubham seems to be running out of time, I thought of helping him
by looking into the above-suggested idea. I think one way to lock a
particular hash value is we can use TID of heap tuple associated with
each index entry (index entry for the hash index will be hash value).

Sorry, I didn't get what do you particularly mean. If locking either TID of
associated heap tuple or TID of hash index tuple, then what will we lock
in the case when nothing found? Even if we found nothing, we have
to place some lock according to search key in order to detect cases when
somebody has inserted the row which we might see according to that search
key.

Okay, but if you use hash value as lock tag (which is possible) how
will we deal with things like page split? I think even if use
blocknumber/page/bucketnumber corresponding to the hash value along
with hash value in lock tag, then also it doesn't appear to work. I
think using page level locks for index makes sense, especially because
it will be convinient to deal with page splits.

What I intend to say here is that we already have a mechanism like
PredicateLockPageSplit() which can deal with predicate locks during
page split if we operate at page level. However, if we want to go for
has value locking technique, it can be quite complex and expensive to
make it work as we have to search all the locks that belong to the
bucket being split and then move them for the new bucket.

True.

One way to avoid all that might be to use pseudo page numbers that
don't suffer from splits. I don't know how you'd choose the
constant, but it could be something like pseudo page number = hash
value % 1024. In other words, you'd use the full hash value for the
'tuple' part of the predicate lock tag, and a shorter hash value for
the 'page' part of the predicate lock tag, so you'd never have to
handle split, and promotion from fine grained 'tuple' (= hash value)
locks to coarse grained 'page' = (short hash value) locks would still
work automatically when space runs out.

Alexander/Thomas, do you have better ideas to make it work, otherwise,
I think we can proceed to review the Shubham's current approach/patch?

I think we should proceed to review the current patch. As far as I
can see, adding more fine-grained locking would remain a possibility
for future improvement. With this patch we get page-level hash index
SIREAD locks, and perhaps in a future patch we could add fine grained
hash value SIREAD locks (maybe as described above, if that works...)

--
Thomas Munro
http://www.enterprisedb.com

#13Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Thomas Munro (#12)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Tue, Feb 13, 2018 at 7:47 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

One way to avoid all that might be to use pseudo page numbers that
don't suffer from splits. I don't know how you'd choose the
constant, but it could be something like pseudo page number = hash
value % 1024. In other words, you'd use the full hash value for the
'tuple' part of the predicate lock tag, and a shorter hash value for
the 'page' part of the predicate lock tag, so you'd never have to
handle split, and promotion from fine grained 'tuple' (= hash value)
locks to coarse grained 'page' = (short hash value) locks would still
work automatically when space runs out.

Thinking about how to tune that got me thinking about a simple middle
way we could perhaps consider...

What if we just always locked pseudo page numbers using hash_value %
max_predicate_locks_per_relation (= effectively 31 by default)? Then
you'd have lower collision rates on small hash indexes, you'd never
have to deal with page splits, and you'd never exceed
max_predicate_locks_per_relation so you'd never escalate to relation
level locks on busy systems. On the downside, you'd have eg ~3%
chance of collision instead of a 1/hash_maxbucket chance of collision,
so it gets a bit worse for large indexes on systems that are not busy
enough to exceed max_predicate_locks_per_relation. You win some, you
lose some...

--
Thomas Munro
http://www.enterprisedb.com

#14Robert Haas
robertmhaas@gmail.com
In reply to: Thomas Munro (#13)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Mon, Feb 26, 2018 at 7:51 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

Thinking about how to tune that got me thinking about a simple middle
way we could perhaps consider...

What if we just always locked pseudo page numbers using hash_value %
max_predicate_locks_per_relation (= effectively 31 by default)? Then
you'd have lower collision rates on small hash indexes, you'd never
have to deal with page splits, and you'd never exceed
max_predicate_locks_per_relation so you'd never escalate to relation
level locks on busy systems. On the downside, you'd have eg ~3%
chance of collision instead of a 1/hash_maxbucket chance of collision,
so it gets a bit worse for large indexes on systems that are not busy
enough to exceed max_predicate_locks_per_relation. You win some, you
lose some...

Hmm, yeah, that definitely has some appeal. On the other hand,
there's a lot of daylight between locking hv % 2^32 and locking hv %
31; the former is going to potentially blow out the lock table really
fast, while the latter is potentially going to create an uncomfortable
number of false collisions. One could imagine a modulus larger than
31 and smaller than 4294967296, although I don't have a principled
suggestion for how to pick it. On the third hand, people who are
using SSI heavily may well have increased
max_predicate_locks_per_relation and with this proposal that just
kinda does what you want.

I don't really know how we can judge the merits of any particular
modulus (or of committing the patch at all) without some test results
showing that it helps reduce rollbacks or increase performance or
something. Otherwise we're just guessing. It does however seem to me
that locking the hash value % (something) is better than basing the
locking on bucket or page numbers. Basing it on page numbers strictly
speaking cannot work, since the same tuple could be present in any
page in the bucket chain; you'd have to lock the page number of the
head of the bucket chain. There is however no advantage of doing that
over locking the bucket number directly. Moreover, locking the bucket
number directly forces you to worry about splits, whereas if you log
hv % (something) then you don't have to care.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#15Andres Freund
andres@anarazel.de
In reply to: Robert Haas (#14)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

Hi,

Based on this sub-thread this patch's status of 'needs review' doesn't
quite seem accurate and 'waiting on author' and then 'returned with
feedback' would be more fitting?

Greetings,

Andres Freund

#16Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#15)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Fri, Mar 2, 2018 at 3:57 PM, Andres Freund <andres@anarazel.de> wrote:

Based on this sub-thread this patch's status of 'needs review' doesn't
quite seem accurate and 'waiting on author' and then 'returned with
feedback' would be more fitting?

I personally think this patch is really close to RFC. Shubham has
fulfilled the project requirement, it's a tidy and short patch, it has
tests. I think we really just need to verify that the split case
works correctly.

Hmm. I notice that this calls PredicateLockPageSplit() after both
calls to _hash_splitbucket() (the one in _hash_finish_split() and the
one in _hash_expandtable()) instead of doing it inside that function,
and that _hash_splitbucket() unlocks bucket_nbuf before returning.
What if someone else accesses bucket_nbuf between
LockBuffer(bucket_nbuf, BUFFER_LOCK_UNLOCK) and
PredicateLockPageSplit()? Doesn't that mean that another session can
read a newly created page and miss a predicate lock that is about to
be transferred to it? If that is indeed a race, could it be fixed by
calling PredicateLockPageSplit() at the start of _hash_splitbucket()
instead?

Could we get a few days to mull over this and Shubham's other patches?
It'd be really great to get some of these into 11.

My thought experiments about pseudo-pages and avoiding the split stuff
were not intended to get the patch kicked out. I thought for a while
that hash indexes were a special case and could benefit from
dispensing with those trickier problems. Upon further reflection, for
interesting size hash indexes pure hash value predicate tags wouldn't
be much better. Furthermore, if we do decide we want to use using x %
max_predicate_locks_per_relation to avoid having to escalate to
relation predicate locks at the cost of slightly higher collision rate
then we should consider that for the whole system (including heap page
predicate locking), not just hash indexes. Please consider those
ideas parked for now.

--
Thomas Munro
http://www.enterprisedb.com

#17Amit Kapila
amit.kapila16@gmail.com
In reply to: Thomas Munro (#16)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Fri, Mar 2, 2018 at 9:27 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Fri, Mar 2, 2018 at 3:57 PM, Andres Freund <andres@anarazel.de> wrote:

Based on this sub-thread this patch's status of 'needs review' doesn't
quite seem accurate and 'waiting on author' and then 'returned with
feedback' would be more fitting?

I personally think this patch is really close to RFC. Shubham has
fulfilled the project requirement, it's a tidy and short patch, it has
tests. I think we really just need to verify that the split case
works correctly.

Hmm. I notice that this calls PredicateLockPageSplit() after both
calls to _hash_splitbucket() (the one in _hash_finish_split() and the
one in _hash_expandtable()) instead of doing it inside that function,
and that _hash_splitbucket() unlocks bucket_nbuf before returning.
What if someone else accesses bucket_nbuf between
LockBuffer(bucket_nbuf, BUFFER_LOCK_UNLOCK) and
PredicateLockPageSplit()? Doesn't that mean that another session can
read a newly created page and miss a predicate lock that is about to
be transferred to it?

Yes. I think you are primarily worried about if there is an insert on
new bucket from another session as scans will anyway take the
predicate lock, right?

If that is indeed a race, could it be fixed by
calling PredicateLockPageSplit() at the start of _hash_splitbucket()
instead?

Yes, but I think it would be better if we call this once we are sure
that at least one tuple from the old bucket has been transferred
(consider if all tuples in the old bucket are dead). Apart from this,
I think this patch has missed handling the cases where we scan the
buckets when the split is in progress. In such cases, we scan both
old and new bucket, so I think we need to ensure that we take
PredicateLock on both the buckets during such scans.

Could we get a few days to mull over this and Shubham's other patches?

I would also like to see this patch going in v11. So, I can try to
finish the remaining review comments, if Shubham is not able to spare
time and you can help with the review. I am also okay to review if
anyone else other than me can handle the remaining points.

It'd be really great to get some of these into 11.

+1.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

#18Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Amit Kapila (#17)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Sun, Mar 4, 2018 at 12:53 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:

On Fri, Mar 2, 2018 at 9:27 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

Hmm. I notice that this calls PredicateLockPageSplit() after both
calls to _hash_splitbucket() (the one in _hash_finish_split() and the
one in _hash_expandtable()) instead of doing it inside that function,
and that _hash_splitbucket() unlocks bucket_nbuf before returning.
What if someone else accesses bucket_nbuf between
LockBuffer(bucket_nbuf, BUFFER_LOCK_UNLOCK) and
PredicateLockPageSplit()? Doesn't that mean that another session can
read a newly created page and miss a predicate lock that is about to
be transferred to it?

Yes. I think you are primarily worried about if there is an insert on
new bucket from another session as scans will anyway take the
predicate lock, right?

Yeah.

If that is indeed a race, could it be fixed by
calling PredicateLockPageSplit() at the start of _hash_splitbucket()
instead?

Yes, but I think it would be better if we call this once we are sure
that at least one tuple from the old bucket has been transferred
(consider if all tuples in the old bucket are dead). Apart from this,
I think this patch has missed handling the cases where we scan the
buckets when the split is in progress. In such cases, we scan both
old and new bucket, so I think we need to ensure that we take
PredicateLock on both the buckets during such scans.

Hmm. Yeah.

So, in _hash_first(), do you think we might just need this?

if (H_BUCKET_BEING_POPULATED(opaque))
{
...
old_blkno = _hash_get_oldblock_from_newbucket(rel, bucket);
...
old_buf = _hash_getbuf(rel, old_blkno, HASH_READ, LH_BUCKET_PAGE);
+ PredicateLockPage(rel, BufferGetBlockNumber(old_buf),
scan->xs_snapshot);
TestForOldSnapshot(scan->xs_snapshot, rel, BufferGetPage(old_buf));

That is, if you begin scanning a 'new' bucket, we remember the old
bucket and go and scan that too, so we'd better predicate-lock both up
front (or I suppose we could do it later when we visit that page, but
here it can be done in a single place).

What if we begin scanning an 'old' bucket that is being split? I
think we'll only do that for tuples that actually belong in the old
bucket after the split, so no need to double-lock? And I don't think
a split could begin while we are scanning. Do I have that right?

As for inserting, I'm not sure if any special treatment is needed, as
long as the scan code path (above) and the split code path are
correct. I'm not sure though.

I'm wondering how to test all this. I'm thinking about a program that
repeatedly creates a hash index and then slowly adds more things to it
so that buckets split (maybe using distinct keys carefully crafted to
hit the same bucket?), while concurrently hammering it with a ton of
scans and then ... somehow checking correctness...

--
Thomas Munro
http://www.enterprisedb.com

#19Amit Kapila
amit.kapila16@gmail.com
In reply to: Thomas Munro (#18)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Mon, Mar 5, 2018 at 8:58 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Sun, Mar 4, 2018 at 12:53 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:

Yes, but I think it would be better if we call this once we are sure
that at least one tuple from the old bucket has been transferred
(consider if all tuples in the old bucket are dead). Apart from this,
I think this patch has missed handling the cases where we scan the
buckets when the split is in progress. In such cases, we scan both
old and new bucket, so I think we need to ensure that we take
PredicateLock on both the buckets during such scans.

Hmm. Yeah.

So, in _hash_first(), do you think we might just need this?

if (H_BUCKET_BEING_POPULATED(opaque))
{
...
old_blkno = _hash_get_oldblock_from_newbucket(rel, bucket);
...
old_buf = _hash_getbuf(rel, old_blkno, HASH_READ, LH_BUCKET_PAGE);
+ PredicateLockPage(rel, BufferGetBlockNumber(old_buf),
scan->xs_snapshot);
TestForOldSnapshot(scan->xs_snapshot, rel, BufferGetPage(old_buf));

That is, if you begin scanning a 'new' bucket, we remember the old
bucket and go and scan that too, so we'd better predicate-lock both up
front (or I suppose we could do it later when we visit that page, but
here it can be done in a single place).

Yeah, that can work, but I am slightly worried that we might actually
never scan the old bucket (say for queries with Limit clause) in which
case it might give false positives for insertions in old buckets.

What if we begin scanning an 'old' bucket that is being split? I
think we'll only do that for tuples that actually belong in the old
bucket after the split, so no need to double-lock? And I don't think
a split could begin while we are scanning. Do I have that right?

Right.

As for inserting, I'm not sure if any special treatment is needed, as
long as the scan code path (above) and the split code path are
correct. I'm not sure though.

I also don't think we need any special handling for insertions.

I'm wondering how to test all this. I'm thinking about a program that
repeatedly creates a hash index and then slowly adds more things to it
so that buckets split (maybe using distinct keys carefully crafted to
hit the same bucket?), while concurrently hammering it with a ton of
scans and then ... somehow checking correctness...

Yeah, that will generate the required errors, but not sure how to
verify correctness. One idea could be that when the split is in
progress, we somehow stop it in-between (say by cancel request) and
then run targeted selects and inserts on the bucket being scanned and
bucket being populated.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

#20Amit Kapila
amit.kapila16@gmail.com
In reply to: Amit Kapila (#19)
1 attachment(s)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Tue, Mar 6, 2018 at 11:26 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:

On Mon, Mar 5, 2018 at 8:58 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Sun, Mar 4, 2018 at 12:53 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:

Yes, but I think it would be better if we call this once we are sure
that at least one tuple from the old bucket has been transferred
(consider if all tuples in the old bucket are dead). Apart from this,
I think this patch has missed handling the cases where we scan the
buckets when the split is in progress. In such cases, we scan both
old and new bucket, so I think we need to ensure that we take
PredicateLock on both the buckets during such scans.

Hmm. Yeah.

So, in _hash_first(), do you think we might just need this?

if (H_BUCKET_BEING_POPULATED(opaque))
{
...
old_blkno = _hash_get_oldblock_from_newbucket(rel, bucket);
...
old_buf = _hash_getbuf(rel, old_blkno, HASH_READ, LH_BUCKET_PAGE);
+ PredicateLockPage(rel, BufferGetBlockNumber(old_buf),
scan->xs_snapshot);
TestForOldSnapshot(scan->xs_snapshot, rel, BufferGetPage(old_buf));

That is, if you begin scanning a 'new' bucket, we remember the old
bucket and go and scan that too, so we'd better predicate-lock both up
front (or I suppose we could do it later when we visit that page, but
here it can be done in a single place).

Yeah, that can work, but I am slightly worried that we might actually
never scan the old bucket (say for queries with Limit clause) in which
case it might give false positives for insertions in old buckets.

I have changed the patch to address this point by acquiring predicate
lock in _hash_readnext where it will acquire the lock only when it
tries to scan the old bucket. I have also addressed another problem
related to transfer of predicate locks during split such that it will
transfer locks only when there is any tuple transferred from old to
the new bucket.

I'm wondering how to test all this. I'm thinking about a program that
repeatedly creates a hash index and then slowly adds more things to it
so that buckets split (maybe using distinct keys carefully crafted to
hit the same bucket?), while concurrently hammering it with a ton of
scans and then ... somehow checking correctness...

Yeah, that will generate the required errors, but not sure how to
verify correctness. One idea could be that when the split is in
progress, we somehow stop it in-between (say by cancel request) and
then run targeted selects and inserts on the bucket being scanned and
bucket being populated.

I have verified the bucket split scenario manually as below:

Setup
------------
create table hash_tbl(id int4, p integer);
insert into hash_tbl (id, p) select g, 10 from generate_series(1, 10) g;
Analyze hash_tbl;
create index hash_idx on hash_tbl using hash(p);

Session-1
----------------
begin isolation level serializable;
set enable_seqscan=off;
set enable_bitmapscan=off;
set enable_indexonlyscan=on;
select sum(p) from hash_tbl where p=10;
sum
-----
100
(1 row)

insert into hash_tbl (id, p) select g, 10 from generate_series(10, 1000) g;
-- Via debugger, stop in _hash_splitbucket at line 1283 {..
LockBuffer(bucket_obuf, BUFFER_LOCK_EXCLUSIVE); ...}

By this time split of bucket 1 is done but the split flag is not
cleared. So, predicate lock from bucket-1 have been transferred to
bucket-3 (new bucket).

Session-2
----------------
begin isolation level serializable;
set enable_seqscan=off;
set enable_bitmapscan=off;
set enable_indexonlyscan=on;
select sum(p) from hash_tbl where p=10;
sum
-----
100
(1 row)

Session-1
--------------
Commit;

Session-2
----------
postgres=# insert into hash_tbl (id, p) select g, 10 from
generate_series(51, 60) g;
ERROR: could not serialize access due to read/write dependencies
among transactions
DETAIL: Reason code: Canceled on identification as a pivot, during write.
HINT: The transaction might succeed if retried.

It got conflict while inserting in the new bucket (bucket number -3).

I think this patch now addresses all the open issues. Let me know what
do you think about it?

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Attachments:

Predicate-Locking-in-hash-index_v6.patchapplication/octet-stream; name=Predicate-Locking-in-hash-index_v6.patchDownload
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index e337439adad..4f2ea7955f9 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -68,7 +68,7 @@ hashhandler(PG_FUNCTION_ARGS)
 	amroutine->amsearchnulls = false;
 	amroutine->amstorage = false;
 	amroutine->amclusterable = false;
-	amroutine->ampredlocks = false;
+	amroutine->ampredlocks = true;
 	amroutine->amcanparallel = false;
 	amroutine->amkeytype = INT4OID;
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index f121286b8ca..3eb722ce266 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -22,6 +22,7 @@
 #include "utils/rel.h"
 #include "storage/lwlock.h"
 #include "storage/buf_internals.h"
+#include "storage/predicate.h"
 
 static void _hash_vacuum_one_page(Relation rel, Buffer metabuf, Buffer buf,
 					  RelFileNode hnode);
@@ -88,6 +89,8 @@ restart_insert:
 										  &usedmetap);
 	Assert(usedmetap != NULL);
 
+	CheckForSerializableConflictIn(rel, NULL, buf);
+
 	/* remember the primary bucket buffer to release the pin on it at end. */
 	bucket_buf = buf;
 
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index 3859e3bd838..c26cf76285d 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -33,6 +33,7 @@
 #include "miscadmin.h"
 #include "storage/lmgr.h"
 #include "storage/smgr.h"
+#include "storage/predicate.h"
 
 
 static bool _hash_alloc_buckets(Relation rel, BlockNumber firstblock,
@@ -1098,6 +1099,7 @@ _hash_splitbucket(Relation rel,
 	Size		all_tups_size = 0;
 	int			i;
 	uint16		nitups = 0;
+	bool		copied_first_tuple = false;
 
 	bucket_obuf = obuf;
 	opage = BufferGetPage(obuf);
@@ -1157,6 +1159,17 @@ _hash_splitbucket(Relation rel,
 			{
 				IndexTuple	new_itup;
 
+				if (!copied_first_tuple)
+				{
+					/*
+					 * Copy the predicate locks from old bucket to new bucket
+					 * only if we relocate any tuple to the new bucket.
+					 */
+					PredicateLockPageSplit(rel,
+										   BufferGetBlockNumber(bucket_obuf),
+										   BufferGetBlockNumber(bucket_nbuf));
+					copied_first_tuple = true;
+				}
 				/*
 				 * make a copy of index tuple as we have to scribble on it.
 				 */
diff --git a/src/backend/access/hash/hashsearch.c b/src/backend/access/hash/hashsearch.c
index c692c5b32d1..650041db0a5 100644
--- a/src/backend/access/hash/hashsearch.c
+++ b/src/backend/access/hash/hashsearch.c
@@ -19,6 +19,7 @@
 #include "miscadmin.h"
 #include "pgstat.h"
 #include "utils/rel.h"
+#include "storage/predicate.h"
 
 static bool _hash_readpage(IndexScanDesc scan, Buffer *bufP,
 			   ScanDirection dir);
@@ -171,6 +172,7 @@ _hash_readnext(IndexScanDesc scan,
 		Assert(BufferIsValid(*bufp));
 
 		LockBuffer(*bufp, BUFFER_LOCK_SHARE);
+		PredicateLockPage(rel, BufferGetBlockNumber(*bufp), scan->xs_snapshot);
 
 		/*
 		 * setting hashso_buc_split to true indicates that we are scanning
@@ -347,6 +349,7 @@ _hash_first(IndexScanDesc scan, ScanDirection dir)
 	so->hashso_sk_hash = hashkey;
 
 	buf = _hash_getbucketbuf_from_hashkey(rel, hashkey, HASH_READ, NULL);
+	PredicateLockPage(rel, BufferGetBlockNumber(buf), scan->xs_snapshot);
 	page = BufferGetPage(buf);
 	TestForOldSnapshot(scan->xs_snapshot, rel, page);
 	opaque = (HashPageOpaque) PageGetSpecialPointer(page);
diff --git a/src/backend/storage/lmgr/README-SSI b/src/backend/storage/lmgr/README-SSI
index a9dc01f237b..739efec81eb 100644
--- a/src/backend/storage/lmgr/README-SSI
+++ b/src/backend/storage/lmgr/README-SSI
@@ -379,6 +379,11 @@ level during a GiST search. An index insert at the leaf level can
 then be trusted to ripple up to all levels and locations where
 conflicting predicate locks may exist.
 
+    * Hash index searches acquire predicate locks on the primary
+page of a bucket. During a bucket split, a predicate lock is copied from
+the primary page of an old bucket to the primary page of a new bucket.
+
+
     * The effects of page splits, overflows, consolidations, and
 removals must be carefully reviewed to ensure that predicate locks
 aren't "lost" during those operations, or kept with pages which could
diff --git a/src/test/isolation/expected/predicate-hash.out b/src/test/isolation/expected/predicate-hash.out
new file mode 100644
index 00000000000..53e500fd26f
--- /dev/null
+++ b/src/test/isolation/expected/predicate-hash.out
@@ -0,0 +1,659 @@
+Parsed test spec with 2 sessions
+
+starting permutation: rxy1 wx1 c1 rxy2 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+600            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy2 wy2 c2 rxy1 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+400            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy3 wx3 c1 rxy4 wy4 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy4 wy4 c2 rxy3 wx3 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy1 wx1 rxy2 c1 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy1 wx1 rxy2 wy2 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 wx1 rxy2 wy2 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wx1 c1 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy1 rxy2 wx1 wy2 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wx1 wy2 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 wx1 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 wx1 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 c2 wx1 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy2 rxy1 wx1 c1 wy2 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy2 rxy1 wx1 wy2 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wx1 wy2 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 wx1 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 wx1 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 c2 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy2 wy2 rxy1 wx1 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 wy2 rxy1 wx1 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 wy2 rxy1 c2 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy3 wx3 rxy4 c1 wy4 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step c1: commit;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy3 wx3 rxy4 wy4 c1 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy3 wx3 rxy4 wy4 c2 c1
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy3 rxy4 wx3 c1 wy4 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy3 rxy4 wx3 wy4 c1 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy3 rxy4 wx3 wy4 c2 c1
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy3 rxy4 wy4 wx3 c1 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy3 rxy4 wy4 wx3 c2 c1
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy3 rxy4 wy4 c2 wx3 c1
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy4 rxy3 wx3 c1 wy4 c2
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy4 rxy3 wx3 wy4 c1 c2
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy4 rxy3 wx3 wy4 c2 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy4 rxy3 wy4 wx3 c1 c2
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy4 rxy3 wy4 wx3 c2 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy4 rxy3 wy4 c2 wx3 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy4 wy4 rxy3 wx3 c1 c2
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy4 wy4 rxy3 wx3 c2 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy4 wy4 rxy3 c2 wx3 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step c2: commit;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 74d7d59546a..cb3b0d8cb22 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -66,3 +66,4 @@ test: async-notify
 test: vacuum-reltuples
 test: timeouts
 test: vacuum-concurrent-drop
+test: predicate-hash
diff --git a/src/test/isolation/specs/predicate-hash.spec b/src/test/isolation/specs/predicate-hash.spec
new file mode 100644
index 00000000000..8c83cb33e19
--- /dev/null
+++ b/src/test/isolation/specs/predicate-hash.spec
@@ -0,0 +1,122 @@
+# Test for page level predicate locking in hash index
+#
+# Test to verify serialization failures and to check reduced false positives
+#
+# To verify serialization failures, queries and permutations are written in such
+# a way that an index scan  (from one transaction) and an index insert (from
+# another transaction) will try to access the same bucket of the index
+# whereas to check reduced false positives, they will try to access different
+# buckets of the index.
+
+setup
+{
+ create table hash_tbl(id int4, p integer);
+ create index hash_idx on hash_tbl using hash(p);
+ insert into hash_tbl (id, p)
+ select g, 10 from generate_series(1, 10) g;
+ insert into hash_tbl (id, p)
+ select g, 20 from generate_series(11, 20) g;
+ insert into hash_tbl (id, p)
+ select g, 30 from generate_series(21, 30) g;
+ insert into hash_tbl (id, p)
+ select g, 40 from generate_series(31, 40) g;
+}
+
+teardown
+{
+ drop table hash_tbl;
+}
+
+session "s1"
+setup		
+{
+ begin isolation level serializable;
+ set enable_seqscan=off;
+ set enable_bitmapscan=off;
+ set enable_indexonlyscan=on;
+}
+step "rxy1"	{ select sum(p) from hash_tbl where p=20; }
+step "wx1"	{ insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g; }
+step "rxy3"	{ select sum(p) from hash_tbl where p=20; }
+step "wx3"	{ insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g; }
+step "c1"	{ commit; }
+
+
+session "s2"
+setup		
+{
+ begin isolation level serializable;
+ set enable_seqscan=off;
+ set enable_bitmapscan=off;
+ set enable_indexonlyscan=on;
+}
+step "rxy2"	{ select sum(p) from hash_tbl where p=30; }
+step "wy2"	{ insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g; }
+step "rxy4"	{ select sum(p) from hash_tbl where p=30; }
+step "wy4"	{ insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g; }
+step "c2"	{ commit; }
+
+
+# An index scan (from one transaction) and an index insert (from another
+# transaction) try to access the same bucket of the index but one transaction
+# commits before other transaction begins so no r-w conflict.
+
+permutation "rxy1" "wx1" "c1" "rxy2" "wy2" "c2"
+permutation "rxy2" "wy2" "c2" "rxy1" "wx1" "c1"
+
+# An index scan (from one transaction) and an index insert (from another
+# transaction) try to access different buckets of the index and also one
+# transaction commits before other transaction begins, so no r-w conflict.
+
+permutation "rxy3" "wx3" "c1" "rxy4" "wy4" "c2"
+permutation "rxy4" "wy4" "c2" "rxy3" "wx3" "c1"
+
+
+# An index scan (from one transaction) and an index insert (from another
+# transaction) try to access the same bucket of the index and one transaction
+# begins before other transaction commits so there is a r-w conflict.
+
+permutation "rxy1" "wx1" "rxy2" "c1" "wy2" "c2"
+permutation "rxy1" "wx1" "rxy2" "wy2" "c1" "c2"
+permutation "rxy1" "wx1" "rxy2" "wy2" "c2" "c1"
+permutation "rxy1" "rxy2" "wx1" "c1" "wy2" "c2"
+permutation "rxy1" "rxy2" "wx1" "wy2" "c1" "c2"
+permutation "rxy1" "rxy2" "wx1" "wy2" "c2" "c1"
+permutation "rxy1" "rxy2" "wy2" "wx1" "c1" "c2"
+permutation "rxy1" "rxy2" "wy2" "wx1" "c2" "c1"
+permutation "rxy1" "rxy2" "wy2" "c2" "wx1" "c1"
+permutation "rxy2" "rxy1" "wx1" "c1" "wy2" "c2"
+permutation "rxy2" "rxy1" "wx1" "wy2" "c1" "c2"
+permutation "rxy2" "rxy1" "wx1" "wy2" "c2" "c1"
+permutation "rxy2" "rxy1" "wy2" "wx1" "c1" "c2"
+permutation "rxy2" "rxy1" "wy2" "wx1" "c2" "c1"
+permutation "rxy2" "rxy1" "wy2" "c2" "wx1" "c1"
+permutation "rxy2" "wy2" "rxy1" "wx1" "c1" "c2"
+permutation "rxy2" "wy2" "rxy1" "wx1" "c2" "c1"
+permutation "rxy2" "wy2" "rxy1" "c2" "wx1" "c1"
+
+# An index scan (from one transaction) and an index insert (from another 
+# transaction) try to access different buckets of the index so no r-w conflict.
+
+permutation "rxy3" "wx3" "rxy4" "c1" "wy4" "c2"
+permutation "rxy3" "wx3" "rxy4" "wy4" "c1" "c2"
+permutation "rxy3" "wx3" "rxy4" "wy4" "c2" "c1"
+permutation "rxy3" "rxy4" "wx3" "c1" "wy4" "c2"
+permutation "rxy3" "rxy4" "wx3" "wy4" "c1" "c2"
+permutation "rxy3" "rxy4" "wx3" "wy4" "c2" "c1"
+permutation "rxy3" "rxy4" "wy4" "wx3" "c1" "c2"
+permutation "rxy3" "rxy4" "wy4" "wx3" "c2" "c1"
+permutation "rxy3" "rxy4" "wy4" "c2" "wx3" "c1"
+permutation "rxy4" "rxy3" "wx3" "c1" "wy4" "c2"
+permutation "rxy4" "rxy3" "wx3" "wy4" "c1" "c2"
+permutation "rxy4" "rxy3" "wx3" "wy4" "c2" "c1"
+permutation "rxy4" "rxy3" "wy4" "wx3" "c1" "c2"
+permutation "rxy4" "rxy3" "wy4" "wx3" "c2" "c1"
+permutation "rxy4" "rxy3" "wy4" "c2" "wx3" "c1"
+permutation "rxy4" "wy4" "rxy3" "wx3" "c1" "c2"
+permutation "rxy4" "wy4" "rxy3" "wx3" "c2" "c1"
+permutation "rxy4" "wy4" "rxy3" "c2" "wx3" "c1"
#21Amit Kapila
amit.kapila16@gmail.com
In reply to: Amit Kapila (#20)
1 attachment(s)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Sat, Mar 10, 2018 at 1:10 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:

On Tue, Mar 6, 2018 at 11:26 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:

On Mon, Mar 5, 2018 at 8:58 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Sun, Mar 4, 2018 at 12:53 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:

Yes, but I think it would be better if we call this once we are sure
that at least one tuple from the old bucket has been transferred
(consider if all tuples in the old bucket are dead). Apart from this,
I think this patch has missed handling the cases where we scan the
buckets when the split is in progress. In such cases, we scan both
old and new bucket, so I think we need to ensure that we take
PredicateLock on both the buckets during such scans.

Hmm. Yeah.

So, in _hash_first(), do you think we might just need this?

if (H_BUCKET_BEING_POPULATED(opaque))
{
...
old_blkno = _hash_get_oldblock_from_newbucket(rel, bucket);
...
old_buf = _hash_getbuf(rel, old_blkno, HASH_READ, LH_BUCKET_PAGE);
+ PredicateLockPage(rel, BufferGetBlockNumber(old_buf),
scan->xs_snapshot);
TestForOldSnapshot(scan->xs_snapshot, rel, BufferGetPage(old_buf));

That is, if you begin scanning a 'new' bucket, we remember the old
bucket and go and scan that too, so we'd better predicate-lock both up
front (or I suppose we could do it later when we visit that page, but
here it can be done in a single place).

Yeah, that can work, but I am slightly worried that we might actually
never scan the old bucket (say for queries with Limit clause) in which
case it might give false positives for insertions in old buckets.

I have changed the patch to address this point by acquiring predicate
lock in _hash_readnext where it will acquire the lock only when it
tries to scan the old bucket. I have also addressed another problem
related to transfer of predicate locks during split such that it will
transfer locks only when there is any tuple transferred from old to
the new bucket.

Added some additional text in README in the attached patch to explain
the new change in mechanism.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Attachments:

Predicate-Locking-in-hash-index_v7.patchapplication/octet-stream; name=Predicate-Locking-in-hash-index_v7.patchDownload
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index e337439..4f2ea79 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -68,7 +68,7 @@ hashhandler(PG_FUNCTION_ARGS)
 	amroutine->amsearchnulls = false;
 	amroutine->amstorage = false;
 	amroutine->amclusterable = false;
-	amroutine->ampredlocks = false;
+	amroutine->ampredlocks = true;
 	amroutine->amcanparallel = false;
 	amroutine->amkeytype = INT4OID;
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index f121286..3eb722c 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -22,6 +22,7 @@
 #include "utils/rel.h"
 #include "storage/lwlock.h"
 #include "storage/buf_internals.h"
+#include "storage/predicate.h"
 
 static void _hash_vacuum_one_page(Relation rel, Buffer metabuf, Buffer buf,
 					  RelFileNode hnode);
@@ -88,6 +89,8 @@ restart_insert:
 										  &usedmetap);
 	Assert(usedmetap != NULL);
 
+	CheckForSerializableConflictIn(rel, NULL, buf);
+
 	/* remember the primary bucket buffer to release the pin on it at end. */
 	bucket_buf = buf;
 
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index 3859e3b..c26cf76 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -33,6 +33,7 @@
 #include "miscadmin.h"
 #include "storage/lmgr.h"
 #include "storage/smgr.h"
+#include "storage/predicate.h"
 
 
 static bool _hash_alloc_buckets(Relation rel, BlockNumber firstblock,
@@ -1098,6 +1099,7 @@ _hash_splitbucket(Relation rel,
 	Size		all_tups_size = 0;
 	int			i;
 	uint16		nitups = 0;
+	bool		copied_first_tuple = false;
 
 	bucket_obuf = obuf;
 	opage = BufferGetPage(obuf);
@@ -1157,6 +1159,17 @@ _hash_splitbucket(Relation rel,
 			{
 				IndexTuple	new_itup;
 
+				if (!copied_first_tuple)
+				{
+					/*
+					 * Copy the predicate locks from old bucket to new bucket
+					 * only if we relocate any tuple to the new bucket.
+					 */
+					PredicateLockPageSplit(rel,
+										   BufferGetBlockNumber(bucket_obuf),
+										   BufferGetBlockNumber(bucket_nbuf));
+					copied_first_tuple = true;
+				}
 				/*
 				 * make a copy of index tuple as we have to scribble on it.
 				 */
diff --git a/src/backend/access/hash/hashsearch.c b/src/backend/access/hash/hashsearch.c
index c692c5b..650041d 100644
--- a/src/backend/access/hash/hashsearch.c
+++ b/src/backend/access/hash/hashsearch.c
@@ -19,6 +19,7 @@
 #include "miscadmin.h"
 #include "pgstat.h"
 #include "utils/rel.h"
+#include "storage/predicate.h"
 
 static bool _hash_readpage(IndexScanDesc scan, Buffer *bufP,
 			   ScanDirection dir);
@@ -171,6 +172,7 @@ _hash_readnext(IndexScanDesc scan,
 		Assert(BufferIsValid(*bufp));
 
 		LockBuffer(*bufp, BUFFER_LOCK_SHARE);
+		PredicateLockPage(rel, BufferGetBlockNumber(*bufp), scan->xs_snapshot);
 
 		/*
 		 * setting hashso_buc_split to true indicates that we are scanning
@@ -347,6 +349,7 @@ _hash_first(IndexScanDesc scan, ScanDirection dir)
 	so->hashso_sk_hash = hashkey;
 
 	buf = _hash_getbucketbuf_from_hashkey(rel, hashkey, HASH_READ, NULL);
+	PredicateLockPage(rel, BufferGetBlockNumber(buf), scan->xs_snapshot);
 	page = BufferGetPage(buf);
 	TestForOldSnapshot(scan->xs_snapshot, rel, page);
 	opaque = (HashPageOpaque) PageGetSpecialPointer(page);
diff --git a/src/backend/storage/lmgr/README-SSI b/src/backend/storage/lmgr/README-SSI
index a9dc01f..a730506 100644
--- a/src/backend/storage/lmgr/README-SSI
+++ b/src/backend/storage/lmgr/README-SSI
@@ -379,6 +379,13 @@ level during a GiST search. An index insert at the leaf level can
 then be trusted to ripple up to all levels and locations where
 conflicting predicate locks may exist.
 
+    * Hash index searches acquire predicate locks on the primary
+page of a bucket. It acquires a lock on both the old and new buckets
+for scans that happen concurrently with page splits. During a bucket
+split, a predicate lock is copied from the primary page of an old
+bucket to the primary page of a new bucket.
+
+
     * The effects of page splits, overflows, consolidations, and
 removals must be carefully reviewed to ensure that predicate locks
 aren't "lost" during those operations, or kept with pages which could
diff --git a/src/test/isolation/expected/predicate-hash.out b/src/test/isolation/expected/predicate-hash.out
new file mode 100644
index 0000000..53e500f
--- /dev/null
+++ b/src/test/isolation/expected/predicate-hash.out
@@ -0,0 +1,659 @@
+Parsed test spec with 2 sessions
+
+starting permutation: rxy1 wx1 c1 rxy2 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+600            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy2 wy2 c2 rxy1 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+400            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy3 wx3 c1 rxy4 wy4 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy4 wy4 c2 rxy3 wx3 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy1 wx1 rxy2 c1 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy1 wx1 rxy2 wy2 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 wx1 rxy2 wy2 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wx1 c1 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy1 rxy2 wx1 wy2 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wx1 wy2 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 wx1 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 wx1 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 c2 wx1 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy2 rxy1 wx1 c1 wy2 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy2 rxy1 wx1 wy2 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wx1 wy2 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 wx1 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 wx1 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 c2 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy2 wy2 rxy1 wx1 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 wy2 rxy1 wx1 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 wy2 rxy1 c2 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy3 wx3 rxy4 c1 wy4 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step c1: commit;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy3 wx3 rxy4 wy4 c1 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy3 wx3 rxy4 wy4 c2 c1
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy3 rxy4 wx3 c1 wy4 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy3 rxy4 wx3 wy4 c1 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy3 rxy4 wx3 wy4 c2 c1
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy3 rxy4 wy4 wx3 c1 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy3 rxy4 wy4 wx3 c2 c1
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy3 rxy4 wy4 c2 wx3 c1
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy4 rxy3 wx3 c1 wy4 c2
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy4 rxy3 wx3 wy4 c1 c2
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy4 rxy3 wx3 wy4 c2 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy4 rxy3 wy4 wx3 c1 c2
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy4 rxy3 wy4 wx3 c2 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy4 rxy3 wy4 c2 wx3 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy4 wy4 rxy3 wx3 c1 c2
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy4 wy4 rxy3 wx3 c2 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy4 wy4 rxy3 c2 wx3 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step c2: commit;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 74d7d59..cb3b0d8 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -66,3 +66,4 @@ test: async-notify
 test: vacuum-reltuples
 test: timeouts
 test: vacuum-concurrent-drop
+test: predicate-hash
diff --git a/src/test/isolation/specs/predicate-hash.spec b/src/test/isolation/specs/predicate-hash.spec
new file mode 100644
index 0000000..8c83cb3
--- /dev/null
+++ b/src/test/isolation/specs/predicate-hash.spec
@@ -0,0 +1,122 @@
+# Test for page level predicate locking in hash index
+#
+# Test to verify serialization failures and to check reduced false positives
+#
+# To verify serialization failures, queries and permutations are written in such
+# a way that an index scan  (from one transaction) and an index insert (from
+# another transaction) will try to access the same bucket of the index
+# whereas to check reduced false positives, they will try to access different
+# buckets of the index.
+
+setup
+{
+ create table hash_tbl(id int4, p integer);
+ create index hash_idx on hash_tbl using hash(p);
+ insert into hash_tbl (id, p)
+ select g, 10 from generate_series(1, 10) g;
+ insert into hash_tbl (id, p)
+ select g, 20 from generate_series(11, 20) g;
+ insert into hash_tbl (id, p)
+ select g, 30 from generate_series(21, 30) g;
+ insert into hash_tbl (id, p)
+ select g, 40 from generate_series(31, 40) g;
+}
+
+teardown
+{
+ drop table hash_tbl;
+}
+
+session "s1"
+setup		
+{
+ begin isolation level serializable;
+ set enable_seqscan=off;
+ set enable_bitmapscan=off;
+ set enable_indexonlyscan=on;
+}
+step "rxy1"	{ select sum(p) from hash_tbl where p=20; }
+step "wx1"	{ insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g; }
+step "rxy3"	{ select sum(p) from hash_tbl where p=20; }
+step "wx3"	{ insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g; }
+step "c1"	{ commit; }
+
+
+session "s2"
+setup		
+{
+ begin isolation level serializable;
+ set enable_seqscan=off;
+ set enable_bitmapscan=off;
+ set enable_indexonlyscan=on;
+}
+step "rxy2"	{ select sum(p) from hash_tbl where p=30; }
+step "wy2"	{ insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g; }
+step "rxy4"	{ select sum(p) from hash_tbl where p=30; }
+step "wy4"	{ insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g; }
+step "c2"	{ commit; }
+
+
+# An index scan (from one transaction) and an index insert (from another
+# transaction) try to access the same bucket of the index but one transaction
+# commits before other transaction begins so no r-w conflict.
+
+permutation "rxy1" "wx1" "c1" "rxy2" "wy2" "c2"
+permutation "rxy2" "wy2" "c2" "rxy1" "wx1" "c1"
+
+# An index scan (from one transaction) and an index insert (from another
+# transaction) try to access different buckets of the index and also one
+# transaction commits before other transaction begins, so no r-w conflict.
+
+permutation "rxy3" "wx3" "c1" "rxy4" "wy4" "c2"
+permutation "rxy4" "wy4" "c2" "rxy3" "wx3" "c1"
+
+
+# An index scan (from one transaction) and an index insert (from another
+# transaction) try to access the same bucket of the index and one transaction
+# begins before other transaction commits so there is a r-w conflict.
+
+permutation "rxy1" "wx1" "rxy2" "c1" "wy2" "c2"
+permutation "rxy1" "wx1" "rxy2" "wy2" "c1" "c2"
+permutation "rxy1" "wx1" "rxy2" "wy2" "c2" "c1"
+permutation "rxy1" "rxy2" "wx1" "c1" "wy2" "c2"
+permutation "rxy1" "rxy2" "wx1" "wy2" "c1" "c2"
+permutation "rxy1" "rxy2" "wx1" "wy2" "c2" "c1"
+permutation "rxy1" "rxy2" "wy2" "wx1" "c1" "c2"
+permutation "rxy1" "rxy2" "wy2" "wx1" "c2" "c1"
+permutation "rxy1" "rxy2" "wy2" "c2" "wx1" "c1"
+permutation "rxy2" "rxy1" "wx1" "c1" "wy2" "c2"
+permutation "rxy2" "rxy1" "wx1" "wy2" "c1" "c2"
+permutation "rxy2" "rxy1" "wx1" "wy2" "c2" "c1"
+permutation "rxy2" "rxy1" "wy2" "wx1" "c1" "c2"
+permutation "rxy2" "rxy1" "wy2" "wx1" "c2" "c1"
+permutation "rxy2" "rxy1" "wy2" "c2" "wx1" "c1"
+permutation "rxy2" "wy2" "rxy1" "wx1" "c1" "c2"
+permutation "rxy2" "wy2" "rxy1" "wx1" "c2" "c1"
+permutation "rxy2" "wy2" "rxy1" "c2" "wx1" "c1"
+
+# An index scan (from one transaction) and an index insert (from another 
+# transaction) try to access different buckets of the index so no r-w conflict.
+
+permutation "rxy3" "wx3" "rxy4" "c1" "wy4" "c2"
+permutation "rxy3" "wx3" "rxy4" "wy4" "c1" "c2"
+permutation "rxy3" "wx3" "rxy4" "wy4" "c2" "c1"
+permutation "rxy3" "rxy4" "wx3" "c1" "wy4" "c2"
+permutation "rxy3" "rxy4" "wx3" "wy4" "c1" "c2"
+permutation "rxy3" "rxy4" "wx3" "wy4" "c2" "c1"
+permutation "rxy3" "rxy4" "wy4" "wx3" "c1" "c2"
+permutation "rxy3" "rxy4" "wy4" "wx3" "c2" "c1"
+permutation "rxy3" "rxy4" "wy4" "c2" "wx3" "c1"
+permutation "rxy4" "rxy3" "wx3" "c1" "wy4" "c2"
+permutation "rxy4" "rxy3" "wx3" "wy4" "c1" "c2"
+permutation "rxy4" "rxy3" "wx3" "wy4" "c2" "c1"
+permutation "rxy4" "rxy3" "wy4" "wx3" "c1" "c2"
+permutation "rxy4" "rxy3" "wy4" "wx3" "c2" "c1"
+permutation "rxy4" "rxy3" "wy4" "c2" "wx3" "c1"
+permutation "rxy4" "wy4" "rxy3" "wx3" "c1" "c2"
+permutation "rxy4" "wy4" "rxy3" "wx3" "c2" "c1"
+permutation "rxy4" "wy4" "rxy3" "c2" "wx3" "c1"
#22Alexander Korotkov
a.korotkov@postgrespro.ru
In reply to: Thomas Munro (#16)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Fri, Mar 2, 2018 at 6:57 AM, Thomas Munro <thomas.munro@enterprisedb.com>
wrote:

My thought experiments about pseudo-pages and avoiding the split stuff
were not intended to get the patch kicked out. I thought for a while
that hash indexes were a special case and could benefit from
dispensing with those trickier problems. Upon further reflection, for
interesting size hash indexes pure hash value predicate tags wouldn't
be much better. Furthermore, if we do decide we want to use using x %
max_predicate_locks_per_relation to avoid having to escalate to
relation predicate locks at the cost of slightly higher collision rate
then we should consider that for the whole system (including heap page
predicate locking), not just hash indexes. Please consider those
ideas parked for now.

OK. While our potential pseudo-pages are identified as
"hash_value % some_constant_modulus", real bucket pages are very roughly
identified as "hash_value % number_of_index_pages". So, page number is
adoptive to index size, despite it costs us handling page split. In the
same way,
locking in other index access methods is adoptive to an index size, so
that should be considered as useful feature which should be present in hash
index
as well.

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

#23Alexander Korotkov
a.korotkov@postgrespro.ru
In reply to: Amit Kapila (#17)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Sat, Mar 3, 2018 at 2:53 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:

On Fri, Mar 2, 2018 at 9:27 AM, Thomas Munro

If that is indeed a race, could it be fixed by
calling PredicateLockPageSplit() at the start of _hash_splitbucket()
instead?

Yes, but I think it would be better if we call this once we are sure
that at least one tuple from the old bucket has been transferred
(consider if all tuples in the old bucket are dead).

Is it really fair? For example, predicate lock can be held by session
which queried some key, but didn't find any corresponding tuple.
If we imagine this key should be in new bucket while all existing
tuples would be left in old bucket. As I get, in this case no locks
would be transferred since no tuples were moved to the new bucket.
So, further insertion to the new bucket wouldn't conflict with session,
which looked for non-existing key, while it should. Do it make sense?

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

#24Amit Kapila
amit.kapila16@gmail.com
In reply to: Alexander Korotkov (#23)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Mon, Mar 12, 2018 at 12:18 AM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:

On Sat, Mar 3, 2018 at 2:53 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:

On Fri, Mar 2, 2018 at 9:27 AM, Thomas Munro

If that is indeed a race, could it be fixed by
calling PredicateLockPageSplit() at the start of _hash_splitbucket()
instead?

Yes, but I think it would be better if we call this once we are sure
that at least one tuple from the old bucket has been transferred
(consider if all tuples in the old bucket are dead).

Is it really fair? For example, predicate lock can be held by session
which queried some key, but didn't find any corresponding tuple.
If we imagine this key should be in new bucket while all existing
tuples would be left in old bucket. As I get, in this case no locks
would be transferred since no tuples were moved to the new bucket.
So, further insertion to the new bucket wouldn't conflict with session,
which looked for non-existing key, while it should. Do it make sense?

Valid point, I think on split we should always transfer locks from old
bucket to new bucket.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

#25Amit Kapila
amit.kapila16@gmail.com
In reply to: Amit Kapila (#24)
1 attachment(s)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Mon, Mar 12, 2018 at 7:18 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:

On Mon, Mar 12, 2018 at 12:18 AM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:

On Sat, Mar 3, 2018 at 2:53 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:

On Fri, Mar 2, 2018 at 9:27 AM, Thomas Munro

If that is indeed a race, could it be fixed by
calling PredicateLockPageSplit() at the start of _hash_splitbucket()
instead?

Yes, but I think it would be better if we call this once we are sure
that at least one tuple from the old bucket has been transferred
(consider if all tuples in the old bucket are dead).

Is it really fair? For example, predicate lock can be held by session
which queried some key, but didn't find any corresponding tuple.
If we imagine this key should be in new bucket while all existing
tuples would be left in old bucket. As I get, in this case no locks
would be transferred since no tuples were moved to the new bucket.
So, further insertion to the new bucket wouldn't conflict with session,
which looked for non-existing key, while it should. Do it make sense?

Valid point, I think on split we should always transfer locks from old
bucket to new bucket.

Attached patch changes it as per above suggestion.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Attachments:

Predicate-Locking-in-hash-index_v8.patchapplication/octet-stream; name=Predicate-Locking-in-hash-index_v8.patchDownload
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index e337439..4f2ea79 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -68,7 +68,7 @@ hashhandler(PG_FUNCTION_ARGS)
 	amroutine->amsearchnulls = false;
 	amroutine->amstorage = false;
 	amroutine->amclusterable = false;
-	amroutine->ampredlocks = false;
+	amroutine->ampredlocks = true;
 	amroutine->amcanparallel = false;
 	amroutine->amkeytype = INT4OID;
 
diff --git a/src/backend/access/hash/hashinsert.c b/src/backend/access/hash/hashinsert.c
index f121286..3eb722c 100644
--- a/src/backend/access/hash/hashinsert.c
+++ b/src/backend/access/hash/hashinsert.c
@@ -22,6 +22,7 @@
 #include "utils/rel.h"
 #include "storage/lwlock.h"
 #include "storage/buf_internals.h"
+#include "storage/predicate.h"
 
 static void _hash_vacuum_one_page(Relation rel, Buffer metabuf, Buffer buf,
 					  RelFileNode hnode);
@@ -88,6 +89,8 @@ restart_insert:
 										  &usedmetap);
 	Assert(usedmetap != NULL);
 
+	CheckForSerializableConflictIn(rel, NULL, buf);
+
 	/* remember the primary bucket buffer to release the pin on it at end. */
 	bucket_buf = buf;
 
diff --git a/src/backend/access/hash/hashpage.c b/src/backend/access/hash/hashpage.c
index 3859e3b..3ec29a5 100644
--- a/src/backend/access/hash/hashpage.c
+++ b/src/backend/access/hash/hashpage.c
@@ -33,6 +33,7 @@
 #include "miscadmin.h"
 #include "storage/lmgr.h"
 #include "storage/smgr.h"
+#include "storage/predicate.h"
 
 
 static bool _hash_alloc_buckets(Relation rel, BlockNumber firstblock,
@@ -1107,6 +1108,11 @@ _hash_splitbucket(Relation rel,
 	npage = BufferGetPage(nbuf);
 	nopaque = (HashPageOpaque) PageGetSpecialPointer(npage);
 
+	/* Copy the predicate locks from old bucket to new bucket. */
+	PredicateLockPageSplit(rel,
+						   BufferGetBlockNumber(bucket_obuf),
+						   BufferGetBlockNumber(bucket_nbuf));
+
 	/*
 	 * Partition the tuples in the old bucket between the old bucket and the
 	 * new bucket, advancing along the old bucket's overflow bucket chain and
diff --git a/src/backend/access/hash/hashsearch.c b/src/backend/access/hash/hashsearch.c
index c692c5b..650041d 100644
--- a/src/backend/access/hash/hashsearch.c
+++ b/src/backend/access/hash/hashsearch.c
@@ -19,6 +19,7 @@
 #include "miscadmin.h"
 #include "pgstat.h"
 #include "utils/rel.h"
+#include "storage/predicate.h"
 
 static bool _hash_readpage(IndexScanDesc scan, Buffer *bufP,
 			   ScanDirection dir);
@@ -171,6 +172,7 @@ _hash_readnext(IndexScanDesc scan,
 		Assert(BufferIsValid(*bufp));
 
 		LockBuffer(*bufp, BUFFER_LOCK_SHARE);
+		PredicateLockPage(rel, BufferGetBlockNumber(*bufp), scan->xs_snapshot);
 
 		/*
 		 * setting hashso_buc_split to true indicates that we are scanning
@@ -347,6 +349,7 @@ _hash_first(IndexScanDesc scan, ScanDirection dir)
 	so->hashso_sk_hash = hashkey;
 
 	buf = _hash_getbucketbuf_from_hashkey(rel, hashkey, HASH_READ, NULL);
+	PredicateLockPage(rel, BufferGetBlockNumber(buf), scan->xs_snapshot);
 	page = BufferGetPage(buf);
 	TestForOldSnapshot(scan->xs_snapshot, rel, page);
 	opaque = (HashPageOpaque) PageGetSpecialPointer(page);
diff --git a/src/backend/storage/lmgr/README-SSI b/src/backend/storage/lmgr/README-SSI
index a9dc01f..a730506 100644
--- a/src/backend/storage/lmgr/README-SSI
+++ b/src/backend/storage/lmgr/README-SSI
@@ -379,6 +379,13 @@ level during a GiST search. An index insert at the leaf level can
 then be trusted to ripple up to all levels and locations where
 conflicting predicate locks may exist.
 
+    * Hash index searches acquire predicate locks on the primary
+page of a bucket. It acquires a lock on both the old and new buckets
+for scans that happen concurrently with page splits. During a bucket
+split, a predicate lock is copied from the primary page of an old
+bucket to the primary page of a new bucket.
+
+
     * The effects of page splits, overflows, consolidations, and
 removals must be carefully reviewed to ensure that predicate locks
 aren't "lost" during those operations, or kept with pages which could
diff --git a/src/test/isolation/expected/predicate-hash.out b/src/test/isolation/expected/predicate-hash.out
new file mode 100644
index 0000000..53e500f
--- /dev/null
+++ b/src/test/isolation/expected/predicate-hash.out
@@ -0,0 +1,659 @@
+Parsed test spec with 2 sessions
+
+starting permutation: rxy1 wx1 c1 rxy2 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+600            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy2 wy2 c2 rxy1 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+400            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy3 wx3 c1 rxy4 wy4 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy4 wy4 c2 rxy3 wx3 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy1 wx1 rxy2 c1 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy1 wx1 rxy2 wy2 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 wx1 rxy2 wy2 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wx1 c1 wy2 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy1 rxy2 wx1 wy2 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wx1 wy2 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 wx1 c1 c2
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 wx1 c2 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy1 rxy2 wy2 c2 wx1 c1
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy2 rxy1 wx1 c1 wy2 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c2: commit;
+
+starting permutation: rxy2 rxy1 wx1 wy2 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wx1 wy2 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 wx1 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 wx1 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 rxy1 wy2 c2 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy2 wy2 rxy1 wx1 c1 c2
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 wy2 rxy1 wx1 c2 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+
+starting permutation: rxy2 wy2 rxy1 c2 wx1 c1
+step rxy2: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy2: insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g;
+step rxy1: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step c2: commit;
+step wx1: insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g;
+ERROR:  could not serialize access due to read/write dependencies among transactions
+step c1: commit;
+
+starting permutation: rxy3 wx3 rxy4 c1 wy4 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step c1: commit;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy3 wx3 rxy4 wy4 c1 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy3 wx3 rxy4 wy4 c2 c1
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy3 rxy4 wx3 c1 wy4 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy3 rxy4 wx3 wy4 c1 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy3 rxy4 wx3 wy4 c2 c1
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy3 rxy4 wy4 wx3 c1 c2
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy3 rxy4 wy4 wx3 c2 c1
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy3 rxy4 wy4 c2 wx3 c1
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy4 rxy3 wx3 c1 wy4 c2
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+
+starting permutation: rxy4 rxy3 wx3 wy4 c1 c2
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy4 rxy3 wx3 wy4 c2 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy4 rxy3 wy4 wx3 c1 c2
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy4 rxy3 wy4 wx3 c2 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy4 rxy3 wy4 c2 wx3 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step c2: commit;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+
+starting permutation: rxy4 wy4 rxy3 wx3 c1 c2
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
+step c2: commit;
+
+starting permutation: rxy4 wy4 rxy3 wx3 c2 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c2: commit;
+step c1: commit;
+
+starting permutation: rxy4 wy4 rxy3 c2 wx3 c1
+step rxy4: select sum(p) from hash_tbl where p=30;
+sum            
+
+300            
+step wy4: insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g;
+step rxy3: select sum(p) from hash_tbl where p=20;
+sum            
+
+200            
+step c2: commit;
+step wx3: insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g;
+step c1: commit;
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 74d7d59..cb3b0d8 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -66,3 +66,4 @@ test: async-notify
 test: vacuum-reltuples
 test: timeouts
 test: vacuum-concurrent-drop
+test: predicate-hash
diff --git a/src/test/isolation/specs/predicate-hash.spec b/src/test/isolation/specs/predicate-hash.spec
new file mode 100644
index 0000000..8c83cb3
--- /dev/null
+++ b/src/test/isolation/specs/predicate-hash.spec
@@ -0,0 +1,122 @@
+# Test for page level predicate locking in hash index
+#
+# Test to verify serialization failures and to check reduced false positives
+#
+# To verify serialization failures, queries and permutations are written in such
+# a way that an index scan  (from one transaction) and an index insert (from
+# another transaction) will try to access the same bucket of the index
+# whereas to check reduced false positives, they will try to access different
+# buckets of the index.
+
+setup
+{
+ create table hash_tbl(id int4, p integer);
+ create index hash_idx on hash_tbl using hash(p);
+ insert into hash_tbl (id, p)
+ select g, 10 from generate_series(1, 10) g;
+ insert into hash_tbl (id, p)
+ select g, 20 from generate_series(11, 20) g;
+ insert into hash_tbl (id, p)
+ select g, 30 from generate_series(21, 30) g;
+ insert into hash_tbl (id, p)
+ select g, 40 from generate_series(31, 40) g;
+}
+
+teardown
+{
+ drop table hash_tbl;
+}
+
+session "s1"
+setup		
+{
+ begin isolation level serializable;
+ set enable_seqscan=off;
+ set enable_bitmapscan=off;
+ set enable_indexonlyscan=on;
+}
+step "rxy1"	{ select sum(p) from hash_tbl where p=20; }
+step "wx1"	{ insert into hash_tbl (id, p)
+			  select g, 30 from generate_series(41, 50) g; }
+step "rxy3"	{ select sum(p) from hash_tbl where p=20; }
+step "wx3"	{ insert into hash_tbl (id, p)
+			  select g, 50 from generate_series(41, 50) g; }
+step "c1"	{ commit; }
+
+
+session "s2"
+setup		
+{
+ begin isolation level serializable;
+ set enable_seqscan=off;
+ set enable_bitmapscan=off;
+ set enable_indexonlyscan=on;
+}
+step "rxy2"	{ select sum(p) from hash_tbl where p=30; }
+step "wy2"	{ insert into hash_tbl (id, p)
+			  select g, 20 from generate_series(51, 60) g; }
+step "rxy4"	{ select sum(p) from hash_tbl where p=30; }
+step "wy4"	{ insert into hash_tbl (id, p)
+			  select g, 60 from generate_series(51, 60) g; }
+step "c2"	{ commit; }
+
+
+# An index scan (from one transaction) and an index insert (from another
+# transaction) try to access the same bucket of the index but one transaction
+# commits before other transaction begins so no r-w conflict.
+
+permutation "rxy1" "wx1" "c1" "rxy2" "wy2" "c2"
+permutation "rxy2" "wy2" "c2" "rxy1" "wx1" "c1"
+
+# An index scan (from one transaction) and an index insert (from another
+# transaction) try to access different buckets of the index and also one
+# transaction commits before other transaction begins, so no r-w conflict.
+
+permutation "rxy3" "wx3" "c1" "rxy4" "wy4" "c2"
+permutation "rxy4" "wy4" "c2" "rxy3" "wx3" "c1"
+
+
+# An index scan (from one transaction) and an index insert (from another
+# transaction) try to access the same bucket of the index and one transaction
+# begins before other transaction commits so there is a r-w conflict.
+
+permutation "rxy1" "wx1" "rxy2" "c1" "wy2" "c2"
+permutation "rxy1" "wx1" "rxy2" "wy2" "c1" "c2"
+permutation "rxy1" "wx1" "rxy2" "wy2" "c2" "c1"
+permutation "rxy1" "rxy2" "wx1" "c1" "wy2" "c2"
+permutation "rxy1" "rxy2" "wx1" "wy2" "c1" "c2"
+permutation "rxy1" "rxy2" "wx1" "wy2" "c2" "c1"
+permutation "rxy1" "rxy2" "wy2" "wx1" "c1" "c2"
+permutation "rxy1" "rxy2" "wy2" "wx1" "c2" "c1"
+permutation "rxy1" "rxy2" "wy2" "c2" "wx1" "c1"
+permutation "rxy2" "rxy1" "wx1" "c1" "wy2" "c2"
+permutation "rxy2" "rxy1" "wx1" "wy2" "c1" "c2"
+permutation "rxy2" "rxy1" "wx1" "wy2" "c2" "c1"
+permutation "rxy2" "rxy1" "wy2" "wx1" "c1" "c2"
+permutation "rxy2" "rxy1" "wy2" "wx1" "c2" "c1"
+permutation "rxy2" "rxy1" "wy2" "c2" "wx1" "c1"
+permutation "rxy2" "wy2" "rxy1" "wx1" "c1" "c2"
+permutation "rxy2" "wy2" "rxy1" "wx1" "c2" "c1"
+permutation "rxy2" "wy2" "rxy1" "c2" "wx1" "c1"
+
+# An index scan (from one transaction) and an index insert (from another 
+# transaction) try to access different buckets of the index so no r-w conflict.
+
+permutation "rxy3" "wx3" "rxy4" "c1" "wy4" "c2"
+permutation "rxy3" "wx3" "rxy4" "wy4" "c1" "c2"
+permutation "rxy3" "wx3" "rxy4" "wy4" "c2" "c1"
+permutation "rxy3" "rxy4" "wx3" "c1" "wy4" "c2"
+permutation "rxy3" "rxy4" "wx3" "wy4" "c1" "c2"
+permutation "rxy3" "rxy4" "wx3" "wy4" "c2" "c1"
+permutation "rxy3" "rxy4" "wy4" "wx3" "c1" "c2"
+permutation "rxy3" "rxy4" "wy4" "wx3" "c2" "c1"
+permutation "rxy3" "rxy4" "wy4" "c2" "wx3" "c1"
+permutation "rxy4" "rxy3" "wx3" "c1" "wy4" "c2"
+permutation "rxy4" "rxy3" "wx3" "wy4" "c1" "c2"
+permutation "rxy4" "rxy3" "wx3" "wy4" "c2" "c1"
+permutation "rxy4" "rxy3" "wy4" "wx3" "c1" "c2"
+permutation "rxy4" "rxy3" "wy4" "wx3" "c2" "c1"
+permutation "rxy4" "rxy3" "wy4" "c2" "wx3" "c1"
+permutation "rxy4" "wy4" "rxy3" "wx3" "c1" "c2"
+permutation "rxy4" "wy4" "rxy3" "wx3" "c2" "c1"
+permutation "rxy4" "wy4" "rxy3" "c2" "wx3" "c1"
#26Alexander Korotkov
a.korotkov@postgrespro.ru
In reply to: Amit Kapila (#25)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Tue, Mar 13, 2018 at 4:57 PM, Amit Kapila <amit.kapila16@gmail.com>
wrote:

On Mon, Mar 12, 2018 at 7:18 PM, Amit Kapila <amit.kapila16@gmail.com>
wrote:

On Mon, Mar 12, 2018 at 12:18 AM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:

On Sat, Mar 3, 2018 at 2:53 PM, Amit Kapila <amit.kapila16@gmail.com>

wrote:

On Fri, Mar 2, 2018 at 9:27 AM, Thomas Munro

If that is indeed a race, could it be fixed by
calling PredicateLockPageSplit() at the start of _hash_splitbucket()
instead?

Yes, but I think it would be better if we call this once we are sure
that at least one tuple from the old bucket has been transferred
(consider if all tuples in the old bucket are dead).

Is it really fair? For example, predicate lock can be held by session
which queried some key, but didn't find any corresponding tuple.
If we imagine this key should be in new bucket while all existing
tuples would be left in old bucket. As I get, in this case no locks
would be transferred since no tuples were moved to the new bucket.
So, further insertion to the new bucket wouldn't conflict with session,
which looked for non-existing key, while it should. Do it make sense?

Valid point, I think on split we should always transfer locks from old
bucket to new bucket.

Attached patch changes it as per above suggestion.

OK. Now patch looks good for me.

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

#27Amit Kapila
amit.kapila16@gmail.com
In reply to: Alexander Korotkov (#26)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Thu, Mar 15, 2018 at 4:29 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:

On Tue, Mar 13, 2018 at 4:57 PM, Amit Kapila <amit.kapila16@gmail.com>
wrote:

Valid point, I think on split we should always transfer locks from old
bucket to new bucket.

Attached patch changes it as per above suggestion.

OK. Now patch looks good for me.

Thanks, I have marked the patch as RFC in CF app.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

#28Teodor Sigaev
teodor@sigaev.ru
In reply to: Alexander Korotkov (#22)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

Thanks to everyone, pushed

Alexander Korotkov wrote:

On Fri, Mar 2, 2018 at 6:57 AM, Thomas Munro
<thomas.munro@enterprisedb.com <mailto:thomas.munro@enterprisedb.com>>
wrote:

My thought experiments about pseudo-pages and avoiding the split stuff
were not intended to get the patch kicked out.О©╫ I thought for a while
that hash indexes were a special case and could benefit from
dispensing with those trickier problems.О©╫ Upon further reflection, for
interesting size hash indexes pure hash value predicate tags wouldn't
be much better.О©╫ Furthermore, if we do decide we want to use using x %
max_predicate_locks_per_relation to avoid having to escalate to
relation predicate locks at the cost of slightly higher collision rate
then we should consider that for the whole system (including heap page
predicate locking), not just hash indexes.О©╫ Please consider those
ideas parked for now.

OK.О©╫ While our potential pseudo-pages are identified as
"hash_valueО©╫% some_constant_modulus", real bucket pages are very roughly
identified as "hash_value % number_of_index_pages".О©╫ So, page number is
adoptive to index size, despite it costs us handling page split. In the
same way,
locking in other index access methods is adoptive to an index size, so
that should be considered as useful feature which should be present in
hash index
as well.

------
Alexander Korotkov
Postgres Professional:http://www.postgrespro.com
<http://www.postgrespro.com/&gt;
The Russian Postgres Company

--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/

#29Amit Kapila
amit.kapila16@gmail.com
In reply to: Teodor Sigaev (#28)
Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

On Sat, Apr 7, 2018 at 7:47 PM, Teodor Sigaev <teodor@sigaev.ru> wrote:

Thanks to everyone, pushed

Thanks!

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com