[GSOC][weekly report 5] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions
Sorry for report late. Our lab's machines crashed for several days.
As I reported in the last email,
/messages/by-id/5b6b452.16851.15cf1ec010e.Coremail.liu-mx15@mails.tsinghua.edu.cn
I tried to decrease the contention on SerializableFinishedListLock.
It works actually. But I don't know why my optimization results in another problems: more contention on PREDICATELOCK_MANAGER_LWLOCK.
Here is the result of statistics collector. There were 36 connections in all, so most of them are waiting for locks.
(benchmark is ssibench, but other benchmarks have the same result)
Original:
SerializableXactHashLock, 0.36
predicate_lock_manager, 6.00
wal_insert, 0.07
SerializableFinishedListLock, 16.50
After optimization:
SerializableXactHashLock, 0.35
predicate_lock_manager, 11.53
buffer_content, 0.12
SerializableFinishedListLock, 11.47
So in the end, the performance did not change obviously.
Even if I enlarged the number of predicate locks (NUM_PREDICATELOCK_PARTITIONS) from 16 to 1024,
there were still so many contentions.
It bothered me for several days. The patch is attached.
To focus on this issue, I removed other parts of modification.
Do you have any advices for this ?
--
Sincerely
Mengxing Liu
Attachments:
finishedlock.patchapplication/octet-stream; name=finishedlock.patchDownload
diff --git a/src/backend/storage/lmgr/predicate.c b/src/backend/storage/lmgr/predicate.c
index 27c4af9..26e76a7 100644
--- a/src/backend/storage/lmgr/predicate.c
+++ b/src/backend/storage/lmgr/predicate.c
@@ -3569,11 +3569,17 @@ ClearOldPredicateLocks(void)
SERIALIZABLEXACT *finishedSxact;
PREDICATELOCK *predlock;
- /*
+ /* Clear at most 200 finishedSxact once*/
+ int releasedNum = 0;
+ SERIALIZABLEXACT* releasedSxact[200];
+
+ /*
* Loop through finished transactions. They are in commit order, so we can
* stop as soon as we find one that's still interesting.
*/
- LWLockAcquire(SerializableFinishedListLock, LW_EXCLUSIVE);
+ if (!LWLockConditionalAcquire(SerializableFinishedListLock, LW_EXCLUSIVE)){
+ return;
+ }
finishedSxact = (SERIALIZABLEXACT *)
SHMQueueNext(FinishedSerializableTransactions,
FinishedSerializableTransactions,
@@ -3595,10 +3601,8 @@ ClearOldPredicateLocks(void)
* This transaction committed before any in-progress transaction
* took its snapshot. It's no longer interesting.
*/
- LWLockRelease(SerializableXactHashLock);
SHMQueueDelete(&(finishedSxact->finishedLink));
- ReleaseOneSerializableXact(finishedSxact, false, false);
- LWLockAcquire(SerializableXactHashLock, LW_SHARED);
+ releasedSxact[releasedNum++] = finishedSxact;
}
else if (finishedSxact->commitSeqNo > PredXact->HavePartialClearedThrough
&& finishedSxact->commitSeqNo <= PredXact->CanPartialClearThrough)
@@ -3614,8 +3618,8 @@ ClearOldPredicateLocks(void)
{
/* A read-only transaction can be removed entirely */
SHMQueueDelete(&(finishedSxact->finishedLink));
- ReleaseOneSerializableXact(finishedSxact, false, false);
- }
+ releasedSxact[releasedNum++] = finishedSxact;
+ }
else
{
/*
@@ -3635,6 +3639,11 @@ ClearOldPredicateLocks(void)
break;
}
finishedSxact = nextSxact;
+
+ if (releasedNum == 200){
+ printf("more than 200 released num\n");
+ break;
+ }
}
LWLockRelease(SerializableXactHashLock);
@@ -3698,7 +3707,14 @@ ClearOldPredicateLocks(void)
}
LWLockRelease(SerializablePredicateLockListLock);
- LWLockRelease(SerializableFinishedListLock);
+ LWLockRelease(SerializableFinishedListLock);
+
+ int i;
+ for(i=0; i<releasedNum; i++){
+ ReleaseOneSerializableXact(releasedSxact[i], false, false);
+ }
+
+
}
/*
@@ -3732,7 +3748,7 @@ ReleaseOneSerializableXact(SERIALIZABLEXACT *sxact, bool partial,
Assert(sxact != NULL);
Assert(SxactIsRolledBack(sxact) || SxactIsCommitted(sxact));
Assert(partial || !SxactIsOnFinishedList(sxact));
- Assert(LWLockHeldByMe(SerializableFinishedListLock));
+// Assert(LWLockHeldByMe(SerializableFinishedListLock));
/*
* First release all the predicate locks held by this xact (or transfer