Avoid LWLockWaitForVar() for currently held WAL insertion lock in WaitXLogInsertionsToFinish()

Started by Bharath Rupireddyabout 3 years ago54 messages
#1Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
1 attachment(s)

Hi,

While working on something else, I noticed that
WaitXLogInsertionsToFinish() goes the LWLockWaitForVar() route even
for a process that's holding a WAL insertion lock. Typically, a
process holding WAL insertion lock reaches
WaitXLogInsertionsToFinish() when it's in need of WAL buffer pages for
its insertion and waits for other older in-progress WAL insertions to
finish. This fact guarantees that the process holding a WAL insertion
lock will never have its insertingAt less than 'upto'.

With that said, here's a small improvement I can think of, that is, to
avoid calling LWLockWaitForVar() for the WAL insertion lock the caller
of WaitXLogInsertionsToFinish() currently holds. Since
LWLockWaitForVar() does a bunch of things - holds interrupts, does
atomic reads, acquires and releases wait list lock and so on, avoiding
it may be a good idea IMO.

I'm attaching a patch herewith. Here's the cirrus-ci tests -
https://github.com/BRupireddy/postgres/tree/avoid_LWLockWaitForVar_for_currently_held_wal_ins_lock_v1.

Thoughts?

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v1-0001-Avoid-LWLockWaitForVar-for-currently-held-WAL-ins.patchapplication/octet-stream; name=v1-0001-Avoid-LWLockWaitForVar-for-currently-held-WAL-ins.patchDownload
From 82296a73d48451b3c16266cb5df529ee60ffe04f Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Thu, 24 Nov 2022 12:08:38 +0000
Subject: [PATCH v1] Avoid LWLockWaitForVar() for currently held WAL insertion
 lock in WaitXLogInsertionsToFinish()

---
 src/backend/access/transam/xlog.c | 40 ++++++++++++++++++++++++++++++-
 1 file changed, 39 insertions(+), 1 deletion(-)

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index a31fbbff78..cffcbda941 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -631,7 +631,7 @@ static TimeLineID LocalMinRecoveryPointTLI;
 static bool updateMinRecoveryPoint = true;
 
 /* For WALInsertLockAcquire/Release functions */
-static int	MyLockNo = 0;
+static int	MyLockNo = -1;
 static bool holdingAllLocks = false;
 
 #ifdef WAL_DEBUG
@@ -1321,6 +1321,8 @@ WALInsertLockAcquire(void)
 
 	if (lockToTry == -1)
 		lockToTry = MyProc->pgprocno % NUM_XLOGINSERT_LOCKS;
+
+	Assert(MyLockNo == -1);
 	MyLockNo = lockToTry;
 
 	/*
@@ -1351,6 +1353,8 @@ WALInsertLockAcquireExclusive(void)
 {
 	int			i;
 
+	Assert(MyLockNo == -1);
+
 	/*
 	 * When holding all the locks, all but the last lock's insertingAt
 	 * indicator is set to 0xFFFFFFFFFFFFFFFF, which is higher than any real
@@ -1382,6 +1386,8 @@ WALInsertLockRelease(void)
 	{
 		int			i;
 
+		Assert(MyLockNo == -1);
+
 		for (i = 0; i < NUM_XLOGINSERT_LOCKS; i++)
 			LWLockReleaseClearVar(&WALInsertLocks[i].l.lock,
 								  &WALInsertLocks[i].l.insertingAt,
@@ -1394,6 +1400,8 @@ WALInsertLockRelease(void)
 		LWLockReleaseClearVar(&WALInsertLocks[MyLockNo].l.lock,
 							  &WALInsertLocks[MyLockNo].l.insertingAt,
 							  0);
+
+		MyLockNo = -1;
 	}
 }
 
@@ -1406,6 +1414,8 @@ WALInsertLockUpdateInsertingAt(XLogRecPtr insertingAt)
 {
 	if (holdingAllLocks)
 	{
+		Assert(MyLockNo == -1);
+
 		/*
 		 * We use the last lock to mark our actual position, see comments in
 		 * WALInsertLockAcquireExclusive.
@@ -1482,6 +1492,34 @@ WaitXLogInsertionsToFinish(XLogRecPtr upto)
 	{
 		XLogRecPtr	insertingat = InvalidXLogRecPtr;
 
+		/*
+		 * If this process is holding the current WAL insert lock, then it can
+		 * never happen that its insertion point is behind 'upto'. When the
+		 * process doesn't get enough WAL buffer pages for its insertion, it
+		 * comes here requesting the older in progress insertions to finish.
+		 *
+		 * In this case, we avoid going the LWLockWaitForVar route and quickly
+		 * continue.
+		 */
+		if (MyLockNo == i)
+		{
+			Assert(LWLockHeldByMe(&WALInsertLocks[i].l.lock));
+
+			/*
+			 * Since we're holding the WAL insert lock, we can safely read
+			 * insertingAt here.
+			 */
+			insertingat = WALInsertLocks[i].l.insertingAt;
+
+			Assert(!XLogRecPtrIsInvalid(insertingat));
+			Assert(insertingat >= upto);
+
+			if (insertingat < finishedUpto)
+				finishedUpto = insertingat;
+
+			continue;
+		}
+
 		do
 		{
 			/*
-- 
2.34.1

#2Andres Freund
andres@anarazel.de
In reply to: Bharath Rupireddy (#1)
Re: Avoid LWLockWaitForVar() for currently held WAL insertion lock in WaitXLogInsertionsToFinish()

Hi,

On 2022-11-24 18:13:10 +0530, Bharath Rupireddy wrote:

With that said, here's a small improvement I can think of, that is, to
avoid calling LWLockWaitForVar() for the WAL insertion lock the caller
of WaitXLogInsertionsToFinish() currently holds. Since
LWLockWaitForVar() does a bunch of things - holds interrupts, does
atomic reads, acquires and releases wait list lock and so on, avoiding
it may be a good idea IMO.

That doesn't seem like a big win. We're still going to call LWLockWaitForVar()
for all the other locks.

I think we could improve this code more significantly by avoiding the call to
LWLockWaitForVar() for all locks that aren't acquired or don't have a
conflicting insertingAt, that'd require just a bit more work to handle systems
without tear-free 64bit writes/reads.

The easiest way would probably be to just make insertingAt a 64bit atomic,
that transparently does the required work to make even non-atomic read/writes
tear free. Then we could trivially avoid the spinlock in
LWLockConflictsWithVar(), LWLockReleaseClearVar() and with just a bit more
work add a fastpath to LWLockUpdateVar(). We don't need to acquire the wait
list lock if there aren't any waiters.

I'd bet that start to have visible effects in a workload with many small
records.

Greetings,

Andres Freund

#3Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Andres Freund (#2)
2 attachment(s)
Re: Avoid LWLockWaitForVar() for currently held WAL insertion lock in WaitXLogInsertionsToFinish()

On Fri, Nov 25, 2022 at 12:16 AM Andres Freund <andres@anarazel.de> wrote:

Hi,

On 2022-11-24 18:13:10 +0530, Bharath Rupireddy wrote:

With that said, here's a small improvement I can think of, that is, to
avoid calling LWLockWaitForVar() for the WAL insertion lock the caller
of WaitXLogInsertionsToFinish() currently holds. Since
LWLockWaitForVar() does a bunch of things - holds interrupts, does
atomic reads, acquires and releases wait list lock and so on, avoiding
it may be a good idea IMO.

That doesn't seem like a big win. We're still going to call LWLockWaitForVar()
for all the other locks.

I think we could improve this code more significantly by avoiding the call to
LWLockWaitForVar() for all locks that aren't acquired or don't have a
conflicting insertingAt, that'd require just a bit more work to handle systems
without tear-free 64bit writes/reads.

The easiest way would probably be to just make insertingAt a 64bit atomic,
that transparently does the required work to make even non-atomic read/writes
tear free. Then we could trivially avoid the spinlock in
LWLockConflictsWithVar(), LWLockReleaseClearVar() and with just a bit more
work add a fastpath to LWLockUpdateVar(). We don't need to acquire the wait
list lock if there aren't any waiters.

I'd bet that start to have visible effects in a workload with many small
records.

Thanks Andres! I quickly came up with the attached patch. I also ran
an insert test [1]./configure --prefix=$PWD/inst/ CFLAGS="-O3" > install.log && make -j 8 install > install.log 2>&1 & cd inst/bin ./pg_ctl -D data -l logfile stop rm -rf data logfile free -m sudo su -c 'sync; echo 3 > /proc/sys/vm/drop_caches' free -m ./initdb -D data ./pg_ctl -D data -l logfile start ./psql -d postgres -c 'ALTER SYSTEM SET shared_buffers = "8GB";' ./psql -d postgres -c 'ALTER SYSTEM SET max_wal_size = "16GB";' ./psql -d postgres -c 'ALTER SYSTEM SET max_connections = "4096";' ./pg_ctl -D data -l logfile restart ./pgbench -i -s 1 -d postgres ./psql -d postgres -c "ALTER TABLE pgbench_accounts DROP CONSTRAINT pgbench_accounts_pkey;" cat << EOF >> insert.sql \set aid random(1, 10 * :scale) \set delta random(1, 100000 * :scale) INSERT INTO pgbench_accounts (aid, bid, abalance) VALUES (:aid, :aid, :delta); EOF ulimit -S -n 5000 for c in 1 2 4 8 16 32 64 128 256 512 768 1024 2048 4096; do echo -n "$c ";./pgbench -n -M prepared -U ubuntu postgres -f insert.sql -c$c -j$c -T5 2>&1|grep '^tps'|awk '{print $3}';done, below are the results. I also attached the results
graph. The cirrus-ci is happy with the patch -
https://github.com/BRupireddy/postgres/tree/wal_insertion_lock_improvements_v1_2.
Please let me know if the direction of the patch seems right.
clients HEAD PATCHED
1 1354 1499
2 1451 1464
4 3069 3073
8 5712 5797
16 11331 11157
32 22020 22074
64 41742 42213
128 71300 76638
256 103652 118944
512 111250 161582
768 99544 161987
1024 96743 164161
2048 72711 156686
4096 54158 135713

[1]: ./configure --prefix=$PWD/inst/ CFLAGS="-O3" > install.log && make -j 8 install > install.log 2>&1 & cd inst/bin ./pg_ctl -D data -l logfile stop rm -rf data logfile free -m sudo su -c 'sync; echo 3 > /proc/sys/vm/drop_caches' free -m ./initdb -D data ./pg_ctl -D data -l logfile start ./psql -d postgres -c 'ALTER SYSTEM SET shared_buffers = "8GB";' ./psql -d postgres -c 'ALTER SYSTEM SET max_wal_size = "16GB";' ./psql -d postgres -c 'ALTER SYSTEM SET max_connections = "4096";' ./pg_ctl -D data -l logfile restart ./pgbench -i -s 1 -d postgres ./psql -d postgres -c "ALTER TABLE pgbench_accounts DROP CONSTRAINT pgbench_accounts_pkey;" cat << EOF >> insert.sql \set aid random(1, 10 * :scale) \set delta random(1, 100000 * :scale) INSERT INTO pgbench_accounts (aid, bid, abalance) VALUES (:aid, :aid, :delta); EOF ulimit -S -n 5000 for c in 1 2 4 8 16 32 64 128 256 512 768 1024 2048 4096; do echo -n "$c ";./pgbench -n -M prepared -U ubuntu postgres -f insert.sql -c$c -j$c -T5 2>&1|grep '^tps'|awk '{print $3}';done
./configure --prefix=$PWD/inst/ CFLAGS="-O3" > install.log && make -j
8 install > install.log 2>&1 &
cd inst/bin
./pg_ctl -D data -l logfile stop
rm -rf data logfile
free -m
sudo su -c 'sync; echo 3 > /proc/sys/vm/drop_caches'
free -m
./initdb -D data
./pg_ctl -D data -l logfile start
./psql -d postgres -c 'ALTER SYSTEM SET shared_buffers = "8GB";'
./psql -d postgres -c 'ALTER SYSTEM SET max_wal_size = "16GB";'
./psql -d postgres -c 'ALTER SYSTEM SET max_connections = "4096";'
./pg_ctl -D data -l logfile restart
./pgbench -i -s 1 -d postgres
./psql -d postgres -c "ALTER TABLE pgbench_accounts DROP CONSTRAINT
pgbench_accounts_pkey;"
cat << EOF >> insert.sql
\set aid random(1, 10 * :scale)
\set delta random(1, 100000 * :scale)
INSERT INTO pgbench_accounts (aid, bid, abalance) VALUES (:aid, :aid, :delta);
EOF
ulimit -S -n 5000
for c in 1 2 4 8 16 32 64 128 256 512 768 1024 2048 4096; do echo -n
"$c ";./pgbench -n -M prepared -U ubuntu postgres -f insert.sql -c$c
-j$c -T5 2>&1|grep '^tps'|awk '{print $3}';done

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v1-0001-WAL-Insertion-Lock-Improvements.patchapplication/octet-stream; name=v1-0001-WAL-Insertion-Lock-Improvements.patchDownload
From 293e789f9c1a63748147acd613c556961f1dc5c4 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Fri, 25 Nov 2022 10:53:56 +0000
Subject: [PATCH v1] WAL Insertion Lock Improvements

---
 src/backend/access/transam/xlog.c |  8 +++--
 src/backend/storage/lmgr/lwlock.c | 56 +++++++++++++++++--------------
 src/include/storage/lwlock.h      |  7 ++--
 3 files changed, 41 insertions(+), 30 deletions(-)

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index a31fbbff78..b3f758abb3 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -376,7 +376,7 @@ typedef struct XLogwrtResult
 typedef struct
 {
 	LWLock		lock;
-	XLogRecPtr	insertingAt;
+	pg_atomic_uint64	insertingAt;
 	XLogRecPtr	lastImportantAt;
 } WALInsertLock;
 
@@ -1482,6 +1482,10 @@ WaitXLogInsertionsToFinish(XLogRecPtr upto)
 	{
 		XLogRecPtr	insertingat = InvalidXLogRecPtr;
 
+		/* Quickly check and continue if no one holds the lock. */
+		if (!IsLWLockHeld(&WALInsertLocks[i].l.lock))
+			continue;
+
 		do
 		{
 			/*
@@ -4602,7 +4606,7 @@ XLOGShmemInit(void)
 	for (i = 0; i < NUM_XLOGINSERT_LOCKS; i++)
 	{
 		LWLockInitialize(&WALInsertLocks[i].l.lock, LWTRANCHE_WAL_INSERT);
-		WALInsertLocks[i].l.insertingAt = InvalidXLogRecPtr;
+		pg_atomic_init_u64(&WALInsertLocks[i].l.insertingAt, InvalidXLogRecPtr);
 		WALInsertLocks[i].l.lastImportantAt = InvalidXLogRecPtr;
 	}
 
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index a5ad36ca78..3c2048224b 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1536,6 +1536,15 @@ LWLockAcquireOrWait(LWLock *lock, LWLockMode mode)
 	return !mustwait;
 }
 
+bool
+IsLWLockHeld(LWLock *lock)
+{
+	uint32	state;
+
+	state = pg_atomic_read_u32(&lock->state);
+	return ((state & LW_VAL_EXCLUSIVE) != 0) || ((state & LW_VAL_SHARED) != 0);
+}
+
 /*
  * Does the lwlock in its current state need to wait for the variable value to
  * change?
@@ -1546,9 +1555,8 @@ LWLockAcquireOrWait(LWLock *lock, LWLockMode mode)
  * *result is set to true if the lock was free, and false otherwise.
  */
 static bool
-LWLockConflictsWithVar(LWLock *lock,
-					   uint64 *valptr, uint64 oldval, uint64 *newval,
-					   bool *result)
+LWLockConflictsWithVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+					   uint64 *newval, bool *result)
 {
 	bool		mustwait;
 	uint64		value;
@@ -1570,14 +1578,7 @@ LWLockConflictsWithVar(LWLock *lock,
 
 	*result = false;
 
-	/*
-	 * Read value using the lwlock's wait list lock, as we can't generally
-	 * rely on atomic 64 bit reads/stores.  TODO: On platforms with a way to
-	 * do atomic 64 bit reads/writes the spinlock should be optimized away.
-	 */
-	LWLockWaitListLock(lock);
-	value = *valptr;
-	LWLockWaitListUnlock(lock);
+	value = pg_atomic_read_u64(valptr);
 
 	if (value != oldval)
 	{
@@ -1606,7 +1607,8 @@ LWLockConflictsWithVar(LWLock *lock,
  * in shared mode, returns 'true'.
  */
 bool
-LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
+LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+				 uint64 *newval)
 {
 	PGPROC	   *proc = MyProc;
 	int			extraWaits = 0;
@@ -1741,21 +1743,32 @@ LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
  * The caller must be holding the lock in exclusive mode.
  */
 void
-LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
 	proclist_head wakeup;
 	proclist_mutable_iter iter;
 
 	PRINT_LWDEBUG("LWLockUpdateVar", lock, LW_EXCLUSIVE);
 
+	/*
+	 * When there are no waiters, quickly update the variable and exit without
+	 * taking wait list lock.
+	 */
+	if ((pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) == 0)
+	{
+		/* Update the lock's value atomically */
+		pg_atomic_write_u64(valptr, val);
+		return;
+	}
+
 	proclist_init(&wakeup);
 
 	LWLockWaitListLock(lock);
 
 	Assert(pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE);
 
-	/* Update the lock's value */
-	*valptr = val;
+	/* Update the lock's value atomically */
+	pg_atomic_write_u64(valptr, val);
 
 	/*
 	 * See if there are any LW_WAIT_UNTIL_FREE waiters that need to be woken
@@ -1872,17 +1885,10 @@ LWLockRelease(LWLock *lock)
  * LWLockReleaseClearVar - release a previously acquired lock, reset variable
  */
 void
-LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
-	LWLockWaitListLock(lock);
-
-	/*
-	 * Set the variable's value before releasing the lock, that prevents race
-	 * a race condition wherein a new locker acquires the lock, but hasn't yet
-	 * set the variables value.
-	 */
-	*valptr = val;
-	LWLockWaitListUnlock(lock);
+	/* Update the lock's value atomically */
+	pg_atomic_write_u64(valptr, val);
 
 	LWLockRelease(lock);
 }
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index a494cb598f..3fae217d26 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -125,14 +125,15 @@ extern bool LWLockAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockConditionalAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockAcquireOrWait(LWLock *lock, LWLockMode mode);
 extern void LWLockRelease(LWLock *lock);
-extern void LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern bool IsLWLockHeld(LWLock *lock);
+extern void LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 extern void LWLockReleaseAll(void);
 extern bool LWLockHeldByMe(LWLock *lock);
 extern bool LWLockAnyHeldByMe(LWLock *lock, int nlocks, size_t stride);
 extern bool LWLockHeldByMeInMode(LWLock *lock, LWLockMode mode);
 
-extern bool LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval);
-extern void LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern bool LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval, uint64 *newval);
+extern void LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 
 extern Size LWLockShmemSize(void);
 extern void CreateLWLocks(void);
-- 
2.34.1

WAL Insertion Lock Improvements.pngimage/png; name="WAL Insertion Lock Improvements.png"Download
�PNG


IHDR� l<�! IDATx^��	|e��������@K��B��] PJ.�^�/J+����`h�U���.Q�kn�+Q���zq�
u)�����bKJ(�tI�4���3����93����;s����69y�w���>3���kd�����!�   ����l��5�l��`�������K/�����@� �A��@@@@#����@@���0|@@�@���%��@��   ��,@�-A@@�,�g:n	  `���d�8�qK��@@@  �Y��[�*�Z���:������,W����g]#��XL
�uT)��A�#����%w�Qm��jh�:�[5���i�7r���VKZq�e�vk�\=�.�in�j!e
��l�4Sw.%/�spW-������	{�Q������1���i/�1���DC]�V�B�**h�N�HIZ,r�%.]%  �>�]PE�#AU�I&�G8	\LL9��
kWj<�bK��?�D
������b��:�"�\x�d8������u�q��^������z�b���t�F�fQ.���ZZa�n����k=l|Q��+����6�F�F���U#vY����b��5
�rci=����7^�p��9���_�^��������Cv,R��%�a�`��N�=��}��'j���a�Hc���(�63�F��;l��,
�N�u��:�����'���	����--~�_s�8��C������I&uQ����/4#�������1>m��1k�H�m��bb|�Zw�/��H��?���'��w]�'^�i��E�
��n\�d�����d�$�X��3QG��&.-S[��/s�mH��f����Hf����|]S�#�of��t~j��l}>$_%H���pWt�xw��[��`�F��o���)�����H��T+�(�^��c��(����}W��.��<5�6�"�
��]��)�R�^�z�e*zo�t���\��	�9|-=��������% ub��'��L�G%u���X}I�w��U�|a���L��������w��� �E{0�V�������^;���<���J�:������^�OK�� l��1�w�>-<'��2��o?��A�1�79M������29zu��4]��	|�m�Zo�qhk9�� ��.��+=�FZi������U�|aIk.��������X!��e~@{������������:i���K-���t�*�1�����1`�q�	�v�`�Ii�/T��p�{��6]�^�m55��y��z/\��V�*��11�_�-�;��*X�������#+/"��
�j���$�����dl"1�U����Crd�Z��qOUc��"`�-��Z|��)�H�f3I�H#2,u�*0����.3�D��03��z��vg�	;AG���5�.����!��������U��}!b��/���#��O�%-p�������]���N�[��m�'���t�t�&#eI�c��i#�10��1�fV�����%c����V�N~�;1[f��<�:�X�.Y�3�m��g��M���~�����Y���\�`�J8���Km�T�Mm�dm������-���Ik����R����K\�����,7m�����sJ�L��
p�%Y��Gv�q�L��O\����<6����v���������`3�gK�
��O&"IK�z|�wq��]��-y[Q�q/��,>-��*���������IQp>4s�.�����t�L<�|����2W�f�l�U}��t����j�%MZ6��u���N�{��&�z%�a*po�8%�7�����V���`��iMv{�V���,e��Y��������w��n��o�}aKz��*�|���-����F�@ 8M���`d�8[�q_�D ����QUa`aTH ���-9��g�|n]�
@@ g	@�s�i`��@>��s��n  9K��M�@@��8�[u�Y��m  �� �������@���l��0�|&���E�@@r�8g�����3p>�.�  �� �9�40@@ �	@���uQ7��% )������t(��A��J��9K����@�Em��T���6v�7USYLM����=&������L�|u7�j��J��mn�jR��~s\   N�`���	���T��H�TOM�1�������_��F����A@&;���;u�U
Z�[9!����b�?���%��VK-�����Zv�P��=��E+�t�"fD�ph�0�,�F��_��
~������44��D(��^���O`���/�Z	��'�����c���!������U�������pgy35Uv��$������^�;LU7�FY�l���|/���|��L l���������`���BL���/M�
��!�]��5�(�'d��\�����r�.(gug]j.����}9��V�	�g%�u��h�".Y����K��K��lj��%&�|�x�I
�`7~��<pV��/��	���.�!��{��;�Y�t(|����Z����X���
]���

|!h��@��"v�
pp���� h��\%_W��3��0*W	���2$W��X���`�X��M�n���_0,�l�J��u�����W���a��C�]=�b���B��b��6��%'�/l|!�`����*U�^��U3g_aT���/v���e
������T����X����C�!�b���T/?�$-|�>?�&E�����6R�K\�D����`��^����C�!�>=c�}o���/��pE��(�1�J��;gE/\��!^,����|��'$��^����e�2�5TPMs��|T����)
U
��T}.��a������?G���d��2��y�=|������oQ�i�+�s4w����i�p�?��u�:�����CI�8�h+
���_��s��O=��!�����@��q�|�q�6����D�,��T���6v�'8&����d$��z��8B�'*���`�h|�`B�"�%D�qM�F2�)�B��WQ��3����ve[#��=����QMP��-c{w�������/	�"�N���\�8��C���.<�*�9_���r����J�?�0���/�9%/����$�������Gr����k���s�hC�	K���h��?�D��A��%Tr�**\x�4��.ba��������5	G�EGk�.��mO�F��i�T���e��X�9��^c=������E�2M��U]Vt������hD��s�W`��z�pU��}9��x��|@�Z+�*�]mTk��������,.��PB�37��o�O��~����>����mht��������s�&^,l��y��v,������1���>���4���Sn���=:�G��K�]��/p��:q�_z}�z���[t��
��_�g�������4�D��_�r�`l�X�oZ���j����fe��S���p�W��Md��������Bf��y���G�S�S�U�S�1���[�X�S���{������2����BS����{�<��M�,B>����}q���h�50�.�,�����"l������K���4����hN0H�UG������KR���N7u�#4��{��o�N�����c4��s=�������?����*��ad.zsE�cc�E�wh3�c]�E�=<��\x�5*8|P���yKh�x:E�i��������(:m��������t����n��
�k�d�����{�PO`��e����=N�le��U�IX2�����{>3-($SM�h��}��y��j�}�����C{����Uk�������3��[�>)���\��kcZ��f�����)�ru�W����sD���e&a�9({%���(�fU
���N���;���iNL�R������$,�*��~�>�������7P��D�eJ��f�C��*�5�Q'�������m.z�����#Sg��C�4g�3���
�\B��->�t*�a0���2�3�`���;��jki�t�>sw���m������m�{9��@�s��M��2�l������y=8�P�'��e���f���
>��?����L�z�������������u��Lq}��&�<k{�K���xW�c���a����'���QJ������2�.-C��y����������wQr3���P^`>:�M���:����:n������H�t�z����w���I�u"�0��T���X�f�(
�+
�C���Y��f���.h�����&~�g���k���c�J����^=!}~���ItE���|��xJ��m���T��Nty��n����0�
��A�� ��?/�w��������TG�fA5����^U$��	�����o?��I��3�u��XS�"=00@�����K�7t	z��L��W��Li�f/��E!�������n����[����>9I��\�k,�2�����DE���mtw[��5&��ur�o&;��~�C�s�8�����_��o�8��]�a}��}u�K&�,��ny��f���F$-s��i�s|�P^�el�e�6'�a�F1��A�"`w�D���S���W��d�����/J)+��w]	KF���d���U� �?X��Y�y""o~�.r��y{P�&��*��D��&�OoQ�;e/8����i#
m����F�G����A5����^U$'�g����K�/c���O(=d����.n�F���Z��������l��5���d�!W6�0&^���� ����>pV��K�r�dS��8��{��9b-8i����Ic'��W3���"���+b�L�������e��c��n��-����W��w��XZP���:���WID�*I&&�q�vH��E�����6���,��
�����+�`�t��
��{U���K2�t�������:�/��4��[�,m�u
���w�sKK
p�������jJ:M?���t���!
�>u�\�����jm\ks��W`�T�.l/0���������n<Zt����0��W�%X������T���X��iH�T���6v�[����z��F�<v�Q�1����MTM��)�85���������H����������B���z�d�k�G������[i��6fu�����IO�,�6���������"�]mT�i���D�=1�(���G��k���\y�+�ll����<���
��3�!�w�+J�]:�u�M4W|K���K���Wi��N+������(Z���tA0Vek�_0wK?LeMuT�	18�
d�� _����RE���������Y��#+e�h�������}��l^�a�g[n�%N�tsX����#f��:���T�������4���E�,�ID���m���;(2<@E�=<@}oRa��B$F�������S����2�N�C��c���&�4�c�����lU
�\�������:]���g��_�"�-��_�5�H�|e��k���_8��t���]�p�~nt����]����|a�����=
��v���H�x�]��.�����g%�.���&�|EI�K����@
7����uQ���O��70�.����l�/
?�E���MM�g��KS&����-��2w�`���BL�W5�S]���kw��RK<C�iy���E+��IX2�~����E�eJ{3��w���/Sn�M&��W7G��^������P��3�H�����/~�h|� E����K����u�'$X{n�����zeG,��W�Z�WI�r�|���Ik�wm�����@�	d�{u�9��)�hl`B|�:��f�?�o�g(=(�`���hy���>��W` G.U5�e{U���0��uV��=�i��k:�Y���>�/�yG@�}hf>���������������7r�&_Ybr��W��l�����������?�>��~�1Y,J�������-�!�����+�	�WP�/k�0�"&W�A��lj��%&���1G�}7QtpbvvdZ)���2M9�b��(��O@�6eD�SN��f^y����j|W��d���H�����'��?������):��
�����}�JN��
.PzP�X�1D������o��~3�T�g�C���W���t����h.�%�>;���	�}d-gP���t
>�J^��t��0���p��2�������K��C����r��/G��^���"���ElI{E(�O�����_J���1"`o�!����R�O�E�������z��9/�ff�%�z��9o���W��Mg�w����3._C%gT�qJ��zj����7#"O	��>��`�X�Q�$���z��Z���7��
8�\pV/�2������_/�2�
_D�����/���.S�=t�K����{=6x����/_�2�?H5��TM�N�+�Y��;�gjG��OnP�����9�']��b��N��/��u�|�q������_����:��a%m�$�7�g�uUR]�&������iU{U��x��&�&S/�Kl0����#^~Tr��h�M��(K��=`�WI�;��k&6�+�x!,��&N*{Bh+�������Y�8������W��Q����Z�gX���O�����������cgbz�����^�-�>?��/�_���3'?:���Fq��7�;�2RPH�>�SegX���\��"i_��/�_���a=4�XotnE�ee}�i�z�������I m$�]F�/m	�Kp7_,[���r�_����t��y"VgysJ�s�E���q��q����KD<��=��o�#���/���A���:63z��	���;O'anX_a*����U��a�*�����$�f�/�������`*�������j[��U����iy���E�I�$� ��������t����h.�%�.]��B���;g��\*��3��4��.�j~�a�������fa�0X�NX|��0��O��������^��\��/�_���3'�,�wS����y���oxH��#���
��~�<
�Q��b�a��T~�4�8�����2��"i_��o�M��|,�bC��������n��^���q���P������c;���m���3���u�����^X�9�_U-�/��t.>��{r�������2>t��id�4��7���W	[���!zu��.�/��G�{i���I�#��Fi<!�����?&��V$|� ����6��o��@���a����8x����G���]�_��_�e7=�yq����RT@'.)M|���C4eN�8�EgI��4-]4�s]T��K%����|!��������9tAK���'F��&�~�?snx�-��w��l\�J����tl�t���5��+I*��-�&�1�X�����]m\U�����S�3��A9�DS�M
{U�t�V�.B�jPE��0v~V�-��w]M���i��k�����\�m?H����z�2_,���:�.|�t���������"�
�&�r��g>��N=f���{���_K����-��.��z�4��M�)�z��13�fD�9C�=9���:��^���9_����q�d*�x;~�+!���E��]�2��/�����E�������t�YG%�`qf~�=��e�_]s��|��m�����gJa��fa��=l6e/���mc���/���6Q���io�M/)���#��zI���L��^�����tlK��~�����[��h8(gl���`oFD���'|3��/��I!��=���|���3���/���T[*���������D�2G�}g3��W���`��x���7t|!��8v�B�PsS5�����;v��R�@�u���~.��*�!���;(z����v��K��<!�l���I�� IDAT���_<K����d"�����J y(�z�'�| yH6�`%��1tRy�J��������S�h5���U�uTIq�cg�^^�'^���9e��~���&&I�mx��*Q��
�QG��$� ����B��J����_+���O�wA[��(���S���>��s��aW�*����'�S��5l��v��l��gk�1'c/�����;^&�����������������k��\z�7#�^���_����X���{>N|��+o�?�L�i���X5qK��q��eN����'�����3��y������e�K�f��.�U���$�_��@�3p�q��^�����ny�5��6�

ht�~��z�T�^`�d��?������m�����z�m.�\���b�L^�+CK>-��3��6�\������F���$�MO���'i�n��'�?��2}��-*.�������irk|e2�i{�������qs�>q����LO�R@���	�o�,,N$>�������������1��5���sb�q��@	���g��:�O�r�<��s	=�2�[O��|���z��G��������O����i�K7�X8o����i��������u;hpxL+*'8���
G�<�W���t����L.0��%!�<��6���jh'^q��VK-�?T�4'&d����x`���w��~�^��]�����Y�T~]~8k�m.y��'h[hr���L���?L���Nn����[e:��������/�Y����|U�h.��I�������,�A4�?�p+���'���oW�Ow?��x�)�����/g������%}��6��u2�z��Ct�o���m��x�5o���2����;��m"���+ro7i`�j�y�W����`�������R������S�V���)C����k;'�9��w�<]��'Ay�%�7L|=���|z��<3�`����mtx���������'���y�1,bD�����o���404��x{����5����G�,�|�b����/X��������Hn0���IR��;��������_����G����b�rd3��Y���G��O�����]/�'^��2�]m\<��r��Dt,�K6}���^��K�r���c�����_�������;�?Z@?�Re ]����,�,�_�����l\=sT������~\���f2e��-��a��^�6��-�(����$��-�S�7��o��_�N�&�����T)l�W{��<D����G�<���{�7'0���K|�����^(�e����y}k���w��G��6�2�&��/+�]�_�x2�u��2��W��|����C������?�?A<E�|���������eo:1���+O�����nk��/{el�I{eh��_yf�9�����m�DG���7/ ����[�t�\��K�m*gMmC�y���a������I\��� |e_�r��W��l���E�X*6����y}��z�y��UKe������m/01wU�-ozg�Qzd��c���!��z%�>?����`^��k�����y�{h�5�}��k�4x!��x��������*���o�q����S�W��������x����C���g?�,��]E=���
�qF��)5e��\���7m��m�[ly�y�j'f1���yS�Q����4Q�ys��h�a�()w������]��jn�&>��g]#����{U���
?u_�eH|��x�[���5Tv���|�)��R�.lX���"l��e7>��4��G�;q�:I��u���ma��a��S��8�V�z]t{h]�j�^���T��1�>Gh�0\8�>���@���,���� �	c'.��O_:;���Wo���{EZ�}����*��SeW#�R}�hB����z��U|�T-��2����R�Ois���C�r���y������g7�H�G��S[����m�k�8��/�u/�"9����[���3�
�5�2,l��>\M�/z�>���,���Lj����1��e��(E>R�����:�+�|��%{?��%&����1����[���-�����S���j�����D�N�l\V&1�}�T�o�2x�����f�����#02�m=#����{�0��;�T���_VL'�O���.��G�m>"�&'/�(J������L[Q������J#/o�h4J�l�K.���_ &���E����%�id��Lj>r����^>��	����y��f�����K�^��vS�uCM<O��r��fsP�*����3�9�,o���.j\�M���������������M����Lc�,�[?@���c�c���L*��h�	�/W	�����)'���������Y���G�����cQ>n�L�[�N&��|R�����.-�s�,��b#}��������<��F���)U���B&���
4����k�[x��]���m����*Ci[�J�,�<n���^�_�1d��O\�c��.-�*���l�t��/����O8�����T��I���|����j���U� UF�0����s��J�m�������\~�4ZZ6C�}mw����#�|��Igc�"`��<C&���J}}k�]��sh��^8��������,��D�k������.�� �0�t�B<!��������!*�VL7�:�s���������+������%#���C4�K�/�I�.�
M9�����8�U1PKq���,���<L/���2��0{zI-��x{�
�����L��3���u����_��2���c���pd��[�I�E��i����������_V^npVw�h�\�����.���Cz��?�XG��Ei<!�F]K�h[j���GN�#f�������l9].�����i]wi���9_��-������Kzd�"��������������<���z�'�5l|y�������	�.�d^�*_0���JG��~�|y��)z�6���W���^?�!V6C����/n��#w����S�����c�%�>z[���i���_�a���+mB�
[�S����<N�we���w">*��3��k���ys�wF�Al�������������d����}�FG�N@zs|�Nl��b���sT=7�������M�*!�,\@k�y1����|p)���#R�5�����c�{�������}C����3�;Wk�+�+����g:=
�����3��/��<��[^�E\J��it���:-��sz�	?).����.Z0����W���/Q��(�h���W,���:J���d,�-c�F�,*�\������
��}���?�DWve#���`l���ed�t��O��c'Q���=����3��+����73_gi=����y�lc��.n��Hs�������i}t���9�!�?xM�`G��=4�h+�b�bZv�����Xt�.8����W
_#�6����c������@����Y�y�N����-����r��!�������9��>����|��Y�����������7~�r��"m�Es��_���0��3�g��e���������?��@m�����=o~�CT�w��\C_��v�b�/8��
�����0fes�w���j���������gc��u"�A���t`���iG-�A�G���G�iE|T�j�������I�>u�L[Q�]��
�G��gt�'.w.��=���Y3��w���/Sn����*c�<Y��W��eW�xu&����zj�(M�>=U�\���4�t����:�����U0	+vR��v��45a��c��/Tt���t�{�������{��3���������H{ yH�����o��{p���o$���l�7�7w�3��sc���-�����?p
�fsP�z`�V���j2k"G��T�8����^�T�����"�.��8�=p=1z�Xw{�5�"���&���j����"i_����#�G7l����y��X��w���mXj���������
�ex���������j[b��������,$|���il}=8�>���_��I�(9l��h\I��4�,�|�]�f�669�[������c����\�yoQ{%��4l6e���"1���VK���Z��&�u*G6.+���.���i�����S?I�|�B�7���[�504N���)VD�G��b?���z�oj%���.Y��{��B:jv����9���}���������� �'U.����U	����k�w>@3�B���=�����[�)���/E�b|GH�r����<������6�U�b-�����y|���������~�����2^���8i��iBVY8&a���4��s����y��m�dV$mP�/b�H�+B�}�u�N$g�����[h��X��1>m^��f68���X��qP�z`����qw�1<\Q������s����8M�2���v��rb�-����tA=\�-L�	{U��/'�����v_��ic37b����p��[N�y}w���io��N+�V�����	%�����t'|q
���S����_�����T�+2�K�@:��0����!���d����e����#t��B���JD��l�������u�'&����=4��V�P�N��K���)�(=���0)mR��������%'V:�W��9���zxdbs�lp��[Mt����b�k\,�?����F)��OLj��������?.��.��S�|��
���Y9���"	��$���8Uk��]4���Vf�W����E�g�?z[U��&�/�w���--����������"�{@T���p���$�_���
�|O�w�'[������������d�T��A��&��������z����9����7�<��WUa���$�_���
�N������������xI�g>���q��2����$�}7�I#TD���C��!��[��<|a��5������O<��~���/Q��(���W,�����r��|x�F�����������'F���_�,���WIDh���l�
�KWU��z�MZ���oi�#����3����_��MI9A5�c�B`�*�I�o6�B����E�n�
b��l5�]���~@�����]M��&[�����os�/��	���0��{A?��hq����c>I�r����@�;�� �ER�uM"�J�r�4l|!��~���Py�s��?�J���_]<�e	�{�����e���� X?����������>us����i�=_�����Fi�E��] ZT � h�b_�E��>`-��=oe������|1m��c�.$���Em��T��D����$��}mki0�o�����������J�����E',�#��
��oS�/�B����_5��F��VP��M��`�\`]��K�9�F�"�4���k�b�Y���D{
��5��H����|!���X���3����:���8������i;��z�[���RA ������ �b|=pR�������������������#@*����|!���X��G��w5u��������;���c�����.���2z����1�|��GD�����E���C*��'�����8'a��{�w�`}#h'Q�F�����t����)gB��os�/�"�����O���w��R���!��I�>�"�u'�]7��6z�f|�q*�9[����A �E
���� �b|�
��=Oe]L���EfQ�-��E��J����{v"9�W���4a��5
��A�;�x����E��4uO=�N�����s����{U��/|�7�;��'������q��K��|�]������t�p]�����\&_�����/��������~�^9�j�����.l��=`��_�_�E��XK���t��[��������[���4�V�[P�/hN�d�7#"O	������7#"�	��yxiR���-�Q���4�����O��X~g���D���a�(�����Ft������b���7��"ES�}�,=l�u�����W��d���D60FLo�z�����^��o�O���x�'�
|A�L6�x�!�#�0,A���|+Id��J4������d0FL��m��I+n������/k�_3���������������rz	�/+��>����;
L�-_N�w��l:@�G��+o�E',���\�/�$���,I��4�,a��Y��W�#WgjG��U����\�g��_�I�U;���"�L}TJ�W�gN�p�0�+������;n���W���t`l���x[�����*�����~n\�M������3uQ[m'�77Q5��x�<v�����[>B���-!�F�=i��G��2��_���f�����D�B[��H�TOM,������>U��Q�������hZ��m�yk�)"�4x!��|���XK�3�Wl.���V���j"��Y��:	��o�j5-��J!�G3�mk��'
�
�*�C��O +�
@��$l|�h���/�z�sG������W����p��@dm;�ZZTy��GYA@����e�|���1���<���9��9����_O%O��N8����h1������+}��a��{���x�g�&%���&���y)`�|W������*�&a����������g�Do��*9�	85��_���f���Kc����j[6$�^Qc��I�>u�~DK���}��N��{�h�������|���XK�3����������o�|z����E��	*��_���`},�WA������[Og�o�!,����5���!��u"�u�N(#�
ar�(l|��a�9({�^����M��y#-�n�����G9�����~Ig���6�!�UE��0���j6�;�|^0�o�����F3�����^�I�o6�B�'�?�t-U>�-(!������_���V�	�/������<�������-(!���[83^��\%_W��3��/"�I�m�et���)���-(!��� O	������;cf����s0���s������TK�~ 2���h��_�~�N�WpV����_�5�?��x�&al7��D����V\}����x!x�'�| yH��	d
_�$��N���
��<&	��zl����_�x�MB�,8+^V}|���w�$�-�\J�^K�y��*�|����8w��<���+�C��O k��B�C.�|������ZT5P{]eF7���
Z4��x�7qE��:���2��f��=`��_o_�5�?@��,�N���(_��\*���D,���5o���>�K�{=6x����`}�ZzP��'�7��������
�����,{�z6���X��C��zsP���h����}9���4U������}7��x-]��;"`Un+'(gUe5�UE�/����w��������T��y�z\S����l<7�'��@L�:��=�\�-�[��O������"��8��K�C+?��w�����#���
�R��nZ�^G��Em��T��D�!�������������h:����i����G���@�	��k���j�eC�UEMsbBV���pI ��n�  9K��M�@@��8�[u�Y��m  �� �������@���l��0�|&���E�@@r�@^��c�o)�L�����ya357U5�dov��C�WSM��ldm�v;{s�����\eloo�2_��<���{O�(����?C���^�{����+�v�r��B�o��������d/��	L��D��V��N�z�VY���^�9-WYk,��*���t3s�U�N��*c���9�������z��F�<�AR��o�r)�f��V��c��	���������a�n�c���!5?L�,kc��[���P�`�zGa*a�����je����N,g��o�6�M~�u��>O9�7��s�oW�nZA�+6QcOL�e��@�a"�C�IDAT���l�# ���UJN��0���u\�E!�]v6u��,��D�2�����}/������.�������!���a�y���4��������m����|��HB6���4\J��b������~4��
�%���
9��z��FZ��������$?
�;��m��gU�cMlZ�N���u�A�e87Ys�w�O0��l���
���n�
�N��ao����m=��/�wZZ_���\���]����[�7o��1��i��������&2	Z�O�����e�T��c{)l\i9�+g;�k��K������(�~5�Y�>gA=N�����%���
]��>	+��f��,�o�
p�[d�c�4E_����=��p>v1��-S�Mv:�"���/9��r�������9��'���a�����?��17'������e���7�S����we�Z�;"�!i�:�����[�`�H� �����`7��@@<�{��   ���
5�� �";���!vC
y@@@�#�G��  n@��PC�H� �����`7��@@<�{��   ���
5�� �";���!vC
y@@@�#�G��  n@��PC�H� �����`7��@@<�{���N���5��7Qu�Q�.jk����������@PA5��t]mT��+�3��VS�N��j��H���z[�O��M���5��TMed.�`���5�^X�s>�
�+�TO ���!s.�-;�&.-�A3��07�o��ud'�]m����4�^aXkyq�X�GG�i��y�C��'Pdj�&,�F�c�w�j'�;�}�c��Y�H�TOM�o3��=����2j�+K���`;����&�Lb�I���\�������l���1����}�'��E3$��V�
�ZA�b�K����vR�9z��R$���[����Rt�?�<RD3���.�r�Y����To�h��<	����E��d���+��A�/�;�^���i�������ULw6����N��s��SKt�K�����l�E�zD���������H�'��#���]F��w��]$q�A���
f�,�4������f�t[ky���W��@��w?��:~��w{�u�Z]s\�>�"�$A5��}�^�m��?�#���7�&e5��3��Mi%gT�������<\Q���Rv��v{�������/8B
�D `!�K���
�{>1�����&(Q�I�����yS��/��A�-������A+V��{������L8�.��
�����_����7]����S�l$��]7*2��84MCE	�;4L�~��?<��e��b�Uz�E���	��F��qt�B�N��{������o_�����q�;%����������g@k�N���������M��E�~��M�30�!/	<������?��@L�y�����K�J�7���!iq�&$+6����X���U��0����2�����M3��D���i��i�z�%���]7�D�c��m�|y������r�)�����$��H�E���������������s��
7Kn��1����������o[|�yTT~j�&�~ q`���d�8�qK��@@@  �Y��[���>  Y �t�@@ ���p���   ����@@���0|@@�@���%��@��   ��,@�-A@@@�m��E���A@@@  ����Iiv=-^IEND�B`�
#4Andres Freund
andres@anarazel.de
In reply to: Bharath Rupireddy (#3)
Re: Avoid LWLockWaitForVar() for currently held WAL insertion lock in WaitXLogInsertionsToFinish()

On 2022-11-25 16:54:19 +0530, Bharath Rupireddy wrote:

On Fri, Nov 25, 2022 at 12:16 AM Andres Freund <andres@anarazel.de> wrote:

I think we could improve this code more significantly by avoiding the call to
LWLockWaitForVar() for all locks that aren't acquired or don't have a
conflicting insertingAt, that'd require just a bit more work to handle systems
without tear-free 64bit writes/reads.

The easiest way would probably be to just make insertingAt a 64bit atomic,
that transparently does the required work to make even non-atomic read/writes
tear free. Then we could trivially avoid the spinlock in
LWLockConflictsWithVar(), LWLockReleaseClearVar() and with just a bit more
work add a fastpath to LWLockUpdateVar(). We don't need to acquire the wait
list lock if there aren't any waiters.

I'd bet that start to have visible effects in a workload with many small
records.

Thanks Andres! I quickly came up with the attached patch. I also ran
an insert test [1], below are the results. I also attached the results
graph. The cirrus-ci is happy with the patch -
https://github.com/BRupireddy/postgres/tree/wal_insertion_lock_improvements_v1_2.
Please let me know if the direction of the patch seems right.
clients HEAD PATCHED
1 1354 1499
2 1451 1464
4 3069 3073
8 5712 5797
16 11331 11157
32 22020 22074
64 41742 42213
128 71300 76638
256 103652 118944
512 111250 161582
768 99544 161987
1024 96743 164161
2048 72711 156686
4096 54158 135713

Nice.

From 293e789f9c1a63748147acd613c556961f1dc5c4 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Fri, 25 Nov 2022 10:53:56 +0000
Subject: [PATCH v1] WAL Insertion Lock Improvements

---
src/backend/access/transam/xlog.c | 8 +++--
src/backend/storage/lmgr/lwlock.c | 56 +++++++++++++++++--------------
src/include/storage/lwlock.h | 7 ++--
3 files changed, 41 insertions(+), 30 deletions(-)

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index a31fbbff78..b3f758abb3 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -376,7 +376,7 @@ typedef struct XLogwrtResult
typedef struct
{
LWLock		lock;
-	XLogRecPtr	insertingAt;
+	pg_atomic_uint64	insertingAt;
XLogRecPtr	lastImportantAt;
} WALInsertLock;

@@ -1482,6 +1482,10 @@ WaitXLogInsertionsToFinish(XLogRecPtr upto)
{
XLogRecPtr insertingat = InvalidXLogRecPtr;

+		/* Quickly check and continue if no one holds the lock. */
+		if (!IsLWLockHeld(&WALInsertLocks[i].l.lock))
+			continue;

I'm not sure this is quite right - don't we need a memory barrier. But I don't
see a reason to not just leave this code as-is. I think this should be
optimized entirely in lwlock.c

I'd probably split the change to an atomic from other changes either way.

Greetings,

Andres Freund

#5Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Andres Freund (#4)
3 attachment(s)
WAL Insertion Lock Improvements (was: Re: Avoid LWLockWaitForVar() for currently held WAL insertion lock in WaitXLogInsertionsToFinish())

On Fri, Dec 2, 2022 at 6:10 AM Andres Freund <andres@anarazel.de> wrote:

On 2022-11-25 16:54:19 +0530, Bharath Rupireddy wrote:

On Fri, Nov 25, 2022 at 12:16 AM Andres Freund <andres@anarazel.de> wrote:

I think we could improve this code more significantly by avoiding the call to
LWLockWaitForVar() for all locks that aren't acquired or don't have a
conflicting insertingAt, that'd require just a bit more work to handle systems
without tear-free 64bit writes/reads.

The easiest way would probably be to just make insertingAt a 64bit atomic,
that transparently does the required work to make even non-atomic read/writes
tear free. Then we could trivially avoid the spinlock in
LWLockConflictsWithVar(), LWLockReleaseClearVar() and with just a bit more
work add a fastpath to LWLockUpdateVar(). We don't need to acquire the wait
list lock if there aren't any waiters.

I'd bet that start to have visible effects in a workload with many small
records.

Thanks Andres! I quickly came up with the attached patch. I also ran
an insert test [1], below are the results. I also attached the results
graph. The cirrus-ci is happy with the patch -
https://github.com/BRupireddy/postgres/tree/wal_insertion_lock_improvements_v1_2.
Please let me know if the direction of the patch seems right.
clients HEAD PATCHED
1 1354 1499
2 1451 1464
4 3069 3073
8 5712 5797
16 11331 11157
32 22020 22074
64 41742 42213
128 71300 76638
256 103652 118944
512 111250 161582
768 99544 161987
1024 96743 164161
2048 72711 156686
4096 54158 135713

Nice.

Thanks for taking a look at it.

From 293e789f9c1a63748147acd613c556961f1dc5c4 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Fri, 25 Nov 2022 10:53:56 +0000
Subject: [PATCH v1] WAL Insertion Lock Improvements

---
src/backend/access/transam/xlog.c | 8 +++--
src/backend/storage/lmgr/lwlock.c | 56 +++++++++++++++++--------------
src/include/storage/lwlock.h | 7 ++--
3 files changed, 41 insertions(+), 30 deletions(-)

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index a31fbbff78..b3f758abb3 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -376,7 +376,7 @@ typedef struct XLogwrtResult
typedef struct
{
LWLock          lock;
-     XLogRecPtr      insertingAt;
+     pg_atomic_uint64        insertingAt;
XLogRecPtr      lastImportantAt;
} WALInsertLock;

@@ -1482,6 +1482,10 @@ WaitXLogInsertionsToFinish(XLogRecPtr upto)
{
XLogRecPtr insertingat = InvalidXLogRecPtr;

+             /* Quickly check and continue if no one holds the lock. */
+             if (!IsLWLockHeld(&WALInsertLocks[i].l.lock))
+                     continue;

I'm not sure this is quite right - don't we need a memory barrier. But I don't
see a reason to not just leave this code as-is. I think this should be
optimized entirely in lwlock.c

Actually, we don't need that at all as LWLockWaitForVar() will return
immediately if the lock is free. So, I removed it.

I'd probably split the change to an atomic from other changes either way.

Done. I've added commit messages to each of the patches.

I've also brought the patch from [1]/messages/by-id/CALj2ACXtQdrGXtb=rbUOXddm1wU1vD9z6q_39FQyX0166dq==A@mail.gmail.com here as 0003.

Thoughts?

[1]: /messages/by-id/CALj2ACXtQdrGXtb=rbUOXddm1wU1vD9z6q_39FQyX0166dq==A@mail.gmail.com

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v2-0001-Make-insertingAt-64-bit-atomic.patchapplication/octet-stream; name=v2-0001-Make-insertingAt-64-bit-atomic.patchDownload
From bbc589de76cd37ffa6f7969f07b4f750a7911846 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Fri, 2 Dec 2022 08:46:23 +0000
Subject: [PATCH v2] Make insertingAt 64-bit atomic

WAL insertion lwlock's insertingAt value is currently read/cleared
with the help of lwlock's wait list lock to avoid torn-free reads
as it is of type XLogRecPtr. This wait list lock can become a point
of contention on a highly concurrent write workloads.

Therefore, make insertingAt a 64-bit atomic which inherently
provides torn-free reads and writes.
---
 src/backend/access/transam/xlog.c |  4 +--
 src/backend/storage/lmgr/lwlock.c | 44 +++++++++++--------------------
 src/include/storage/lwlock.h      |  6 ++---
 3 files changed, 20 insertions(+), 34 deletions(-)

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index c153c32a77..26e841ef6c 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -376,7 +376,7 @@ typedef struct XLogwrtResult
 typedef struct
 {
 	LWLock		lock;
-	XLogRecPtr	insertingAt;
+	pg_atomic_uint64	insertingAt;
 	XLogRecPtr	lastImportantAt;
 } WALInsertLock;
 
@@ -4602,7 +4602,7 @@ XLOGShmemInit(void)
 	for (i = 0; i < NUM_XLOGINSERT_LOCKS; i++)
 	{
 		LWLockInitialize(&WALInsertLocks[i].l.lock, LWTRANCHE_WAL_INSERT);
-		WALInsertLocks[i].l.insertingAt = InvalidXLogRecPtr;
+		pg_atomic_init_u64(&WALInsertLocks[i].l.insertingAt, InvalidXLogRecPtr);
 		WALInsertLocks[i].l.lastImportantAt = InvalidXLogRecPtr;
 	}
 
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index a5ad36ca78..f7556dcb7d 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1546,9 +1546,8 @@ LWLockAcquireOrWait(LWLock *lock, LWLockMode mode)
  * *result is set to true if the lock was free, and false otherwise.
  */
 static bool
-LWLockConflictsWithVar(LWLock *lock,
-					   uint64 *valptr, uint64 oldval, uint64 *newval,
-					   bool *result)
+LWLockConflictsWithVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+					   uint64 *newval, bool *result)
 {
 	bool		mustwait;
 	uint64		value;
@@ -1570,14 +1569,7 @@ LWLockConflictsWithVar(LWLock *lock,
 
 	*result = false;
 
-	/*
-	 * Read value using the lwlock's wait list lock, as we can't generally
-	 * rely on atomic 64 bit reads/stores.  TODO: On platforms with a way to
-	 * do atomic 64 bit reads/writes the spinlock should be optimized away.
-	 */
-	LWLockWaitListLock(lock);
-	value = *valptr;
-	LWLockWaitListUnlock(lock);
+	value = pg_atomic_read_u64(valptr);
 
 	if (value != oldval)
 	{
@@ -1606,7 +1598,8 @@ LWLockConflictsWithVar(LWLock *lock,
  * in shared mode, returns 'true'.
  */
 bool
-LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
+LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+				 uint64 *newval)
 {
 	PGPROC	   *proc = MyProc;
 	int			extraWaits = 0;
@@ -1734,29 +1727,29 @@ LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
  * LWLockUpdateVar - Update a variable and wake up waiters atomically
  *
  * Sets *valptr to 'val', and wakes up all processes waiting for us with
- * LWLockWaitForVar().  Setting the value and waking up the processes happen
- * atomically so that any process calling LWLockWaitForVar() on the same lock
- * is guaranteed to see the new value, and act accordingly.
+ * LWLockWaitForVar().  It first sets the value atomically and then wakes up
+ * the waiting processes so that any process calling LWLockWaitForVar() on the
+ * same lock is guaranteed to see the new value, and act accordingly.
  *
  * The caller must be holding the lock in exclusive mode.
  */
 void
-LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
 	proclist_head wakeup;
 	proclist_mutable_iter iter;
 
 	PRINT_LWDEBUG("LWLockUpdateVar", lock, LW_EXCLUSIVE);
 
+	/* Update the lock's value atomically first. */
+	pg_atomic_write_u64(valptr, val);
+
 	proclist_init(&wakeup);
 
 	LWLockWaitListLock(lock);
 
 	Assert(pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE);
 
-	/* Update the lock's value */
-	*valptr = val;
-
 	/*
 	 * See if there are any LW_WAIT_UNTIL_FREE waiters that need to be woken
 	 * up. They are always in the front of the queue.
@@ -1872,17 +1865,10 @@ LWLockRelease(LWLock *lock)
  * LWLockReleaseClearVar - release a previously acquired lock, reset variable
  */
 void
-LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
-	LWLockWaitListLock(lock);
-
-	/*
-	 * Set the variable's value before releasing the lock, that prevents race
-	 * a race condition wherein a new locker acquires the lock, but hasn't yet
-	 * set the variables value.
-	 */
-	*valptr = val;
-	LWLockWaitListUnlock(lock);
+	/* Update the lock's value atomically */
+	pg_atomic_write_u64(valptr, val);
 
 	LWLockRelease(lock);
 }
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index a494cb598f..4227e59298 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -125,14 +125,14 @@ extern bool LWLockAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockConditionalAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockAcquireOrWait(LWLock *lock, LWLockMode mode);
 extern void LWLockRelease(LWLock *lock);
-extern void LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern void LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 extern void LWLockReleaseAll(void);
 extern bool LWLockHeldByMe(LWLock *lock);
 extern bool LWLockAnyHeldByMe(LWLock *lock, int nlocks, size_t stride);
 extern bool LWLockHeldByMeInMode(LWLock *lock, LWLockMode mode);
 
-extern bool LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval);
-extern void LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern bool LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval, uint64 *newval);
+extern void LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 
 extern Size LWLockShmemSize(void);
 extern void CreateLWLocks(void);
-- 
2.34.1

v2-0002-Add-fastpath-to-LWLockUpdateVar.patchapplication/octet-stream; name=v2-0002-Add-fastpath-to-LWLockUpdateVar.patchDownload
From ab047556c0bc93bc6f5d28cef1975ae9ccdf46f8 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Fri, 2 Dec 2022 10:30:37 +0000
Subject: [PATCH v2] Add fastpath to LWLockUpdateVar()

Add fastpath to LWLockUpdateVar() when there are no waiters. This
avoids unnecessary lwlock's wait list lock acquisition and
release.
---
 src/backend/storage/lmgr/lwlock.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index f7556dcb7d..9d90cf9adf 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1744,6 +1744,13 @@ LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 	/* Update the lock's value atomically first. */
 	pg_atomic_write_u64(valptr, val);
 
+	/*
+	 * Quick exit when there are no waiters. This avoids unnecessary lwlock's
+	 * wait list lock acquisition and release.
+	 */
+	if ((pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) == 0)
+		return;
+
 	proclist_init(&wakeup);
 
 	LWLockWaitListLock(lock);
-- 
2.34.1

v2-0003-Make-lastImportantAt-64-bit-atomic.patchapplication/octet-stream; name=v2-0003-Make-lastImportantAt-64-bit-atomic.patchDownload
From 5b1380553e69f1f85343441697d7c6af0b6a0727 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Fri, 2 Dec 2022 09:24:16 +0000
Subject: [PATCH v2] Make lastImportantAt 64-bit atomic

WAL insertion lwlock's lastImportantAt value is currently read
by GetLastImportantRecPtr() with the help of insertion lock
acquisition and release to avoid torn-free reads. Make it a 64-bit
atomic, which inherently provides torn-free reads and writes, to
reduce unnecessary WAL insertion lock acquisitions and releases.
---
 src/backend/access/transam/xlog.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index ec289e0f70..f40e82ba5f 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -377,7 +377,7 @@ typedef struct
 {
 	LWLock		lock;
 	pg_atomic_uint64	insertingAt;
-	XLogRecPtr	lastImportantAt;
+	pg_atomic_uint64	lastImportantAt;
 } WALInsertLock;
 
 /*
@@ -873,7 +873,7 @@ XLogInsertRecord(XLogRecData *rdata,
 		{
 			int			lockno = holdingAllLocks ? 0 : MyLockNo;
 
-			WALInsertLocks[lockno].l.lastImportantAt = StartPos;
+			pg_atomic_write_u64(&WALInsertLocks[lockno].l.lastImportantAt, StartPos);
 		}
 	}
 	else
@@ -4603,7 +4603,7 @@ XLOGShmemInit(void)
 	{
 		LWLockInitialize(&WALInsertLocks[i].l.lock, LWTRANCHE_WAL_INSERT);
 		pg_atomic_init_u64(&WALInsertLocks[i].l.insertingAt, InvalidXLogRecPtr);
-		WALInsertLocks[i].l.lastImportantAt = InvalidXLogRecPtr;
+		pg_atomic_write_u64(&WALInsertLocks[i].l.lastImportantAt, InvalidXLogRecPtr);
 	}
 
 	/*
@@ -6124,13 +6124,10 @@ GetLastImportantRecPtr(void)
 		XLogRecPtr	last_important;
 
 		/*
-		 * Need to take a lock to prevent torn reads of the LSN, which are
-		 * possible on some of the supported platforms. WAL insert locks only
-		 * support exclusive mode, so we have to use that.
+		 * We atomically read lastImportantAt which prevents torn reads. Hence
+		 * no need to take WAL insert lock here.
 		 */
-		LWLockAcquire(&WALInsertLocks[i].l.lock, LW_EXCLUSIVE);
-		last_important = WALInsertLocks[i].l.lastImportantAt;
-		LWLockRelease(&WALInsertLocks[i].l.lock);
+		last_important = pg_atomic_read_u64(&WALInsertLocks[i].l.lastImportantAt);
 
 		if (res < last_important)
 			res = last_important;
-- 
2.34.1

#6Nathan Bossart
nathandbossart@gmail.com
In reply to: Bharath Rupireddy (#5)
Re: WAL Insertion Lock Improvements (was: Re: Avoid LWLockWaitForVar() for currently held WAL insertion lock in WaitXLogInsertionsToFinish())

On Fri, Dec 02, 2022 at 04:32:38PM +0530, Bharath Rupireddy wrote:

On Fri, Dec 2, 2022 at 6:10 AM Andres Freund <andres@anarazel.de> wrote:

I'm not sure this is quite right - don't we need a memory barrier. But I don't
see a reason to not just leave this code as-is. I think this should be
optimized entirely in lwlock.c

Actually, we don't need that at all as LWLockWaitForVar() will return
immediately if the lock is free. So, I removed it.

I briefly looked at the latest patch set, and I'm curious how this change
avoids introducing memory ordering bugs. Perhaps I am missing something
obvious.

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

#7Andres Freund
andres@anarazel.de
In reply to: Nathan Bossart (#6)
Re: WAL Insertion Lock Improvements (was: Re: Avoid LWLockWaitForVar() for currently held WAL insertion lock in WaitXLogInsertionsToFinish())

Hi,

FWIW, I don't see an advantage in 0003. If it allows us to make something else
simpler / faster, cool, but on its own it doesn't seem worthwhile.

On 2022-12-02 16:31:58 -0800, Nathan Bossart wrote:

On Fri, Dec 02, 2022 at 04:32:38PM +0530, Bharath Rupireddy wrote:

On Fri, Dec 2, 2022 at 6:10 AM Andres Freund <andres@anarazel.de> wrote:

I'm not sure this is quite right - don't we need a memory barrier. But I don't
see a reason to not just leave this code as-is. I think this should be
optimized entirely in lwlock.c

Actually, we don't need that at all as LWLockWaitForVar() will return
immediately if the lock is free. So, I removed it.

I briefly looked at the latest patch set, and I'm curious how this change
avoids introducing memory ordering bugs. Perhaps I am missing something
obvious.

I'm a bit confused too - the comment above talks about LWLockWaitForVar(), but
the patches seem to optimize LWLockUpdateVar().

I think it'd be safe to optimize LWLockConflictsWithVar(), due to some
pre-existing, quite crufty, code. LWLockConflictsWithVar() says:

* Test first to see if it the slot is free right now.
*
* XXX: the caller uses a spinlock before this, so we don't need a memory
* barrier here as far as the current usage is concerned. But that might
* not be safe in general.

which happens to be true in the single, indirect, caller:

/* Read the current insert position */
SpinLockAcquire(&Insert->insertpos_lck);
bytepos = Insert->CurrBytePos;
SpinLockRelease(&Insert->insertpos_lck);
reservedUpto = XLogBytePosToEndRecPtr(bytepos);

I think at the very least we ought to have a comment in
WaitXLogInsertionsToFinish() highlighting this.

It's not at all clear to me that the proposed fast-path for LWLockUpdateVar()
is safe. I think at the very least we could end up missing waiters that we
should have woken up.

I think it ought to be safe to do something like

pg_atomic_exchange_u64()..
if (!(pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS))
return;

because the pg_atomic_exchange_u64() will provide the necessary memory
barrier.

Greetings,

Andres Freund

#8Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Andres Freund (#7)
Re: WAL Insertion Lock Improvements (was: Re: Avoid LWLockWaitForVar() for currently held WAL insertion lock in WaitXLogInsertionsToFinish())

On Tue, Dec 6, 2022 at 12:00 AM Andres Freund <andres@anarazel.de> wrote:

FWIW, I don't see an advantage in 0003. If it allows us to make something else
simpler / faster, cool, but on its own it doesn't seem worthwhile.

Thanks. I will discard it.

I think it'd be safe to optimize LWLockConflictsWithVar(), due to some
pre-existing, quite crufty, code. LWLockConflictsWithVar() says:

* Test first to see if it the slot is free right now.
*
* XXX: the caller uses a spinlock before this, so we don't need a memory
* barrier here as far as the current usage is concerned. But that might
* not be safe in general.

which happens to be true in the single, indirect, caller:

/* Read the current insert position */
SpinLockAcquire(&Insert->insertpos_lck);
bytepos = Insert->CurrBytePos;
SpinLockRelease(&Insert->insertpos_lck);
reservedUpto = XLogBytePosToEndRecPtr(bytepos);

I think at the very least we ought to have a comment in
WaitXLogInsertionsToFinish() highlighting this.

So, using a spinlock ensures no memory ordering occurs while reading
lock->state in LWLockConflictsWithVar()? How does spinlock that gets
acquired and released in the caller WaitXLogInsertionsToFinish()
itself and the memory barrier in the called function
LWLockConflictsWithVar() relate here? Can you please help me
understand this a bit?

It's not at all clear to me that the proposed fast-path for LWLockUpdateVar()
is safe. I think at the very least we could end up missing waiters that we
should have woken up.

I think it ought to be safe to do something like

pg_atomic_exchange_u64()..
if (!(pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS))
return;

pg_atomic_exchange_u64(&lock->state, exchange_with_what_?. Exchange
will change the value no?

because the pg_atomic_exchange_u64() will provide the necessary memory
barrier.

I'm reading some comments [1]* Full barrier semantics. */ static inline uint32 pg_atomic_exchange_u32(volatile pg_atomic_uint32 *ptr,, are these also true for 64-bit atomic
CAS? Does it mean that an atomic CAS operation inherently provides a
memory barrier? Can you please point me if it's described better
somewhere else?

[1]: * Full barrier semantics. */ static inline uint32 pg_atomic_exchange_u32(volatile pg_atomic_uint32 *ptr,
* Full barrier semantics.
*/
static inline uint32
pg_atomic_exchange_u32(volatile pg_atomic_uint32 *ptr,

/*
* Get and clear the flags that are set for this backend. Note that
* pg_atomic_exchange_u32 is a full barrier, so we're guaranteed that the
* read of the barrier generation above happens before we atomically
* extract the flags, and that any subsequent state changes happen
* afterward.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#9Andres Freund
andres@anarazel.de
In reply to: Bharath Rupireddy (#8)
Re: WAL Insertion Lock Improvements (was: Re: Avoid LWLockWaitForVar() for currently held WAL insertion lock in WaitXLogInsertionsToFinish())

Hi,

On 2022-12-08 12:29:54 +0530, Bharath Rupireddy wrote:

On Tue, Dec 6, 2022 at 12:00 AM Andres Freund <andres@anarazel.de> wrote:

I think it'd be safe to optimize LWLockConflictsWithVar(), due to some
pre-existing, quite crufty, code. LWLockConflictsWithVar() says:

* Test first to see if it the slot is free right now.
*
* XXX: the caller uses a spinlock before this, so we don't need a memory
* barrier here as far as the current usage is concerned. But that might
* not be safe in general.

which happens to be true in the single, indirect, caller:

/* Read the current insert position */
SpinLockAcquire(&Insert->insertpos_lck);
bytepos = Insert->CurrBytePos;
SpinLockRelease(&Insert->insertpos_lck);
reservedUpto = XLogBytePosToEndRecPtr(bytepos);

I think at the very least we ought to have a comment in
WaitXLogInsertionsToFinish() highlighting this.

So, using a spinlock ensures no memory ordering occurs while reading
lock->state in LWLockConflictsWithVar()?

No, a spinlock *does* imply ordering. But your patch does remove several of
the spinlock acquisitions (via LWLockWaitListLock()). And moved the assignment
in LWLockUpdateVar() out from under the spinlock.

If you remove spinlock operations (or other barrier primitives), you need to
make sure that such modifications don't break the required memory ordering.

How does spinlock that gets acquired and released in the caller
WaitXLogInsertionsToFinish() itself and the memory barrier in the called
function LWLockConflictsWithVar() relate here? Can you please help me
understand this a bit?

The caller's barrier means that we'll see values that are at least as "up to
date" as at the time of the barrier (it's a bit more complicated than that, a
barrier always needs to be paired with another barrier).

It's not at all clear to me that the proposed fast-path for LWLockUpdateVar()
is safe. I think at the very least we could end up missing waiters that we
should have woken up.

I think it ought to be safe to do something like

pg_atomic_exchange_u64()..
if (!(pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS))
return;

pg_atomic_exchange_u64(&lock->state, exchange_with_what_?. Exchange will
change the value no?

Not lock->state, but the atomic passed to LWLockUpdateVar(), which we do want
to update. An pg_atomic_exchange_u64() includes a memory barrier.

because the pg_atomic_exchange_u64() will provide the necessary memory
barrier.

I'm reading some comments [1], are these also true for 64-bit atomic
CAS?

Yes. See
/* ----
* The 64 bit operations have the same semantics as their 32bit counterparts
* if they are available. Check the corresponding 32bit function for
* documentation.
* ----
*/

Does it mean that an atomic CAS operation inherently provides a
memory barrier?

Yes.

Can you please point me if it's described better somewhere else?

I'm not sure what you'd like to have described more extensively, tbh.

Greetings,

Andres Freund

#10Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Andres Freund (#7)
1 attachment(s)
Re: WAL Insertion Lock Improvements (was: Re: Avoid LWLockWaitForVar() for currently held WAL insertion lock in WaitXLogInsertionsToFinish())

On Tue, Dec 6, 2022 at 12:00 AM Andres Freund <andres@anarazel.de> wrote:

Hi

Thanks for reviewing.

FWIW, I don't see an advantage in 0003. If it allows us to make something else
simpler / faster, cool, but on its own it doesn't seem worthwhile.

I've discarded this change.

On 2022-12-02 16:31:58 -0800, Nathan Bossart wrote:

On Fri, Dec 02, 2022 at 04:32:38PM +0530, Bharath Rupireddy wrote:

On Fri, Dec 2, 2022 at 6:10 AM Andres Freund <andres@anarazel.de> wrote:

I'm not sure this is quite right - don't we need a memory barrier. But I don't
see a reason to not just leave this code as-is. I think this should be
optimized entirely in lwlock.c

Actually, we don't need that at all as LWLockWaitForVar() will return
immediately if the lock is free. So, I removed it.

I briefly looked at the latest patch set, and I'm curious how this change
avoids introducing memory ordering bugs. Perhaps I am missing something
obvious.

I'm a bit confused too - the comment above talks about LWLockWaitForVar(), but
the patches seem to optimize LWLockUpdateVar().

I think it'd be safe to optimize LWLockConflictsWithVar(), due to some
pre-existing, quite crufty, code. LWLockConflictsWithVar() says:

* Test first to see if it the slot is free right now.
*
* XXX: the caller uses a spinlock before this, so we don't need a memory
* barrier here as far as the current usage is concerned. But that might
* not be safe in general.

which happens to be true in the single, indirect, caller:

/* Read the current insert position */
SpinLockAcquire(&Insert->insertpos_lck);
bytepos = Insert->CurrBytePos;
SpinLockRelease(&Insert->insertpos_lck);
reservedUpto = XLogBytePosToEndRecPtr(bytepos);

I think at the very least we ought to have a comment in
WaitXLogInsertionsToFinish() highlighting this.

Done.

It's not at all clear to me that the proposed fast-path for LWLockUpdateVar()
is safe. I think at the very least we could end up missing waiters that we
should have woken up.

I think it ought to be safe to do something like

pg_atomic_exchange_u64()..
if (!(pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS))
return;

because the pg_atomic_exchange_u64() will provide the necessary memory
barrier.

Done.

I'm attaching the v3 patch with the above review comments addressed.
Hopefully, no memory ordering issues now. FWIW, I've added it to CF
https://commitfest.postgresql.org/42/4141/.

Test results with the v3 patch and insert workload are the same as
that of the earlier run - TPS starts to scale at higher clients as
expected after 512 clients and peaks at 2X with 2048 and 4096 clients.

HEAD:
1 1380.411086
2 1358.378988
4 2701.974332
8 5925.380744
16 10956.501237
32 20877.513953
64 40838.046774
128 70251.744161
256 108114.321299
512 120478.988268
768 99140.425209
1024 93645.984364
2048 70111.159909
4096 55541.804826

v3 PATCHED:
1 1493.800209
2 1569.414953
4 3154.186605
8 5965.578904
16 11912.587645
32 22720.964908
64 42001.094528
128 78361.158983
256 110457.926232
512 148941.378393
768 167256.590308
1024 155510.675372
2048 147499.376882
4096 119375.457779

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v3-0001-Optimize-WAL-insertion-lock-acquisition-and-relea.patchapplication/octet-stream; name=v3-0001-Optimize-WAL-insertion-lock-acquisition-and-relea.patchDownload
From 30af9146f8915df08d70845776a4a43d73051e4f Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Tue, 24 Jan 2023 08:10:00 +0000
Subject: [PATCH v3] Optimize WAL insertion lock acquisition and release

This commit optimizes WAL insertion lock acquisition and release
in the following way:

1. WAL insertion lock's variable insertingAt is currently read and
written with the help of lwlock's wait list lock to avoid
torn-free reads/writes. This wait list lock can become a point of
contention on a highly concurrent write workloads. Therefore, make
insertingAt a 64-bit atomic which inherently provides torn-free
reads/writes.

2. LWLockUpdateVar currently acquires lwlock's wait list lock even
when there are no waiters at all. Add a fastpath exit to
LWLockUpdateVar when there are no waiters to avoid unnecessary
locking.

Note that atomic exchange operation (which is a full barrier) is
used when necessary, instead of atomic write to ensure the memory
ordering is preserved.

It also adds a note in WaitXLogInsertionsToFinish regarding how the
use of spinlock there can avoid explicit memory barrier in some
subsequently called functions.

Suggested-by: Andres Freund
Author: Bharath Rupireddy
Reviewed-by: Nathan Bossart
Discussion: https://www.postgresql.org/message-id/20221124184619.xit4sfi52bcz2tva%40awork3.anarazel.de
---
 src/backend/access/transam/xlog.c | 14 +++++--
 src/backend/storage/lmgr/lwlock.c | 66 +++++++++++++++++++------------
 src/include/storage/lwlock.h      |  6 +--
 3 files changed, 55 insertions(+), 31 deletions(-)

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bde..4e427c85c2 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -350,7 +350,8 @@ typedef struct XLogwrtResult
  * wait for all currently in-progress insertions to finish, but the
  * insertingAt indicator allows you to ignore insertions to later in the WAL,
  * so that you only wait for the insertions that are modifying the buffers
- * you're about to write out.
+ * you're about to write out. Using an atomic variable for insertingAt avoids
+ * taking extra lock for reads and writes.
  *
  * This isn't just an optimization. If all the WAL buffers are dirty, an
  * inserter that's holding a WAL insert lock might need to evict an old WAL
@@ -376,7 +377,7 @@ typedef struct XLogwrtResult
 typedef struct
 {
 	LWLock		lock;
-	XLogRecPtr	insertingAt;
+	pg_atomic_uint64	insertingAt;
 	XLogRecPtr	lastImportantAt;
 } WALInsertLock;
 
@@ -1496,6 +1497,13 @@ WaitXLogInsertionsToFinish(XLogRecPtr upto)
 			 * calling LWLockUpdateVar.  But if it has to sleep, it will
 			 * advertise the insertion point with LWLockUpdateVar before
 			 * sleeping.
+			 *
+			 * XXX: Use of a spinlock at the beginning of this function to read
+			 * current insert position in this here implies memory ordering.
+			 * That means that the immediate loads and stores to shared memory,
+			 * for instance in LWLockUpdateVar called via LWLockWaitForVar,
+			 * don't need an explicit memory barrier as far as the current
+			 * usage is concerned. But that might not be safe in general.
 			 */
 			if (LWLockWaitForVar(&WALInsertLocks[i].l.lock,
 								 &WALInsertLocks[i].l.insertingAt,
@@ -4596,7 +4604,7 @@ XLOGShmemInit(void)
 	for (i = 0; i < NUM_XLOGINSERT_LOCKS; i++)
 	{
 		LWLockInitialize(&WALInsertLocks[i].l.lock, LWTRANCHE_WAL_INSERT);
-		WALInsertLocks[i].l.insertingAt = InvalidXLogRecPtr;
+		pg_atomic_init_u64(&WALInsertLocks[i].l.insertingAt, InvalidXLogRecPtr);
 		WALInsertLocks[i].l.lastImportantAt = InvalidXLogRecPtr;
 	}
 
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index d2ec396045..27c3b63c68 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1547,9 +1547,8 @@ LWLockAcquireOrWait(LWLock *lock, LWLockMode mode)
  * *result is set to true if the lock was free, and false otherwise.
  */
 static bool
-LWLockConflictsWithVar(LWLock *lock,
-					   uint64 *valptr, uint64 oldval, uint64 *newval,
-					   bool *result)
+LWLockConflictsWithVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+					   uint64 *newval, bool *result)
 {
 	bool		mustwait;
 	uint64		value;
@@ -1572,13 +1571,11 @@ LWLockConflictsWithVar(LWLock *lock,
 	*result = false;
 
 	/*
-	 * Read value using the lwlock's wait list lock, as we can't generally
-	 * rely on atomic 64 bit reads/stores.  TODO: On platforms with a way to
-	 * do atomic 64 bit reads/writes the spinlock should be optimized away.
+	 * Read value atomically without any explicit lock. We rely on 64-bit
+	 * atomic reads/writes that transparently does the required work to make
+	 * even non-atomic reads/writes tear free.
 	 */
-	LWLockWaitListLock(lock);
-	value = *valptr;
-	LWLockWaitListUnlock(lock);
+	value = pg_atomic_read_u64(valptr);
 
 	if (value != oldval)
 	{
@@ -1607,7 +1604,8 @@ LWLockConflictsWithVar(LWLock *lock,
  * in shared mode, returns 'true'.
  */
 bool
-LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
+LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+				 uint64 *newval)
 {
 	PGPROC	   *proc = MyProc;
 	int			extraWaits = 0;
@@ -1735,29 +1733,47 @@ LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
  * LWLockUpdateVar - Update a variable and wake up waiters atomically
  *
  * Sets *valptr to 'val', and wakes up all processes waiting for us with
- * LWLockWaitForVar().  Setting the value and waking up the processes happen
- * atomically so that any process calling LWLockWaitForVar() on the same lock
- * is guaranteed to see the new value, and act accordingly.
+ * LWLockWaitForVar().  It first sets the value atomically and then wakes up
+ * the waiting processes so that any process calling LWLockWaitForVar() on the
+ * same lock is guaranteed to see the new value, and act accordingly.
  *
  * The caller must be holding the lock in exclusive mode.
  */
 void
-LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
 	proclist_head wakeup;
 	proclist_mutable_iter iter;
 
 	PRINT_LWDEBUG("LWLockUpdateVar", lock, LW_EXCLUSIVE);
 
+	/*
+	 * Update the lock variable atomically first without having to acquire wait
+	 * list lock, so that if anyone looking for the lock will have chance to
+	 * grab it a bit quickly.
+	 *
+	 * NB: Note the use of pg_atomic_exchange_u64 as opposed to just
+	 * pg_atomic_write_u64 to update the value. Since pg_atomic_exchange_u64 is
+	 * a full barrier, we're guaranteed that the subsequent atomic read of lock
+	 * state to check if it has any waiters happens after we set the lock
+	 * variable to new value here. Without a barrier, we could end up missing
+	 * waiters that otherwise should have been woken up.
+	 */
+	pg_atomic_exchange_u64(valptr, val);
+
+	/*
+	 * Quick exit when there are no waiters. This avoids unnecessary lwlock's
+	 * wait list lock acquisition and release.
+	 */
+	if ((pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) == 0)
+		return;
+
 	proclist_init(&wakeup);
 
 	LWLockWaitListLock(lock);
 
 	Assert(pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE);
 
-	/* Update the lock's value */
-	*valptr = val;
-
 	/*
 	 * See if there are any LW_WAIT_UNTIL_FREE waiters that need to be woken
 	 * up. They are always in the front of the queue.
@@ -1873,17 +1889,17 @@ LWLockRelease(LWLock *lock)
  * LWLockReleaseClearVar - release a previously acquired lock, reset variable
  */
 void
-LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
-	LWLockWaitListLock(lock);
-
 	/*
-	 * Set the variable's value before releasing the lock, that prevents race
-	 * a race condition wherein a new locker acquires the lock, but hasn't yet
-	 * set the variables value.
+	 * Update the lock variable atomically first.
+	 *
+	 * NB: Note the use of pg_atomic_exchange_u64 as opposed to just
+	 * pg_atomic_write_u64 to update the value. Since pg_atomic_exchange_u64 is
+	 * a full barrier, we're guaranteed that the subsequent shared memory
+	 * reads/writes, if any, happen after we reset the lock variable.
 	 */
-	*valptr = val;
-	LWLockWaitListUnlock(lock);
+	pg_atomic_exchange_u64(valptr, val);
 
 	LWLockRelease(lock);
 }
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index d2c7afb8f4..f19bc49193 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -128,14 +128,14 @@ extern bool LWLockAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockConditionalAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockAcquireOrWait(LWLock *lock, LWLockMode mode);
 extern void LWLockRelease(LWLock *lock);
-extern void LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern void LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 extern void LWLockReleaseAll(void);
 extern bool LWLockHeldByMe(LWLock *lock);
 extern bool LWLockAnyHeldByMe(LWLock *lock, int nlocks, size_t stride);
 extern bool LWLockHeldByMeInMode(LWLock *lock, LWLockMode mode);
 
-extern bool LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval);
-extern void LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern bool LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval, uint64 *newval);
+extern void LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 
 extern Size LWLockShmemSize(void);
 extern void CreateLWLocks(void);
-- 
2.34.1

#11Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Bharath Rupireddy (#10)
1 attachment(s)
Re: WAL Insertion Lock Improvements

On Tue, Jan 24, 2023 at 7:00 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

I'm attaching the v3 patch with the above review comments addressed.
Hopefully, no memory ordering issues now. FWIW, I've added it to CF
https://commitfest.postgresql.org/42/4141/.

Test results with the v3 patch and insert workload are the same as
that of the earlier run - TPS starts to scale at higher clients as
expected after 512 clients and peaks at 2X with 2048 and 4096 clients.

HEAD:
1 1380.411086
2 1358.378988
4 2701.974332
8 5925.380744
16 10956.501237
32 20877.513953
64 40838.046774
128 70251.744161
256 108114.321299
512 120478.988268
768 99140.425209
1024 93645.984364
2048 70111.159909
4096 55541.804826

v3 PATCHED:
1 1493.800209
2 1569.414953
4 3154.186605
8 5965.578904
16 11912.587645
32 22720.964908
64 42001.094528
128 78361.158983
256 110457.926232
512 148941.378393
768 167256.590308
1024 155510.675372
2048 147499.376882
4096 119375.457779

I slightly modified the comments and attached the v4 patch for further
review. I also took perf report - there's a clear reduction in the
functions that are affected by the patch - LWLockWaitListLock,
WaitXLogInsertionsToFinish, LWLockWaitForVar and
LWLockConflictsWithVar. Note that I compiled the source code with
-ggdb for capturing symbols for perf, still the benefit stands at > 2X
for a higher number of clients.

HEAD:
+   16.87%     0.01%  postgres  [.] CommitTransactionCommand
+   16.86%     0.00%  postgres  [.] finish_xact_command
+   16.81%     0.01%  postgres  [.] CommitTransaction
+   15.09%     0.20%  postgres  [.] LWLockWaitListLock
+   14.53%     0.01%  postgres  [.] WaitXLogInsertionsToFinish
+   14.51%     0.02%  postgres  [.] LWLockWaitForVar
+   11.70%    11.63%  postgres  [.] pg_atomic_read_u32_impl
+   11.66%     0.08%  postgres  [.] pg_atomic_read_u32
+    9.96%     0.03%  postgres  [.] LWLockConflictsWithVar
+    4.78%     0.00%  postgres  [.] LWLockQueueSelf
+    1.91%     0.01%  postgres  [.] pg_atomic_fetch_or_u32
+    1.91%     1.89%  postgres  [.] pg_atomic_fetch_or_u32_impl
+    1.73%     0.00%  postgres  [.] XLogInsert
+    1.69%     0.01%  postgres  [.] XLogInsertRecord
+    1.41%     0.01%  postgres  [.] LWLockRelease
+    1.37%     0.47%  postgres  [.] perform_spin_delay
+    1.11%     1.11%  postgres  [.] spin_delay
+    1.10%     0.03%  postgres  [.] exec_bind_message
+    0.91%     0.00%  postgres  [.] WALInsertLockRelease
+    0.91%     0.00%  postgres  [.] LWLockReleaseClearVar
+    0.72%     0.02%  postgres  [.] LWLockAcquire
+    0.60%     0.00%  postgres  [.] LWLockDequeueSelf
+    0.58%     0.00%  postgres  [.] GetTransactionSnapshot
     0.58%     0.49%  postgres  [.] GetSnapshotData
+    0.58%     0.00%  postgres  [.] WALInsertLockAcquire
+    0.55%     0.00%  postgres  [.] XactLogCommitRecord

TPS (compiled with -ggdb for capturing symbols for perf)
1 1392.512967
2 1435.899119
4 3104.091923
8 6159.305522
16 11477.641780
32 22701.000718
64 41662.425880
128 23743.426209
256 89837.651619
512 65164.221500
768 66015.733370
1024 56421.223080
2048 52909.018072
4096 40071.146985

PATCHED:
+    2.19%     0.05%  postgres  [.] LWLockWaitListLock
+    2.10%     0.01%  postgres  [.] LWLockQueueSelf
+    1.73%     1.71%  postgres  [.] pg_atomic_read_u32_impl
+    1.73%     0.02%  postgres  [.] pg_atomic_read_u32
+    1.72%     0.02%  postgres  [.] LWLockRelease
+    1.65%     0.04%  postgres  [.] exec_bind_message
+    1.43%     0.00%  postgres  [.] XLogInsert
+    1.42%     0.01%  postgres  [.] WaitXLogInsertionsToFinish
+    1.40%     0.03%  postgres  [.] LWLockWaitForVar
+    1.38%     0.02%  postgres  [.] XLogInsertRecord
+    0.93%     0.03%  postgres  [.] LWLockAcquireOrWait
+    0.91%     0.00%  postgres  [.] GetTransactionSnapshot
+    0.91%     0.79%  postgres  [.] GetSnapshotData
+    0.91%     0.00%  postgres  [.] WALInsertLockRelease
+    0.91%     0.00%  postgres  [.] LWLockReleaseClearVar
+    0.53%     0.02%  postgres  [.] ExecInitModifyTable

TPS (compiled with -ggdb for capturing symbols for perf)
1 1295.296611
2 1459.079162
4 2865.688987
8 5533.724983
16 10771.697842
32 20557.499312
64 39436.423783
128 42555.639048
256 73139.060227
512 124649.665196
768 131162.826976
1024 132185.160007
2048 117377.586644
4096 88240.336940

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v4-0001-Optimize-WAL-insertion-lock-acquisition-and-relea.patchapplication/x-patch; name=v4-0001-Optimize-WAL-insertion-lock-acquisition-and-relea.patchDownload
From 74c5bd8cc4f1497aa7f2fa02c6487039dc91e847 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Thu, 2 Feb 2023 03:42:27 +0000
Subject: [PATCH v4] Optimize WAL insertion lock acquisition and release

This commit optimizes WAL insertion lock acquisition and release
in the following way:

1. WAL insertion lock's variable insertingAt is currently read and
written with the help of lwlock's wait list lock to avoid
torn-free reads/writes. This wait list lock can become a point of
contention on a highly concurrent write workloads. Therefore, make
insertingAt a 64-bit atomic which inherently provides torn-free
reads/writes.

2. LWLockUpdateVar currently acquires lwlock's wait list lock even
when there are no waiters at all. Add a fastpath exit to
LWLockUpdateVar when there are no waiters to avoid unnecessary
locking.

Note that atomic exchange operation (which is a full barrier) is
used when necessary, instead of atomic write to ensure the memory
ordering is preserved.

It also adds a note in WaitXLogInsertionsToFinish regarding how the
use of spinlock there can avoid explicit memory barrier in some
subsequently called functions.

Suggested-by: Andres Freund
Author: Bharath Rupireddy
Reviewed-by: Nathan Bossart
Discussion: https://www.postgresql.org/message-id/20221124184619.xit4sfi52bcz2tva%40awork3.anarazel.de
---
 src/backend/access/transam/xlog.c | 14 +++++--
 src/backend/storage/lmgr/lwlock.c | 66 +++++++++++++++++++------------
 src/include/storage/lwlock.h      |  6 +--
 3 files changed, 55 insertions(+), 31 deletions(-)

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index fb4c860bde..95aed0e97f 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -350,7 +350,8 @@ typedef struct XLogwrtResult
  * wait for all currently in-progress insertions to finish, but the
  * insertingAt indicator allows you to ignore insertions to later in the WAL,
  * so that you only wait for the insertions that are modifying the buffers
- * you're about to write out.
+ * you're about to write out. Using an atomic variable for insertingAt avoids
+ * taking any explicit lock for reads and writes.
  *
  * This isn't just an optimization. If all the WAL buffers are dirty, an
  * inserter that's holding a WAL insert lock might need to evict an old WAL
@@ -376,7 +377,7 @@ typedef struct XLogwrtResult
 typedef struct
 {
 	LWLock		lock;
-	XLogRecPtr	insertingAt;
+	pg_atomic_uint64	insertingAt;
 	XLogRecPtr	lastImportantAt;
 } WALInsertLock;
 
@@ -1496,6 +1497,13 @@ WaitXLogInsertionsToFinish(XLogRecPtr upto)
 			 * calling LWLockUpdateVar.  But if it has to sleep, it will
 			 * advertise the insertion point with LWLockUpdateVar before
 			 * sleeping.
+			 *
+			 * XXX: Use of a spinlock at the beginning of this function to read
+			 * current insert position implies memory ordering. That means that
+			 * the immediate loads and stores to shared memory (for instance,
+			 * in LWLockUpdateVar called via LWLockWaitForVar) don't need an
+			 * explicit memory barrier as far as the current usage is
+			 * concerned. But that might not be safe in general.
 			 */
 			if (LWLockWaitForVar(&WALInsertLocks[i].l.lock,
 								 &WALInsertLocks[i].l.insertingAt,
@@ -4596,7 +4604,7 @@ XLOGShmemInit(void)
 	for (i = 0; i < NUM_XLOGINSERT_LOCKS; i++)
 	{
 		LWLockInitialize(&WALInsertLocks[i].l.lock, LWTRANCHE_WAL_INSERT);
-		WALInsertLocks[i].l.insertingAt = InvalidXLogRecPtr;
+		pg_atomic_init_u64(&WALInsertLocks[i].l.insertingAt, InvalidXLogRecPtr);
 		WALInsertLocks[i].l.lastImportantAt = InvalidXLogRecPtr;
 	}
 
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index d2ec396045..27c3b63c68 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1547,9 +1547,8 @@ LWLockAcquireOrWait(LWLock *lock, LWLockMode mode)
  * *result is set to true if the lock was free, and false otherwise.
  */
 static bool
-LWLockConflictsWithVar(LWLock *lock,
-					   uint64 *valptr, uint64 oldval, uint64 *newval,
-					   bool *result)
+LWLockConflictsWithVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+					   uint64 *newval, bool *result)
 {
 	bool		mustwait;
 	uint64		value;
@@ -1572,13 +1571,11 @@ LWLockConflictsWithVar(LWLock *lock,
 	*result = false;
 
 	/*
-	 * Read value using the lwlock's wait list lock, as we can't generally
-	 * rely on atomic 64 bit reads/stores.  TODO: On platforms with a way to
-	 * do atomic 64 bit reads/writes the spinlock should be optimized away.
+	 * Read value atomically without any explicit lock. We rely on 64-bit
+	 * atomic reads/writes that transparently does the required work to make
+	 * even non-atomic reads/writes tear free.
 	 */
-	LWLockWaitListLock(lock);
-	value = *valptr;
-	LWLockWaitListUnlock(lock);
+	value = pg_atomic_read_u64(valptr);
 
 	if (value != oldval)
 	{
@@ -1607,7 +1604,8 @@ LWLockConflictsWithVar(LWLock *lock,
  * in shared mode, returns 'true'.
  */
 bool
-LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
+LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+				 uint64 *newval)
 {
 	PGPROC	   *proc = MyProc;
 	int			extraWaits = 0;
@@ -1735,29 +1733,47 @@ LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
  * LWLockUpdateVar - Update a variable and wake up waiters atomically
  *
  * Sets *valptr to 'val', and wakes up all processes waiting for us with
- * LWLockWaitForVar().  Setting the value and waking up the processes happen
- * atomically so that any process calling LWLockWaitForVar() on the same lock
- * is guaranteed to see the new value, and act accordingly.
+ * LWLockWaitForVar().  It first sets the value atomically and then wakes up
+ * the waiting processes so that any process calling LWLockWaitForVar() on the
+ * same lock is guaranteed to see the new value, and act accordingly.
  *
  * The caller must be holding the lock in exclusive mode.
  */
 void
-LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
 	proclist_head wakeup;
 	proclist_mutable_iter iter;
 
 	PRINT_LWDEBUG("LWLockUpdateVar", lock, LW_EXCLUSIVE);
 
+	/*
+	 * Update the lock variable atomically first without having to acquire wait
+	 * list lock, so that if anyone looking for the lock will have chance to
+	 * grab it a bit quickly.
+	 *
+	 * NB: Note the use of pg_atomic_exchange_u64 as opposed to just
+	 * pg_atomic_write_u64 to update the value. Since pg_atomic_exchange_u64 is
+	 * a full barrier, we're guaranteed that the subsequent atomic read of lock
+	 * state to check if it has any waiters happens after we set the lock
+	 * variable to new value here. Without a barrier, we could end up missing
+	 * waiters that otherwise should have been woken up.
+	 */
+	pg_atomic_exchange_u64(valptr, val);
+
+	/*
+	 * Quick exit when there are no waiters. This avoids unnecessary lwlock's
+	 * wait list lock acquisition and release.
+	 */
+	if ((pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) == 0)
+		return;
+
 	proclist_init(&wakeup);
 
 	LWLockWaitListLock(lock);
 
 	Assert(pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE);
 
-	/* Update the lock's value */
-	*valptr = val;
-
 	/*
 	 * See if there are any LW_WAIT_UNTIL_FREE waiters that need to be woken
 	 * up. They are always in the front of the queue.
@@ -1873,17 +1889,17 @@ LWLockRelease(LWLock *lock)
  * LWLockReleaseClearVar - release a previously acquired lock, reset variable
  */
 void
-LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
-	LWLockWaitListLock(lock);
-
 	/*
-	 * Set the variable's value before releasing the lock, that prevents race
-	 * a race condition wherein a new locker acquires the lock, but hasn't yet
-	 * set the variables value.
+	 * Update the lock variable atomically first.
+	 *
+	 * NB: Note the use of pg_atomic_exchange_u64 as opposed to just
+	 * pg_atomic_write_u64 to update the value. Since pg_atomic_exchange_u64 is
+	 * a full barrier, we're guaranteed that the subsequent shared memory
+	 * reads/writes, if any, happen after we reset the lock variable.
 	 */
-	*valptr = val;
-	LWLockWaitListUnlock(lock);
+	pg_atomic_exchange_u64(valptr, val);
 
 	LWLockRelease(lock);
 }
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index d2c7afb8f4..f19bc49193 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -128,14 +128,14 @@ extern bool LWLockAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockConditionalAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockAcquireOrWait(LWLock *lock, LWLockMode mode);
 extern void LWLockRelease(LWLock *lock);
-extern void LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern void LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 extern void LWLockReleaseAll(void);
 extern bool LWLockHeldByMe(LWLock *lock);
 extern bool LWLockAnyHeldByMe(LWLock *lock, int nlocks, size_t stride);
 extern bool LWLockHeldByMeInMode(LWLock *lock, LWLockMode mode);
 
-extern bool LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval);
-extern void LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern bool LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval, uint64 *newval);
+extern void LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 
 extern Size LWLockShmemSize(void);
 extern void CreateLWLocks(void);
-- 
2.34.1

#12Nathan Bossart
nathandbossart@gmail.com
In reply to: Bharath Rupireddy (#11)
Re: WAL Insertion Lock Improvements

+ pg_atomic_exchange_u64(valptr, val);

nitpick: I'd add a (void) at the beginning of these calls to
pg_atomic_exchange_u64() so that it's clear that we are discarding the
return value.

+	/*
+	 * Update the lock variable atomically first without having to acquire wait
+	 * list lock, so that if anyone looking for the lock will have chance to
+	 * grab it a bit quickly.
+	 *
+	 * NB: Note the use of pg_atomic_exchange_u64 as opposed to just
+	 * pg_atomic_write_u64 to update the value. Since pg_atomic_exchange_u64 is
+	 * a full barrier, we're guaranteed that the subsequent atomic read of lock
+	 * state to check if it has any waiters happens after we set the lock
+	 * variable to new value here. Without a barrier, we could end up missing
+	 * waiters that otherwise should have been woken up.
+	 */
+	pg_atomic_exchange_u64(valptr, val);
+
+	/*
+	 * Quick exit when there are no waiters. This avoids unnecessary lwlock's
+	 * wait list lock acquisition and release.
+	 */
+	if ((pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) == 0)
+		return;

I think this makes sense. A waiter could queue itself after the exchange,
but it'll recheck after queueing. IIUC this is basically how this works
today. We update the value and release the lock before waking up any
waiters, so the same principle applies.

Overall, I think this patch is in reasonable shape.

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

#13Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Nathan Bossart (#12)
1 attachment(s)
Re: WAL Insertion Lock Improvements

On Thu, Feb 9, 2023 at 3:36 AM Nathan Bossart <nathandbossart@gmail.com> wrote:

+ pg_atomic_exchange_u64(valptr, val);

nitpick: I'd add a (void) at the beginning of these calls to
pg_atomic_exchange_u64() so that it's clear that we are discarding the
return value.

I did that in the attached v5 patch although it's a mix elsewhere;
some doing explicit return value cast with (void) and some not.

+       /*
+        * Update the lock variable atomically first without having to acquire wait
+        * list lock, so that if anyone looking for the lock will have chance to
+        * grab it a bit quickly.
+        *
+        * NB: Note the use of pg_atomic_exchange_u64 as opposed to just
+        * pg_atomic_write_u64 to update the value. Since pg_atomic_exchange_u64 is
+        * a full barrier, we're guaranteed that the subsequent atomic read of lock
+        * state to check if it has any waiters happens after we set the lock
+        * variable to new value here. Without a barrier, we could end up missing
+        * waiters that otherwise should have been woken up.
+        */
+       pg_atomic_exchange_u64(valptr, val);
+
+       /*
+        * Quick exit when there are no waiters. This avoids unnecessary lwlock's
+        * wait list lock acquisition and release.
+        */
+       if ((pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) == 0)
+               return;

I think this makes sense. A waiter could queue itself after the exchange,
but it'll recheck after queueing. IIUC this is basically how this works
today. We update the value and release the lock before waking up any
waiters, so the same principle applies.

Yes, a waiter right after self-queuing (LWLockQueueSelf) checks for
the value (LWLockConflictsWithVar) before it goes and waits until
awakened in LWLockWaitForVar. A waiter added to the queue is
guaranteed to be woken up by the
LWLockUpdateVar but before that the lock value is set and we have
pg_atomic_exchange_u64 as a memory barrier, so no memory reordering.
Essentially, the order of these operations aren't changed. The benefit
that we're seeing is from avoiding LWLock's waitlist lock for reading
and updating the lock value relying on 64-bit atomics.

Overall, I think this patch is in reasonable shape.

Thanks for reviewing. Please see the attached v5 patch.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v5-0001-Optimize-WAL-insertion-lock-acquisition-and-relea.patchapplication/octet-stream; name=v5-0001-Optimize-WAL-insertion-lock-acquisition-and-relea.patchDownload
From d13f00494e7a74caa41dd2d79b8b05add6dc5c28 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Thu, 9 Feb 2023 02:42:58 +0000
Subject: [PATCH v5] Optimize WAL insertion lock acquisition and release

This commit optimizes WAL insertion lock acquisition and release
in the following way:

1. WAL insertion lock's variable insertingAt is currently read and
written with the help of lwlock's wait list lock to avoid
torn-free reads/writes. This wait list lock can become a point of
contention on a highly concurrent write workloads. Therefore, make
insertingAt a 64-bit atomic which inherently provides torn-free
reads/writes.

2. LWLockUpdateVar currently acquires lwlock's wait list lock even
when there are no waiters at all. Add a fastpath exit to
LWLockUpdateVar when there are no waiters to avoid unnecessary
locking.

Note that atomic exchange operation (which is a full barrier) is
used when necessary, instead of atomic write to ensure the memory
ordering is preserved.

It also adds a note in WaitXLogInsertionsToFinish regarding how the
use of spinlock there can avoid explicit memory barrier in some
subsequently called functions.

Suggested-by: Andres Freund
Author: Bharath Rupireddy
Reviewed-by: Nathan Bossart
Discussion: https://www.postgresql.org/message-id/20221124184619.xit4sfi52bcz2tva%40awork3.anarazel.de
---
 src/backend/access/transam/xlog.c | 14 +++++--
 src/backend/storage/lmgr/lwlock.c | 66 +++++++++++++++++++------------
 src/include/storage/lwlock.h      |  6 +--
 3 files changed, 55 insertions(+), 31 deletions(-)

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index f9f0f6db8d..affb8c77c5 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -350,7 +350,8 @@ typedef struct XLogwrtResult
  * wait for all currently in-progress insertions to finish, but the
  * insertingAt indicator allows you to ignore insertions to later in the WAL,
  * so that you only wait for the insertions that are modifying the buffers
- * you're about to write out.
+ * you're about to write out. Using an atomic variable for insertingAt avoids
+ * taking any explicit lock for reads and writes.
  *
  * This isn't just an optimization. If all the WAL buffers are dirty, an
  * inserter that's holding a WAL insert lock might need to evict an old WAL
@@ -376,7 +377,7 @@ typedef struct XLogwrtResult
 typedef struct
 {
 	LWLock		lock;
-	XLogRecPtr	insertingAt;
+	pg_atomic_uint64	insertingAt;
 	XLogRecPtr	lastImportantAt;
 } WALInsertLock;
 
@@ -1495,6 +1496,13 @@ WaitXLogInsertionsToFinish(XLogRecPtr upto)
 			 * calling LWLockUpdateVar.  But if it has to sleep, it will
 			 * advertise the insertion point with LWLockUpdateVar before
 			 * sleeping.
+			 *
+			 * XXX: Use of a spinlock at the beginning of this function to read
+			 * current insert position implies memory ordering. That means that
+			 * the immediate loads and stores to shared memory (for instance,
+			 * in LWLockUpdateVar called via LWLockWaitForVar) don't need an
+			 * explicit memory barrier as far as the current usage is
+			 * concerned. But that might not be safe in general.
 			 */
 			if (LWLockWaitForVar(&WALInsertLocks[i].l.lock,
 								 &WALInsertLocks[i].l.insertingAt,
@@ -4595,7 +4603,7 @@ XLOGShmemInit(void)
 	for (i = 0; i < NUM_XLOGINSERT_LOCKS; i++)
 	{
 		LWLockInitialize(&WALInsertLocks[i].l.lock, LWTRANCHE_WAL_INSERT);
-		WALInsertLocks[i].l.insertingAt = InvalidXLogRecPtr;
+		pg_atomic_init_u64(&WALInsertLocks[i].l.insertingAt, InvalidXLogRecPtr);
 		WALInsertLocks[i].l.lastImportantAt = InvalidXLogRecPtr;
 	}
 
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index d2ec396045..3a7e3e11ba 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1547,9 +1547,8 @@ LWLockAcquireOrWait(LWLock *lock, LWLockMode mode)
  * *result is set to true if the lock was free, and false otherwise.
  */
 static bool
-LWLockConflictsWithVar(LWLock *lock,
-					   uint64 *valptr, uint64 oldval, uint64 *newval,
-					   bool *result)
+LWLockConflictsWithVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+					   uint64 *newval, bool *result)
 {
 	bool		mustwait;
 	uint64		value;
@@ -1572,13 +1571,11 @@ LWLockConflictsWithVar(LWLock *lock,
 	*result = false;
 
 	/*
-	 * Read value using the lwlock's wait list lock, as we can't generally
-	 * rely on atomic 64 bit reads/stores.  TODO: On platforms with a way to
-	 * do atomic 64 bit reads/writes the spinlock should be optimized away.
+	 * Read value atomically without any explicit lock. We rely on 64-bit
+	 * atomic reads/writes that transparently does the required work to make
+	 * even non-atomic reads/writes tear free.
 	 */
-	LWLockWaitListLock(lock);
-	value = *valptr;
-	LWLockWaitListUnlock(lock);
+	value = pg_atomic_read_u64(valptr);
 
 	if (value != oldval)
 	{
@@ -1607,7 +1604,8 @@ LWLockConflictsWithVar(LWLock *lock,
  * in shared mode, returns 'true'.
  */
 bool
-LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
+LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+				 uint64 *newval)
 {
 	PGPROC	   *proc = MyProc;
 	int			extraWaits = 0;
@@ -1735,29 +1733,47 @@ LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
  * LWLockUpdateVar - Update a variable and wake up waiters atomically
  *
  * Sets *valptr to 'val', and wakes up all processes waiting for us with
- * LWLockWaitForVar().  Setting the value and waking up the processes happen
- * atomically so that any process calling LWLockWaitForVar() on the same lock
- * is guaranteed to see the new value, and act accordingly.
+ * LWLockWaitForVar().  It first sets the value atomically and then wakes up
+ * the waiting processes so that any process calling LWLockWaitForVar() on the
+ * same lock is guaranteed to see the new value, and act accordingly.
  *
  * The caller must be holding the lock in exclusive mode.
  */
 void
-LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
 	proclist_head wakeup;
 	proclist_mutable_iter iter;
 
 	PRINT_LWDEBUG("LWLockUpdateVar", lock, LW_EXCLUSIVE);
 
+	/*
+	 * Update the lock variable atomically first without having to acquire wait
+	 * list lock, so that if anyone looking for the lock will have chance to
+	 * grab it a bit quickly.
+	 *
+	 * NB: Note the use of pg_atomic_exchange_u64 as opposed to just
+	 * pg_atomic_write_u64 to update the value. Since pg_atomic_exchange_u64 is
+	 * a full barrier, we're guaranteed that the subsequent atomic read of lock
+	 * state to check if it has any waiters happens after we set the lock
+	 * variable to new value here. Without a barrier, we could end up missing
+	 * waiters that otherwise should have been woken up.
+	 */
+	(void) pg_atomic_exchange_u64(valptr, val);
+
+	/*
+	 * Quick exit when there are no waiters. This avoids unnecessary lwlock's
+	 * wait list lock acquisition and release.
+	 */
+	if ((pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) == 0)
+		return;
+
 	proclist_init(&wakeup);
 
 	LWLockWaitListLock(lock);
 
 	Assert(pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE);
 
-	/* Update the lock's value */
-	*valptr = val;
-
 	/*
 	 * See if there are any LW_WAIT_UNTIL_FREE waiters that need to be woken
 	 * up. They are always in the front of the queue.
@@ -1873,17 +1889,17 @@ LWLockRelease(LWLock *lock)
  * LWLockReleaseClearVar - release a previously acquired lock, reset variable
  */
 void
-LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
-	LWLockWaitListLock(lock);
-
 	/*
-	 * Set the variable's value before releasing the lock, that prevents race
-	 * a race condition wherein a new locker acquires the lock, but hasn't yet
-	 * set the variables value.
+	 * Update the lock variable atomically first.
+	 *
+	 * NB: Note the use of pg_atomic_exchange_u64 as opposed to just
+	 * pg_atomic_write_u64 to update the value. Since pg_atomic_exchange_u64 is
+	 * a full barrier, we're guaranteed that the subsequent shared memory
+	 * reads/writes, if any, happen after we reset the lock variable.
 	 */
-	*valptr = val;
-	LWLockWaitListUnlock(lock);
+	(void) pg_atomic_exchange_u64(valptr, val);
 
 	LWLockRelease(lock);
 }
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index d2c7afb8f4..f19bc49193 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -128,14 +128,14 @@ extern bool LWLockAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockConditionalAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockAcquireOrWait(LWLock *lock, LWLockMode mode);
 extern void LWLockRelease(LWLock *lock);
-extern void LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern void LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 extern void LWLockReleaseAll(void);
 extern bool LWLockHeldByMe(LWLock *lock);
 extern bool LWLockAnyHeldByMe(LWLock *lock, int nlocks, size_t stride);
 extern bool LWLockHeldByMeInMode(LWLock *lock, LWLockMode mode);
 
-extern bool LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval);
-extern void LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern bool LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval, uint64 *newval);
+extern void LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 
 extern Size LWLockShmemSize(void);
 extern void CreateLWLocks(void);
-- 
2.34.1

#14Nathan Bossart
nathandbossart@gmail.com
In reply to: Bharath Rupireddy (#13)
Re: WAL Insertion Lock Improvements

On Thu, Feb 09, 2023 at 11:51:28AM +0530, Bharath Rupireddy wrote:

On Thu, Feb 9, 2023 at 3:36 AM Nathan Bossart <nathandbossart@gmail.com> wrote:

Overall, I think this patch is in reasonable shape.

Thanks for reviewing. Please see the attached v5 patch.

I'm marking this as ready-for-committer. I think a couple of the comments
could use some small adjustments, but that probably doesn't need to hold up
this patch.

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

#15adherent postgres
adherent_postgres@hotmail.com
In reply to: Nathan Bossart (#14)
回复: WAL Insertion Lock Improvements

Hi Andres Freund
This patch improves performance significantly,Commitfest 2023-03 is coming to an end,Is it not submitted yet since the patch still needs to be improved?

Best wish
________________________________
发件人: Nathan Bossart <nathandbossart@gmail.com>
发送时间: 2023年2月21日 13:49
收件人: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
抄送: Andres Freund <andres@anarazel.de>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
主题: Re: WAL Insertion Lock Improvements

On Thu, Feb 09, 2023 at 11:51:28AM +0530, Bharath Rupireddy wrote:

On Thu, Feb 9, 2023 at 3:36 AM Nathan Bossart <nathandbossart@gmail.com> wrote:

Overall, I think this patch is in reasonable shape.

Thanks for reviewing. Please see the attached v5 patch.

I'm marking this as ready-for-committer. I think a couple of the comments
could use some small adjustments, but that probably doesn't need to hold up
this patch.

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

#16Michael Paquier
michael@paquier.xyz
In reply to: Nathan Bossart (#14)
2 attachment(s)
Re: WAL Insertion Lock Improvements

On Mon, Feb 20, 2023 at 09:49:48PM -0800, Nathan Bossart wrote:

I'm marking this as ready-for-committer. I think a couple of the comments
could use some small adjustments, but that probably doesn't need to hold up
this patch.

Apologies. I was planning to have a thorough look at this patch but
life got in the way and I have not been able to study what's happening
on this thread this close to the feature freeze.

Anyway, I am attaching two modules I have written for the sake of this
thread while beginning my lookup of the patch:
- lwlock_test.tar.gz, validation module for LWLocks with variable
waits. This module can be loaded with shared_preload_libraries to
have two LWLocks and two variables in shmem, then have 2 backends play
ping-pong with each other's locks. An isolation test may be possible,
though I have not thought hard about it. Just use a SQL sequence like
that, for example, with N > 1 (see README):
Backend 1: SELECT lwlock_test_acquire();
Backend 2: SELECT lwlock_test_wait(N);
Backend 1: SELECT lwlock_test_update(N);
Backend 1: SELECT lwlock_test_release();
- custom_wal.tar.gz, thin wrapper for LogLogicalMessage() able to
generate N records of size M bytes in a single SQL call. This can be
used to generate records of various sizes for benchmarking, limiting
the overhead of individual calls to pg_logical_emit_message_bytea().
I have begun gathering numbers with WAL records of various size and
length, using pgbench like:
$ cat script.sql
\set record_size 1
\set record_number 5000
SELECT custom_wal(:record_size, :record_number);
$ pgbench -n -c 500 -t 100 -f script.sql
So this limits most the overhead of behind parsing, planning, and most
of the INSERT logic.

I have been trying to get some reproducible numbers, but I think that
I am going to need a bigger maching than what I have been using for
the last few days, up to 400 connections. It is worth noting that
00d1e02b may influence a bit the results, so we may want to have more
numbers with that in place particularly with INSERTs, and one of the
tests used upthread uses single row INSERTs.

Another question I had: would it be worth having some tests with
pg_wal/ mounted to a tmpfs so as I/O would not be a bottleneck? It
should be instructive to get more measurement with a fixed number of
transactions and a rather high amount of concurrent connections (1k at
least?), where the contention would be on the variable waits. My
first impression is that records should not be too small if you want
to see more the effects of this patch, either.

Looking at the patch.. LWLockConflictsWithVar() and
LWLockReleaseClearVar() are the trivial bits. These are OK.

+    * NB: Note the use of pg_atomic_exchange_u64 as opposed to just
+    * pg_atomic_write_u64 to update the value. Since pg_atomic_exchange_u64 is
+    * a full barrier, we're guaranteed that the subsequent shared memory
+    * reads/writes, if any, happen after we reset the lock variable.

This mentions that the subsequent read/write operations are safe, so
this refers to anything happening after the variable is reset. As
a full barrier, should be also mention that this is also ordered with
respect to anything that the caller did before clearing the variable?
From this perspective using pg_atomic_exchange_u64() makes sense to me
in LWLockReleaseClearVar().

+            * XXX: Use of a spinlock at the beginning of this function to read
+            * current insert position implies memory ordering. That means that
+            * the immediate loads and stores to shared memory (for instance,
+            * in LWLockUpdateVar called via LWLockWaitForVar) don't need an
+            * explicit memory barrier as far as the current usage is
+            * concerned. But that might not be safe in general.
             */
What's the part where this is not safe?  Based on what I see, this
code path is safe because of the previous spinlock.  This is the same
comment as at the beginning of LWLockConflictsWithVar().  Is that
something that we ought to document at the top of LWLockWaitForVar()
as well?  We have one caller of this function currently, but there may
be more in the future.
- * you're about to write out.
+ * you're about to write out. Using an atomic variable for insertingAt avoids
+ * taking any explicit lock for reads and writes.

Hmm. Not sure that we need to comment at all.

-LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
[...]
    Assert(pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE);

- /* Update the lock's value */
- *valptr = val;

The sensitive change is in LWLockUpdateVar(). I am not completely
sure to understand this removal, though. Does that influence the case
where there are waiters?

Another thing I was wondering about: how much does the fast-path used
in LWLockUpdateVar() influence the performance numbers? Am I right to
guess that it counts for most of the gain seen? Or could it be that
the removal of the spin lock in
LWLockConflictsWithVar()/LWLockWaitForVar() the point that has the
highest effect?
--
Michael

Attachments:

lwlock_test.tar.gzapplication/gzipDownload
custom_wal.tar.gzapplication/gzipDownload
#17Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Michael Paquier (#16)
2 attachment(s)
Re: WAL Insertion Lock Improvements

On Mon, Apr 10, 2023 at 9:38 AM Michael Paquier <michael@paquier.xyz> wrote:

I have been trying to get some reproducible numbers, but I think that
I am going to need a bigger maching than what I have been using for
the last few days, up to 400 connections. It is worth noting that
00d1e02b may influence a bit the results, so we may want to have more
numbers with that in place particularly with INSERTs, and one of the
tests used upthread uses single row INSERTs.

I ran performance tests on the patch with different use-cases. Clearly
the patch reduces burden on LWLock's waitlist lock (evident from perf
reports [1]test-case 1: -T5, WAL ~16 bytes HEAD: + 81.52% 0.03% postgres [.] __vstrfmon_l_internal + 81.52% 0.00% postgres [.] startup_hacks + 81.52% 0.00% postgres [.] PostmasterMain + 63.95% 1.01% postgres [.] LWLockWaitListLock + 61.93% 0.02% postgres [.] WaitXLogInsertionsToFinish + 61.89% 0.05% postgres [.] LWLockWaitForVar + 48.83% 48.33% postgres [.] pg_atomic_read_u32_impl + 48.78% 0.40% postgres [.] pg_atomic_read_u32 + 43.19% 0.12% postgres [.] LWLockConflictsWithVar + 19.81% 0.01% postgres [.] LWLockQueueSelf + 7.86% 2.46% postgres [.] perform_spin_delay + 6.14% 6.06% postgres [.] spin_delay + 5.82% 0.01% postgres [.] pg_atomic_fetch_or_u32 + 5.81% 5.76% postgres [.] pg_atomic_fetch_or_u32_impl + 4.00% 0.01% postgres [.] XLogInsert + 3.93% 0.03% postgres [.] XLogInsertRecord + 2.13% 0.02% postgres [.] LWLockRelease + 2.10% 0.03% postgres [.] LWLockAcquire + 1.92% 0.00% postgres [.] LWLockDequeueSelf + 1.87% 0.01% postgres [.] WALInsertLockAcquire + 1.68% 0.04% postgres [.] LWLockAcquireOrWait + 1.64% 0.01% postgres [.] pg_analyze_and_rewrite_fixedparams + 1.62% 0.00% postgres [.] WALInsertLockRelease + 1.62% 0.00% postgres [.] LWLockReleaseClearVar + 1.55% 0.01% postgres [.] parse_analyze_fixedparams + 1.51% 0.00% postgres [.] transformTopLevelStmt + 1.50% 0.00% postgres [.] transformOptionalSelectInto + 1.50% 0.01% postgres [.] transformStmt + 1.47% 0.02% postgres [.] transformSelectStmt + 1.29% 0.01% postgres [.] XactLogCommitRecord). However, to see visible impact in the output, the txns
must be generating small (between 16 bytes to 2 KB) amounts of WAL in
a highly concurrent manner, check the results below (FWIW, I've zipped
and attached perf images for better illustration along with test
setup).

When the txns are generating a small amount of WAL i.e. between 16
bytes to 2 KB in a highly concurrent manner, the benefit is clearly
visible in the TPS more than 2.3X improvement. When the txns are
generating more WAL i.e. more than 2 KB, the gain from reduced burden
on waitlist lock is offset by increase in the wait/release for WAL
insertion locks and no visible benefit is seen.

As the amount of WAL each txn generates increases, it looks like the
benefit gained from reduced burden on waitlist lock is offset by
increase in the wait for WAL insertion locks.

Note that I've used pg_logical_emit_message() for ease of
understanding about the txns generating various amounts of WAL, but
the pattern is the same if txns are generating various amounts of WAL
say with inserts.

test-case 1: -T5, WAL ~16 bytes
clients HEAD PATCHED
1 1437 1352
2 1376 1419
4 2919 2774
8 5875 6371
16 11148 12242
32 22108 23532
64 41414 46478
128 85304 85235
256 83771 152901
512 61970 141021
768 56514 118899
1024 51784 110960
2048 39141 84150
4096 16901 45759

test-case 1: -t1000, WAL ~16 bytes
clients HEAD PATCHED
1 1417 1333
2 1363 1791
4 2978 2970
8 5954 6198
16 11179 11164
32 23742 24043
64 45537 44103
128 84683 91762
256 80369 146293
512 61080 132079
768 57236 118046
1024 53497 114574
2048 46423 93588
4096 42067 85790

test-case 2: -T5, WAL ~256 bytes
clients HEAD PATCHED
1 1521 1386
2 1647 1637
4 3088 3270
8 6011 5631
16 12778 10317
32 24117 20006
64 43966 38199
128 72660 67936
256 93096 121261
512 57247 142418
768 53782 126218
1024 50279 109153
2048 35109 91602
4096 21184 39848

test-case 2: -t1000, WAL ~256 bytes
clients HEAD PATCHED
1 1265 1389
2 1522 1258
4 2802 2775
8 5875 5422
16 11664 10853
32 21961 22145
64 44304 40851
128 73278 80494
256 91172 122287
512 60966 136734
768 56590 125050
1024 52481 124341
2048 47878 104760
4096 42838 94121

test-case 3: -T5, WAL 512 bytes
clients HEAD PATCHED
1 1464 1284
2 1520 1381
4 2985 2877
8 6237 5261
16 11296 10621
32 22257 20789
64 40548 37243
128 66507 59891
256 92516 97506
512 56404 119716
768 51127 112482
1024 48463 103484
2048 38079 81424
4096 18977 40942

test-case 3: -t1000, WAL 512 bytes
clients HEAD PATCHED
1 1452 1434
2 1604 1649
4 3051 2971
8 5967 5650
16 10471 10702
32 20257 20899
64 39412 36750
128 62767 61110
256 81050 89768
512 56888 122786
768 51238 114444
1024 48972 106867
2048 43451 98847
4096 40018 111079

test-case 4: -T5, WAL 1024 bytes
clients HEAD PATCHED
1 1405 1395
2 1638 1607
4 3176 3207
8 6271 6024
16 11653 11103
32 20530 20260
64 34313 32367
128 55939 52079
256 74355 76420
512 56506 90983
768 50088 100410
1024 44589 99025
2048 39640 90931
4096 20942 36035

test-case 4: -t1000, WAL 1024 bytes
clients HEAD PATCHED
1 1330 1304
2 1615 1366
4 3117 2667
8 6179 5390
16 10524 10426
32 19819 18620
64 34844 29731
128 52180 48869
256 73284 71396
512 55714 96014
768 49336 108100
1024 46113 102789
2048 44627 104721
4096 44979 106189

test-case 5: -T5, WAL 2048 bytes
clients HEAD PATCHED
1 1407 1377
2 1518 1559
4 2589 2870
8 4883 5493
16 9075 9201
32 15957 16295
64 27471 25029
128 37493 38642
256 46369 45787
512 61755 62836
768 59144 68419
1024 52495 68933
2048 48608 72500
4096 26463 61252

test-case 5: -t1000, WAL 2048 bytes
clients HEAD PATCHED
1 1289 1366
2 1489 1628
4 2960 3036
8 5536 5965
16 9248 10399
32 15770 18140
64 27626 27800
128 36817 39483
256 48533 52105
512 64453 64007
768 59146 64160
1024 57637 61756
2048 59063 62109
4096 58268 61206

test-case 6: -T5, WAL 4096 bytes
clients HEAD PATCHED
1 1322 1325
2 1504 1551
4 2811 2880
8 5330 5159
16 8625 8315
32 12820 13534
64 19737 19965
128 26298 24633
256 34630 29939
512 34382 36669
768 33421 33316
1024 33525 32821
2048 37053 37752
4096 37334 39114

test-case 6: -t1000, WAL 4096 bytes
clients HEAD PATCHED
1 1212 1371
2 1383 1566
4 2858 2967
8 5092 5035
16 8233 8486
32 13353 13678
64 19052 20072
128 24803 24726
256 34065 33139
512 31590 32029
768 31432 31404
1024 31357 31366
2048 31465 31508
4096 32157 32180

test-case 7: -T5, WAL 8192 bytes
clients HEAD PATCHED
1 1287 1233
2 1552 1521
4 2658 2617
8 4680 4532
16 6732 7110
32 9649 9198
64 13276 12042
128 17100 17187
256 17408 17448
512 16595 16358
768 16599 16500
1024 16975 17300
2048 19073 19137
4096 21368 21735

test-case 7: -t1000, WAL 8192 bytes
clients HEAD PATCHED
1 1144 1190
2 1414 1395
4 2618 2438
8 4645 4485
16 6766 7001
32 9620 9804
64 12943 13023
128 15904 17148
256 16645 16035
512 15800 15796
768 15788 15810
1024 15814 15817
2048 17775 17771
4096 31715 31682

Looking at the patch.. LWLockConflictsWithVar() and
LWLockReleaseClearVar() are the trivial bits. These are OK.

Hm, the crux of the patch is avoiding LWLock's waitlist lock for
reading/writing the lock variable. Essentially, they are important
bits.

+    * NB: Note the use of pg_atomic_exchange_u64 as opposed to just
+    * pg_atomic_write_u64 to update the value. Since pg_atomic_exchange_u64 is
+    * a full barrier, we're guaranteed that the subsequent shared memory
+    * reads/writes, if any, happen after we reset the lock variable.

This mentions that the subsequent read/write operations are safe, so
this refers to anything happening after the variable is reset. As
a full barrier, should be also mention that this is also ordered with
respect to anything that the caller did before clearing the variable?
From this perspective using pg_atomic_exchange_u64() makes sense to me
in LWLockReleaseClearVar().

Wordsmithed that comment a bit.

+            * XXX: Use of a spinlock at the beginning of this function to read
+            * current insert position implies memory ordering. That means that
+            * the immediate loads and stores to shared memory (for instance,
+            * in LWLockUpdateVar called via LWLockWaitForVar) don't need an
+            * explicit memory barrier as far as the current usage is
+            * concerned. But that might not be safe in general.
*/
What's the part where this is not safe?  Based on what I see, this
code path is safe because of the previous spinlock.  This is the same
comment as at the beginning of LWLockConflictsWithVar().  Is that
something that we ought to document at the top of LWLockWaitForVar()
as well?  We have one caller of this function currently, but there may
be more in the future.

'But that might not be safe in general' applies only for
LWLockWaitForVar not for WaitXLogInsertionsToFinish for sure. My bad.

If there's another caller for LWLockWaitForVar without any spinlock,
that's when the LWLockWaitForVar needs to have an explicit memory
barrier.

Per a comment upthread
/messages/by-id/20221205183007.s72oygp63s43dqyz@awork3.anarazel.de,
I had a note in WaitXLogInsertionsToFinish before LWLockWaitForVar. I
now have modified that comment.

- * you're about to write out.
+ * you're about to write out. Using an atomic variable for insertingAt avoids
+ * taking any explicit lock for reads and writes.

Hmm. Not sure that we need to comment at all.

Removed. I was being verbose. One who understands pg_atomic_uint64 can
get to that point easily.

-LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
[...]
Assert(pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE);

- /* Update the lock's value */
- *valptr = val;

The sensitive change is in LWLockUpdateVar(). I am not completely
sure to understand this removal, though. Does that influence the case
where there are waiters?

I'll send about this in a follow-up email to not overload this
response with too much data.

Another thing I was wondering about: how much does the fast-path used
in LWLockUpdateVar() influence the performance numbers? Am I right to
guess that it counts for most of the gain seen?

I'll send about this in a follow-up email to not overload this
response with too much data.

Or could it be that
the removal of the spin lock in
LWLockConflictsWithVar()/LWLockWaitForVar() the point that has the
highest effect?

I'll send about this in a follow-up email to not overload this
response with too much data.

I've addressed the above review comments and attached the v6 patch.

[1]
test-case 1: -T5, WAL ~16 bytes HEAD:
+   81.52%     0.03%  postgres  [.] __vstrfmon_l_internal
+   81.52%     0.00%  postgres  [.] startup_hacks
+   81.52%     0.00%  postgres  [.] PostmasterMain
+   63.95%     1.01%  postgres  [.] LWLockWaitListLock
+   61.93%     0.02%  postgres  [.] WaitXLogInsertionsToFinish
+   61.89%     0.05%  postgres  [.] LWLockWaitForVar
+   48.83%    48.33%  postgres  [.] pg_atomic_read_u32_impl
+   48.78%     0.40%  postgres  [.] pg_atomic_read_u32
+   43.19%     0.12%  postgres  [.] LWLockConflictsWithVar
+   19.81%     0.01%  postgres  [.] LWLockQueueSelf
+    7.86%     2.46%  postgres  [.] perform_spin_delay
+    6.14%     6.06%  postgres  [.] spin_delay
+    5.82%     0.01%  postgres  [.] pg_atomic_fetch_or_u32
+    5.81%     5.76%  postgres  [.] pg_atomic_fetch_or_u32_impl
+    4.00%     0.01%  postgres  [.] XLogInsert
+    3.93%     0.03%  postgres  [.] XLogInsertRecord
+    2.13%     0.02%  postgres  [.] LWLockRelease
+    2.10%     0.03%  postgres  [.] LWLockAcquire
+    1.92%     0.00%  postgres  [.] LWLockDequeueSelf
+    1.87%     0.01%  postgres  [.] WALInsertLockAcquire
+    1.68%     0.04%  postgres  [.] LWLockAcquireOrWait
+    1.64%     0.01%  postgres  [.] pg_analyze_and_rewrite_fixedparams
+    1.62%     0.00%  postgres  [.] WALInsertLockRelease
+    1.62%     0.00%  postgres  [.] LWLockReleaseClearVar
+    1.55%     0.01%  postgres  [.] parse_analyze_fixedparams
+    1.51%     0.00%  postgres  [.] transformTopLevelStmt
+    1.50%     0.00%  postgres  [.] transformOptionalSelectInto
+    1.50%     0.01%  postgres  [.] transformStmt
+    1.47%     0.02%  postgres  [.] transformSelectStmt
+    1.29%     0.01%  postgres  [.] XactLogCommitRecord
test-case 1: -T5, WAL ~16 bytes PATCHED:
+   74.49%     0.04%  postgres  [.] __vstrfmon_l_internal
+   74.49%     0.00%  postgres  [.] startup_hacks
+   74.49%     0.00%  postgres  [.] PostmasterMain
+   51.60%     0.01%  postgres  [.] finish_xact_command
+   51.60%     0.02%  postgres  [.] CommitTransactionCommand
+   51.37%     0.03%  postgres  [.] CommitTransaction
+   49.43%     0.05%  postgres  [.] RecordTransactionCommit
+   46.55%     0.05%  postgres  [.] XLogFlush
+   46.37%     0.85%  postgres  [.] LWLockWaitListLock
+   43.79%     0.02%  postgres  [.] LWLockQueueSelf
+   38.87%     0.03%  postgres  [.] WaitXLogInsertionsToFinish
+   38.79%     0.11%  postgres  [.] LWLockWaitForVar
+   34.99%    34.49%  postgres  [.] pg_atomic_read_u32_impl
+   34.93%     0.35%  postgres  [.] pg_atomic_read_u32
+    6.99%     2.12%  postgres  [.] perform_spin_delay
+    6.64%     0.01%  postgres  [.] XLogInsert
+    6.54%     0.06%  postgres  [.] XLogInsertRecord
+    6.26%     0.08%  postgres  [.] LWLockAcquireOrWait
+    5.31%     5.22%  postgres  [.] spin_delay
+    4.23%     0.04%  postgres  [.] LWLockRelease
+    3.74%     0.01%  postgres  [.] pg_atomic_fetch_or_u32
+    3.73%     3.68%  postgres  [.] pg_atomic_fetch_or_u32_impl
+    3.33%     0.06%  postgres  [.] LWLockAcquire
+    2.97%     0.01%  postgres  [.] pg_plan_queries
+    2.95%     0.01%  postgres  [.] WALInsertLockAcquire
+    2.94%     0.02%  postgres  [.] planner
+    2.94%     0.01%  postgres  [.] pg_plan_query
+    2.92%     0.01%  postgres  [.] LWLockDequeueSelf
+    2.89%     0.04%  postgres  [.] standard_planner
+    2.81%     0.00%  postgres  [.] WALInsertLockRelease
+    2.80%     0.00%  postgres  [.] LWLockReleaseClearVar
+    2.38%     0.07%  postgres  [.] subquery_planner
+    2.35%     0.01%  postgres  [.] XactLogCommitRecord

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

TestArtifacts.zipapplication/zip; name=TestArtifacts.zipDownload
PK���VTestArtifacts/PK[��V1�p�BG2TestArtifacts/test-case 1 -t1000 WAL ~16 bytes.png�ZeT��)�\$�i������A���.�fAJi��s������<7��s�=g����7g��g&���8	��Hq�y����D��:
I�����n�L9�>#%a�t����!�����J��Y������	��p�t�TVq����,���C���,+|���(�����f22�g�'��J;�;�����5�&���������	��94�']B���k�B#��������/_�|�<^�.G�1}�\zO�����2�%b�A@~���&�L��zC�����H S��>���.<� �������|�����m<�[�J��(����.�Iu����Lx��G�E������i&����'��5���_��3���@������7�v��7�O��-�H��5sM9]����{�tWpKm�2_�=��T�&�h�{H������e�<���RX�Xc��42(ku��.�PF�*�@#�P��eP�����w�"6:-R}�Vr}!+�`�CC4����������A�mX�&��%w��P?p�w�BO�6TDDd<��/b?p�n����7���t���KiVV����n���B����o*r�/Ilb��*����u-$�������nW��g�+?�E�s�i���X��w��+�A?��aS�����2=aP�%+�������d��Yf[[R�4�V|d��VR�L��o����Kr��>������0�pE�-���M�	]~�h��]/�Blcd9��L0������0`��l�-p����|�b���	Q~��!"��At���/
[�L��:��|cl36\����	.}���$�y{K���T�	�Sd���5�(��.��` �Z~�/�z� �o������x���EEp�l��zc�5:�����h�I��xQ���%�	��?��hT��4.���D2e\�fU:���$�4���q��K�HR���*����� ���wWt>9�j(^�H
J���W�o�)�,Yx��O�}|�6����w������]�5��T3��qn���0�3D�����<~Y���6#���O���i���M���^Nn���'Z���z���[/(��W�������+�E��������g����L�fb�����i~���TO��d"pb�
���R�����~�U���ZO��t�(�J�I��������-m2��=\���
��n=��\��~�x�Ws!����N����<��@�~�frVN����O+����&�����z�r�F����W���l�Ac����Z��MJ������K����%u�d���������S��Qg��t"��ZDS-���w��+���w,���1���kN����b���������0��jW��R��5���/���?��D�c�Z����4eg��6���G5g��8CS�@m��O�UT�:�
*��Z����Dd9�)q\�B����dE��=�"e�����2�����x�g!�-�*����{EQ�1��e��s:�J�F�=A��r!J�KK
"3TkaS���l�#�E�q�������,���d�-SOX����UC-H`��U ��m.\}��a����%&����	83�e5&�4�=���x^����3!�����
��p�Mv���F�HgufFw����D����7F�L,A�����/����+�������W�1�*���<�d� ������
<j����t]�;�q�������KV+y���kv���^,a� D��c{�{0��-��? ��1-aptC'�j�#$�����)=SnV@�b7q����ee�7%��l>�Z�~��l������	i��No,���1�[���n��y����'�9��#��B#����TY%_I4;���+�D��;MQt������"�r2�|5�e���|dRw�w�����0����|���O{���'a-�]M���_!�������(�E�o#v�z���rm-,��H��z�+F�d>r��^��B����*S�04ym�u�B���>]]p����_>�������7J���G���$�d
%*���:����H�,�����������J��)�7'�1
AK��g�P3��o��vW���������G�a�U�VUi�;
���T����4�{p�#pf~L���)e�gi��%�����-h
���a-�Gy���c�*J"�D��E&�WPPb0+���������}������7)����)�����t<��	7�	�4�s��������H����!kN�Zf�{�A��'��j����������������!r�vQ�-F
2YS�E��)�4�������7/]n���*������%z_��;��W{V��7��vP%%�Z�V&��2|�
G-����m��6	�j��'��� �v���/r�����u����`Lq�u�+d'>�{��N������]f�B�c�t��O�\�.��Jwy�9AGuf�s��J�y5g��U��2h����C������r�;�.UA*�L�Zu{�~P���F-p(�\���,X}��7G�E�{C@@��`1P�g����d��5�4��a����`Ul�N?
Lj����j5�TX�#����������|u/�9�)�*5�� X}�d�W]����sn$%������*)���(����
R�0��"�vE����d���@����9��M�������0���b+�E$�!73�����w#�Vd�&���B�����}x����tf�y��V�����l���c1�'[����6<#HSj:2P����������_���^�|?9_s��(]����.K=�����g28:Gv�wk�=
Ae��_�U,��qu�os*�@���w=��=B�"
f���W%?�ex>�B��[6����)��r���B�aN��*�L<X��+d�:BY�	�\gI�kF��g�"�aif�0��TEP�|}EP��#$�M��~1�$<7�B�7a)B����KO����.q���7���|u.�g��������9�OyF-��n,��v"�L��Dn��[�7����Z�N(��@
Fd'r���^��7���Y�
5Jax�0M}�,�X	}��q6C�h��~��YiV�#���5~����+X�$��M�~���1&��k
���,����ek��_�J6�1:������1�&c��a��g��������4�<��N������~�B�~f;��'�]�D���tPb#i���g������\�������,�Hn��W[`���9�`�]�������"�l�36iM�d�N���uOd2�|B��xQ�S�Z����Ia����m�[WZak~��d���w�!L&<����j�,�@����z��������N�s�\?8�m�	�=0L����j9�7$�Z�=���vR	�����+~���,^����%���lp����\2���K�^�*����xUR�2������z��������xU�����o}��9{l;���>�-�m�� ���?ZU�^1|���SJ8�]k�l{�r6�)�*��
��(����Hh�;��(�P�����<����Vpfr�V�yUj<i1�5���ks�c�g�X0����+���3X����x2�Wa���s���y�v��	�?�,'������k�<A%�N���bV'�:�5�$dj��Vu��L=q���>_����>���W;��|�\^n���Hs4e�0;S4
c������b{�q����B"��������eG_[g���
���Q$n
�{�R�B1n���g�����dPj�S�ZP�~�q��'@����u���}3�)S�n��OE��:��=��m�iR�H!��1��vi��zr�w���
)'������/qGa3H �$��
.��_��	N�L?��(>�\0<�����X��9��d|k�A�Id�1|�����S�L�>i������_��S�L��J����s��i����<a��J�8��K����R��� t��U9�X
�vR�D���K���Ryl��}d2���Z�b7��[W���{�.���:��|g5sE����3���e�g��ZJ3���������U;&K�0W����/��'G�Z�
Ri�W/���FfP����c���Y5\�Zqe���t�_���6�}?\C�}@�>pUQ����g&��6�� ;�,��h��/�+�Y�;>.��E������Y_
�g�4#xJN���%�^������f�*�<lv�5��(�><k�E�#EU����!Oq3K��G���!���I���+_0�Q�S	U{%��8�rr����4���NE�$ .i;���Z�Tm2�X�p��;�##u��.\H�
���U),_�:�W�[M�
Yn���)��@�����%Rhm�f5�W'b��v1D�8��n��xv�.��o� ��Y��T~��`��D1�����(�3�c���t|y>�b��U��d���=�B�@{�^Q�B81�a������$��(��/�y�]^o_Y.�[�2�,�h���`��&#�6������sn�kOZ���H~���k����A�B����g���p�Y���C��t�1�&FL)1Y;I_���>v�����_$t�~�^Y�Z�j���L��r^������@�4������M�2S�fZ����w���r$�����l����������g��/��y��f���s!�����X�f������������r�H��U����xN%�U"0�(z�*-(F}t��Ef���	����mEw�$����T��D��6F�a����u�����B2j��dL��F�X����V"��x��&
�,�����_8X!�h�y��#c��K���nf;%�.�Ki���+�������3Ze�^�D;-�����c�0�E�{=�>y�fg��6�1�j#�������4C���j�:��������E���u���u>r�m���}�ku��4�}�m�9U6Z^�TG����.��0EBM�"�Q���n$T�!R���M���w�������I�U�����e�r�D�K�E�������"�����Q�1huUEB�$�~s-C�?����6�f��{?��L�O_��~��2BB�����"�c�	�[y����
����_<]3�O���
�.�p\`A/�;�)��fe�cj���u����0#�x}�#�
���h�g�b�"J�gf|�-�:�v�+'J�O�!�FZ����zm��`�������&��p�e:�c���pv�����\���M�����������l[�M[%}p����s�r3B3P���?$�����hi�9�f=�����r�GU��5�
��
:(����N�\��e�'R��I���J������f��z�A\����
�Xo6#^x\t�{~;��\�c2/�q |-�-(�X7}���;�No�������x�Fs���!M.��D�o�Qu���]��Xvo&�_}R��)�
 -��E6�F���0�����J;��U���C�����R~���������������4&fdv�rH�����L������B4���W����-���Pb��I�R���
��C� 5���NiM���{#/>�1���,��2�^�JO�������W�z��Zu����t��A��bG����lv�X����~SB�4�{��������H�����V����$c�4}�&�����f������p��#w(��%�>���A��EvPu���]rB} �9� �E<���j�
[��\���d��c������ ��*U��Q��&��.5Vr�4����JM�*z[���(� ��MCo:��m�����=�����%��� ���w O0~��-,��'Kq�L5e�n�f�u�.�)As�{����lg��7�����h?j�5g����X�����-�����F;I�)�=�$��K�&	x�d���}�Y�Q�z�D,��M$�|,��s#.�_%5���,*
��h
�U�IH�f>!������-�ME5�����K~�=��#������B�5��.&�_�Xp_ng�
/��-G��C���
��iUdGdo�96�i�C�F���.�����j�0'��R�;K��T�Y�f�-����)��d�<�mV�4@�8}�he��;3~"6=�q��9�f���{�-Z����
����.��dvp��P�@l��)�6��faQ���ab����rg8x�g�6d���������*5E7�����{mvI���HU�:�9/�B��;cC�;��+��QN��v���N���7��J�M��O�M�����x��F2�E���"?�}S��}q������g�uy�z��+��vc@A���V�2a���K|J�z��g�F�u�/19�2���n���gbm,�gJ�k�]i��@g��k��}�������7Tz��e�p�����!�s^eU�p%�������h;(��m�!o����L����>	����B�KH��=�-�Z=a/�:�|?w�B�)�W{��Bm�_1=6�G�v�R�-|�z#|�v�<�4=P�zF�����jn����E�$�?�_���X�KYZ�� ��;�������������������R|]�5i���M�<;��wz��z�w{��f{3'��)�B�m�"�{��^��O�c����]B������eS�f�O�d
s8U�2y�����I�3h$P���_�-�=f6K�I2����0>H!���/&g�����.w7-2{��^��*H8�Z�I(<����y��yyHze�|�p�����j�0�kB���B�	�����"��&�KG�$�Oj�0u�N����uLX�d(��^��b��3����2N�t�^��E�fI�+�E��h�a��U������������#sY�j�����UEC���U�mmw_��������1����=�Wf0G�}����E���3�xL2!�ic������8���7�h�y�"�\^�UU��[��G6^h������(xNe��}���%�O4�8R���mx�\��:�w�d�y���
=�j����9����6��O	���AO�
 �����ug9�,B>@���w8��ov��� b�.�uV��VU%�-��"���<cP'�%<%L`��gx�j��$����J��v�� �P���_���oq��Ya">�v�pMR�at>����(��S���C�W6j�Si�`W(�d(��'�4�W]�4�a�~���T��L��d���z����Z��T��V�9�{����2�y�7d����]��bq\����%hK�R������>v�(�81�d�����ct4�%��;
���u�����>�=C]iFsV�������[�����/S���sb��L���X��q���Y�����?���W.s�4�_�3�K��8d������k1�\��������=p����������c��v��9�bXh���K$Z�M��c{�����P'�.�jH��[���%!���3�H�MKp��u������Fz\��	�jZ>�����	�M)������C��~�6�]Q���\�����N����G6��XB������E)�zB2�������?F�/�m?K�3��h	������$��
7��.����d��+�Nx�{}]�������������v^�T(*��Z[��T�,&�t�����W
��c���.�@09��T�mK�x������`v�:��vs�wEs�N�k��G��J��W���e�{}q��u�}�>m��J���q�������R?�������pr>HE�����Z�Gqt ������d8������v�V��3�������
�@��v�AG�6�_3��J�w�A=bP� ��@�k90��N�Q�)���_��XlV��O�2U�N%���y���j�q��_��W�z��NR3_���s�����*����%�v��}������DF�J���� �����_���[����0�A�5�P�����w�6��./��*Brc��ua���~���qVd�������v������ms:���l�F���)���nq��e����{���s�����}.�@�D�����H�6�A��l�����sy8��kN��1	��{58k��&7��Ibl�Y�F3��!_�?W��6e}�d�3��g�S������b�
Q��D�~����?H C����;E4�P������Z����~SY��W=�2]���6�����K?GADr������9Kn}�b� 1�����^g�F����c���X�S��;VIv����]"�8�:5�bHvv|���E;X��� ��]#���X{`d#X�@�g����X`:O�n�fy��U����W��K���$�''�G�����c�����g��E/��e������n��^]9���|u������b:VF���F\H�0�qQ��Ih��8���W�����I��x��c�*D�U��2���c����(��:�KQ�X���~'3�yN��o����"��]S������e��p�Y�x�e��X�����N o:��;��0 ��!l�q��]<���'��n((�@��wi�s�DXM���}�w��d6�j�����/�-gNo3O��v~��v�/%�>�
����>�Z��%��Q.��^O1����1!�ar�2^Y�^�\��,����OF��%�W��\���{���I����e/5
)�<�H���$U�=��,]A{�92��F����c{����K������+W��Y)���l�b�a��$�J7_,#�Q��{��[�k�
s	y�0�������
��*��Lx�6��y�����S���:y:=��rYA��3%��K(<��,��������P��Q�)���}�a��:�
����_#�pi�h�WO+]D�j ws&��0��9��*#��XQH�������Pl���w���b�w����|�T���R5Bm��WKkd0=5�j�A��"9�����Q�u��pk�b�__����s}���h����C{�/��gp��W�b�"�`2�13�O3�
��8i��nf��w���-�=;tW�5O�9

7���*�!R]?@O�m�j�}s��Q�7�Q���5��>6\lV-�+���+���R�rfq_'������K�~
>�n{��p��o����/�R'g��w�����\�F�9�6sZ*
�M6/�d�s&>���zNB����O�2|3��x�cf?���3F�t����H846Y�P�nM���Wl�Y�K]A���lStL����\��s'�U��!��K����*����������E~�v��	�C�r����5���?m�������rQ8�,�-A�ST��rc�v]���P�-E��]���n�d��uNU+��&���#�p�	Oul����.�?��(�I�}�=��5�,���B3���:�b�^�{��'O����I?����
�_.����u��O�c����|h�PG*_gl����V��h2��3f�S�h��xt�zC��%���8ZW?���m"����/\���5���x8���-�	�&��<]HMk�G���Sb���$���'s���-�72$p�sa(a#�L"%
�~R��ZO�gF�t�k>�}d�Q*6%�e��K!���&[�
gRy����N�����3o��B�����M72�&�<c#�1hW-`��;s�?-;S�2�!�d�9v��f�����*�aX��/�����>����9���uO����^'��k���g�]�BZ������n����`������5����d�r>�_��1�=p�Q�T9�:���*U���z�e��<962)�}�}��	}��S�`����.���*����>�q�F=x���:���v��$�Ix�C@g�-����8��:�{������t�wp�M=W�
�0@����tqza�sX�G^d������c�3�]���H���r���U8>��tnG����7B�f��t����z6��RkY8�Q������$��"=g4�TJ9�4@Vk*�RAD�M���y��t!Ku����d����l�4�����`���[�].���t�7�|����{�z!�4k~W��cH�A��t'Jz��o������O��wf�k$B�X��U[�)$�9CO��c����[��i���g��[o��}����v�����Q�BI��-�B�Rqh��+"hl����
�cl��k#������#��
�{�3I�����������E�P���c�����p��)�{G��R��]�U~�B���#,����i]}=��
�T�>�����z��c)3
��*xi�����1&=����uVe�{1y�;rP�r��A=i-���Z��Z�:?��0!|��h��MUk{���(����������'�����O���kDO[[-n���GgG�t+�
����<�����;/�	�u��Vd����1����_���y	�c]?)^=�9h"������"�+������������U�z��� ���=�I3��"S�,�xNJ����'�j�V�~�������A1�KpIL�����E���[��w��cw4HvQ��u�����R�'����/q������,ON�OM	�����$���9	EOP���A/�
��~����P�_?�^�@<h2&��%k<���iA���(%���6������q�}��\��q&8�������(���z
_g��9<-�;��
=�h\F�!�����|Oj�����7����g#�a������j��������N�	�����..R���4TTr�W]q�?/��yG�n)��w5[�;��|�N��������o&>�4�*����I����j�2�������\���V���	�����"9} �CiO��\9��������|3�R��.�|E�V����oI�3�wz��q���T!S�����8R��1v~L��Vg�@�����b�w������T��u��tj����$�����Oa��n&C1^����9.����j.��9N�x���;>"��f���I���B����#��~,����AA3���H0���@?�^����R��\4��R���i������\���5�3������Q��&����f�9�9oMs��V����]�z#h���de�~��:�x��.�7%�_['u;0�� yW����Q����iL��A���<�k��G��pv�^�q	��]@���{�`�������t��^P��s��<���I��m���1���6*6m26t��wI�����r���'l}����;�L�j��nWN��1��S66I7�W��F�#Z
m��yv��O���?r�l�aV��k�����#��1�V�������con���3��{�
��PMy	t<Q��n������&�v�
��+K@��x�,�qK0pxKi6��c1����f��\q��o�����S����Mzmb����5!s�����8��z0��w�������x<����DoU���S�V��W�	s�����<X(�6	^R�3>y!���0�fsS0�������d��.
���]��nm\��)�S��W{��r��)����XOM�n�M�q����0�XBA��h�^i�[�.�n�W�F�Td���G
����J�������I�UbX[�Gr3	;��:�{(��NF��>��f�/�_�_����&�(,,MA!�*%B�����P�P�t)K
���.�	�+U	�P����(U�*6n9g�����_���3�����{����O*�D"o�n4���;��rZ�������;�UX�
8@����+rZD�0.���%:���(�5����� <�n���Q��h)>Bg��=|�ql��"���/����*�e~���p��A��TuS���JS%0I�
������)~��Q�^sn3���0�K��G��{�!i�h��G%� v�D����*(�f��?/Jw���b>�Tu�L�z�O6����������(Y���l�}��5�`�$m�W�
e�[u]F�������G)p�
���V8mS���8�\�}�#�n�P7�m��I7��}7�ZGHs�0�����h�tX�M���x����s�������%��_��&�J��<���
@�(����+mTg_��g��[��l��7�i�[�dzI:��d��;��<����!�?��bIqVS�v��Y�B�mIme��XoBB�Ql��}��Q��=��aGUK��Z���I�[
+�q\��<��_�>�%�k$yYDN=[�~��������*���i��VY��y�����f�t���`)��fy�R�����6�9�/��r*��u({�J������f�8�:hw��b�L������\��o8B�8��p?xO�K�[�����>}��j-���]b�E�����T�}��$U��X��&��Fvy��,����Z��fU�k�_�r�7d8K}���;z*|�*9�W���[���m=P�><�xZX��u�L�p���z���$P�1U���j���3�Z��v�?��v&Y1W��a��H��h�h�9�R:
�����i3H�L-H?E�3.�&S�h���P�r'X�s�F�Wv��?0#Pn�����3z_M�J���^����:V������e7�z>�����~q�L�}7��7���y�>�����������L�9T���g,#/�yu��y���d�Q��LD�����x=�u����m������������~1�������ZZ�|�+�M�,�5��o9���x|�w�T5������g�Uy����dTD�\t,_��$$�4���[����!4W=7
����]\��{%M4�Ch.���e*�������D�f��\lU|KQs��4_���pnz�I�k� �hh�P�%��9��u^:9�����)O��r����<�����s�*T=�;Hg6��y6Kgr��d50u����m�R��Mbm��1������1���G�Wj����E�.�^���TmFU�U�q������o��q�.�m�;'f�KV~d����bU%a��Xr��=Iq�	�$���`�jU���r?��i\�������7XC���KGf�#,2o���-����q�Ig�Q�Ds�,�"����NS��w��	�Vo7�#� ��X[��k��h���/�M�K���Ge�bjp�� 523��bj
s����g��o��qL��j�B��f����q���]]����m�A+�Kx3@������e�zB���V������^H6�� o���|d��bjH���f9 ����m�D)��'\W�(��<W�v\���Vn2��K�p�3����YX�Y)l�s����l'>�p�n[����r}��#��j���f���Fl��V��xK.� ���i�_T��u�����D��������'C��M?G5���[q�X���|�gI
r�<S���'K�+����������r�����/����������\��zZ�@3��:��"8f�@����k�:�0����f����	W��4cH/�ee�{����9�;31�����4`�k�8i�/��R����3�7��i�Q
����*��r�n�DOt�i�K����C>��+"!�Z>VkS�����U���pt4$1n�>�����\cg�������M��6'�	��zM���;�	�O�(�*�Y�O$�B>xY<�����
��d��k�X��d����w�:s9l���`0g��".bO[���^~����	���R�z�<�,EQ����Z�
'*-����E���8P"�+��~
^Y���������������pKQ;�$�����p����+��2��i���
�}�a��g�13�v��l_����c��hk[���@���qu�'�**�m��_�*=_�
�[/3|�8�������D�>�KI�6���(K��R�G���KI�9�	sP*�3�t�(�g��K����9���
��h`�0I�P��-E2�B�F?4���%�������{�����j�HA� {**�t�_�j�aDDv����=��+�B�?�V�O���A��t�X�Au���G��F@e�[�oFz�T`@���`����G��i��!��=��)��R�1��e�d������l���t�z�N�)��W��X�C2�_#�8&&������DMD��$���8�tq�\�D=cw��X��L��w$��[L����R{8�GX���hf��!���AA�=�e{���xz��C����^�D�gviP!�t��Kr@��Z�Dh�_V�UBL�}��s�D���A��W�`��C�?,���U:�����m��|
@yS������i�B��4���h�b��q�kC��7cui�y����#�� NA��Fx
���b�6����_+�NM@��U&8�C�����y�F�K���8&�]�bU����FC�)�Q�����W+���o���U��@9��n�If�|�S��@ ��R�
���>R�rJK���h
,����i�W7�U^nY��Bk�WYd���DHg����.�P3>9��2���'�"���[U&P�h-�c�p_��g��Q�ud&2��p�Y92���������T�7���<��`-����l�����������+C�6A��^�&���f��z0��$w�=���E����Yk�8uO��g���OP��Q�t���H�4�I�nb/��w]�G�QTlM���B����b�����sM��q���M�$C=j&�)tl��9���Cu~ D�G:�	<���g*]�|���2���;	����K������q�l	�;�	5}I{�i���	�={F}{>����T9zu�1mI][E~?N�����������J�����@'0���}I-�	��n}i��]ZC���J��������K�#c�_�:��$8���r4#�t�vs��_��������n�Kfe�.��Xb;&�m`��8�����fh�D�5��� 7j�X�L����.#��������P���CL��j'd�l������6���J{�Gd��Cg��0����,�����(�%V�*�NC�!���y��>P^?A|�'����������7�QU��d2`�O�IL��>D����K��}w.��-<��w�N	�J��u��/��D���ws�\2��L��Mwn�r.4�����T���{��V��]H-������}���X��s^�8�)M�"�0w�7}����!��\��=�'m7!l3�L���+07i��F��z�����#������nIF�0���p�}�����b`�\��=���9�C?�2;�}������m���6+<�7��j�~V*�<7-���*�s����pK�n�Y��2C����^��Y�n
����0b��{e�m9��J��
8�>���9���z��8�^C =������=)?x�o���,�����}JU��jB7p�5&���o�q<��0���.&��q��ka���S���RM:q+��kSq�+����;#���p�[D
&���%H��5_���{��M5
N�Z����d�����jw3�l��%��9�)7M04�1Jz��W�&�������~p��4���Ar2kZzo�\_^v����1�"����B�C����S�s�4b���9�����Ha��}l����FT&`��i&�{�����J�M�������MM������[�`,��"���{�#z��Tv��v����G@�`��ZO����{I�9�z��P3*�;����T�n����~Sb���92�C�H��
��E�ki�Q�S�e��Q�/}$.���*f�/���W�2���E���p2��,,!O�Z�����,�JM�9����3]��9rL�B��u����mG��0t�w)_�"��M;�)�N9���#�-��l�toI�����"j�����[�F���)����z��IARkpY������n3��l�z��������)m����%K��e�H���9�,"!�������##n5��Z�F�/Va�aY��QS9����h�����;�����7B1i*�����S��
9��E���4q~�n���eU�NL�t��������y��w�%�T�K��gG$Nq �����7��~k���4P��m�L5g������)y�|+E=�2�	W#�o��m�e�	�f�S���4N��U�!��MS
�`�k�;����|�&?��o��+]R
qE�����a!*t�I��'`4&�������4���uo�6�����t���A�]aC��� ��'r��t����o�GjG4�FycP������R���b�o������O����iW�o)�:�k.�86/:�D����M��J�hAd�t��r�XS�>�J�s*��'��q�����*��u����k�b���]���c��rKN�n;!8��|n�g�t�5�w�����2c��{�l?6�e6(L�5��0���K��1XKQ�T��v^���+��e�������E�>G�"�{Oy��s ���
��~����M#�����N����
�uL$���.����\���	��z�U�Y�PQ�q*.6��$15h �W������|��������Il��[�Lv�W�9����vAv���9�M���+��ZAu�C�<%���������[�sr�'���hq1��~�a�(~r/��9s.����-���Id�������6	�H��)�����^�����|�n����l�����C�5L��i�����)�h��L��d*7@���nro8>�>e;�����)������j"���^��D��!,���2�p�<��PND.[^B*��J�h��G@�c�:9��0B*e<=��h,�O@�=]�%"�E�����w�D|�|Bh��\c"#����)|�y�%yBsPJL����4!����m�(B^K���'.:���p����R����V�����Z�n��PKB��V�?�G�K/TestArtifacts/test-case 1 -T5 WAL ~16 bytes.png�Z�[T����VP����[@��.��!$�C������A@bh��a��������>����}�>{��z�}�,��E������NN��2�G���D.�e5�;iIuw��tlw
����t��%����9���c�T{���t��#[���)���~|�QH�:.
e�iH��!���cB���'!�k&���`��V�HJL4�����Ni�X�������Qe�~������_
5�����������#!����03��	S�3V!�Leh(�H.	�;��>H?�F�D����O�&��x=C�7i@����x=A�n�Xr��[�W�N����@��<A���G��`��b��������O�O���G�Kx�����}���#��}��	?�@l4��'��B�[�4@������s!.�R#V���[���u� ������'f������3���Tm���J3~�P���}�b��fc�]`_�<�S�6����Cio,	������Ki��|��J<E|�>
�s�d�����f+W�^�_8����������L�������o��2M��{�0���c$����(��{�>�&��R�����o������t ��lc���3���m���[2{h�9����EJ\���w	�J���G�n���a��>��j�����e1�p���Ks_���������Q;����X�z�'
�f�H��0�Y5����9]\��r�F��^)Xp&;E�NPhl�����]I�H�d%5>yJ���Z��B!$�1-��uU�m������s{
o���@���'����{B4��5b���e����F���X��'2	GVg18��
������4��� �z� ch����rob
�~:��j��TpE���,@��
R�N����a�V�	L�K�T������B�P��u����i��t�(�dSz�_���a���Y/��������6J�Km1�&g�H��5�9k�;�B7��)���/xM�8�1�[u:&lSVq�������B��Fh��j�����`�:�pK���)�%�o�9�i�{t�(�AFH�+C!�������2���i�q�8;�0�gH~r��!5F���s"D���W3$�X9@	@j������%�0���(��?�����J�������[�I��W|�������/������m��n��������?_^+X�o3qs�d���l���XpH������@���E��/��OHM@]k������h���L�N�T�9�������J���U���]x�,7���D�w(�7B��������d�V����-��L��ah����l<����������"l�N�����m:���A����t��g��h�de29x�\����6��Q�
u�X&�]fS����M�Kb�BC1����9p��Ty.���%�n�KK#������9u�	�@)a��!G����m^�%\���V��T�_������8V�=k[�e�/�"XNl{���g��T�P��>c����D���I$���^Wh��\p��x�QCD ��)rv��iu�`M��.d��!���������J���P �'y	0q<+�evCj�h��^n�#)�-�x���%lH�t�bx/���>�yQ�*����,��^4�-�D_�x��}�f���A
Wh{e�����	�0�E�lBW��JN��v�<����NxF����V*�V�9.���{����{�C2��XXB�-�D�<�}1B�����:F��X&�;�1�~|h����
��>��La����J�����c��z����=>\�aLI�J�I��E��K����E2����\c�C��Q�V��(�+ �%������.O�~F���(�p�SC>L�0��y��_���t����iPX�
��&����7	��`�K��A���h�5A�q��K&�G8���l�3}��T��`���-(�^ ���CW�T������zBG8����6\�z�>���7-%Z�. �{
���$� 3:�����q��)��W����� ��S'�q��Cf����w�^�qs&3?�!��r�+���5����]I`<u�9����,��L�����>��E
��MP��q�j��q������3������vtOt<����([k��/9�����<Mg1��Y���
f[;��1��+��t2-����5�Dk-�C3R�����K`1`�����V���wP }����q��)M����TJ���\�;,g�������JN�t���,��7��`��TL���U��t���@Q��6*#I)�:�vM��`��9�y��^���e��')%�9�WZipN���_���1�R�j��tL��VgYmOJT��+����)C$uC���p�\'����Kz&�av�nsD��I9����l2���yu"��TD���E."#
�fJ�!���N���"�lkd�4�=\�P(���D�������W�xnrR�>f��9(�;��]%�XP��19/�P�q����^3B��|�G�~�q��*~����q����jW�@_V������.��?��A���we�k��d-��N$^�hKx�&`�Wa~:������������
a�'�-*���O����q�{�BUoU:M����H��:H����������=��K���7�a�rs�����c~F�R�st��W�V��Zya��	E������^V��O�\-�!k���)gZ��������P�-�,k,[^^7	�1c&B_�fWo��*�@n��<��#�^�~r>)#$&�kZ`��rGN���*f�voH�"x�������{2d������-��h�'6����~�b��������M'��
~�y���r�*&��kn�����bp�w(`�v!%'��"�B�����7}
��G�|;G�����;���w�aM -_\�4�a#A�9�a�.�o���Od+��z����qYW~��1��WQ��S\��*|��Y��%Pg���=q����*���L&�f��7�����P{�0D>Y0�F��
�T�}� �����"�-�>oY-���JX>Y��u���8$T���:)����qw���fv�h
����S���|���������T��caT�E%�w��>��J?`�������\�q�z���.��zK�c�����s�
H#>~�����L����.����'"�Le�`�h�p�e���f���d)Q(���>���?�Ly(�?�Ef`�d���V��������E��	��D� ������	���D'J��RF��^sS��\I���c��F�_L�)������6�L����j9��r�������]N�G��,���["3�+M���0#��|�G�X��0+�dce����7��6 ����#b��}J�$H��;����1k���Kk���5����$������
��z��q�O�m���5L2�����S��K�u��{�����+�H�����9{�Fb���Y�&��W�������&����x�����:6�*�>{)����1���e�3V�������i	�����|W�G�;[�xh�9�  :��q��s�k���0w�c����(|jnm��Yls"Xj��,m���{��OW_g�u�j���3���a ��g##����VQuO�5|��t��v�t���C�6-N�`WF�6��bew��4���!,�[$��i ���l�b�k��=)��6��'�C`*�c�UT���?����ae�;T�w��V.�)v��H9>3<���Y���kLbJ���I�Hw�j#��1��z���]Pk�'.bJ�2�w��_"3�Wt������-<��q��o��bf�����HZ>[z��Y6�KHtl��Y.�-��&����#7��@:r��c�����������_���k�&o;�j�+�/^;�W�?k��$�������@O��B3��D��Z�Mxh�`J�I?�Bt����/���d��iyf`���Dh���m�H#"0�VF��v����r8,�a[�����a>A�P�1ms4K������{���a�j��f��������[����)�<��@l�V��O� ��T��cQzED����w��TIOM
gW�'j�1
�$P2��SU���.:x]������^����bx*
�sf�mV���u�b��������,\�����z�N��
�Y�nkB����B 3!{���~�0{{����s_�lI��X���������O�1|�"���M�W�����)���S��9t�~����m�Q��a��R�ca��pF������fz�H��	Q$��x���O���?����6�����#��aa�M^KaT*�4�_C3z%�0���5"v����#��]L1){e�������^5O[u4}D�_�$���z��:�M.[&���_5��z���Bm�	���)v`��"`����[�8��q!������J�4��a���_�����=���9:�`���tB��b�[�f{���#�J[�?��UQ��l���T�W)+ u�����j��(�BXH�K��p�mf%��=����0�H~�h$�#�k�����t���OP_���Q��:5D8����s}��&����j�*/e�Tk��N�BH ����6�B
�����+�}o�$�v���7��HK���Y�|�$��s�@iE6��:��W��&��ve8��Szv�4�!''�"�� �p�~C������^�
�U���
�I�A�*�����v��������
�P�q�$t�����l�s�)]��c'mYb��	�m���fNS���r��Z�\��2�}��\���&�S��@8?���s�[��U�&'2���!>8;)��D��T\�-��;���K��jI9���v��������8A��7d�e��R>�f�x
�j��K38�����)�����$�
^��Q���J�I��n�Is?�ur��"�M��u�	��\��z����ag����2�����_��A�r*Z��.8����XL[G
[�q&o�������l���$O��V�����R
�h����S���������G[�1�����k���pD�d�z�(A���6�8eH
�4�p����p2{�wus���Gr	 <��������d���������-�:��F��f<�A�z�"[John������Lr��y�Z��f���m�a������i�sY�5C�9?%�[�1^�/�T"[�2�{��{vq�w�^Fn�R���Kr��#�&�	��W�������:y�G�����m���ZM��g`u��e�Dx�����n[���-����>"Hn����������g���R�����6�<��Z�����B�y����+Z��Pe�W�����#���79Am�kNsq��������T�G�����&��~�����EI~:"8�Cl
��
�A��HB
I���SH�UR��J��2�����gJ%A}���b��Ii)�5`+{�w�L� ?}��BR[���0���<���/�U�$��kl3?H+�Q�7��y7������Y���d���'�^��8�:zb$�|@L�L�a�����b�N����r���?\R�����S��I���G*`�'O9��#r2�jU��oS�D'�BQ���L��M���E��=�<��yN/[yv!| �Q��<��l��^�U�+Mv��e�*_+�-]�zu ��MTZ���>�i���=I������csR�
�(�/%�����!���3{�8N~���Q1�����G�Yl��^c�����&{��nT��D�$+7d�z()!O�GH��.1����W-+�����^����jer���y1�j��e g����n^��&"8+$�c�z)~�+����	1��5A�-����k����~��J�`��<�}K^���a��wbaZ&FX��2,�>AOq5L�������w��"k;��k3U�!�thq�`r��?'(@�j��f~����v��&bcWv�����;�_;>��t��_������~�h��Y�|O	]��N�a�B�x�^Z��zZ����r!�q��#.�����"L�����,�����f6���I����o%zz/O����y~����@��rCO��!Hm����VQ�u��,�x��O�.���-��S��DI&�t�����6'Z_[OJ[���������$��������Aqg�k�}[���d\j3�6�j�y�����$����^��u�>�'
OW���K@oR!����>���@�_�1��N�\2���}(N�w=�5\�y�r:��;���<g����e ���cE
S�~�,/)U��];�MK�J��c2������������I3�xc�.s�s{t��_��R�eu�#������������]~�NLl�j�`��AS�$E��:���Q�������B�O9����y~��2a2��}�QYn��!		h;d�~O���S�e�5�&u_?��
�����x���X�g�Q`v���"{2���G�B�9z�>"/<�c[��J��-�N�\e1��d�����$�m�9��&���q������ab���-���b��CW��N��<����XK�z��V��=��u����~C�cT7}�U`=��9�#��+l#�}���l���Gok�uPT���;'(y5Oh�=�L�ui{��(~�5#�i�)y�c4�������B�|nz��v����T�t.`�AC�e��z[������h5f�������&���vJ�GL$�d,V^|R�`�����K[��_��DV�1���zkSo�<5�
m���v=>������`���R�7l�V,�V3��~M�2��4����eD����9g
kL��\>i/b�����s0gCb{�<�Xu7�Z�Y�����#i�g"[��,x�9��/������Q�$�>�L��h�b������|�RP�Q���1��2�k3#[��
�s������C�b�L�x�O078(u��![�|�B���&Q�~����bT���N���>�xq`	\�&9����E���^��UU�l��nT�FL�7�o����MO�cs��ssak���8l����Lei��i�E��t����;u������^A��]��b�(+��O����i��_%����j�6��]�b�}�n��69����_h���1-�KT%1)g��l�o���RU���,ckx�@�����yV���C�4�c{��@�8���mQ����M�K~��u����������K�mM}�1e�����Nj�/��6��p�����=��n5����*������R0��K1!�P���v<�+y�r:\��XX����s����e�h����Z��������#����$�#;n��/:����+%�U���_R-���9Mk���s[-������'�G�f$�&�lmd����[�g��+C�
����[3�x��9.�kMo�����(w��#��|M�"���?��ez���������q��#�?h�E��Id�U�V#���i���)_��a=��������Q}�mg@�s`/�e�?��{+��>�f���?���\�d����J�������
�_�V5@���������&���W�CQ���V,'+�N�
�]h��E�2gch�%'3WP�%4��z����S��e	�`�,�ig<4�u �"w�(^�������`M��\��\���&��=a�5���c��f�Jb�C������������:LJ}�7-.���l����^F�V�Ed�(R�����'��g�I������4&�I��^>����8Z�c9Cs�����Q!���/�s�u5C�+��7Y�"������_"�69�H��Cc��7�Z&��[%�����R���q�`���9y���H�;b��N#�JLg��������� F|+��l����m�3��L�gA�j���(6-?�:5�o�zh��8���+��W�q_@�e.�oI�,w8�"���C�zF�{��%�h���-�[���]^�e/	�E��RO��c�B��MW��m��|�[�\��)�;c�41�]Ym�����g�6&Hz��e��)P�N��K?�+=�j����n�~�tG����L�sm}j��{,L������j����l=(��v�'\�0���t8�����>2K{����/}1�K����!oE+���K�
�n�*��~��Y��`I��^����cx���^���4ps+.�zl!
��}�	WVS��h�+Q9�tg����J!/�6'dm�[���&�h���dm��}����s�w��/�Q@Lums=�Q�/�,�Kz������K'���|�-qc��mI|2��b�
������P�iu�"c��������f���
g������.��?��~�.�n��;Gkl�������������_4
��1���~�m|E��������T�q\�	�Y]�gK��>g�
�Q��M�(%�x��\�d�]/�t������o�a��\J3"��<��hZ�?��*Cy�E@:��%�*D>�Qq�q�F�|E�����{�#~��5[sx7`����b���0���<v�[��s����
t�F�����D�?��
�-���j���k���_��AvL�s�S�� �������!����j���3nZ9����M��B�m##��+�[��J��6������:��m�=�];�-E�j�#���o���uT��^>������=<��O
�z�Y9?�(�p�<_��k����%'�R>&�� !3�_WU��	�a�������-�S@�*FpeN.��C5���]d3����lq��t�n_"���{�V���Ic0����4Q�T��zc����i�*z8V��t&^�����28���H�a�|�C������,��+��|�z=�����>��k�'9����M����`�Qe�C���L!���������$���iNv h/�p�H�#�����qK��������p��@s3@��q%����T��%��-U���������%���:�/R���b�_���y�csry����X�iyk��J-I���.�b������c��1�G��:z���QKw[��.;�}$�r��;�P�[9��U"�T���Y��n��~������>�'5<w��@y��Mp+����i�E�)��@?��^�c����F4��]g�K\��\�m����������D�[6�/\�;�Dv��� Uj���r�����ck���5�7��7��o����k�J��lhI�nM�f����Y�7�77Skg��#2�o)c��>7��*�b��p���'L�W�0E���~����r���F��>.WJ����#��DZ-���!����`�D��l�&Ro�9<�����p���zM�q��s�'*�E�Mf�'�t��I��!���q�)����,�I
}����+S�����zr#��$�a�z��xJ �mXk��s��?$��^[���� 72$�S����������/���u�/�D����������������2x��w���j#O(��,��;)�c�����x�������WS�r*�C��6��x@6yFB��`�b�6�;s�C��y���M�E`r����G�����������[�^��I����W&|<�ab�F�^�0����t��4�������"��*��!���>ov�0��F?��z�����^a�����:��2R��}���O��=�#�J��ky|����!�
A�H}��kz;r����3P?|N\�QM{�x��kKVn����i���I	��]/iD����Q:�c�6�(�dS�oNl]�alw�M|�3��O2��-����jKP�/�t��!�����u�k����FH�n`\$��P�82\>9,:�=g�37��&� U�z��R����;����1�e�|d���^���\{L^�����j�����$���<$*k�s�H���8�Z��k�F�)����;����i\�2�1~\������}�Zt�2/(�������d6�l��[���b/�3���1����m"&��3/=1��������|%�@��}��we\`������U�r&���19'0Y�	��sG�2��mv��aa3�m��O%i#����7�3�j��~@�f:0��a=�'���h�s3�_|�PrI����q&�!����YQ�azm��&������C���,�I6�X7�d:�iO�BU`e����e�.��TM��jU���&���T��������1��Y�Q-'�)�Y���"O�k�f��������D46����(sX�W�8��f�R�wW��5���)������<5r�g��K�%�lX�%]0e��~&���dT�p�>�I�6�Ap��������;�j���>m�`V������`�y:ZP��i�O�Z����6������>�%0P"���4W��
5
��ddR��
�O{RE��U�p[d,��xj��O�n��0@���amI��nI����bI�W��O���5����`����z�O�j���b�sF�\RH��w�q�h�b�Z�C*�]=mT�����6��������t6u���������hha_����������*�����<����_B�~b�����"�f�0ZVe���YR��#g6�L�D�Z6�rb��������UEn~�\��R[u7����
��,�Bi<�kC��r�G0�����{����ag�#����1���S�U�j4�?����u�A��4��=,��n2bt����+���������b@����g��|��PY���z��J�)[�2�+1�1���
m����u"��� d�������N�@��
wh
��~�0��k�q!�,��5QN/�����oP������?`S��������f�6^R��)���CF��X���y�R���5����k?���z�Y���Y	�#�d��6K(Kg����7�b�E� �W�BGpb��u���)}BL^6�j�8��wQ���P`���8�^��7��
��U8�Xz&c']�������A�M��8�b�������"�����������M?}�������:�P>�gO�eP���Lf@��>�~e�L&M�4����}�:Y��\v}��r	�v ���%2?M_��q����J/�G�#&,@��M��@
5R�#�����s��\|4>'4c���������������N��<��m�����]n������0��%�)$��]��N�9"���,�}����JE��A�LP0�h�������$��'�>���b1m���g��&WDi��d��-i���x���zMn��[�����\~�����?'V��|x���]�dZ��K������$	s���^2vsG�C��j��a-6�_�I4k������������o�ae���F\^\�:W�([��S�-������R��k-�w(&J��W�������r�����Z�����G�;���6���Z��Oa#c[dRT��
����,�6���o�����mV������U3����������[��3�1�P{�������2~m�h�����rKL��p	~��)�]��}^�p������,�%o���S�n��}Qn��(| �$����&0�@�yB�U�p����X�A�+	�l����?�7��6�0�SCF����Q����+Z�wHqh��-2���"A�;AB� ���4x����B$�������a��sr��s�{��#��6E��p�������N1<�����!9�3�I@��kX�?LK���
>Z�>��`��	/�^*�ch�."�8��]���op+��<L����qQp��������~N��_?7L�@PH)�p��u0�}Y�<]��?c���}k�}���T��C;��x��S�y��A��k��	�����M1.�4�$������	�������&��X��JT*W�5k����9V�b���l�=���|���C��v&��@���PY)�]�+�G�
R���b�70�5��4�*�o<�#�}�<9�2�����_%��������=����%0zX��H�>�LQ	�S�J�[��������� ���l�F�(���U�N�k���B~!�`������,a��">������
|����)�;���u@���� Q���^V��������SM�q<y�2e7�^�v�$LU�o�Z���l�����=.|�fT���Z���M����%���[�?�qA��!N�vY����<����V��f��bdI%'s���t-������j��(�B�V���//�`7>Mk�na��SA��{��#�����o���P;-�F������+���+T�u�t-R[@�����J�2�Y��m���+x�B���h�-@�4��!�%�L��V������

6���X�g}���g�e��H�x�1�;�������q������.y-�����m4�\�s�?4��&���z/1lB�����"�s�]��Q!2���>�A��9_=$abo������}��oV<�?�����P;Nh�i��b�3m���m����zOo��	4����N��������6��D�,���%�������wc��������j��P~�_Lwr��1�}�i�]�)�]Q��8b2�9�x#tv�����r��&��7�9���O��e��mV��h��51����Q�x����<+n#������[�v�H��-h���#9E������ �?�@��X�h���}0X��H�Kt)e�M=����j�HU����@�e����#�C'�
F����jFH[jNL����)��Yg.7��<��^~�^�b`(��YiU��e��!����1�Q��h�,���[]!^����U��s���<8�V%����)���q`��/��W@�z�`����h�l����2�p��6|��p���6fv�;����+�7,���������gD�SiW�UzP�H J?�C�^��$$�X�&&�Z/��3�Dn��
����b2zM�Z���T��`Y���������	s����Y��iY���/}n�<���fL8�"Q���H������b1?c^d9 ����s��+�6��W/��i=�ddX���Y���5(���)1mp��*G$1d��� A��i������U3���{H_T� >m[����Z�=�����*{�b�I���������l1��=!/������{��Q��^|��Lo�M�����\Ra>\�����u����0Q�B"pm����H�K�!d���~��G���A����<���d}fou%'�-����C��cST��H�����o��-y_nQ�����CLDO���k$`"5(�a(�����[on+�t���`H�y�nr�r9���z�m~x��:.�7��Q����_�KH /���e��'��G���<$���$pA�t[�������]w?��5��gbro�%��$N�}��������&rF�k?�c���NZ:��)�kJU��3��N	������6���mk���n7�2��p~-'O�j����SzMy��r�A�����H�;��v.D��9���=T���@{�����.-�� "�*����5�L`�
 -X�����+u,3�Re�0`QvE��O�w�"�����$V�Vs��T��9`
�]���\��Ip�G����S��f��#����	)�xJQ�x��X�{��I4!?�~?���
Z[+����"�m<j�t����n�#u����R�5G���fC���CK�vr2������9�N/�?�j4�q�|������%���-��e��3���������m�^h�k�<`�4�;2"GH�����1�t	b�������pv�O#v����Z�Y�����N0�M���$��x�"���F�@�����%�>�L��ju^)�H�y�Z���'���<K����;�!���+�,�Ox�J���9tT������e�.���Q����r����?�J+v�Z��u���y/����;S����$4���'�S_\�[��~w!T����%H�>�8���>v�Z�<?^B�>�8�����CZ/N�L�#����[�6�~�H�O��~l��}��b���LmH����`��W�QK��{�����A�a)#������S<�3���P/�T����e�� �{�*�>�$N��~s���D��D�~{���p����V~-�VQr�������:���%�E��1������%-km�MQ�A�I>Z����etM��["��?�V��"K�n�"������!Dl�����&�P3r���/�JB��7=�pl^�X�r��Jj�C����zEo+h~�x����!	g��~������v�WY����+�����Nc
��?��d�L�S�O
���V����h&k��Gl��@�,�v�P�ma��;��*���*5PZ�����P�n�lTOZ|QGj�y��;�B�) :���}F+���m��T������Vu�d;ki���Q+������(C�����NkSC*q�81�I�5��y��~���\���p?U@nYP�yx\:u�m�0uh
2����M���������B��2I+��j�����,1�
�iw�����;J�n�!��(,��&-����|~��W_+!�T0��C���!j?E���YG�N�dWP2W	q������^WL�Z�X�E���"-+�%��K_�1E&2s"�rk*��N�
+$�+�\�P�:f�#�n�5���p�^�]6�[���K�pt	���~����,�kz]����[q��
�J���������q���������Zw1�����4�_W65�@��T����^�;�+�0����B9b�g���!��m�i��9����z<!����x��6r�%�0s�W���`V]'��F��4�W��vB�c&4A�)e*�Tj�������_Z�X����}Vm��1@���FQ4���(�������Q�+y:|g��%�����k.n��h��GD��W�OZL�����M���$a����������MPo��z�_����p�Z7��:��|�������k�X���]��y�v>��?&�e��K�J�Tg��n�I��]�
�H"e)��f��[cw�pO��?^�o+<�s�z����{�?�d��B@?����g�
��g0{��E����+(A��O�>��4�'`�g+�:"�O��+h�3�7.���d���C��!����FLNfw�&���F;BF[3�\����t�J�G���`B4���q�4Z�I��o9%i`��50�i��w[�<���&%����w�����[�W.�/t��b7�	q����k�"�Ms�_����6)2�T�s��K����dMP��!`m�����,i�Tu�����pt->*Z�iD$���{%�E�z�=�������E�_���x����N�{�G�1PBo~��T�Dz�,'�yb|ZK�,�Nnme���80�
������^/p�Wq\O�	@c8��MQ<���]U
U�RP����l:�z�*���R
�)�sF""���{G=pMG�C�HF��$}w�z���~n���sC|?�������`�����Y�R���!o5s4���s+>�*�N$5iiy5����#�w�7��~$P�1|6�����t�_�b�l�����#m��'�J/��uZ��g��>������v(�;�7���1*�	���6}����d���']]`
?=I���Y���*& 'sF>yw��l]8�RUoh�0�	�2�DU��fx}��C����)��\������5��u5�J��|��yZ|��O�����I$E�����T����6qu��'�0
�����N��'$%o�x���No<Nrqv��~7e�p����dR��v5�e.0/3������
t��)�U�U������[>����i�W:�J	*���-[��uC��+��A�q^��t��"uj!��W#��bP0�-�y�"�]��G �"��q�})+���?_�aW��XB���2���S~�P��9��-0[�Z��PS���mE2��v��Kh�7�
��QmJnEp4��O@��W�����J���V.���YG��u6���"���m�)�{�Rxl�t��)��z���0	 
wG\���Y�/�y��]+WY��B��N|w^���:r��IK���{���:gI� Q�x��>f����"�!����s=�����"����� �C	j7�9!�����3
�5
IK����RQ�n���YX&�|`!���k'��������t��@�~zjbZ���2�~�|_���CIr�Hn[��;����7���c�!)}$T�Z��J�@�^�ri����ZP�)�W��Nb�����u)��?cF~qp���0F�BfP��3���d�w'�a��2*�K������c/���������V�Q����%�
J�f�I`hx<1�C�av���$��W���F�4�xoQ-=��;���C����;�_����������=���WXQ0�0��F����I�N�XE����+�����V�����j;�Q`�,��w� �n�{�_K3�)B�����?zY~��n�&�}��t�/�j��i�WB�W��Jt))<W	�{�����=� k�DD�Z�Bx(�^��7�B�&�''���S�a��7� �T�w��R�h�%��r�0���t�6������Hb0^��A��!������{�g�7Z�1wp%\���G�T2���p2�>��� ��g-L��f���ca��k�x�3��x(����c�[v�����tl���o":�f,^�n6���t��� 7:����^$�k\r�L��V�![�O�5�[�/��7�+� �03�������,?���:R�o�+��G+�{l�;�]��z�~�s��L�P�T������n�5���,���#�
�[}�ro���eO���BH3r��B���QN����i=��nC��1����l�b���������� u��3%��t�����4��s�!"J���-��X����8;;s�����d�N,��LAk������*H	'Q�R �����?�1�'��3BIo�Q�����m�*��O�t��6�,9��F�
�j�O8��)Y���77G{��`7�m7{��*�A�����l��"��Y����}H�y�y"�/���W<��� �����#�M���M:�<�'���z�[���������$:�[W�E)����wv����j��c��>CeO��d�\���Fg~�Na3rgN.{v8#�I���;aX���)��dIZ-+�MB8�Iu
�s�����8�A`pV�x�E*���vy��� (�.���d����������>'�-���_p�����
�l�9%��n A��a�:(K;a�He�9�k�P/4�Bf&#;�M-��I���J��r��LRF��Gl���ix�\��$=���)��	��S�M=VJ'A,���������_���3��fH��=�q4��;nU���\p�������]���8>�>����2�m�I��_1Q4����k���?�%����3pH�X5B�������C��/E�Z���Z
��U�������Of��G��t��6h�#RP�^H7��N�8��������E��bb���=�['��\K�+;f^g����fe�Y��{���������J$��D����d�R��/?��R���f~��k������<.��k�X���CU������L9*xK�?��vE���K��k.>?�.�2�-7�4��T���=Y�������4�k(XE������y�J��q�9b'{15�~��TH!�����\Z&�T]����%\�+���I[��F���(C[�4�0����t�	Z�������>;��������XWB
E����������
��_i4{%1sW�S����t����j��H����	�H/����F�����v��#�������Xy�v�[p�{p��)����Y���`�a������F���WV�
�RD
���{���'��9���p��0//���y��V�2
ge��
ov�	r��{�<��r�$tZg�����mC��[�x\h�u�V�H�a4ly�'�(����wG�P��)�t]�O�@�#aG}3���$����#�1,{**��o�������u�2	�'oH.�I��P_U��,=v5��[��{/�I�s]B��G��rY���zN�tY�| �����H.�$A���X/%����#�F_VC�����v����/^�,$�Tj��544���5+<_�s�>Tt�k�l�~PK���V�]J/CMG3TestArtifacts/test-case 2 -t1000 WAL ~256 bytes.png�Z�[��$�X�A�.���HK.HKK7��%KHI���H�"��tw,
w���������{v��9����woe=|������


�"����D~#����EN�����n*�'��.=Fa�*�5
��wl�h�~4k����������d��]��4�����-AL}Tq�Q������[�h#l��U��CG`:���{;i��9������0<����PR������mz<�s�X'fyy�]��C-�fbb;Ua��i�@}4��
����-7��)O��'�������~BC8h�(��S� �Y��S�$�L�=`��E�$���,��J�F���s����/�Q���3�&��[���$4<G�;�� I�{�?�A2�����e����"����X����)QC��?��P	�I�QT!��)���2���:*�����>Mo���7n���2` �����J�I�d^��X�r��������+�>����O�-T5�N���`�Q���?C���!$�3���+�����1�R2��*�P������)A�G��������1�OS{Q]UV��~{~+�_����GC����}yp}�C%��	�t��k���)���s���lg�-4��m���S��/�zk��/r����%�+�5�s��n��S�P9���L3��!����m������B%�6"��N�:E��= �d�4U�D�?T����P�c���cg�z��E]����z���>K�>��!+�/���h�h%�)z������8O��|���/�^�d�����5L����F����J�������[^~C�
�"�G���AP�����p���_Z_�%������T�#Z3R��A_Q���e�����$N�������<�$*�	���K�J#U+���t�@��fu��l"���O������*��.��Jd8�����pi?�L*���6���H_^����KHB�Rt,I��V��bF��>zZ47��U��,����K�V����D�#�'�` oM���am�H!�k&����J(v$'vf���{�3%��+:#P�m����/w���8�C�EYZ^�9��q���!5�3U��!<���Tm��1'2�^�F���~k�7B��q�����XT#5R�CM}��_�T!���Qa i��>�cSL������'���D��q&��o���!BJ�Ca��Zn��6N6�B���F�H��P:a���I���	��m����V�Z���C����6_#����!�z�����
;���q=`����)9�	���_2��	�|{U8u�n���CMi.������	yuMi����%bM:��Vu�,��������%��%	�d��,���L.�Q&�����;V�i+
�z��a�"�Ru�>��_�b(��Y16�o#�p�����2�u�w�i��7B��v^ID`�M�s)�t�V��*�-�������`���2z��_t/���,JI�x&���(";�HUR
P�Q 3��
��<o��j�\�Q�@��)�����	d��A�`���#
�L0@���nh�g�,��}W_F)| �|B�#K��wY��)��������}b�+���v��~)iy��CmmA�����z���=	����A�KT���}��B��rr��Un�o�R/.��d���5��T��
S��?����v�8��k07��Ovn�&����"��1��l�k�V2 ������g�vh�����?Rd�]�rJ>���S3���������kZZ/����~�G��?.S&q��y9�
6��?���+b��9����*
^M�������ne�w��U����	O��[Z�.��8���$���2e�)�o���gN�]!C�6%PdD*���O�1%��l������N�;�	���:��,���w�1]�g��2�l��#���~�*S��y�M�/������
X�x���Se���b+��]�6��v?k���:����Z����}u�:��v
7�����}�
�$|�[>��2w
Z�c(bE����yo��4e�d�p�����g�UOT��1)��1+��N���Z9]w����>��9[�8~�o�61$�8���/��/��Q�,���oHx��`IO�#~l(��4|3@��%*RXV��go5���R�e:���H
f:��R�������� �f�G2p�1��oi���O}VWEb�?�����;��.vc�C��,��P�g/��oNI3Wb-������w���#����	~������@�}�[\�w����cB��G9[xL���"W��� �oy��6����Fz�q�3?d��lr��F�]�2e�ZV�!f��A��{w�)����?''	��g�a;4D�����#����TK	rq���{U�]P��y*�O�.�
�)���XO�<�Y��IOJ>�%�\�l�!{"�>�����F�c��	���C=e���@��5��K�X<�7/�
ne�
������������7&=>\7���=T����Z����'�a�U\�u�t
f��L�
;
J��Q@AvC��8�?lK��S���}�/���H�:�����
�xI�����c\i)�g�t�u�(��	�w�>��^��]�Pj{���} �O*�?g�d����<�x�x����W��,S�1,�nA���aT\E?��/m&h
����3��B���CCB����9#>H��w2��t"�NX"=0M�C/�-�(�%��_��U��P���*\��������`gI�1�����2��.�i���A/��%�t�Ae���T�g6��Tw&s�J-\�&�)����p��2�M���B��+
h��
��X���������y��z�gN����zs���k������On)N53������b���N���	��]��>I�}�*�J�;�0��O^p�<�a>����J��)��cj��#�~n�Mn�~"��[��Y���9,vKMi�>iT�V���pnB�wG]��g�hK-�k�2K��w�#���*0�W���*1?���\��O5+2�7���}z���&>8�����3����hN�[;��j�|��P��u��������������'���I�Y&�Y1s����!����/�X4�=�25�^W��2��"�8�l�F�j2���hX�tKY���9��&�f�"�����^�i��	90���WF��C�M,�}�^*����XU�Y+���V���s��3MU�U7�
����1���v����6���c�fF����9|fw6	-[�������m�Ca�Np�t�/h&���1'���M��xw('�F_VJ�.e�`��^Mb�#�~�:�_'����E5U�!��]�`�[S���e?�v�-5�qY9�AP98u�J�)L�>�NW�������N��9
�gXy���F�Z<g����,�Gs����ov{N�$�����U/�
�o"�%DE�-������k$���`v4Em�����Ii,���L}�-���8"�D�E���B-z1|���-����V�O��O6��M�����uj��:����+c�d��?�n����y��j�kBn��e�(���XQX%TN��L�����H�������U���&��(O�E�ib��Kj�������:i2w���@2EtP� <��[Z�T�I�`U����9�\��qx�bM�NW?w&��7|g��vc�7��_�^u:�_��\�v�=G9�
����k:����B�_%�|�
|�D��K��/K�>-}'o�,����q����6��-P'p����S��V����~�>J�-2'���}s�g#��m����n�7x\ix�0�uQ�����\��w��8��A�L��U����rOU30�����}*X
)|Jl�6�)�@
�"6�7T��7A�=y��Q�+\�M
qR�rl`�KY�5`���]���t��"Dk���*��
P�>��d�N�+�(��V��+�f�TC?a��s�X�m&4���>���}�"��B�h?���2�I�OqqL�������	���mqx6'�[�SO���,�~f~�
.���r����e����GC��y�#��-�A~����"
H��(�����>�{	�����n�bm���;����� �m�����hMJn)�� ��)���#��Ha��)���w�����b:7��]�����xH��S!�g{i\�A����*&(�+�����{���N�����4��.5�� ��)v��-t[qQa��3^��D��)������b���P��y�%����S�	�c����(�E��l��o0�S�G#Z*����v�Z�X"��M��A�����M/J�vFU�����p�Q��6d�������c�Ik��@CO�Cg0���e��p�_|e�Sj(��C�r�r��q2�l)$Z��&op}�����p8I;�u����Qi��� %���d��kI���/��uh'�	�$��0����E���3�E7�G[l^��v}�'��JH�����������O���������j�n+�$'�RS� ����z6'���#��7�
Rn2v1�}eC(����H ���-���8)X��{{�if�kS$\"&�b��| �G�(3Z"g���']����{�31�(q{t;xJ��:ff�f�3����u��� *�d����#�[*K��y�N�=��I@���
8]���1A�
�m_�?�B��Tv������@�
�L�������B:+V�����d&v���n��fk�PI�W����P�!a��P�@dBb�M+�4��x���*�	��vt�f�U�RGl]1C��^_�@Nz�u����'A?k�)L��snx0��;��j�6 ���0�6:E���6�����Q0g$�,!P��w�������V���� Q�bg9���������
nS&��K()W]��q�	�k��?XI�w������FO�%����@V_64������[;t'�������G��k�V�l�^�{~EN��{p�"@����:�z������xlEFVj�brm$����]
z���=/�*N��:+��Ig#oh6���	�]u����sb�n5��Uh	N�BnvI�{����� �w`�����.-��{�'j8�/\����������O�"%h�\�K}�e0���u��������k��7���3��pXo��%�����i_���
	;{�sr��:���tm�Y��A���/���u�7��^���"���.!�;O�r�F*+>�g�wK��/J���|��b�=U�Z�&�gd��C*�K�Bs���)&��usD����+�%���k��KB��L��_�[��2qTH�W����^�:P���<��vB�F�G�<O�uT$��4�� 
C0V�g��U�U��/����g��A�2�S��}��X=>�>�zt:N�U���F���-3��)��Gp���#��<`�O�M}�t���iO�AE�����X���M�����gk����2����
s���%��oS|�,SH	Xe��P�*����&0?�1�uS��>�k���6���9n0�����
^(Ip��5x���\�F��Q`�-/�N[Nw�euL��]���b��8����4Lh�s.xuW�Gu���`�Ale�)r�K�z.iQ��
M�M��U�Ro������K������:����Vc���sZ�`k�{��+�����\�Ty,l�"
�����:/ �4RZ��
�07�E����(�a	r�7�c������vFD������oJj<u18��%�R��-0L��(����X�YuT���i�Z9H3'��4y�c��1N�w�i�>H�;�h{�dc�n���4�����j����}���t�(,#�� /��kj8���)UN�h�[�C��[\�����PG�k{�C'g�3e!�D���eSN����f��6b�1�&�	��=�����������C_����=�c�����!/l%���#}��y���2H��8�[o9	�:��6�)�s��*�{��� o�����M0��
�#��FY�����������t� n�?�'**2>n���9��~N�#�a���3jg5`v�>�(�&��������{����p�U�|s�Z��D��%��:5�W=�k�D�����FX�������X4�m,O���?�/�L�g���n?xb��$%��z2������+
��������3��7�>@��A��B�X��4@fH���x�N!�����riT=zN�2f&�����]	��-WM���M�����Msf:��t���*��.�8�J��S_p�a��x*9��>u����Ae"mTN�D���
 }���Ltk+�(�����U��L�����Pd��5N���L���5Y����
���&�l���3�����Z��;�����85<)�3���=�@kO�vyV.[W��fA��	��{L�����h�h�Z7j_��"�
:��u�����Y���8P�������"������������V�7�r�G����J*I��>[�,�����|���$�A_p�
X8���%R�/P`S�0�J�0�zpx��U&p�r�rAy�\�K�^�\B�s��&6����;��^�7��'X�H�R#��r5���B�>"/=x�r%�o���j�����6�'^`WP�w���:��������e�m�u������R�����>%I��$�>���+)x%(��1�������������)���p`K;���~����l@L�U�4
�Gb�hl����bL2��^�����J�h��/�`�|��Om��P��cx�@GL��q��f�����`O��?u5�2OI��d=;Lxw<���<����'[Jd����r
�u��8�]���BL:��t��D�_/6c�����$���I�B���<���-|B��]�����nlJ��
k�;��8��]�sjR]$YIE���oZ}��u�0�A\�D�_��?1W�#'��:o�������#�L<�RuT,p�W����^�Q����I��X��8:�������=����}������T�!����dh��<s�B�h��{$�j(�����i�T�M�~��������}�k{��+2�g����*Xd��~vG�i��? L��Z}-��~Z��{���CD1yvw��X�{'��p�h����C�F��5�:XG�����o$�U��@s��L�%�F����a6����o�-�����/����
q��jJ�V�8���e3��<,���=�v��+�u���F��V��`�w�}5t"//�SQ����F�{}fHW��=��K��9���)����Oc���z�HlhE��mN�pK>Y��|K�)��Y4����w#
q�5��P�2zv%��86��(���*���_@k�
������0k{���[��5"�w�s����
)����J����=4��z3'U�>��:P�K@��������g������p���,�,���T����1y{�k�.k:���y6J�U�oS���U+��u��F�_��S!��>�~Km������6)�r�U���}"cb2;v���a�>��	�	�W��J����^GV�|�_����,5���t��� }j"U8�})9���.>����RJ��v���Gn����{���O�vj���i��n�������<e:�Nz�]+��������OcmD��F������5�k�=+I�b�b��i%T��P���lB��U@~~v�+dM���z����<����|�r���z<�8;&B�\l�QO�a�k)�0;?�-��Y��%�������:>f|�X�����<�#�e0-�^
�
�9n��������������� �P���b�.��F�����!C�WR�r���Y1'p[��i�G�qR�sF[r���bE�"�C��u����4��LpF�fA~�X�&_����������2�QUX�S_ ��]����
9����2�;�GT�8�6��3G_���1P�����
J&cQ�?�]Y�.������E�v����h�����7]�
���9O�}���_��d'�]NC$����ka���'A��E&�����8�b�Dbb�3j�b��7��L���:����>����F���'?���_a������������
��I"�Z)W*E��l.��@��a�,<���i������*�b��4���=���=+n��j\�XsU�(�3a�����9����������8��o�!�c=���[$H;�,��T�jn�=\C��C�Kc&C����9��?8��Zk��4C"�<�`�Mtp�%m�]8\�H_
�S:��)��l��f3���o3/rNv�k-��~rl�?��[
��f6O����Y}%
=�����-��v���e��AG���!���x}b<����w�����k�Zt@1h�����U�]���7mu��M����6�B�
�>�|l#�/B�#�<>xI���q�V)eJ�1��B�v@EH��}�<��a��f������6��������}	M�S
gs������g�����
w�;k�c~+�G�B�#~���f<\FF
��)�t��IJ�2CI�����D��z�������{������;��K���-�#�F0)�Dat7��4��=�U��*���>ZR�l ��i��@�7e�����V����.��q�eF��]f����ml� d��_d �Cif�5�Ag��+>r}3�n�����&�^���q ���I0�E������F������!��M+k����3���*&x�a��rKg={B�K	��������w�92�������P�����T3��d�����HTZ6���~M/�v�����+���c�o5^7���������4�7W����)~;f�^�oY�������$Gf!����k�������Y�����?�%�$	�����������u_��84t�^�)�	o�y�N,����O��jq������{}�����R�{�����o��C{p����7-��3>��-���'�����)9�v���d��������k�
�������C�m��8�v����.p��0~ ���~�e4r�R�"���H/����G�5��a���Y[8.���F%A���^���r/�fJ^h���3~���O�Eg���J�1t�R�	>n�� ���6}�����<-���s����<)� �j����`�;@�s�O�
��<��{��EpE�*��V=K�ILs����e��)��$���5T+I��L?\OA�7���6�*�<��Y2c"���k�P�"��Kp/�p�e��<������9�}��q��g��W������V
�C�u]9�Me��hBW�/L�SC�7z���@7�z� �p�����b��z��pl�Z���m�M��'|�	�as��Z�����)�D�m���]��p�,��`-����p����8Z@�����I����MIh�D�����t�x�Dw�CN����~Q����q��Uabb�RM�\������	������'u����:�4�����l�1��]��J��j8��A��y��?����\��\��D���'M?l���*O
��&�8EB2���u��uuM0����~����|���?������x�����z�����mdS�`�!g$��$�6���Fa�u&��r�H 	����� &�'��aKb����V7��a��3E�]��ZYP��!�����!1����)?q����Q}wO�m�'�bi{�f��*� ��p��M|�y�g�'�i�hm�����j�������|/�aE^�����<��f
�rd6Q�L���m0�o{�C�I��nL>���Z���0�������c��Q	K+�;�:�!��
���!TWS���I�n�������	����
�7c0���N�����<�^?lZ���V��rDA�>����v�"����,H`�~�ty(��Gq�J��HA6UY,�h��H�R�U�����N�#����L8zd?�Jtp��c��Gr�3�Vx��?F��p���sGAMl��k��[7&�a�<9���������=�jt�����`��&g@.�)��<,���+��k�_��b���'2w�[{�=��Gq������+�OG�-��/g'gTq��Y���<����W�4�����W�q<����T��n�bQ$C*x���Nx!6���a��4�9y�X�M�������n�Y��e���3���G����O��1c���bC���H������X%���KNN�G������V��H
��S��O��t�����GW6D�!Z�O+Cd8vra^�
&�c�]J������f��=�s�zO�aG��(/�" M�����$R����V��3;������g�e7�����'�mK��4��&�^����N0��r�-�xP���2�z���d����/�X�i���!����~VU)H���cl�YX0��uT�����j�/t	��;�1�/=�	B�7�,L��`�nV(�T~_����W(�
��{�#.KhXx��bk��Q1�X�sn�BqY�\j�iHO	��|���� ��U9����B�6<n&�|����q�,���X��3-#A�(���=/r�C�,"�&"����&\��l
�^�#V�/�0��G��Y����t����,�)�WB�c�*�24��k���)��*���
w8�8���D7p|[����v.�B�?�����B�{�-��o/+}
����2���[�;�_�-�� _b�����F����Y��i>���d&�O�[f&9;��l���^3�����=�����;I"79t-+��M��HL|�M��hh�n��>��s�Ik�����J32+���y���~i�bV���ofw������Cu�#��4�R��o�7��B�i<U�����h
9������+5�7y!g7��<�_����������*W��A'=�U�u��{�2�DhH^��PSX������}��l7�8�J���k�U<��+Y�?���}���U�_�e���-2T�8����#=��-����-�����(F_ �S�#�����~I�SlY������"��Rb�E
qK��yFU���-�X�A�	Bv��/$����r7���2y���;�
]>S|o�F��O�DX���n�:��K�K_��w)K��%�sY����\�em�����/��	�`�#NZ����so�U�B�����������]FdD]i����+������,��?��`4��b����
{:���f9N2J(B�3���"�}�����o�wvzXM�]6O�.�g���2i�/=/������7;�j|U	�)&�n���LR)�Gi����K�O!�����s����b���Q��V����h#a��>�����K������
��svZ�K�<;qy<?	xz��C����}g��-Cx��s��WB/9����#����_b���R���0m��#]�$l�D����l#�-����Z�JP��~4[J	�����}!��������g>�r�Dp�`�)�e�I�Z�u��v?*�OS���l��WQ���3!�����6��S�c�.b~������O��E�6KJZ�O��Gb����O�7�#"OU�
����k�oWi����b�E]p�8���7)s����
�bGJ���Nj�1��Z��0�����l�m]-����PY���q�����:k������g;��8-�u��"fE�-�jS�f�p���GQ���h��v)1J��v�]��h��7����w��{����+>��y^�<y�����vu�Zx*���AQ����u5UQ�"���'
�<��(�B�\���o�����l�S�(��{��{�6;�W�v��o������ Ov�B�|�����C������~{F�5w��]�$��;�M���J��N]����+r��G��`r)�T�;!�&�p}�2J#R��AJ�}�����5����_�"�mQC��(�WQZ����p���13�0���3�b��A������������K��9cpZ�����u�����t����0��+zR��gQ C=J����U�]��k��%�My���
�q��kK�5;����F&��T���`w�����@���~���cf�Z2�b@�8uY	�Ca)g����?�*PL);p�g������}N��qU��~�����B�;���P,c��'��|/��el�\23������}bF�6I����|QR��#Vk�r-���&Jm�y��En����"	�u�J�
�,���u��a�
�K��O�w��zp�����)�����\��nn��?�@���|�;��M{�[~S��I���;s.����O���G�"I���3��5�{��qi��Z=����J�<�.��7��r��5�rS�L��Pr�T��Dj�F��T����$7��q�V�e'a�O��L�3�r�j����fwn�{���w����}�{��$��D�W��I���Q2�)�:����rz�Ea������0���x�g=~&�������k'�|�#:e�� ������n��q�q'Zs��d��m�&*�r��e�_��Wd?������a�I��49�x�o�1�������b���(Fk����3��D����3�%�Z�?v��~a{���y���7>Sa1�,�0�l5���l7+ ����:D��8����['��q����!oy+�5�W�9��/0;�*�q������x�5��
�Qg��!��������A`=9%���M�`�?��I�L*_Mz���J&���S�����X�YA�T�c8�S�^�����>I�$kL�u�������������|�20[��L�?V�U7o��F�R
W��f�jK����!��F��M�s9��o��Rm�Cm��e"��'uGi���UR3��jsx��n���5����D��G��V���j��V�5�D/85#��k��]����t4E�y�;��{0�ax^BDf����/���Ly�{dI.��[!�pMHJw�����,p���8Xg�����M�bI1Z�����q�5}���U� �#h\?���E�x�f*�W��F7,����=�����������)#�����jO��}#��X����W��!���2�Bx�c@]��*���Da�����!31A=�z]���0y����):Y��Bu$��4�Fy�{bq
'�00�������-YnE��''�h�=���v�T94�P�+EG�Y�J=�\E?!n��\�	WM�`�1��uQ����%��d�7:����[*X�i��f�x����DG�>�X���(zJb4���Xk6���~����xn7�)�<hI���E�XU��%(S��3f����
��V�P�XK��n��}�FE�c������[w����>n�$'�]x�V�g�t��`������0Zh������C&���f�O
��j��/t�U
Fm%
��q��a�M������)?Vj���_����1m��LQC�����4cG�T�}\0t'����G4�uP�I�4@�fXM]���r��I�2
P�U��}�^�{x���
$��t�+<;�~Oy�o���e�86|
��13��u����1?b[K������"Hr�i��
���Dp{i��[����8�5�qI����1g$����9��|p(�����'�@~��������.��W>'�c6���H��=U��]��p�e�w5yF���
��J��u���3���� N>�lA���>d�#(7�?��f"d��*���_��_��p#�������R��i���[9�/��,���g��P���[$�S��m;FVI-j�q���}'�����BJ�*Z�,5���B+y�^�����9�w���c��
������
�����x��e��V�X�z���z,e����}�������Nx	�'z���|I)������a~�zh�bG!�
�$��5��wh��X����u������%�P*�9��n��|',wW�U]���M�N���8D�Ff�����D�(�V��DI9B�������c=����7�o=���)�@F�1oC��n ����TS��Pc�E)��a�NM���z������P��nQ��/ZW�2���Om<{��'����&h���W�nNL��E�r���������r�}�Ji=�+�4����'�_*�����9����@�Sw�Hc��@&dL	k�
�xE$��(�k1$9^t�R+^����ZZ��EcTm/���!�����XV2H����5_�����3P.��'��z��1";��)��)�s�;M��p���C�t�"�0�>�������>c���|�
�00��XQ=V�@������m�Gf�6_�vs)��:�uX�|Y�[J�z[����z�
_>��Y�]h���"�;iNS�zOn�D��tr�B�c'D��5�b����F�U����G��l?�G�9��.Y[����G]��zz��7�O�/u%�/�	v���eo���q�m�iY(c���]�HV�)���YFUb����P���<��Z����h�w�.�EF����b)k	����M���I�$�%5�Q���z"�2�B�kC�q��S	��e�:v��[���f:Xi�@��� � �_�m�_���q��!���%I���2���_��Vb�/2�� ;L����4oA�::�k�5�����/�~D��r"O��@�|sX	K3�0�f2��X�'5��w���M0_}�;�9���8��������t{g���t�x��:���6+.l�(q5�6���u� ��uv�g�J���.�fl|��$.EyF4Ef��7�2�G=~{]3:��5�Yn-��3nM^�4N7�U�e�2�m)��k9��S�k�R�����oq����c��f5�5���S����9��4��e?�����W5JN�=7X�6�V�Q��j#���~~[�n������QT��P�V��o�&.f��{��[��p�|��,z�+�ks�V��L&�����UB�uY��������v�:��.�_�����D.Ki�����������%/:���	�t��]i���H��\���C��%CUif����}�������v8"�`���	[���V�^����3�x����(� ����7b��\�o������Y��`�����2�����se����e�/����	���Q�x�m��_�{���u
���1�
��q��n�|�X��I��}w���	����/����0�� ��s�e���\�5��D�����oh�s��.`y���#l��n�=xx�T�C"�
��.��3�0:@_��|)mB,�=%oq��FM"P������)s�8�D7���������4w(x����K��r=����e��0w�A�����0����TB=1e��W��;6T�{����U5'Q;����DD����P�����>f��IQ��{=W��������/�N�c�����7J��~���c�a��k�9��_������K.#�������$s���0H����Z3���������`C����I��������&��WW��#*,\�b�W����N���+��V����Y�R��o
�P�E��@�}������Z��?�P������T�K��_�.�W�<�M�������<�����o�s3�.�'�Sf+�lg��M'q�xN������~��!�*a��#
�Lb?Wh���:T�2�]�����LMA��d�m�(�ne(x����"rM�Q�������h�_��'�������:��9�]���N7\_�_���.	.�������QM��f����2����lN����O����xyah�.3�q�DY�f�/S����T�@99��h��kk���RO]�V�-YEW��ys���RM~q��R�Q_�?��N�09��qe���i�1�����[_\�6M�q&��xU�h1qMk����+��
eP[�Ij~�����J�1��&h���C���j�������M�u����!��K��asc�*�f���� ���;�"WQ���I�#'��n0������OV�����M�w���c/_]��IY��F����g�	��"c�OJ�`M��D��;HVP(����]��
��V�Q��	)���J;���#��0y^����V�0������
s���E����h>b���f����A$�`dP�u]��`I;Pv�Nr�f���n�ex�����r0�G(��
�N�I���o��� ����e����J"�]EB�7��1_���6�bN��_�b��6<�e���
����`����-iR���c��M&��Y��c��� �
��q�ldf�,�jVdd���6������j�;R�w�k�]�l���C@������@u'������<��d�]�vSl�������U"M`)����7��q @]�}�.q���zIz���u�Mh��N���(-�6�U���f���<���#�g ������2��/������eZ&���E�>~j��E��"�2�(���*s&��!��N�$[��Z�UInrk��-%���-f��<u"�=��%�8N�������;$i�\}b����{�
��F9$i�rd������19��>��R^��S�4�)�X�tRvo�jr�g~Q��c"���a�m������5� �(�s!b�"O�x�o���7��+�8F����I�IO���:'!Cq�ug���aLW��]D�R��<E8���O���]�}��1����J0#>)Q���d��������.�n�����-��E	6��p������PF��]�=##�$]�����q���e��Qo��n�aHn{:��B��r��O_QB���+=��n����V�4e�j�/Uj�A�Mi����lRM��N��������du���?{��R�^�4L[X���@����Kb����2r�9{��P��w#�*�9r��$���j��8��H>�&k��j��W�L�RT��l��~�lg�"�[����� ������$������Y��&$n�oC��H�������G����-y���"B�G$hM|�H a�{\�W�	JNH��K��zv�@P��G���BB��5��?r^�K�&�t������c-Ld�?��K�����@b���*^D+����!_�����PK���V��)�G�L0TestArtifacts/test-case 2 -T5 WAL ~256 bytes.png�Z�W��'e	���t�.�pi)��.�]@�����.i�����������w��9|p�{������on����s�hppp�e�������a?/�`�� �������������Bi�Ryo'���lE���Rx��d�:�������oN��y����'e�Wl)��p�6")�YD�����=Y���I����?��ixN%�;�9��U;(�{O���|�vx�q�XXV�S�MGK |����~D���c�v=����a�n`:@B������T[t`��!==��H��u�� ^?�"�����~z���6�������
������_K-}���;���_sJ���C����`0�0�?g����.�	���
<���?���ZW�)�9���/���>H�5����C�R�������X�qM\+5����� $����N��U�?���`�x�B�����EA����?�U����t�X&��g*G�Q��g���7 P��=[��D[�0]����#��:/���i<��
���>�J+��������
��������|���2f���7�b��?�a_����!��N����@{H��^�80���en�0.
����'�:+�h�<{�\Z��������������*OdYn\�Q/i-_�
�����6we�O(�����T�����qO�}���?��O��H8��p5������_Z��&�^6���
�xc���4�m�j**��mMG�+������{�Q��'�N\��5B"= �6��|���xIp�^Q`u�a
(�g� 7V��pmwO�h+!�JJ3~]����U�:�U���e�Y�u�8�s����D)�`'���C�����plW%_1wL������%2nI�`��d��p��+�� ���
j�<T1�TQ���{eH.��Z`�:RN@ {����D�eL��_}y��7�2�����y���}p�����t{����s�"}c�i���h�J���7h�i��:�em��!D�L(�CE�V2.{I�:�W���vjRu��$F�2�� �}{��f,;C��u�K?�.�n�
�Nr
%�W%���g�G�k��Ov`\�9����a�)��
��{O6;��.�"��Q���@�A�_�_��+9�(������x��o���W�D�^)t����2L������t]*��p5��?�FC�^Vz��O7���B����:3L��������R��N�_�a��f���`���(�P6{��I�����oz{��/���e�e~~����},���������Z��4 >��]t�������t�c��+�����]?�-�H2��U������7j�u��H>f�1�?�<���T�'#��P-��S~0��t���������/�@���*=m�J'n��{u:0!(�R��_%� ����'m��i��s����~��P��E)�z!��V��4CV�_��G��4^@���)-�TU�R��Y-���z���������'��I��p1�pm�c�4��#83L��&�Nh<38���R4�Cb��.���-������Gv#��b������i�8B^�~"�fj����j���,���U%��+�].L�HPJ����q���z�8%e-(Q�U��hW�����h���APv>MA���+��!���C�S�;y�H{�c}+��� ��ov}��z�m�#����8����H��_�����r�U(��B�����Ex��SQ�/����>�.����o����W���WG0����#j@ ��$�����������%�_J^'dv2iw�����C���o0P����(w�k�F�[�2f����T�����\M������]�{��cZ���iNd�U����,�0����L\%ZP5��]���k�{�%2QS<���9�z����F��O&������iJ���TRD	S�'�Y�q��Rr�zj��m�	N���-�XP#�s��95���B� �'�!�j���������������WV��:��L�<�d_�Z�~��U��UQ��Tv���`0�8o���X,z�Ks���qu��1D���
5F��n���H����c���7a������E|��btI��/i�o�(�Ib,.&O�w����o��<be��!���2��������E��y���|"6[QV�Z�H�rnAZ�����F@��h�34�V#y�����O	j��W��l�b�6�|������Np�
��v~E��lM����jC&�
�m/>�b�T�}�h�G�u����4B���A��*�y����~���"������:+��;�&���.u�������@��C��-<�����4n�h�!�Uc�%�7�w�Y��%A+�P�:e�SP��+�%7��}��sK��T���
��R���Qp%7�7���M�[sA������V���T����~2���Nf��&���P��8&��+���@D"���7�
���K�{��#<�z��~MnHW^vH��E���3� v��iE���a�l��>��z���AKB^��>_l�}���LS�������Gq�W��E���*(�����}�;-�
=[�Z��L�+�w��1h(\���M3DW��������3U�6��_���/M���FQ@������{�I��d�1)7����oi���������?+����+���R��6��q��Qh`��6r^�i����0�F����et�E:�����|�������q�)n��{�B�
��t;TG���	����ol�<�lE)BlW�fk4�W �=<������	��S�[��<�B(0��mA������YD?�i��z9��/M�Z���}q�G�X�}�3
���u�I'�;�sy����������E��l�y��U�{Y�I��q�L��Q�{^_N�t�>��`�$�w��#��=;U�yN�
��|�I�:�z�G�=�t'4�1<���9PC���~�����qY2�mI_	R.f������J�N3���~-����E��@��A�)gI�Y���L�8
L���We�e�A�X�b���F�k<�K}7m����	�%i���F��B9��.�!��}�I��U��]?.��)����S"��^n���OV������]>4[�=P%*/�_���bx�-���Fi���o�����2-���+�;��+��~5@L�2R�T���B�{����r�M��0�M#�����L���t6-T�D��1�����X���y�G��
���Yj�Z|O�� 4\7�Hm���U�N�������%��
�EbX4B�KI��nA�d�){�Q)7f�.Kh�{�?P�����*�L��v���R_���K�C/���Ja�@�&��8�{�<=��zx������$�����y���c~�������u�pT��������E����'�
�����BZ�=���m#�)��f8����A9;K��Z�D��W ��8Pns��b5U���s��w�t����o-��;f�dr���T�m,�����������k�v�����������
J�/���#����r?X��;��v�?�.~6,�@x�rS�5��Pd�+��l�
�v��jh��F������/����s��	-����78<�?ko�X�+�{p.��~�nF�lX�����OVl���Q��)�
*�'�K�}?q�*�\�K��S�����wJi[fS��R��_-\|-@30$�YW>����1M�N�KI)�qxKV��_�Y��v��Uq�����X����(������a���xv����]�Z!��	��"}_v�	'�0ZdOP0�{�n�����Q}e
���'���^G4�P�U�Z�t
���|���?!/$�yc������nkl��#M�q�c����q��~��;��$I�`�y��w�A@��
�9"�o�"����q5*=�Y%L\�T�T���&w�
%d�����l���seK���v9�rM�B�d��g	��9w�����0�=��U%��_1�?6�J��@�9��������	C�������I�N�>�$P���-�����0��?�eP���WK��)�
G����~�=q.��Y,�cYZU��q�������}��8�}!���v��AW�;a2_Y�B~�o�NK�Hv �0��{����	���������X�z����7w(d���Z�F���%���#�Z��L���#W|-������7f	���"�G���+t��Y3�N���RA���K5����1d����� 	�:\��n����]Zu2����� �T{��8��s1�
m=�[�������qy=��b�I_��& ��l40���C��A�t�����Q�ya�EoBT����c�����"lzt��o��6���"Xm_�a�
�f�i�u�{6!C�`�:�g"Y������G�����+t���)�!���w���9W5��S�l�5�"�1�3�������T�����[&����I�a�<3G�`4z����w"����ThJ��p����h�t��:�K�y�8I;�8i���kA�S-� ������Q�����h���j�'����38��~�m��+S��"4�d����D|�x-a� w#}3��EK��y�'���1�HMN-���t��a������1�qv�A���.;������9��y�J������/��A��pO�YQNp���S�G1�/��RD��t�����`W��[�A��7���r��:��T���a���J�����K�aI��G�4��zP����������R��)xQI����d���l��i�1�� D]��o�$k
u����%�����>nn�Q}��%-sz�M��&��@E���]d>���+�	o����L�F�����0LG�S���B�J����*""S�t�x0ak~|Y��}	���}aK���oh���pR����t�Q@�� 9������Y�����M�A!�O�9Y�_�dW�F: �z��94�
��A"RU�+����$V[������c��v�hU&�.>���i�+i�E�����5X�bW��j�9^�.��p��tV��XxE:��,��g�~r��������%	��E���E�����@�e��g"y�|����k.]X�?�����N��V�Z3X�9�e�<��Z';������E������k�C$�q�RD��&���J��t;��bZ��f����>[���a2���=>��^��"���+��w�^����l���^s
���4K��{���a�f\����H�N��]��S���'-/)����>������7z���\��R3G������������e5���(�sb�qk��
����V�l�:E��O�cUpbZ3�)
�	�"��}t�n?!��fd�&�xL_@g?�|#Uf�f[��f���-��T�%N.�9��c)of���]F3��	r��1��7��C��w'�"l$����_��-���E���������uq���k�P�:;���|��\�����o���
`?������
ib�7��qo�*/�t�-'��Q�]�,s�O���}5{�0;n�'��r�L�����g��-��m]�Z�����K}�g{�5�����Aq���Z��
z�&��Z7~�a�
�K��g�%�P���w��hp��t
�e��)B��Ft�V�
7F?�&�~���\�EM�����CB���������?].���+��Z�1��	u�&<����2b!�|?����`��!����5k��j�
6�,�?N@���.�I��Bd�W�C��~�����,�}�I�9]1�h<��h�P��M,��d0��.m�3���P��R$�+V��xt5$
U��(*����h�4�~�d;�<��<~��������9x��o�]�Q|�[���2`�>�Y��6�5���M�����������&���s����9{.2+���������-�F�;��re��+�2_M:#w�\R33��f��0���mV�,�����=�������*i�]���S�+D�t`�0�*����~��NU�����EO��d4����M�O�A��.�O�#5��|�$c8�Ho����4f:�Ut����Z�M03(�m
+�xt�T��N��u�F���aS�
���"��)K��,(���bM��w�v?i��v�;S�?Ne���y����6
���~|o�_j���������u	�N��q�K �~~�W���1i��$�c�[?�2"���/+A�V:�����L��s�<W���^n��
4`T������u!�������G��PwpCAp�BS��z�T��� z�S��x�]�Y���Y��k�m\����G�y��62a�4��z���������_W
2���
�I���;yRi�c2$��R�e����?1c��v�(iT���#�s����M��B���f��y
��{U��S;_�.�����i�jQ�,�>���$��j���T��v�W�5$p�4,�o�;����r���P����Me�;X,������C��21�vq��b'���0�k���(�W���L�=���mY�����8E����,������7	�0/������������EMjNyX�����\W�N��� ST����?m��o7&��P�21���6!0��g*m��
G��J����Q�]XCjjm����{G�0MX��>�@F���mA���g?�8o���'?WR��>n�n���s��JV�}yCIj��B��l�5~�A"���F@��j�n��q_�4����+�+~�
f�h�T��|�����_���17p�LR���r�kj��8�����DM��b�
�	�
�3�]�m��Y-/V��.�7���������/'��5��\k^����_��^�dlt�NA���Sh%|&8-��$�:f�KG�XhV�����|jonU9��n�t��l?������2A��2�;���j�U�#	����uP�Xq��z����|L�
.���L��@N��8 ��0h���I�����J�6��Q����sKCND����^���-V�h���n�v��8�Jl|�L*U��p���[�3�]���9�I�@�N���A�N!����w/��&_"0�~�T���&d�<$*���d]T���{��JZ��6?����aAI�1}��z�(�����)�����!@j�"W,���4�IH�V�>���!$�)����m��s�R��*������]���5��������O��Q�����[�9�Z����Q+-��[�Q��Rs��?�JM��uLlL�B'����@������^0��7
P� d�#.b�m�Pm�������V�h�xJ0v�l��T~p��������>P�y<:��|�7�$�?:2%�4�$�b��e���b�3�6\*a=�9sb>��O���U?��{��_��������ENr���C
�|F��p�b�pE�T���r�P�x���|)��g. ��o�*cUDrN��xym�����LcQ�+q��N]���}J,2$�!VXFW��J��N�^��}ce~ +b��j�q�Nq�����H��M1�6-��]&���7�Y�n�
iJ5����rN*��9������^���;W�oD�f������
.���}��n��E���Q�Qb	�I,�{���g�S������@�j'x���lI ����$����th��5r���]����{UkR9��%
�p�C�F�I�)`�5��c�@��OK4��7������-���;:!�vIz
X���]RX�_:d�R���/�������>T&����}9�!&��,lO
B�,�z�)�lAE�TT�qM������S
���|�@C�k�<�n1��lS|O�#D1H�]�+�a���X�i��%"
S���\C�i���|�����8�Gkb=E
�������.�C����:������c:`8��ly!�����q3�����|��MY��^�N=�>;������W;���p��j6���������I��Vd���s�����7??
*��vn�$�3����Q�/5�|�Kr��(,i4'������C����Qq���=n_rF0��"����}�}�
K��a)0�!q�s ����� �tv��������}<Vj�,
��c!������9��p�� kp��aiz`��G�����{��~�G���Ld��_����������-��g��:��D�v{�e�Npaj�.����t]��T�>��d$�V�q�O��
����x���������}53���T�����73Z!�0����r��_d)K��S����%�|����b���u[���:@]�gl����f��*k	��"�1{!7�M�����v9���$�s���]2�G��6�-Z%�*������������(�2J6h\��x�z�����;H6��*������n��el���]��3��BdY��O�b����Y�Oh@���4$��Y���������^�����mA�-��R�FiE���������#Kc����a�D\$$���@N��K�~5Xmul�c~|�7zO�v`��������������L�,�����t������>�*a�#,������:'&�A����|#G�U:[��u�������L�������4��X
,����B��r����,�mrv��%s�N�������6��G���
$�W5��[�'l�c/��>�4�[��`{\�������:����=B��h�{�q;�������vr��>���a���["0L�y�{<��
������bRm�[���%�4�����;���3KZ2�E	���tDB�wZ!��hy������!�����HhRRy���]��/g���	F�
su���������B�j����p��%�}�����bf��Q����ez�T�f%T?�M(��j�>'Y��lq���PgP���RV�po�Fb�a��WUo^�\��b_H��$����k� ���5LP�@���,���h��Z�F�����I�w|�A<;��l
���}�
p�D��j�ec�����F��O�R���j ���7X�s��=��?���������3��n����]�E8��HKg�{�;��)?���&Y�mL%�@F���6��|���F���\���L������Y�K,"��Y��4VK�H+Hl����Z�����CUM
B�7��
$�[g	`�����$w
t&��g�Y�P4#���X���Y����A�;�7������r�g��;
��d}������t��@4��%�='t�5�>��o���D<�3�H%���U�m�\0�R��j�pKn}1d�������M^qt��(;��z������@@�����	� �xmRb�����5��a�F�d����i�P�������b�K������|W����:��.�	%�%������N���l��7�BU�n����Gm`w��S����L���[cOi�r�M�(�T��(���@y�UA����K���x�UH��&�	����#��:�\=��[�TtZE�*n)O�w��H��b�.4�c5����U��K�6�F��~i�p�xn��#��i�X_K��'#�0�S�)����i"'gc�qA���f��d2)�Sfpy�m��\R����r
��KM���������=�B,������nb�����?.	��pt�8xU&
X�k��iE�o�����21d�=r�w������]	�J'H���As����<�l~4#��bS�i'�����
������!�J��\W'��*�b�*��WA�h��F���l��kW�m��4�E&��K�?
�tXRa�qi��������@[Jt���c����$�W�jx�x��H~p��fr���_m�
����R��x4��L��G�gv0���oW��y�!��#��0�����4r�M�V�B;]�\-m��#�[�T��/����f�k+p$F�b����n(1���4�~v��x/n����f��@\�5Bg���k��n�Tj���q���OtSf{��+������F�%|y��q
����D7�/��y������'	���[F_{�1y����^��4Nz�U�Q�F�U,�V�����C����V{nm������Ya���z������u����6��kk���<�&W/u���s�!���F^6	�E�����+4��|k�)�S�ev]����(&dv�3��Sr%�����\����d������I]z���b6�X���+���]����&�L�9��)C��e�V�&X�'��w��=&����B+~Rv����;1F�K^��o������0��tW���
*n�����A������d����'`M����?K�*�SG����-BL���~�N��=2��f#��lD �.����,�?��"�4�k��	��w�{������t���.�=���'����qt�*}T�vUdL�������
wVr��R���Nf\A��>\A�Y���i�1���o����$y7�G�a���bnv��w�1�g�esNy7iE�Zk���������3g�sy6H]�y�t���E69���Z����-�2�g~���
��H��@�Y�-���A�iL�h����f�l>M�>��������@�z&��zv����]���g�[���)Px�jl�G�������*�dECI�:d�P@|�uF���Zb�X}�~ngr6��[������6�����$q�"��=&}�"5w/$�t\�$���w��:?��1��_�'T@�*f^2_Xj1���tq�����/W��*i�C���P��2wS����l������=]���q�A���/�#�)��	�e���=c����g�!'*�<]qta')������iVB�*�E��/���=��9�&�&s�����������S���]+���}��qQ�d�N	�i�H�D�o"'[U5�J���NJ���1�[����b�E�ib�1�	��Gb������r�����[T(�]��f�fP5�O}�p;��r��?�ij~db"���yl�QC����z��������]s�t���u(��I�[W��.V~���2��������0&�]�\��!����Vd�����FZ1��fv�Q���f���N��q#���nl!����;������������8��ua�57�%���+nq%�yv���/\jPDZ�pt�vO�Xe�m#�l�zuP��\���0U�+)���3d����jo9�q���P�n}��?�����<`X�z�����vIv>7����:stb:�����A��!����#�s����K"	b8�Y���D�m>P���
����u'�1h4i���8�����7�Hr��+��:1��FZ�N��@���`9g-���}��C�����,B&���)��O�WV��x��/������$B������8�h��Q����c�[�z��J����j��>��g0�7���~e7.�U������Jr���,�� ��w�l��Z)��z������u�z:dNyKxHc�Kt^m 1']��?C�Wn��<ujt%�LsZC��9wl����4�'P���I�����h@�'	/�i�b���|�)�����-w;����At�{�,�dB����0-������%�=RC�������|w^b���3NTV�Z�OLI�c���b�#�:0�b��IfY�W��\eX�Wfmq�A�S\�[[
l#�Sd8��)��0�Hqwipw��a��`��� ��6f���������o�w��{�{�{��*�7�8r���������
��'d�n����]3��������k���F�X�%�XG�}���#V0S%����?
���^E�D��R��aO�����g	m��P�g/o^y����Q����s�������#�T��[������5�dU�lk�u�D� �$���3Q<�Z�&O��(��o�=�D���qC7�
��XYO^�\bO8t@��� �^��qH�Z��$D�9 2�K��l����oP���D-���t�2� <�N����w&Cz8P���^k���� -����m�X=W~�yZR_���W<��g��l��w���N3����?����6�Z�eO�LTKCr�^��g����5��5�}�_���h��������K>�G7���y^��~�����=�f����`��O�0p�^�����A�i��4-9�
N!�[�C(��0��zy���=����_�X��.64�{���s��m{$t�k�/��:��i�.�����i6���1ui��.A^�i{P~�K�Y�~s������[T�|>)q�����H�j�{�`�tW�r<qc4W(�T��9����ZB�sl"n�|s��@_Df��7�ga��1p���0�4d��in��.#t��X�I�w��z{x������Y��0Y��2������*9����w*�-�6���/g�J�������s�m��7�N�Vt__�?����`��o�"aYlI���M��n�u�S��90��y��������v�Mr�����	3��h!����}�/,���si����}d$�u������n��\�y��4z�����5z7�,����
�-��� 5���t'�{��c���S�M�� f��H[�\H���e��HT_W�l�,�l�o%����qI��[`3����mO����:Q7�+��"�zN-w���Uu��L�v?�'_<�������
����|]q��1GR�	�{�2�[~����Nqu�����x�Uu���yY�*��/KV~(?��:b[E�=�Mf������ot?��6kV,\���w�}�W��)!����gl<b���U��V����)��\�N��������N���u`��	%���?���^|����G������_<�v��Xe+b\��`�
����(�r2�������A���0�a}�	O&��I��������
=W������8�{���6����)]���D�g���}^n��W����u�~9�(�v�Ky��0+@���e��!�HL4%%�`/���`���{Ma��`��5��L�t�e��OPV���F������t']����8��]�Ft�h�>�J&Rai9�Gj<2�)���K&�%�s�������3�^�E!�g�����JR�/��m~��6�2(����Spl�V� $�["����RA��W�p���d��^ch�f��3�b� ������	TX�k� �q"*����6��_��Y��e���U�����Kl������W������F���(1Z�D*���T$"=��tq���CP�+(�q�z��]�T��	��QH4���K!��z\���AWpo)�T�B���!1�������v���&10�)D@N���:�F��C���N]e����X4
<��i= V<�h2������x�m��g�������9G�����������Q	.&e�b[^����t����K_d���[B�5a{)�?~���$i�{h��q�^8���n��\�\�3�J�Q����E+���c���D����):�Q�%n�>�&���s9T������^a����7�)i<y|�m1xh�g�k)Y�V�����u�6z�����*����&�?��B�X�<�a/�%4�� ����P�����U�F���B��1�b�h��j�[�o�R�yA�%;����b�Z��?�{��iP�e��&2����?Q�rt�Eq�����,�#���a���"I�@�s�����n|���)0S��7��G��g����+�k|�7��/P�k�~h]g�������&9f���3���v&C- >����n&���*&)��\��;��v���*�q�>�e�8����+�� tjA��k�i��*o�o������oI����>HG'��W�A����q6�����}�����})��Z�!j%��p�Uxo�!�>W^�O��o]�][�Hr#�#�mDuc�����%];�;�����]Xk
�8Z;�,�Y�gj>Q�t^B��]�[���z�r��x�^�_9���Sw���|}.�R3_<y,�	[�������4
E]�;��1�R_�#4�P�������("��yF�����;�Bz*x�Z�5}
���JT��B@�V���u5���A�%��������1.Q��Sa[*�k�I����U1�xD[��	���R�N���q����y�������r��Z����6�����BL
Y%%a�R�G_mGv&U��n(m1��
��g�V@����>�<�#��t �������~bA`��Z�Uh	��t���OS�`�}��{��P>��2T�g����<�r����� ���Q0���*!���!��5Dj�d��@�o}�t7�����JGS�^��P_,��uUF1�!Y����Y�����������������
����3���Q�x�N����S2^���6��uz34�p��bG����^<r(���v!#�l�iG��B����K>C�{V�������d��M�.+��B�����0^@DO'��f`���:��$��<H�=> �56+���*��X8�.����4��L���$����
:RgM�b�#���sg�L2���-����h��(y����*�l�����P�/T��!�4w������Rrq�K������dq��LJij$��#L�<'���#�1��U8��_��P���D�#���.�X6���M'��k���-�{����Z�&v��h������YO�#Hy��������P�x����i�K���������
�gW��R'j��vM�:�����F�����~��x�������f�|X��_KDun��|��+_+V����-C=m���H���p�ASJka�D@���L@����Zbi�(�%������PW�SI1��F@��W��);����Q,�L���>�1^)�[��-��`E�u,��z��.���$��cj!�'��T�����I
#���ah�3�jo�W���Z�d~da�1�t��$#�����W�{J����-������!�=�	3���K�����K)��W������h�,��O{����q�@����g���wWw |������{)�QXiNl��2t|},g�;��Az��f0B�-�	g�������VZ���r�G�
���S�r-����_�!���)"z���J�G���>����k���y�x���3$�>5�����hx��K���Z8�����Z�/���j���c�h�lA�9����8-������R��tY���E�O���.�j�q��Yl�M��
b;��v���n��*a-G����U
7��I��,�U�
k
�U|6+kU���� 8���������#��S�g�s�N�����+u6Od5�}��*Xj������=�`S�6��N�:�S���p�:�)�
z����|e��F�s<��)�,���}�!��,�K.�������?��K�>�����+�����fH��r��E��]bQ���6�����(}� d��@�F:���8"�i��O�k�d�������<��oE��2sGJ�@6��{�EN�o����bV�9�tp
���|j��	&�����������!���^d�p���Gc4%�
�\��Pp&��u����B]����Bf�"��ED��C����=���)��|�����
y���c_dh�k����	��
�J~Z�z���}�oq�XX�}���;h
 ����\�>>��j���4�rM����=�N�n}F^O�V��8��2���3�.��z�cZ�����er���C
?�E���.S��fjz����q*N�%���;���\h-}��R��.�������<�v�58�r`�|'�P�������V��K9?���NGb�F����;7f�}RD������x]K��
K��jE����$A�'�������raa�v P))u�|�g�ZM�����ZWJ�5Z��X0*i�9�^|`0+�P�8'S�������"�h���E�*��%:%��g��G|)������t� ��^��F�Z�|-���N�xk��#�d�+C�C�����f�E�0��jM)����
�&}v�}>v	[Q���'�Uj)���'��4�zX���p.����([��vK�]�����t��|��9���Q��H�'�GD"�Lp��!{�E�y��W��2���ug�[_����,���Oelis�Y :,aW�i-k2B���i��4 7��P���nrN����l��S���>������}L�Kk>�J��Y��K�&f�L�]�uud�xn�������Hh5t�������@���:����X5e�*Z%"��,^h��*�q4��m�D6.u�n��c
�g?F 5��R9�u�(���Li� ����B+�~��_�\K���R����o�L>�HB'�G��_w<�Hfi��Y8����
��Ey��w9cK6'���8I�ZU�����_Z#f�$���d�U�����������+�8Gy�?�Cl)4�Oh�=G�j�r�R�K�8�z/�'xF��� D���t�2x �P(�������s^��
�YP%��"jl����h�G�V��P�Q�T�������f5��{~�Z�7RY���)���N�]��Q���n�������%�/	���*�>� �Z��}F�����-��\�\A�O�X@g�����]�R����Xd�n�*�L�y&>B_���8�1:�r���:��s5�Dl��:���m3�:����d.Qw�d������{k�l����	��
������z37;����m�Fi#s
mt�[X������������U��f���r��)���kY�SW����[�I����0�����Jq�S��� 8tY+�;��r�
cx�������������neG"���q(:�K��)��p��\� Ot#���L�@������B���j�4��E�X"�ec�^�d`��=��T4�j�4A�����e�X�I�r�$�y�x��Z�J�r��rl��k�����P[�nA�f'������X�^o�U�-������k4��r<r�RD�����A�K�
���k�b�<*&@*���)��\��<>>9�
N*��\�fR��N~J\F������^�����8�|�7Xh��w�-�.V)	.�=����=�}��6u5�?t���`���8J��u�kU�m�e�oB��p�%��_����@M�������G)�\��H^C@4�'RegA	�=���i�8�vMKGo��C$��D�e��Cbv>u��1�����$��J��;	�����C�V��+������JU,��~'h*��z�� C}">��R���h<kr~(p�&,-<R�v���'pPZZ������1���$'���������b�Ld6?T{-�N��f=�Q�,@��sZ�"���<M��L,�Z�����
���)���R��,�|��h�����c���c����=��q�|�+-�Q}0������r3�*'�����D�
w�>��������u��Zf�����\�Gey]�'z�gU�a9������0oq!���������V��J�8�]I&6��APz[��48����i�ew���=���0�0�zRJ[P���z������@x85�����p���6R�c�K�dDFH����n���z��c�)�v���g$���L��\_r��l/�����^��N<��m#I���!��|������&��8OiS���7x�$R��F-����p����JL}f��	�jwk?~�O
(-=�a��6����leA,�G����,<�.5�t�S?��>p0�z��Z��C 2��y9��C~F08�f�j�C�f��� �	x3H6z�8��+��������j������e3����G�rp
��n���v� ��4C�l_�@[�x��!�l_P��-�
+
���"��;ixCFR�`�bi��v��`:�G����?c�i�
V����2��W�z�|�=nH�����a�|��K}T������a���������gLK�����s;�Gjxk�����:���UY]�\-:�m���*���U�g��x4 C4���)bH�{���Jj��7P��P�]n�:��8��dZ�1YsT��GU�������&��3.�a�9r�o���h���
���d��������/L	���nK����y�����+�/�+X�8�n+�44
���VK�!m2�/���j�hK'�v�}V��z������������P��Yr�p�p��������)�iC}
���������.�{xxx�w����c���0���@e@�7/�~PK���V�=��A!F3TestArtifacts/test-case 3 -t1000 WAL ~512 bytes.png�ZeW��i$�$��;�����ni	iP�����.IA��S�;�>7^�u?�]����}���{��S��D{���V��
�{����\�f88J��	
/��6T �����(6�vN>��d1�H8�t4���U
$�8�!	9#��f��$zI9)����������9��!9��������J��YuB�����������������Y�c���G]2��uh,�9�Y��������#������G�Du@@�u��p�0�("%�������@���?�Q���"�g�!���}/�M�����;  ����
�>�� �Y��R��[��D ���"�������� �C���o
8�a�1��uVBy��[���	vE$/D�{q�o
pT�����.\}F��, 6���PO^)ScG��4�4'+�r7W���<�k�� ���<b�z�~-��{z�q)�CkuiC��i�S|r����[��/�r��UUq2��w��pt0V ����B��R�.T��x���)��7 ��<��B�P���\��a��D�a�}5Il�_-��(�i�O��%�������#J�i=��v(?&T]	o�w�������M���nLZmc�y��������YJ�3��W���4�����d�k��Z1��,�q���rf.�vW�O2���`����6��%��E�^�kN�#��hb�����:LE� 6���&)P�2�R����y�I�s;y��n���?D->����������M
�*�+���N������a�����T��(��O#@�&�0X_W�\A��b�Q���]�Q-���z���S�5���C�]!=�e0Yq�F4��H��&��5<�6"�%o�U?�y����E����g���T-^A��	��#��U?1�>��_�1`G	�J�m������&�5E_�$"D��`�9`�
�w2=b��7�|�j,�M�|K!�pg�N>"K}zcm.`r����Y>���V��(����58��e�U�x$��
eTf��|D���Wf�(E�5�������R#����}��:�N��t/
���L���U0s��_����p�(HP�gH������0���q�Q~�NK�ap <i���G�~-�L�'�����"������l&b�^B
B�OF������Y�K��iX��������mh.�	��I�nPc��)�
z�Hp ��_��Q���j��c7{�I�?f�J+��9��IdP�������H�����;Q���#c�>��s�E`O��Q��W& ����/��A��x�G�8lp~�;���P��A-�J]��?�_S��_��3�XynyL�zC�8YK�.�4�iHJc~�Mb����</��������?=R2""@��\O�sx��m������z2<8��z������$�N���t/�����kJ�`]�&;M*#����Fi��%���'U�I�F���g���o���:Qk'��pDw(����]`9J��wd
`UGt
�b
=�-��-��i}����G�^;�N��vN�v��2*bQT|��n�}jT.#��%�j-�����
z���m�,o�k\��^�06�Jv.�s�J��K����2o�g1�<�L�����_1aI��4w�e��3aT�
�f�(�[�.�G�ab�V�x��fl�*g�"����=�|Mu���;�r8��U[:)�����'���6"+�8})�T\��D���O��z�Jg<t8�0��v�i=:�t���,��t������M�����\;���Y�g�/V����[��������k��s������xU`�l8c*IQ��Ii��i=)��
t�;69����x�t��	���BZ����,%u�y��"��}?��d����)V��&
;����B��������^�����8�w������x��T"�Rd���wyr5�����l�I���Sq����C0lZ��������-Py�I2��s��b�zq���d��k�����	O��2����Br������8�����W�������_z�_��F��r�/=�N����6N�`�������k�}�I��
�s��s�t�;D
���v��R���^��g��~N��<@��<?�<�z+���K�����^�.�u�0����EJ���+������Y��5���tF-�57Q�G�'�6��I��p�x�����U3��^jJd��v�x�"���)s�B�����j��K����>�{���6CH��J����
un�e�������EL�J�Q��{&
��)�`iK+k�� ����B�@���__\����3{����O�_�/���\������(C"t���w��t|����
��X�4b	J�B��N��.%��XF(p���\O<��������r�������?V�@�;�����X�����F>b��p�Zm���M��y�F����H����H����q���n�qMa�������}N�L�<
��o��ZG�7�u����N�B���������gkAuv����Y��|����4*����S���#�F��%���|��s0���X�E-�Q�:�Y�`��V�#��Y�6�I|���XB�t���Y��m?����|6��)#�W�9��f1|!��H�aU����p�A%��z��'��G���S
��_,�*.4K����
~�t�/>��A����XzYO���'���7�j��q<B�������T��O����>G+R�xE�3���G������C���M���lb_6��v�}E������������@h����F�����Y�K�l?�g��I����yu�����
��g)�r��*!�j}lU���)L;
Fv�p�%�������K��Y�lD��uZ>��aV:c�./���T�/�FS9a'����b
�$E�����W��$
�#������6��AF�n�!��g=������������C�=i����?���i��g���%�x�)M[���o���8���U3��k���������lK�]ys����������:�0��������a���WQ��2QAn:��_�{6�8C�������v�6�Q����(�����6%�}�������;M�m���%O����U�I����i�������g�j%j.�_��}����B2�=���i���w���
	7ux����g�C����Z%�j??������bq1U��@��'������:8��\`Yq���uz�E��ml��)�'mu
�F�#�a�t<����8w�*���]��Z���%�����
8�!�k!z�M%;���D69�G�a[����:�����5���_�����z�6���S�S�n����������/Zg��K�b`���%��G�vB���N�����zDU
��F���r�R]l���v���h�����������k�V���:�I����s���y-���>����!��;��p+����4����Z����A�Df�R��f��	��-&�O�����n]^qpn?����-�$vl�>��k#}R�������u������S�x25sQI!����]�]1T�^V@����>��y���q�s���-8]C6�s�,�
c�4�-����b���������>��=�x�>�-*�v
�2��J���������|S���y��l��f���i�vIL%��cA������$r/I���T���
�)���s����>����f���*a���%m�_��m�4��*���&�A�Z�����XJ�!�Z��4��v�;R������%��)1�]�n����qb	�B��E�Y���fr�2Q�����z��z��a��-����j'#����9��dB�u����7�L��C]F�>�KGs��;��`](���KO���~��|H���e��Z��U����y�g�s$R�V����lZ��}��o�aOz��ps%:0����B���/u�U���������I�x���!��W�{Ln_N5@}�LU.oj�"C���K�q���i�F��5�;s�����RyWC2��0H���9S�>���A�8F���Q'�R������P��N.�AT��������-�.&�g1H{���^��
����Fs���t�����|��s~�6�Kew�N=B<+Z*���=���p�u�a\<3/��7�u�(@��N(yyLL���]%������!ne=������cmW������.#_I���E�%
��E@t3���w
g�S|�hi����0o��W&o���a���Y�����������x�/�E����g�3Y����n�m:�'5��4����AE�
��K��D��o�A����N���Nu���c35�G�����5��O���L��P
���QCQ�����"������)!UFT`"�����2)_&AK�n��W�.��4}�6�C�\��@T���M�q�N�kt?����������W�Z��5�XJ�/��<[O��>�������d^)c���8�}8��67>f�
��I �rT�s��0=��b��vB�n=�����a6��&mI��6A�����`X��t�Jz=��1�7�`�u������~g�gu���E�o����E(��R(�������04��)��8��1����������T�eW�9�p��l�"�w �����z����'�4s������|=��WL�E�"_�6G��XI��u����� �������M�A$�sWe����v�D��6�#k=���.r!�u�����@�R���!���[u�O�}H�E����S�$�
e� �y7j�v����Q���@�+H��.x����0_�R���Re�to}��dQ���t�@f.B
�����bo��0������,�W�y^��������D�{
@��X0��xv�8��g���������7>��Dx���j���/�{�s��BC�o���E�"�6��O�x�7L�a*��m�8���y'�V��@�}!��l�0Qv�o���>=h�����!��V���*8)������{���Zj4����������A���A���
�t���D����������~bD�_��w��Q���_S��.��7��J���o�) �_. f��9~��[��WMD�J�?q�y8M��n�D]y��_�X��S�H����1Q�K�F��R��R�0�]Z��p�"��Hp�>��$M&����U}}���l|�t��p�	E�Z!UZ�I9�H'B�y���L$�le~P*v�(OB���K��_������.)�)��BT�	p@�U���-����=;a�hhb����Fv���[��;�~�M�l���Yg��@�E������(�'u+���4�_�8��u�T�s{=1-
�3�Q0�{�Bh���_����m(�l����N	$�M
��>Wd�:bU�'<'�� �{2���W��(;�'���5p�|�">�|m�e��beV�1H���TP;�xb�?V��)or�����=c6������U�|��iPm�<�j-�����/�?�Jv*��X}l���r�uV5����A�am�|��T-��(�\��<\{����scY@.� al�p��������MX��k���#�ca����[:Y��B����1��%��������F{.x�W�Ym�/��9�G�9K}AS��X�E@�C6�0	���N19�	�$�v����n�;r��wo
��H�+h`yOD�����g	u���^��T��������j�d����R�F�T�y=1m=4�U�����]��'}�A�?��W�������~���VmG7��Z�Y��~���Jg�JR�2�d_\f��3��E�M�L��re4�d�
�SCl�B���C�j����6�5�����~z�4��O��?�M3�P�7{�0�U�S&"$����{�G����K�<�c%��n��� ���Dk]���zu�Z(N�8tm��D?�k�.��c(��aSh{�?2�{-�8�/a�~�e�C����a8�c^\�q��HgI���]:Z4-�L��z�%H/���
8�f��P�V���4��&(�|lo����K�7U�Oq��y������,U=�����b�/8������wF���]��x�/W���d��C��L�����?��pK�#t���"���f�[�~3J|�FV-�T���1����
����Hl���>����Fr��DX-L_�5*���&
�Y�����~��k�$��{D�~0�'�!�S���CkZ
��
k��c�o�I�m�����='�'�h�_QkBe@,��[&F��J
g(�)Ve���7/7�����f#���c���_��I�G������-{NT���\����(�fqIeG���x�K��!���#���S[c��Q��m�W"���k
����?6x9Y�H���rMX������%������4,-������{��g��[��y�<�3��q���%[��*�����u�_����P�n�h:�<�Q�������M���=.���C���W��@�.h���g������T�F��+���e�'�&,P��|�W�����\jx?5%7�7;��.���M���q�B��C�	��w�������}B���I\�����Au�)��yr2�]d5AP7�'��|�q����Uj���X(v6�g�psV��Z��bH�h�n�
d'��w�y�*�$N�i��\-��fV��>O���Xa�x>��������*V�]��P��J�}U�����XHp���K}���ia�N;�cO��*���e�/M�%�R�@~@]W��$[L���[��c|�/O��C/B7�3��vo�������VYF\� n�y��gX����c0�?�r���
�3A4��	�����\�#F�����.rl�o$��QV�)��D��zQ��q�d$�<m���4����0!��Q��Z�:�2��������6qoo�*P	`e��.�Q�W!�e7$�=��^0�Q�A��V��=�j��v�H������g�q��o���KK1��+�Lzi:���*��}b����s+�gfE.��->���lP����(���z~GQ����$C�"��~m�!S[�@)o��k���l�luh�q4Qs�k�Ey}��c�@}�DA�w��k�/��A�1���J�����u����"�;�lu��\����LkV�[x��C#�������W��e��ZF������*D2c6�p�����?P��F�H��������Z&A]Fn��}��%�E?%����#J��z�[�Ue�EC��R������Q|J��K�K9$-F��G&�^U�EV��9�X9��Lf�
�C!�|�v�ky�U��"�B�$|
���=�����(s�?d��]B;�_�D��'�*8���^��p�9	:_��}XO$��]��9_L�����Z��kZ��h���y�@vt1�/��s��R����lv�^���Pu�z������d���N��zgu���h@8��5�X�9�J��Z�f�[e�<Hj����s�9����?�����bD����%�����_F��:vG����l�����D�����&8�����x]�a�����B?�d�A���yy�"��%={�~%�<f��������^(��x�M�����?�d���}H�������E�O(<�w��
!�X�p����2�9���O���C��;}�bXs�A�������4_����u��O���6�V�J���	+{!�@��Vt_�a����A����8�dE��9�2�������``�e�^��W��F�PD{������}���4&'6l3[�_j�BH����LK�=�gL�No@;=�Pw^�K�x4��"��XX�Oc��n�t.�>�����~���]l!=Zs�]4d�F�H�D~Bok�������$r*�dW���v���az������-^�>���Y)����mW{�<��������xZ,yI7����-�^;O��4�F������)���X@�Kz0��`4�1�t3������:m(kq[��!qb;���0N�9043��uv�6�����>Z��u�:���������	�&����Fo�)��fe�_K�}�@������%�`������0�Rw�����|y�����bG�ga�`�K����O?���'���Z�u������������U���!�T������V�%���)���9 ^���"���
�|��S>4���z~O�l��/���x����l�f@�O)7�x�k�����mg��e�'��<����F]��)E���H��1��������5 c~�f�
�2O-�#��m@����K}4{�����
�#��]O��z�9y��-��yai������j�}C�����.��2	�?�<��|�`��v��e��'[,�L{+�80�UU�k�A��ly���s{>�k1E(��/�mR�lO��w�����<�`�����u����)r�i�I6���~(��P�HXM�#�Z���6���]��x@S�M�o+sY�6�^�]�]�W%�tl�v��F��&��r�k�B�_o�9~�1m���Wf�5zK���H��_��(��@����y4�B	��{U�,��g�8���w�3���|6�t��X�V�����+��_�[�>��hH �p�w�
��4�F���*%�h���r��-�����y������a�1����Dc�3 �����Q�'-����I����0kx��pz�b��]����/=z�LGmM�{���V������>C����x�����i�?3�O(w��f)��q����G�|���qu���JkqG����B�c�����23�Q�!�j��_!�����'�
;���w����$�I������D-���)��U��f��x��y%��'Lj&����54o�X�$�IqU���[h�m�$"N:�{AO����/e��o6���������l��vY��*������z���6g#��DN;�N��+�h�(H�:;���[�<�Ce,��p�'��=�m$��Y����R����!��5���@w5�L���ac�`eQS��=��LNw�d���,������J�6G������}az���N�:���e�.W	�g=$�!TiFl�]��+���%~���N~���_��w������x�
��z� ����w�\j%e��5Mr�����3��r	-5�7���y�-qfQ<�/�vb�*�Kv���<����,�P�V������0�x��^�[�����!Q��<j(�^�k���J�>Kd�4���������O*����'a����|kg����jt���t�d��p�A>��W�D�:Ht���{?d���]�x/��nY�J�%V�gt�K��z��b\���v�~��=�KjQ��f9vC�r����xp��be�
�/�>4A��eq��2��+Z^z�Er�P/��<�L}e10����gMB�;r������^eS�iDk�{��(�
�����`���r�(����[?��J�G��`q���P��YO��+a���I�_����y�����O���D|{����>~�'
ewn�Q���M�0>�ex^%�n�����(�����`�',��5�������~��5�g<�u�G�!���\�E�|'�g��x1.o[�H�&��=w[���>kP��Uf^U���?BX �g���k�u��	H8��G;���)��<����,[K�Z��'�@�@����������?,�=���N=��U.C\��`�6	v�Z8��7���\=�jytC90��y�������#]Z��<�R\I�`Q���\��v�@D)��29E�������m����b|b��||��}�d����o�B]Z�[�z0�Q��7s���edem��t��j��w��0��Y�{���Ds�;����
&���5�)o��WqvmtU,��{K�\f�gf��ZF�oDE%�'P�k��$������M�-Ep�����	��ez�/���&*9�Hw�x��"��4����6oRF��QM�^��z��L\6�e�1�J	�|=�[e���%��ma�h?�������%�}�*$�k<���q�,��m��4@��t�.��}�
w�t��.��e�e���*ci>gd�P��)����4��`uh��b3���E�J����8s�c��@!}��G��C���1+�%���
��������9�sR�����N8��
v3���O)���k�^���X�0���1���p$���
��\�lM�W�6�vLG����y�<��+]��M3S���u�AG�)5����3Of���[�kyy�H�L�e�>I�
4I���Z�S���C������u������L
;��U�Fl�����������p)�xu��;����Dz`�lh��f�W�9��,�<*(�F��7���A�G��b�����/�����A�U�$W�?S"&b�\/y�����%�-��3��	�`���������
Mt������>�9�${#�)����!�(8����G6��W�1���d��<�G�����s"���Mv�(�*J�r��Pe0��9�V����s��P�F�CKG�v�5���W�bNx�i���������t���a������K=D�`�8�~�lE�|���a����(�o�[�g����.�|�-HC�[�./
��
K�x��\��]l��l�X���o��>��|�����}M�D��;)j���}���c�vz��K�9�h��gy��%�|�>}�`�\�z�G%�&vR��z6���\���������y��~�q,���C��?�%�o�&z�g���Ch|���j���n��R��9M��;��.�9^gI.L.�J�p7��	��HR;�f����-��E��}w��1x�Hr�K�7�F����W�o�;�����x�H���wV:���R�\�[��T�`�{����?6�hI�Z�9��������7z_�����n���=��veu��4c�#���["���������$!�^�(<d������O��,��w2��=?i�u���_��V��^G&'��W\_ON�>-�s�R(�;�d\.���?���l�� ���J���)��3I��)��<�?����������d�?]x������X,�#�r�-��k?wV�xCG�B0�0���
���������`\��ilF�1��i�:ih�IL�J�f�J�DK���������b�gg|����^�1C��|�g���o5^�;�}��F���N�<�����!�p��MV�S��A,C�=�H�I�}���j�NI$
a����O�?������t�1��e��������Gr����\
���
%_���?�9������l�%��Ui����-L����6'��z��l-��P���I���	GW�;�xfe�/��o������9��V�+�*�@V�Y��sX�|v�Y��
2��������L�����A	SD)�����c�LI��y��{_k�31IA9����O���j��<V��x���P���e@~�����d�l��B�����Oq`I0]������|~bG��->�o�V!0���-|����f��R�qR�rA&�������Da6|���_h._�V�cFa��*�����E��y"q@���">{�YB�+���%I��nM���mQ�E�}�z?��@Du�����p��R#���6�.*��z�����]�3��ux!���%�`���+���q�|���>���?N���V�=�R(�%�=`}�o�4��!�>�M�����G���0��1�{P%���Y�����
���h�@)T�����Cd�^/I b���[�>W�v���Z�4�5�j0D��&K��3C��4�#�����-4B��X�oc9�q$d	c�&-�0�-$��i��~����{]�u=�|�����<�}��G��Z:��-�dE)>�(�k3K[���b'{�^s������Z� �
8W{��{�e�7�!�x�,��5���Wm.�Q�L����<������&4IC D�	����I9b����S�B� %�A�rH��/��G��"&���O@����G�A�(]��QM+g�|XG�W����lo�T��%���c�c�9y�Q���L��w��#��o������z�g�DWs��~]
7n����*,��$������n\��\��6����ia����^;�c�~DZ\�_�v��D������l+����+��q^�BVC4d�,������=��@9���K�u"?�d6�f\���G{����������M�,m�/����Me����s����b���<.D'sC�����ENnh	��o�+GME��2���8�@�nz�)WR�-���BsS�?����f?���$B�{�.��X�-2/(��43�d�B��P��?'TU��8���2������+�g������2?s�� ��
��Q�{����� �c�/�eK�'�1���{?�S9}�������c1:+%�.���l�����"�7��v��������I��g����'��� �D���,��|<����R� ���%�6���7pyI�_�zk@S_C<������`��}��������n���y�8o����D�h�1��T�������[|��(�a�����S^��A7fZ�C������	!����-���KP�����"n�i��>�����"%L?k�!�w��-zJ���%�y"]���+_�j}�8��"%�����j�U6m��f�z�����k8�����s�P���5����"��T�����q�}5�z��*�����>�"�VV<-a>��}#�W��t`��0/�+{�Je0o���$18</�L�V�{e#����C�}�z��������NI��i�n;��6f�������u�OX�=�p��Zs:���]��g����W��D�29��O{���z����ee�p�[[������Q*��Xe�*h�A(�������s���f�6�p4�G��i��$�u�fR�(U,d�F��h���q�"�n��������uq#���������K'�29�
��o���}+,G��<���S�~Gm��J`"l$V<M�Qsrgi��A��*y��b�$
��{a.�g��7fD�~#L^
�_6J��%�|�g���Q�R\���������s���A�2\�<��a;�i�t}���r�������Erq2�������wk�l��j���r�ra���*��oYsPB-�o����`���%`���c�>,:�yN��VB`^��V/D`�wy/�<�[����Vi�����������K������M��Q8�i��0�5[�y}F��lb>�����k��V���Cwp��1 leM�Y��Y`�li
'���s
�����f$��L���n�������f&@?�w�#Y}����R��8q}&z&^�n/K=Vz(�����p�g�n�H:�FGM�#�57s.�����n2;w��W��s���~$^.)���~��c��\�����0�
�aZ��n����q���/y~��������t��&~D���n8E����lP��C�QSOqQc��b��f���Uc���w�`�\~����\�|���������Y���T���YA�����������g�t��R�����w3�t�[r}"f_|�WO,u>c�/��6�$�%5�]J��q���t��P�6���$TG34m.k)K����5�u�-�������{rNq������GR����I����(����TT�����uu�ZO0�ON�dT�^����F�8Tj~D�+�}����31�>N������+*���������n~���|G�NCt��4Q3���f��%�3T]e������sK3y�hT;�T0�_�����$�W�C�
^���%�����Y�8~]'�-�4H.|R�C����I2�_
�M�Y��9I�{ib�l���@4��q��e������,��@U�Qm��k���N��O���_����o���^1��Z������������5Xv������
<pD�?%��o7�wDZ���d>hl$�=\z7�u���������.���S�J���8q�1��mJ��&�x
	Y�#�=��a�c�Ha�}�������$sy.��%(6��%������#cl'�CE3��6�r���lO�0s�RA].^uu���9�e����[����
d�R�����1�
����r3��Xt�LBu$��������}��)�L~������i[b��*��P����g�����C�$
6o�W7�%���6���F���~��I�^}����md��)6Aj���g������Q��a��*�y��
�&��
{��+�A��������}�4/����{���.�+�Eq����:!���sa?�C���H��kP���Sl��t�{���������<
�
�O����=bul=�B,���!�d�*e����3J��<����g,�����]���G�>$6�������~t�^#P���\��Y����|�[u-	������%�]�J������hRo���~����8�p��lxW����������Vn��7j.c�%R�;lw�x,�$�S����gE�S��k]���g��/��^����4uS,9��,�h_�_�(9�����z� �.�������:.aI(
�&>��}2uj���r�'gX
�y��cm��#?23������q4�
� Wt^9�%��9�/6�z�Y@Z_��J��)_L��"[���<Y����;�v
�����\��2c�. ��F����ly�b%>#�6�b�����@� ��6�\��$�fY�+f�a��]7��h����oa�&����2�#��&H�R����������
����|)4QU|�_8f��o����l���=1����)8����R�����w($�#��6a3��0���W�����4b]=�����F���H;��4��p0ki>Q�k-p:���YxF�����SYV�kP!!�,6:�����t�H9����7:1@O�a�E-���'�#����_4�a��}��DS</_�W�-�*���jQJ�'hfv*l�5�/������W��v�I��N�Loe{2x`{=N=����R�%�x�z4��X������Q���7�1���B}`�Us��9l�S{��.�eXG�qS�y�`��>�j&1��:K�Y�W�8!�\�X���$~�w�������I!49�pd�������{zhC��}ce�D=�A�5�N�v��
�Qs�[��br�n����Z<I���I�<�o<��Ao4������*Y����$6�G�}�
�������fO%��H;��(n�33�z{�������L�n�������t9��Y�z9�h�8`6!����r&D��&�sq�������g^�/���ST��S���M�j>������):���9��dq3M���i
aY����'����tnkc�:WL����W,Sx�G�;y�����n�^R�Y�Ce��r���\6g�ja�I����q��&k�����IC�8��>�m����a7���%yW�f��%31e���
�%V�vS\���t����s��/�k5���L�rBk��!��p��������<2��a�C�����Za�d�����o����M��������`�zJ��f�Y�i����	��(�e�v��m�B�\:�)u���,�P%TZcg��k�k��u
���",�l{<P�y�P�y|v�� 3�Ja�v��=���C�?���c+��W���F(�	i�������E&asH��MX�����e:!O"����d�
�����[�6�~��ur1ncFFZH��5Sg�o����8��`qQO�� ��
^���>�������S�h�������7�76����0�	6_��h��!�D��V�-]3�(�yeE�*X������S=��p�#����No�_
K���8����@o{���b����F�<&�l%t����S�w�J�U~�[����}h3��(���K���BM0��cr�(��%��fw@��O��S�
C�n�I�n�2a�e�p�b:�jL\�&d���c�}m�TC�,�`T�n9�*�5�5����z��jO�e�o���#C���U�*�����lf�SA��S���[�z��KQ����(���v�}���a<�)����!0���DE=��{*�)��Q4�I��P�C�V��������:m3}�i������4��$&,�Y�5�.���6,�T�!���6��H�( :U��k�Tb@hH<�b�Ye�4e�����F���yHx-;��,��15�}��r��)���t�����.�����iZ1�T����������'}A_^���dVcO�I��f�7d�u�W�v�n&���x���
%6���K�|�U�HZ�7���T���(#�c����h�b���m��L���$������x �s+gNQ6;�&���_
@i�+������i&g���EJ`���p�~=(
�~>�����������]9^�"����1d��,�15%b�����i��
�3<o�&�/��X����������{Q�������qu���c~�O��g�6>8$"���[������w��.=j�?���	�����m���L��pZi��+N��'P� ]�}���}�Y�;��q�p�@W��P:a����K����NR\?�j"�u��1 &A���4~���:��J������8��Hz���$�3�(�8���ci2���/0�	pH����O�v(����S�R��jC�)355y����j(�	���g�Y�S����"���e�>U����1�J����PK���V%QjE�I0TestArtifacts/test-case 3 -T5 WAL ~512 bytes.png�ZeT��7P@��a��.�niX\�{iIA�n��X��$�������X�.��=�~�_�9��9�����<�����7�
��(�=x�[IQV����+�=�S�?��d��=P��~�
������l���M/>���4��%(�uZ����|�C�������s��8x��DqOX�������*qyK���-<��PY�[�����x��]s�xSU�����g���A�����N��������0KQaa[4�b��F'����F������4(����b���<y.��������.�u������}| ����Nz|T'Zv�?�Q/���!��
����?C�@��q[�l���{1��������'��| '�������oBw�{>#��(���F���1����0>"Z�k;��g�+�B�� \4f��\ko��p��]��ER��I�Y7��9�k�������8�/��P�R�J*�o	S�c�A#�-
��bZ����4��K���M.�h�D���0/����-5�aN3�Lc����fP���sd��Q
�y
�{G������[����V-0F�f>�:6b%$��D�������
"��p=$��IKF�Ky W�7�-������������Us�>�SM��;�����i���}�����X������_���|��*W����@��sQ������	G�����e{.m��_�PlU����`���1��#*�V~p���B��O~�8~ yRkz���ZZ�����epg�Q��Y�v��������uQ���O�������<&}cn���P���z����a�3�nk\��C���+��'X\j���?��X��%�����k12b��jJ��jUPR�U}���a���%"4FV�� � ��r�f�ta����.����h���q�����Iz��
mf��c�W���g���q��L�a��f��q��u��;J_�0���K2P���Q�rF�)���(��?����2V:7=�i��/Hf-�+�<c����l2�J2��������f���~���o��LI9�_q��	���j%B=�q��e9�
M��Jy�vWa(i����l�2oV���R|�HG��h��S
���&��1�[��W^1���0�\��_y�J��Z9t����.+�tT!�����E�����j���O�
?V�����Q\����R��<i��#����{��b�8�lH,����~@�#��+�kZ_<J8�����U0
�������V�D��h�B��
���JE��1��WV���x*������\�o�����$3���g8+�������_����&���
so��97�����BPy������tt��G�.�S0�Rqn3�
J��<o%=����{���bx��M��'Q���Xo#��4]2����.�Q�x�d�k=��$�+�����9�a����7x���^��F������z��Q����p�l�X�����+����{����/-&I@gN��.��s���9H����i�����1�(�\]��n3K�����!��gc�a�M��jd��?����8�{/x��������9h5��p�%����-'Y��wId���� �8�?� ���C�{�u�^<o�����^���t����I�{����e}����-�\p��4����0����V7	����Y
�����SeF)����LH�b�1Y������������9IJ�fUW��`������f�9���M{�
�	�|��fL�>E�j�:J����?����7e��v�%��-z��r���?N���R�X���v����~A�t30�O�p�V��9���7��~���_�68M��a�gm���W��T�#"1�OBF��F����DN�Q��Y�z��|}��v���	Ts�������O*�&���A���`/�T�=�b��S�o��pi��//�dj{�:��@��l����wZ�K��5n�~u��*GB���eq�L����
�9�Q{pd���W���'�(���s~(%�)X�%�
�k���2��b�z��JsA�R���E��i������dp=��C��<�i�qN��%Z_A��+c��S������{��ocU	�@��gv��ogY������?��������w*�:'�q �p�����T"���'?>IB�|�VcN�����9��i��?�hX��<b!A{z�����tx���>5��|���]/��WVD������mwA���s��V��Y�
i��s(�r�����b"��Qb�]m$�����eS�-�""c�^�l����t\���V�{���$�{�'�����l�&-d��>�U���,]*�>j���S4��*7a&�i�rt�HP!+S������Te�o�:���Nk�������\q*���p0^���~��lV�����{<D���-�����>NI�S�sf��Il��y�1d}��T@�S�
�}6A����-}��B��]���`����n�]5_S#o\w��cy��%����4���c��}���]�������9��*�G�u�f^8���u���������	�)'�O"-����^9�����'r7�t�\<f��U��^9u���9�����dp��)��#vH��c8������~�Jr$��-��g��c`J��IU��nP��<�80������ZN����������u�	�@Ou1�
�ij����X%��p�*��-�8X������3��7=�$��em�E�w�D{%��8L��%=�p:����j����_*?�&�_��#�:��x�A�ad����f�M(!����)��r5k�?-|����k����n���L+�8�-������_x2j���)�D����_���`H�������i�j5�������~����4���'zr���C��R{I��&L��{�q7�b6U�-'�C�~�OR����?�s�b]�����b���[��A}��v,�4+����6M`{�H����,��V���:�|wz�4<M�{ST���6#gI�0��b�k��m[m�q���'���MB��_��[��^�N>�c�����T?�!��w�_�-�M)+��P��
��bvn9P�4������K��6�������H��f��y����lH�k�z�j;M���x�n����1�x�|�vEJS����lAy��a���SD&[Xn|5yc�3��g�B&��|��=����Z��IAPy=�d3���hc��>�O�����C��r~l����I������\o��#�=b/�b��\�%�z���R�-����.}(�Zz�T\U�^����2%��������I���S��=���~�Ob�hTnT��o8�5q��t��La���;��h��\�3������vZ%��\����]:{��~�0�h����7WR1���"��b�{���[?���r=0�|G�t{]2�8a���n"����LL?��HUMK+��$���J��O^F��#�04���q���o��G������W��7U{�W���,��T�����w��s�Yp�bK�]��ad`��������x}j8�3[\��������AE�@H�ZN��8�3�\�����%?l}�}`���Q*�<,�I�+�1f��y�#U����r@���������{%7E�f��a@�)j=��|2�G�	������ya�F�@����~'��?���1Fu���e�e�JF>��&���|������!�P���3�j1J��\~
��p��AK���
|��[G��@Ce�tL	�7����&G����n������������S;e������o����8W��%�M��b�O~-���������*h���E������T���>;�b0�;�	��6�PfF�����*�]C�6T���	�����Mk�{v�8F�"�K&�����H��U����gU��x���D�XFPD�y�������	��"��)�.���F0�����Wu�����q��u��<�@��%�W�g���$�'��
��j�����	5S�csx�v��c�C
���+�xF��r9}r4��H��o�����:&Qi� ,�A�=d��'e�D����[*7����_i�[�*z>k�����s����J^���/�+g���tb��{^!��-o���h@��������O�<���MO��R��g�{l��������}�n���-g���g�9p@)�\D�l��@�o'� ��Cc���"X�feeLa�#����(��F�]����?!v���D�����������|�����H���G��AMqV^a�8��Y�9�%I.������p���O�P��=�9��b��c��
�A�ci$J2@�0�������\�������Ak)"�1�=��iy�O�%��}�y������b��vN'+�2�1��W�DL|��z�W�_Un_��E����U�
��Al��?�4���P�N�e+�%&U���A���0�'� �������Q}���Yn����6��G.�����\�o"ki*��|
��X��c�8�0��J�,e#=
�������q�-�(����$�?����~�|���t��z�mb�l?�.�X.�w�W�[�$2(%�����J�F�$q�H�	��I6�g��e"���e
�����J8�pH���'���&vR���������:�v��u��q�j��t���s�������i�v������N�x�C���������syF��g�8��g>��+�lot
�
'~x��}S������\�d��q�@G�������D��[�i���zI�9�9�P���f����R�T��e��C��g����9���H�0����h�����`����K�M��A�M�zv�(�|;N>S��f��f����)��I$����VC-������V��Fz�+�)}WN��iVQa6�)^��O]d��&��C���)����C���q���)����P51B�w2��=����I��z��
�MS����N�X�����0���pq���b��c��/����6�cu'�~#�1f[�WV����F��7�`�1N������[{����sQc��x����k��T87���1{���������[\��E~_~)�'�D��c��udK��vBZ�f����q�����"����'���h����`�Q��y�d��� ��������=j^Nm?�������Lb��7d�K����O>g��j�����r�(���u�r�����K�#c4���Mx���2v���X,=�4�<�n�{����i��"
TsU=��`<4��|�����:��,���V�h&�>��Ojh]��s��'N
YW�O��r��2~{����,p�����U[�x���4}�%B�3����s�������3�����'DMFv����-I?�to�)Y>�7��m��o������/;�����jN�@S���K��Q�fz�����Hw\�k�)�ynM�:1���s;�������\O�����jx����N�Y���8�u��Lj��"w�`F�	z^_�
8����g�
�m��;�������p6��uD��g>d��B��Yz�����O�C�oK��QLX�g���kC+.��m���n����]��w�������8[�Ds��������T��wMJ������d5w�����\L�C��,��O#ox����	v������2�>BJ?�
������J�����"�DDu�6%��������0��V
<�����)���_��hn&�>�ut��u_�b0�(8�����}K(����'�t�W���qs����Ah�I����Bw��`	����J�/�@&�~�\�(�C�Z�c���,$v}�U�'�>��"���t��z���A�������a�j;f����*���P�>Q��;my�k���?��� �;��6o;U���3�U�������R�����?��iJ���m�e�V=�f�qv*� �^������L�*|�$��BU��P��0B�xc4� ��Y��dT�>D0�.��_O���4�&UA:QJ��}�V���5�C-MN��|��k�6)%R���WdA��!X_���?-K	��d��x������������1���+j��(�������g��c����k|���y0�a���1W��Fr���&���,q"�[�)WY�G��U{�����^�����-���������T�&�w�v�����+Z�VL����������O6�L_"��h�����'T7#�k�3��~��4���p�2\�*����H����������c{t�mcd�h�Uzo������c�g�k���e4pV!\^�<��GO�2�����o0`�w����]��P6i�81*yq�XWr%�
��8p����]������%`��,�������Yd�eJ����TR�q�k�o����?4af6My�U��
���V_�i��E[/�}� Y�1���44��_������h���U�g��kU�1T��
g��59$|��x�����3���hA�@h-/�����W��'�Y3����>>��%V��K�0fV��6�8���||T������l&��nb/�RJ�����~B-�����r^}�-b����1p�q��7��4���t0n��D��8(`cWf��s`j������zo(�lqG��,�a�HU���
��,���7����g�lB���H7L�8�?p	%ex���j�C�q�$Fgb^�"�-���)�����
�nf	����z���	8��q}|�������P�W��^���9���P�S���}k�j�9��iO���4�!��z�5�+W��O	�D��0����������X�,Q��\�����p20���!z���SF8)1n���w��u�T��9�M��\2��q��5uG��zZ"X������5�a�{lY�������rK���[�{��
�Y�dt
���������H`��Q��\��/�-LLC��}�
��+2#�p�`�@_"S��w�wt�<��c8�";�W��T(�����@Z��3����3;Yb���T��\�g~����}(gSI���/8��#6G{�Kn����dP���t����h�[�Ez��S}
WJ��[��J&�d(���L�v�=�F�r�.������9P����P�����	_���,�Nu�\��LoZ}�iw����FU0�i�]��y�NS��<6���%_G�|nU��K���s�u�3�w!��R^���m	
�Bf�������$zL���_�Oc�HGU��!U�<f<�t�|�u�<�������	��JM�5Z�z�9�t��P���BtT� Ps�>���T"
;�5M����:X�Qx���p�#d:�::[
�b��/H{h�~yy=�T�u��^�is1���c�.�v/2](�!�����hgi6.�
�����U�-4��������1L!�D�������_����oV��d��6e3��z�>��}]��vZS&��fw��m��Fg����r��1��sUAZF���e�,��(����)���6F��{^����/6��O#]�g�}��($��PQ>T.����� ��p��'����p+��P���h��A�_�n�U)�=�������S8��Y,�����I<2	����"naym�1v�P����e��W�������'&�������^r��R�D5������,Zl��|��c7��x'%	"fc���T�i�����}��������
���0>c�]��� ��3��0c�g�3��]�vm��)��i�5R�6�������%
8�#M��fv������>��(j!%r��>����l�I�D�$�D���~��UlY��=��v���Af{�g���rk�
ei9y2�1��Y��!��!hd�fb$�7�rr�C���]�UA�e��cEmZb����7mi�������[o��]��K�$���?�};�'�0�'����Q���)��a�w��
c#�?t�{���7�d-��|f���1�3��E��Q�oe�]���.n�Y��t�������d�F�����k�����\��H�������W�*{_���}h8�u4'v.c!#��}z�IX���!�
������R�o�FS��q)�`�K��<Z����y�{�7q���|A��2�FM�*&�\{CJn'�����
L�B��k��$����F3c#��K\,�k1c���R;t}��>��P�{�PU���5O�Z/���������+��/�f"aZ�[��YD���S6�=S�s�����s��wK����;^D5q��&�����U.XDh��O���������	�s~�/~b�,�moW0U_���� w�8����q�}�('|����J���8�\~�����$,����nof�������+-@D����mkj@�"7��{s�lV����?�� VK��6O�z�{hH��������Ap0ya���������0�{��9C
I�EY�XP|�s���<�7���Aw�����l0~'_(*$�%���Jk'�(7��}���t	�`snWr^b!��Zp�AC���H8�(�Ksj;�#���<�
m����b����6��^2� 6Y3��)�m�}Z3I|q��U�~J;}q�oxp����PG����V��X�i��N2Z�N��e�*
�����^;���Z)K�W�����2�Y������W��P]��r��jD���_KV�������gYSM�AxZ��I^_O�l~}��C5?'�����l�{9��v����:�c�}�2��gJ�Y��vG���"r�`�S��J��CW\2�?����|7���?�X���pu)���r_��=��X�������'b�
�����$����'������p���r��`��E�P�N�$I���\v��U�S��������'_\�G%`�Gj��sN/���fw�7�n-7�����|PM(O�v��868I$�_PY�Y$�RW�u���rlr_O���3��z_L��w+�aI&�L�������D����X���^�������h+��%��@���E�w��[uc����6�;�����0��+F������}
���q�=�}5Cb��h�����[}/nP��RITE��_A��&kv����Uk1PU�����9�����S��h�PrNa��_�T��C>�\���}����n�����m���Z�}��V�}a������{� ��{�������>XfX����8�%�5[��)<���� L�hbMAE��/*�g�k�
�k��Tl�>d(��*����M��e�����-x�O��m�,���Jf�K��{��8j�]V����^��l�X�
o��En�GS�6�+g�POm�������2���.[����!��\��SV�K�.�@������8WQ�Q�G���!{'�V��Hu�����Orug���R]�e%s���{�LaI���E8����d�x�������������?r$��.��_�r�!��u���u��%�w�!�@&W�i���_xmf$�P�!��:l	d�px�
/C[5y_"���"�G���\�)c�vG��z�E6�2�����SnU���.Y�yMx�j��I���4��������������("r���G�wIr9��k	��:-5(d�����Y�r��_q�����r�~����E.}���s����wp�@2�$�y������>/2<le��J�h��~� �&lf���ZOwSC��������i�@V��v,#�;*�jXh���Q���9���C�B������\��y��.����2��?�qP�g��N{�^tn�t��c{�P��'��VV�������E����P9M��s�~_u~*m��z����~�*�����Oz���4�{]�'�?�a"���$
�������`�98�A�4n�����>K��Q��Ss��v��O&�>�-����Ov�5��q4r��F����
���.�����R{]���X�!��S�>�����;��Qa�6�{j�������W����Md2��\/S;`��cy"G��!��6�$�/����8`�1�]��3����%\b�Y�B�xv)�}��>����i��R%���,:8��y�;����"-��GsyzS�c�/��}�j�f����%]�Z6�g�R2���z?*�uV��!8b7����*���)���1�VQxHbz
_��xI%c�=l��A��j�e�����1Rsi�e\5}�>o���B7>�n��� D�����������-C�K�[
��S����-�f|�������'0����O��[�p��r�K8�<����E�v�$e:�qw2�Tk=eSQ��A���G�9}n�GE��E-��!��I�^NO��J;���wVz��1����DC K���eO/�Ct���b�Y����� �$X�	��'Q<��
6������D)S�Uh��n �?9�-�-����Ls�?7Sz5k��1���z@i*-c�[�e���[���oP]�:�q�����CT��� a����b��A��N�gq��	���o�U������P���!mM�R�����
q�w5+��]��RW�@���@�,��l�
\	��NStak_3%����*x������������"_�Hi��NrQ���k��=�H';U2��q,���5�	8K$���I�L���>����p�W��D���x��/�S��V���J�(x�\����t�6��������������4p��t��!"@G�<�^��?��j�L��r{w_!S:k�i��y
���Q��L�u4
7�Wu�,�����zn}�.�O����8*� ����X�!��O,��;��=�py��@�|[c]*r
��p�1k���r3��R��K�:F�"��mn)u�7R��|��p��b^�ly��#k�s<�����!GrvM��w�{]y������<������dg��l���N�g&Gj�$��P���9)�I	����& Q;�6i�>�B��4�1�m9����h��	�A����\�9CqC��5�Z�9��?��M�X��s�>B;0�w?�"��x��������~����1��	�?g����W�0�u8����{��_���g��=��<��>i|7�����e�Gn�Nh6�/#
����f�c�~��������/I��\����1.���T':F�1��	��Y�H�����2��42�T��y�7Jr�B�����������u�r�����H.��	��K�5K�%���C���^��C���s�F��)��8�Z��s�U�:��6�78�������%�C ���^1t������G5�T��3���@���X���a��dUv�[��M����+������T����v_���E+4p���� �9>��t�\��G~�|������U���6�DFs�De����]��[,Y�s��.��]m�NR���5'��O��������&\���R��F�%����^����z �{�]L}�s����sW�MHib�T
��*��q��hBj�(m�L>����q�R8��;?�
�>�j�7��J���5X�\���G�����.��(��(A��e�kt�����v�������i�3�����<�<���i����Z���1`��|��E�����v���:��R��FL����w�����3}�b��3��D���nE<"$���-��*�����d��`;g���}G]�N���������)R�0����u��������_�`
t04����W�V��FS�!����($�[���fq�`�w)�@Bp'�S�Yl!�Cp�>��9��=���w�����y��|��&9�A%/�s���$��M�X)@k�[������W����q�GH�N-�{^*j�K��������|G��������i��}xl	y�D����5�K����7,m�+Fb3\�_�nn��B�.W����^������T_6a�����/����l�sx�$���=X�
�EfP��x�����Jz��
���P�s��v��Ad_l=�� �����+y�E5��>\t�n���l�?��?f]- �T���*��;�C��2�M'���!
�?�+��v��x�1C't	��i
n|���{��Y�r�k4}�����d�f��:>���E���R�}G-g���)������qwX��~��)��|����MH��O�8r)���Ii�`VS��������+5ux��h
v�(���i��<i��c�L2��o����������j;f��Ej��?�i�/�UWBpB���V���	Y�{��Q��f���
�q<���0�<�Z�}G���6e����9�����8�v]L�-;:���,�l��9��_<�d���s��W\��v����������j��;p.L��)��N���a�����B��&��G���^�0�z�'%<~\���5D\T���Z�}�k�cS}m{R���z�1Sd���~)j�+��O�hi����p���A�<H���%�s��'$ [|��"�������K�.'��J����Z�����4�����g��R@F�@�g�k�f�����T�Kf������+���a
�x6����V.��9�?j���`�|�
-��#�����m��z�9��HC������~�t��f����������bR����<E�����S9�-f�g&�,��w���ek7�yvH���p��Z�O���������]�c�kD,�ED��UV+I)�������,G�h���Z�C��V�C�c%��QD����go>,��/��G>��
�-��{��*;��])OxE����@J_�\(�b��� m�p��� x<�!7po���yZ������O�:4n>i�jW��By��B4��L1���MF�v��I��e���c.����
�h�[�}�����F�E����U|��b������r5�����t/�C�Q����	�j���~�����P�����s�7��{�VW~�	���w��!���G[�������"{K{�<�����?�<S��3
����C������������_�������:o������U�� i/V�`y����Y/�U��K+���+m/�L#�����\1��^�t������=����$	?���l�\4pu����������'���T��-��r�r���/������������C<�C]��t!�1��>�(+�@�����'�1��$�\<'���Rf�1���0!-_������8��C��1f<o�B��y�\.���L�� ��ny#�56����6#��������s��PT����3'��f]��VU��$0I_�h(��5R�k{�Ye�=/���?�:����X�azx!����c�����}��^���5��i7��TP��;��q��IX���k��@��oX �eZ������C|P�?��`C�c�����~������#������� �w'u8
�sm������E��Y�Z��p�du�r��W�(c\
��E���	N
��$t�=�Q2�5��
��`�{����s�!�8N�a�yl�:�[����7��8����w@���L��F<��s�'i`��E���"~�����������=��d��n�(������������� �i���i����5���T9[[~)������,����������V�^��9TwP���5���_�������G���������c�z�>��a@���bc������1�+o�u������7-�[c!`��tyGy��	h���Y�s�cl�O���k���0A�V�}^jt�P#�mqg�k4�sr*���/w�@����.2����BKc�2�1
�S�idjTg5$PkQ3��z
OeZA�x�2?�Kx��3����2��������[���	�:��r����	N!�yY_1���:���txi��mq��	��  F(�uG�Y�s����QD�BH	�q������u����LZ����|o^�����k}����(d���� 5u���G.�n�d�h�wm�^�/��t�v\��.L6��y){D�3��������8�{�E�����	�����7[��,/@/1�^�<[w�+5:�9���
M=���H5j`(�*%��D�S�����v����n�^d%�I]�����MU����F����{�������>!V��FY�tCw��;5OF>����F�U�a��A��1�6��@M���Y`���'�G�?C8�"�����5}?*�rdM�����b������@JTw	�FS���hUHg��vu�X���&�;;����-|N��:mi�D~���V���s�E�����WY����B�����>�����N>���$���MZJ�����w:S����F�SD�����x.��2������*��v�9$l%4���u����;��egT�!�*�dQaF
�����+8r�W�@���Z�B�4�����y����=A$w�{����>�@D�.`�:�X�?S�PS��1��M{^^�IoF)�@���	����c�x
�G�	���u��S�����d��?�Z����X����-0���������T���_,������YTwmBM�e��6�N���@�NNlOw~J"�eNu�������m�������Y���
Vr��n�������@17�eVA��jV�'q�A</yD7�
#
I�8��\C��IH����yn�y���d:�My���0��WZl%�4b��c���@%S���$�.R�����bN�wM!�&�����?����W��?�}B$I'�H�u���)�����5�]�"Q��@B������F{�e%�6k��S������Q����������`����/������t���QaCe�
=A��{���?S�w�!J$�H=��;�\�R,Uj=��`mm��c�xr"��Z��D��FH/=R���9������Kn�a����K��be�'�SJ���o��*�&��"�o�t���l+_?s�����=���Rc�Z�?tF�2��a�.���j�*+aA��O�xR��HL	g:��v�����iZn`�)��[����������V<�M�w+9��b]�N����m���c�j������ d���7R�B��fs=�:5��}����&-|2��G��
���PPQ���Z����[U.�9���s&��a%�����G�+/�����~~�Y�������?�:������\���\������9/e�K��4�����r�0$���~������&�2�qYI	#(�:�
�,�3�"�M��NSo,�1N���D����wP�9/�![`V��X��!y���Ws�Q����T���uT��F|��y�q�2+��\6��K�;�5X:!�N2���X�;�����6=����f�'H�s�2m����=��|�3����ej�McPl��!��a%�����G`������ ���_�|��e����&%��4��I������Y����K)�:��F(
��2<���`����A�A��M��c&�F�4��;KZ�>l���n
u�I$E��T
k�����#�9Lo���l�+-��������6�NJK}q��!��!�kH�b�f�L��u�����o��tBd&pL���B��8j�vqXf�����T����C���w���9l/���k�`@��f�$X�H/f*=�M���c���8V1����L���`��`Sh����r�NV��������B�0S/{�I�F���}E��fjh���ay7���%7���e'��y�"}�d~-bV���Z�X�I���@��2
����jO��n}�*7H+H0Y��#^!S]u���}�B����g]���v����U^z��)�b5��7����0;fI%
�����2��7��y��t��-XU�u����v����g��9�1�!��<�q�TarJU�	�Wo6@$�L}���T��4#�s?��O���pVq�����������x����G~/QArH9��y������t�6�z����	��{����@���.K���<��o`�Y����u,�=�V/:o:�L���q�47�jvwa~17���&\�3[����Z.�c���h4	l�����J$��(���;+�M�����7h9�_�����FG�efZ���}��m���~�V�))�D3��/�{7���ge%�u�������X��������b;�v��?c�[W8��l���y��p��
�EV��9��?�����J?6|p�E|���odl�{�Q���K�#'��R�M���~��D�.�(��#��E�\��]0�����";�[��q�[cY�e��3A��,�0���t��!1�����q��6o"�*��������9IeN�H��	u�c���)����Xv=��$��J����p> �9��s��U�@)w�U
���������t��^���p���\R��e!�i�;���R��W�B��f>&Y�O�x��]9#��b�ab`E�-Y��c8�&�{i����t�������e�+�[�.�f`�)PGv�U���V�_��96K�A|�R`��.��Gjr�u5�c�}I���W���O��� ��=���I��
���M��)������H,,i<��<���R���M������t9[������i�"�
|��� ��N�B$����:���s*��9���D����&���������K|Ue.�����e��w-��6�c0� �KX~��'K���Vg����<.�]�{��`���hF�'�@5�����d��)�8
����L��������~j�,~���hC�b�}���5������],	�9@?�#W��e�M�6(S����R2.��}�,c���������&�J��O�\�!Dy��g|�T��`/:����+��
��y�Tn�b��:!}��G�&��B��PHG��g�}�&~�qC;_���?3���q}h_�����������!?�����Md��P.+��R^(�-���ue,ul�k�D81�O)^�ev��]�Mb��'�����b���oq�o�����i�*�"���8��ua?&PB�O0���g�}g���f��m�/�J�KW����'���Z�[���MD��t;
��������c��!nJ:d�2���W8��O�%K����Qi<�f)�t")E�:uzo������
k�Vo"A���J������<>���K�k�����cI3�I
v�nL�7>t��7.Yn��<�V��9���;���]f ������jk�����W�����B��j(���^��
�����������R��7����a1�Z�(5
LyJ?�g��p�<��W�[krY�q�����Q�mof��
*~�����*��,���3�
)A~�&���"�L��������:���u���^�
�����;^[1�����l*���Ze�g]��b3���39�1��Y��U�f1E����^��2�9T7�?W�c�?F���G�<�;h��p��_i������o�������4fW�Y� �� -^d���U�cphl�f:�'y�;���$C1k�
���}��%!��J��'Y��$f�w��d��wWwG��l�����)�,����9��xFFr�]�5].%X_�>b�V�8l���V��8�_����}��u��}����S�c�-�������R���x��'PK���V�Qv�>>�B4TestArtifacts/test-case 4 -t1000 WAL ~1024 bytes.png�ZWT�&ex�H%��t�3��� ��4���!�  -]R
Jw�tw��������3���s�=�����VR����������=���@AA�>t�;�/2Y��((�(��OT]�[ixTju�����3��pA����^���`6����0����r���'�����w8\vQ�!���[7�*�K��a��v1�="��\}|o�y��V�67{�{6��M��O����6M�D{SgEVR��&]�_��%����n/�UU��)|oB�u���~>�EA��;�J%�������p�R���	���CP��$��������J�rsq{c~_PH*��g�?�d��M&# ��m���?�!�4(����}@&�"�8�C���U�F�
���j���bI�J����Ej�Ev��%CA������ t>��<��;�����g��f�k����KdM���Kp�7uX�"�@Yc%��D��1�>���~$(���W����g���R�Ae�_H$h���p���a��	K�S_���W�l?+�B]�"���M]�aG��o�7b,k�h�A�S�}i�������@P��dT�%�����]���d�2>(������G�W	?'����_k#a�#�2&�I]]�O*�e#�Yi�����g�����]���������<e>�7�N�gS�USc�<��>�>�7�0(�R�AW41�h*�Bpj������L��|����S����~k?SkWck�>iibCC���{�|%��%��g��}�1x��s�d�,�x]������4V�b��,�������`l��4:�G���!�ES����X�xy��S3����D7^���y�T�n�w>	������%$�_9�����#�eyj)1�
|z���<�M�P	eh�|����z5j
����c��l�O�~������&]���q���5�\�`�=��IX�!�{����:������`�4�F��9<	 iL�&�j�MG�����~'
�}S���Q} F�
��8Y���2O�(�������v�[rm`D���|��1{��zx&�.���|����V�^B�����t�h�fP��x�������;>>\��x��r���4
 ��|X��V������x�qkzb7ghX��sk'`�a<-�[3����vu��[�x��5��>�l3�8B�������@���Q�o�i?
�3j�;��l5��+U��
�Q�l�j��q[�p�z��a�6�]0mK�!����B�0��9���G�l��z>����#�_�]���b��vT2�S_~�x��_i[$�}+�����7>���������{��w#v��%<
Q����Z��M[��:jkV��V�C�r����o���F�)c����A6��:�0����$�"����3o����_��x���F-�X���pBfuq��~�f�V����$���V���Z^Fnaaa����Y2h��wQ�m����y�c�.�`�C�I����,��v�����J����H?�I�<��L?��a���B�o�H���g3n�c!�?�C�����aj�E�?���c$��g"zBs��T����5
4��~=g5E����)Iy6g��J�.���9���&&�V:<�h!��0��]&*U��P�A���<��f��%8�[w+�L���c��g��>M
�I���������(�(q!�
�oL@��-�IyK6t�_:d��{����������'�c��|��������N[��&fg?���<P��`���B�,4�����u��`����������1�������q�t��-��Ai��/�d����_�	�~OXW�{�L���R�@�7���p��^�H�.����Z
g����S����G��g�C�Sc�"�!D����f�x���d����$��'<O��[mAu\gI^4�����T�RM��Q,�|�p��
Z�S!c�P�$z�.�g�9���U�!���G�yE�Ks�Oy���,SK��y%w���;���������A��7�����q���Jg�6,����hu��,�P��������iO$i�x�����H����3a`k�������d����h]�c��ew�w��T�|_G����LE<�x�pg��f�A#���>S~n����U-������'M��X`�;<M�?)��p3�j��A�!��lP�_L\�nU�`fi��UB
��8R�������!�~6���
09�}:)i�g�����~���mcX�nj�����5�~��hD�:�-u�'�wwF4���eK����k�LJaQ������a�4M��V+~�ICgg�����MR����u�z�R"���i��2�����W�.|��Yg�%�5�) A��&K`�m�]j�����f���w��6 ]����]�
����d��Y�9z�RxK&^�9]�9�&�D�������eb���P��J5U�|�J/b���k�Dc�y�]L��J����e5U�)���,Cf4]*Nv�������QF�����[�G��+~f+dw�k���4o���O���P�D�)������>X�g��{{���E}l�Z��F��EQ�x�bJ<��v
����!�V��mh�R~��u	��+��]���4(��7���V�������q��#�Z�}u�+L��bc[���"�����B	�WZ��U��j����<�kn�w�k�j��^�����\�(�*�������/O�f��to�H��5}�����Gj,���n�yFE�N�I�����Q�kcS���:���:��STRdc�'E�����
�-X\,��@������������}��A#_�,�;L���c�r��_��(L@���E,?8(��p�N��K�4x��,�`n��xn�(.�����z(C�-��o��+����"$��
�%���y��O�����O���Y����F,����M�qV'�}B�����������O]�G.���@��k��nrf����������;g��r��q W�JH[G�zS�����M�rm"��#����?/�a��w��?2���	��_��/���~���@��eS$����R4������A�P�����FX:�
k��a�5��]��S�������xi��*!]%��.���HHl:���L����,kEd��P,����#r{,*������mj�w�3�:/Q�=�&.������$OzV%�����tP�KJj�'W��K�4��� `�&�F:H�""����b?��������/� ���D�1����a�i	W��Y�7��f����H���gj��P��&�r�]L=z�����h�E��fM��r#����<o}X�$Z���=�w}Oj\�������r��~�iTe�:� ����6�)�6�ZaY,D���e4=:HW����/i,��������g�eh������kq��6�N��Kw��R�2E/��VlD�:[&��?���C���XwYL�U0�}V�o��Q�:z�Z6U/�"� ��^���.[&W
[;��CY��7���Mz	�.��*H�e��,��A|R8�u�����S�1�w�$<S���=�2l��������m��xG5�7�C�����o��������4�'�
��^��d�p���������5qL6�D���8�;���Up�)�b������k(���Xi~
l��0��8�	qC�����>�a��h���,����k����hYQ]y1����`qxT�3n��7�FZ��Eo*���t���PO<��HN�tN��AU�������?�G(<�|��9k��B��s
�K�o���*#�5N��������~	�s�C���8`��� �%9Q�*98��x���j������ic���S�	����	���e"0�T1�?�3���:�����q]�B�GO�e5��_d�rR�����L��:����5�����	k�RaS���$=�����T������?�J�)��!w�YHe_G���3�}kd��b2���|�����AeA����<������7����K���X����+���D�M[�	�Tc��@�9a��H2��r�H�e����E����'��>_P�S��������s�T&|\g������T�����o-��3U�$���H�<�#�2��w����_T(����~q.�A��^��F�I�{�V�D����)Y�$�����J[h�����h�����
a�U�$�7���I��K�����tW��H8N�/U���Q����!��$����Y����*N T��JC�
w���b�t���g8.��*f�
�T�N������c�����xSu �|�U���*Z'h(�������)�� ��k��G�,��\��g:�[�E��7g�:�CJ:���=�'�x.�E��9��X�O�!������\*V\g�d��-��=����?�	md��Bw���FE�TV��MXx��nD������J��3�*R��hI_�;2i�!�(f��;*$
�^����c�&��R1i�F����x=�0�/��v!`�bYYkNO��|�����cOs^��nh�f�F,HCc2�l��kV`�=0:��x:�(�S�.��6=dC�o�`�����eh��hN>�>�vm����F�������j����	�b�On"�j��7p�CE���&�?O�y��q��b���ZM[q��"3./�/h��	(����U��.��pm��&j��
��@A���+�"n���F�����X� ]�nA�X�C�o���9���YT�6a�-Q�m���*T�oD���������[�G�}\M!���k�Q���!!e���b�8	[_�,(��q�R���V�
���O/��FDkJ14z�����{3-(����A���t�>a�/U�iRxz����*$���S|BD�����Y�F�u�x<?&����+�+���2���y\0�OV��g��3��\�P��.�$7u���.���2oA����B6?ph\���[N��$������������k���
7��!��0�<���w]���Q2�{/�Hz�&��H�J
��V���d���:�|��\��g�g>>eF�>�8�{Z��m.E��?�o1��Z�\�J�7|'���ZJ�F��k;N�k�p�Z6~u��8��:nA g���5����� r�A��L��
��hv�j"'z�d�HXo��*t�����5��FuT/�>��G��p�?.�
!��-8���(�h�v���m�Y�l��������D���M��X|p
^������5F�SP��y[z��Ce+�5+������E+�r���bS3C��z�Y�U� $��{����m$��}���%������e0h��o)�(�?[QM�t�y�1[~�Q)���g-�����`�cb�i��7��"o�ug����^	L�������p�����W���E�Pc�q����%��fO�njV���u��gAqm<t���9����bG��\�PR�S�M�+:0�����*��9 ��)����M������[���(���� ssD�'�����[[j&�)P���>��Eo�>5��C$��.�=��:c'e�X�8�WO��w�KrRZ�{iA�>BbS�s0v�(n-"��<W��,���yMA]�,�_��)����q`���u��o\D^:��i.
c��*.��d�L�^ca���.iVZ^R�+�(7b]w�i�%�G%��[����9������;���"��r)�;3�x��*t�89��i�r���q7�[���+rF{D���j�V�����&}K�-��Z(������l�����f�\l��������������H�b^w0v������A�9����r��&�IQTH��.{�3��kV��=����[q<�W�i�d�SJ�}<�[zs��CC�A��gqS����cg-�x)<uW��?���z��U��Q)@I:Vfp��~#����+�J�������&a_�^�n����f�`��,7��p��"W��v��P�>Kndz�QV#�n�"-R������x��ORL��q�B���'	�|8���y����M��q\�����/�k���x]�
�~�'S��i��]�F��}��;�@��J�+���r����?F�L��!`� �������d���T�$�?^*��(�*/�B��� R����0�v�ku�nN$\����8j����4:�h�����������Y��m�SkH��_K,m�����o
�����]���jz?^k�f*x<��`�v6��Y��o3���������Fd�.�N���s�/���x�>����K/d�we�Z"�s�����UI������������^&Qsn���<�J�����5�(�j/ �q���:,�������eS��g�>#?������21\�1�?A:b������FQ�������l��+����J��.�W���Jj2�^���/��7�f����e����S��
�>F��8LQ��%9)��_+h���/�����|X�l�2L=�7]j�4�e�k�S�\(u_F��W���"!���7��(s��1���7G`����������f�B�|
P�������������^���R���?Oh�0����M�0U5�\Buc��
���
��y�����M��/�b -��=|��{Z��K# �dG%�)^���������*C�xo�K��p�/��^�����_�k������4��|��j0��6���7�d���,����R��ah^�����mXK3���	&������R�n2��J}j&t�r�GNP���������Z��3�~��W����������Yq���7$D���u }-�����U�)�SI�p&�C=z�|��+�?#
����
��0�o��--�F��V��t���y=|�bq��bgkE*.s�!?�#���\�>.6g�����C��$�D��@��� �o�*'_����P�oj3��gh�F�4J�T ��
�=����
�d�����~�����.��������������{R�q���-��Yc����}������Y{����"<|n�������2��
X7�e8��{��B�A�ixy4�S�o�k
��Oc���k��%o�|�y:DP�j#&\v5,�@Ul_�gAx[(�o
q������{ �`�;\.q%l�t�V�3�r��FC��b�m~��t�R'�sn��K������x��S�wZi�������q�E�d�Ds��i�<T���`����'W�g��:�u��zp8�Gc��/
���OW������X��~�{EQcp��W�
k���3�����.[��N70n�
��H��"��)�~�"��F 7�C
�������$���VI��t������-��5�����w�����jU�M���o2��OXL�z�L��!���������l��g:M���
�>�����;4�ZK�����w�<����*��(y�L���(,y��������`��wU������Qzm���������B�]��`�����]p�����>���������]�=��5 &$+���j|�F��h�G^�~1�w��S�Rc�����n,�����%�K{�_�<4��uU�r����)��l9��,;������j;�dS�h�����HQ3��u?S�j��P����^��.�_�\	�bs�xD������#t�]�Yw	v�?J;������kp���2��!K������^���,�c�\\*�;��f���=9������=���(*/�V��v�g�{q����&������!i����Ks��j7&U��$+�Y1b���Kq��'[��RI�����������qo������{�vF�<�<
������H=����>_+���t�'1�z	�~��?
w�4v�g���b��z9��+��Y�������R�uk���#�yd�����+�$x4�*vwD�}��Y���3*��of);��<E�9?���$���88II`g��#��>��z]�9Qlm��#�P_66���}�^���Z-r��!9��>�3n�K&�<�r�j�p���l��)����LS�q�zn�$�n�����-X��=�(�c����3��V�!����u����D�i:�s��\%3.�-~C�s��L
6�/��j��*]S�����Sy���?�����5�� ��kX�&a��ds�)		IIW��Xx�TC��Gw��s�gR���/���D.�3��$jc�oA�>��ag����~`�j{�<ScF����=���W�:�
��a��!z�}>Y�r#��L{��=o9���G3�P���#�TT
RUR����s�6.A�BXA�'J����Cu�Q]�������)���$��7F����1����`�E���$���!0��S;���(��H��.��&��4(���5�G�����v�%!�41?I����B��5a��_"�"���
l� ��/�G�_�����{�vH��EH�y����& �,�	Y� RE��rx��Q5��7@����:@>�b9�O��H��K�v��b*�~�\
NIM*�?Z:�9}u?��!��l����K{�Qf��z!"���xC�������Z��DY�7|w��[U7�X�K�uE����{�_D��4����%7=!�Y;/o}t�yH��4Pk���Z����Z����<�O]��o���:I�^��c'��6,��V!���������`� �HK�������MV�c�d�y�:g�5M|��rY�
?.K�y�,����
?�s'i�X*�sR�^V9�[==��n~�l����R���}�W�S��<��}��h�Ex-4xw]�<�U���r��+��6V��!������Z��s��p
V����&�!�R(�C�V�?n#�w�I�{����b������2���M2��
!!yZe�D���{��h�����X�������Kr*���@�q& (���W��j�>�
�$0��N(Uo~�
���\g���~)H�(�|���r���]���g���\*\�H�d���ktA{��l|��p�t[��A��%���@������hD)@<�����g�9
f���F�b��6'<�o8���AX�	����6����^D�$rJ����[q+��S7������Q�x v@]Ww�3�����E�J���l�VL���+{�r��9~)z>���\@9�	P)>��'SGY�Z6s)<��>�����xE�n����2�2��`��������5-�5����s�D�%������#4�kh(�����t���w��?���4P���r�����Kii������b���N���TJ6r����3����LI	�x$/�:)=�@b.���@�Go��
b&Hn"�|p�"���?6seF����KE�	�C FZ%[�������F������Q����X���4�R1�{U
7���G+���=@</65����
 �Y�G�n�0x�u���r.�:��T��_�"�\��2*��fG��=������B�M��x{a�mGJ���=a<��g������=�vB��g;���������u���P������Pgo�$���2��kV�5"*���\��p_�K���R!"$�9���+��AkOm�C�$)g�q���g�~���j��Nr�/d��b�o��f�j���hSZ��ETM����@%=��2l����w��3�����r��
��q��P���������?��7��9h�]�3������S����JV�m��H�����j_���e�o�$F�]���/�]K6�s�T��U��,����:�-_�N��-��?�F��Q�[����C���6���@�G�^�t�P��9���:�yk�����
���J�H�e��Y�����+��J�*(u|o�^_��k6�Zny���?y;jj~`� ��������������P'z��bOxx��M���
�A:����#J�j|m��"P*=)2�q���0JDd���{n���(����-���p�� yW[���������k6�75$�P�����g����jZ���7"���P�&��I���)(E��uN���M"�}V���qg�����
�n:�h��U����h��GU�5��?X�
�&��DI���n5��=@���r��??}��
��
����m�`����3�����t�{0�d*(��v[ ���x[�c�X���&���1��5�U�m=������=����K;	[8Z�����.t��,�=��7�kJS�����l�}}���hEt5S�Z�������S���'A����HS��G�d��P�ou�����+��[�k�v2����]�x�A�t�|�,�!�B��{��%�����;,��;���a]��tj^�nzo�@�K����w�d���6A��W`�m�m51)�l?���N�x����
��OVP
=}�����]��{�LU��&���$�y'P�������w��_^���j�/��
�g��<��"��m��{�k]r���:��������|k	��s�oZY5���y9�/a�o
������A.���[(��������<��B=���'l�{-@��b$j�|g�O)�����{/zBwy��>���@�9}�+��v������VG�W�	�W^���s������2SU�r�M
��+9	7�I�?z����koe0��[6�a�f����w�����c�C��@~����5Jk��3���V�	�?o����h{�7����p� ,�v�R�7����@�2
r���WApGRLRK�����7�ZA���J�"��������������\'�T����M*���������Q"��I�t�:\~����<��4�wp/�L��i����#��1��|/N�������+��
&���A�Sc�E&��z��4��^���5q���=�ok��5�+b�7��@4cQ�T/��Qn����^�h�7��);l�K����:�x���H�,�U�,]iK�@�=�s�o���h\�$����T��O������N�����!��"c*2��io������
��F.ek8��:!. �]�>�U
��A;|��b_]��k�>���;g'��LO�z�
�������� ��h
j�'@�O&^b����Uv�V�?�5�.�� �~�/nU�J��y-F�h�6���\S���{�����w���������Fl�f��#7g�������_��+Q�`��|z�����h��Cyj�k��%��NS���*Q�MI38d��x�SZ
R��������[6��ik�*�	,���������u7>������1�+���-��O�*��"�OjlT(�������b����.pn
(�rU��R�A�y��g}.V�t�3��O���@K�h�[����@K��|�_|o[j����R����v�@K�J
9	�mK-�R��s���m���R����t�(
4�y8�J�f�)(5{n%5���Nw�{WQ�/|����4%��^>3ne\����t���X>�-��7�v+�yq���G���+J�<�����:$�r ~`UW_��d��=^�i���0����i��n���[v9����$x�.���,����I�*�;��Al[W�'&����IY�N���'���5���������\�3�^�?m����R��9>Yb�h���.Gjs�cU��\c��R�<^�a����5�tV*&%��O�bC��v�����^5�M�	�et���/b�i�
�-�i�������"��m{.�]'��n��A,�*��lO�^���b�����/��(�<�B�U{��L�������!���4��.��[f1�%�k#���n���
�Z��	���/��r+�r���,�}W��eg�%��b����0fl����R����fd���z��!;cII��V
�-�p��_�y��=g��=�s>���('>���Y�H^�z��G*S����e�&X����������T����H.�X4~v+u����]"�	�����&�L��*�Q��,3���l��Ww<�uX;c���%�(u����v�_((�����/��m���K�^4r;*f�,������i��0)�0f�X�DW��N�����do�R��-3�S�����GL��8��(�v��������48}�2qFeu7%4��
9 �� ���u+��hO�[`����'���w�{�E~���/mC����O��<�������	"nh������TI�d	�;��.|��{J��v	�=@)k�D*�p2M�sLFw�-O	��lyz�����B�Q���N���w�!���p
0!�<����0�b����~i&(Hv�D�P�p�=;m�_V�(�����
���s������R���='�����J��Q�+Y�K����z�}�Z���/��\@`���J6�����:53^r%�l��>8mmr*f#
�D��Yr%^@��������Tx�������)�S���|�����������U���������X5�z8������O�p3��c��&L��'�L�Ic�iR���>�p��:}'��U,������P�W%�r�T,�b�E�i1cw�(/����_�q��<x����>Df��y���
>��<2o}b�NW�i��T�_;-�[k�[���g��z����	O�t��4]�WP8�)��p�7�,�;)�-�k��j'K�_�we|t���V�6W�����6<�T�D��2*�?�8�t���x�m�����a�� ����Y�N���(��5���<����X�M?1���7�S)t�g>��)�����.W�����D��k���$D}(V�j s��V�@�.��/R�"�j9S��F���3��tr�	[&E�E��~"���P^�>\�1�>
3���������A+k'��-R�Bm����6��=��F��_9+
���m�c;���V:b�""��{�^����H;t�Q	2�n<����������\��s.�1����%�[Z�*|)���.A#�9	��w���>C�o+tfy}�F!Y}����&���T������9��J�lt�y����.y*�I}���99�1X��*�f0�=������<��Ixv�xr�e����/ea��}�����H�S��1�����"����,{2������:�d�<�?�V��K�|�����n�R�@���|�����d��3L������[65�!K@����q������������y�F������iQ�2�e\E�����YU
u�~=�����\����j�ZdV����M�4�PQ��_Px��2�@^���|3����_�B"�������K9�g%�����I���R^�w�F�iT�^���PZ��3#���H��uy+i�`������P�z���L�H��F��I'����M6��+��������:��jQ�qle�c3�u�zS�98�+���8X��������hU&��A��F��:������v�-����.�'$t�
��������$��Zz�l����Mk,w�I?[�x�aa��%p^?���l������@��n�99���8�6���9W8�e2���~%2���QI��t��rG���pl�k��W��,q
�l�o���B�xDy+0�L�%&wliU��(~&v��>�o�R����/�����H���L�0��NS���}������F5�a�YIfN������Q�Ax$��dUg�#?�xxVU�������"��=I@ �v�H�C���ET@$����d���lMS�����+���0��ov��qS|��c;"�m�8+3���@+������oJ?@|��[f�93�G��}�]��=����*��<J)��� ^#��
�����
��"Z��$�u+;�-2��N�{������u���
����z�[��0���{ �pF����.Z1�'I<�'Q�����PU�B�I���:�d��l`t����n&p����2���_�w�K4�j{Lx(�,�wj�2Y��l�O�W�a�������%��x��W�z����L��+u x��?�Rv��T�.QL���kv����O�	0?�3�n���m�s9]�� ���Fs���z} X�����^���g��2Q�&����-��(-�E���X�`w��w';P���f����JAwI)Plx�	���ZXF����.f��x�m	7�_E1���[�pl���'Q�d����YE<���Hw�7��~��=������a�������*O���^jO�@
��-�����K1�0��W��h���NT�3T�����:qr5�%S?��V�b��k���K���J����D�Qy��!�i"����{YI����T�;��K6���u��Zw�*�����mj��%��C��e���� �M�y�S#<���&�����q���������<�����#fe�e�Uo��{jTBfe)nL"��q��T��k{p��sw���HBg�C��N��@c���JQ��U�Q���������3�;w�N�e�����)�/���2������s�����>�IL�"^lF�-�Y�?P��p�LA�<0r�����&,w���oG����4[
��d�O�h>��T3x���y~�#���74e��m���Y%3e~��.��5�u$W�&
�G
�
�`���z8�e��60����G������*�b���7�����������E=��x���o���~�/�^P%�����G�u�}�������Z{���1�-�q�	���PV���m�yz��^�J��=��=p�z(�-{|9��qd���w��oJ6�����#P��������.���-��'F_e&
W%��������#EUu����z�8�7<��I���>F�<�6Vo�(>��e���y|1�>�����<�����T���0Lsm�7������H���WW�Do0��jm.��1�XZ����@�<1�j-�
���9���>�4�(�d���5B���o�Y������y��n�{l���
�8p��@0A2b�/�	��?]�������`��q��m��$����-���i<8��g���QJ��������������_������o�R'�sZO�"�t�d.���'R�y�)7��W���l�eO��,��%<<�'�E����q����	ec���M���+Xh�+���f^/��sX[y���0�I�Z���n
^4� T��GSX�g��nF��I�?������m��2�S�T���J��np3�P��^f���X�f�`7'�w\_+qtz��K�,<�yY/�����sZ��_��P���f)�gQ��g��F��?(5k�+��pKS����������a������Y����9�3�2L�!C�Z�I��1���0A;�~�9���R���Iuz�:1���D��j�H�(
�do��n�����jd��v��qI����QA����u2�v�b=�b�&��Am�j�._m����O�?b=s���l���9�/���aa��Kr�����1F%���w��<�E�����?�i����P#+!���j��/h�$������b�s�����r�]���)C�U;%����F�����q�������atg�&��1�{���!�9!��[��o:����o�}�%��FH��9��O������l�kU\?���W
��a��N�d��(�I~����GvJ�������z�Me)z�5�����~�ch��s���:X��(hH#,|���x���+��a�
��,��*��L�.�����,� ��1rdj
Dg�eM	%�@Sx,�D��@ �GS���*I`��.����@4��%9���.7t���@-M����R����3��@q�=|Hyx��~>XJ:N|uM`��w�PK���V'P��?FaJ1TestArtifacts/test-case 4 -T5 WAL ~1024 bytes.png�Z�[��DAAZB)�Xz�;D����.��%�.�.Y:��C�{������������~��gf��9�3�|f�T���?{������{�E��k�'����HkPP�Qd%�~p3�M��R�W�#���kZc��;u��$2���������B6V�[���^��{|���;hG�-A�_�"�4
���9��0�����7zE��2��dN��^~��vo��+�	�I���|��W�g��H�Ek+�"*���.fbB�6S�^md��2���Q��ib�x�}0����?4j*t����_Ld�G`���=�g!|���B�X���gx����V�P$���u�9'rL�����-b�����"(�����O�6C��G)x�/�;	Q?�6-WR�S�+���#��b���SV=��p�(
:������M=V^EN�K(��5@:��>�����K�������9���h_dPz�E���e�MM{X�-*�2�����_o�n�*������z��;��8��RH[�����%6�f��SZ*}����+�"\j�p�MJ��j!i�Aw"
�=����2�����u���C��@E%C�tK����������c0@�[\�����~N����������P�>y>���fu{���h<<B��U�� A�v��#4()'f��R7�"�l|���G��s�V��X�N�[q�M�
&i���L[+��[/�}�6��%�+��MQ�
������dw����
N����f�$�>Y;���<y[J�������59����VIB%9B���qxH�:QQP%�q��$_�]:?n�������&���K`�-���N_�d�NIo!���@�Ar�6To����t����$���F��+���t-Ai&d�a��9Ip��
���(����_���V���k�Z�|�hiQ��CNB���k��"���W���?+�����l]�1���;�-I�1g���%�nS����.���[V:;T[4���01`���Y�&��4��g���A�j�u���ba�F����L�����
l��b��������tq�P�W������{I:-�74�#F2�J�O)�<��s��|�����E�l�����=�dhQ�����E�����o\G!WF�K���oD?Fn(C"���� �A~�W
la�A
�3�^�p=
�HR�O#k�Q���H�DY��R�?�*������\��������!�7h��>�)S5�;� -#��oav�A�2��^C:�8��M������kH��E���`������/���8z������)v@�4�8���
��?�A��������F5V���Y�����b3g/���A���%2M�37����Q
`g��J��%���]8�?�	L^L��0����
��������Q���/Z��E�w���FQDO������A�{a�'�&����m�(W��(H��+�2��P��y �!z������a���@�����Oi��X3	4��kbb���/}14,Lh��<��H�%�*s��A�I'��6��)���d�����jY\�D���6��t��)�&.e`	���_�r>c���F���Z���8@�
@Sq�
�P��d5��keNMOL��UT����`=1�s������������'� !���D3[��BLt4EX/L�QB���"�Y{��$��D-OU��
`A4(������ x��g�4D4��q���?+IK��(�L������/����(��[U!M�-����?
X�
-`��������
��}tNIT������j_k3�r��z�^���p��%
�eH\f�G]����||$O�V�p�I��-vf~�_�V���5wW��V(>6��������W�����+���[��|�����<�fC4�2`���'�.�+�X���Ms��[L������)������Qq
yn��{$�G���	�K��5��;\���e�R�]���c��R�	,�T[fS|Xq���^u3	@�dv����h����$��dxyf��Dp�z�(���n�����`AUI�%�]w��'�[a�gJ]A����JJj��jF��+=����:�����o��WX�z�	]�r����}z:���A��J�1&�R�w������M��|�f}�p�O�W�j>q��?���vN�8.|�R�`��S:Ao?z��G�����E��o���n��k��L��l6t���V���/g"X.�B'�
��y�T��N5�~{�d�\RZ4����J5��3}Xw��WduQ�V���r��G�����U	�p���Y[�����V[����6t����r�g���b������E+�}]��g�Wy������HcU��a3��d���Uz��7R�[�����PM��taL�������^6N����FDl�}�����XgW�P��?�q�:��U.��
��2��(���yI�9|��%��j�6���bW:cVV�7gR����1%���L_`�^�����wx�%7~cR��a��v�{/�1�60.��v'p��U�xX��sr0�(?e5Q�Z�fWv�v�<�3	|����yW���j6�x@����`�4N��%yG�����[ZI
f����������i4k�m����j\�������`�����1�R��B1,� O����o�+m���/0U��L<�5�@�?����E���L)�����#A�AS(go��X��Q�{zXu�K*�c|>'��k������Yvy���F�LC��@���������b�C��k�f���&���wy~�G�����O�6�u�bAL���0��?��V��cmRP��P���V1�A^�|9a��R(�&s��zwv����^�B�>x�f���T��B�0y�Q����Y�^h�TR�^�.�P����7	`�NW�I��{v�v����sr|�����-G�\��	������!
���@���>������'!W��R�V�|����xb�-����X�h���T�
>2n����N�p���k�s����`z�P(�M����3�z����@@�|��M����p5��k�����������������N{�0�O����s:�]:��o���6�h���h8=/*0Xd�+-i({��)��je�t$:��	Z�r������]W�W!���|�
 >�B��-m4-Wp�b��N4_�i�6���+B�<�1�C9U	4���t	f�����j��_j8�}7�����6��8����r��P�� �A�?���lt<<=�Z�B�-�-�z�,�����q���
�\9����W�����?�J�u<7�e���)�z���.b����������������9���7�G����<,���D�0�����XNKc�������}�%�I��_����s��.VW?�->3`6�j�F3��*���n��#���G���o����3��N�3!8����@�.��F����?��p���n�Fvr�&�VJ��|t�T=~:����zA�6@���&q�uMG�.���+���#�����d�����95���"
W��or�4E��r�[������J�&��=:iN\g����x��� n��5�^_�3|����B��9��w��ny{��95�A�
=/Z���0������#����������|���_FF��B�{c����5�����*D���z�c��m�)�%�����Sn���a����1��p�E�#7����W�{�U�m����WQqz�vY��%z�gK�����;����3)���t����c�M��2�8j�o�(hA.�;[�G����#�2+�'d�79#������?�$-Ve<����}j��&WV
���7Q��(�%����TZ��)/u97r���1)����T�\�?l��!}	����ynH��j�k���l���X�&�Kn��Y�B����M�i�������}(D�1�y{��}1����� �=�K�������)��-M�����w���
0W�{�2z�[!���K���q���QO���a���MH�a�X��Q%Ze�N��?�������zK*k~��d��sVZ�w!�1#N��-a��n2�J���5����7�x�����)N�W?�m���ewv��j)o8T�GX�����>$���R����TT���'�&a�������;WP��������/���]C��Z�Ko�Wq��������G�0V��Q��Vx��o<xA�*DA*h���s�_��.2B���4�#��=��4s}*Br�^[��AV��.�������$�F��>��?`k��"�'A�]A�:���6��\6���UR����/��0��?���"�z:+����;�"Jx�O�!�S������=��c������F�-��443�Q{[�����K���T�c?�>�rnBH��	.���[Y�z>�^��v7���6?
1��8�=������/�<+X1b0t�K:�V���x�$�/�W��_�7-^'m�����P�L�N;5��W��t��~q��B�����)W���Im�\��@1�!����hg�E3����{�Vh���������*����:�)��C�7��!������D3�.�M@�<�u�"�
������t
x�aD�u�?F�<L!�'m��e�l����>��p%�����f��`T�T���^0^�v�{2!=�`�����(�{�����m;��>nJ*��` �JH^��b�%��r���kU5:5'Y��6���PS4U���"k�mr�$�6g�X���=)lY|q��xG�la��vG:�]7�[���(�ni�r�����g��a?Z�������Sr���h'6�����C�!���|5|����;�~�1r��]7q^�H��v%a��I��3��v�O���$��.']2��\O�M	+x2c�G�����#�������a��=�����l��;]b��c�3���	�
��.�$;����'�#^_;��"@��_���s�~IL������W�2�X�X�����/���r�8F�����m^�[�w�	� �!Hg�Vf"?3/1����h��l��W��W�����uE�
�/�����&�j-z�ur�Ok��k7��$�r|�^��ov����b��`��mG��k&5 ����$���q�!����^�n$����q�2' ���/����LI��L���HAc���9�����I?�l���\��<���O�]M�Z9�����\%:������h=����oJ4��Y�L0�+�yFB_���5�o����T�Z����KiZ��I�s���:������V�������,�l>f���A�|53����<�1`�5M�y�p�c��y���������r"Cy{���,���hd�QhId}��#�u�:�eVA���U{�%5�kNWS��l������U~���*���:��P�����!�[�=�$&��=_Q����'+;�0U���P=6D2&�]$��X��y��7���}���A�|�Y���e"[�)�����^����^�nM�$�l�����u!����[R�7�
� �_�'|����Y��T�#�o�u`��������P���	�t�og��Cf5wRp�y�}�S��������
�l3!x������h5:�q�'7���[����LC��O'���R����j�h�L�}��I���(��	^"���
{n��������%a��{_�
_��>����R����2��G�r�O��8���|��<�I�q �F�L�
����;�#z�wTL�������w�XR3��'��n��������X}A��ho\��X�.��������,)&�B"k��f��^y5},��;=0X�i��o�������
�8�#Ne��|��Q��ut'��jL����$/�R\��+��|�����=
����z���z]S}E_����Q�i
���D�����,�z8� �3���v���L���FG��@�+�D���	�fc��c9�r�UUP�������e�>p&�CI�vhH	lX��8I8��b�`|��P��ZF���E�����vhGWi�9�}x!�`!��6zj��p��]uZ���������85Sf���W�Nx��a��OZ���#������x�����"J�����%7�M�_�L����8=ywTur���kA�y�H(�S3�gn�����#��v�j���U�{����E�"��U����12�'�(���m��)������uA$����O ���d���(~SB~��1�!�����@nm^�x��J������,���r{�p��4�#��a� -��oD���W�mN�}����SMJ]�*%D���>��{h�������a�>�����{��R_�����n�E���z��T�"R,����8������������qi,��VT���
����{~�XR����w�	�l���������@2+.d�;$�6���;��Ux����h�����>��@�6Ho+\/��
>��,���WV�k��m�IX��B���P^������H=�����~�m��A�|2s�6�&���$��}���VJ����
7���0��QJ�#��	b�)W�4dtC��MW!����&�3��K��W?������@M^��v�x���:l~*0�
��	�'RSx�����y�
�����A��e-;xt�d:��s��a/?�h��u��6��������B���;��OMAg����e��$t����\x�HY �����`mo��=�{���	����#�������]^P�r���6�`����a#=�~M�
�/�����Vd3�����l��
�2�����0!��;1���`\����3��yM�Z�_�EG58����w��#n��G���X��#w.��� ��/�H�?������d��9��r�w��w���s���������/79�^�����_5�W$,��.�N�1����S�uQ���v;����g9[����%��({^����lr��jPR+uHT�W���8�E�?�����K=-�I��W�(��b��j�N�0N���,�\J9s��utb= )�B��!,&����1�M�Y���-<���X�%q�TER����RS�w������y �X7X���
�%K\}����^�+���t�%d���m���`(gV[@��H�����*�T^T'*ND���$\}�d�&���h���E�n�m�#H�tI2@��rv��2���S#�T2/���D7J�+�^##x7c�����7I:���[#|;Q@U�K3��D�1y"J��������l���|�#��$m6��_�����`����P���|W���J����/S���>��zb��$����)haR-�zD�;� W�*���<�E�#=����������o1`M��&��u�o7�����������	!�q�5:���1Ae\�('�
J��=�����Zs��
;�rf�1�\��a��\Abm���uO\�QQZ��3q��UP���~V���}�m�
�"���T8Zg��wa���R;�k[�d_M\o������m7�E(`�,�*���T�
v8��x���f�&z����6��7�L������s9����7��-��������#��������.��G~#+"s6C�$D�>%k��_o2�����6v�>����P��;AAp��B��&�,oY`<�G����]�uo���s�9��9��������gl����b��5�Om�����wj���Q>;����*��O�o��(�"��/R�8l��~����Y�D��������P~q�TG=�)������W��n�d"���&��3�� WAS�Q/bA��*-rN�~�+��z%����a,�_lW�mk'�n�d���r��h���mb���9|�@�x��+�
����d&e��h����G����o9����)x�e�zeUo�/R#�H�%(|_4��i�s*z�F�T-:�3��������_
�w���u]�j�s��"�R��n@���_.��h
�E������-�f��HGE?�l<4d�������?.�R��J� �X��*.�{��c�����At�i@��J��V����])j���D�m}����0���_�w������r�����H?��$�����=�=|>l���{�z�����n)�Y�o�<e�GHa������>X����e�=�mV��v�b�{����$C�J���zCwG���"�V��Z���&02�!�or,�O��V��v����JO0�]CC��,����>1dq��/5�j��l���RH�+���#�f�����7\!
��m���&����X���P6�	]����k���~{�t�#Pt�����2U
��
���l%�2e������`����`v�Z<s���K���@�+KE�@�������iE����~�dv��}������n�G������}���(��%M$����].K:���tG\+	jEJ"��
���p�ccPbOD�2Q~�P�����>|%�&2�e+���R��h~
(TYa��Tm�?7	�|W&��CbM����������y�8/��.F���C�5�g���@�l�{���hZ3�i�N��Q'����2jd�o@�()��$����[NG�%�.�iq�-�'X�����q$�Qd�k�v���+<���o&���s��������e�X8��q����SD���J[���5�.����^
�~�j�:4���y����v��F�������P����I��Z��m�@���ae��? ��X�Y-�B�J�C��[���TI���v�4��yW�T��0m��HTH�z��ke���*��99OQ�F~P���P�+6k��H�wx��##�
�N��.=i8���l�J�nk�	���G����h�`�3��wN��������*M����b$��u5�����3�`CW-�Cd�����#���~����m_J���j����46�����p)��/�9A�� 9��E];j�\n&�ua��e��Eug�����"�)7s�*
`/RFK?�Xe6�Ly�Z��Yg��=Q�f��2�L����t���;�`vp�"e��"�A��N��1����5}7-��a._��0�m�C{��� �I6��j3�/�j<�_��~����T_�!�Z�4H����-��UMp��#���=��y�2��
��W
��Z�xx~�Q���#Gy;�3���o��p�������+�v���Fo�9TB%N8!���2#��3�M��'!�����p��eK�:�K�+�d�re�T����g���=�9.���v��w���T��J��%���xQ�2���zH}�CVk�	���e���?�3E����?7�!i�C.l�7����nA_�+|K���Y�����w�����������u��seOZ���Rlb��M���B��P���%�q3���rj�����f������	���T��kp�'��f-)"ORa��zj�r��{�1]
���9����\�&��������vm��a:K"��~b#z�[�������fsN;S#��r��u�c���f���_-���(����D���&e�u�4�������/gm���U�A��"4V����O����.����p+|�:|���N���m����t��U�h�Ym�
�8��@�3���W8��.�$���\	<��W����w`���6^1���H>��kT��C����\��b������H��_ux���&���3�n@W����U�}�9=Y|�S)XS�������vZ�w�o���?�,��H�n����������rp���A|�J0�b=����=�36�}�[V��lp@�!�T�mc�:�+'�;N�v~�k�*�_~��X���F�J7����(<�eM�J8�w�����$�?��C�$��
^������	C;��4&3���
��
��<;�5���2�����2�����FC�J��v'LX]v;_���<o8hbr"�P��+���"S�D�g�' �D�s�6���~G'�Y�N����$�n!
���b����Q,��E�hk&�Koz�����|d*����|�W������0�� 11���Vw�W����B��%^8V,��3V�X����U�c��xG����"�E�vY�?5���UcQv��Fk�l��V�������$��P���]�+k��m�����)�+qQ{n�+M���7����t���|1�1���y��Hw�2[��w|�{��:OM�W��W��Q]&W�o\��
�XQ��O���nK�>V����8�,_X���A�S���_�(F�+�;1�%H��\��{f	
y�jBi*���g�NJ�T��`�"KR<iX�d.������0$�[R���y�������S�+;Oa��;�R�`�8E2U��d�u��k�I`y��n��G�����2o Co�X�MQ��������:�XW��!/E�{����e��#���#�s����6��������Q��9=46m��3��y3f����(�,�������x�>����sJXlC�m75���*o���i��;���UXx_�
Y.��f������b3eG��x���C��t�Q�}4��
��Ce�~��`�.+5��������O�D�o�����E2�
S+����R�(��`�H6V8���z:.x|��9ib
���F!��:�~��d�0��F������=�A��1U��//<!5OA�x[�8��S�D������9�a���,�����[�I��]U�vcW��2�u��&��I��vE:�UR���,WO�,5�^v���g������m'����"�S������4`���L�<���H�� �l��-�@�b�dW�B3_�@�����6��:��e���?�46����j�U"�i�6��#���ov�����:��^�0���=
&7�zv����t"	/�m}qZ��Mo��H������K�gq����`�m��-��}���M�z��_!!*u�����	3A�&ig�j+���A1����[���OU�W%�����Qw��8#���h*�e�tU�����r�K�j��Z�{V��S_�����by/�� a���]�����Ikv�7��"Nl��A(x���a�{HKkO?<�3���/�a�o��ju�M l�N�4�k4sX���1�T��� ���7�/��
�!���p���#�C)��;��C���K��u}��eI�=��_����t����-�z;Y'O���!O(�6�.��wc���%v"�b���Y������g[����bl�?+�-�hm���I�O�$��h���y�E�mZ������K%�`_�.�+�~����*�,��������qW���Q9�>9Q>OV���{b��+��9�L�^�}"��w��~��>�>�	�C�>��c��4�di\��p�yn���
�{H��X(�>����#��5�
��O�K��+E���+�f�!ymm6��8���>����
�-����'�S�20�Vc�j�zo1>�Aq1�;87��_�83@�I�>��OW��A�B��5�O��������{�3"0���O�,Y��;���p�����
����\�V
�k��`��a���p>C7�&������HLW~�c3kk�P���]<��cl����V�:���B62��*/UQ��q��Q�u���E��jA���5D'$:�����5z��%D�$�FK0��F�QG� z���dl��v���9s���-������n�F|g�n9f�;]X�<[6?R\�l?S�����F����0K�������I%h��P��=����1u�����-���}w��e#C����n���'8/�_����P#�j��u�D��O-n���4�_|1b��
y��]�e������#L��(n����L������PL��X���������g�[��V����������~�y�I4A������]����P=�DT��n[���D�g-��$VI��Y,����6�VT�2��H����R��b�N�9�W<�����
c���}��
��0�5�\��ii��y������Y�~^��0��x�W�N*��V����-/i4�_�usxJ�w1�T$R����cNK��'�n���el\���I�K�G�Id�>��*�[�JR�_r�IC8�4�����OH�b�'�p��*N�%�bo�@�z�B>68i�)m���A�g�u�bG2�o.x���+@�$��	Ot��S�Z���w��Z*3[�������}�'��a;IFp����B�y@�u�9����{�K�V����y��\q�����5:���#�IpE��/���v����������J��P}b��/xZ����&K��2H��MW��Vxsy�Uj��b?�f�
��� ������W���$S_D'�h�=��oAI�A��N8\��x��d��I��n�!H�,�B6i-�b��	��P�Iz�:@<�t`���c��v�i�U��wf�crE���V
�O����#�c�����+]�)v��n�s4�hX�Y��?�����qX�?�U�1�����k�~����/��8�;���7y]�g�W(���E�q��3�YX�(Zj<�#
��-����G�7>���N������r����/����^��`���Ty{T�pP�J&�=���]���/�E�����A��^���0�w����������B�����sA�z}4��s�{P�����@L���OL�+�����d��0��%~�����<EFgb����S,Lr��c��2������)D�O��0�������.fa<�� �s&�����!!���\��oz��	�y��p��-�!��`��E������v;��6.U*����1�v����?��L"�O��v���'����-�{���K�N�5�4���|'��B�������K���c")����;�����QtdU���5���	[O�D���`����GM������V�?�m�`����t8����p!�lr����wM��y�M>P�����pM�S�x�J?\�
o�
(��"��'h',�fz�}��@���n�R}J�'pI��p����UM��h�%�8� ��./H��� PL'��2�.�^o�54s��������(��)��Vl7��!p���SHWyH�%�8�~���}�u�����x������B��$�Y�u�>��R��e����F-�
2L�9����9P���b�2+�C������e�#[o��l���5�M%����=��;L��%s9v�
�g]��+��c��VOr������|I�PM�=�������y��s�k�S���+���G���';��F��n����W�9��6�P1��j�NDfx`���_� �N	qr�P�
��������%+�9�O�M{�7�

u�����/EY}<�J~�t�y��7���;�g�{_
ZV-w;n�|�p�np�5j��Rxwz������%@���\� L?C����@��7$���[�czfO�Ti�O����~����N#�h�	�4A��cV
>��(���������p��uQ�!Vv������8m��]�e�j0r?�	1�n�_��:UsH�x�a\.������h�8)dPOA!^������k�a1�
��M��#N���9��X�Iq�^�o@�#�����R���e�C]���~�nE���r�@����&�i��;P$n�����l�)�F���F�5n�|JHDY�u����	�����%��g3."T�%��#Za|<7? A74�aO�Gf������|��j��y��gm����G.��NL�%	�}���
	�X�$Q+�z;qRF����{+��A��[�@ �#%�2��ur�R�pRE^b����s0��Q�e1��-Ap��j�����9���[��2lB���sg������Jc��4&��>����4O�`�K�D��mn��Q@M~i����{/�A�Q�P*(^Y!��!�]�Z�,��u�B����wJ����*�N�[BSnN��x��`%t|��������aL�P�����u�`mS�}cZ
>\���g��>CJ�� LS�P�P�%E��t�@�i�G��WQ��
����	u}G�$����L��RD���r�1�����k'�_a���!�~�9KB�����f�����r���U-����Q11�y}G���j��]~������vl�@C�:�E��M��3�?g_�ts �L|��o������T��FN7�E)N���x��2k�=��������C���ef7��+�1��l8�m|<���\[T�4w�nD���������8��������9i�dAI�����l
�`U	��>k��������R����Z�0)A~r#��
B�*���y�kesE��i�2D��A[�[:�h���!���l���j����=��Sw[�A��~�i��Tm���K;������RFJ�L8���z���0�!����^�F2�k� �g������T7H��_	W	��B\��pe��q
2f���&.��H`�=�ce��~�q
���������J��?'��n��F������uluP�)PCU�+�I`��R��]�k�����^Y��`s��W���1�����n|MnC�:��	X�n���n��R��s^k���mhjR��Jy��K�Yc�Yn��O[Ug�`x(�qCW���OY��pm����b�4fH�=�S��_Lc.����u��������Z������H�^�0c�{'�	�I��e]��s�O`��������>�
okA�
����7rf���1$*���������1������X����������Y�ox�������I�2�e}�i�����R�\L���uK]a�D<�����iTZJ�51�U��f4(9f�3Er��|�����=��_�j��������F�������r��g[�+�����������1���x��UT}t��N�����L��7�,��m��-U:����+�3HZ1A\�]B�j����8�^:E)-��$�b������*�����.y�R��La|5���g��:<UJc���;�v�������i<k���%$���X�4+�V�����3���aJ�����TTr�����[�����j�J��nr$�p���?:�3�H����]xj�q@G�s��>0^���Z�c�,M���D������i���������o�l�l�ahK�y^s��6�2;��{o	*�	<��u��j3�����7PAG��������K��qw��>4��5�y{l6�KX67�+� ������3W�������{����V].���0V�v���2�H7m�u.U��K����n���@:����DG2m��N��XI�Q(��b)���1Y�u1!�=8FAN�PJ��T�H�����=1�d��L!q��.z4+&��H��0K�S=C/0�.����L���^�v���{<YU� �S�fIfTV�b ����Kd;���-���-|3�/�
kV%:�$��q�hzc�b�Yh�b8Yo��~��% a��B�X�h�v��"{�>�=��i0��.����v�^��LQk1Bq�����+}%X��1U	V�-6��w��D���#���#��<O�P]���3�}��?���@��	-��
��(Z�=�����h������7"���A��^��������>��P������~�k`��9����g��.N�6����)z�%�jC2��z� 0[1���h0�z���&21^��7S#r0"7Z���JbuBF
��}^�!)W�?�m(�8�~<_�L�+���V�.Y-���X��3��V�
a0:Vq!J+�D<���lgq�1�5�������TP������	i+�.��.�O5�5u���g�K����,�Q��;boRq�%��b`��!�
#:�#
3Sf�5:�_����QqZ�1�
�	��p�6���H�t���B�� 
�
�?�����@��]5gD�F6y�[��>�����^������<B������{aU��h>���������Sg��XS��Y�z�j����>�(�q
���<9�zd�`T��1���������P�H���)��$4���J��-�$|��o�_��YnD�!����x�$_q�KM�pp��z�<��_�b!+����*+a��|
�X
���x��-D���0�x
"���x�;���zj<���p=f���;�	D�x�����8d��u���s��BG'C*l��r�:����
rE��j����A������0�qa���7�����J&�tI}�������e�pHM�#|.]	�<=id��+���������9�h��.����7��r��l"Y��PX�W=%
myO���9���*D�%���B��(�3����j�Un�q�l~��"�������:���DQ����?���I?�� ���{�d�Q\N@U�����Y�K�o�yE|�o��~��qU4q%��Xj��;V	�)6w�������Ebz��z�C�1bf�I�L���DN��C����������SVZ@�zz��:��=��������$�����!�L���7�)�Oz��Y��M�Z���0>�G�{9We��O���Q����*C^��P-����/�1�=��!����"��'��7�R��������p�9
%P(�b�(�y�$�b�{Bd�^/qa���C��x\��&+ Pl��lQ,�G�����cx�'8���eh����X+���
�Fo��3�~x����*��K��b^~�aG�Q�	]_����^�%�Ro!���-�e�M��C~�B�\B��\~,�T�	��:�N;O@�{z���}g}X���1��k�8h�X2ul�k��2oC2�<�pG�������t��k� H����z�������A��.����J�?JO&SMK	�Cc���F�R0�77����=�dO	U�*�j�O��E���)�u�@P��}��e��_iz����Kr�F�����2G���C�3e�	A+++�1��7�����GJCl����\�Ux�`x�1k[������0�&go9��U�q��G}��&N���p��A��
���,�����W��baFK����'�1��������,����h*�����(�GfTx]�4��'��l��pB�O{����^L��\=����R�����%����O�f?!J��W-������K|�QK����1d���r��$Qb������-.�{&-O��;K�{�qT}��"��K�{]MLz`�����Kj���W�G/��S������������mf�s��r�6"���H�5�D�P�%�&�E{$1�I��O,)����Tsn=�M���trui:�p����-?Fj�/�����9���?����F8:�k,������:[��g��
Y�AV��.�h�V2�2���=���J�����zT��j��rK^��'��d �PzL^L?!�3��3��MS#\C��EH0�����<���%M������/"��]�h������������(�y�Y�T��4��(�&r����0	�� P����I�x��NT���O�]���b;����*#�K-f�q�F�N�i���Nb�
���#>c�A�������bnR���,Q��w�m�������O����=������[-k%��
�
��E%N��]8�9
���w�Ed�5Cf�cS'�M
���|A~a�����&}&��w����L�4���?�Xfb����g��<��6��l3�#��ad4�Z2����y�w��"���;6�LY@�)�t��iT;��
	���q_��������!�����B},���\#OR�mS��^���#��J%W������(<k������8u�a4����D����R�cQ��������p,��E���<��2<��k)\p���
�]����G�kH>���mP@�c��_���
KM�8�`�x����u
��)i�����PK��VD�+?.D4TestArtifacts/test-case 5 -t1000 WAL ~2048 bytes.png�[eT�]V@B��4���������R�!�t��tI�\������~3�����^����>����g���D_b�|����G����=_��#������4��g��=��sP�IDt����s�/9t�Y�u���/h�`���7[4i�>�R�{M�i����"��{��9R��9�b
u���S�"��4�
)FqL=�k��=�����I�5�W*yf,k+�sG�=6������_��XX`]?��<>�2ZZ������@���H��g��&D4���O��^�����I��1a�� h�w���v���IJ�<����=��N
��#T�����@	����ypR���rI�����U�P��@�����q���\�����
>��e'�),��+�A�x;���82+JS�Cj]b:ti�,���FTLu��S��a���o��~��!��iYU8&�(Ft�o�4c���sE�(	�����E�3~>��e�M����	�z@�q��]I��MA�;��V��;_�r=F��u�OPX�%7���c�R�!���B���O�
�:����S�4�Gc�>�}�"v�+*P��6jCs+�6��c,�������a7����_�'�3tP��R8_Q���#�2�V	��g6S�����*�v�6������Q�b:�u�x(�5
[B�br�w��vC���rQM:����C����f��3�D�&q�������	��`�H����&������*�Q��6��U��	�����1k5q{*���Y�
��Q`>�q��r0#T>v����+gdc8
aW)�df6���\<J��w���~�!L�[��6������h����Cn1�B����\��"5GYb����J��k�&����l�y����[�
��3�����!@[���\�U.���bo� ��	uM�R(�r}��-iI�HDZa~7���c]�N�� ������o%LCQw`�^��z-�G�<w}���8��-Z!,q����f�� ��p��w�9������x�it�������w��|������?��61��U� ��V��|&T���C���� �@�1%E�E,��/�<< �~��c���<�Ad��� �B��������x����~�,2a����-�}��^��U�!����@��),�y���"����3�A��tJ4{�("�s��Q� ��-�[������lI
������eW����D������Q
���kR*~i�#H4���/��"��t�^-�x&�����)|���Z������J�@,��������	(+�^?�w�?����L:jj�_�Q��Q��T~��>�����_�E�]=�����	���(50�M��_��a���c�����e��nX�]�����12"G�9�$��T����fA(F�>~��}�d^W��y��R ��	��Oi������Rww@����iS���`e��yZ2$U�mAw/j���g72!
�T5`���Pme�O��|��.&9�P���5(R�\��$���Z�1��1=��f�ui$_+���]0/�A�E_l��A�2��2^���1G�5�V�r�J:���
)3�U�*72���*3�6����G�G�K�Or���8�3ie�G��x���.�W��lKE�Rzxh�	�������i^��!���m����&�j���(���g�#mB��RU[�����x�\U~w�M��t����� h���	���Q���G>�G7��K}�.���&62H����}
�_��!���D�������~l�T��!����b8�����V������l,7$�%���H|\1}�?e/�&o]��T��^J����b��-����'aD*j��(��w��^J`<{��^h<��L���n��Y�G�M�(�g�x��;\�C��o�5�?�.�"�`�ZV	$
��i���G&H"��N1���������{���8�YA��������6~���1��3���@�Z���VG$"���sBL�=�[c��a|�!1`8����pD
�KC�~�Gz�k'��2���Q��i&�����<�2"&y�.�S�b+�==c��IT�r�5��H����SL�*{"|�EI�D%Zu�[��V+jaA�r�����{D|�����^g�eXQt"�b-���v#�t�v8��_��XD����R)K��0���j�E/o4�~��6���R$���]�I���2x�7���������y�����O���k��P��f�����)�������\~�����hUfuz��X�ak�y�.!)1ps�S�F��nF���Eb��v��)�<M��tL$%n2{�L��pw6�=�h��������Y�����p����t�i7�
C�O���;��d':�B����P���=x�!E����@1a����g�7x��u1+���#��(m�+'�BHi4r!-A$�
�o�V2AY_'��H���M����C�J��*3��Cqff��-a'�����~1^����`����[[P��^`��������~F����hB;�@7��|����:��<�Kk�E�X��:N���:8�H�.s���
�����8k�o7�B�Z���H���}c�',�V��qHegfA�JB�b���y9��������,Iu$����{�N���\��Ya�!|�/�9wx���E��b�n
������[K>�{���~d'D���j�X���M�K�Z���C�������%�����F����I��f*t6���������3�"X,�����DN�jmU���$G�+�J}�=�S}���E�������/�Hr/��[K9&T�K�(���,h�m���2�@9: rL��U���8jL�����d~�
_�x��E�T\sP.F�v��5m�!�������r��)�6�l�����w�o�Vz-��7����f����7QT�(��k��/z�oX�C��t����j&���X�?c�;+�J��gS�����!]�1UZ�������Jj"�5���(�q�U#�nXsm��C��������-�tW�����7�������~l�Z������JL�k�O�O�wV�e�t���V�&�9����R���i��� *���]�k{�]��-1�;��������^�mg�#�W���r�N"���YC�y�����V]���/(	���J^�z�9�O����8c��6��-�]_Z8I�6Th��_D.�#'&E����5i�f;����X|E�{�
2g�������:.bz�����Y�x4������1c����0�VO�z��}gS�����5�xG�M����i��������3�35�����z��6�k������,���K�E��\�����Y/�
{�!��!���m�����h!������c��-m9jvq���r�1-�b���f���j��}����_eba7}L�B�]���7�����������O�'�^9D��������Y��C����b@���wn�u~��ZaC2�5�����n;�s.���P-}�K��}��������r�����V�\7�y(e��M�G�y�z/$��W��n��[O����]��@o��1I
�N�VQ�1,xSR�ot�\��ob)>��8>M��M��'1���A��Bf�f*W��j-_��!h
[s���W8��V"��o%
��
��h���?�t�)��+W�CQ���xb_�y��\�8��4�j:����.&�����7#������������rZ�����Y���}9��`�#�Z-M63\��,��}S$�6�X���+�(�3��q����1G����0��O�-�����@*�R����������0\��T�w�m�a&�+�����.�NX��T6)G�C��}KI"M/��)��N?n�}�����������}�6ez��~"�����'��>�x��a�U�_�i��.kMl�W��Q����)��t����[���
�R3�{(3NT�X��^��������XO�z��|���7���f��60�I�@5���T��mZ;^@d����~`�[��\,�k����AH����c8�Mv��rk�"��X(U�'z���n4�{SN����d���McZ"�+�����k�_=�R�%k	x�`�;]&d7�m�|����7;tH2"�r;���<#>Klr��T����������L����t����^�t�z�P���<�H���,����
�d&f
��h� ��9mM4h��
��y??e��P���N����t
q�a~�b������*���7v
{�ar�1��Y�T�_-^6��7���o�v]��N.�'%����d�F��]3�����R���-��4���,��8>MU?�����RV�_-�����gR�U���<4�D�)r��M�����=m�cU^Q�B����������.+�+�}��c�l�T2�����z�`@�N���8�
�^�;!fu��py��E��2���6o�E��7�K1��cf�9���`f��#�KG��da�TE�U:�\y�R	 R�x�Rs{l�����o��'*��DW9��1��M%���>$����+�L���J��7}���Ts����;Jj�'�L����1�o����/�Z�f��J7�f��Q�=��(�����eUNN��Q�r~��&UnCu���@��
tF$����^i1u�3�xv�4�������^�D}	��?A�',��5���Ez�U[dJm������[����ms8E�>���f�b��U_2�w��v�}������Bn�q����QXr�+�A��r���t������sP6��9��*+������4�TO�4�6����Z
$}��n��z���gdH��5��b��f��S�@G���K|�-��-��
}P�c���@���x����%������X�S`=�$�%��7��yG�]b�T7��N���
��}-�k	��{
���/G<�zg:���T���X��!K��f�s�o��w�t
\,k���R'X-�V�nC3����8������O����GP���/v�� ��R0���$�a���*-��'���T�w�l}u6��](xLY�V���)hn:[��X��'�
H�5��+\����1��GkBU�<����RQ�]����#��#�"�i�U����f_�D'�?�_����:�L�4��+;�:�[���������J�1�s�L>9<#�*�w���~��MX1���1!6/���`d ������932�@������]������219��G7z���������dQV|A�7�=3J8G?���lY>���7���o�� [���H�,���HE��TY���B��|x��b��Zz:nE�������>���N���,+�����(MNy~�O��U$�&�pB��4�%%�Y��Zh��N
�X�fG>�=�ei�����?s�F,���8���j���1�d��[-�N�J"c�+7Sh=+��a����
����b�E��Z�\�i��+�C��.��������� \�>����5��i��pD���$e��ke���P��'o,V�c��e�5���R8�~�+U���hL�[?����F��xn���~����;�B����[bU��[�����u������_�q�
��VA�8*�tYi0�}��%F�����U%	z���4-��,�������E�����_/g��ux.�@;��\�k1U��~RI��_��^��~��Ar�}��v����D
00D�����O1���)J`�v�7�+@����g��"��:k��l�PZnE�u5R�*o�|���������M�A���W���`)����e�,*)vCxj�T(�.�T�"I��`j�$�������V�Zq�����yA�����u�sr��{��4;1��;�.q{^���� ���)��YW����|���9�r�49f��4���x[~#�����NW��>�&���V��V�jt����~$/������R
1��@���W?���(�m)�����>�v��}�g���9�{�f�����]$�nAsE��r{[�m���;2��O�3���#H_+��H@QV�N����u���ff>��3���6
�!��@��c���f�4k��T��^Fdp
mv�"�Q��5������4L���,�=W����nX�MN���G��j!�� i������J����������\��R�>�`p	�������9��s�m/�(�Vt����������M+[@QF��(;�d#���:��>�)-�,�r0���(iQI��5�mF�IdQ^l�!�MDr�$�A+��������D���p
�,
�
D�Z��x��yUvg0��Xv�qd��v������Y
�k�����aB�_`��*NI����%���������x�D��5�l��?�.!5���*�����m���0HO�������A���rG�u�hu��b��^P
�H����6�cX�&m�)��:������Hl	�0��N�nzU4�s�vi�l�2:g�T98���#�W�$�S2��0������,����
P�����%�����u��*hk*���c�r�M��:�v�-J��H���w��4$����i}}�W!Z�N��o��]
�J�\�h�u�_���e9*��+��������O@��jW���%0f !�Uk�6Z�=�i�4�mj)=PN�
��,��kkj��Ko�s��_���`��<1�F �����O�1oy�L��	
~�Z���IH�i��E	
���R���S�M�n|b�0x���h:�^wz3�����(2��#���n@�%�u�m������>�0F?\gy�VG��F|��\��kA��D:
=0Y%�R,���P����LzB[m��LN�DpE������[M�3D@��#�:����<�j���G��l3����r90��:)h.F����PA�eU��n�p���^K��e�q�)�e�����s��%N@;0����>Fkx��xH���v���h��l�;z��F�1,���G��k�}"_����FI*���|�LL�,��$�(��.�t5�H���_����,2��5�~��R�\�6�c�������)>KN���v*�����o����>� �E={F���!�B�	[V[����L�9@b��	Q���#Ht���e��5�l�����Z��r�a���&��	5��<�jI�O�^��7�����x_�=�W4�[*`Y*��;��������%�$p�+�j�!��-�����9it3g<Y�j]P���SOE��
��L���0h������1��k���8�������Wrq��ZG	��i�����"�#>������vy�!X����{���5`��U?������N�m	([Oq(��w�E�M��RC:S��H�ROG]��������Y�MR���������g�����w;>��v��.Pl6�d��-R
�_:�g�6���-����?m�������=/��:��12�����~M�C-��x���{�OQ�f��G����+�������.�����I[B��2Q�<
�^`��1T�m��W�{My�h����_`�XI��qq$2�Sq>x�9":r�)+���p��F���i8���yi?�	����`�W��?�GI
=�X\�
eM�wv]O1^����e�����v���������z��Y����������	FN�M�����Uk�O�:�E�b�x�m�������(�H�^�2��>q�a��0mn~��D��j^��C5�����}��9%G���4y{�b$(��?$]*�H�Z�BE�!S���+��A��q�^�r	!�n�m�@jO�)��|������#e��i���K&���������g�G	X� ��O��d��AL+���Q���GY(6�mj���oo�b[=�?:L2�����957���(�k	���q0;Zp�I?�R`T+Ej��!xp�������Z.jp����t��d	��&m������H����1��0��t/����G��Lv����wn�������k �9S� 2�/����7O K9\���������m�~�n7��x/$��9/�{0��$��~�������$dG�:_��g��:4�h�*S$z����v�<Bj���++��bc��D���|���r�Zue3����?%0��(�|u�;��J�aE�jl�F�S�s����������$[�H�y!�������-�oe��Q�	5�bg_n���WC�[���I3������~_��z+��W����(�;X�o�F@��F�����V�
�j���I� �B�=��c�f��q�|��1I_a.;~���n/�&�Ouk7��~kgL�I�B������iR�-��:�������^��3/�����Xr���~����*�K�
�a��@�=�O��z���+e�G����p��!
�.O�_NS�v�-�x�#���v�:�/Q�/�C��	�j�1�K��`SX�!�Z�_9���\��HI
��|B�+kq����!�����a�����T���'�����1T[vbC�x�
��f���%���dm��%^�7��k����X�lE?�@6�m@�������N2��m.���w����=���,�W��\��E�)�����^�/��*x���p���}�?B���� �����m�[�J�#�=4%)����V'��7�v��[�eTk}io5�I�>�2��D��x�@�h���4���z�e`��]�����������3�
�������G�<|^���%D^���-=��~���`[<�]�4=,g+@~��O�t�U����^�x}y���wf�I�����T��&����[��m=3[Cm�%4h����#J��[E����
�����x��y�����7���b�����O�����f��B�����SC���T���|��S��>�������o���Y�W�c��[qF~1�=G6�;����0tv�W�
����}^sS�,'�|jC ���wR3������3�P��C����_�7��H&wt�zv� +Sl@y�'{<Itv2��>xo��x������HX�A���p��C���aq���S�Mne!�d��k������������Bpr���F����U�Q��G.��������]��4k`r��%������v�rU�a�T��i���]uh���=cF/64p�!��������Hr�����(����I%!��������u���u����R1������.��y,Y�%
���=������4=���W�am����4H�����x�4������C�:����D(�,�19MA��W�� ��p?z�����`5�J������	�"�m��S*���������M��H����l)�-�WA��a�[n��o�>���YH:��f��=�h���g�;��R�E�F����������U��W����1Ku���5z�b��X���o��V���v:�EFRV�{"�Z�I����y�
�V��"n�Y24}���y�L\�#*�J�=M�����,������8�3�D?��Z���U���WR�������k�E�C?���t���%��){�g �u~�N����37�
���f�����T/��
��'%�2iW~+:+}I��!x>0��H�7���~��q�s�_�u�u��fx�3�����J�Z%~��p7��\5x��������������y}�Ye���������.!bpC�����D����������?�~�`��T4�7H���:$n$m]N�������aAt�����_��=�cu������C�������N�|6�u�jXu�$�q���bq�$�);�
���[O����w���/H- ��j�8�g����� ��	��u���-���o>A���\q�U[��M9��C�n��?�
��������
t�Y:����C������0/Si5gE����L5�.��h-Jj
�s���W}b�
AP=m���k���K��5���M�"���)�xa��]�����z5
���9MFm;��/� pID��a��O���5h<+����.v��h�n�������!�2��~��R/t�cB��
�{;�vs���M1,�k���`��{����-��R\_�^,�O��jks�:o{�m����!m�&�����[�d����4)�vcI2��8���8D�7�2��1�_	������@R�G6�LK����	����&���
�C��M3E��������nHn�\hm�.����=I��{m��\���-p�g�pf�k�RdU�������T��7Q��)o��
����������X�V3j�:�9�qF�gB�^!���D!����m����Vy#�d�S�����^K���C*���$$������DW�cn������>�����0�q������{�_�b��p�b��0JM��u��D����cC�p����
	
�X�hh`��%���L���
'��4:��a���4:vg>���v����,K���vK���d��h���.���j�����\�{�{`Q���3����K$U���f�iEai�`��K��K�=�P�����{��O"W$��-�k�y����G_0
jJ1V�7dmXB2����J�����b��s1:KrW����.)�q�6��
ev#�)�:�S�I�D����l>M�f��U������2��=��C<�������0$#����2��uB��X�Z�9h��85R\g��/M�
��<��c�R*�X
�	7��<��������)����c����g������l�y�����.� ������d$l6�R���HH�c!7)Ea9xQO���)�"�)i�)Bc�w����\c��	�����{l�X�`{�K���Tlkn������[b���{�Qzd���)sQ����o���h�g�
����R���h��;|Q'���kl�J��,b�VE���COd|u0����Q�o���������w��3ea�2���I������)�/"Y�%oN2��2�N�#���:7��:![(�������I��"d�.ez��Y�����A�����PP_��C[��|'��'<�T8P���1l����*��Yt��^4hh"��]����w
�/�#8B5	V!6�p4-��O|+��$���)H�Yo�DP3��+�9���|QS��;�J��:����g��q���:�o�8i�z�����M��6�9!�u�B�-s �@DHI�B1Au�������r|F:}����:N�r\
<�xjs�������K�I���c)���i��V�xJ;0���(���/���x��A���E���(v(�5��X`u2���t������h-�V�Zm��v	�B���	F��:�'�G�pP�r�fB��������[iZ������y����(Cs���&JKB$�����kV�Q����wP%����g~���[�7�����t��Yo��?�����/����F]U�R����5���p��;���H��z��_���>]V:�3�G	P������>f��X69On<��}w�uey���Z��k���e����|����i�(1*��tn>��l�}����|���~#Ns<��c;���L~;�����"��U��r}��@b:���	�`�/.�y�(����-���5�YL����[�)E�����@5&<����fx������`^~�F/O���>u�D�Q@r���2��������D��"d��v�`�i�Y2��]�W�����,y��2��3*d��f�t��cG:=��-��qv���j���2�yPA���@j�v��Oqf:���q2L����7v,,6Y��Hdi]�.��uXi������Y����}�����R��qV3����&��T<��
��������� �Q������R$RdBQ t�H��I� !�� ����	U��z'tB�"Q�������������s�{������?3�����5����Je�2k�&_��2������&)�N�[����~)1����,�W���ZF����%p�t���u �oY��%r53����|�V%�D�ul��Gp5��)+
Y��-�q�����D��kZ���������'�#v���������x���+�cZ�.����)��y�����d_�t
��{�r?��-x�2���pq.�e��D)�Ks��4\�[,a��:r��hm�gc^��^�XSY��8_�Q�k��`�1��j����
�/X��:0D��[���=�������"������N��/����5(1g�}�����P�������4>��`�����������}�����Aw���B��%X$$}��jh5�i��	�&��'+�B��.�-^����g����L���V���|����������*i�����21�$���(�!�����Q���V��������L����������A�x���A�I]�`�������M��a��g���[�����) ]��&�5���i,3"��Z"hk8.I�������?4(]3ip6�V�2x��Dg=����u��N?�.\~��`��3�x�c�Lz5�����CC���_���Y������������T�RQ�Z��y��O��q������P�}$����1�Ty�-�AR�s�Qp����'�D/F�hZ0iq�����,��r�,����@��"g��&���TF���y��_"�r����f/6�j+��@�}�:��_�)��;����2"��:	S]��I�9�,�|�_m5KR��V�k��gW�����92�P�� �Sd�����[0�D���p4���4d�,'�C�kE��V��:���+{����Q�M>��a��������!gs+=���f8�l��F�3�^]���q%����z��0��?�e�+g&���m�%��
@s��PV���H�����f���U����Ms&����u��rnw�"Y;��E ����L��NR]��m-������U#��33��aGk�}0����VQ�A$��F���W�p�%�S�����"D�eLi�u�7�(ZH4
������X��G_����u����]���U{��3�!��)c����;AS�&k: ^V`$��*����ah�L(���s����W�4����L*�|���Co�6�lWB���t�0����W�m�S�3�6��=��	�����=)	�>	��/�?�2��W���������)�d�R*�E7��s��Bp�����l���H��$�6�n��pcp(����s�����c����2@Gm�7��t�X�Z@�4���{���=�
�}y����]��&�^��S�*�AeAxPT��L���{fvN��"�T�y&��"���]���o�k������u�E%��M�l���$#z�S�fx��<:X0�@�/���65�������9~.�-U1��sw�r����8^����mQ
�[gow�����6@�Q��E�S
NlJA�T6~�(��]vD��
��������O?�5I=im���1�y��aF �����������pw�A�W����y�	�lw�yr����})�6�k����>��"�o�r����������������YK�@�A���c?���������@�3��\^��G)��c�������H:!�|Y��1#)�y}�`���d�:�~�9��s�mh_�������s��;�� ~����D9�Q���$�>�`>�^6�]�x������jM���\����z����%����Y�u�gk�P���}��D��(+H��H���]�C��k����:�O������yUH7�AFo�U=�tIC��T�"��1[���z�!�����a�d�+�������66k?��u��E*R4���z��_:���D@����3���)
��U�R�r-�o�����4;o�����E��,�*{c��9�a
N�P�_�#u������J�r+��n������x�*a���b��.��4p�?�|���]6Y�Jo�����yk�e)�����b�r*�������Ch��/��������X1�D��*!��9��}���vq�Ra{6��X=�U5y�j����Yn?�����6�2[��<�S_R%������+�=���^��~������H��c��}�oTz��P�C�/�8��������b	}���F��9N�Ve�QM+��s�c����:���v��
�(��\O#�h����3YfDK�����vp�LK�q�dmV��5�b�K����v���E�})����^���~��:�����'q5 .�Y�3}���;�lK:'l��iH�a��Sk��s����]J��o�a�aLJ��dT��`������[J��B����32��@��9K)�f�����	����8��,�>7�$�W�X�)������:����EBO)gh���[HF�{_��1@_�T��C>��c_|���)������ R��	�m�%�j�#j�
^,$y��]����V����o2�H?�1{&� c��iLz�����A�Qd��&3���K��C�����1j3R�x=F�|]Y&��8��}u��B�-FI�;����Nhl����d�+���J��� =Q%���{�97��N�e��z�L���y3�"k���������@:<�o������-}�w��R�{X�@RU��M��|�1A��	�Q4�����;�����Z7&���q�*B����"�1�_X�D�)hE�0�� ����;Q1�����#�J�����V"J*_]�]��m�W
��X]}2��� !cbg2��u��FX��n�t��������/�SYM�7v�*�B��C�{t�����VK��?����*�t��4L�~r4,i��2���h"��P6��;:�w��Z`���=d������OM.����60W���2;T�����K��N���%	�`����&��uS#o���X��o�3�Z����n�k���+���Q>�If ;��;���$W�W-}�1v�Zf�rN(�;�\����7�H���G�
��5��4��b"�V���?H�< ��X=Y�  <>�q' ��  ��b���WS�e��DO��WNw)����r�4����f����Z��]�H��e��hV�T�R���IX?Z�1�jU5���6�G�Zd.-�W$�����Iv|���9!���8���kOm�����������5��{S0��������y�p��y��E��[�mhh�fN��-��/o��'�+��9�o'���W�#��nigK���=~F=�������/�r`W��{�(�%B����&-@�&�,q`����)����p�@���� ��s��$s�2�-R(BH^��^/wU�c�a`�F�|�@�6�������&?�#��mg������!mI��|��BV<W��Yq��HO�� �L�����E�^��9�!V�>�g<5�G'�r�g���0�����A�t�G.3�G��z�5'��S�D�*�F?��F�	�CD�>~��������_����,���ob��_����+��Y�QH
y���c��2�v^�BR4���t�TKZ����MYy���P�TW��o���]�X������H���Sv[�;����qc\c?�i����*������M!�W`��-���K�c�W����\2/x�>�rf�W�\^�+���5�F��zFVW��d�A��u�{��%�eO�O�&�W~~��8n�u��w���������H���V�0�{iei��_�u��^ f&i{�%:D<����~?~gJ��;#� ���+�����'J�@�[Iu����P%�����5J���~7Q������"!j+mn���-�|a]2w��C�/�F�n�pY�����l�rr�u����U
���9�$�${v����c��)��)X&fbgiOnn�R��x2��
b�o�p���p���-�x��P���^�Z8b�%��c����A
���.��z$��`I�B/�B.^;�,�V�d�p������Y+<��6��j��Z�����}���F��F����������5��!��3K�����K)���JG��g����TS�c���1�xQ�`GkL�������������P�j��6
���	��Q����PK���V�E���E�I1TestArtifacts/test-case 5 -T5 WAL ~2048 bytes.png�ZeT����E$���F�n��V$	��;E�%D@��%���Fb)�%�kiX����9��;����{���;��	QS��{@����[x
�2�n��=����c���$��u��-I
W�f
�'���O����u���X��5�D���CZ�����2 j$6n~D���$�R7c	K�`�0��w��Wq?e����W�0��`M��^�]��n��SS3��e�|�G�I��2����?�s�}gu����-�~]�j����T��q�Z�-n=��~�('1�������j+�W������"q����t�dN�����Z�C���T���{�#_��	������A����wMbx�/�Cn�T����]c���&FH�7�ul�e!N��a��x��*�
���P�d�*gF������[`W4����o��9t�C�����yad������B�R1�(�MSC��,-�gx��o�����t��(�������M�,����� ���&TV`|o���<��6�k����OS�-!���G��0�#�S���mC&��������=Kew�L/qx$4
���uh��K>����j���Z��)��f=�	�YH�Xf%�]�Qd7�RVKDD��pB~�zn4]��r.[���]��ve���#���(��1I�d L1������GH��h���Q0��K���nyAtP�(U�
w�������c�/$��:�T�H-�X�����E`p��G<�[�&��*�d����*2�k�@����=��F�p<N�������13�k.i��O�]����������$F��EG���{�6��m���#���z����{��?t�"	�=U�0z+��
��?B������prYm���T%�����0�kx$����o�
b�����"�<Q�cn	�����&Jd0z���
gF<w��_x����/'�b��^�|�;�$b����435�����A��69��G����W��K����J��x��K��2�b/�W%<+�d�����-�.�6wY������q��<��f<>��,���w�^|�'���|:!F�q����7��+A����2��O���������O����_����R�h��%)|�&vG�#u�u:����?2�7 @�7s����%�o6?��� �G
�����a����W���F��Y���&�n����K	����t�2����*���	1�]�O)y�Y-�Z��V����/,@4��x��MW9Y���V���T�����?W����b�
���"�X���Q����q6�MQ\��/������_��5����57���?E���K��	,�l�|4z�$,9�S8k"K�Z��O���v������~V9��</�����c����1��?�~|�:�T�)�t�[�>�@1����P�L��S�N+�x�)��S�`�6t�$��(@>�������,|Ol�����!0`����74mb�p�7�m��-�d�/ �����,K�W��eWa`�E���w<
P}P"G�7�|t�Q�U����gn�LS�����S?>��
�n���eU�� $��a<�:�x��t:��c�W��I'hh������X*q��_���S����c%:RClA�O�R�r	\_��'�]��������El��N�����M#��}K���x��
*��R=�oyL��O�2��oQ���L4X:#�,*�z���~�`;��������K$�'�s����pL�BH���G�4� ��5�x���Q���4��^��h\��5�.�����K
C����Mg5�^��C������\�80	-������������-������E\MTt�$����"��XZ���36~��R��������K�-��z87x\re�s��$��u���0�,�L�����6d�P	5�F�F��>~w��.S����=�����d�VvG��?���n����G��#<b;O����Yjw��4�|���H�����vF���
-c�����l�7��/b�m�w1:\��������� E2r��{��F�C�:���;�7
�y*��x�F+^R\oB&�Zc1l�Y�V����`�@�Y��8C�}��&�<�#�[������0T��D!���������MgNj�I���o�Zh>DV�P���_&�e�;�\]����"��z����Ya�+G0S,���,S*�%��
Yu��'�,���^��|tt7,9���5#����
u�Hh]����(����%i0���q\�f5e����Q�H�� �k3j�xi�1�������Yu-=G!%|9�Z;��Y��%!���	�o}QP��,����,�/���Z9������g��(	b�%�&��S:e�I6/�k~�����PU��.�V������J������S�y��&�U�LX�Z������|�^��_�d�a�N_���y)�C��t+���$����Jp�@��W���j
a���-�����[2��\�����O���4����5�������U���Y�X����!�h)"������i�J[W���hd�������m���yA�����W�yJ����'���<,��UJ#��r������k��\9S�E��>���U�����j��
���ldCWa� 3:T���q��|8�t����~��8�P�C���s�:Eh��3���y�����	ifV�l9��t�V�j�^Od^�q��1OT�g?"/�x���M	gvA��<~"�-�M��Z��zkPo�Vx�c}�����:���V}��������}.�#f����cz����\|��%�S�7K�]����{��W���"�B����3�c���$�bd��X6
�,bc�+�wxp��~G	�0���O����x���.S�y{vT�).$��1J-RK�?�jl|<�6`2xP3�����hd+�{<����h UO
lj�j(?�5�	-�S%w��>��k��@��m��lg����i��#}O!Og���R���=����j��1E��L>}p�>#���� �����Z1��5�6F��Pzyd�����u���q@�2�(�<2*{�j�C�Wnc��`'�����,���u�k�05�F��9���J�^������V
�[#+�(�+5����Dg(c6�x��pJ#�&$��F�kBi�yp:.�<��%����L��F�l���Q���@y�YC8`��(Vc���*%��C'���l��N��Z}�p�@�$��v���p )YZ��V���U����
�$���9[�45�c���D2m;��)w���"��#j=*!�&����^s/�d�@�������M(p;�z�eH�"��=j�+-3��}�X�\���_�}?Z��q
�$���]��Z�qJ��{55?��?��H������t2�}��A�q��2,�������m{��������Rc�kyf���C��T�@?��bH�����<W���
���>*��~�%(���+Q��=�Tv>5�2��gL����>���:D���+����h�C��&]c��������/��H)�}$����l!�(z5�S,����U?��@M
����+��K��9����k�'�3O���\��3+��^vFOyj��*,�Za��r|&g<`-�n�g��\V�.G2mY��
����z�1�e�O(�|jF�d�0F��f\�(�qGrEJ��Y�9y*�f�Z_�|����c@�g��~UpY^��Z2�UD;�S�
�����n�_0<>���`��6����s�l��QY\D�Nv�D�M�������z/�y���d�?!_�	�a!^wU$D��_�����[�d�VX<O����"<'��c��>��|X��SQb_�p>\��*�h���#�e�2~��!�!��?ew�E�a����tO#��Y&��~��[�����M��u�
�;s>��1I������,�!!}�a�e�����@S���n=�O��	����� >R�Tz�n���m�;����%��	����&��]����>k�A���i�,v�qU��U�&��C~�H��z8.)�Y(l�o�9O`b���Gp�V��O�<��6m#6���IZ����� n4��2-|��J�x��D�k�u�n�V�$���O��{�\	����HLN%I7��"��t�Mb�p]dE�Qm�"����s#A(l}��E/B������/_B��*�od��u�(
��/K���wLV@V��0���k9��@��_�����cg������D��y�g�Z�*�N{�G���4E	����
3$�%"|��g����Q3 �oc��
7��&�k�;4�7o�o����9���������X�`�o�[�1
r��_��4|����l���q����~���*�LX������'��������C�HW��~(����q��������;�I���Qb���a��,����`��k�H��x�g�gQ���|�&���g�������-^���V�Xy���S��5�������%�w��wlw?��(hs�*��
4�YQPw|���3,l,?�*|��#N�-�$�C��\T���TH�[���$:\����5gw.�%����Ir����y�8[������h��$V��?��0�l������*�+�x���T��Yia5����{5���!���Kut����K�JPS%��'b����vR^�o���"2F�]���Mc��3c��-�k�^?v�����)[�7=|�?�4��q�����
`"�*�s�j���k���������H�����:�
w�]�P�%�<aIu��GE����t	�����-q�Op�T���-�>��4d�V?c�!��U�k�?CtW����^x������\?1I�
KB[����%aR�?�}���Z�����iIW�����V0��8}X)�C�����A����(:��jR�����?��;�6W�u�Z�wl�_�zpQU�*���{��2��gt�/���.W?���y(AA�9p:92�2�1@�G/���@m|��j�%?y��
~/������J,����c-������x�'���,�<{L��?��*@����,�$U�`�%b~����H�,�����^�G��8�JW�b/r���6t����������b7{�`m8�4����D<G��y${���C]1��t��d�0�k4z��~�����b�;I����R����Z�R	��
�j2���Ea^T�c��g�!�%p��'s��]�P���vb�:6��1��D��K�+�J���SiM|o,���Pz��&r�F��9�o���hIO�)�P;u���e��R
!8�Mci��n/��H�x�=
I'�Y�/����V�����ZE��p�|�02�8�j�����-����q]�N�Q�/���rd%�v�5�W�P�9�}��Df�������}��Q����Q=73t4�W��T�8|<�[��j����r��8XP�5`6-I�h:�P�B����������E��wvWP����x��~��,��-������z:>�D�����7�cTn����� �:��t���p���9w��'i��p(�D�w����3O�<m��it������N��)b'f���i�o��������@t7�_n��k���v{������e�a&�����D���Z��F�.H���0�=Ztc�����U�)F����s2���f%�9�Aa8g�&_)i�������x_��)�P�]�
�m��n"?�����)�n���Mt����D�H������~H�v�S�&�`4tY�<�B��roT������F�l�PX>����h8����6:�p%s�e��7A�Jlr/T
������=a�)�=�F�������~���.�l�}.lF�|�O��@H�:l48���\6��eb���c!��3I�=�2����$��5���#�IO��&�����s��n}y��b\�K�n�������;>G������KQ\l��/�	7p'hGS����6.�	u|k�_ ����(��y�����>-#��#ZlA~����J�x�	��RQ���`�=��d;o��&Q;�����
I]���q�6P����$�R�.�S� 
�*��o�� h�^�Y���8�r�Z���(u��tQ+���*i��6���H���,������0�M*�aR�-���|��y��,���=�~�~E����}��1v�X���w����,'U�L�v���k%}[��X����n�e>����"�S`�0�@����9{�\����x�/�������gny��j}�����$���Y;R�������2���8���3������b����������j�z��$�K\�,���.�Wz��+��3�P��j�n<9_gv���e<��*�a?���l_R���
y�����>�u�|
Kn`Hh���lf@�������������~����F�����������V��Z��9{�h"������g�3�`�M��d'����v�`}��N|�?	EF��"gC�/�,��!���$K��r�Jo�A����`�����(�e���KF�����O��5�_�dVM��p��������$�W=��x�d�K������g�3�oq��$��Y)n��������F����c��)�����(3��#��s|,������l��#x�������L��e���<�kr8��lM�/���r`�M$�/��h����Q��@d,���M�`*5f�mX[:8-���Ym��H��^9Gd�*��c;�s?&��,5�IA�S��qnS�P1bsC_)�Xpt�L
��������.
����n��Lp|'��i�8�uE��Py<�1f\aNIZq�&p��s	).D,b�y�{q��]��<
��c�
���IF��|�,��
�
Z}K�����0����2:������'P���������Q2c��r�,}>��'i��[{���n0%�w~��hVs��l���~�C�Ar���tL{�B�
��n���s�d�E��]<�i��~�X�\�����x\\�(��gx�Fb��61��L;��Y\X��x��������d`#����E�46�2w���(8-�%l��������J�K�J�h�W��ey1i�*�Q�����7�TSML��2G�;M��e�������<T��CV�f�=��������[�����}8i���v��G�}��
I���AZ�2�~�� ��Xa|ur��$d�adU���czbZj��sk������+-U�M�����?�O>��v,��Q���H���QGH�])/�����h�����&>C,���)��
H3�R��v�N���AaX�j��v���exlb�vC�����N���,�\��d�\@:?n3�u��GC��y���4�4��rb�M`y:���C#�q?i���T��sd�\�o����`�w����l�'����y���Y���*K��?�;��^#zq�y��{��R��SCR�����"�����L�]!u�����;Q��f���j`�k��CH����:���}�`!�n[�Zz������
UZJE��_B�t�����[�������_S{@FU���o}e���*������a�>�{u����}U~��#?N��@}S�q+��G7�,8����\�w{�v/;�mYi��a�Smx[��7�����w�c�US��=w�H���������3e�V/��i��Q�D�cL$xg�}re�e��� ��"55�4	��S4�H��p��`��������J���
�Y���\��:���(�u��0�������C�

zK��#C�>t%�)?[�}0d����t���
0��C����#�����}�qW����"L+!�<��U�:DC��SWX��N������]T�v{`�_��7�HB�~V~D�:-$#�#��x17��4x��?�,Y����J�Qmv��wJ�X������[���U��Feq��^m�S)�������
�YQ"�Q�-g���
MwNS<�5`!�p|+��ql�) �=<d��w*�W�t�9iy��3Gj�X��t
v�C��k���lN~�/v�FC��@���a�.��s����U����)���l�x�tS=9�{GB���WKQ��JL_��!IW��$���f��6�j1������)o�k����~u A�Xp-������BR���22�d&�.���>D������i\�,��g��FKn�u��_7��������N���V�q����oI#�����}������+E{DV<���������!|kJ�Pm8�
��tl��w�4QP
kp��,l�F�L�L�������@)d�@�_�;N�-�*p��I����?=�1kRJ3]&XgH�8/��?�TG�lR}�}���_	8��_��)��t<�3��;�S;�Q�,����V����=����.HJ^��C!FQc!�o��~���`|��"OXx�`�����
��M�=}�n���*�!�ve�@��_4)�3��&���y�����G��8�*G�1�(��?I[��!5
�����a�@C Vu�Ga6���&���X��/��)K
�Eow��$��������6�/����m��':�	II��8$���c�en���h_{G�=������������I���z��������8�Y��Zr�p��R\x�u
 +�2��w�4���	%�X��y�r�!���n&�w�������x����N�[`I����B��sSTi6!��X��f��C�r$��a��J5��rl����Pr�|��l�e����T���z��!�� :��m7�n�E�i2abk@�4���O��3-�2p&������C�!��W��d�q"�o8���E���c��L��S�(?r�@�=o�����`���f��n���r6�J�[,}"��j��#h<��alT�)E����Fr����|�����:�����Y@n	�u����J�K:2���gT�m��6��"<�=�.�7���e�l�3=vww8w�������B�"S��;T�Eg����_#�:��<��Q��-8%u�8��u���ID�NY
�Uq���� ������g���I����h�PTC���uK���vi�aNo�@XG���%��C�!�\���_%2S���>���E�7���"������9�����m5������	���R7^����x����)8. ^�`E1��He����V�`G���qJ�|6���{�%K��I�9�����Y�����a%>�Y�z���h�S�A�qU��/�}���%�4�\b'c����� By��H�����P��]{jF��\��=�28���8s����� `�z|�7��t\]�[��z��Ysoe�g�9�i@r���_��d7	�{i�P�\8m-��Ij0����.Wj��/�\�-��=gR��[���K�������������>d13�k������r���Sc8_���|�����[|���qI�[(��N�Iiv�#���/&����FT���b�W&M���ar�*N�����5���JWCM��n���EEw�<N��o�>������\Y�]�@�K��2V�v}AIc��(3�qc2n>fN�x������bp�*kj�1B>�z��`��~�����[�Z��[>�m�'G�h���'�!P7��r�����-&G�ww� _^����:a���cT-��^����8��Y>�[�I,Q���y��?��Y+� �$���H��X�x���|��B4R��<;�h��:06�b��m���g������-��-o���S�hE����'��9.�0��.��o)��a��!�G���9B�����R�Z�5rn/���O:Bq������WP����#:�n���#W����y�����8K�'���Z���}��-b"V�qO���-��eso;�p�S����6$���������k�^�$2�_�&�ZQ��*���}@!�����mc����}~��#�k�����/�'7\���
��1$���)eb���T�X�GC�$���L�Sm�����d}w�c���
�~]DAg��������	�;������������q����P��-����_�^����AV-[e+:��hz�0P��b���F\���'{�a}�tW���=.V�i$�v��,C���2�P7������c���5������1� �H�EW{B�ds�*�����~1��/�??
�1Z��F��b
+P����{�[J��w�s�u1*ncN�@�Z�_
,��&��{/cr �*r���P�;��~P�X����l�L���'�e~�J}~��j�V1$:�x�o�b�i�z�����55�{s��:�y6y��x�a\���txj��t��l�J�z��LM��,R_W��wA�x��U[�J���R���V�����1�4���������BeKS=�4���reE]:
�����b.0�����<gYF9����Z�
�_���Pf��lS���������5�����_�V��9�G���;~��c$(O{u~�m��=�����o�#�\�
�,�� U��	y�������~�T�h
RX��W��8a�n���y�nKw�bK�m)���o���]D�/pl��<J-���GY�[���NW�+����
@t��e����#����6���GI���67$�����vtB[�F����8u��L�e�s���������L�H`B���}e����c�1���%�j�H1{�sx��k�j��a�P����	d�]�uO�`���+\l�Y&Zv�����;����|
l3�����J+<��
p�����E&���\=��N����U�j��z(tg����qx��.��n}{������S��=A�H~�y�:4l��.�AQ`�bVB^T?�cF� ���	�ntE1�\1t��%\_�-���b��_[k_^u��Je0�%?��%����8��en�2�\a�
>T����-�M�������������;P�X���;R=LO����:�q2
zg0;�GS5����D��&���H�9��e�;2k��y�g]�l�Y@�
?�3�O�-�0���
;X���p�(~�3B�}�=^"�����rL��Eu�v��Z�\����UW<hJQNb���CP��V���gb����x�_����y�%j�^��;��o��_�z���(^����!V�����f(?����2���,���
ux�x2XD������[.]�g��R7��c�xw���Yv��]�G��q�g����4Bg�%P�o3C7�
��V��KAT�A4����D���q~������v��[$���������0����.�����\R�!4Y���3��S��0�g
k�3|&��_�0�JU�T���qt��-&d\���'e����r�����y�vX�*CVx$1q���q���_r4�2�L�������C�$�f�:]w�����d���A�c*����M���_�)�c�
��o���7{	���\vH�5���6[7V��7��kDt���sr	���%�O^[�'�x)�O�����{�d��%ZF��D�.��}��"��������*���+�1�����L�N�U��j���h���L[�T���V:;�m����oX^U;�O������_�g
�R>�Mj�\�STqQ1y��]�^�����j*����
HYA$HBGz'���{���^BBYz/���*�5R� ��Tin(n9{���9�39������������t"�o����?p�TN';��B�-l���C�m��������V����o6�k�g����+�a�-��(DO������V�$<�m~C�*�r.��{n����	9�bR����[��a����yf�/b�����jS�JJ��p�E
�����Y��E!�R��F�s�����������+��9�95�nZ���K��WLG�//^EP�����E^�0�mj���w����o?"*���O2��W9����[�6�2��x���P@�a�h��;R�01����s!�7v��y���w����b����W�L�\����)�����4���2#
zD)4��� [�V�� o��F��I���{���c�5��D\�p���~��]u�x
�>��j�\`,����O	�BS8�cd���=O=������HL�uEC	�8����)�" ���~�c����J��;��K
6I�����+�?1�2�_�F�8u��6�Q�V����1
����\��&wj}����O��������U�$/gM����D���e���=���g-���N�p��O�)�&o2��'w�c�Yp�/�0��j
�BOF_�j�W�������Sfu*6�:�-�y+��U�/C�)W�����F0��/�YC���+�B�-�����e4�WO ��
��8@�t�d�u\�O��X2�;��-�������8Ec ��bn.������`" B�"�A����F	��k�I�.�Fh��=BS!#]A+�W���+t�Ei��pe��s���@`:i������g�6X��+���24�X��\#�B����g��Z<{�|�u`�t������A/u��������<m4f��th`j}L���=�
��F�~�:$1����q�i�h$��-������fF��|_�0��2N�B������V����T^�����v��X�M���8��a�9��E${2g]a��q6�yV��5�y'gCeT��5={�S��I����J3;	�|���������<0JG�|�\���3$�)�#�[��q�
���?��.�n ���=��������|L���m�z
�a�U�������B�y���Z.z�$��qM#�����:�.z+J&|�fq-����:��l����<�{��M�����-�Yw
&u��Q���������N�
n�%A����]�b�{
!8�K,��������O�'���?��t,��o|s�,^;�_G���c���\#-�����y �[aEyD��������$p�)��7��^^�h������H����;TY��7�\�����J����
���&/�z&�r����e*�}��@���M�TY�z'9��Mr4��
b�|��yH�����k�I�G^�]5�AP�������D86(�\n��������)AZVXj�r��qSs�3��|�m#�������l�~���@�/���6�\��5��mjd�P��P�3���������	��ou�0L���C".
q)a�,u��FZrWQ����<������^�J�����"���8xk�YK�g����V�I���b��:���1d�����Lt��y�'���)b�E������>�~q�0��{e��?�������6�q���C��G�b�N��,�B� oH��3�Pz�Jo�������4���w=������/����"[�^���)�&���E�/)�M�P��,��k����]xNR�e�}�R9�V����6BZ��?_�_����������I8P��#]�ke
Y.��V���^����a��Zr��M7/V���������������2{GI�y��2*��,!�"3n���3�����_����������D���
�;U����3��C�4o�%��B�W��W
����r����G�p"Q�S'����D�����s��{z��V�=���uU����q��,,�����joT�0�����:X��C��[r����vJ���-�m����`�'k*�����~�@�Je:�����<��c��
�J�<����$8'��p���/o������1�XIV�m���FL���	���������;�����J��j�9�e0J���2����	�R�v����]�W��M���M���[�'�1&W�l��Q�T��A��Z�u+�;aA-�fd����D ~i����3n��� ��4�5�`�z"B��$�o<��&����Z���Q'��WP�vk;K��C�3��&<L;sk���x�	��!���JR��b��x5��u�����L�!4��0g��1���q|���M�*�2L�;��;s5��W@b�P/�;�Tl���T��i�L�mp���,ol���v�X<>p�:1�����cib*s6�5��F����p��Ha��6?#� ����R�,:�>��%4��cPj��qmu���/C������E������)+���	}��|��`%*�k�����jmHd��C��`�������}��`r9c��m���C���T�d�W��\R���PT��!�~:���3)��%��LX!W��������� ���]Q���$T�M�����W�Y
H^;o6I.�Op\�������RC�����&=��hKXcCi\��EaR�����CX���1'���v(v.dQ���%����]�������]]����$CQ����2$h������{.2������u?������&�6#���W}K��4vb�=>�pz3�Z>���K
*zZ��x�
oz��J�
)��}#����Y��q���c����j:�N)D����mf�>]�M�A ?�=w���f0�*�����u�����l�p!-/���`�O)�L��Y���]L�����,���-o��/�=��3B����C�u�_�!�Y��n��0���M�x
5%��uh���{�A���H[kc"�0�5�Tl�T���5��������fUv>(6��o��[�o�=�<�*M�V������{��G�������
�?>���F����w��E���o���=q�6���!��Ad
H�s�$��R3���Yx�LPM�51�m�8�0��"��������(����2QR&T@ysx:��F�64���]�������q�3x�����"�<�����}�B������tCM��12����Z�x��"c��{��D���m��KtS����MP����
i�9���^�@�:D�;��:2)�����:����j�����&�5r5����G����3y�8�h�	:���j��	��D�z��h�x�#ou;�>������O��	x�������"F)G������w�`U����x��VX�e��/�^�����5�����U9��1/Z	v���G�U@�co3*,}��R������!����S��lu��BL��[���]c��o�������x��1�XZZZ����uZ��xM�-���
�3D��JS���~�d�I;.���'5��v��;w��aR�L�*����|�6�
�n&diF���$����ntID�����Hm+���������������B9F F�c�J��06�ce��GZ�/+wb/u�z:�J�g������%T��:�	��#�D�\�G�������f=Iy�~��g�����WS�����?�)
�����>8����'�����EN�2���<��_3���)�%b[<=��24qtZ�����������[�!���p.e ��93fS�y�~�v!i��E��<��������'8_R#��U�����oJ��3����R�#_���Q��r�T�m�nJ�B�T����mD:�U��d�@�iK�T��7��F��L���v:��w;��� ��:j�jE�?�=$�_8���6���HV�D����a�t]_�jI������G����S���d�&SA-3Vs��,�G�U�<�5���i��w��V6tH�D�X�!_]�T�K:�)��t��e+���O������s�s$��9��t�$D!�N.����)��6�� ��$Z��)������l���B���Iv�}�;��S�W���q�&�	M��X.��~�2��M�7H��
<�o����|��[��O�,��t�u��
��n��P�8=�8������h��}���V�]
Mx �t�d����g��"������n��G���������@u��O@�a��x�_�@�]���Z��v.�(�5�<P�v��q�N���uE%�k���-���2\�����;��	e2m�%_��Q{?�c� ��1Q[�6w�j�%�������E��5�iH����$s��kg$Ht���@l�SU�_�����.��ZNQ��mPc���� ���vT����mlw��R���<��8��z+8�����C)fO��C��3����P?���k��� afb1��3��w�N�[�w�a�Q��	 q�c%_4m�fB��UV���z�������}����-%��t���Bf,5���R��������DVo����2��SkcK4:
l���>��O�*�K+��g�Y��z	���u��L,}�te?k �#��_'�s�����]'9#�����R�Y�L�����|^-�9~V3l��� ��
��yWv�}w>��W9wO����R���^�
'}\����S8�5@�������d ��2�8u<L�����2�A /T�w�u:{J�)�C� �D#�y\f��8M���gs����$tE+
�~e��p���W[��H�J��{[q(J�C������V%h�>���|��3;�����U,�9:�-��k�x-���z+�M�����du�`�<$/9��d�?rO�_�\6��mDLTH8r����rU-�]�����Sbm�i��0���Z�>_0]����8�������\�>�0Syfr�L��D�q��Ti���Ux�6?Id:����l���7K8��uw���-��on�
S�n]��d���fk�Q���wRt&fR�F�.��S�I5���,Zq��d�.���F�e��D��A��Z���S^*����
(��nUQ��T{�����x�ik��"���{_ M��O�A���������P�I����xZ���3xmc�=� :�r�K��8��fD��Q6�@�Dj*{nN����N���U�:��V�'�����P�T�����]Y�U}�K�D�������f��[�&
*�BkY�������V)��~���e�	��y��#|bY���i
�F�l�f�����r���k�G��<�HG#��
�F����{t�0��h�X���y')���.�2��b���:���5D4T�9���B���
<4�.0d)�f�P$�A
����b�L�����T�S�{���|GC�T9l�\D_��y�,u����!}
�s�C�R��Jp�;�|�X@T�it������D��R7�'�fbt����U�M;]<)�<��`^?����z7�,GhuQl?���1�����.In(P9��voBd������Jv�Z�=�?�d�����}������x��A��S�����7N��J.�6L��:% ��f�!n~{��������n�onU�,�D�����3(����8�� ?��e~T��&0��M���^�T9��G�t��[��H�,�J�^_�_8Y��V�n���Gx�w� ��*Vy��l�V���]�{1�:��������F���
S��y��e��H��������1��_�9PiiP��9��f?H�6�X��
y��"����~�7��3��U����!F�y��CIt�cn����<�P����m����~KbR<r,TT��:A�{��Jy��W�_�*1%H�.����?�����=����+� �&����t�%�q~�����+���A@l�=���^�@����T���`�w�<H�[]��������n��b�v�S\��K�2��=����Y\��D!D=�B��RB���!����DH�`�
� ]ZJ=�N3�u����@HG������H�jy��/~���������~p���Mk�o����@*�L��PK,��VbM�-�:@4TestArtifacts/test-case 6 -t1000 WAL ~4096 bytes.png�[UT\[�� �k�����pw������%Hp�	����4����L#��f>g�����9u�v�����DE&D���A}-%�v	�"<��#��_C�ZL����N*���a��WLn��H&���Zq�7d�A?P�|1�pZ���2��[b����>k����:�E'��f��	����_!�"30X���&��q����$�
����[]�g&�0
]y����n��q[m�+k��ED��b��>^.��r���D �x�/�tf ��_�=�a����z���A)�?c�������E�s���s�-�2<���o=����>S�a/��W=�s���Y(
 ��6�x#� ��%8�p�YI����K���aBh�%f�frj����~�~L�����F�p6MJ$��,����)}z��A|n_,z�}�L�vZ���*��kYu+/�����	0D���e��a/�^����P�k��R�m���>�w���]D�>*����*�22��p7V#!�"��H_m��y��(#��~�d����	`�)�����L��q�!�0���Wig�:��_��nM6�b�CnH�I��+6K�$U�/����(UfT�`�	��3v�������4�.`!����c�iq2�#*`�!��� �H@��)0W�$�)kd�{�P��X�1�{�� A���n��m�8�(I=c^�k�����/�B��J�T��W{����h�X�6��I�=~��BA[N+�r)RJ,����9��|R02�t(�)��A�#�n�r�B1�o����X�%w��{���q���8�$E���_���v`G��g��|12(��Y���"ePz�B� �@2�I�Y��F�j:�h1pq"+96'l���������
s�����Tb����MYF����C5��iX�"7"��=k��=����`�dX��m��>����M�b�'A����xHq�����b?���p|QP(`�=�/���Np=C!�������N������6��kl��?��97Q[4L s���
��&?�=~�gRdP�@�"2�t��~I���U#@=QcN���C������jA�������n>'~��0�W��/s�GY�PYb���7���*��S�8���,d_�,�z�G��������gF�QT��o����kh�����,_2�� ���2;���k��y��%l�,��q��*�v�N��u�eg<�#�b��)��k�6ym�M��9Z���e~���W����y��ypg���9qf>�4�7���	>S��~���
(��#�C��v�<+�(G�5�n��z%vg9���|eR�{��,<_�>�DU��K�������K���n�����@�p8K����v��=i!��"����v����!f$��`y��;f��/Y������������������!/���Y-��$
���e�&���I]V����2���@�R�������h��F������qo���o���A.�d	�\�v������ZA���j�U����#u��������,BJ��O}]���HO�5�x����- �
��F�5�O��T="��.w��.��b`��b��meJ���8��U��Fs�K�Xg�����`�H��Tg��&�������rx��)����.Z�\&��p4R�a��(q���SY\�F��z$q/Y�!�3�}"�_���?�������7�����>H��-��ZI���V�v���W��7$w0k���^��~S�S.�%��D�����u{�����/B�����<����i�(�,X��������f9��+8���!����G�r���hz�3
&���J{3����H�R�o����
���5�������-
r�� ���T��+���ve��Y,t������[�8cv��*��.���m�A�����/B��m���tCJy7r��Wb-h��
�J6I�S����J�����������F}���?��(����������DF�78��"�����P���E!���rI	x^ ���b��I��Y36�K�6kO1��rC�E��z�
|�;}�y+l�������L[]U�}Z���(����7���!�-ZH��AJZ����^��H�d#RT)��i=lMS���n`o��[�FN��w/�(���;j]Hi���������i�����C�~��C�r5S�R�[��v�~m�������dR���������
�y8Ri*�4Z�����-����lX�P��=Jy}/��m�p����Ut��M�1���s<�	���&����1�V�c�G3*��a��	\����d�����z�3m�#y��6DWSfjX�Zwj�{E���h���~U��a�u��k�>�T�1�U�h�P�WX;mjh�;r������,����yaf(���Q��"u�;�<�����N{;�/b���b�<�������"(9�����0�2"�h6�b�T��/
��c�e�18W�b�$�|(����FA�S�:T3�4����}?#����9���Z>��gn0t>;�
���[�t�i������5�����t��'�4E�B���K�h�g;yJ��?�:�qr;C��3@�	�����������������m�������z���nd�v�x��"�*Zc����?�����F��fI���B������J���V�t!� z�k���(i�G����@���~;�+�`������Q���6�b�9�����G��u���g�b���&S[w$-��~����Pk%�;q��G��Y�����+�H��������|�A���N,J��@�^���$+�):x���
=*z7KfD���?������CZ7Z�b�g�LR�ja�e����?�Lu�I�y�v�z�y�#�+yM�����.H�O�&d@���j�����h��
�����g�r1�s����%��<�{F��OH(".��6!|nRdS����:���By.�@M��@X�������!�����������5��[�#�������-Q���GA4&E��R����x:-v%�[���z;g1}�iyWW@��8_u�7]o�3g�'�������!������\�{��
�>-��;T������Wy�6�(���#�]�E|bi���4>����V��GO>�8.�:��9��~���s5�/��"}����f��N�V������_d�3\�M��usi�}�"i� '���rbPv��r�D�1��V���}l-�;��D������N�8_x&8`%�>?�qF���`�x�[I�Ke@��
CmCZ��t�p��=f	#��
�D�����uR���33�*!�Sv��&�<�7&��tW����~M�&�98y��i)�.�c�=�<��v?��z��X��h_d&�H����9*vo+���\U���=K��������(���nEL�yEFD�]o�\���
�/m�rvhT�[ ����~�r�o�,Y-��>]����<vw��d��)N�%��.�Ks�PY��c~XfR��O�������E�����������Z�7��������CU|WS~#�}����t!���}��(�G������R�r�Fiz���3�������w}/_P"�Z��������y�t���Y>{�����:��ip��]:wQ�|�f%B��N��uhF���c�>��s�~sJ����$D���"|�g�i{�G�������.]�6^�����sV�1�v�5�0�a:*�S�r�HVv�;~��R����[������+�Nc�H1@�3
���$L�(X_�0�>�F�c/�t}t�-�C!NT ����Qi��8I�qC�^?u���L�����Oz6�k��6L`�@��E�\�!�>Nv�T�������a�����9W?g�\�����r�v����fqH�
=KU�����i�����e�h��
����i��C�1�����N�E|����~��|��R�{�`����"����6�`u����)�2:M�qs��V���]q^�of�o�!�7�9��{�"U'����.������cG��X��?��O<�ip�/��:�l�I��������g6[?v�0!m�}�����
~+c���7)��-����eP��lz��P:]��_	#rQ�7X������v6;1�+��e��*��$� F�c~�*���Kg�����l�>��v��C�"0������K����$����*���73�DH�WU���h���7�uM��d��71t�k�C�-����M�q����?�5�����n���U���O�H�KLg���
��t��
��nw(�k�D��)�(�O
�����D*H��z!4sv�PI~PsEN}X)LjJ1��1��Q<B}�W�K?L��T5f�VM�"���s4����Z,�I�bA�a
,�#�O�kPF�U�Q	!�P�&k�6���xq��x��B�����Z�c5����S���OS�y��R�o-��Z3;�J��}��9}^�������Ll�#�J$V*\��*����{|�P�b2Oc{�����i$��?�F�#f��4~�g�O�\�r�����]���D@��g4;���
*�%%���P)V*�^V�5���G�\E��l]n�B�����~��z"6�pr���h��F1Q��Di��TO��
��G�����<�8Lk\��2x���������j������^����QAh�������$��9�O�	�oD�N~��jg�����)�k�2�n���vf	?��|@3}bGt�J����'^���&:k�������Uc�"Q��?gAHf����@R�U@`����P��lv��%;����7����E�����q��6�������$���|�s�\�Y�^�a�J��pX��=OA���~L����3��6Po���wB/�2���7	-�b19rH���������p�R����v������i�3���]���-~�����9�3t�^NO����R���r���k�g0���������>�����z^�������f����������r�z����5����JQl�����o&5#��8�ywIK�+���^���������C�@dv�8'������K�_���fVPm;��T�UT��[��P�g����agM�f���A�V;�S��&���8�}��(��o���E�����'}-WTC
$�Z������o���g��m{�sH�-�H~X���>q��A���(�4fK�6�KD'���~�����M���������B�n��O ��H��nj$�5���:vi��l1��k�N���V�����8W�>�R�4�����v,�=�3�QgZ�Aq��!B�r;z��z��RkLa��9��C$��9x���6��~{�>��{�4T�E�y��U�}+�"Vg�m��V#""�Z#D�@C.�~7��4��Hl���|���A�'���5���� ���mQ���Ay����u�d�Q�..��.]s���ezp�$�V�pu{/{M���k9�ddfmi�@��
W:^��������9��H�z5}bc��A�n�E�KAz�.��X?���u����X���~�P@-�G� ��9��hQ:�J4�������~�M��.��M����Ca�N����F�l��?��LS���:J�N
���������dC�|8�������e�|W�g��IK"��0��}@�r�����?P����h��y��lP)�F��Y'�LJ�A�����+C�rY_�s!u�>8�W@��>�X�������_�>��;�ce���k��D������C���+��D��`��e{_�:�"4t��Y�H��J��s�(+/o~��z��-�%�$y�c��q���t(6s��_^_�%��z�x����-&et~����hq��(������-�OF_�/����0���@��r�n��x$���s���j!��;!��>�}d*3�B�w9{����"O�+�A�,��P->��W�*X��Mq��Xj�@�����"\�^�x�yR*����sku�1��Jyc�K���T�Y�,�bW�&�����C���Y�j����[�6����U�����MW���/�wj9�7x:�p�����0�{�7�k���+TL@�6���S!��G"]+p��5����z;����Y�
Os'���pb�e_�?n�L}��n�Og�2~�������9�k6N7������"E��6���<� ZD�#������/*�j������xCtY���v��Y�/j����c2����
d��OS��^��X�R?	�%b��T>b�5$�[�p.o M�����\w=�f��]�/.���c9���#F�����f�`'����ZH|���'���Q��n5gT�y/�j��lSV��>�cC8F�#��H~8-)62�2W[���	fO����l[��������f1?�x'����f�{f6�8��fzXVe�����]
]�1����&�%����E�<��'�������X$�N�d��/�q�;o�V�%8:��f
G�3��
�t�`�X�2!�DB7�/�c���^r�2���n�P�:Q���
U��`$C�,����:���P�flh)$e\}j�����h%�7���i{
{v��k����di��tg6���������W��������4]�+t�����>��[�U�kw7��@���'wk�z�S��f��>������
�RM�_Xo
�t�c����a��������&m_p��M�H�zr��0�e=�$�\������N\���@�;�����qc�S?�p��4�#����S)b�C�b�~c�{�������pt����������!��R��S�������r����y�uR��kW�`}0)d�P�e1���gB����! c��4���E�a��0'����#bq�
M!M6���?ev���tq������-!�]���Q�}b=8�������J�j�.�j�����Z�*i�YgsxS#iI�0SA��
�g�B�O��8��*���k��,���\�����uuW����](:��!>�\&��������z
/m������z�g��5+yK
��z?2�Ih���O<r��)���J1��#<`n����1�����{/�t������V�8=X��B���/H�X������&*>�6����}���Q����"�`e���K������r&O>��*����;v����.A�|��YO]�mv��d���o�[��?T���V��03������6UjRJ�X��`���J�������\��h3���M��L��������I��Z�����U)xd�O:�p���H7<���2�;�=�#��RT����P�l������S����e��Yus��r8��^��1���~C����c���$PFm���C����\Y��1qH[o&�7N��n@	hI$!�mU�*Lbc��� R��K��������y�v��R��t����.������$���\OU5qBM:�F�)9oP���;������\��zz��2�}H9��r��g��5��Z�70>v(�P���6~�������ve��bGh=(��=8P�R�}���?�P0���%9�N�3�f���/�I�� ���-G��e)6�����K������3Y�_{���1O56���L,!��n!e�K���R�V�y��ph�;���}��mU��"�XpA;�P��{s��AZ�
Z6�	N��L�P�_���dt�I�,���}�N���9��7�K�z�[.�����]�2���S��SLm�&�7'��Cr��O��5�u}����!��e�����Mz��{s�u������,���z{�7��"��$K�����f@9P���[�>��u�]���~y��n�������2�����q�(�Z
�*��D��WYz�J��mn�g�����1�fS�mh]�I;����~#-}�AQ	�]�k#c�q�ib����\ga4`h�
�8�����_b��HI������?"�����m��E����)q�V���?�ecb��cf!��\��^��7���!�}��������ut�P������=��v�m	�]0�H��HK�mRd���A'��c��v^�������>x?�)r�k������J81M��9���E��.���o�~�!������Y�,|_������F$���^6X�y������Cm�P�k�fs����	��2���Rg����m����}�5��<
z+��{�x�I���\�K	������7]������q7���*{��7�CE��h`���`=~���i��Az4'����a$q���/(IU7��_���v��L���?rf_�b�� �0���������������v
:�L�h?�jv8����LE����m����C�$q���vy-���O��M�~j8���@�����j�#�(�I��r�x�E��j����DL-��[yb���Pt��O@�J��������tT��:��vq����SWQ�l/N���������5E�Ai�j3w�����]��z��\���~�2��'6
��FM�T��c��T@���>U��>����J���*0)�%M�S�����6e����)�|[a�2������	�&e��9#�����%q+�����:��*���bO
��o}� a�F�!��$�X%�
����Q._�S'�T^�%`��������c�bP�����mx<H�(�;I�[�{"���Y2R�lu��\��Mov5��?��$��}�����6�v����>��-��%A.��vq"�f��	4&=&�ey*��ym���U8{�i~4Xq��>G�������gqr��h=r*G���'�$u>��(r�&��9>|
���F����p���La�}�@j���4?.�W��\~�uK"�����������}7}����-�����L?��-\\�g\���Bh"r�w�{��%6�^��mW�O5H����3�+�����S6�������-	��lx!�$h5��`u���vbK,E�;_�>+e[p�������;m|3�����L�
�*Ss�Y�S�Y����K�<)��`
�D��������~!z-��.�}���z�v�M�i��l�4����py0�d��G[6��m��V����55x��g}�?�kE��;�	��4�.���7��[M���#�F�S��"�|���A_phJ*������!j?�=�s�u�	�bJ��b��my�PK��<d����a@)���%�l
���z��Ss�ih��f�/ro�������va����TRC����y���E�Y�Jd�N�s��^��!3V�Z�d��'(~4�e?}y����B��9��j����D,��!�G{����[ c��a����?�}�d��@�w���uE���>��O/�i����]�$M�3i�k���ZAPFF�����Ug�������7���(�x���*q�O������k�5���*��8G�V,�Z3��N��'c���X��F�=�>t�����z��!�+�����nw���CY&���zC@�w��fclN\Y���,�g[/�7�#��
�|<���?Zr
�B�sq�������~���q���#�F�4�������?�KD"`��w��N�g�-�<:��x�8��/��	�23�����N=��	�r���r����|��O�	����;��u�r4k0`Fe�������u��I{��7)�7N��)�%ue�Ay(���]��^<�_�73eX�T/��^��Q��-eX�����n�O��n�OJJI%���4�6�X�����cqG�����N�@��nlB���,s#�&�q�F��O�|����
�tG�����N����������F���p��gFE�}�h����&T��DD���$X���_f���j���TQ�������_�"bD�K��&tp�G�-wH���BM�=tqw���a�-����!�}d�a�o�;�K�of�Jt�w���K~��%����G�L�B���"���I~�s��������nh�#�n:y�4�����Oh���+W�J���T���7���BBZ�q����)�s�0�J�wX�wp�`
Q�����@�j=�,�����QC��k��{��H��"����x�S��������6�wO1������
LR:���D��E�R�6�O�v,�����.�L/��������|i8O)-B<��'��Vb���Y�IL��{��8@C���x���D�H%�-��M���'��*��D����a��c��W�6�CCM����H~���T<&��$,�O-�5_���n���[mI�>��;g�p����t�G��s����}w^3�&>,���f g����2b�Kac��@ ��F��Y2�zV�WO�JJG�E�yP`
	4�n�	��'cGYc��G�XC<�*�����c%��?�����H����O�X*!��`��8}�������>�
��,�_��n��e���i*.ig�Z��i�V&cQ��k���i�W��~Jq�7���B:�X�5��4�H�RL(� �R��N)���>�!��N)��#�>p8���T�?���O]��R^�)�S�w�u�
A�����=��U�������Q��k�V�v�lN�h���33���C����L*?a���_}��8Q
�8��!#\�2���� [eaA�����Xd�t�h������O��W$��l����	��)�F���)��X�a�&�T>>�/��2�Q;C� ��3�����������cB��*�:���40-�a�����{�]R��R&�c��jV�'�5��/��\M����OFK�c��Z\(��"o+j�5����q.�]J�,}|z��g�&�%�No�P��<�9W��il��u��U�X��Y%K�?������@�>o_��$!�s	�+��5h��������������T���j�����uK>�v��!!R`�UQ���k��R����@^�g��|��eJ9�8�%����<>V�W�r���r�,y!�������3�4�Q1WJVK.j�<�\�����1 LY�����V����"xKn1�2�WU�39L1T���S�3��wm�u�=X?���M�g��r8Q���b���zM~C�cRG[���C����W�d�����.Y�VN*�����V��
�&��O�*�����	k��,�v1�1R��\�Z$�q���������G��2h��D�y`��	;�/�e����<������U
i7���2��=����;r?or��wG�W0J
����*�!��i���;Z�{��%Q0�������jI�Q��e4��]�PI�i��:7����i���	:��������pP�B��@����A}p>��O��"�)B�����y��p�>��
���~�0z�'i�X7�N�'��@��_�$��Zx:���e)2��
1�c<I���m����_d�����?I[L������=����v�<I��6����_���������e%��d_��<kUPV\����q�*J�50�6 �	���'���=r
'�Y'������3����l6bA�1�Z�&$��C�5�����E�dfQ���J�����X,�b�q,ABCe���:�G�s���#�f���2�u
O���c����+�1��h�"�8��JD��?���fJ�������TL51�$[�X��Bu�S���M�����K��'��0�l����mg"u��Mb�fQXp3y�E�^4�����<M��������g����B��/������sk�	
��3�_�x+�65�hzqB�'�;�c�a���;\IX�zp*���IH�m%�����&K���W��e�y���5MGJ�Ub��"�RP�-:�W��(�U@��A�e�<���J��5�=��$�}����O��3�N^��*�~�T���F�b���Wf���[���0����3��,��W�I]L(
	��"���	���
�%J�"5J��AY@���"�i�@��������+���������3��}������7C�M5He�����(-��{�%_�3Zy�YT��gze0n'j(T�����|�G4��� ���]�F��\��:gB�@���I�{��E)���s�A�8�h�_x�Q�\��M���W)+��zZX�M�[�(�;��]��\����P���H��!�y��nnI`I�S9*>�I�k�[��%����o��x9�u��_�\om����T�u:5���T�T�TQU�|T��s	I<M9�u�"�,
��-=�~.�[�9��8��[�!{�!g��m�'U��
���N�*����� S��������o9_~���������|���If/:="nAo`��p��\��x����2�C���

� ���g�Z�Y���Brv$��Ng��J���5�#���r�4�,!@^�~!(�+�07lx���I�|�T�����;��B�*iy,�2t��Z��_���uB���p������8���&�i��C�M��	]"	-^�X��]*/���
�a������ne�|�y3�S{��3�����O�����$�wnB�b������WzY��L�3�GzPye��
�4��?.��zxq�����Ck�]1%���hx6w�h��$�)h��+�I�Jq�S���>m���t%��{��u~D�j.w�:���i�������H�����W�e���Z�$CC~=0�5�v!@�f�T�R��^���_�����7h�G�!$]%XcQ�V��H���E�%�y;V�%J��P�F�������(�����#$Z�E�rP��!��v��A_�U������N
�PB�^��	��L:���-�5��}U��/�4Z�6WF��7����?�^����r���?
����v��������;q*��o��b�&8=���I�����
GH�wC����=G-����4�j��9�6�@�o����T(��*2�'L���_'j�g�&�cS���f���\�������;9#�wF�e;.�6��W3�R�{����'��E6��O�����%��E�l��3����Mw��������f�W����v��I�RoS7�W�1��u����:�����W����Qp6����Z��}���^��,[��$$~hJE�����aj�������AGp���E�����������
*�{�v_����D�~N���N��.���TB�����������,�f��U�2��@�����x��,���g}��Z9rx�������?�D���pG���*��M^Y}NW���#����]n��[>��f�A�{���uv&.���-��*�����
]_�WH�vJ[�z���V����$�/sH������<�������K:XT,�&/T5�5)�������������&�f}��4y�'x���_���E86�E����[?���X+�����l���-.V��=nZX���azU����6���Og�9���%��>Qd�����;������;Ry���Yo�p0��"t�����g=����t�������e?i����Fg�Hu����n�3-G�{�)����V����qE��Jh�?������`"���Nr��i�6��I�~��OLVf���`kAN{KjOt%%|�{���#o�@��V�"���`�&��[��_B���:���z�}�g�P�XvO�4�yhr��	�{�`��������"���b���D��h*N���nr#S���H�2B���[>�b��Y��-8��*�0�w�%[�j�i��-�aF���<`�]���
Pk�<����[-o,�g�o�y�2�DE�L�O��^�������@���5!��c�_�Y>uz/���uM4�j�i�d�S��G��W}���������lT��;f����Jx����)*N��v��UZ����\(�y�������=���G.��]<b���	(��jD����2�ce ����5�;�=��W��Z��f)�r n:*o���\M�-��P��1�X5�6{�
��.�~����9_���0���;����+w����gg�� �+=�3��0��S<G��2��T�����p��>#�V[-s��4���5�"1
�_�(�_@��9����[b��G�l�|^]:H���Lv�B5����4�����m|�j��K�5:�g4>���1�xP��";W���gt7��{I|f��A�s���W���jvE���+d�0�$K*�y8h�w���~[���\���MRp������v���uG�tw]wV���0�
�����=<io4p<��R��1���I�lR`��t����_�^Qyw&��=��b��k�����)�Zc�B��c�)�����K��F�h���I�O��Pu�F�/�v>�O���ioO�T��&?�*R�"��@�k;\��]����)��f%��YoP���'Pp4��M��.������>V��BA�*9���������������|����&0�Q�k{��>`�x#N�;~�1W�h;�]h��h�t���1��H������7�)�a��S�HW��v�B�����OY;2�72s�R�2�w���d�����E������C�z\�t>e���/=;�j�,&9v|X���u���=�g�B��VHV��3a���
}�������������w4x��c�[���4m�Mn���0��E��a*��cnvq�D��-�y�F4�7f�!]>/���`�q.���e=���
!�CbN�)C�H��*-l���g����U�E���+�w	����������|���!���8]w}����������u�U��� ��:\�!�gV��j��9�U6 ��@&��-�����)�K�T�4h�/����1p�UX��B�����"x�9gr�nDqU������H�V%V���P���E(�������jM���^G�ub� uO��1��S��#KJ3������������-5�qr_���
�gFI�}+L�i���z�����������~��+��H�NJ�Xp��.`_�J�_D�8v/��$���J���zPHSD�����PK��V���HA,F1TestArtifacts/test-case 6 -T5 WAL ~4096 bytes.png�ZeTT�V)���C:����F��KB����4C+�4C������H�����n�{��k~��}�w�������j*rXd<�z)/�����K�?Z�����g��=x@��������n���e�����T~�����IRo$���B��R������V�E���NFZ����<���@��l�>����e4��1��jN}2$�7Y�)3�A�=�����@�u�`��������>�w��������d#�~����j�@����E�\�fw��������������_�
�|�I~���<����=�C0��{d���U�=���0�����_�h��.�C�st3T���L�QT���h�?"
�����5��s�L�
4���{��@���h0�OTeUZ����d	`�����WUf��

���vb��b.H%�#i�>��b=a��!�
�O�&��j�g6�0����6��"��D���`*�|F�^p�}��@��Q��> ��QNZQP��uV7�&K��mD���&�Z��Cz���!��q[����ag�0	�<r�HI�f���HFI4�_Q���[��=5@"X���T��\�op�:����)/�[[�i�����:�"��02Gk����!�.���/�����dP��&	�k�Y�_�Yof�v6�?��Z����X�|E�\��'���������$����&3X�����!���U�>	�PL;W�sCg�6-R?#�O�Qx'�|���OZr���J����JB�)�?�0Y���J�I�'l�M!���%/���y6�cz-=RyL���j��0��
��OW{*�D�c}*�l����E"EW��3C��F���V�S�T	��?�������>�$c4q�����pS�_N����=��c�'�i01���8w���������U�����Z�pu��T�B���3@k�Uh�����mc��IG��T�y����Nc�r�D��m����3��>���No��#����[���<01�2"�#@������&�����Cgv�g��m�!LK2p�hW�c66���uG���Y��?K>���y������
���$����pV���=�_����.��Ch�P�b@����:�F?�
q��@�q�! A{"<�tg�/�[��X��w���E,�lb���V�-�#3� �G�%�q@����U�-#�G���T�L���U��p�B��(���~����MlV����-��-��s�{�b �V�����tk�v����B����.�+�����u��<��g�O�%����[�B�t��!\@l�(eh^�KrN��M�49�5���l>`m�@�m�pG������e����X$�����	~e�;��W'^��Kl
�-�����?���%��K
����OI�J��nfOZ��h��BOX�$3OJ�=��p��YEK|oP����"���pI�����q���9�[�V�3������9��zG��y����3��(E��f+�h� E��3��*�����=�Jt��"cC���k�uo�K��xY��d��G�(���(+��I�B�Q�X���$�L�����7Nl���1EW�JT	]#�/y�!������4	�4���7�ak�JJ�_3���
a#Z���/������M��������\�e�/n���
4+���L?�it���\�v��Vv����I7Z��aK6@�F-�����W����(."Jl�gZ���E�^��`��9�'�������&Kg����q.Zo�������~9��BV%���� y���D���#"�|��o.��F)��|�d:�����c���z�+�)���G'.��]R�����z����z�|Xl\�����6��)��Q�Z����|{�]1��J�������j��cZt��/��������V"S�]U��K��R�K���&��y_�I��&[^�(+L���Y�X@T������z���������&���������Bm�,
T|$}��}���l���{�]o�a��C.W|����������P�[Pj��Kc�ir�o����cKo?�A6���$BM��M�b4^�#+�Th�@\��P�F�v�f�
z��!Pm�����}�o|�%�+��n��^�O�uR���6�mJQ��J'�+btp:G�����
Y]���U�]*��F0��������o?�#vr�ri�����z���z���[�I�z�d��G�MboI1��_!�&!K�r�����MDpe�k�>�!-_�'����;�����!����^|�S��S3/j���|������Nv5��B��"]ae5��Qx������-K������s��O[�-�i}E)��;����K�o>���������i������f@������(��;!!!�w��c�Y������h�����:x�Mw�S���T��b��>�u*��|k>u�����e2�Yc����LbN�m+�*��P�� \X�8=�DXW��n��%�MG"E�����`	���
m�eu������C5��e������z:�I&f���G
*[P��-�
W�����z��9��
�7	�Y�C�wC������X3��f��uB�%�����
�8F(�d�"&;������#�-`a/�Su��%S�{ae�lw��^N��.KGM1��a}a�%���i��[~�$�s_���[��0�������������i��2��A�p����v�W��|d%�So6z����O���*�)����y��qO[L��{�FZ�,�2���^�g�����|?G�D�����w���{���%qZ���%�����1F�82+�#G�:�4����z��1_��S~b�����-�M4S���w�Q,�,��j$K�*�^���z�|G ����z��a@bd�jE9����v�:�/("����i&cE���"�Pb����jI����3�c��q6�2�/��6.7X��+~��3��_A��u��0)�8����������Q�����A��<��a.��Fn{����M��I�_&�)
F�'���B�{�|0{�����!;�|�dy�FP����:�g��W^���O�����5^�o�����#�\���t��)(�5e*u����gO<�r6�;I���u�&J>�D�:�����`�� ������*�>���:p����"��g�}�kG��;��/K��:��|5%N�����(�������������nR�C2B�������k����d��\�m�e�����'[2���n����5
�jJ��J���9,��me��Q%��M��+���y���UN�$N9�(8Z���l�(�2>3���0��i� 
d���;��<M��D��E�DD���C4�_r��v^$�7�zX�{�u����zUe��=�e�����w������(��[�	��a��^�p?�:ny�`!}�*�-�y
���z�G���
�����S��wg����x)��9Iw��/h���e�/#���_�}e��)�n>�jOn�>�X�y�Wc+e��N��Pe���}�f����t:F�
k��GRK�0����2<
�}qD���vu�#4q�x�v��q���t�CDpy�@w���w�>�p����ud�f�Qim�(�T]jN�o�����}����$��u������r�"�+D0���?������L���w��U[���^
���,�����eI�:���l4�;F�)T�5�)���E��:+3�4�=T���zO��r��v�f:&+o�I��:������A�O\L����:�e&��{���v����LV�Q�~���n�V,Z��3m�3W`���}5�!I��Gv=�Fr����	\�ii���}
	�{�~!�m���(P����5��w�9�Xp�)�br����w �����9f�UZOT�Z{��'����gs:MO����l�WR������n@���B}���[�8��{\}�*��J�'6��-��/�[}v>Z%�O
'��+��88���,��>Z��i�<g�@�j�*�m(�Ga���`���n�Xd������9IL�����[��,|��Y#W�M�&k3�������y!��t�Xo����s|����	�T3�~��A��L<��2q�Z��"~?,2hr���5*��b���z�/�s5���w
n�m1�r6�3����7p���aPP�R�eo���,3�V_ei��H��YsZ���
����3g���g�77n��@��l���._�J����N��
@���>��xm@�|��T^���(����� h�|j�
�|�����Ho��\�d�J0.0����;1�	)��'�i�cq2c���=U&q�y�C	Kv%�����Gw~��](�������z�����P������<PLz�NyXk�~�/��u���T����t�X�LZ�@vq������5Y����gQ:�T������1�����9���~M]�g-F���B��P@mX������7.\�'�e/O�$B�}oI�|�RM��t?���06���9 p6�v�r�Z�P�o�r9��7V��	R_�������J�G+�
Gm�mJ����N�S�e�������
��L<�<��}FD!����b8�&k���6s7����Y)�?��H_���s��m�~�Y�=_�L�,����'�}���h7|�v���[3���"�E�H��{<�t)�?��q\�	�������Cs���h��L`�i�j]����;'�O\-�1g�<L�S$���
4���;����O>����=�w��c[��iB
e����y����2~�	���������%�B'�WN�{�E=F?�l<C)F���cyl�w�']����.BDD9n���o"%c_���b�L��P��7��E�T�.a�O��R����x����4����I����_h��(u�q^�����6!?�b�l�&��Fv7�����e���iwo�=w
�Ek�J�S��q���9�X���6Fs7�
��s�������I�#��sJ�HG���/�2o">��/d�^(�n�JJ���?�x4���T��f���Z��m���@K%�8�M���s	����|��W�7�,.�@ �t[��y�;F�����������/�
�O��/uR�����DK�s?�����u&?��#�������\T�n�`<���19�luL���O6l�`���Z:��X���J�#btm�U��S�<�������kF�K�RA(ffi����Sv�i/V�'�����_,?��o�kg��;?F('�D���sl��j���5�~AM�N���S���1���j���k��L*b����S�3.A�S�������/����>�e��s-NmB���Gn{�i>��=����/~�|����!�o����bO�����U2�5��f�����z��
Y)�����N�^���[�����^{\���$(��x,��5�\��_��}i'������r���������/�BQ4��N�4����[���Ir"��;�����<�0��D���2}�ND���m�"���������B��3�����4�
���>
����a��&�����)�<�~;}h�"����.�-��W����r��`Nz�Ae�z�8�Z��b�AU��D�)OH4�����u�fUO"���!����
�G��r�/�M������(���6�L�1{��p��������#�HW&�#�g����	�&j)?Bu�D�������f=6�3�W{�p���O�x�
�;J�xb|bdWJQ$���c�bQ����A��u{;Q��~������1-�Qb�y�Z�8|�����>�g��K���?<��H��jCQI�oY&zjv�$�������%�+�Ia���hY�D��K�n����=-���L-R4C�������=Q&R*�hU�D#�yz�N�qS�5U�@j["�@����,Z~3}����
P��\��y�T���P}�|�*:�q�3���Z���!(Ye��?�l���U���a�7)����T��X���8c��Vw�p�b���������gK^�zEk��	���}N�1���	A�G�WI����S�$��a��WHM���L�Yn.I��9��*�1���T�vl�F��#���^r���/�P�yxv����$t�b�� $�t�+*-��'?%F������������Ct���`�W����)A`�����@�^��hFC����w�������#��$��3�d��}��~n�_�|�}T���|
�6z�C�>pq\�2�]�0�ozr�O�
�ltK����=�W�	����v���k�/'��+)����-��J�D��N>�"�l�d?dF�9=CV����n(�*%�������N�������}��~�cyY�'?��0�J�?�`��r��QL`��R������r�J�t�AY@Y��������q�O�w��;�tf��`�]���`��t�?���E7����]��C;�?�X,�(���\�b4}�4�����:��	s6�~YM��z"7����=�V_I����h\��3���W���=��f��
�6*>[�.J�������	����.��wC��
!�%DM>��lwI&�i���������x�sT?�/�E��-����
����u�2�;H��,(�
��n���������r���G&�1�]��2h����u/���a�1���I�w\�����Omef�����m�	2�"����>�F�v|����E���K=\4�?L�Ti���&$q6:���.������C���$	�&���
�s��5���;x��-,(��9d�Ao�l���w"�=���(�no:RZ����'���5��qt����f����JE�F��{M�/��b�� �av!��pdw�q.BA&���0�Y��Z�/f�mh��@j�H��zv�f{��x����a470��n��a�+�-��
�[�y�l�6�������
��d}����6�Q�f?Z7c�C�[���_l��J��wIB�Z���W�.�,V��d�y��c�����o�����[-��x�����Q�ZJ�"�?�u������L/J�Xr�`�h��P��~����������g���6��3�����E���Zo����Yo1W�&���>������)2?��4�M�M�"��;�=�+{��Q�|�C�T�����LE�L����-���'7�'��Q�'�P�������E�����20d����c���D#��I���"�R��0t���a�(�}���[�P���#��m�d��Y������KQ�$�n�>�]�5`Y��
`���A`��������*��I�Z'��V8����?Al�?�m���]1�>o/���|'���
��/x�r(�gOz�i���u>a��_���"�.�x~c
��4j���[gd
Y�DK�<��=�D��`�F����~q��
/�\i6��V_N�:��V�	�v7L�� G�=������x'�������c>�n������'�s�x�]_NPM�C�Z���:�C�;aL�a��[��n����{�d��5�W��/
A3>
�:�M�y�����M���DH~J�B3�e�����G��Q����%r�"d=��Z�/]���n�p����m���6����^�K�IQub�7��0Lv���<�YCs�rq����@E�`��5�8i����r3�W�7Z�l�L��]�GD��F�)�����l~��/z���m�aG7+�$��2�����^<��,;^��I�C"�=�zXY��h����KPh �9��[����}�G&�!%
#AW%��~�1
�%hx��J�K�tj�u�z���:+�@�g��R�=8��OM�F*izs��[�����!I��-�����~�gE(��X\(�@��E���l���4�5�Q��[����|V��1e����M����z������
?�w�R;s�'<�ig�Q��t�G���B
K?nT>�5x�|�i��t���s�0.��@R��������	���O9�eu�~���	��V.\���O"U���s���P�;��r��/�����J��'2��0�����s�EA�b�I���ql�lc"���-���<m�y?9�u�7`~�u����+T9���%c��"��j��jb���r7���k�
0,�������2�$lyp���� &����<��
����K��o}k���M��c,wf��H�������m��B�W,�b&=��M~�����������D������w�fH��<���y(�7�X��:�T�~~B[��6��=�>�����US D.)��Q�?�QiC���%5���#��+wL[�c�r�C�!I�?��:�&��H_V��MDD
(p�Fi�#.W�k������|J��,�����r�Q��u���
"2m#�m��]�Km���w������r���k%�eJ���V��K��&Q;�b��Z�w?�����<:na+���?�Z4��"I�l���:5�Z)]LY��7��A���n��G�nt4GlI!zv300��R����1~�i;5���,P'x
 E�o��_������utM"5l�		��������
wVK���b�	�_����Q��yZp�(��r���b=�T�@U��O������H������E�a����9"fFJJM�t�f%��?�+��d��s��\*��/�lO�S�1��"j����3"%7o�U��X({�W�<@�����0��f���_����������
f�����*C=&s`������z>W����p�]�lwQl-q��c��	�jL��(��!��cw���yr:�l�+��6rlS��O����	,9	Mi����tg��=��8��-�b�h������wWcI3?��!Y`�)��&���u�rF�����n��C(=�Ow���P|�t0f����.Q���,��
���1�R��W,�>D�eB��~*-��/Y�&l-5]����r��P�#M(3�f��N�0a9xH����bw:8�z(�����E����`F��V%��^HQH��R����s��*���2S�[db��1�i%�*��04��a��Y����t0w��t�d����]���59|m�{���}����b|tX�[���t���Ai������
���{�
o�J��=�X;���Uj�
��y�o�P��=O07�"X��?,V�.D��H\�5�ZYu={�-8��d..��[{w���(�sg������E��������k�K(�	R�X����C]�d`�����#��R�
`��/��c0����v�v.��e�w�tB��?*�h����&��`n��Y� ��j�"��~�����!#�mv����4_����Kf�C��8z[�(���u�����%o���moD����������f��s���n|z���{0���A�u����y�y�_julb6kX�]�d�Oq�&��\�,__j������6iD���A��v7�e��
�D;Nn���)�i�����r�t/�5�����Wd�h��_4�\�{.���]y���(|�D����v������(R���������vHZk����Y���Mu}:Y]���m�A^9G�H Ia���\�����bg��$zw����'ry6���7M+^=]&�tYP]���(�Y�#��(ON��y���+hC6Gja������������7��>�]�8��*���BUU�o����v�%i��ZI��l����3�$,_�*�;�7�l\��Z���^��XT�KunIlb�����{���\U0����n�p--�a���N]��Y��W����C�q��inF�l�GU����Q��%
��>��|#F��#����
xaQ�?@��7�u�Y�M���s����pf�"�;O�j��/�^��T��d��b$:�0��m����YVY�ct+�MZ9Fwm��.����T��cA���hq�����Y"Z���������@
����?7.�CF�����e��:z�L��������Y�f����K���)�tA
��2����\K�XF�'2��l\��� ��,d�*{�{W����,D�)(��->�	~������@#{-�|a�b��?cO���U
���]Bb+,��������G,T`����0����mf���u��{����������/:k�w.��j|p�(<)jI����
���63���3,hv��AT��G�+:��+gA�.�+�s�y�6G�ZU	�+XTyJ�`@���������#L��F�%���Z����r�������3�?�z����j�q�G�^���b;Kzw)s��������
���a-�9�Kc8 ��516�6VT�s���e���M�����c�|_�]/�$��YL��������S�o7D?.c��p�md�j����e�"<Mn����"v2�
t4_���1����V�QZ����Q���U��_�����TAF�JPi���qW �
��=���F
�������0b���}�^Xa�g�D��4�����@LI��������}������*&8�1��@����}�B�3}
���L_�����������_~�S���?����'��!��A
��\o�i���t����Q
�+���7�7?7��E�]��0����l�:��qD�@���o��{����@h�4��_��H��/������������|��D�K��\e���$~�(Q��������<�%����<2��Y�|�bQ��A��P��2���r���T���;���;�����������f%B��6��08?Fu�<��5h�,Q�{a=L|���Q�;*9�����o�V<��[9xnq�0�d���@���/�	�Y��~����'���Y��,+�|H��=�_�g�(tp����������TQ�[�7�
lm���������!���UrP�s��x����M�����|9D*��{*���SSKn�oh��h`vV��.�1	������L�sd�3�C�
�������[F���*���������#��
��r�P�����R�u���~|���������jt��_F�j���B�5��< �e�����5����K����.���
�{��^�
��Y�t�XQJ�=�<A���0n-����[�x������>_l�����B�j�%p
[#p��cum�sfY�0ef?��$VA�<��
d04Z<���V(�Q��v�����4�@lYc��!��&�@:�K|�J`Z=���"���<Aa�������h`9���_����p�}1�~�:����LP�o�� 0O����"�Y����H+���MZ���Ry�D� �����/���}���\��jyv���Y�%�0����[?�\m�C;��h(W����MY��O��#��*�E�$7S���`����kJ����"H��5��Z���������_��m�x�{J����������������qU������w�y����|�z�A#N����]t"��%���A?e���/��%���x���:B}v�v��~�V
I1�g���p?�I�.(��5��D����Qj�&U�\I�7��k������}F*�^�X�_���0\5��N���y�v�}(�OyK��?�
�W�b�{�|���������1D�A�jNG��$Y�x�
��;���c9!���S���W@7x��h=�o�k���s�w.���1��������\����2)���e���\�����I��������������NH��H��b�����%	G������i�����u�wQv-=~!�������!1c���]�i�����A���^EV]aWD������x������gT�W��!K6�A��-S��^a��D�={�
K���$  a#�0dO��"K�Y���l{�����4����yO������=7xk�!c�d�T���*q��VJ2����z��K7r�h{kj�88����	E�e�_��`���_������?mk��uT�i����y;��������,��$��U������$w
K�u���H}��s�bZ�00�� ��5
��\�x}��1� ����'S��p�h��t���L�v�f��e��E�F�
`����2>�
{�>�-�A������)������b��f{�!��������Yo�&5<����r�0y,�k��j���Z|������]��MiO���^����l�+}�HQsr�B�@��u�sS�������h`�y��P�~g�*�����&��@�i�cc��%��O�����9|3���fd���,�Avug�����C�*���Ee��hL�F�K���X��V��t�`�8U����2����:�8�t�:�:[����	�''v��l��l.�=�.��?�=l�:��M*�R*�#��^�����=��M��q�+$��um�o���������|��5
�(/9}v���d�����CN��.0���g���=�i�N_??��[WL��kk�9��1���!�����X�t�� U&�RxO����	lc�������ed[�A��u�#�=���[����cd����B6�g�h�8���F!>Llm��~�1I^v�	R4��k�>��#L�}�w/�����L\Q�o8��s�eD/H�
����b���_���N�\Z �[�X����;�*���y�F��'\��X�z_�����������i0;=a��b�h��4'��A)0Gh"(�nS6��$���c2�o~�<�uc��$�6��m/������4�A]�l2����lUG*�g/�(��>Ak�����0���>}�0��.���){9u�~b��S���p���Sy
��C_��J��~�4��]��xM�����1w��l"�J^����'XV���#�T�A����40�e�'�����6nx�����FP1�����e�L}D��@�=��p�5'dS��(�W�
�f%�L�4�����k������9��?��>!���G��Tv��K����<>-����
��s���Xy��7����'3h4���3~�ba���������$|�C�3�1�C��,�TvZR7�%�'m��#�
�I��]���z7.=�v+��H�����:t�$!�p�;�
<W=y���#w|K��ogHR7����u4:��+�i���A?�T���)��������4�|��r�N\EP#H���S��`�������c����Wh�Y�>��H&:�H�Y�Y��$@-��0&"��
n9���H���K���`9�en�@k���#"<@f�?e����t�Ii����2m"m�|k����Z�� 1���.�<���������qi-6����/�?d��"'`'e����&�e3(!���"��I��^�6C��h��_.�((z�H>��l
4YD��������(��b5�#]q�!3p��-��+",�.�.3?2V,��}}#�%7�v�������(V�����Co4@��E��UA=�
:�pvE�T����l���|��coxw�
�:�E^���1%���(X_C�#�wC�<K~�a��6�*��lg��+�rT����~���F���fh>
	���j�f~�-#�_s���<v�Q�K����i������unb��e��T��O�g-�x�?�I������L����� �<�>A2�������&�u�[����bV�xy�EX"f�4��!,�k�J�i�"�~�5���?J�O����R�$HH�j��4�G|��!�����N������dp(gz�C �+��e�m���3���>��U.��+j���q���q��+7������MR7Jl�?��Kl���\��;
�JPr�S��}zIy�|P��-t�� ��f���(-���1�1�P��G�7�b�,L�@����s�3N�}�6N����/�h�e�9W��Y����aE5��D5����m=�A��/:j��I5�}��9g��:Z����F�+K�V����>jI���i?�z����x��Egr$x�s���"��\�P��tZIvdu�;x���m�D~�	2�F&����-�N���y`�$?���`1��I~R���������"�V�Z�!.c.�l���_���2f�f�v�*�W�'��������$n�Z�����93������C'{W
�s�|.�u�����2YR��� ������^���a4��*=���1�GRQ���^�Q�Uz�'�3�h��>��R	�Kg��g2��	�^��6����Y>�x�"�`pD�(��y��%�zhn��.��G�|'N�����5�O�3#�a?��S)T�s�����������`zt��i�>�#>���y>R��������X��m��f0����u�f*�6G����Y0J����o@����!�W|���Rz�e��2b��O��- H=���'�V�rT~���T���]��FK�W����������Z9��:��Y�� ;��	�Q��f4�C�C%�'���
}����dQW�U���h���s�?�P���;iRU��
��}��1`hv����^l�i�Of&��`���T����M����1��bn!4�]Q��^�d�.���J��zgRH�1����p��~�LrDM�@�����x����4���*�BPq�4=l{|5h�7���{�gf��/>�d'SF$6�\e�K�D��$�[XD����~�~����&?j�%�Ar��������_H2E�M_i�����<'���M}F������2�L���dfc�������S�1��TS�3�)R!��.J4���3�N��L�@�|�~��������9��`y���S7����{��S��KU���$N�r��v��[��%&d�.A�.'�n-���]�Q����2/l���s"��a�ri8��@6R���^�Z��Z%�����X���@T�wQ���l����=o=FdD/mEFZ�#�Q���UUJ��F����_#�NpN��$X��3E�*#���NC�,om\a��[����L��?2��I�T��Q*�����>���
����q��$5�.���iY�]�^�������	�F�Q���}j!7��w���G�w�F�����C�v��v[Y#u`���������{����
�Kq!1Dv�V�Il�EK����%:�a��Z.�!ad6�p�4��td��d#=��Z"��*�n��&���Z_�����(����T�@?�l9Mx�~q;7r0#�3>��c���r3]~�r���J��[��Lc2���H�=U�����X�=;�
�+W24	��=M?�����Sz�M8��t;���p��,8�M4T{�cO6�J.�R#U���� �����[wd���h�����_5��6y�Z<��-�[�E~�\o/~�0�r����.�!����\�//�t��sI�w
i����E����eh����V���-�o����To^+�
���<�9a��Bd�7�}��6���Y��f`������*����������JM�7�l��J�_)�|~1:���+���<N�R	�&y]�n���=5a.�.]�K�5���II��"��0b�y� �8����+#L��OKa��������}�����B��0�M�+����&�����g�����ow���F�(���tL�=(n���p�7������*:�8	�W�}S�D����Gl�H��F�0x��n��&?_)������c�?���g.E�%4���'AT���g��SQ/u����8A~T��,�jO(�����JR�1��D�xx)65X���s��W]��3�����8d�L�}n<yoEe�u�n�L�j
�S���r���pD����9�=B�}������/���j����8�l{�EK'��I\M�����
�`�l9�b��j(������Q�4��	�vl���G��.#E)��[�Gd�����]��/����L���X��s��Cv�����"n����,�
O�>v��*�=M#������%;��8����Nl����t%>pS{�DJZ�����+��=)L[���uAm�?��>t[�~�c��`�s�+cd)����t�Kx<�Y7�6�����.[�j��q���d��r>������?�����h�><�-Vi3Jq�iz09)�1{I������	r�KeN����7���(D�����%j����Z_n�Y����5ky=D�`Q�j
6fqV�zEt���Kd�D-�q�h^?!�F���ZL�Dfp�ma|dbja\@C���xvB]5�)'z��F�M��j%Q�����X&J�%�l{��>6&^\��$	���if�J���u��mk9?6����y���T�#�ZqJ P�����-�wE*�-�*�H���	��-��%��e�����'�EDhk:�wi��/���KyP���	�~-�C:���B���g���t��t0�k�^�<c�o>e��ZN�������52gq-���@�z���c�����j2v�S�@�n�������UE�.����k�V�/,0�z�o7o�O+]��n)�YF4�w�1��IZ�3V�*�I
p�*�����[
��c��������C�V��$���=��f�~3m6X�/^�&'4��f���5���M�,?i#	_An=�0�����R`�N��
��_���++b+��i
��p?��������{Nx��1�q�
�QU+�yh�PKH��V���U/=�A4TestArtifacts/test-case 7 -t1000 WAL ~8192 bytes.png�ZeXT�DBRZ�n��T�����C�[J�	�N���!�������?�y�G�s��{���^!���hO	�>z������G����������zQ����Go�D�\t��P\�G]Ooc)�gKX	aej��R�ah�S��>�}���9)��N�X!��r���6e�L����5���oC�|�[$h@�����[R���U�a���;l������|������Q���9���~�����4>�v**���(������qpph`M�������e(��~.��������/��~�Hp�6�������������������W��mP��?S��Gc��'	�}
W2a�EE��3����A(K!����~�3�y&{��N�
����������lN�?f;��m�5q,^>cM�a�_���o31�iPKUa�{(�'��?�����p��VH,^����r�7W!C������o9%���@q@��y+�7�mj�-b��;J�����'���������u�,����e�ZB��DN=�%�z��O����.o?��f"��tWP�7�m�:��E��!�T�n��)���8��`@"L,��4�
+�PPN��O��%We���Jg_s���W�a/����Wtc���<���LE�k�t��>@��Vpc7g����Q��	�>��4
��U�������W���"���hc+^^sHY,xZ�^z��F	��X��Na"8mI��0�T),�s��W�
�I~�:��}/��d��6����Kl��K� ����4��+����sKq�\�<�Jf����pr[J�)��C�&�z��W�3����bs���g��;�e
[���L9�lt�Hno�u
e1Hd2�+l.�tmO�4w>��Z��,#Ru���&��d��a� �1X3�b��Q
�%��T&IW9�A�xbQ�_�x�H�RY�3h�L��&N{���B	����j��W��A��xW�f&�4���wah.L9����EN�|E��w�BR+Q����%�7p��Iku2��!	�|r5T�\���~��D>��Xe��������q�F-���0��'��H���{�%��[#{����x�lr����w����`�;�rx���4w�r�JF���KL��b�	D����������oQ�E>���{?8�p�a��0�{@����D��"4Ex�
�aqy��b^
�"���N���Ny|�n�?���;Z�����������?_�2��m�)��'	�����DH
�3�2���,8���lQ���!�u����LL7%���7\�P[T`�;�a��[%��h�+I��+��C\-����'��M;�
��������Q�R��a	A+������v�Mo�`��Y�M�;n���}�q2�~�������W�[��7��������V����9��W�R5�wT���bK|p��k������T������S����Vy��!%�t)��n�A-]�u
�d
�=����c��cd�?)��Cp�$L��6���nM/Q��*r(������$��Qj5���S�G�m#2�1q�(�7�aV�r�Oc�g�������6��'��a�W���~�a[��H}����1vOu:���#>AK�����o��&��z�����D��_����g��)�#fC�(�x���N�wO�"X��]�IG�m�d��Hd�5>D�m�l�@3P.�@0|����5cY-2��Dq�\fvL�yp�f����*]�E�q��������T��R������{���h��{ZF��'��������Z���3I^ax��\c:$�rB�`�nvc���1�V^��������f^���.q=������+5	��qhI���_T��H�$e�w2�F������F7����V}-�@��-���`��wf&�����FD��C;���������s�b����g����s�V61���0`T�&*�	��M��u�c���2�,�����#�N�p�
7&�XI���<p�q������ �O��`�r�+g�o�d�J������k$�i���$APs�CC���0"%�����L�1���?��%i, �O*a��o��<HFd �K�a�+�xK0����a��1<�������D;������.U�WC9Z�7p_��Z���.b��W]y��k,B��[BN����	C�~���V������YD�]��w�R���71��W.��Q�s�h�E
��Y8s������|0]sf�6����)���I+�����������~�e�`x�*)��1�O�#mJ����Rq��7���NX�t��75	����jO.�h�`�--��A���v:e��Y��c����7�N+6�X�,J 4��zC�}{.M=$r����g��'Ks.�e-�6�@J��AJo������K9��r)�G
����^����85���Re�d���;�*2EqV]�
n��k	Z��je*��9��W��jf���%�+x�+��
O�RJ���v�P�
du��A�F�6��b�7�}���;j�a[`h��p�b�`����)~du���������5�t�a]�q4<�:Q������n7���c�qh�/���6�PB�Mb=��7^�|%��VoB�����7��O��2a�&�����m�0���M�/=�6��5y��E���UY���{�t�Y�d�
?"E���p�H���Q@���T��n&���|��>�A���Qs[�����U*��l��F`�����d���
���lHp}���]�q����n0��n��e����Y��y&��!y�ODv�BK_)fq��f-]��;V��d);bz��Q���X�U����n�����8����$O�q��A���G��L�/�%kW���f8�>tUh��#��Oo��t��y�=CC{����b�-A�����s"�y{��=��D�*�����>�����o�6�$������)������gl������	8�y�����P���<E�`W���:�O7�u�OB���,�J���c�lQlE&����D���9�2������BugN�b�4�c�Tgq��n��{�Af�}dC&�6���M,1i����D<4�'���g�����[i�.������R��e
yG��5�VH��U�G�����H��>	�Hh�y��e�	}�o�8Xu6
,�b�sLWr$-�9u=����}L����(�����3��A�4w��)u4�������.�G���WZ��PVu��q�5��R�22����q���z������y)�(s�����u�>�$�8x.�~�&}7��j�LZm�y���E���~|��q��0����V{��I^px�~��y�	|�>3]���bV�oG����O�yX��z4S,�t#a���#:O�\��ti�&XC�N��>
S(N���� ��,?��V��"X"��,u�rg���x����M�A���3z��wGu�N�P��{�����k�~�.p���|(=�W�IX8"'�3����0�zw^��_�&����NG�����_1F"N��F�-]F��Tu�Q]y���k���o*j�xB����R.++�����_�B�Y\���%��e�R�{s��!%�9e�L�kb�Nh�x������O��x��'Qo�&��abv��'ia�a���x���P��
�<Vt�N��'D��%��~*$R��U@�Hm��rK�+8��?�>J������zW��e�u�I1M3K6�Nd� ������l0�.'p����xgh��T\���I�U1qd�Z�Ab�_7�um�
��G�B!1��L��p�I3d4��I'����F����A��A/���Y�
1���SS��My���b5�������#�������>Gme� ��Qy�8[�s���|�|���c_���J.�R��%��}��9�>��Ae0Bm�_*�s5�*�a%�VoPy�#%�7�5?XQ�n}���_+@����*v����I��ns����J�b��U�A*��2�a��_�������A��W��g��R��##�rZ^��b�PW�W�*8G��'�������4�U9���*t�Is��D�����4��"�n9
-�e���0I���%}ImM_E��a���mlv[����2�^rj� ��4XJ;��f9^�\�R~���t|#����iS��b>^�M��H�[o��������@����[�����@��X��+��t���;��Z�l�V�4��5���&�����<�FD���]*d6@�?R�Q�cYr0��!���M���]��8�u�V>xM�\��C �9aS����:Ep��8���_���=%�(Bv?~�sN1���(l��t6-�f{��#A���W�,��yt���?�d�B��m��2|B�W�-�V��te;���8j1�P4wD�Y'2�$�{��AN����H�r�\���d(������p}�
�r�l�[�����q���0Y��!���O���S���Q���&���V*����u;_w�d����Y#����;�����=-&
�q���|�b#�X���e�����1���u�Z��L���I;L��
*F����FJ	
��x������ac3��>��^w	3��^���^o>]����;������_�DNAJ��l�4���a(��3}�#k�����VG���~��������c
'�@|2u�E�}�[c�tZ;�����*�OZ[s�z���Xv#�lW�����#k]����L����bX��+;������u/yTd,6����"�������CC����F�{M/��k��WE'�e%Vzf{+���R��g�������F�g0;���z�{;��������Y������Q�`�����EZ
����H�]�C�����{
����{M�.�uu�z.���s���H.��\��!�S�wP�Z�NuT1���,U�o�D�m��BT�K��7����Ep�Y��"){��0����� ���5����uk��+����%Qi��x����B��>�U��j<7�f���Q���A����o��5��zX�7e	��Mx��2x��|�� 4�p_)E�T��&�FDJ��Q;ipp��u{WW�z���E�F�r�^u#�K-��}H&��DRS��z�����OS�I�o��������&�:LYG�w�A��(�k�m����/vq�om��3���ZBC���}bK�@�$�b�����MV5T�����d�*�e�ZE@�Y'_.�r�a_[�k����}�o��#��J	H�5R����E��/����u�R�d�g�������~q��A�_�8�G�+���~J��{�@��I���Y���t{�����Y^���v
�����a%u<�u���<t16��aA�����2�P����t_����������iv��D�V�����Z���L�r�da���
!��"|��#.���Y��*w�E��q���f�v��
<�H���-��$��^����z��&P0�L��R��2Aj��F{`���!�/"U������:���w�j1�����UL����������/��yq�+#\+ixn=�/76K]`e��a0Q�����E���m��/�"��bpX��g��$C�q4=� ������w�4H�&�����$;�~7/rv�z�������<�R���Mijs���������������}���M046M���0�}��m����vPXQ��e��L�����
�=!\k���(�/�������`@6�Q��;�2���w	�������8\�'���a���x��i d�@�w���7��(�s?mh$��?��m��0������@
Mz���8��
�*<@��
���<J����L������w6\u���QDbU�%����a�mD�f�89�%�6�_��|m&�������T��gE��5S����HO���7)���lP�������ng���e�����my K�
��t?;�\���7^�[������1�}����������2;~�e�z^����"@�`v���kSqr�����E���������������J���wH����������z�I~�����������X�����AB��]�	��EX��!�����SC�?"�=��%JM�x���$������#��;���Z�&Lw~���J2�
6\lb�t&m��<��>�3���A�������z��8���~���h�T0,D5�7���Q=$ ��6iE=A	��?4u�Xq{��}�t�7*>i�|��������p���-.��)�
�O�M��:��"� ]l�c�p��5��1&���&�8��Q���nNx���|^,��9���r7i@�5��)��~-��|����n��)3d��w���,�K�+�&�N�	�
��w��[a�K��Kd�Jx����w��a�#�|������50b�6���tyd��	�>�ep��]����Lp86�7t�'��[�v��?Cihs���!��L��*���tD�.x��+�4��xqi�:���7c:���&��]j.��u���:���g�j(��;�D�?�
��+��7���������*x������OY6��g�]c����6A�{���X�!��i���B�G\u!�U�b�gp�������,���A�}��|E���=���o�ju5�r�a�l�xVS���T�%����)v<�N{VH���OI�vr��������6�����q�B>�K�0��S��������_<��^\���JK�
��p�Z�%$r�L�Y6?;;j^�O�L��M��P���y�C�U/;2�<�tn��!����Xq��3�!	)l�:��n�e����#�K���eD��ZED���������M���G!�M�aaMJ��:�~�Y��%6D_��~��y�8�WH�*��\[�����
��y��m'sk�����)3�*���NJ����53�����/?����%����n���������4�).B�1g�m�h���s�����^��\�)��0�W���?Ce����|�zv��Q����] ���\��@���A������'��������O@�+�y<�41��Wm��[a%Y�Z�.�}&�Z�nI�q�,D�=�<u��������H	���������}���������sD����h�fx�1������S ��T�O��jX���w6�Q
�Q�
����'�����:�;�!�[��Jq�
�2���$�l]�z7�m���e�k�L$�����F�bxr�z�p�^g:x��;����t�~��sn�2�:0�����~T�K�.��3g�}��AT,�����0����N���H�OM�Vg�{�	�>��4'n�6]�C[]`d��J��%���K���5@`]� �5r���J��D�*"����)S�A#{���6����1J��H����q"�����M�/
(�����#l\�U�$���R�B�>�xL�_�������r����e����j�*���;b��s3W	������j���8����x�Y�p����0�����z�P��<�u�\b2[B��3���R��Y@�����d��)����X�V
Eu������v!{g~�QX�o#Hef�Ieg������L�I�)��8��
���8\|�#��q��%���]r=�������+B-g
$�(�	���/(�h��,�����P\�S-W���C��9|i\��#T���%z$	�Y�
�B0*x�k��Z��;t���]�Iq�����0A�������d�@B:U"|GM��3�f�c�m�)����:_\���V�|-����|J���V}�}7����p���I�#g��#�O�$Yu�w&HV��)Q����q����/����k��[C�7t��S��8� �>j]�6/����}����+g�P1��:�h��\��
�|^��������%�+�=���-c��{����LV5W�{��?Y�n��ouxO��{Ft6W��n�Z�����M={���7+��l�Xo��K��{��+������(����r�~#���0��,��|}�����1�����T��On��%[�O*X7��w�.����F��9����*'��h��F�������BEb��*Tc�����)r�Ud���q��P���q��i��^���S����[i�2����O}��q���B�7J�����z:��Rg���K��59A����,�f���nQt�	�����	�?�����B[�L�������	�3�/��/5p�k�����)��j&.P��I$�����}fL)�G${\��R:V,��!��b������F�O6���C������z$q��_����s��dO�Q��Y�[�����0�lK���\���)U��"=�6��S(���������<�o9�D��^�0��#.TL�9Swt_)P=B3�%�'1�X:FB��W{&����c����$x5Q�w���h�i��X�a_��!%��AN������]��?1a�C4[��UF��Q8}V����	-���\(�O&�o���PGa_E��|��w���{��6�l����+D.�}�`y�t��T���2�h�(�������ru=���G��*	�v"a{����N�[n&>�9��3Mki}���C�R}0�
���0
����)���NWn#��yX�?&y��\��>���\v�5���P+���$��k�]8y��P�e��!�=���'���h��p�&������(�+�?36:Z�38�q�Z\3M�-h���c]f�o����3��S������@�8gA"$��Z&����}�2^���<
n����GD"��N����zH��Ri�B��| "���s����
��O=���U��'>� ��"m/���g����=>!wJ����*I{
K�zNo5���i2h���o�6����6�X��n(���H��C������[������v��-������;@#)[A�����(�%�y����Kw���f���b<�&��Xy&��v�\�l�
�l8���5����) r�WXZ�#!-�H�#s0����]�����L���"�?b��� �{���q����(d���	�����gH�C>dp�N�-.�����4�Q>6]�~�iGzi��|{�.�S/��j-����.�#�oo�,�����y9��y1��q�T�7fJP����^d[uT�J���{���f�����H}hp!U��������@�9	���OEg�f_x�R*|�vFdv�V�����
I"���eqnHO"2�4��Yk������|a�G��j;�i��������3��'������s�KD-�.����/����?xY�����s3�,��%W�_L���D�t=>Sc5�U�}`��xmX3��OM�	E�.p�����f���� �fN�M��>�~B}WWmK��Y-m���S������|�	�����>Y~�D��vV��R=��j�]����rk��`|����Q��J��VzdrQ������r}~BA�u�$���eNb41r)@���X�q����jY����S�����JU�HQ�f�K���f�Vz<�s�E�;��a���<�������Ue�
]����^z�7������������H�����G�%�]���*�����=7rF��ik���1�l)�=���W���([���������e�UK���Hu�*"t%��J<�a�DV�t��6��(�8H.Q�}B	�9dOr]��Bh�;�����x�"@J�32�y�O�9�B&����Z��JY������P	�v�7�����Z��Rd��5)�i"�zmkE�p��{_U���ks�y&�����;�kO��
��������P�z�B0������b=����b<�v���.���2������[� �hy��@�����W�^��P8��e���y�|��\C�3����4��/���1�c���4�>��iQ��A$���1_A�n���B	M7gr�|����$n5;�7��IZ�u��Su���#���V���CL��lX�@�T�|~��	^��`���K��
�I�v�\B��V���5r��6�&l�H7y��!����	b*.0���K�r���N��?_�h�}bO���{W�<���Y��:���a7�@�e�}��=e3�JVKg^��r4V�U����A�Z`S(���%l3�n:��0�z��\D������#������!�7���������4����^�rWQ�`�{hE�;��r��� �O?�@+E���?�.4u�wM4�g����(F�K7�3s�a�)whr����'%�=�����(��U�`Kvu|���!!�J���������xg�s���y��R)�W/��=J�
-�	������,z��
����uz���M-h��NI�t�e
����}�J������I��)��������r��A��U���f��t�q��~��SZ���s�W�w5�����fY�\������3�gR���uX���b����~�C�`�|��A�{$�S� ����{��Yv]�K[�j5;I��h:0������&�Y��=��<}������8�#�y6k��Y�?<*�����B��A�t#o�����U���q����Qp���A�[�%I �)6����������_�G����=�[+r�_�]�A�W0<��������a�<�1��p�ol'$����;�]�T�����$$���#�V>aL����ydD���M�o8�n��50�[79��o����K�$����������>���$���aw�~�ec�C�����1�����6�zS�ki���xB94$6@T���A'P�1�]���-�����������K#��`�O����RW�
W��R�������5�\������\6�`��}d��M�w������|�b��=\�u��?�m�y\�:�`�;�3D�}��r���z��t����G�����w�4z!�9�����i@�q<�w��]�|#�FH��iL�}�
"�t����i���y���u��tC:'�;��`�t�(]��z]'�S�
��irwT�u��s�����'�������*<Q���y{?_d~8����I��.e�b�l����a�hfl��v�u���u�<�:}!���^��P/-���x�"�f&<(�����
.��h��R�=Tj�3�����o��DV�kJ�-�))��������Xx�;%?+�&+D����d��gM�Y!������g)�4l1J(23mh!���^��,llLYT�A�J&x�]��g��T)�97r��W�$�&Oz��b���Y�1�s`�ckoE�]\Ya0������j�U�
���2�WQM��4K�D����3������o��(% K�?�@d?���2e��Q��T���Nqz�4a�Q��1��U�|7�\U����No����W����n��M��Ikc����F$Re��I���aPx�k�I�7�.����.�����dD0fK���{��k>������,q"W�����I��a�(9Kv��<��f!����*O��w
�`�DT��_O�=J����"(�&����F��h�ZV�7���[$R'�����<�@��+��O�`��}�g�(mSM�,���KK��3���#�o��L�eD1�����O0��_�1�{&�z������g���]�\O�x� +���*�D=���Mr�=�����H�(
6��>���Y"<��z��4�w���!���g�0�����;/vShI�+��j
�8��wG��/d���p)5��
�����,������������� 2l�E
(r��Bb�J�/��m�R�)��n��~>Q��A�����,�V K,�����j������jzo�f������V��P�Ct��C��T��+����������0�h���uE5�EQ�j��"R���� & %BHBS�^�����*"�T	�{P�B	!�(H�"P�<�c�|���y�}�u�{��s�9���N��l��O ��p9�%��u��l}�U_d��Rn����U}T��j�
w)��1<�s!����B��kk��'�|Yo:��IR����Z��?A���J��o����!B��fOz�3y�����#�%pC{�E=��a���@�iI��>��L��M>�[���{�
Y3���8�!������.��[D*	�����|���1��mp����GX��X����,���\w=��zx�u:��2+��z �q���p��s��q����]d��gs��|�/������S��Q���v�|�C�k��~&[�����Ls�t���������Pq9�W����c	
�}k@h� ���c ��..��[k�@0%A�_��2���C	�TpT*r�
�?����?�V�!\��Js(I��{5�@*r�54#�������\��
�8:5�f�V����h.qT�v�7 1�iwr����$.$:,�z������S`��39��&_eG0��5lj��R�@�Gf,��;���]�:�A��8~�)���^�o�3w��sG�M��*�Y���LM;��cn�	�����\�(%}���T#'��P�����6~-�W?�P|�i#�&,�@Z�O@�$�bzD�����J�T��%��.��8����"V�)�F���jY�����������������a�#(�����}�0��&��[�6�I2W������g1%2[nfM+?DY'��������!��2��T���j���$����b ?+�\L�@/U����Z�;z���dW��Zb�	�q5����Y���x���CY����
���s���,#��"t�*B����8j[Di������EC����m&w��^ooL��:\�i,����h���4
���Y{�����|���L
�����$��\W{�S�K.�����F<k��*V+���/��l	�hQi��]�����~)U��#vQ�V�9,���h�0�� ����,j����I��h�(_|pO�����c�����W�U�&>'�}��F���MY�-��9E�������W��a}���^*� �vWX��IE�R�I���QqK�����K���L���Ev[�=-�p�*��_�SBn��U>rzCu�������a���R������]r}]���~E����������6�G��,5xx6�^R.�T������L�AVK�]w���
��!�L�DwI�\tX������c��.h�������c{x`�|4\k<K:����C�u����L���UpH���\��dY�o��,��^�����M� ���v�)1��������w��
�W������N�x�@�8J"D�,e��l�n�o�I��(�8�����zly|��]�")��]�Cu�����$1������N�;w%8��^�����i��Dy�$hx���y$5�w��0��Rj/U��e���T���?���@����!���4Rh��.��2)�����C����;���M���m�)�%�d�~Ai�Q>Zu���Y�&1S����L��r�"ig�m��������Y�DL���������WZ��h����k�f�>�E������H�
��Df��{Xl���Y"����������69����Yl�t���|�_<~h�l��
����#�����'?���^��7����$��Gr�)B���t��lm���E�T�R{ER�K�b�\�fl�XD�U�;���L���sS����	D5���D�hXh�\��S���6��!+���o�7����o�91I�[t����)�-�G����_��7b+�]�3N"�'iX����x.���>����������������_k�mK���s���x��E6�Z_��?���J������0�?�z�o���s���v��|d�����#�/�p� ����}2��l=� �3nb%s$���	'U����p��;\nd�����]Xqh���K�~	;�l-�@����[����x{�s-��#n�����&%	X�R��63��y���~y��|���~A����g�eA��b>S�5'�Gc�?�3����oF�H
��-�\&���}��{�!�.�v���D����+i���������w�V�s���Q�z�V�����-��dau���T�(J�_��jM�|���,L������-�4���'H��:����!y��8p�]��T�D���F�
M�ctZ�}|��f��*E�J���N���!��;�x��2ox0�i�O��2�1���,"�#W���G��I��
,-��6��;?�g"�EM��Rl���0�Z���H��������Vuo7qe>�,��O^����?�(3��>�V��'by�
hD3��b������`������W���,v����U,�(e��.1��
Q��3��@��b8����\����4�������>.T�*�5�Z�`�������wK�^�|B����j5�HU�Kx�)�kz�NeP�"��b����z/�U%�p�H�����)j����u�q��Dr"�E��R�E*�v�p�3u�U���=��A���C#��KV����O<?h\��������/�����"^�9��0��W������wF$�E��j�cdEw���H��F�/�H:�?mar��%�;����-O8~��km��b��'��+K��~�g��.^���#�6z�jN��
�X���;���l����W�p�0������YqGz���2 ��`�_*{�A4��l�Uy����`�{����2xUy�x�B�����J�=�lS��R�
��K�������������������fP��'��_��p^���c���V�w�6���	1����)�G����
uo��JeU���p����
���~�+t��[�,��N�,5d���`���[z���j��\�'m
���M��<���r�q���Y��zg?��>�ncK�O���>������(9�*�^���W��y����Bd�dn/SH}v���ay�N��&5}m�P��	j�s�^�L����l��~:���W]�$H��$���i6��Ss��vC�F�Q�7�����`:1�1R'�v�n
�LO5!�Q��8�MR�I|
F�j�T9o;�[�u���� �
����:L�g%/b��ax)���<�J�n��CoK����d���v{C,X�����Q��?/��i�J��.
<��S���F�����X^��uC�\
&���E�����d�Ss�(�>��[k:���.���~ojQ�F���M�_���E�+�n��
Mi6F��YU\�]w�	���/�A��Z�;9
W
�Dv�h(��^�u|\Y	#(�/l���;���7�M-��7�W��9�#f}��k��[��-,1�^.��h��a�5��@�.�V������8��)�'&��V��}
{)|v��@%v ����Ps�R��t��_�r*�
K�������C�+.�d��03y���i�@}�K��E��`]1�6�a�ef�1�z1��UA�-���b�qi�[a%Xtn�W�
i���]e��Di���i����=��#�=A,X(c�Wwo�`Km�dc�,��p���Z�ys�c����U��r1�u7�#>�J���ajf��:�ur=j�;\&�+�r����PK:��V�My	1=YB1TestArtifacts/test-case 7 -T5 WAL ~8192 bytes.png�ZeXT�V/
�KJ�"+��, �t-�Kw�� �
�!)]�H,Kw# �wo?���w������|s��yg��	��{�����{���y���y��9�#�?�_����Gy�����d;	�-Q)��#�5��1#
�#�������+�;ki-������n��3��D	���z����o�F�(6��T���R �E������Xl��16i�$w8�~��c��
]��9����=V=�g�W����e�|���0qO��k	�SP b4V�5?�����~���5W����6���]JB�����p�ww�BTB�,��wo������$�|�3+ �����
�'e�?�x:g����=0�����{9���P���L�o��Z���1���c(B�4hy�TR�-����r�(V���/����
	�3�����f����G��w<�LaE�2�,��mV<1�]z>
P����{�|�)4X$�<���Y&�g��4��q��>�|�����)y����
����u���BRS�=r��m`hG��_���������W0��v�8Vo(���c�
?x�,������h�w��������.���=���HPR�����ry���*�����cz�v��t���?rI}����p���?L�����z�6��a?j�1`��oX���������2�B������n�d�&4�76{�Ccc��g1�O'���C�i1
)���P�
n�T"
*�����/e=�n������8�~���}����#��h��c����V��p�����p�Q�xI��	�cQ���2�G������T1\�T
�\���
���t�_�tJx���=������mV�^2�L>&��|b�������L���|j3%��HsE�st<N�^Zj�Dz����/?z�&��({-�����GHm�]3�Py�NmzV��jBD��=4�3�����r$�NO�%Rq����<*�G�����s@T�=Z���#C0J$rO�K'���������Be
���\��1_�[����*�E�!���T��
2"��f	}������p���>A�S�V���h8��q(�to��M�����f���3<*����x&��,��P���}����gH�
*C����DZH�E*��@��=����p��;�|����K�]��R�f_��+��0�T���$F(w�����F�WT�{'�YyP��BTw�~\��5�����N����G������Q���M3�Y7K��n��������<O)]4\�/�/nM��x�����y%��p��G`';T��p���<T�`g�_�8�h/?��I���������Q�h 4�c���2�����~'��(����^Kd����,$��q��e���H���nn��V�����'��,��3KK�l�t������ys�8��oM�=�jk9�E���������57	�����~�?3
*�W
��h)���������%2����,���
��������g���lm��I����
h��
d*�����`����*Ep�-b���/6)���e�_"��mb��Ymm)��~�Rp�j6N,��Tu���{<p�O\YH���qvaq��Hc�8��U#g9��Au0�K�cT(�Q&�|[�?y^\��[vS,>����������d��rHJ������c���$�
��q.���F�^G�������
?]"���d!����$���s�2�$Q�,���"�J���e* ^U��5n�RV��{q�x�uV��z�U��P��[�4<��qu���kWcw5��9PP�9
�?������8�xan�O�.��V���z��>+�A4��������'G�����[�U[�.G�K�I�cx�=���X�S0�_����A*���d����	��x�p�������8Si]3��(��'�t�������Q�/��ya��0'��vLJ��{�3��
}�����,)9��M�
w7F�mg����C-B�^�YO=b�J�`OP*S[��":B(�OK���f������r��O=�{1Jz��G-�|�����c�>Gq��Y���I�������i�q����}��F��g�~���Tj��j�C��}.������.�.���
�8�q��<a�b`���]��j����;���6�OU	f
���W���G�=��@�b��'\�HY0���\�%�}���#"����q�������?�OL=x/z�E��W��%F��k���g����� ��zU!F:�`i�����f�6��r0I?��"|h�K�:&�����>5N���V��1]�y���vd��"��!9���w�:���}�����r�w��]Rg�{u���NJ�0x
[�OY���L}�[��!b)���9���u���lnpL){`�P��=����^�{gX(�������Y[E�j:M(���qK�u����������+�~�)��\��'	n��f�f)�:�(������`&�*V����<�U��R���(_���<����7*��K2�(�id�,�����S���bd����#K>f�2}�x�(�c�)e8��
�1-+$��1��S���1@��*��6_�e�)"b����d�'�$��� �o��,�"}m4(��"O�'��\�@U�]�!j���hn]2���`�����4���������#�T�p�h��|e+�<����o�I�������+;�kn�\U�����x�de�<��,k�+��Xr��v���Y�e���=��}�����o�d5��^����x\u
:���sFZ�O�u/6�X�e�����9�&�zS��:�8�}�^\���pdhX�*�������I��5��h@�UZ'V��������M�Z�j�	&:�u}�N��j�2$r=\S���
����[R���3�a��4t$1�w[jDw�xu���{��4�j�b�����66J�}M�����MI���4���v������� ��B�w��".vA8?`�Ff�*��nb6��Eu�����?3�����
��q����E`1�T2����hl�=x`��*u2""a-��+��1���)]*j��+�:R����R>�����1 ���'s��<����f�E.��F������W����W���!4_��&����2����-5�/��UX�&Dc�yT�����'�]��$f�������PX��/�}���L��)Jr!�}���4m^�F�G��{��v�mA:�����+tZ�S�L��~��vW�4�'��K����u�WHC#U|��b]$�����ty�>�T��Y^�a#'%��f�<*!���������=�?�!��3��k�|^�"��"������U�����4����Olq�G�TT>�f�5�<��Q�K�n�����.�+=��h������5�0��|���wv��N����]Rt�d��sf����Y|�������EK�R�����O&G+�hA�j���
��*��^���P�
m���5�3G�	�r� Y}- >�����1j���.�kv����O�X��E����U>[�������x�Y�J�YUDBs���8�INk��|����`m�=g���\������2���UuoS�5D-�/�1 y/ ^u+v<kN*:���U	�5w�m�������������\+"����k��X}(�8�7S���L��@�I:����_��"�#fH����RY��
�����5@��P��1kYiG�8s��in���_���O-��,j�(Q�����F��V��������f��Ae��<i�^:O�8�c�����G+	�x��rGX�je���B�cwJFZ�*��M�D�{��t����-�� ���6��+�������>���m���gT�z)��:���L[t�����t�1y��Eqf�[/��7?W������.��RH���|��g�@�{�&�_�$��es�I5a����E��C4�f�l�H�����mC���W�o����&���c���n
�/��{�B��t+�U��z�#�K��B�WrF��|T��������V����'H�jB���������h����+�(o������Zb[�x�WK[��N����,���������?�X����,f��r�mjT�M=��_�yT���_��&V�~���lGE��cf�+�HFt*�p�h)���I�������
\���i�;���W��H$�S���#R�)�y���/��_��'++��#��I�����X���M��~ 3�����g ��e�>alm�Hb��	�5�_�i/����Aa�;m��+U�K�e��2a�����K4�������9��zQ��������[�
�m�-,�R^���ni����s�����D�=Y�����~d���e������
��
�U_O�FH��������y���<�<=��~�)~��Mz���K�7Pp����JT��BD�YT3n6�t�������m��K`0]3��v��Nn�?�N�?0D�����=���Z�r��c��	���ND�j���=+��Pw��tU��[��������A@��������o	���Wh�K������V��`X�.Hm%�l�����E�P0�b�P�����+����Ao������������dV���;����k������]KV58���agI��hq��^�v����1��i�vh88����'���{F���Jl%��A�zNLZ?d�:��0hh�^����?F9a�A82K�cG�>*}�\J6��� B4��~
^f�.AB`^��@�����G���p���^�yv���������/hJ�?�*�_���3"����Hp���|���}�^�)7>R�bVvt�UX�k����G��-�Xh*EQ�V-D�8�����'��l���K�&d������Q
f��Q�9�<���m�>��B�w�y���&4���K�c��_�W�a�[m�_���x��w#n����h)\8��������������S)2������;��lH�G:b���^he/���
����2Nt9��������[7�e���y����l=�>:��F^C����K�L��oq	�����J1���4�*Z���9-�:�������82��@�s�"������G����VFNv�:��t[�n��%������{A=uJzRx^Ju���.�V}9�m<���'�����un>V�OA�-��V]j���a�T)9�P�FR�5������W5����P4�u�S{��n/�d�Y�5 �[eq9�x+�1n���2���o��oj�t�;N]*�{uFnB����M�T���A�+F$`�?)���OP�Wy�r����5�{��<q4��*�C�����|��0Q�Bi��v��������.�VC$���Z��\���:�-6nn����/�~U�<��1c1E�D�o��]�/�J�=5�����C���a�-d��tv�"��;�M���m�|d���D���j��|�"3m&@�"Vw}�"g�B����}�D���N�v�;�V�~����N��<��
�-9SbH����v |�q��j�+��13n����/Z���r~>S�3���u��n"
�;d�I�$������EI�C����_n(;)��~�NN���M�I����������k�������j���S�����7L5� ��>�v$������d���7x��~se��	v�w����m����Q�u.�d
n��u:��T�|��7�'�E�!��7���`��	��^/x���1Y��p�3�B�����+�b�/L�����v�������M���>�K��NZ9\��o���T0����>J��\�C�V���1�T:���4�D���UD�
��[Z�7�TX�|2	d�{�+����JB��l��[�CB�E�p��*�r��������E�A�=�u_z����K��T���AB
���7�}��Z����c�R/�"��5%\�K��_o7����f��&��*SLz�]q����>	�${g���q��d�,G�
��$�J�H�:�I�^��ln��5`Z�h9;�z�%�>��r�/ee��x����|�'�(�OXf�=p'&���\��P���O��R��^�|)��b�$C��s����\��2� �"
r���2���O=]���T�`�3���D@��~�OA2�.x�P89�kg�������	���zB�X��c��Z��������]=F���I:�Z0@-��i��X��	��m*��Z��o���(���}��V������g��(������l^&�}��E���\z�J���<����b"��w�t���!���el�uT� ��E0�K}����jt��
?h���b��w6�x7������<�;_#������2�������gke�jJ����>����s���6Q����^����}�tZe���Z�{�������3r�#�j)Z�<�����*����_�P�([����7/c�B�>v��?:��������y�~O�o��Ncf8[zK�	��U�����L2rivL�3
L_"U����j/,����C���B�
��/+�R)H��G>��o	�e��/4�Xl��]�-� K��[g�g������Ps���s�,	����VB��o-;vvr��1��o���x���@�[��p[�����q�*[�M��Pe����Pk�L�����Y���Q����r��i�l0�$�%���,���c��C�|K*�K%p
>s_%�cXO'j���Mrr�v8�5hgIN��$9WY�u��s�"5?�"��&�pf8+��'Z�5a(ZBV:	��fs�%��M��u��������c�����z����u�]��n��P:�"�3�!���������D�j��>6����X%��e�478Hk�%=��w��9���R�K��(����y��%�C�vK����.���b�X�%�v���a�cglv �b��k��-���W��?�)��������>��'�Cw�n���V�	���b��E��>Q��'<����M���\��6e-uGS'��bhl�J���	�����9��H��h �/I0�!�pc��^m��1��S_U���(&Yv���6�s�q��>�	��,V�U�������i_�5{%v�S���&&q�B�r�Yr��,s�M���^�����Le����/�E���;3j���s�@E������*�Wy�WD.��E��K4��_`pf#��]Oi{����._mqhB�Hz<��/�/7���;����E1R����6&j_,'�~"'�C�f���~�Y'��gh�^�GK#����:��r:��mg'�\?1��)�{��I����:snd�&U�	�h��"
�0Mp#;��VaigT�!l���^->��p�����Q��qr�>����a�����
r�(+��4�$�P���%�"K�r���Mx���T.��r���8��Ii��z�f���w�f(��������G����Wi#!�H�?�������O����i?
:;*N�h�6MgR�aqh������
c"��B�L������j�)�?E������/.Z1��7+�}�����4������^n�e�g������E=�W��\!�F��{�s������!���c�vM��c�='��yZ�u�<�i���7�
Ee��G&Z��2X�	R�Y��c9
%���������',k�9��3r]{����C?&�|X
�h-j-��������zs�h=8����� �9L9����A*U>����`U�������V��������+����+:�$sK�l��������-=lE��_,��E�("������A��O���/o�YK��a �eSON������f�*����Bg=w���K}_���I��L�zw�?�3���!�c�7z����=�9������SJ���C�S[3����z�H�
�"����o��6r�{�����q)�S ]K�Ei�F����p��d��y�����(v����+�������r��+���9uk�~�����rP�~��������Qtp�������X��Db�W�m��o�b��"��k���a����ajX:a_k|�P(^���IB��\5����\��?�	�G���@|�d���Y0t�1i�����~���U�����a@��Is	O���Q9��	��6x��[����P����nt9�%�wD�������|�0]
(�Y��i-���h��o���Y�{e��^�U�������d�b }a���%����|�{\A"o���M@�NF��i�/�`+
�k%�z9������������������%89�O;�p�bht1E_-�se�ch^�E�h����[K����qx�=%1��={���'���\M\��"�����v���N����\/Q���4�����h�Qo�g��6�Gx��
�j��mG��[��&��_�W��.�(^+T?�8du�I�V����W���R�&��)�����������cJ�����;����	Q!0������a�S�}r���\CS.|7�|*����CF����an#�|�B�����$,��Oxm���1y7jb@�����c.�Q�}C�Q�_NxE��Z�^�����[�
&6��1�`�����-��TK������I����/S��#�Ao�kq�"����uH��Hp�_��[���hg�������~��^H�,�n�X�9{����������������v�{�dy�)cG��y���_z�P��YX��x��t�s��o5�D�&}��E����'<=��|��������sH����+�
������e���U�ZZ�Vd����Z�5����:������[d�tn#nn�9+�:G�<�c&�?�;t�>��Qx`�w��C6,K���
��N�����.�F���a&v�M�a��+��J@��dl6�+hf#Zv��	�FZ������qzaC��{]d	����0�4vs�m^�%[�+���o�|�:r��3_
��a8��YZ�=�@��AF��n����
�j3���"�G�/����dw���+�a]�g�MA���#{U~���jB��U�B�v���+@��I\~w�v��C`���E�W�Q��	�b}k�]�J�#�T'���4]3f�������h�����e$�q�
K��s�1�����%�l�� �S����q�c������u7#�1r�>{����$ z���M�aw��$�p�[���T^%����������,���qoYaWK�h3����������9�`_����iodM�n���t@��M��y��,��
y\~��
�;D�������=�������j'1�k�q�L�m����y\/��������I�5?^m?7�@-������Qxc1�
��b!�4���\����Mt�:4���M������WiR)�G��Bg��Uy�#P���g����/ fdl�4ov���yy����p$L�^K���v���5�K,���N<[��rv����A_G���T�C�v?�1Mi������+�����}D�e�a�A�i��Z���A�l�!���W�����f�-�t!����4������.f:0��z�U�K��Q����:��\�i�3��9�nrE�w�<j���g[Q��g�p���q���s}�R">�-�3�*���j<�{l���:%��f����Y���
i�����q��"�|	5n������+�#��S��g���dp��'��~}`��O�&�^Z#���@qC�K���/��o|M9j�~ 1Gjm�C{���I��
w��j2w{�D������k�=z�8mNS����r���KtJ�7������[��O�#���I��k��"��$���D~u]euN���X��h��e��,p�H�!��X��"�����8��j1������7�\�������g�R����~A�K#hV��J���{Np����L��T�<���~i�
����[��	��s����}��K�����*�)��$��	�1�5�cR3^��*s��r����?%Pi��0��RQg��'O�*tW'
H�h��x����/@\�vX��+�ekB���6� ��]��?�P"(�E��r)���aN���?{��T����M�y���=`O����7���m�����J���83454�)�M5
���I�?�T
�8M��5T�c
���T��m})Qf ^;87�<���W��e�kt�V�rt����8$�<��O�uW./YZ_��e+�cYM5�����\g�s������eS	i3=�/;�_���1&��zp��p�h�������B���BM��?}�G�=���q�@:3�%w��JoRb5�-o��0������W�Nu�,�Qe���q��4�'NU���r�4��7-���;�<I�����-+=�����������k��
��$��&_�:��Ab>����� q�Qk,[
:��Mx�]_��W�� c��!Y��3kV����#7l��gM�4���r���	�/v�_k,F�l��R�����<���Y�A�x��_i�������5
�����e*`H��~��Kc^[�r�b�7�~��:Z�
�
j�fS��/��/k�s�|\�F��v/9�������:
�E��E���\�2�z���U�y�T�"06t�6}���+g���KP0��,%:q��,!"8H����QmKS
�nze�����}O�f|�]j������m��V�`�F��@�����)sW�P�b
i�u<���	l��}��m���K�<4)�2���f����@�E$��:]������7���N|����M��}�8U��3����eA��T �$�G��;��L��+
�w�n��5/�e��}������x�f?#}r��r���7�'LbR
���?�>%�u�aT�&�MUT�k���x���������<N���6��v#q��DO�]:i'B�%��@&c��|�����Q+�n����(�v�N�3�t�!'L�d���t*����+��xYh���8O��mn���������6]yI��RI��@�����D%�bH���F�n=����\"�?�0��O�����#w��W�N��Ga8���d�E?u#1����'����"����^�������������h����Z��,�Yg�)C��]�;w�tR�����V!cw�.���m���w?w��n��Z}��''"���*NDv?n��ab���z��\����d���i-���U<�vf}YO�����7���������?[�/���C�3�y���i�\$�=`�������wr9���*`��w}�0a���~��Nb���N�o:���.]�!f�v���>VsoB?y;��
�!6J^^���X#DB���O&�
���9\�:���e\\����}���f*M�e��D���wiM���^u�:�P����+ f��w�mR�N�����?����&��5�?����V\����?}�4�._:����T{nzm���x�^�A�Fs�~�\E�M+|Hy6^��K��`���d�q:n?k��cPY��-�Ti g{��QS��m��5��%Z�n�J��u�h���V�6c���E�V'������s�fBm�<t��I3>^�B	:�<��[�&�	m���.�E���B�k�I��D8��Ap��&��{"�iF&x����Q0� ���l!Y�6���Rd|����x��g��6�H������P�"pk�8��_vg�8L,rI�I
`�j�������l����,����&���j��q���H�%�o���4jq��x�y�o�_����&�0�CQ��"%`@d��B��@R6aDD�,��"���^e��({����)EJHCiO�g�s��~�������y��>�����cXh���5!}O�*��Fi�:�z>������2#]{�����f��&X3�n������
��O��f��D���T��\P�
#-+zL&a�����&����
B��NI�s*���i����c��!�����k����x���{�{�<���N������uQ��J���c�v��+�Y����w�� )��Vp����I�H��G&.���g���{W57�"����d�;'�~�0��"a����������^���k��6W��u*J\���%7-����m���F!0v��LB
�E�����!�"|�^�'w�H�~L
{2	.|;�������Y��QGO�,�,�Op<N�mx&�D=]���������������� Z�P���3��{Pb�����+
Am�9>�����T�$sV�P����(���f�����$S�%1�����#�Vj	��$��Av����V��-
4����[�����sX���Rh�i���7���z�2�2{m�OW��NC��b�g}�����8��(�!�;
�����nNWs!Q����u���
n����x	����/��
�x������X��s&�3����g���`�K�3L��(97����^x�����X��n��G�?���%����kP�<� .�;X�����4�������-�CgM�~�;�leK#����}�`o�3�F|,�1g���".��}�Y@��?<)|����v4��k�kEw�2z��Ww=��1��|�C��uCbp����<�W�s�Y(�A�3��G#?P*�G6V8��� ��(��B8l��F�i�@N$��|���>���h���	��%F��p=�]To=�(���	D�Mk��@5[+����C� �]�G�=��u.}�N�7���s�
�����}�t:�7%&bQ�T���%K�����5��k�i�Z�$�q�l���/�.�c�(�����p0�5_��3�6;���<��J�/���E[��)�
����t�ci������pj_G�{���[�		v�9��K+j{�Y�]�0,�}�,��&"N����a�����;��`��>��%o�B�q�hIY!V��y������[��"�F�r��x��z�6�d��QP���y�*�.@��M�>�����\����ffAr���+��Ab��?��-4r�b1N}�2�*!�1����3.�+�-�ky�����h���M�m.���]��7ua-�Z�1�J$��J�!�OZ�}��W�����t:���E����-
�P�*�������������R6o�iO(yG�a�,�Zj�����������=H��U�=�:��%K������;+%�KO9_ �������NV-��s�����m�9CyFq�&jk�)h���	S�f�������vA��+��_QI��U�ws�����A��!��8�rw�7���A��L0h�xw�>?��C;�Io�A��1vk�F,m_���^S�����Rp5=��2��s��^��Q���i���&��}%%���t���:�s��{�_��,��j�)�"�k0*Xd��1F���>��1g���0`U��(s��`�����l��=q��������8v�����,�Ha����D�v����qt������B��~��G ��G��[����:x��4U�`$�.�B��o��1��f�s��_�[�����L1�9��B�V�;�3���B@uQM��������3�@���k
��1J�p���`
Bn���U����x��)`���}u�-��!K�0�~��$3�| V���C�j�t��e��L&������a��q����������2��2��5:6���V_1��o������~�Q�Qx���&��|��
cqb���b(����]��Vl�c���z(��}���O�o��n���=�tf*�K����1wL��M��<�:4��n(g�Fm��n����|�R�w��r�9���Z:5n=�V��Npc��q+�
q�ac/N
KQ������:uf�SM_�:�S��"�������i�#���,�����W�k>���Q�����l+�S.���"a�\=�00�4��:O�s-[�~9]Bp(��*�cl��� &���.aR�K�:"�<�z�!�U:�]�:h�I�-6��&��|��&c?#�$s����C�o}�}��`I<�V�R�Pc'�jw���aL��j�Z�h�j� %�}���o�����W�+e�,4�Fe0��o��l�d+���^3W�{����"�Ik5?��������>���~�' ��xT��w3�T�Ck��U���Lo��b��.b�����*:*���o0�ev`�+��[�ihh�{����:�%��(��T���)�N"�Q���"%�tBsK��PE�RbFgg�OJ�q
��x��u9ts�PN��y��EB{��o����1���8P��<��f�Y��-�����+d'����*0����l���4�0�@N@�+����B|&�9o{�x3K���4�|8RK	6��x��B���5�dF��y�R���~����S���N��+�9)M�L9Wb��&3���{GU8��R��������~�d�w�T���}|@���uiQw$���i0/��_��%����Ui�����[��XY���g����COBT,7��/�������_��O�1������'����t�����*{"'���F�fQ����x��M�[����:jZ�7��X^�T�h��{����~��u���~3��1�q���zkg�lS�9x����Ir_��M�����Z�,R69���b�>$�J��
�5F��,}��v�����v���!���X���VY����L�${��	a����D��s���V�EK����;�$_Fu:d:��Uhgpo�@��&,��F��)|����kk�3�.,�>���rK������s��H�o_����3�����uoP���m�Z*1��t����WIT��.����s����+�v��Z����\N�/�9e���{N������5�IUI!oM��������F��!����I�W)�uU+���mI����b�W!����U#)RrU�g�E���PF�c�K��	8����������%'P%�P�����)/�/��I��Q�)g�������C�Sk��������� ����&���u���RP��h4n,�;���90���V�@?�Qp���J��\�vVRj�#�i���"��{���c.�j�9�7j�O�zA�� w�(u�u�*3qm����C�~�H�J�_�J|-[��98���W��\U9�+4m"���(���M�#3b��������}r�'����K�!4��U.L�����/�t������0��{���r���=6�e~
Jbm��O2���2	��zg�/(#�����N����Z��t�N-�8�����?�@��^����r�W��u��}E% 0�:t�<��n��������������){����X�7��/�M}e��-�sO�n+�M�L�w�zT��!��%*��Jb���%Y%���cp����Cd��Hv��Z6���xm�:�f���8�z:��������,y���x$��������m��	�����h}���h
;����G�(n*��'�����Jy�
N����y�����.O��8����`�$|�z��Vi�,�*CC�t���J�{?�PK���V_��TestArtifacts/WInsTestSetup.txt�RMo�@�[�!�)�8.1RJ�F�J��Z�k���������w��h�Sd�Z��7���,f��_����4r2Q0�^����J(��L9�T�Y��4�t�a��$����=�4AN���F�-��	��q0�����i�l�B��4
�a�p�i���;T�)_���mW���)�.n_$����f��:�|��q[����z�V�a�O�@�q��^3a*"�Pu"@�`S����#h ������)�g�$RT1%4c�:���"�NV����R�����;������G�olh�y��$����0�}���J�R��L�:��##�%��NS&U4Dh��:%���RE����vo��9/�{	v	/��1�+������J^�>xb�{���~:�(�p�v0TL)�fgZ���b��.@��}f%����������@�&�b>�x���.=>��p�|7B�^�#HD{�[�.�����+i�d�K��B����������m>#������L��~W7
��k�������PK?���V$TestArtifacts/
 8����8�����>����PK?[��V1�p�BG2$ ,TestArtifacts/test-case 1 -t1000 WAL ~16 bytes.png
 yN{q���x��������nq���PK?B��V�?�G�K/$ CTestArtifacts/test-case 1 -T5 WAL ~16 bytes.png
 �BzT����������y�7���PK?���V�]J/CMG3$ �TestArtifacts/test-case 2 -t1000 WAL ~256 bytes.png
 ��'����F������Z����PK?���V��)�G�L0$ v�TestArtifacts/test-case 2 -T5 WAL ~256 bytes.png
 �P������������������PK?���V�=��A!F3$ �TestArtifacts/test-case 3 -t1000 WAL ~512 bytes.png
 ��*����
�������"����PK?���V%QjE�I0$ �XTestArtifacts/test-case 3 -T5 WAL ~512 bytes.png
 IAZ�����������L����PK?���V�Qv�>>�B4$ �TestArtifacts/test-case 4 -t1000 WAL ~1024 bytes.png
 
1��������������PK?���V'P��?FaJ1$ ��TestArtifacts/test-case 4 -T5 WAL ~1024 bytes.png
 �7�����������,�����PK?��VD�+?.D4$ #TestArtifacts/test-case 5 -t1000 WAL ~2048 bytes.png
 b�>5����}�����65���PK?���V�E���E�I1$ �bTestArtifacts/test-case 5 -T5 WAL ~2048 bytes.png
 y�$�����~�����z�$���PK?,��VbM�-�:@4$ ��TestArtifacts/test-case 6 -t1000 WAL ~4096 bytes.png
 %��Y���	\{�����;�Y���PK?��V���HA,F1$ ��TestArtifacts/test-case 6 -T5 WAL ~4096 bytes.png
 5DyJ����E|�����}lJ���PK?H��V���U/=�A4$ �%TestArtifacts/test-case 7 -t1000 WAL ~8192 bytes.png
 ��y�����x����Z�y���PK?:��V�My	1=YB1$ cTestArtifacts/test-case 7 -T5 WAL ~8192 bytes.png
 e�|k����$z����>�kk���PK?���V_��$ ��TestArtifacts/WInsTestSetup.txt
 �;����k*x�����&$����PK��
v6-0001-Optimize-WAL-insertion-lock-acquisition-and-relea.patchapplication/x-patch; name=v6-0001-Optimize-WAL-insertion-lock-acquisition-and-relea.patchDownload
From 4835d9f1c990940e7c73cbae16cc2cfdb855add6 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Mon, 8 May 2023 11:31:08 +0000
Subject: [PATCH v6] Optimize WAL insertion lock acquisition and release

This commit optimizes WAL insertion lock acquisition and release
in the following way:

1. WAL insertion lock's variable insertingAt is currently read and
written with the help of lwlock's wait list lock to avoid
torn-free reads/writes. This wait list lock can become a point of
contention on a highly concurrent write workloads. Therefore, make
insertingAt a 64-bit atomic which inherently provides torn-free
reads/writes.

2. LWLockUpdateVar currently acquires lwlock's wait list lock even
when there are no waiters at all. Add a fastpath exit to
LWLockUpdateVar when there are no waiters to avoid unnecessary
locking.

Note that atomic exchange operation (which is a full barrier) is
used when necessary, instead of atomic write to ensure the memory
ordering is preserved.

It also adds a note in WaitXLogInsertionsToFinish regarding how the
use of spinlock there can avoid explicit memory barrier in some
subsequently called functions.

Suggested-by: Andres Freund
Author: Bharath Rupireddy
Reviewed-by: Nathan Bossart
Discussion: https://www.postgresql.org/message-id/20221124184619.xit4sfi52bcz2tva%40awork3.anarazel.de
---
 src/backend/access/transam/xlog.c |  8 +++-
 src/backend/storage/lmgr/lwlock.c | 67 +++++++++++++++++++------------
 src/include/storage/lwlock.h      |  6 +--
 3 files changed, 51 insertions(+), 30 deletions(-)

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index bc5a8e0569..92b0b87d1e 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -376,7 +376,7 @@ typedef struct XLogwrtResult
 typedef struct
 {
 	LWLock		lock;
-	XLogRecPtr	insertingAt;
+	pg_atomic_uint64	insertingAt;
 	XLogRecPtr	lastImportantAt;
 } WALInsertLock;
 
@@ -1495,6 +1495,10 @@ WaitXLogInsertionsToFinish(XLogRecPtr upto)
 			 * calling LWLockUpdateVar.  But if it has to sleep, it will
 			 * advertise the insertion point with LWLockUpdateVar before
 			 * sleeping.
+			 *
+			 * NB: LWLockConflictsWithVar (which is called from
+			 * LWLockWaitForVar) relies on the spinlock used above in this
+			 * function and doesn't use a memory barrier.
 			 */
 			if (LWLockWaitForVar(&WALInsertLocks[i].l.lock,
 								 &WALInsertLocks[i].l.insertingAt,
@@ -4611,7 +4615,7 @@ XLOGShmemInit(void)
 	for (i = 0; i < NUM_XLOGINSERT_LOCKS; i++)
 	{
 		LWLockInitialize(&WALInsertLocks[i].l.lock, LWTRANCHE_WAL_INSERT);
-		WALInsertLocks[i].l.insertingAt = InvalidXLogRecPtr;
+		pg_atomic_init_u64(&WALInsertLocks[i].l.insertingAt, InvalidXLogRecPtr);
 		WALInsertLocks[i].l.lastImportantAt = InvalidXLogRecPtr;
 	}
 
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 59347ab951..3cca22d2b9 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1547,9 +1547,8 @@ LWLockAcquireOrWait(LWLock *lock, LWLockMode mode)
  * *result is set to true if the lock was free, and false otherwise.
  */
 static bool
-LWLockConflictsWithVar(LWLock *lock,
-					   uint64 *valptr, uint64 oldval, uint64 *newval,
-					   bool *result)
+LWLockConflictsWithVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+					   uint64 *newval, bool *result)
 {
 	bool		mustwait;
 	uint64		value;
@@ -1572,13 +1571,11 @@ LWLockConflictsWithVar(LWLock *lock,
 	*result = false;
 
 	/*
-	 * Read value using the lwlock's wait list lock, as we can't generally
-	 * rely on atomic 64 bit reads/stores.  TODO: On platforms with a way to
-	 * do atomic 64 bit reads/writes the spinlock should be optimized away.
+	 * Reading the value atomically ensures that we don't need any explicit
+	 * locking. Note that in general, 64 bit atomic APIs in postgres inherently
+	 * provide explicit locking for the platforms without atomics support.
 	 */
-	LWLockWaitListLock(lock);
-	value = *valptr;
-	LWLockWaitListUnlock(lock);
+	value = pg_atomic_read_u64(valptr);
 
 	if (value != oldval)
 	{
@@ -1607,7 +1604,8 @@ LWLockConflictsWithVar(LWLock *lock,
  * in shared mode, returns 'true'.
  */
 bool
-LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
+LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+				 uint64 *newval)
 {
 	PGPROC	   *proc = MyProc;
 	int			extraWaits = 0;
@@ -1735,29 +1733,47 @@ LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
  * LWLockUpdateVar - Update a variable and wake up waiters atomically
  *
  * Sets *valptr to 'val', and wakes up all processes waiting for us with
- * LWLockWaitForVar().  Setting the value and waking up the processes happen
- * atomically so that any process calling LWLockWaitForVar() on the same lock
- * is guaranteed to see the new value, and act accordingly.
+ * LWLockWaitForVar().  It first sets the value atomically and then wakes up
+ * the waiting processes so that any process calling LWLockWaitForVar() on the
+ * same lock is guaranteed to see the new value, and act accordingly.
  *
  * The caller must be holding the lock in exclusive mode.
  */
 void
-LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
 	proclist_head wakeup;
 	proclist_mutable_iter iter;
 
 	PRINT_LWDEBUG("LWLockUpdateVar", lock, LW_EXCLUSIVE);
 
+	/*
+	 * Update the variable atomically first without having to acquire wait
+	 * list lock, so that if anyone looking for the lock will have chance to
+	 * grab it a bit quickly.
+	 *
+	 * NB: pg_atomic_exchange_u64 is used here as opposed to just
+	 * pg_atomic_write_u64 to update the variable. Since pg_atomic_exchange_u64
+	 * is a full barrier, we're guaranteed that all loads and stores issued
+	 * prior to setting the variable are completed before any loads or stores
+	 * issued after setting the variable. In other words, a barrier here
+	 * ensures the variable is updated before waking up waiters.
+	 */
+	pg_atomic_exchange_u64(valptr, val);
+
+	/*
+	 * Quick exit when there are no waiters. This avoids unnecessary lwlock's
+	 * wait list lock acquisition and release.
+	 */
+	if ((pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) == 0)
+		return;
+
 	proclist_init(&wakeup);
 
 	LWLockWaitListLock(lock);
 
 	Assert(pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE);
 
-	/* Update the lock's value */
-	*valptr = val;
-
 	/*
 	 * See if there are any LW_WAIT_UNTIL_FREE waiters that need to be woken
 	 * up. They are always in the front of the queue.
@@ -1873,17 +1889,18 @@ LWLockRelease(LWLock *lock)
  * LWLockReleaseClearVar - release a previously acquired lock, reset variable
  */
 void
-LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
-	LWLockWaitListLock(lock);
-
 	/*
-	 * Set the variable's value before releasing the lock, that prevents race
-	 * a race condition wherein a new locker acquires the lock, but hasn't yet
-	 * set the variables value.
+	 * Update the variable atomically first.
+	 *
+	 * NB: pg_atomic_exchange_u64 is used here as opposed to just
+	 * pg_atomic_write_u64 to update the variable. Since pg_atomic_exchange_u64
+	 * is a full barrier, we're guaranteed that all loads and stores issued
+	 * prior to setting the variable are completed before any loads or stores
+	 * issued after setting the variable.
 	 */
-	*valptr = val;
-	LWLockWaitListUnlock(lock);
+	pg_atomic_exchange_u64(valptr, val);
 
 	LWLockRelease(lock);
 }
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index d2c7afb8f4..f19bc49193 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -128,14 +128,14 @@ extern bool LWLockAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockConditionalAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockAcquireOrWait(LWLock *lock, LWLockMode mode);
 extern void LWLockRelease(LWLock *lock);
-extern void LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern void LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 extern void LWLockReleaseAll(void);
 extern bool LWLockHeldByMe(LWLock *lock);
 extern bool LWLockAnyHeldByMe(LWLock *lock, int nlocks, size_t stride);
 extern bool LWLockHeldByMeInMode(LWLock *lock, LWLockMode mode);
 
-extern bool LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval);
-extern void LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern bool LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval, uint64 *newval);
+extern void LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 
 extern Size LWLockShmemSize(void);
 extern void CreateLWLocks(void);
-- 
2.34.1

#18Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Bharath Rupireddy (#17)
Re: WAL Insertion Lock Improvements

On Mon, May 8, 2023 at 5:57 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

On Mon, Apr 10, 2023 at 9:38 AM Michael Paquier <michael@paquier.xyz> wrote:

-LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
[...]
Assert(pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE);

- /* Update the lock's value */
- *valptr = val;

The sensitive change is in LWLockUpdateVar(). I am not completely
sure to understand this removal, though. Does that influence the case
where there are waiters?

I'll send about this in a follow-up email to not overload this
response with too much data.

It helps the case when there are no waiters. IOW, it updates value
without waitlist lock when there are no waiters, so no extra waitlist
lock acquisition/release just to update the value. In turn, it helps
the other backend wanting to flush the WAL looking for the new updated
value of insertingAt in WaitXLogInsertionsToFinish(), now the flushing
backend can get the new value faster.

Another thing I was wondering about: how much does the fast-path used
in LWLockUpdateVar() influence the performance numbers? Am I right to
guess that it counts for most of the gain seen?

I'll send about this in a follow-up email to not overload this
response with too much data.

The fastpath exit in LWLockUpdateVar() doesn't seem to influence the
results much, see below results. However, it avoids waitlist lock
acquisition when there are no waiters.

test-case 1: -T5, WAL ~16 bytes
clients HEAD PATCHED with fastpath PATCHED no fast path
1 1482 1486 1457
2 1617 1620 1569
4 3174 3233 3031
8 6136 6365 5725
16 12566 12269 11685
32 24284 23621 23177
64 50135 45528 46653
128 94903 89791 89103
256 82289 152915 152835
512 62498 138838 142084
768 57083 125074 126768
1024 51308 113593 115930
2048 41084 88764 85110
4096 19939 42257 43917

Or could it be that
the removal of the spin lock in
LWLockConflictsWithVar()/LWLockWaitForVar() the point that has the
highest effect?

I'll send about this in a follow-up email to not overload this
response with too much data.

Out of 3 functions that got rid of waitlist lock
LWLockConflictsWithVar/LWLockWaitForVar, LWLockUpdateVar,
LWLockReleaseClearVar, perf reports tell that the biggest gain (for
the use-cases that I've tried) is for
LWLockConflictsWithVar/LWLockWaitForVar:

test-case 1: -T5, WAL ~16 bytes
HEAD:
+   61.89%     0.05%  postgres  [.] LWLockWaitForVar
+   43.19%     0.12%  postgres  [.] LWLockConflictsWithVar
+    1.62%     0.00%  postgres  [.] LWLockReleaseClearVar
PATCHED:
+   38.79%     0.11%  postgres  [.] LWLockWaitForVar
     0.40%     0.02%  postgres  [.] LWLockConflictsWithVar
+    2.80%     0.00%  postgres  [.] LWLockReleaseClearVar
test-case 6: -T5, WAL 4096 bytes
HEAD:
+   29.66%     0.07%  postgres  [.] LWLockWaitForVar
+   20.94%     0.08%  postgres  [.] LWLockConflictsWithVar
     0.19%     0.03%  postgres  [.] LWLockUpdateVar
PATCHED:
+    3.95%     0.08%  postgres  [.] LWLockWaitForVar
     0.19%     0.03%  postgres  [.] LWLockConflictsWithVar
+    1.73%     0.00%  postgres  [.] LWLockReleaseClearVar

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#19Nathan Bossart
nathandbossart@gmail.com
In reply to: Bharath Rupireddy (#17)
Re: WAL Insertion Lock Improvements

On Mon, May 08, 2023 at 05:57:09PM +0530, Bharath Rupireddy wrote:

I ran performance tests on the patch with different use-cases. Clearly
the patch reduces burden on LWLock's waitlist lock (evident from perf
reports [1]). However, to see visible impact in the output, the txns
must be generating small (between 16 bytes to 2 KB) amounts of WAL in
a highly concurrent manner, check the results below (FWIW, I've zipped
and attached perf images for better illustration along with test
setup).

When the txns are generating a small amount of WAL i.e. between 16
bytes to 2 KB in a highly concurrent manner, the benefit is clearly
visible in the TPS more than 2.3X improvement. When the txns are
generating more WAL i.e. more than 2 KB, the gain from reduced burden
on waitlist lock is offset by increase in the wait/release for WAL
insertion locks and no visible benefit is seen.

As the amount of WAL each txn generates increases, it looks like the
benefit gained from reduced burden on waitlist lock is offset by
increase in the wait for WAL insertion locks.

Nice.

test-case 1: -T5, WAL ~16 bytes
test-case 1: -t1000, WAL ~16 bytes

I wonder if it's worth doing a couple of long-running tests, too.

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

#20Michael Paquier
michael@paquier.xyz
In reply to: Nathan Bossart (#19)
Re: WAL Insertion Lock Improvements

On Mon, May 08, 2023 at 04:04:10PM -0700, Nathan Bossart wrote:

On Mon, May 08, 2023 at 05:57:09PM +0530, Bharath Rupireddy wrote:

test-case 1: -T5, WAL ~16 bytes
test-case 1: -t1000, WAL ~16 bytes

I wonder if it's worth doing a couple of long-running tests, too.

Yes, 5s or 1000 transactions per client is too small, though it shows
that things are going in the right direction.

(Will reply to the rest in a bit..)
--
Michael

#21Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Michael Paquier (#20)
Re: WAL Insertion Lock Improvements

On Tue, May 9, 2023 at 9:02 AM Michael Paquier <michael@paquier.xyz> wrote:

On Mon, May 08, 2023 at 04:04:10PM -0700, Nathan Bossart wrote:

On Mon, May 08, 2023 at 05:57:09PM +0530, Bharath Rupireddy wrote:

test-case 1: -T5, WAL ~16 bytes
test-case 1: -t1000, WAL ~16 bytes

I wonder if it's worth doing a couple of long-running tests, too.

Yes, 5s or 1000 transactions per client is too small, though it shows
that things are going in the right direction.

I'll pick a test case that generates a reasonable amount of WAL 256
bytes. What do you think of the following?

test-case 2: -T900, WAL ~256 bytes (for c in 1 2 4 8 16 32 64 128 256
512 768 1024 2048 4096 - takes 3.5hrs)
test-case 2: -t1000000, WAL ~256 bytes

If okay, I'll fire the tests.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#22Michael Paquier
michael@paquier.xyz
In reply to: Bharath Rupireddy (#21)
Re: WAL Insertion Lock Improvements

On Tue, May 09, 2023 at 09:24:14AM +0530, Bharath Rupireddy wrote:

I'll pick a test case that generates a reasonable amount of WAL 256
bytes. What do you think of the following?

test-case 2: -T900, WAL ~256 bytes (for c in 1 2 4 8 16 32 64 128 256
512 768 1024 2048 4096 - takes 3.5hrs)
test-case 2: -t1000000, WAL ~256 bytes

If okay, I'll fire the tests.

Sounds like a sensible duration, yes. What's your setting for
min/max_wal_size? I assume that there are still 16GB throttled with
target_completion at 0.9?
--
Michael

#23Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Michael Paquier (#22)
Re: WAL Insertion Lock Improvements

On Tue, May 9, 2023 at 9:27 AM Michael Paquier <michael@paquier.xyz> wrote:

On Tue, May 09, 2023 at 09:24:14AM +0530, Bharath Rupireddy wrote:

I'll pick a test case that generates a reasonable amount of WAL 256
bytes. What do you think of the following?

test-case 2: -T900, WAL ~256 bytes (for c in 1 2 4 8 16 32 64 128 256
512 768 1024 2048 4096 - takes 3.5hrs)
test-case 2: -t1000000, WAL ~256 bytes

If okay, I'll fire the tests.

Sounds like a sensible duration, yes. What's your setting for
min/max_wal_size? I assume that there are still 16GB throttled with
target_completion at 0.9?

Below is the configuration I've been using. I have been keeping the
checkpoints away so far to get expected numbers. Probably, something
that I should modify for this long run? Change checkpoint_timeout to
15 min or so?

max_wal_size=64GB
checkpoint_timeout=1d
shared_buffers=8GB
max_connections=5000

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#24Michael Paquier
michael@paquier.xyz
In reply to: Bharath Rupireddy (#23)
Re: WAL Insertion Lock Improvements

On Tue, May 09, 2023 at 09:34:56AM +0530, Bharath Rupireddy wrote:

Below is the configuration I've been using. I have been keeping the
checkpoints away so far to get expected numbers. Probably, something
that I should modify for this long run? Change checkpoint_timeout to
15 min or so?

max_wal_size=64GB
checkpoint_timeout=1d
shared_buffers=8GB
max_connections=5000

Noted. Something like that should be OK IMO, with all the checkpoints
generated based on the volume generated. With records that have a
fixed size, this should, I assume, lead to results that could be
compared across runs, even if the patched code would lead to more
checkpoints generated.
--
Michael

#25Michael Paquier
michael@paquier.xyz
In reply to: Bharath Rupireddy (#18)
Re: WAL Insertion Lock Improvements

On Mon, May 08, 2023 at 08:18:04PM +0530, Bharath Rupireddy wrote:

On Mon, May 8, 2023 at 5:57 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

On Mon, Apr 10, 2023 at 9:38 AM Michael Paquier <michael@paquier.xyz> wrote:

The sensitive change is in LWLockUpdateVar(). I am not completely
sure to understand this removal, though. Does that influence the case
where there are waiters?

I'll send about this in a follow-up email to not overload this
response with too much data.

It helps the case when there are no waiters. IOW, it updates value
without waitlist lock when there are no waiters, so no extra waitlist
lock acquisition/release just to update the value. In turn, it helps
the other backend wanting to flush the WAL looking for the new updated
value of insertingAt in WaitXLogInsertionsToFinish(), now the flushing
backend can get the new value faster.

Sure, which is what the memory barrier given by exchange_u64
guarantees. My thoughts on this one is that I am not completely sure
to understand that we won't miss any waiters that should have been
awaken.

The fastpath exit in LWLockUpdateVar() doesn't seem to influence the
results much, see below results. However, it avoids waitlist lock
acquisition when there are no waiters.

test-case 1: -T5, WAL ~16 bytes
clients HEAD PATCHED with fastpath PATCHED no fast path
64 50135 45528 46653
128 94903 89791 89103
256 82289 152915 152835
512 62498 138838 142084
768 57083 125074 126768
1024 51308 113593 115930
2048 41084 88764 85110
4096 19939 42257 43917

Considering that there could be a few percents of noise mixed into
that, that's not really surprising as the workload is highly
concurrent on inserts so the fast path won't really shine :)

Should we split this patch into two parts, as they aim at tackling two
different cases then? One for LWLockConflictsWithVar() and
LWLockReleaseClearVar() which are the straight-forward pieces
(using one pg_atomic_write_u64() in LWLockUpdateVar instead), then
a second for LWLockUpdateVar()?

Also, the fast path treatment in LWLockUpdateVar() may show some
better benefits when there are really few backends and a bunch of very
little records? Still, even that sounds a bit limited..

Out of 3 functions that got rid of waitlist lock
LWLockConflictsWithVar/LWLockWaitForVar, LWLockUpdateVar,
LWLockReleaseClearVar, perf reports tell that the biggest gain (for
the use-cases that I've tried) is for
LWLockConflictsWithVar/LWLockWaitForVar:

test-case 6: -T5, WAL 4096 bytes
HEAD:
+   29.66%     0.07%  postgres  [.] LWLockWaitForVar
+   20.94%     0.08%  postgres  [.] LWLockConflictsWithVar
0.19%     0.03%  postgres  [.] LWLockUpdateVar
PATCHED:
+    3.95%     0.08%  postgres  [.] LWLockWaitForVar
0.19%     0.03%  postgres  [.] LWLockConflictsWithVar
+    1.73%     0.00%  postgres  [.] LWLockReleaseClearVar

Indeed.
--
Michael

#26Michael Paquier
michael@paquier.xyz
In reply to: Michael Paquier (#25)
Re: WAL Insertion Lock Improvements

On Tue, May 09, 2023 at 02:10:20PM +0900, Michael Paquier wrote:

Should we split this patch into two parts, as they aim at tackling two
different cases then? One for LWLockConflictsWithVar() and
LWLockReleaseClearVar() which are the straight-forward pieces
(using one pg_atomic_write_u64() in LWLockUpdateVar instead), then
a second for LWLockUpdateVar()?

I have been studying that a bit more, and I'd like to take this
suggestion back. Apologies for the noise.
--
Michael

#27Michael Paquier
michael@paquier.xyz
In reply to: Bharath Rupireddy (#17)
Re: WAL Insertion Lock Improvements

On Mon, May 08, 2023 at 05:57:09PM +0530, Bharath Rupireddy wrote:

Note that I've used pg_logical_emit_message() for ease of
understanding about the txns generating various amounts of WAL, but
the pattern is the same if txns are generating various amounts of WAL
say with inserts.

Sounds good to me to just rely on that for some comparison numbers.

+    * NB: LWLockConflictsWithVar (which is called from
+    * LWLockWaitForVar) relies on the spinlock used above in this
+    * function and doesn't use a memory barrier.

This patch adds the following comment in WaitXLogInsertionsToFinish()
because lwlock.c on HEAD mentions that:
/*
* Test first to see if it the slot is free right now.
*
* XXX: the caller uses a spinlock before this, so we don't need a memory
* barrier here as far as the current usage is concerned. But that might
* not be safe in general.
*/

Should it be something where we'd better be noisy about at the top of
LWLockWaitForVar()? We don't want to add a memory barrier at the
beginning of LWLockConflictsWithVar(), still it strikes me that
somebody that aims at using LWLockWaitForVar() may miss this point
because LWLockWaitForVar() is the routine published in lwlock.h, not
LWLockConflictsWithVar(). This does not need to be really
complicated, say a note at the top of LWLockWaitForVar() among the
lines of (?):
"Be careful that LWLockConflictsWithVar() does not include a memory
barrier, hence the caller of this function may want to rely on an
explicit barrier or a spinlock to avoid memory ordering issues."

+    * NB: Note the use of pg_atomic_exchange_u64 as opposed to just
+    * pg_atomic_write_u64 to update the value. Since pg_atomic_exchange_u64 is
+    * a full barrier, we're guaranteed that the subsequent shared memory
+    * reads/writes, if any, happen after we reset the lock variable.

This mentions that the subsequent read/write operations are safe, so
this refers to anything happening after the variable is reset. As
a full barrier, should be also mention that this is also ordered with
respect to anything that the caller did before clearing the variable?
From this perspective using pg_atomic_exchange_u64() makes sense to me
in LWLockReleaseClearVar().

Wordsmithed that comment a bit.

-    * Set the variable's value before releasing the lock, that prevents race
-    * a race condition wherein a new locker acquires the lock, but hasn't yet
-    * set the variables value.
[...]
+    * NB: pg_atomic_exchange_u64 is used here as opposed to just
+    * pg_atomic_write_u64 to update the variable. Since pg_atomic_exchange_u64
+    * is a full barrier, we're guaranteed that all loads and stores issued
+    * prior to setting the variable are completed before any loads or stores
+    * issued after setting the variable.

This is the same explanation as LWLockUpdateVar(), except that we
lose the details explaining why we are doing the update before
releasing the lock.

It took me some time, but I have been able to deploy a big box to see
the effect of this patch at a rather large scale (64 vCPU, 512G of
memory), with the following test characteristics for HEAD and v6:
- TPS comparison with pgbench and pg_logical_emit_message().
- Record sizes of 16, 64, 256, 1k, 4k and 16k.
- Clients and jobs equal at 4, 16, 64, 256, 512, 1024, 2048, 4096.
- Runs of 3 mins for each of the 48 combinations, meaning 96 runs in
total.

And here are the results I got:
message_size_b | 16 | 64 | 256 | 1024 | 4096 | 16k
------------------|--------|--------|--------|--------|-------|-------
head_4_clients | 3026 | 2965 | 2846 | 2880 | 2778 | 2412
head_16_clients | 12087 | 11287 | 11670 | 11100 | 9003 | 5608
head_64_clients | 42995 | 44005 | 43592 | 35437 | 21533 | 11273
head_256_clients | 106775 | 109233 | 104201 | 80759 | 42118 | 16254
head_512_clients | 153849 | 156950 | 142915 | 99288 | 57714 | 16198
head_1024_clients | 122102 | 123895 | 114248 | 117317 | 62270 | 16261
head_2048_clients | 126730 | 115594 | 109671 | 119564 | 62454 | 16298
head_4096_clients | 111564 | 111697 | 119164 | 123483 | 62430 | 16140
v6_4_clients | 2893 | 2917 | 3087 | 2904 | 2786 | 2262
v6_16_clients | 12097 | 11387 | 11579 | 11242 | 9228 | 5661
v6_64_clients | 45124 | 46533 | 42275 | 36124 | 21696 | 11386
v6_256_clients | 121500 | 125732 | 104328 | 78989 | 41949 | 16254
v6_512_clients | 164120 | 174743 | 146677 | 98110 | 60228 | 16171
v6_1024_clients | 168990 | 180710 | 149894 | 117431 | 62271 | 16259
v6_2048_clients | 165426 | 162893 | 146322 | 132476 | 62468 | 16274
v6_4096_clients | 161283 | 158732 | 162474 | 135636 | 62461 | 16030

These tests are not showing me any degradation, and a correlation
between the record size and the number of clients where the TPS begins
to show a difference between HEAD and v6 of the patch. In short the
shorter the record, the better performance gets at a lower client
number, still this required at least 256~512 clients with even
messages of 16bytes. At the end I'm cool with that.
--
Michael

#28Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Bharath Rupireddy (#21)
2 attachment(s)
Re: WAL Insertion Lock Improvements

On Tue, May 9, 2023 at 9:24 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

On Tue, May 9, 2023 at 9:02 AM Michael Paquier <michael@paquier.xyz> wrote:

On Mon, May 08, 2023 at 04:04:10PM -0700, Nathan Bossart wrote:

On Mon, May 08, 2023 at 05:57:09PM +0530, Bharath Rupireddy wrote:

test-case 1: -T5, WAL ~16 bytes
test-case 1: -t1000, WAL ~16 bytes

I wonder if it's worth doing a couple of long-running tests, too.

Yes, 5s or 1000 transactions per client is too small, though it shows
that things are going in the right direction.

I'll pick a test case that generates a reasonable amount of WAL 256
bytes. What do you think of the following?

test-case 2: -T900, WAL ~256 bytes (for c in 1 2 4 8 16 32 64 128 256
512 768 1024 2048 4096 - takes 3.5hrs)
test-case 2: -t1000000, WAL ~256 bytes

If okay, I'll fire the tests.

test-case 2: -T900, WAL ~256 bytes - ran for about 3.5 hours and the
more than 3X improvement in TPS is seen - 3.11X @ 512 3.79 @ 768, 3.47
@ 1024, 2.27 @ 2048, 2.77 @ 4096

test-case 2: -T900, WAL ~256 bytes
clients HEAD PATCHED
1 1394 1351
2 1551 1445
4 3104 2881
8 5974 5774
16 12154 11319
32 22438 21606
64 43689 40567
128 80726 77993
256 139987 141638
512 60108 187126
768 51188 194406
1024 48766 169353
2048 46617 105961
4096 44163 122697

test-case 2: -t1000000, WAL ~256 bytes - ran for more than 12 hours
and the maximum improvement is 1.84X @ 1024 client.

test-case 2: -t1000000, WAL ~256 bytes
clients HEAD PATCHED
1 1454 1500
2 1657 1612
4 3223 3224
8 6305 6295
16 12447 12260
32 24855 24335
64 45229 44386
128 80752 79518
256 120663 119083
512 149546 159396
768 118298 181732
1024 101829 187492
2048 107506 191378
4096 125130 186728

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

test-case 2 -T900 WAL ~256 bytes.pngimage/png; name="test-case 2 -T900 WAL ~256 bytes.png"Download
�PNG


IHDR�!�`s� IDATx^��x�������p�Db���{,���~�^4~��T����j��1=AAm�i�s���z;j��hM�V}�XK�C�m1*
�/��@BH��yg�If���YsY�g��O���Z��o�k��]�f���
�{��!   �(,,�Hgggt�����g���~?�	@��&������B7�4��:N	  `�����8
�qJ��@@@ 
 �i��S���>  i Nt�@@ ��Hp���   ����@@���0|<&���H
��R}[-Uy\����������<jn��r�����c%J�L#�����'��N�
�h��L�����:&]��j�i��)��Rn�PX��� �D�b���f��Q�px�W����o��J5-�i^s��?����W&�n���lD�6\IA����@�@<��� ���@��1 " �"���5����lS?���Z7���������f���
Z���WO3W���'���m$,]g��>����6,���Yq���Z�oP	@��  D �<U@\�aj��6�f�]^1��XQ�QC�#eU�j�l�`hYt:���(�b��3q!�HYH����KvB &���%&��$�"�G��&�F�sL��x�T����&F��
�;�F����2`J�	�h�~c2��+��X����-�f�����j�m  �X"\�I�A'�l$���R<�
�'��u�!�Lr\x@�D�
�_�������8���{Ed	��h��p�R�������Cp� `��.$�v�kC�q�^Q�f4	p\�V�fU����,����B�!���Lu��[��pa>��
�e-���p� ���� `�@�{����r)J�g�IKZrs75P��*N)��yG^SR��zE���-�p������#d"D��X��&�����` ��@&�gb���@@O�*��  �H����k<p����d"p&�*�	@@ � ���"  �� ��X��&�����` ��@&�gb���@@O�*��  �H����k<p����d"p&�*�	@@ � ���"  ��l	pGk
���a����ZkZ(�1A�M�T��1�,���3�BpM  �A@\�������j�iU~ol�Ds�j��n��9q�����f9U47Q5%�u�9+<)qN�@�z��Zn,��.�:jb��G��w����k���;��r����%Duj��8�
MC����p�  )	8�.j��/��[�	F"����D����
�����"�E�Q�+��J/|�7�@�|"//�>�`���n[�y"�������d���3�hNWpx��4{�l�0�:W�����*Nz9�W.b�_=����6�����s��@�f��olr�*���d+�I[6&aA�������3��K`�W$��	_��?�pG+�h� �����<��a���3��o�������-y����C�EI9K��:�g�\�+J�Y����{!��|%� �r�!l��?`DD�$��6g>���������]���%f/=�=^vS��]b����=^NR�1����%�U.v�_D�r}�_�g�b�2"`��A_�\3_�}"`�Z��3�Y]����fA��B���"`�}������`��j����%_G�lecD��&����r��/�"������A���8g����'�| �H�.�	d
_AcKQ
4l�I��!�^�5-�|�89M6�`0�ik���
T��$���@����C�!�
�i��u��iM��_1NNS��/v�����C����"	���'�5l|!�`�@�v�$l�uZ�b��W���Ta��C���v�|a�`�@��H�.�	d
_0,���&	[�{���X>���4U��B�!�`��] _�:�+P�.���xY���4l�I��!�^�5-�|�89M6�`0�ik���
T��$���@����C�!�
�i��u��iM��_1NNS��/v�����C����"	���'�5l|!�`�@�v�$l�uZ�b��W���Ta��C���v�|a�`�@��H�.�	d
_0,���&	[�{���X>���4U��B�!�`��] _�:�+P�.���xY���4l�I��!�^�5-�|�89M6�`0�ik���
T��$���@����C�!�
�i��u��iM��_1NNS��/�#�h������*�5SSu��{W{#5,��bV=��V�����X�,
�A��u��A%���6`9H6�`/���Z;���EW���a�m��*���A�5������I�F��1�: �B�'
[�{W�PF���8Q��B����2F������P]B4��)���\��E�,B�y��u��y]��_J����/�k��w	Q]S5�2�,Sh����{�3l����_�M&�0�	�����N�>]�k��U��b��`Ap������%_G��3��0*G	����0O�Z^�8+py������2��d
�������9�m�U3G�W=-&a9�o�S��P�6���i�������)w�i���e�k�|�{c��<!�|�^D��T���/O{���$����y&O�������4���*;��m��u�hX\#��P��c)R2������zwP$.�t�}�^�"�W���_g�Ds��()��������9�t��4f����34��*�_��^�!��&�Z����������0K���
�ef��D�*���'��!�B��Df�:�y��ZJ{^k�r���)�S��*eXy��R^�z�~��-4������h���&�����F�r��23�Z"r�|]���`!G	B�dg�������}d����N�����g������w���-��������?Q��},�@������\������\'c�k'���(��#�A�C���R��N�=����7�D����GE3�S�x{�����A=���ho���z��(a��Z����m�L�4.�����'6`��f�_�����m�t�K�M�(��ly��������W;I6d�;���^C�=�1/��o4�^�2�Qt��w���v����]f�a�W$����x�����?xE��0F,��\����+i��5Q*g�d_�������,�����D�KJ�
���X�$��W�C�o��<�g?=M�N���R��Ex������	�9����������c��Zr�=���g�;l|�z�f�_�B������w��l���Qg�wx'\�g���z|�:���5���",�~9�W`�W$���0-�q��l�`y��V�,|,�����m�Hq�V/�����U���z.U"a~�lu��C��V5��{�u��*w��B�!�4{�l+�N�����XPt����f9E����J�YH���W�4����YL�'*�^�8��!�^�����L>a>Zds7$��%�� ����LV&A�sX��\4�;�x^����}�~�:4����K����	���'6@�8��^|�O�w��ArV`b>�N9W���:CO
[�{!ha���o��4Q���1������x���w_�.��,n~U��%����?��F�<3�gH'��${E�	��Pr�|���6�`D����YtY|Y�Y��.�=������U��*�h)v�K<9K�`G�����`�!w���;~V����w?t����X�%���}H�oz��������
j���e�]��>l.�u_��J_����6��7?{���KK���0t������Q��xv4O�z��-��>�u�A��+���|!�bF���a<��VJ�����/U#������
e�@����wz��b��\X����G�Ew����O�
$J�nT���z�o27�+���W._0"��G�����;�����zx��]��P:�v�!�GN�*����at�r;\�_�D��"`1N�Sq��0�,5f��)�S�������u�6���o��{�57��/��X&���u���{�������������z�']�����k	7���uC�:/�Z3r�"l|qi^�`7-A ���N��Gy�W��4V��l��W_T���;�_����X���Q}N�#L|�y����+�K�=M�x�I~�+r��C�a��m?��n�HA�� 8O����F�|�V�I�p�*M��B�����Qf����6g�,���V=B���U����q���f���nX�;��6u/cm����U�v^I�� E�a�{%6R�h���7���}4��q����g^�*[7��Y��_���J4�Wn�����!]�����������r���
�=S]����W���)UZ+{�}a����j1��kJ{e�U9V�Z���{�+�8���������/��+�=	)S;uceXZ[�c���Ii^w���kNN�_������/��������j������kI�����7�K/>�b*>�N���(�{=:��b`�+|������`l�C��]+��]�k$^�1��*e�����
��������,~]���+���#��(�z�����E�Y�E��
���QsS5���vPkM���Q9�]W{#5,��fV=��V��z�����X����'��_^�y���J��S{9
�YZ���0+9�:_^Wrj���y��zM4�<�������.�����j9Q�^��h�����u�Sc�&��VKU�H+y����<�<��B�_����C�<���������<j�OZc!�9A+��:�A7���
=��a��D�u���Z�9�]Bu�TSJ��:�S/>���vW"?�g����v��5�?l
�{&h)Kv�Qf���&R��i����y����N���_1NnR��1=���9�M`�t��fUt�M����j�#'�Vg5A��H��l�+�{^�_/(�����r"`���������`E���6��m���Mkh����P�8�>�v��|�% �0���T��Q*���G{'E���Pe�@�"��M�p"���Z=���tK�g��-����~q��&^c�g�k+h������RU6"4iMA-|�7�|��'�`u��*���d+�I[6Z����|��U�5F�'h�("�����
-x���vs��z�y�����a���2���!�I�)e���E����?��mD
���O��
�)^}.��^0Ge�#
uB��6Q�����G��l{yS~o�����r�����w�e�������^M,|���{"��+I��`��o;W��vPD38�������&��%�C��'h��������w���z�����/�&�O���X�S�,�K���+h�����o�������U�{rpVc����+�}�~5�����7����1|#�l%���gQ�P�!��7��I��@�������h|=q����\�C�=���pU��QDwFKi��/y����1�+h);+�v�a����&���g�7l|!�`�����+"wR	�
S�E��0�O�7]f�zR�����^�!��+�YO:��
T��$��	5�<�+��iJ06&�g�6<j��n����TW�?���5j��h>��?�vN�?�|�16JK}W��K�������
�����/jt��^��a�]b����=^NR�1���$�Y|�[���m����T<��n�����9��Q��]�'�!��>����h��L}g���sQ:��8��1:��a��6������}�}����*x�~��J9�����Ts������_�h�1�����,�kP�z6��7\��������������^�H�G!qc=��X���=���}j#-*o��v<G���������t�O^���!�����b�b�R�!��"�
+�}���*8�tu�C���qyum��+�horI"���,H����3�G&�@y{>���<��]�/_OO�m+}�_���s,�+P�����U��^z@]�R]�C��y��/��#��yjO��{�sTa�+�/���}}���3�!���v!�����p5��-*�E���f��w�Z^��e����� /���������f����G��G��yg������c���D�J�?H/|�����[�O��	XWL�3����:���3�`����N����c�F��]������?5���W�*�ZC�H��3�+����Un��"��{0F����IV���K*�y��.�;aR�V��k>�_,{��?t��W�����J(��g���?��;��P�jZf�-	*1�A"\�h���0"`Wqpd�s�t{�b�(���������IXs�V'c�f�)�p���j�y��aOS�
����g�\�E3k�P��	1�A�n��_'���1c��Y.R4VYlt���=�R��e1	���4Q=-w]���:�v��[Ms������k����QI��2��?�Cp�M8��/w�O7��7�Q��D�����Z����p������ks����\��Y.�t�}�����_>��=m��+'��f�y�r��5o}F���O�U$~%���W�;��(O&�;������txmi�N��U����=��yd�t��:7�Zr����V�����Xx��mE��>M�>��\�~�h\�c�O���|lQI��?�k�[iB�{���o��#f~'��
v�2�_���?�f����p�*E���N����(R����L��I|������kEh��<�1:�G�b`�2�1�V��������'����{�v��{^-�_>����e��_;6�}%��>'�p���u���`�*��Bd��|��Gym�oU�p��7^���R&�t-_��o3'��E5���A%ZU?��v���N��c��}���5[�����{:��P��������	����)<O��O@�-�N}�w�*j(�G��x�����;���7�IX<���W����,}&��Q0G����6k��
�3b~V����#���f'R~������NEh{����*���jQ�/���	��<a1g�i����\�w%X��1��~�,������Y-���J�R��g��"-�_>NQ�<���kV��+�=�y>a�I���������}t�%G�)GL�J���_�o�����^�{��X���)8�k����R'�d_���n9a��
�������C����r�f��Q>��;y�Z��>s3��1/���k(�g�r3[H%�,����i�
=O��O@�ST�6��q��I���kT���G�y�$j�w�-���m&�������z��r��B�����=
��S��	"_y�!_>�g�,����R��
�P�7s��U�~(o��j>�Z�������D��l��^p�Z��������������/�C��x;����E�U�l�������X,���4��G'���1wpn�l����h~����~�l��V5����
D�����*�J5,�J4�+���������^p������V����_}q�0t��������	�*_� ����T�����Om�4c�N0����k���r_������o�!a;����U?�����(z���%��x��K��Zf\�_�B�S�f��o��<D��'n?(��l��N�L?�{�h6,�'L�YB���'���+b����M(�2�:Sd�C���6�I����?S�c`�+�O~��?�Or��7e����!g�e�d���5��y^�q��la���6�������HWS������Z��l��������"�_�/b�H�kL�#a��wW)b�r�;�<T�QM��c�ak��j�oP��K��0����5����S�����Lvf�3���b[�j���x�&��o��f����(/�q��U�@��)_�Z���_}���x��(�mWGGV�=���E�U$
����Q�A{�Z������IDAT��6�Z���9��v�2\}��3<�{h�>�]�/h�M�"Z��Hh;b��\�Q�>j�Oh�f���6]��"`�����-y�O���[�Y����C��u�y}h�#l
���jbu�Rd���,�}�%T4��e�d0��h6h��9�Y����Wn4��m����D��Z�{H�N�����f��4��E��
���QsS5iK�w�7R������UOm�U���?u�+V��
������}ta�c�.9�Q��vr8Y���wr=Fy`�7$c���7��F���DV���\�b�{������q������z;�Q���n6�*���HF#"����_$���j���0�~�+��Ywi����
�����TQ?�V-'�����6���Z��x��&�&�������s�z��7�)�k�t�bu�#'/����=����I��g�"���0�e!�`��4��;��sB��!S��������6���"Vv��M����;l���f���x�a�]2"��.�:j��)��w��������4h'|�mk����i��m4��IW�+�z�����.�j}��-M���-
L{A9L�|��9z���R*:�;�U
���/1����uhjB6�Ex�/����,`UX�+2ya�H�_J����1?��_]��y�����=��h��
R���ah8���n�;�|�Z�O�J�+���D6�|��5(���/"`s������[��.���k�������~xAM����[�����23�Z"r�|]���Y���?4~������tn�S���s��n������w���/��WO�m��@@ ���i�:V�f�$,^:�����8���j��+���'t��o���;������p7.�=��o2����g8��J]9�����8�h��������'d��\���N��5������G�]O��l�a�9V���������t����h.�%�,]���U��f���;�����+�7����>����������3�9�H��s�o�����_�����^�H��%�1��U��"��6����=@WDo#:�$���~�Ykzv����e���������1-�g�
_���?��0Xx%,����?���vW+`!��yQ����S��
6:�!���|!�r} �t�9���Hggg4LC\v]���������u���k"^�����{�I�^0oO��B���+V:1NNS��Srb���0"`��g@���
X��|��&�f�E�����`�X��4�:%'�|�8�I����������{�"�+��oxI(��D"��pV;D��_����_;���
_D����#��~v���v�;�D�t���[�@m��'�m
���50�+P�.���xY�W��$`�8�m��'�4a�{m�
X{N<����w�v���ah�E9��R�
���'�������Y|/����sw������	Xw���+.�>���Z-���.Q{���/����.1{���N)���:��@������(BC���G�k���=|e��F��ak`�W����	����N)�.x�n���J#�����1tC�U��/�%��kkCW�_<J���JA�!h�X�|M	�O3F�\��=��o����vX|5T[#����������e�k�U
r���|�l� �
�Oo�"���m�n�M��D*�\������3D&��UJ����g�|-�J6�|�a��/{!��s��s>
l��("%�i��v�XRe^��6Z|�[4yb=��d�4���������I�r�W._�9_��f������(`$
��Qi�B*8�i�������Y��G0��g��78���}`��i`��8~s�O����TR��B*9g!�M����~�/]����>4�U:r�IF6����B)��G_�~G�\��G���h��W�������~����3�CO���8�d���_�M&��������A��_JG��Cc��JG�.�%�~�mk�wa�vH�������'����t�!��y�:������{i�u/R�(�,��C��u��tW�	��%�G�+0��/"`1�$,��/[��?Z@�e4��O���0�~2V���P��G�����z�E�/��9bG�8e|�������D��E�}���nR:O��	Y����jf�@�%�M"\�h�_D�b>�����{�����/��L%g_GE��+F��T����N�/�z��g�W:\yl=�e����1"`���,�B*����^������G�E{%ilq.��,�QV�OW�=X��L^^$�K���_���\HW�������Is^�.
�����W�[�Ig{o��JR��)�������~;nL>����@�0B���/�B���X"�+V�F�-?{�.��B{&�@S.�O��E�����m_-���I����!�5��/X�@�%�5`~����E4'���?����� �
����u�i������:���4��@�����`�|��W�z��Z:%wmZ�H�d^�mJ>�����7}A"!wEC ����
�V��}�����ccJY����m�����.�1��I�_.�K����\�;ROob���&���P�����%�V�|���G��Z�B+�+��G�M��R���H
�6���UOm�U��^}.Z%f��KW��u��_0�&/|^�8��������[��w���"4%�����8�jYN�u��7����{<�a
�pZ#b��W���T����x>0�+��Es��;|��vjl�Ds�j��X��SEsU�G��V�����G�����T�FS�����$9��Y9*�����-9�]��Z��m!�����u���_����h\r|��\wl ��.�:j��)��w����k��T����e����r���/]Jc����|I������]Ol�g�|4|���{��3�&(�)9m���wG��/��)m���;���vx1����o���D=��e����"+|�7]��y�|��g��C����U�
��d��w\J'���,�;\�~��m��7$?'�r�H4B��2�mp\��
�S��chLQ�p������^Qfo�{��]��qY�����W���t�����\`lL��<2�\����<q�����W_�\�����_�P�8;�����l�K[����!�������'|��u}��3/���[�C�&�����}�v�������(B_:�8!��
�O>i|.�9q���~dE}�{�J
rh�ic�9�����x,�D�5�zF��
���=;���8K3P�������\�;�����5����kP�w���mu�����z���b�����{�n�:i<M����	�`��BWee�g�GV�<V`�w�4i����������`�L�����������	oX�x��fz9��l���IXZ����"�0�n�Y>���`����4����!������b��9����3N�_�vhM��Z~�.u}�7����\:���J����^��[�\�h�G���2��#F ���/Qj�Eq��V��|��*./�e��m��>��)J(���D(��D�7o��S�����cLV�`���F6��j���~1v-�	�)1����o��pK��$��0_�W��B�&a�+���tN�s�:��|Ec��Z���E��J���
���v������oB_:n_/NaY��H���n���m�12����i�g�o>,
�H���$-���������Bnn�))}�cV����[;��O,+��I����m�or����@�)����Ou{m~��������_`^k���
tYa��������?��9�L�����0E�~U��1�~�������E��{�$l|���'w�Z%�g��qp��1��������g�T���<,hy�E�y��1����L���~�Y%�.x�n��I�x�k���'ZJ���������~U�W{�"i\����4j�4�GD��`L�G5������E��hDE4��t�F�����7�P���D���w������o����6i��hkd2u�Sv|MjZ�*�����^�4`�����\K�?��6�L��f���A����qV	��7�r��b���7�*���(����aR����+��X�Y�+��K�����L�����T��8�x���i`K�(�����	��Yn��Q�_�o�$�W��p�F���aR06���tvvF�x��a����M���~�\ZO����2#�eTZ��
�>��y�*��ID�rI�/��@�V	t��i:�o�����B*9g!�M1/�)�U.h�_=��\����x�w+��o�����T��.�������,�k�������,a�6���z������������F��b���K�{����o�#v0�����Xt���������+�e;,�Cp���W��d���`����C�U�%7��\���Lc/[JySty�����^�����X�$��W���!��|L��k4��}�I��%�=����W�{uM��+����/�B���X"_�5�??���J#�(=��f��3�+�;~V���������?��a��!hA�qGA[��>M�-�9M������������|�{��	��/"`�>�X"�g����?��m�@e���_R���Jt�h��0Z��p�G�/�x6��Q�0|�3��X�M�j
[��4�\3_p�����J������	��N7r9�/��kH�+�~�pV��~cQ5�<�nzhx��Np��*�����^�4.|��\H�
CV���KU��M����wGJ���t�7��o�
�
�$I��<���+�E����t�A�kH�^l����*���|�\u�|�|�@^��50�+��y�e����q0�~�������o��r�����C�=��sx�Cp���W��d���`����g���^����EC��qyo�`4�G��kj��������dak`�W���/��	�� �Y*����e:���42��owt�Yz-�������]��{]T�@V���"	���'���Ae�$���.ys���]�It��O������r��/��9���"��x��S�0���j
[��4�\3_p
���[)z�Y�������������g���f39��f39��f3y��B��P�_m�U��N��r��C���&*8��n�_��50�+�7�|������js�x���H
�6������*����e�IJ~f!V9��O��������.��
�r�|��6����w�Sc�&��VKU�A�5�������S{�7�EP��D��%Tq�_]==����`�\?_�
�
:8���������&�mvi�/<���0a����nE�+o�{��A ��9�,�_�\��	_�����_?�?�i0��3C ���������?xE�����E���g�N��n����"l����q� �g��B��P�?���5�����[�m18��@h�����H�7���N�Z��{i��;�a�x����Dh�ash�����	@@@ ���b}Gk
���]F����	YVU����������)������/��U6|  �	�B��S�	@@@�g`���t   � ��Hp���   ����@@���0|@@�@��8%��d�;�GX�K���-_W�W���j����M�utQ{c-����]�����`�6[`&����
.c�U������=u��eD������5��7�����p	K��(�Q���.���lf�������1L�w�~&�ZNT��j�F�l7����*k�ekG��6�z�AelfoP����������U1�V���
�A�����7�����tC��(��X���SvCv���=)RrcZ2"�vm���${�EL
��uh{Q{�4�
OsUu�m��w��������0���2�c���7������jV������+&�v����k`o����^����!mmk��+_��5a`�p`Y�] �xdz���*a���
m�������#���0?�y������_���M7_���?�H$a7��T��8��cP�r�h2���hob-���a�U43,#:{�'F���M��	����n���U�c;=���Fs����C��
p0Ys�/���+��:��dor-�������^j��B-�������Ox��T�������a0��-	�S?q�g��NH��L����t�&,�6�I@�>�B�s��y��u�SX53i{��26�W9H�����j)����Y�mg~�?3{��U��&�A}�j�.bOw���[w���LYN����hV}[��0����~���#ij��N3[�~��6����r���l�6=P�yB����K����e�����b0'������o��^��P�I�H�5�aH7��`o;y�  ��{�%����0�0*$�@��c��@@@@�X����w ���DI    L,�
	A@@�;`�X�$&F��   ��w,Q����BB���;�(	@@�	@��Q!!��xG�K�  � ����@@�#��%Ja`aTH  ��{�%����0�0*$�N]���������5���Es�����Z9���5��u�RM������k�eu4g�S[mU��*����d�G]��5��V����j*'}y�T������9g��LuE�
2g>+^M3�j�H>;Zkh5�Rz�N`�����0�0�v�1�7�m#����c����HK�����f��w�S����T[���G`#���7�[�0FO�X�C�nc�z}������<��T�5��A���J� ca�YN�����"Y��4�i��4o��)*���F/���jv�&�����x$�?�H,"�\&��_0�%�	@����3�����n���3�^fn$B?�����Jm^v����!��!U��aY�|��p�!	'���i�`��-��k��(�gW,G$��\x�*�����u���X������g�}���c�� �>"�>���=�Y��g�2��?��8APu�d;����P�K�1Q�AhhH�]��v����u����t��x��g������������={�3;�78��	 �G������R�i���E����]p�$%��YT�>+�vz�=*C�o�0[VcE�I�����/���O��h�7�u�~� ����T$
5p���xv���������@�������i�W��� N�x�T��F�������N���Z;����)��;��#9�T�������m0I|<�1�u�p|�rw�����JT��$�X�+��
xH
� �p��#�����������D[�O?���J���!)q�"$3V�~���:6�U�	��^�_C��F;�{����'��sJ�)������X���yG�;6O���/�|nXF�7����c_[���2�Ag�A�3�r��������|�}��a����W���&g-����H}j���;T��1T��+����Z&�pH'p:���R	l���^x���1���t���T}/|��
���3�y�=��*����@ FO�4��:N	  `�����8
�qJ��@@@ 
 �i��S���>  i Nt�@@ ��Hp���   ����@@���0|@@�@��8%�����;�D���@@@@�'999��y�@��9�IEND�B`�
test-case 2 -t1000000 WAL ~256 bytes.pngimage/png; name="test-case 2 -t1000000 WAL ~256 bytes.png"Download
�PNG


IHDR�!H�� IDATx^��	t����h����[��0f7c�<��/D���$d#�
D�0���Hp��������AI�0c�M�Y����j����z��zf�z���f�sk�V����U����������"|@@@�#PPP@�������S�;*�   @�6m�#@@@�a/���   `�#
@@@�#a����   F����G ���aA@@"��@�=�����D9  �{��0r@@<"�<  a���xD"�x@@ ��P@������N���
*QP��*[�����R���RQ�����5�@&�gbT�&�Z��j)m�����jJC��n���:Z���,�da.�@���
�q �H	$��4���*����q����&�M�T^���jk(���[c����=A��1L��@Ra$���'���
 �"Pa�K"���G�h�/��Ul�6���~�Ak,���%���Cu�;o��v-/��U���udL�c?�*��z>��	g��n	5��i��������0���d"�p&FmRD |}�C`"K�D��m���x���k�d#��u\v��m�����sx��p6z�\Q@P-hO"�}���Dx+������D���@�����,:$�2�	;�p���3p����>�J@��`E��I�GDC-�!{��e5��^�����D��E����A�@ %�pJD0��������D8�]�k�:�p��!b6��H&�a$p��=�o&lW_��������D8���������K�m70i&���D/�� �
 ��e�Q*��-E�� c�T�Ng.�*�n���J���.G��D��E)���d'	��[�>��g�����TZ�k68�9���b�������&��KV��-UTG������q��[��?R�>%"��us�'\����G��/@�}8  �� ��u�@@� ����F�l�:�  �a_�N��d#�p6Fm���/�'@@��D8��6����D�a�  �H"��QG�A@|A"��0�	�l$�����  � �E���@6�gc��f_��"p@@ 	@��1�h3���/�pS}9��
����&�/�������)�"�&���e}��p@@���F�o*��RC^�WWm�y
TB�W�����F���)_F�jk����:���<>   �1~�i$���E��^��lw	��&��Y1�{�����#�j����d9g"�f�K���������25D8�3�HI���PcH�)<����\������o�I��O	�/�@����_�I��n����3�M��i�oZ�R��o^^M�2%e�d�0���lRmt�9�	���
4��?"�r�J�={�^����2u�TW�%� ��������WF�&�|��J��|a6���3ph��j�)sV��\� ��`���d2��_!�j)����"�TO�����Gb��F�������U�@%�?�oi��=o�!������&�	;��H)�r��DW�|�������:-��DX����n",_'����ue�2��5a��"l�[��X��n9�C,T|!�a�.#f�[������n9�D���B�!���E�R��p�r�
�X��B�!�B]F�X���k�r�:�2��C������n",b�����P��C������n",_'����ue�2����aK�:DX8��t�	�+b���u1c�:DX,�N�u�	��$��e�"��/���u0��p�����W8�B�"�2b��u0��X|�X����I����/D"��_�-u�`a��-'��p��
�/D",�e��u�`a��:��-'���(��_�0D���[��� ��!.�[N�_�_�0DX������ �b�ub�[N�_'Q�/�a�0����A��C,\@�����!*�a��P�3���A�����Z����N��_|!�a��"l�[��X��n9�C,T|!�a�.#f�[������n9�D���B�!���E�R��p�r�
�X��B�!�B]F�X���k�r�:�2��C������n",b�����P��C������n",_'����ue�2����aK�:DX8��t�	�+b���u1c�:DX,�N�u�	��$��e�"��/���u0��p�����W8�B�"�2b��u0��X|�X����I�����$n�/��U����j�����wKc5U-��aV%5T���=o
�\�"����n"� ��Et�	�+`As��!�-�T�TBLx�WWm�y
Tb�75Q}�2�T[C�d�Q�}H��>a.L��t�`a���.�[N�_��:2_"SG���4U��3+f�7�U�}d��aJ�mt�`a���-�[N�_��:�_�"�f�K����K�*�6��C��u�t�`aY[�n9���J�CwM���4DXv�"aeX�e��k%�|�(�ls��I���t�		\DV�P;����	L�:��	J���9���l\E�g���!6fI��#P!�e����0_��pS=�G�O
��M)��K���k"6f��rf����Z�����Yn��_^R���W�r��x[
"���nLg�k�M��������4��j�7Ym���[N�_A(�a<�C�������t�c����%�P_�~
���3l4
��o}'������b�����!k�:XD���2�z��O�V6����qJ�x����.��@{{;��z��8b4���<����Q��S��MP*��������'
��7�T
>���-����q�/�5a�mVZ;����5���Y��e�`O��H�l}���m��%���f
vT�{F �p,�v4����}�(c�������������x����7>n����$����X����a��p3d����7h�_o�L:d�
�3r�y&���6E�l5�n�����\C\]���y�������D?�S��n���4f����{�7�TX^'��m\UW�����BC����L���L���*�����M������$:i�p�5��1x>y�7��1ul�hLc��K,n��a�i����	 ��C5����(�w�'��nWk�:e�Nv�2(r����G�^b���g�F������ �W��h1��?@�Ko�`G��>0d8
+]H�N:/=��K���o_��m�n;�c'��O"Q"��y)�w�G�D%S?�'���g�r����+
�������;r�L#���g(��_��gaN|`7��6#������m3g���N""&�f���"�WUx�f,������{�Z�x3���k�4���Tp���)����r0���8��KH5������NL���*y�Y�Ve�I�v3!����}���������z��c�p#��x�"g�V1�\/��l��6�A���h�O^�!a�'�:(nb2�SSL�G#UI6������L��i�q��.��n�D8Uy�������/ ��3gU�[�e���q��u(��)v�~���h�?��b�4�N�U�
�UI'ej�&��N��x����V�}�W���[@]��a{��	n��gS��3R
�]s����]Z,�B�$��"*r���E�/7*G��������v�Rj�H���g�N�lO���
���|c	����Wm��"�6��j��]����_Qpmv*��n+�_G���g�� �E�W�S�M
"� �T���l�s�����W�'�n�`wg�}Y����W5���u�����_+7�"�6���N���/=B�>j��~���S����.h5�����#@
��3���$U�/�B��@�<���,���������3���_�V��xu�X���3�U8@ ����|1V�~�p�#!������_���t

�{�3o�=�-!�o���Q^7�����'�|�3a�9��3av�����������!����g^�����u�0V����U����0����:����j��~��*���������g�|vi�koq���QV�`P�E���_��7���j����S%���������f�C����?�>�����d���g��6��|�:i���=G�'�]Z*�[���s�1��?�lG��UH����"�|�M��5�r���a7�B����Q�����������������<Y~�o��	��M!�|���"��s�$�W�/DX���v.�������[��,v�>�	f���F����js|��ac����=>a��7�,�a�;������n"�2Bu���WmN�/D�f���6�$��n5���M4�����;�Xn��5l����������A$�g�n9���B��a�w��������|V�4���A������t#�������#$z�o���4��Y�z�6t3ae�M7���������k%�f>@����P�{����l����_!G��LXA����3��y��Z��+�",JL����.����A�;���&,����P�8iP�|3����Pcu-�2��)�"�C���������Z��j���/�*��������y���c+>�#:��kD����z�//+'v�
	'Q+�[N�_���Z�obb"��vM��I�����pu���>XK#UWm�y
Tb
�Q���JI���3��"�nAb�"��P_��=]�}�4���Tp�\���H�V�)���U��+E�-�j���J��lw	�����ZF��o|+��H�<��w������s��oy���G�����ySh��Cx����n"�>Ut�	��6'�W�[����jM���%5{V���M��������0:��m�0�Y�O�W��k!���A��r3+�r����e�W�[+�_v.i��L��H`�:uNI����o�b
Q�!j+���>�<��IgT�#9bT.-��)�D%  �	���k�a���������4����
4��_"��'f��k��^N��������1��t������+������t�W&���������ng��	�O�r���	�uk&ln�ZM3en�J��K`b��k�Lx�����;�����{�m�'��k�?.;�N(�����u�`a���[N�_�9�RD8�V$��Y��o�6��w�;�������.E�3Wd}��6~��O�"�%<Px4]����Bc��/�6����Nj���6�pH�=g�\�����0������f�~���� ��3G����js|���� ���O"�^G������3�nn����f.3_|N�������t�c��P���u�C��q8u�����o���j�be��s�Jl�u��� ���ul�[N�_���*�a_���s�S���}�Cyt{�u�9����d	��;�����+!���A������t�	��V�S_���"����24��v~���s��+�[n���JX��ai:��){����&	������� �;Zm�����r4�	���e���J
���9�JZp���;�y�*��_3�f�<V	;�Jy����M�|��j�|�"�0��`O7�|�;��}��sm<�f�*;6ikyv}���n6�����1GSH1�r�[���j /cY�K���.����|!�
s�K�������ohg�XZqb--(?-eKE����^Z����L�Xp��������n���x���f��/DXax%�;�����+���k��E��~���"�5]��;����t��#������=D�����n>�_�I��V�^�p������Kix��0���^_�-����=]������`7]��E����VHs`�����\����3�U�5��a�9���s��T��oz'�t��?b�Z�R��>�o������Z�
`��n>�_��A-\�M�������ji��
t�FP�������:���^i�������+��c
�������N�'��n>�_QO\��/f�
s�����U/��+n0[�yv-���/	�,���G7��wv��>|����K��NZ
��o:�M��n>��t���,��f���&?477u��D$!��={[i��si,��MG��Y��I���m:	�6j������L�9����C9r0�P:��8��:t��:�2��g��2_������]4|H>-�7������X�v�Q6e���Z{���"��/����@�<g���L��3��[��r���Zi_M��*u.A����f	�z��	���}��wL'F����x*�n�DXR�������j����W�7�=�d�5�H�$�����<����t�1#����SC����eg@l}��
���y����;*�/������?�:��v7�"�0��%��`�p�O�(g;��-�|������J�4-�T-�Q�=Q�%���,U���N�|��j�|������|�m��w����v�/�?{tG_�(����f��������� 1���W����it`?
����:Pp$����2t��	�����Z2�uGEt��:
3w!�
�bc[�����>m�������>hi�2�����O�������"<rX>=��s�T�VY7�"� Tw?�:]��������b��f<Yp--��Z5����'jOJ�
/��m�R7@��7mx���3���C�L����*��z���co7m��%�����#�|�s�#o�+�GS��F�c+p�v�����������MZ;z�����/;��9u\:.�U��|�;��?��N��H-�#��;�sP�:������i�����:��
n&l�p���3��y�z���*��������������oXTh���C�=�6�?z0>�s�	��iP~��3{F����d|�����zb�u�P�Q�u�C���v3i*Fn�D8U4l~����� �&i5�x����0����7����F8rD �L���B�e!LZ�n9��_��e��lfj��I��L��F�c������fUl��	��G�e���������S�2n�v��?�����?�d��t���jT/�_�e�y�i�Iw��	�6\��,�Y)���L�K�{[�\OU)��{�����1`BYv%���m�p�_��`21�=^�C\@������wP!���St�yjT/�?�{��d�2(�k�J�g76-G-�u����|��x1}���iH�h���
H�^3��}Zg�v�V1E>�������v�G�^G���qGp�Y:�W@C/ZH�]����ET$��vd���������_) �T�����>��idS��c=����3��$��e�f��rG�`��y�N#�W�M�a��D�X'_��Et~�KD��Cc��_����� ~��gFk�iblp��[�-i��75)������&��t��������I�&����w���kV�9�����%��A��
[7�B�#x�#����^G����F����7�x�R���L����&�v/1b������{���d���X������H��;y�n�J{FFg�*�157����{��Q����#�N������ #���|S�d ;[�7��Yp���4��N�4���	�f�=������GU�K�N�
x>n���S*�|f���0Y�sv��\����w��T����)�=����� �����:��N/�_�0��=��?&�[��*���(7'@c�_N9������/�-I�.}�^Z��t���L�����>/��r,��n>3�x�7�sv������'���K���-e���<Gt��#��`]�esP%�N�B�|�;�v�rn�f���aZ�e���Bsr_$:�k4����%���L�x����6z����eD��w�%��|������
�����hv������egx��N:��������[�Mti*!�+�D�qs�V�IDAT��N{_9�M�IP�[�����R���Rd��XMUK7��2��*J����/�D2�'��>��W�E�i����I��^����e;N����NS�4=�?'$��R���	��*��3��{E?li�Y�����]��n"���e6�"knP2�������
_�|R��UO^���+ �MT_��&U������GD���������
*��Mm
����V�e�����=�Q��ci�
T�n&@���70=�����6��}i�)�����_����{'�f+�G���7����E4�2X}L&�l&<b�zf�g���5�}���];�~�a��
yI9����D8\	�%�"�f�Kh>���T1��|�[��G�����0{7����Pq�Vz�-4��Kx���O	������Z��g1�\2A>s�h*9~���}�p{����3�#����;�l.joo��CC��)�������;v�� ���HN��9�a�,�c��1��E�"����_>���[��/'��.|5�W�JZ�&,���z�^bB`}�I�w���;��sc�9U����3��2�v���k���~�>���/��������_s"Q����������p�X�����������(�1��6�F�:t0�d��x�'����Q�!�s+�e9@�L���f���,���X����z�}�t�	k�o�O^|3�o��pQ��EdK|__����=�c���P���j��a��}v�~���w���-������#ri���k����8.��$�a�-�5V���_��d�nK[�����1�����k�~Mm����\�2�r����w�&����d$9��������"�v>��*K�gU6�������?��F7i���7N7f��>o�s
M�}�r�|����x��:XZ��
�1&�F��WA������M�"��=�t*�+^z��|�
��Q7�@9���rL��n&@:~F���/�[7����L��R-E�u�"���S�=@_��w*/x�	_Ge0�:�&T��p	��02��+�K��M���aK����Utb�{��~B#��L8�N��N}����_��eD=y���U���|� ����i��h�8Y��,E��i\��(0X����������=�3)�"��m�W��H	7�b&� 2�	���Q�x	��_��/m[7 mg�
t�3aQ�LX=���'�&N��n���s����;�iZ`�W�������L
��_����C��S�{���1
"l���m����R��}��K�w����LF����5���	_������^�7z�
�&��(<������i����>��`�����:F�U|�096r�/D8&&�7��
]��8��!
�z�.�/�����n~�L���_�6��:���S���+�n�i�p��~�u�r��iX�-�B�VF��{~r����f�h�n�B�eD"��"D�+�n�i�p�_���tr���7��GA'������f�h�n�B�eD"��"D�+�n�i�p�?��l*v��|�������	 �a���u��z�a��9�A��Q����(��b�3h�p����f�h�n�B�eD"��"D�+�n�i�p�}�D{�0����{��B9��4���\�7@F�t�",#�a�!�^1vsL�[�������/���
h�E���\�7@F�t�",#�a�!�^1vsL�[E�����`3u����f]�U��{��	++(��eE���_+7�"&��'{��7s(�zh�-/������w3dt=���LXF�1VO3a��9�A��Q~��et���h����<xJ�Wga2������/DXE��z���{���1
"�������������������G��f�h�n�B�eD3a�q��c7�4�p8�M�����z.���O��U�����2���`,#�a�!�^1vsL�Q��� ��sC���n���U�!�.�w���h��A���"�:�6��~�ot���hWA1M�u��R�`�����20V�|��J���%����:u�������+W����z��������i�Q_�S���COC�F���e������
N����t�������h���GN�0�p�U��Z���a��zn����w�c�����]�Hc'��Sh�L!\��z�X�Q�c�Z�n��z���f���rj�IG�zd��7@F[u�K�2����r���	�M�7�E8���OG��S*R����#a9A�a�i��(
���0�4�qu�o����?����y�Z���N���#<�M�L���_��eD3a����t�L�����D�����hU$_���������[��j���/�*��������y;A�����q5M�}�Z�r����V�����������:DX=E��W�����pu�
oZK#UWm�y
TBL�����*%I�3���$a���Q7�@9�c9kTk�f�h�n�B�eD"��"D�+�n�i�D��v��|�)
�e�����2�����D"�i�{4�����,OD��q����-���0Y����7�	U���]�..�5�W�l��2�st����B��������Cp�����C3�u����|%�����_v.i��L��h�����<N%����N��-��@@2��[�$�0QS}9���@s[�%���V����>~����S'm;}>�8�����ga2���X����u���WmN����hK��f��4S��D�n�b����4��(v���Q���#���69kG�r�J�����Q|9 �a�i��(�&��g�1���U�@�;��Yq]�����b�~Y�s�4M����|��;��
p��9�i����V���������LXF�1VO1�����R��m�p���EXF�T�a�O�MC�%h����kU��U��	��P
#�����:DX=E��W����N�7,�:M��d@l[���Z��W1�9��	 �����u��z�a��9�e�o\�2>+

�G��J�fT���/�*�a�����d4
���QSK|���:f��=��t�?n1�
��������.w�OI�J��DG=����ja�o���J���m���o���q4����F�A��`�	cA`���+L�|�	���7+E���O�������<�v���Qo�f�h�n��6��3����X.UK|���J^y�B:u�����k��o\����2p��5+c���|���OP�|v��d�Q{���q	�~H/|������CH�	+�me`��1��/D�/�n&���G{�E��C�oy�����r�
�z�`��1��/D�/�N�7��@����v�N��������V��c���|!�|9�u"����4m�����"a�V�DBI�z�|1���@�������oN��!
���.����6>J.[aS��2_����@V������N���1oN�4� ����ys�/aeYA$d��LX-I��t�Y!���q����oN���� �jy�/�bL�����Oj�9)��7'!a�V�DBI�����L��"��m�tdp��X���Iaw�DX-o�_�i|9�"���K�?|��oNB��%�,+��,����%	���7+D�=1���n:l���G����OJ*}��$���]
"��7��/�4���={6���}���~����/�2�\>BXaS��2_����@V����f�:@�������#��0���X-c�_�0_d����_�4M�z�\����I_�	0��M�
�����,��f����C/uY7�f�����Z�/�a���}��4�r�x��B�f�N�����Z����7u�c���K]|3�x^[�m*�xK�u@ RwhdPL>�L�:U�A$��|���*��b�t�����Aa�������DBm^�o���w\HG�~:��_�V;@��"�NX��Rf��;�8���`?���`�9��������qD$,�4M�8M�)��/�B��r ��	�^6�x_�H�r�K|�<���:�e����mD������n�jV%5T��l�g�������xm��^`SO��2_�����"��H�U[h^C�P��/�I�5TZ���l)���X���C���(��-n�#��0���X-c�_�0_h!�l���SMXu���k�������
d��d�����	�%��n.k�n>�_�Ltn��������2V�w.*��@_�D���!�~�
������`PH�a��|1���������v&������
�K�t� �L^|�"��+��n�C�hg�R�1����/����@@����������r��Z�09���q��4r�O���]!��w���3n��,����   � �������nU�YqYmt�V2�^|�F���|l��`>m-�����r_��   ��#T   �i ��Q�@@@amBGA@2�D8�"����hC"�M��(��@��gZD�m@��	�4�L�(�  �
��a'�!V=�:�:
?{�=}�jkJ���1��������h)���������v���u��������+k�����n�i�K�,P�k�F�4�gos8�E���.eu���A�n��a��M��&�����oG����3i�2��_1M��k�'������
��M%T�^�ie�W������ZX�l���|��n����Ii5�
=����k���=f��"��=�rzT�Z��!�����)��O9����N��_�E}t��8�����h;"����&d>�����&�wg'��:_�]���i������m_����1��z*_3�f�������+����g�����`)�8v�#�,l�V+��f�[��}�Zw��)����YT�PA%�K)v'�~a;C'<���������r*�1�������5c��,]u\��YD�����������`.5Ph5��3�Xc#�G�����i�.+
��n�_������SMD8��^�0D��x�+�����h��-b0DXT�Co��k���I�o����������/��K��ZCyZ���V�E��7@n����J��������}n���4W���-�N�C,OZ9k���z���C����"�L�W�C����q���-��ZS�O|��Y=U�E�r
[�����0���������D��u��e�n����z0�l6Y����S>��l�7j�U����_���m�������n�/[]�Z����l�H���L�\��%�����/��]KM'�^�o�_-��8A�e�\�����eg�;VJ� ���zHD"���@�=�����D9  �{��0r@@<"�<  a���xD"�x@@ ����D�#�8,��@��   ���G�qX�#@@@�#a����   F����G ���aA@@"���Z��z	��*-�7Q}u��)���u�*ZG1��Z�����nk�w������-�UI
%)=����vk��}�f�����QmM)����d��������������G��@*^C3*�NB���i
�2Dz�Ed���D]���f��!���k��9�}�C�oi��%4�j�g3q�[�zy�T%9����w��j�I��Ca��PL�=}�������
��!yw*���Ak��eJ����/�I�Y�OS	�tS����Y���)0xx�"6�[E8^X�9��Y��I���P<3afu2���uE@� �H��%���m���w����lcn @?��$:��a�mN���]bec�!0kB3`sv8#2[�fn�t�����fj��
�jY�����b,�I2N�f{83�x"��m��r���?X�@?�0�A�}v3-��VW��s�TUvl�X�f�1�j'���3�xa	u��(�����5����\����R6�vvvsM���6�����ND8�ux����qU����$FVhC`���K���?f0=y�i�p��%��YTi^;��}���4�����a/�L�
e&�2'�-gT���KX�������^������r4WPa�=���!D�<���_�����snn��5�(*;w�83���/;�u����L���m��Z�H������1�����s�/
��5<b�	G&ms�������.��z�
i�B�`�l���#)2�������{�Q[g���O�
_N%��[����!(3�����Vm�g\#f;��E��u7���\�zpkL�s��4�?,x]�^���|���;n{
����ZJ�;�����^Q�[�2�Cgp� ��lo�M;��g>���!!f;����b�p���D���_��k��}�����Br��h��.�Jh4��D�Q��l��A/��i����tB��[r���z���>���W���?�l��t��>�� � ����D�#�8,��@��   ���G�qX�#@@@�#a����   F����G ���aA@@"��@�=�����D9  �{��0r@@<"�<  ����{���>�p�@NN��I)"����IEND�B`�
#29Michael Paquier
michael@paquier.xyz
In reply to: Bharath Rupireddy (#28)
Re: WAL Insertion Lock Improvements

On Wed, May 10, 2023 at 10:40:20PM +0530, Bharath Rupireddy wrote:

test-case 2: -T900, WAL ~256 bytes - ran for about 3.5 hours and the
more than 3X improvement in TPS is seen - 3.11X @ 512 3.79 @ 768, 3.47
@ 1024, 2.27 @ 2048, 2.77 @ 4096

[...]

test-case 2: -t1000000, WAL ~256 bytes - ran for more than 12 hours
and the maximum improvement is 1.84X @ 1024 client.

Thanks. So that's pretty close to what I was seeing when it comes to
this message size where you see much more effects under a number of
clients of at least 512~. Any of these tests have been using fsync =
on, I assume. I think that disabling fsync or just mounting pg_wal to
a tmpfs should show the same pattern for larger record sizes (after 1k
of message size the curve begins to go down with 512~ clients).
--
Michael

#30Michael Paquier
michael@paquier.xyz
In reply to: Michael Paquier (#27)
Re: WAL Insertion Lock Improvements

On Wed, May 10, 2023 at 09:04:47PM +0900, Michael Paquier wrote:

It took me some time, but I have been able to deploy a big box to see
the effect of this patch at a rather large scale (64 vCPU, 512G of
memory), with the following test characteristics for HEAD and v6:
- TPS comparison with pgbench and pg_logical_emit_message().
- Record sizes of 16, 64, 256, 1k, 4k and 16k.
- Clients and jobs equal at 4, 16, 64, 256, 512, 1024, 2048, 4096.
- Runs of 3 mins for each of the 48 combinations, meaning 96 runs in
total.

And here are the results I got:
message_size_b | 16 | 64 | 256 | 1024 | 4096 | 16k
------------------|--------|--------|--------|--------|-------|-------
head_4_clients | 3026 | 2965 | 2846 | 2880 | 2778 | 2412
head_16_clients | 12087 | 11287 | 11670 | 11100 | 9003 | 5608
head_64_clients | 42995 | 44005 | 43592 | 35437 | 21533 | 11273
head_256_clients | 106775 | 109233 | 104201 | 80759 | 42118 | 16254
head_512_clients | 153849 | 156950 | 142915 | 99288 | 57714 | 16198
head_1024_clients | 122102 | 123895 | 114248 | 117317 | 62270 | 16261
head_2048_clients | 126730 | 115594 | 109671 | 119564 | 62454 | 16298
head_4096_clients | 111564 | 111697 | 119164 | 123483 | 62430 | 16140
v6_4_clients | 2893 | 2917 | 3087 | 2904 | 2786 | 2262
v6_16_clients | 12097 | 11387 | 11579 | 11242 | 9228 | 5661
v6_64_clients | 45124 | 46533 | 42275 | 36124 | 21696 | 11386
v6_256_clients | 121500 | 125732 | 104328 | 78989 | 41949 | 16254
v6_512_clients | 164120 | 174743 | 146677 | 98110 | 60228 | 16171
v6_1024_clients | 168990 | 180710 | 149894 | 117431 | 62271 | 16259
v6_2048_clients | 165426 | 162893 | 146322 | 132476 | 62468 | 16274
v6_4096_clients | 161283 | 158732 | 162474 | 135636 | 62461 | 16030

Another thing I was wondering is if it would be able to see a
difference by reducing the I/O pressure. After mounting pg_wal to a
tmpfs, I am getting the following table:
message_size_b | 16 | 64 | 256 | 1024 | 4096 | 16000
-------------------+--------+--------+--------+--------+--------+-------
head_4_clients | 86476 | 86592 | 84645 | 76784 | 57887 | 30199
head_16_clients | 277006 | 278431 | 263238 | 228614 | 143880 | 67237
head_64_clients | 373972 | 370082 | 352217 | 297377 | 190974 | 96843
head_256_clients | 144510 | 147077 | 146281 | 189059 | 156294 | 88345
head_512_clients | 122863 | 119054 | 127790 | 162187 | 142771 | 84109
head_1024_clients | 140802 | 138728 | 147200 | 172449 | 138022 | 81054
head_2048_clients | 175950 | 164143 | 154070 | 161432 | 128205 | 76732
head_4096_clients | 161438 | 158666 | 152057 | 139520 | 113955 | 69335
v6_4_clients | 87356 | 86985 | 83933 | 76397 | 57352 | 30084
v6_16_clients | 277466 | 280125 | 259733 | 224916 | 143832 | 66589
v6_64_clients | 388352 | 386188 | 362358 | 302719 | 190353 | 96687
v6_256_clients | 365797 | 360114 | 337135 | 266851 | 172252 | 88898
v6_512_clients | 339751 | 332384 | 308182 | 249624 | 158868 | 84258
v6_1024_clients | 301294 | 295140 | 276769 | 226034 | 148392 | 80909
v6_2048_clients | 268846 | 261001 | 247110 | 205332 | 137271 | 76299
v6_4096_clients | 229322 | 227049 | 217271 | 183708 | 124888 | 69263

This shows more difference from 64 clients up to 4k records, without
degradation noticed across the board.
--
Michael

#31Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Michael Paquier (#30)
Re: WAL Insertion Lock Improvements

On Thu, May 11, 2023 at 11:56 AM Michael Paquier <michael@paquier.xyz> wrote:

On Wed, May 10, 2023 at 09:04:47PM +0900, Michael Paquier wrote:

It took me some time, but I have been able to deploy a big box to see
the effect of this patch at a rather large scale (64 vCPU, 512G of
memory), with the following test characteristics for HEAD and v6:
- TPS comparison with pgbench and pg_logical_emit_message().
- Record sizes of 16, 64, 256, 1k, 4k and 16k.
- Clients and jobs equal at 4, 16, 64, 256, 512, 1024, 2048, 4096.
- Runs of 3 mins for each of the 48 combinations, meaning 96 runs in
total.

And here are the results I got:
message_size_b | 16 | 64 | 256 | 1024 | 4096 | 16k
------------------|--------|--------|--------|--------|-------|-------
head_4_clients | 3026 | 2965 | 2846 | 2880 | 2778 | 2412
head_16_clients | 12087 | 11287 | 11670 | 11100 | 9003 | 5608
head_64_clients | 42995 | 44005 | 43592 | 35437 | 21533 | 11273
head_256_clients | 106775 | 109233 | 104201 | 80759 | 42118 | 16254
head_512_clients | 153849 | 156950 | 142915 | 99288 | 57714 | 16198
head_1024_clients | 122102 | 123895 | 114248 | 117317 | 62270 | 16261
head_2048_clients | 126730 | 115594 | 109671 | 119564 | 62454 | 16298
head_4096_clients | 111564 | 111697 | 119164 | 123483 | 62430 | 16140
v6_4_clients | 2893 | 2917 | 3087 | 2904 | 2786 | 2262
v6_16_clients | 12097 | 11387 | 11579 | 11242 | 9228 | 5661
v6_64_clients | 45124 | 46533 | 42275 | 36124 | 21696 | 11386
v6_256_clients | 121500 | 125732 | 104328 | 78989 | 41949 | 16254
v6_512_clients | 164120 | 174743 | 146677 | 98110 | 60228 | 16171
v6_1024_clients | 168990 | 180710 | 149894 | 117431 | 62271 | 16259
v6_2048_clients | 165426 | 162893 | 146322 | 132476 | 62468 | 16274
v6_4096_clients | 161283 | 158732 | 162474 | 135636 | 62461 | 16030

Another thing I was wondering is if it would be able to see a
difference by reducing the I/O pressure. After mounting pg_wal to a
tmpfs, I am getting the following table:
message_size_b | 16 | 64 | 256 | 1024 | 4096 | 16000
-------------------+--------+--------+--------+--------+--------+-------
head_4_clients | 86476 | 86592 | 84645 | 76784 | 57887 | 30199
head_16_clients | 277006 | 278431 | 263238 | 228614 | 143880 | 67237
head_64_clients | 373972 | 370082 | 352217 | 297377 | 190974 | 96843
head_256_clients | 144510 | 147077 | 146281 | 189059 | 156294 | 88345
head_512_clients | 122863 | 119054 | 127790 | 162187 | 142771 | 84109
head_1024_clients | 140802 | 138728 | 147200 | 172449 | 138022 | 81054
head_2048_clients | 175950 | 164143 | 154070 | 161432 | 128205 | 76732
head_4096_clients | 161438 | 158666 | 152057 | 139520 | 113955 | 69335
v6_4_clients | 87356 | 86985 | 83933 | 76397 | 57352 | 30084
v6_16_clients | 277466 | 280125 | 259733 | 224916 | 143832 | 66589
v6_64_clients | 388352 | 386188 | 362358 | 302719 | 190353 | 96687
v6_256_clients | 365797 | 360114 | 337135 | 266851 | 172252 | 88898
v6_512_clients | 339751 | 332384 | 308182 | 249624 | 158868 | 84258
v6_1024_clients | 301294 | 295140 | 276769 | 226034 | 148392 | 80909
v6_2048_clients | 268846 | 261001 | 247110 | 205332 | 137271 | 76299
v6_4096_clients | 229322 | 227049 | 217271 | 183708 | 124888 | 69263

This shows more difference from 64 clients up to 4k records, without
degradation noticed across the board.

Impressive. I further covered the following test cases. There's a
clear gain with the patch i.e. reducing burden on LWLock's waitlist
lock is helping out.

fsync=off, -T120:
message_size_b | 16 | 64 | 256 | 1024 | 4096 | 16384
-------------------+--------+--------+--------+--------+--------+--------
head_1_clients | 33609 | 33862 | 32975 | 29722 | 21842 | 10606
head_2_clients | 60583 | 60524 | 57833 | 53582 | 38583 | 20120
head_4_clients | 115209 | 114012 | 114077 | 102991 | 73452 | 39179
head_8_clients | 181786 | 177592 | 174404 | 155350 | 98642 | 41406
head_16_clients | 313750 | 309024 | 295375 | 253101 | 159328 | 73617
head_32_clients | 406456 | 416809 | 400527 | 344573 | 213756 | 96322
head_64_clients | 199619 | 197948 | 198871 | 208606 | 221751 | 107762
head_128_clients | 108576 | 108727 | 107606 | 112137 | 173998 | 106976
head_256_clients | 75303 | 74983 | 73986 | 76100 | 148209 | 98080
head_512_clients | 62559 | 60189 | 59588 | 61102 | 131803 | 90534
head_768_clients | 55650 | 54486 | 54813 | 55515 | 120707 | 88009
head_1024_clients | 54709 | 52395 | 51672 | 52910 | 113904 | 86116
head_2048_clients | 48640 | 47098 | 46787 | 47582 | 98394 | 80766
head_4096_clients | 43205 | 42709 | 42591 | 43649 | 88903 | 72362
v6_1_clients | 33337 | 32877 | 31880 | 29372 | 21695 | 10596
v6_2_clients | 60125 | 60682 | 58770 | 53709 | 38390 | 20266
v6_4_clients | 115338 | 114053 | 114232 | 93527 | 74409 | 40437
v6_8_clients | 179472 | 183899 | 175474 | 154547 | 101807 | 43508
v6_16_clients | 318181 | 318580 | 296591 | 258094 | 159351 | 74758
v6_32_clients | 439681 | 447005 | 428459 | 367307 | 218511 | 97635
v6_64_clients | 473440 | 478388 | 464287 | 394825 | 244365 | 109194
v6_128_clients | 384433 | 412694 | 405916 | 366046 | 232421 | 110274
v6_256_clients | 312480 | 303635 | 291900 | 307573 | 206784 | 104171
v6_512_clients | 218560 | 189207 | 216267 | 252513 | 186762 | 97918
v6_768_clients | 168432 | 155493 | 145941 | 226616 | 178178 | 95435
v6_1024_clients | 150300 | 132078 | 134657 | 224515 | 172950 | 94356
v6_2048_clients | 126941 | 120189 | 120702 | 195684 | 158683 | 88055
v6_4096_clients | 163993 | 140795 | 139702 | 170149 | 139740 | 78907

pg_wal on tmpfs, -T180:
message_size_b | 16 | 64 | 256 | 1024 | 4096 | 16384
-------------------+--------+--------+--------+--------+--------+--------
head_1_clients | 32956 | 32766 | 32244 | 29772 | 22094 | 11212
head_2_clients | 60093 | 60382 | 58825 | 53812 | 39764 | 20953
head_4_clients | 117178 | 104986 | 112060 | 103416 | 75588 | 39753
head_8_clients | 177556 | 179926 | 173413 | 156684 | 107727 | 42001
head_16_clients | 311033 | 313842 | 298362 | 261298 | 165293 | 76183
head_32_clients | 425750 | 433988 | 419193 | 370925 | 227392 | 101638
head_64_clients | 227463 | 219832 | 221421 | 235603 | 236601 | 113677
head_128_clients | 117188 | 116847 | 118414 | 123605 | 194533 | 111480
head_256_clients | 80596 | 80541 | 79130 | 83949 | 167529 | 102401
head_512_clients | 64912 | 63610 | 63209 | 65554 | 146882 | 94936
head_768_clients | 59050 | 57082 | 57061 | 58966 | 133336 | 92389
head_1024_clients | 56880 | 54951 | 54864 | 56554 | 125270 | 90893
head_2048_clients | 52148 | 49603 | 50422 | 50692 | 110789 | 86659
head_4096_clients | 47001 | 46992 | 46075 | 47793 | 99617 | 77762
v6_1_clients | 32915 | 32854 | 31676 | 29341 | 21956 | 11220
v6_2_clients | 59592 | 59146 | 58106 | 53235 | 38973 | 20943
v6_4_clients | 113947 | 114897 | 97349 | 104630 | 73628 | 40719
v6_8_clients | 177996 | 179673 | 176190 | 156831 | 104183 | 42884
v6_16_clients | 312284 | 317065 | 300130 | 268788 | 165765 | 77299
v6_32_clients | 443101 | 450025 | 436774 | 380398 | 229081 | 101916
v6_64_clients | 450794 | 469633 | 470252 | 411374 | 253232 | 113722
v6_128_clients | 413357 | 399514 | 386713 | 364070 | 236133 | 112780
v6_256_clients | 264674 | 252701 | 268273 | 296090 | 208050 | 105477
v6_512_clients | 196481 | 154815 | 158316 | 238805 | 188363 | 99507
v6_768_clients | 139839 | 132645 | 131391 | 219846 | 179226 | 97808
v6_1024_clients | 124540 | 119543 | 120140 | 206740 | 174657 | 96629
v6_2048_clients | 118793 | 113033 | 113881 | 190997 | 161421 | 91888
v6_4096_clients | 156341 | 156971 | 131391 | 177024 | 146564 | 84096

--enable-atomics=no, -T60:
message_size_b | 16 | 64 | 256 | 1024 | 4096 | 16384
-------------------+-------+-------+-------+-------+-------+-------
head_1_clients | 1701 | 1686 | 1636 | 1693 | 1523 | 1331
head_2_clients | 1751 | 1712 | 1698 | 1769 | 1690 | 1579
head_4_clients | 3328 | 3370 | 3405 | 3495 | 3107 | 2713
head_8_clients | 6580 | 6521 | 6459 | 6370 | 5470 | 4253
head_16_clients | 13433 | 13476 | 12986 | 11461 | 9249 | 6313
head_32_clients | 25697 | 26729 | 24879 | 20862 | 14344 | 9454
head_64_clients | 51499 | 48322 | 46297 | 35224 | 20970 | 13241
head_128_clients | 56777 | 57177 | 59129 | 47687 | 27591 | 16007
head_256_clients | 9555 | 10041 | 9526 | 9830 | 13179 | 15776
head_512_clients | 5795 | 5871 | 5809 | 5954 | 5828 | 15647
head_768_clients | 4322 | 4366 | 4782 | 4624 | 4853 | 12959
head_1024_clients | 4003 | 3789 | 3647 | 3865 | 4160 | 7991
head_2048_clients | 2687 | 2573 | 2569 | 2829 | 2918 | 5462
head_4096_clients | 1694 | 1802 | 1813 | 1948 | 2256 | 5862
v6_1_clients | 1560 | 1595 | 1690 | 1621 | 1526 | 1374
v6_2_clients | 1737 | 1736 | 1738 | 1663 | 1601 | 1568
v6_4_clients | 3575 | 3583 | 3449 | 3137 | 3157 | 2788
v6_8_clients | 6660 | 6900 | 6802 | 6158 | 5605 | 4521
v6_16_clients | 14084 | 12991 | 13485 | 12628 | 10025 | 6211
v6_32_clients | 26408 | 24652 | 24672 | 21441 | 14966 | 9753
v6_64_clients | 49537 | 47703 | 45583 | 33524 | 21476 | 13259
v6_128_clients | 86938 | 79745 | 73740 | 53007 | 34863 | 15901
v6_256_clients | 20391 | 21433 | 21730 | 30836 | 43821 | 15891
v6_512_clients | 13128 | 12181 | 12309 | 11596 | 14744 | 15851
v6_768_clients | 10511 | 9942 | 9713 | 9373 | 10181 | 15964
v6_1024_clients | 9264 | 8745 | 8031 | 7500 | 8762 | 15198
v6_2048_clients | 6070 | 5724 | 5939 | 5987 | 5513 | 10828
v6_4096_clients | 4322 | 4035 | 3616 | 3637 | 5628 | 10970

--enable-spinlocks=no, -T60:
message_size_b | 16 | 64 | 256 | 1024 | 4096 | 16384
-------------------+--------+--------+--------+--------+-------+-------
head_1_clients | 1644 | 1716 | 1701 | 1636 | 1569 | 1368
head_2_clients | 1779 | 1875 | 1728 | 1728 | 1770 | 1568
head_4_clients | 3448 | 3569 | 3330 | 3324 | 3319 | 2780
head_8_clients | 6159 | 6996 | 6893 | 6401 | 6308 | 4423
head_16_clients | 13195 | 13810 | 13139 | 12892 | 10744 | 6714
head_32_clients | 26752 | 26834 | 25749 | 21739 | 18071 | 9706
head_64_clients | 52303 | 49759 | 47785 | 36625 | 26993 | 13685
head_128_clients | 98325 | 89753 | 83276 | 62302 | 38515 | 16005
head_256_clients | 128075 | 124396 | 111059 | 97165 | 56941 | 15779
head_512_clients | 140908 | 132622 | 126363 | 119113 | 62572 | 15919
head_768_clients | 118694 | 111764 | 109464 | 120368 | 62129 | 15905
head_1024_clients | 102542 | 99007 | 94291 | 109485 | 62680 | 16039
head_2048_clients | 57994 | 57003 | 57410 | 60350 | 62487 | 16091
head_4096_clients | 33995 | 32944 | 34174 | 33483 | 61071 | 15655
v6_1_clients | 1743 | 1711 | 1722 | 1655 | 1588 | 1378
v6_2_clients | 1714 | 1830 | 1767 | 1667 | 1725 | 1518
v6_4_clients | 3638 | 3602 | 3594 | 3452 | 3216 | 2713
v6_8_clients | 7047 | 6671 | 7148 | 6342 | 5577 | 4573
v6_16_clients | 13885 | 13247 | 13951 | 13037 | 10570 | 6391
v6_32_clients | 27766 | 27230 | 27079 | 22911 | 17152 | 9700
v6_64_clients | 50748 | 51548 | 47852 | 36479 | 27232 | 13290
v6_128_clients | 97611 | 89554 | 85009 | 67349 | 37046 | 16005
v6_256_clients | 124475 | 128603 | 108888 | 95277 | 55021 | 15785
v6_512_clients | 181639 | 176544 | 152852 | 120914 | 62674 | 15921
v6_768_clients | 188600 | 180691 | 158997 | 128740 | 62402 | 15979
v6_1024_clients | 191845 | 180830 | 161597 | 143032 | 62426 | 15985
v6_2048_clients | 179227 | 168906 | 173510 | 149689 | 62721 | 16090
v6_4096_clients | 156613 | 152795 | 154231 | 134587 | 62245 | 15781

--enable-atomics=no --enable-spinlocks=no, -T60:
message_size_b | 16 | 64 | 256 | 1024 | 4096 | 16384
-------------------+-------+-------+-------+-------+-------+-------
head_1_clients | 1644 | 1768 | 1726 | 1698 | 1544 | 1344
head_2_clients | 1805 | 1829 | 1746 | 1869 | 1730 | 1565
head_4_clients | 3562 | 3606 | 3571 | 3656 | 3145 | 2704
head_8_clients | 6921 | 7051 | 6774 | 6676 | 5999 | 4425
head_16_clients | 13418 | 13998 | 13634 | 12640 | 9782 | 6440
head_32_clients | 21716 | 21690 | 21124 | 18977 | 14050 | 9168
head_64_clients | 27085 | 26498 | 26108 | 23048 | 17843 | 13278
head_128_clients | 26704 | 26373 | 25845 | 24056 | 19777 | 15922
head_256_clients | 24694 | 24586 | 24148 | 22525 | 23523 | 15852
head_512_clients | 21364 | 21143 | 20697 | 20334 | 21770 | 15870
head_768_clients | 16985 | 16618 | 16544 | 16511 | 17360 | 15945
head_1024_clients | 13133 | 13640 | 13521 | 13716 | 14202 | 16020
head_2048_clients | 8051 | 8140 | 7711 | 8673 | 9027 | 15091
head_4096_clients | 4692 | 4549 | 4924 | 4908 | 6853 | 14752
v6_1_clients | 1676 | 1722 | 1781 | 1681 | 1527 | 1394
v6_2_clients | 1868 | 1706 | 1868 | 1842 | 1762 | 1573
v6_4_clients | 3668 | 3591 | 3449 | 3556 | 3309 | 2707
v6_8_clients | 7279 | 6818 | 6842 | 6846 | 5888 | 4283
v6_16_clients | 13604 | 13364 | 14099 | 12851 | 9959 | 6271
v6_32_clients | 22899 | 22453 | 22488 | 20127 | 15970 | 8915
v6_64_clients | 33289 | 32943 | 32280 | 28683 | 22885 | 13215
v6_128_clients | 43614 | 42954 | 41336 | 36660 | 29107 | 15928
v6_256_clients | 46542 | 46593 | 45673 | 41064 | 38759 | 15850
v6_512_clients | 36303 | 35923 | 34640 | 32828 | 38359 | 15913
v6_768_clients | 29654 | 29822 | 29317 | 28703 | 34194 | 15903
v6_1024_clients | 25871 | 25219 | 25801 | 25099 | 29323 | 16015
v6_2048_clients | 16497 | 17041 | 16401 | 17128 | 19656 | 15962
v6_4096_clients | 10067 | 10873 | 10702 | 10540 | 12909 | 16041

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#32Michael Paquier
michael@paquier.xyz
In reply to: Bharath Rupireddy (#31)
Re: WAL Insertion Lock Improvements

On Fri, May 12, 2023 at 07:35:20AM +0530, Bharath Rupireddy wrote:

--enable-atomics=no, -T60:
--enable-spinlocks=no, -T60:
--enable-atomics=no --enable-spinlocks=no, -T60:

Thanks for these extra tests, I have not done these specific cases but
the profiles look similar to what I've seen myself. If I recall
correctly the fallback implementation of atomics just uses spinlocks
internally to force the barriers required.
--
Michael

#33Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Michael Paquier (#27)
1 attachment(s)
Re: WAL Insertion Lock Improvements

On Wed, May 10, 2023 at 5:34 PM Michael Paquier <michael@paquier.xyz> wrote:

+    * NB: LWLockConflictsWithVar (which is called from
+    * LWLockWaitForVar) relies on the spinlock used above in this
+    * function and doesn't use a memory barrier.

This patch adds the following comment in WaitXLogInsertionsToFinish()
because lwlock.c on HEAD mentions that:
/*
* Test first to see if it the slot is free right now.
*
* XXX: the caller uses a spinlock before this, so we don't need a memory
* barrier here as far as the current usage is concerned. But that might
* not be safe in general.
*/

Should it be something where we'd better be noisy about at the top of
LWLockWaitForVar()? We don't want to add a memory barrier at the
beginning of LWLockConflictsWithVar(), still it strikes me that
somebody that aims at using LWLockWaitForVar() may miss this point
because LWLockWaitForVar() is the routine published in lwlock.h, not
LWLockConflictsWithVar(). This does not need to be really
complicated, say a note at the top of LWLockWaitForVar() among the
lines of (?):
"Be careful that LWLockConflictsWithVar() does not include a memory
barrier, hence the caller of this function may want to rely on an
explicit barrier or a spinlock to avoid memory ordering issues."

+1. Now, we have comments in 3 places to warn about the
LWLockConflictsWithVar not using memory barrier - one in
WaitXLogInsertionsToFinish, one in LWLockWaitForVar and another one
(existing) in LWLockConflictsWithVar specifying where exactly a memory
barrier is needed if the caller doesn't use a spinlock. Looks fine to
me.

+    * NB: pg_atomic_exchange_u64 is used here as opposed to just
+    * pg_atomic_write_u64 to update the variable. Since pg_atomic_exchange_u64
+    * is a full barrier, we're guaranteed that all loads and stores issued
+    * prior to setting the variable are completed before any loads or stores
+    * issued after setting the variable.

This is the same explanation as LWLockUpdateVar(), except that we
lose the details explaining why we are doing the update before
releasing the lock.

I think what I have so far seems more verbose explaining what a
barrier does and all that. I honestly think we don't need to be that
verbose, thanks to README.barrier.

I simplified those 2 comments as the following:

* NB: pg_atomic_exchange_u64, having full barrier semantics will ensure
* the variable is updated before releasing the lock.

* NB: pg_atomic_exchange_u64, having full barrier semantics will ensure
* the variable is updated before waking up waiters.

Please find the attached v7 patch.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v7-0001-Optimize-WAL-insertion-lock-acquisition-and-relea.patchapplication/x-patch; name=v7-0001-Optimize-WAL-insertion-lock-acquisition-and-relea.patchDownload
From 8e4eeccadc106381bc2898c1887109f96c3db869 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Thu, 18 May 2023 05:17:05 +0000
Subject: [PATCH v7] Optimize WAL insertion lock acquisition and release

This commit optimizes WAL insertion lock acquisition and release
in the following way:

1. WAL insertion lock's variable insertingAt is currently read and
written with the help of lwlock's wait list lock to avoid
torn-free reads/writes. This wait list lock can become a point of
contention on a highly concurrent write workloads. Therefore, make
insertingAt a 64-bit atomic which inherently provides torn-free
reads/writes.

2. LWLockUpdateVar currently acquires lwlock's wait list lock even
when there are no waiters at all. Add a fastpath exit to
LWLockUpdateVar when there are no waiters to avoid unnecessary
locking.

Note that atomic exchange operation (which is a full barrier) is
used when necessary, instead of atomic write to ensure the memory
ordering is preserved.

It also adds notes on why LWLockConflictsWithVar doesn't need a
memory barrier as far as its current usage is concerned.

Suggested-by: Andres Freund
Author: Bharath Rupireddy
Reviewed-by: Nathan Bossart
Reviewed-by: Michael Paquier
Discussion: https://www.postgresql.org/message-id/20221124184619.xit4sfi52bcz2tva%40awork3.anarazel.de
---
 src/backend/access/transam/xlog.c |  8 +++-
 src/backend/storage/lmgr/lwlock.c | 64 +++++++++++++++++++------------
 src/include/storage/lwlock.h      |  6 +--
 3 files changed, 48 insertions(+), 30 deletions(-)

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index bc5a8e0569..92b0b87d1e 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -376,7 +376,7 @@ typedef struct XLogwrtResult
 typedef struct
 {
 	LWLock		lock;
-	XLogRecPtr	insertingAt;
+	pg_atomic_uint64	insertingAt;
 	XLogRecPtr	lastImportantAt;
 } WALInsertLock;
 
@@ -1495,6 +1495,10 @@ WaitXLogInsertionsToFinish(XLogRecPtr upto)
 			 * calling LWLockUpdateVar.  But if it has to sleep, it will
 			 * advertise the insertion point with LWLockUpdateVar before
 			 * sleeping.
+			 *
+			 * NB: LWLockConflictsWithVar (which is called from
+			 * LWLockWaitForVar) relies on the spinlock used above in this
+			 * function and doesn't use a memory barrier.
 			 */
 			if (LWLockWaitForVar(&WALInsertLocks[i].l.lock,
 								 &WALInsertLocks[i].l.insertingAt,
@@ -4611,7 +4615,7 @@ XLOGShmemInit(void)
 	for (i = 0; i < NUM_XLOGINSERT_LOCKS; i++)
 	{
 		LWLockInitialize(&WALInsertLocks[i].l.lock, LWTRANCHE_WAL_INSERT);
-		WALInsertLocks[i].l.insertingAt = InvalidXLogRecPtr;
+		pg_atomic_init_u64(&WALInsertLocks[i].l.insertingAt, InvalidXLogRecPtr);
 		WALInsertLocks[i].l.lastImportantAt = InvalidXLogRecPtr;
 	}
 
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 59347ab951..65a7f64314 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1547,9 +1547,8 @@ LWLockAcquireOrWait(LWLock *lock, LWLockMode mode)
  * *result is set to true if the lock was free, and false otherwise.
  */
 static bool
-LWLockConflictsWithVar(LWLock *lock,
-					   uint64 *valptr, uint64 oldval, uint64 *newval,
-					   bool *result)
+LWLockConflictsWithVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+					   uint64 *newval, bool *result)
 {
 	bool		mustwait;
 	uint64		value;
@@ -1572,13 +1571,11 @@ LWLockConflictsWithVar(LWLock *lock,
 	*result = false;
 
 	/*
-	 * Read value using the lwlock's wait list lock, as we can't generally
-	 * rely on atomic 64 bit reads/stores.  TODO: On platforms with a way to
-	 * do atomic 64 bit reads/writes the spinlock should be optimized away.
+	 * Reading the value atomically ensures that we don't need any explicit
+	 * locking. Note that in general, 64 bit atomic APIs in postgres inherently
+	 * provide explicit locking for the platforms without atomics support.
 	 */
-	LWLockWaitListLock(lock);
-	value = *valptr;
-	LWLockWaitListUnlock(lock);
+	value = pg_atomic_read_u64(valptr);
 
 	if (value != oldval)
 	{
@@ -1605,9 +1602,14 @@ LWLockConflictsWithVar(LWLock *lock,
  *
  * Note: this function ignores shared lock holders; if the lock is held
  * in shared mode, returns 'true'.
+ *
+ * Be careful that LWLockConflictsWithVar() does not include a memory barrier,
+ * hence the caller of this function may want to rely on an explicit barrier or
+ * a spinlock to avoid memory ordering issues.
  */
 bool
-LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
+LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+				 uint64 *newval)
 {
 	PGPROC	   *proc = MyProc;
 	int			extraWaits = 0;
@@ -1735,29 +1737,43 @@ LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
  * LWLockUpdateVar - Update a variable and wake up waiters atomically
  *
  * Sets *valptr to 'val', and wakes up all processes waiting for us with
- * LWLockWaitForVar().  Setting the value and waking up the processes happen
- * atomically so that any process calling LWLockWaitForVar() on the same lock
- * is guaranteed to see the new value, and act accordingly.
+ * LWLockWaitForVar().  It first sets the value atomically and then wakes up
+ * the waiting processes so that any process calling LWLockWaitForVar() on the
+ * same lock is guaranteed to see the new value, and act accordingly.
  *
  * The caller must be holding the lock in exclusive mode.
  */
 void
-LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
 	proclist_head wakeup;
 	proclist_mutable_iter iter;
 
 	PRINT_LWDEBUG("LWLockUpdateVar", lock, LW_EXCLUSIVE);
 
+	/*
+	 * Update the variable atomically first without having to acquire wait
+	 * list lock, so that if anyone looking for the lock will have chance to
+	 * grab it a bit quickly.
+	 *
+	 * NB: pg_atomic_exchange_u64, having full barrier semantics will ensure
+	 * the variable is updated before waking up waiters.
+	 */
+	pg_atomic_exchange_u64(valptr, val);
+
+	/*
+	 * Quick exit when there are no waiters. This avoids unnecessary lwlock's
+	 * wait list lock acquisition and release.
+	 */
+	if ((pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) == 0)
+		return;
+
 	proclist_init(&wakeup);
 
 	LWLockWaitListLock(lock);
 
 	Assert(pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE);
 
-	/* Update the lock's value */
-	*valptr = val;
-
 	/*
 	 * See if there are any LW_WAIT_UNTIL_FREE waiters that need to be woken
 	 * up. They are always in the front of the queue.
@@ -1873,17 +1889,15 @@ LWLockRelease(LWLock *lock)
  * LWLockReleaseClearVar - release a previously acquired lock, reset variable
  */
 void
-LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
-	LWLockWaitListLock(lock);
-
 	/*
-	 * Set the variable's value before releasing the lock, that prevents race
-	 * a race condition wherein a new locker acquires the lock, but hasn't yet
-	 * set the variables value.
+	 * Update the variable atomically first.
+	 *
+	 * NB: pg_atomic_exchange_u64, having full barrier semantics will ensure
+	 * the variable is updated before releasing the lock.
 	 */
-	*valptr = val;
-	LWLockWaitListUnlock(lock);
+	pg_atomic_exchange_u64(valptr, val);
 
 	LWLockRelease(lock);
 }
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index d2c7afb8f4..f19bc49193 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -128,14 +128,14 @@ extern bool LWLockAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockConditionalAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockAcquireOrWait(LWLock *lock, LWLockMode mode);
 extern void LWLockRelease(LWLock *lock);
-extern void LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern void LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 extern void LWLockReleaseAll(void);
 extern bool LWLockHeldByMe(LWLock *lock);
 extern bool LWLockAnyHeldByMe(LWLock *lock, int nlocks, size_t stride);
 extern bool LWLockHeldByMeInMode(LWLock *lock, LWLockMode mode);
 
-extern bool LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval);
-extern void LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern bool LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval, uint64 *newval);
+extern void LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 
 extern Size LWLockShmemSize(void);
 extern void CreateLWLocks(void);
-- 
2.34.1

#34Michael Paquier
michael@paquier.xyz
In reply to: Bharath Rupireddy (#33)
Re: WAL Insertion Lock Improvements

On Thu, May 18, 2023 at 11:18:25AM +0530, Bharath Rupireddy wrote:

I think what I have so far seems more verbose explaining what a
barrier does and all that. I honestly think we don't need to be that
verbose, thanks to README.barrier.

Agreed. This file is a mine of information.

I simplified those 2 comments as the following:

* NB: pg_atomic_exchange_u64, having full barrier semantics will ensure
* the variable is updated before releasing the lock.

* NB: pg_atomic_exchange_u64, having full barrier semantics will ensure
* the variable is updated before waking up waiters.

Please find the attached v7 patch.

Nit. These sentences seem to be worded a bit weirdly to me. How
about:
"pg_atomic_exchange_u64 has full barrier semantics, ensuring that the
variable is updated before (releasing the lock|waking up waiters)."

+ * Be careful that LWLockConflictsWithVar() does not include a memory barrier,
+ * hence the caller of this function may want to rely on an explicit barrier or
+ * a spinlock to avoid memory ordering issues.

Thanks, this addition looks OK to me.
--
Michael

#35Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Michael Paquier (#34)
1 attachment(s)
Re: WAL Insertion Lock Improvements

On Fri, May 19, 2023 at 12:24 PM Michael Paquier <michael@paquier.xyz> wrote:

On Thu, May 18, 2023 at 11:18:25AM +0530, Bharath Rupireddy wrote:

I think what I have so far seems more verbose explaining what a
barrier does and all that. I honestly think we don't need to be that
verbose, thanks to README.barrier.

Agreed. This file is a mine of information.

I simplified those 2 comments as the following:

* NB: pg_atomic_exchange_u64, having full barrier semantics will ensure
* the variable is updated before releasing the lock.

* NB: pg_atomic_exchange_u64, having full barrier semantics will ensure
* the variable is updated before waking up waiters.

Please find the attached v7 patch.

Nit. These sentences seem to be worded a bit weirdly to me. How
about:
"pg_atomic_exchange_u64 has full barrier semantics, ensuring that the
variable is updated before (releasing the lock|waking up waiters)."

I get it. How about the following similar to what
ProcessProcSignalBarrier() has?

+     * Note that pg_atomic_exchange_u64 is a full barrier, so we're guaranteed
+     * that the variable is updated before waking up waiters.
+     */
+     * Note that pg_atomic_exchange_u64 is a full barrier, so we're guaranteed
+     * that the variable is updated before releasing the lock.
      */

Please find the attached v8 patch with the above change.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v8-0001-Optimize-WAL-insertion-lock-acquisition-and-relea.patchapplication/octet-stream; name=v8-0001-Optimize-WAL-insertion-lock-acquisition-and-relea.patchDownload
From b74b6e953cb5a7e7ea1a89719893f6ce9e231bba Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Fri, 19 May 2023 15:00:21 +0000
Subject: [PATCH v8] Optimize WAL insertion lock acquisition and release

This commit optimizes WAL insertion lock acquisition and release
in the following way:

1. WAL insertion lock's variable insertingAt is currently read and
written with the help of lwlock's wait list lock to avoid
torn-free reads/writes. This wait list lock can become a point of
contention on a highly concurrent write workloads. Therefore, make
insertingAt a 64-bit atomic which inherently provides torn-free
reads/writes.

2. LWLockUpdateVar currently acquires lwlock's wait list lock even
when there are no waiters at all. Add a fastpath exit to
LWLockUpdateVar when there are no waiters to avoid unnecessary
locking.

Note that atomic exchange operation (which is a full barrier) is
used when necessary, instead of atomic write to ensure the memory
ordering is preserved.

It also adds notes on why LWLockConflictsWithVar doesn't need a
memory barrier as far as its current usage is concerned.

Suggested-by: Andres Freund
Author: Bharath Rupireddy
Reviewed-by: Nathan Bossart
Reviewed-by: Michael Paquier
Discussion: https://www.postgresql.org/message-id/20221124184619.xit4sfi52bcz2tva%40awork3.anarazel.de
---
 src/backend/access/transam/xlog.c |  8 +++-
 src/backend/storage/lmgr/lwlock.c | 64 +++++++++++++++++++------------
 src/include/storage/lwlock.h      |  6 +--
 3 files changed, 48 insertions(+), 30 deletions(-)

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index bc5a8e0569..92b0b87d1e 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -376,7 +376,7 @@ typedef struct XLogwrtResult
 typedef struct
 {
 	LWLock		lock;
-	XLogRecPtr	insertingAt;
+	pg_atomic_uint64	insertingAt;
 	XLogRecPtr	lastImportantAt;
 } WALInsertLock;
 
@@ -1495,6 +1495,10 @@ WaitXLogInsertionsToFinish(XLogRecPtr upto)
 			 * calling LWLockUpdateVar.  But if it has to sleep, it will
 			 * advertise the insertion point with LWLockUpdateVar before
 			 * sleeping.
+			 *
+			 * NB: LWLockConflictsWithVar (which is called from
+			 * LWLockWaitForVar) relies on the spinlock used above in this
+			 * function and doesn't use a memory barrier.
 			 */
 			if (LWLockWaitForVar(&WALInsertLocks[i].l.lock,
 								 &WALInsertLocks[i].l.insertingAt,
@@ -4611,7 +4615,7 @@ XLOGShmemInit(void)
 	for (i = 0; i < NUM_XLOGINSERT_LOCKS; i++)
 	{
 		LWLockInitialize(&WALInsertLocks[i].l.lock, LWTRANCHE_WAL_INSERT);
-		WALInsertLocks[i].l.insertingAt = InvalidXLogRecPtr;
+		pg_atomic_init_u64(&WALInsertLocks[i].l.insertingAt, InvalidXLogRecPtr);
 		WALInsertLocks[i].l.lastImportantAt = InvalidXLogRecPtr;
 	}
 
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 59347ab951..82266e6897 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1547,9 +1547,8 @@ LWLockAcquireOrWait(LWLock *lock, LWLockMode mode)
  * *result is set to true if the lock was free, and false otherwise.
  */
 static bool
-LWLockConflictsWithVar(LWLock *lock,
-					   uint64 *valptr, uint64 oldval, uint64 *newval,
-					   bool *result)
+LWLockConflictsWithVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+					   uint64 *newval, bool *result)
 {
 	bool		mustwait;
 	uint64		value;
@@ -1572,13 +1571,11 @@ LWLockConflictsWithVar(LWLock *lock,
 	*result = false;
 
 	/*
-	 * Read value using the lwlock's wait list lock, as we can't generally
-	 * rely on atomic 64 bit reads/stores.  TODO: On platforms with a way to
-	 * do atomic 64 bit reads/writes the spinlock should be optimized away.
+	 * Reading the value atomically ensures that we don't need any explicit
+	 * locking. Note that in general, 64 bit atomic APIs in postgres inherently
+	 * provide explicit locking for the platforms without atomics support.
 	 */
-	LWLockWaitListLock(lock);
-	value = *valptr;
-	LWLockWaitListUnlock(lock);
+	value = pg_atomic_read_u64(valptr);
 
 	if (value != oldval)
 	{
@@ -1605,9 +1602,14 @@ LWLockConflictsWithVar(LWLock *lock,
  *
  * Note: this function ignores shared lock holders; if the lock is held
  * in shared mode, returns 'true'.
+ *
+ * Be careful that LWLockConflictsWithVar() does not include a memory barrier,
+ * hence the caller of this function may want to rely on an explicit barrier or
+ * a spinlock to avoid memory ordering issues.
  */
 bool
-LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
+LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+				 uint64 *newval)
 {
 	PGPROC	   *proc = MyProc;
 	int			extraWaits = 0;
@@ -1735,29 +1737,43 @@ LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
  * LWLockUpdateVar - Update a variable and wake up waiters atomically
  *
  * Sets *valptr to 'val', and wakes up all processes waiting for us with
- * LWLockWaitForVar().  Setting the value and waking up the processes happen
- * atomically so that any process calling LWLockWaitForVar() on the same lock
- * is guaranteed to see the new value, and act accordingly.
+ * LWLockWaitForVar().  It first sets the value atomically and then wakes up
+ * the waiting processes so that any process calling LWLockWaitForVar() on the
+ * same lock is guaranteed to see the new value, and act accordingly.
  *
  * The caller must be holding the lock in exclusive mode.
  */
 void
-LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
 	proclist_head wakeup;
 	proclist_mutable_iter iter;
 
 	PRINT_LWDEBUG("LWLockUpdateVar", lock, LW_EXCLUSIVE);
 
+	/*
+	 * Update the variable atomically first without having to acquire wait
+	 * list lock, so that if anyone looking for the lock will have chance to
+	 * grab it a bit quickly.
+	 *
+	 * Note that pg_atomic_exchange_u64 is a full barrier, so we're guaranteed
+	 * that the variable is updated before waking up waiters.
+	 */
+	pg_atomic_exchange_u64(valptr, val);
+
+	/*
+	 * Quick exit when there are no waiters. This avoids unnecessary lwlock's
+	 * wait list lock acquisition and release.
+	 */
+	if ((pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) == 0)
+		return;
+
 	proclist_init(&wakeup);
 
 	LWLockWaitListLock(lock);
 
 	Assert(pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE);
 
-	/* Update the lock's value */
-	*valptr = val;
-
 	/*
 	 * See if there are any LW_WAIT_UNTIL_FREE waiters that need to be woken
 	 * up. They are always in the front of the queue.
@@ -1873,17 +1889,15 @@ LWLockRelease(LWLock *lock)
  * LWLockReleaseClearVar - release a previously acquired lock, reset variable
  */
 void
-LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
-	LWLockWaitListLock(lock);
-
 	/*
-	 * Set the variable's value before releasing the lock, that prevents race
-	 * a race condition wherein a new locker acquires the lock, but hasn't yet
-	 * set the variables value.
+	 * Update the variable atomically first.
+	 *
+	 * Note that pg_atomic_exchange_u64 is a full barrier, so we're guaranteed
+	 * that the variable is updated before releasing the lock.
 	 */
-	*valptr = val;
-	LWLockWaitListUnlock(lock);
+	pg_atomic_exchange_u64(valptr, val);
 
 	LWLockRelease(lock);
 }
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index d2c7afb8f4..f19bc49193 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -128,14 +128,14 @@ extern bool LWLockAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockConditionalAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockAcquireOrWait(LWLock *lock, LWLockMode mode);
 extern void LWLockRelease(LWLock *lock);
-extern void LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern void LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 extern void LWLockReleaseAll(void);
 extern bool LWLockHeldByMe(LWLock *lock);
 extern bool LWLockAnyHeldByMe(LWLock *lock, int nlocks, size_t stride);
 extern bool LWLockHeldByMeInMode(LWLock *lock, LWLockMode mode);
 
-extern bool LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval);
-extern void LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern bool LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval, uint64 *newval);
+extern void LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 
 extern Size LWLockShmemSize(void);
 extern void CreateLWLocks(void);
-- 
2.34.1

#36Michael Paquier
michael@paquier.xyz
In reply to: Bharath Rupireddy (#35)
Re: WAL Insertion Lock Improvements

On Fri, May 19, 2023 at 08:34:16PM +0530, Bharath Rupireddy wrote:

I get it. How about the following similar to what
ProcessProcSignalBarrier() has?

+     * Note that pg_atomic_exchange_u64 is a full barrier, so we're guaranteed
+     * that the variable is updated before waking up waiters.
+     */
+     * Note that pg_atomic_exchange_u64 is a full barrier, so we're guaranteed
+     * that the variable is updated before releasing the lock.
*/

Please find the attached v8 patch with the above change.

Simpler and consistent, nice. I don't have much more to add, so I
have switched the patch as RfC.
--
Michael

#37Michael Paquier
michael@paquier.xyz
In reply to: Michael Paquier (#36)
Re: WAL Insertion Lock Improvements

On Mon, May 22, 2023 at 09:26:25AM +0900, Michael Paquier wrote:

Simpler and consistent, nice. I don't have much more to add, so I
have switched the patch as RfC.

While at PGcon, Andres has asked me how many sockets are in the
environment I used for the tests, and lscpu tells me the following,
which is more than 1:
CPU(s): 64
On-line CPU(s) list: 0-63
Core(s) per socket: 16
Socket(s): 2
NUMA node(s): 2

@Andres: Were there any extra tests you wanted to be run for more
input?
--
Michael

#38Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Michael Paquier (#37)
Re: WAL Insertion Lock Improvements

On Wed, May 31, 2023 at 5:05 PM Michael Paquier <michael@paquier.xyz> wrote:

On Mon, May 22, 2023 at 09:26:25AM +0900, Michael Paquier wrote:

Simpler and consistent, nice. I don't have much more to add, so I
have switched the patch as RfC.

While at PGcon, Andres has asked me how many sockets are in the
environment I used for the tests,

I'm glad to know that the feature was discussed at PGCon.

and lscpu tells me the following,
which is more than 1:
CPU(s): 64
On-line CPU(s) list: 0-63
Core(s) per socket: 16
Socket(s): 2
NUMA node(s): 2

Mine says this:

CPU(s): 96
On-line CPU(s) list: 0-95
Core(s) per socket: 24
Socket(s): 2
NUMA:
NUMA node(s): 2
NUMA node0 CPU(s): 0-23,48-71
NUMA node1 CPU(s): 24-47,72-95

@Andres: Were there any extra tests you wanted to be run for more
input?

@Andres Freund please let us know your thoughts.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#39Michael Paquier
michael@paquier.xyz
In reply to: Bharath Rupireddy (#38)
Re: WAL Insertion Lock Improvements

On Mon, Jun 05, 2023 at 08:00:00AM +0530, Bharath Rupireddy wrote:

On Wed, May 31, 2023 at 5:05 PM Michael Paquier <michael@paquier.xyz> wrote:

@Andres: Were there any extra tests you wanted to be run for more
input?

@Andres Freund please let us know your thoughts.

Err, ping. It seems like this thread is waiting on input from you,
Andres?
--
Michael

#40Andres Freund
andres@anarazel.de
In reply to: Michael Paquier (#39)
Re: WAL Insertion Lock Improvements

On 2023-07-11 09:20:45 +0900, Michael Paquier wrote:

On Mon, Jun 05, 2023 at 08:00:00AM +0530, Bharath Rupireddy wrote:

On Wed, May 31, 2023 at 5:05 PM Michael Paquier <michael@paquier.xyz> wrote:

@Andres: Were there any extra tests you wanted to be run for more
input?

@Andres Freund please let us know your thoughts.

Err, ping. It seems like this thread is waiting on input from you,
Andres?

Looking. Sorry for not getting to this earlier.

- Andres

#41Andres Freund
andres@anarazel.de
In reply to: Andres Freund (#40)
Re: WAL Insertion Lock Improvements

Hi,

On 2023-07-13 14:04:31 -0700, Andres Freund wrote:

From b74b6e953cb5a7e7ea1a89719893f6ce9e231bba Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Fri, 19 May 2023 15:00:21 +0000
Subject: [PATCH v8] Optimize WAL insertion lock acquisition and release

This commit optimizes WAL insertion lock acquisition and release
in the following way:

I think this commit does too many things at once.

1. WAL insertion lock's variable insertingAt is currently read and
written with the help of lwlock's wait list lock to avoid
torn-free reads/writes. This wait list lock can become a point of
contention on a highly concurrent write workloads. Therefore, make
insertingAt a 64-bit atomic which inherently provides torn-free
reads/writes.

"inherently" is a bit strong, given that we emulate 64bit atomics where not
available...

2. LWLockUpdateVar currently acquires lwlock's wait list lock even when
there are no waiters at all. Add a fastpath exit to LWLockUpdateVar when
there are no waiters to avoid unnecessary locking.

I don't think there's enough of an explanation for why this isn't racy.

The reason it's, I think, safe, is that anyone using LWLockConflictsWithVar()
will do so twice in a row, with a barrier inbetween. But that really relies on
what I explain in the paragraph below:

It also adds notes on why LWLockConflictsWithVar doesn't need a
memory barrier as far as its current usage is concerned.

I don't think:
* NB: LWLockConflictsWithVar (which is called from
* LWLockWaitForVar) relies on the spinlock used above in this
* function and doesn't use a memory barrier.

helps to understand why any of this is safe to a meaningful degree.

The existing comments aren't obviously aren't sufficient to explain this, but
the reason it's, I think, safe today, is that we are only waiting for
insertions that started before WaitXLogInsertionsToFinish() was called. The
lack of memory barriers in the loop means that we might see locks as "unused"
that have *since* become used, which is fine, because they only can be for
later insertions that we wouldn't want to wait on anyway.

Not taking a lock to acquire the current insertingAt value means that we might
see older insertingAt value. Which should also be fine, because if we read a
too old value, we'll add ourselves to the queue, which contains atomic
operations.

diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 59347ab951..82266e6897 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1547,9 +1547,8 @@ LWLockAcquireOrWait(LWLock *lock, LWLockMode mode)
* *result is set to true if the lock was free, and false otherwise.
*/
static bool
-LWLockConflictsWithVar(LWLock *lock,
-					   uint64 *valptr, uint64 oldval, uint64 *newval,
-					   bool *result)
+LWLockConflictsWithVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+					   uint64 *newval, bool *result)
{
bool		mustwait;
uint64		value;
@@ -1572,13 +1571,11 @@ LWLockConflictsWithVar(LWLock *lock,
*result = false;
/*
-	 * Read value using the lwlock's wait list lock, as we can't generally
-	 * rely on atomic 64 bit reads/stores.  TODO: On platforms with a way to
-	 * do atomic 64 bit reads/writes the spinlock should be optimized away.
+	 * Reading the value atomically ensures that we don't need any explicit
+	 * locking. Note that in general, 64 bit atomic APIs in postgres inherently
+	 * provide explicit locking for the platforms without atomics support.
*/

This comment seems off to me. Using atomics doesn't guarantee not needing
locking. It just guarantees that we are reading a non-torn value.

@@ -1605,9 +1602,14 @@ LWLockConflictsWithVar(LWLock *lock,
*
* Note: this function ignores shared lock holders; if the lock is held
* in shared mode, returns 'true'.
+ *
+ * Be careful that LWLockConflictsWithVar() does not include a memory barrier,
+ * hence the caller of this function may want to rely on an explicit barrier or
+ * a spinlock to avoid memory ordering issues.
*/

s/careful/aware/?

s/spinlock/implied barrier via spinlock or lwlock/?

Greetings,

Andres

#42Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Andres Freund (#41)
3 attachment(s)
Re: WAL Insertion Lock Improvements

On Fri, Jul 14, 2023 at 4:17 AM Andres Freund <andres@anarazel.de> wrote:

Hi,

On 2023-07-13 14:04:31 -0700, Andres Freund wrote:

From b74b6e953cb5a7e7ea1a89719893f6ce9e231bba Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Fri, 19 May 2023 15:00:21 +0000
Subject: [PATCH v8] Optimize WAL insertion lock acquisition and release

This commit optimizes WAL insertion lock acquisition and release
in the following way:

I think this commit does too many things at once.

I've split the patch into three - 1) Make insertingAt 64-bit atomic.
2) Have better commenting on why there's no memory barrier or spinlock
in and around LWLockWaitForVar call sites. 3) Have a quick exit for
LWLockUpdateVar.

1. WAL insertion lock's variable insertingAt is currently read and
written with the help of lwlock's wait list lock to avoid
torn-free reads/writes. This wait list lock can become a point of
contention on a highly concurrent write workloads. Therefore, make
insertingAt a 64-bit atomic which inherently provides torn-free
reads/writes.

"inherently" is a bit strong, given that we emulate 64bit atomics where not
available...

Modified.

2. LWLockUpdateVar currently acquires lwlock's wait list lock even when
there are no waiters at all. Add a fastpath exit to LWLockUpdateVar when
there are no waiters to avoid unnecessary locking.

I don't think there's enough of an explanation for why this isn't racy.

The reason it's, I think, safe, is that anyone using LWLockConflictsWithVar()
will do so twice in a row, with a barrier inbetween. But that really relies on
what I explain in the paragraph below:

The twice-in-a-row lock acquisition protocol used by LWLockWaitForVar
is helping us out have quick exit in LWLockUpdateVar. Because,
LWLockWaitForVar ensures that they are added to the wait queue even if
LWLockUpdateVar thinks that there aren't waiters. Is my understanding
correct here?

It also adds notes on why LWLockConflictsWithVar doesn't need a
memory barrier as far as its current usage is concerned.

I don't think:
* NB: LWLockConflictsWithVar (which is called from
* LWLockWaitForVar) relies on the spinlock used above in this
* function and doesn't use a memory barrier.

helps to understand why any of this is safe to a meaningful degree.

The existing comments aren't obviously aren't sufficient to explain this, but
the reason it's, I think, safe today, is that we are only waiting for
insertions that started before WaitXLogInsertionsToFinish() was called. The
lack of memory barriers in the loop means that we might see locks as "unused"
that have *since* become used, which is fine, because they only can be for
later insertions that we wouldn't want to wait on anyway.

Right.

Not taking a lock to acquire the current insertingAt value means that we might
see older insertingAt value. Which should also be fine, because if we read a
too old value, we'll add ourselves to the queue, which contains atomic
operations.

Right. An older value adds ourselves to the queue in LWLockWaitForVar,
and we should be woken up eventually by LWLockUpdateVar.

This matches with my understanding. I used more or less your above
wording in 0002 patch.

/*
-      * Read value using the lwlock's wait list lock, as we can't generally
-      * rely on atomic 64 bit reads/stores.  TODO: On platforms with a way to
-      * do atomic 64 bit reads/writes the spinlock should be optimized away.
+      * Reading the value atomically ensures that we don't need any explicit
+      * locking. Note that in general, 64 bit atomic APIs in postgres inherently
+      * provide explicit locking for the platforms without atomics support.
*/

This comment seems off to me. Using atomics doesn't guarantee not needing
locking. It just guarantees that we are reading a non-torn value.

Modified the comment.

@@ -1605,9 +1602,14 @@ LWLockConflictsWithVar(LWLock *lock,
*
* Note: this function ignores shared lock holders; if the lock is held
* in shared mode, returns 'true'.
+ *
+ * Be careful that LWLockConflictsWithVar() does not include a memory barrier,
+ * hence the caller of this function may want to rely on an explicit barrier or
+ * a spinlock to avoid memory ordering issues.
*/

s/careful/aware/?

s/spinlock/implied barrier via spinlock or lwlock/?

Done.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v9-0001-Optimize-WAL-insertion-lock-acquisition-and-relea.patchapplication/octet-stream; name=v9-0001-Optimize-WAL-insertion-lock-acquisition-and-relea.patchDownload
From 95a70093687373c677786c5d1e2d41e0e756a085 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Wed, 19 Jul 2023 14:24:04 +0000
Subject: [PATCH v9] Optimize WAL insertion lock acquisition and release

WAL insertion lock variable insertingAt is currently being read
and written with the help of LWLock wait list lock to avoid
torn-free reads/writes. This wait list lock can become a point of
contention on a highly concurrent write workloads. This commit
makes insertingAt a 64-bit atomic variable which provides torn-free
reads/writes. Note that the atomic functions pg_atomic_*_u64
simulate atomic operations with a spinlock on platforms without
64-bit atomic variable support. This reduces contention on LWLock
wait list lock and improves performance of highly concurrent write
workloads - for instance, for a pgbench run with -T900, each
transaction generating WAL of size ~256 bytes, more than 3X
improvement in TPS is seen - 3.11X @ 512 clients 3.79 @ 768
clients, 3.47 @ 1024 clients, 2.27 @ 2048 clients and 2.77 @ 4096
clients.
---
 src/backend/access/transam/xlog.c |  4 +--
 src/backend/storage/lmgr/lwlock.c | 47 +++++++++++++------------------
 src/include/storage/lwlock.h      |  6 ++--
 3 files changed, 25 insertions(+), 32 deletions(-)

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 8b0710abe6..75814ac947 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -376,7 +376,7 @@ typedef struct XLogwrtResult
 typedef struct
 {
 	LWLock		lock;
-	XLogRecPtr	insertingAt;
+	pg_atomic_uint64	insertingAt;
 	XLogRecPtr	lastImportantAt;
 } WALInsertLock;
 
@@ -4611,7 +4611,7 @@ XLOGShmemInit(void)
 	for (i = 0; i < NUM_XLOGINSERT_LOCKS; i++)
 	{
 		LWLockInitialize(&WALInsertLocks[i].l.lock, LWTRANCHE_WAL_INSERT);
-		WALInsertLocks[i].l.insertingAt = InvalidXLogRecPtr;
+		pg_atomic_init_u64(&WALInsertLocks[i].l.insertingAt, InvalidXLogRecPtr);
 		WALInsertLocks[i].l.lastImportantAt = InvalidXLogRecPtr;
 	}
 
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 01d738f306..518714a15d 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1547,9 +1547,8 @@ LWLockAcquireOrWait(LWLock *lock, LWLockMode mode)
  * *result is set to true if the lock was free, and false otherwise.
  */
 static bool
-LWLockConflictsWithVar(LWLock *lock,
-					   uint64 *valptr, uint64 oldval, uint64 *newval,
-					   bool *result)
+LWLockConflictsWithVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+					   uint64 *newval, bool *result)
 {
 	bool		mustwait;
 	uint64		value;
@@ -1571,14 +1570,8 @@ LWLockConflictsWithVar(LWLock *lock,
 
 	*result = false;
 
-	/*
-	 * Read value using the lwlock's wait list lock, as we can't generally
-	 * rely on atomic 64 bit reads/stores.  TODO: On platforms with a way to
-	 * do atomic 64 bit reads/writes the spinlock should be optimized away.
-	 */
-	LWLockWaitListLock(lock);
-	value = *valptr;
-	LWLockWaitListUnlock(lock);
+	/* Reading atomically avoids getting a torn value */
+	value = pg_atomic_read_u64(valptr);
 
 	if (value != oldval)
 	{
@@ -1607,7 +1600,8 @@ LWLockConflictsWithVar(LWLock *lock,
  * in shared mode, returns 'true'.
  */
 bool
-LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
+LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+				 uint64 *newval)
 {
 	PGPROC	   *proc = MyProc;
 	int			extraWaits = 0;
@@ -1735,29 +1729,32 @@ LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
  * LWLockUpdateVar - Update a variable and wake up waiters atomically
  *
  * Sets *valptr to 'val', and wakes up all processes waiting for us with
- * LWLockWaitForVar().  Setting the value and waking up the processes happen
- * atomically so that any process calling LWLockWaitForVar() on the same lock
- * is guaranteed to see the new value, and act accordingly.
+ * LWLockWaitForVar().  It first sets the value atomically and then wakes up
+ * waiting processes so that any process calling LWLockWaitForVar() on the same
+ * lock is guaranteed to see the new value, and act accordingly.
  *
  * The caller must be holding the lock in exclusive mode.
  */
 void
-LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
 	proclist_head wakeup;
 	proclist_mutable_iter iter;
 
 	PRINT_LWDEBUG("LWLockUpdateVar", lock, LW_EXCLUSIVE);
 
+	/*
+	 * Note that pg_atomic_exchange_u64 is a full barrier, so we're guaranteed
+	 * that the variable is updated before waking up waiters.
+	 */
+	pg_atomic_exchange_u64(valptr, val);
+
 	proclist_init(&wakeup);
 
 	LWLockWaitListLock(lock);
 
 	Assert(pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE);
 
-	/* Update the lock's value */
-	*valptr = val;
-
 	/*
 	 * See if there are any LW_WAIT_UNTIL_FREE waiters that need to be woken
 	 * up. They are always in the front of the queue.
@@ -1873,17 +1870,13 @@ LWLockRelease(LWLock *lock)
  * LWLockReleaseClearVar - release a previously acquired lock, reset variable
  */
 void
-LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
-	LWLockWaitListLock(lock);
-
 	/*
-	 * Set the variable's value before releasing the lock, that prevents race
-	 * a race condition wherein a new locker acquires the lock, but hasn't yet
-	 * set the variables value.
+	 * Note that pg_atomic_exchange_u64 is a full barrier, so we're guaranteed
+	 * that the variable is updated before releasing the lock.
 	 */
-	*valptr = val;
-	LWLockWaitListUnlock(lock);
+	pg_atomic_exchange_u64(valptr, val);
 
 	LWLockRelease(lock);
 }
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index 34169e5889..d77410bdea 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -129,14 +129,14 @@ extern bool LWLockAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockConditionalAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockAcquireOrWait(LWLock *lock, LWLockMode mode);
 extern void LWLockRelease(LWLock *lock);
-extern void LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern void LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 extern void LWLockReleaseAll(void);
 extern bool LWLockHeldByMe(LWLock *lock);
 extern bool LWLockAnyHeldByMe(LWLock *lock, int nlocks, size_t stride);
 extern bool LWLockHeldByMeInMode(LWLock *lock, LWLockMode mode);
 
-extern bool LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval);
-extern void LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern bool LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval, uint64 *newval);
+extern void LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 
 extern Size LWLockShmemSize(void);
 extern void CreateLWLocks(void);
-- 
2.34.1

v9-0002-Improve-commets-in-and-around-LWLockWaitForVar-ca.patchapplication/octet-stream; name=v9-0002-Improve-commets-in-and-around-LWLockWaitForVar-ca.patchDownload
From 8bc204a469818de0ec6ae4ba81fe07af1c140c01 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Wed, 19 Jul 2023 14:50:38 +0000
Subject: [PATCH v9] Improve commets in and around LWLockWaitForVar call sites

This commit explains why it's safe to loop for LWLockWaitForVar
without a memory barrier or a spinlock in
WaitXLogInsertionsToFinish. In the loop we are only waiting for
insertions that started before WaitXLogInsertionsToFinish was
called. The lack of memory barriers in the loop means that we
might see locks as "unused" that have since become used, which is
fine, because they only can be for later insertions that we
wouldn't want to wait on anyway. And, not taking a lock to
acquire the current insertingAt value means that we might see
older insertingAt value. Which is also be fine, because if we read
a too old value, we will add ourselves to the queue, which
contains atomic operations.
---
 src/backend/access/transam/xlog.c | 11 +++++++++++
 src/backend/storage/lmgr/lwlock.c | 10 +++++++---
 2 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 75814ac947..2fec31688c 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1495,6 +1495,17 @@ WaitXLogInsertionsToFinish(XLogRecPtr upto)
 			 * calling LWLockUpdateVar.  But if it has to sleep, it will
 			 * advertise the insertion point with LWLockUpdateVar before
 			 * sleeping.
+			 *
+			 * In this loop we are only waiting for insertions that started
+			 * before WaitXLogInsertionsToFinish was called. The lack of memory
+			 * barriers in the loop means that we might see locks as "unused"
+			 * that have since become used, which is fine, because they only
+			 * can be for later insertions that we wouldn't want to wait on
+			 * anyway. And, not taking a lock to acquire the current
+			 * insertingAt value means that we might see older insertingAt
+			 * value. Which is also be fine, because if we read a too old
+			 * value, we will add ourselves to the queue, which contains atomic
+			 * operations.
 			 */
 			if (LWLockWaitForVar(&WALInsertLocks[i].l.lock,
 								 &WALInsertLocks[i].l.insertingAt,
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 518714a15d..dad53591c7 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1556,9 +1556,9 @@ LWLockConflictsWithVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
 	/*
 	 * Test first to see if it the slot is free right now.
 	 *
-	 * XXX: the caller uses a spinlock before this, so we don't need a memory
-	 * barrier here as far as the current usage is concerned.  But that might
-	 * not be safe in general.
+	 * XXX: the caller (WaitXLogInsertionsToFinish) uses an implied barrier via
+	 * spinlock before this, so we don't need a memory barrier here as far as
+	 * the current usage is concerned. But that might not be safe in general.
 	 */
 	mustwait = (pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE) != 0;
 
@@ -1598,6 +1598,10 @@ LWLockConflictsWithVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
  *
  * Note: this function ignores shared lock holders; if the lock is held
  * in shared mode, returns 'true'.
+ *
+ * Be aware that LWLockConflictsWithVar() does not include a memory barrier,
+ * hence the caller of this function may want to rely on an explicit barrier or
+ * an implied barrier via spinlock or LWLock to avoid memory ordering issues.
  */
 bool
 LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
-- 
2.34.1

v9-0003-Have-a-quick-exit-for-LWLockUpdateVar-when-there-.patchapplication/octet-stream; name=v9-0003-Have-a-quick-exit-for-LWLockUpdateVar-when-there-.patchDownload
From 5623acafc6f9729ff1ed2c539e26b00c04c23fef Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Wed, 19 Jul 2023 15:35:51 +0000
Subject: [PATCH v9] Have a quick exit for LWLockUpdateVar when there are no
 waiters

---
 src/backend/storage/lmgr/lwlock.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index dad53591c7..bd84f63d11 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1753,6 +1753,19 @@ LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 	 */
 	pg_atomic_exchange_u64(valptr, val);
 
+	/*
+	 * Quick exit when there are no waiters.
+	 *
+	 * The twice-in-a-row lock acquisition protocol used by LWLockWaitForVar is
+	 * helping us out have quick exit here when there are no waiters without
+	 * acquiring LWLock wait list lock. LWLockWaitForVar ensures that the
+	 * waiters are added to wait queue even if LWLockUpdateVar thinks that
+	 * there aren't any waiters actually. This avoids unnecessary LWLock wait
+	 * list lock acquisition and release when there are no waiters at all.
+	 */
+	if ((pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) == 0)
+		return;
+
 	proclist_init(&wakeup);
 
 	LWLockWaitListLock(lock);
-- 
2.34.1

#43Michael Paquier
michael@paquier.xyz
In reply to: Bharath Rupireddy (#42)
Re: WAL Insertion Lock Improvements

On Thu, Jul 20, 2023 at 02:38:29PM +0530, Bharath Rupireddy wrote:

On Fri, Jul 14, 2023 at 4:17 AM Andres Freund <andres@anarazel.de> wrote:

I think this commit does too many things at once.

I've split the patch into three - 1) Make insertingAt 64-bit atomic.
2) Have better commenting on why there's no memory barrier or spinlock
in and around LWLockWaitForVar call sites. 3) Have a quick exit for
LWLockUpdateVar.

FWIW, I was kind of already OK with 0001, as it shows most of the
gains observed while 0003 had a limited impact:
/messages/by-id/CALj2ACWgeOPEKVY-TEPvno=Vatyzrb-vEEP8hN7QqrQ=yPRupA@mail.gmail.com

It is kind of a no-brainer to replace the spinlocks with atomic reads
and writes there.

I don't think:
* NB: LWLockConflictsWithVar (which is called from
* LWLockWaitForVar) relies on the spinlock used above in this
* function and doesn't use a memory barrier.

helps to understand why any of this is safe to a meaningful degree.

The existing comments aren't obviously aren't sufficient to explain this, but
the reason it's, I think, safe today, is that we are only waiting for
insertions that started before WaitXLogInsertionsToFinish() was called. The
lack of memory barriers in the loop means that we might see locks as "unused"
that have *since* become used, which is fine, because they only can be for
later insertions that we wouldn't want to wait on anyway.

Right.

FWIW, I always have a hard time coming back to this code and see it
rely on undocumented assumptions with code in lwlock.c while we need
to keep an eye in xlog.c (we take a spinlock there while the variable
wait logic relies on it for ordering @-@). So the proposal of getting
more documentation in place via 0002 goes in the right direction.

This comment seems off to me. Using atomics doesn't guarantee not needing
locking. It just guarantees that we are reading a non-torn value.

Modified the comment.

-       /*
-        * Read value using the lwlock's wait list lock, as we can't generally
-        * rely on atomic 64 bit reads/stores.  TODO: On platforms with a way to
-        * do atomic 64 bit reads/writes the spinlock should be optimized away.
-        */
-       LWLockWaitListLock(lock);
-       value = *valptr;
-       LWLockWaitListUnlock(lock);
+       /* Reading atomically avoids getting a torn value */
+       value = pg_atomic_read_u64(valptr);

Should this specify that this is specifically important for platforms
where reading a uint64 could lead to a torn value read, if you apply
this term in this context? Sounding picky, I would make that a bit
longer, say something like that:
"Reading this value atomically is safe even on platforms where uint64
cannot be read without observing a torn value."

Only xlogprefetcher.c uses the term "torn" for a value by the way, but
for a write.

@@ -1605,9 +1602,14 @@ LWLockConflictsWithVar(LWLock *lock,
*
* Note: this function ignores shared lock holders; if the lock is held
* in shared mode, returns 'true'.
+ *
+ * Be careful that LWLockConflictsWithVar() does not include a memory barrier,
+ * hence the caller of this function may want to rely on an explicit barrier or
+ * a spinlock to avoid memory ordering issues.
*/

s/careful/aware/?

s/spinlock/implied barrier via spinlock or lwlock/?

Done.

Okay to mention a LWLock here, even if the sole caller of this routine
relies on a spinlock.

0001 looks OK-ish seen from here. Thoughts?
--
Michael

#44Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Michael Paquier (#43)
3 attachment(s)
Re: WAL Insertion Lock Improvements

On Fri, Jul 21, 2023 at 11:29 AM Michael Paquier <michael@paquier.xyz> wrote:

+       /* Reading atomically avoids getting a torn value */
+       value = pg_atomic_read_u64(valptr);

Should this specify that this is specifically important for platforms
where reading a uint64 could lead to a torn value read, if you apply
this term in this context? Sounding picky, I would make that a bit
longer, say something like that:
"Reading this value atomically is safe even on platforms where uint64
cannot be read without observing a torn value."

Only xlogprefetcher.c uses the term "torn" for a value by the way, but
for a write.

Done.

0001 looks OK-ish seen from here. Thoughts?

Yes, it looks safe to me too. FWIW, 0001 essentially implements what
an existing TODO comment introduced by commit 008608b9d5106 says:

/*
* Read value using the lwlock's wait list lock, as we can't generally
* rely on atomic 64 bit reads/stores. TODO: On platforms with a way to
* do atomic 64 bit reads/writes the spinlock should be optimized away.
*/

I'm attaching v10 patch set here - 0001 has modified the comment as
above, no other changes in patch set.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

v10-0001-Optimize-WAL-insertion-lock-acquisition-and-rele.patchapplication/octet-stream; name=v10-0001-Optimize-WAL-insertion-lock-acquisition-and-rele.patchDownload
From 731041bbafcd75698de0fa84141fa79f65f8d508 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Sat, 22 Jul 2023 07:34:40 +0000
Subject: [PATCH v10] Optimize WAL insertion lock acquisition and release

WAL insertion lock variable insertingAt is currently being read
and written with the help of LWLock wait list lock to avoid
torn-free reads/writes. This wait list lock can become a point of
contention on a highly concurrent write workloads. This commit
makes insertingAt a 64-bit atomic variable which provides torn-free
reads/writes. Note that the atomic functions pg_atomic_*_u64
simulate atomic operations with a spinlock on platforms without
64-bit atomic variable support. This reduces contention on LWLock
wait list lock and improves performance of highly concurrent write
workloads - for instance, for a pgbench run with -T900, each
transaction generating WAL of size ~256 bytes, more than 3X
improvement in TPS is seen - 3.11X @ 512 clients 3.79 @ 768
clients, 3.47 @ 1024 clients, 2.27 @ 2048 clients and 2.77 @ 4096
clients.
---
 src/backend/access/transam/xlog.c |  4 +--
 src/backend/storage/lmgr/lwlock.c | 46 ++++++++++++++-----------------
 src/include/storage/lwlock.h      |  6 ++--
 3 files changed, 26 insertions(+), 30 deletions(-)

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 8b0710abe6..75814ac947 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -376,7 +376,7 @@ typedef struct XLogwrtResult
 typedef struct
 {
 	LWLock		lock;
-	XLogRecPtr	insertingAt;
+	pg_atomic_uint64	insertingAt;
 	XLogRecPtr	lastImportantAt;
 } WALInsertLock;
 
@@ -4611,7 +4611,7 @@ XLOGShmemInit(void)
 	for (i = 0; i < NUM_XLOGINSERT_LOCKS; i++)
 	{
 		LWLockInitialize(&WALInsertLocks[i].l.lock, LWTRANCHE_WAL_INSERT);
-		WALInsertLocks[i].l.insertingAt = InvalidXLogRecPtr;
+		pg_atomic_init_u64(&WALInsertLocks[i].l.insertingAt, InvalidXLogRecPtr);
 		WALInsertLocks[i].l.lastImportantAt = InvalidXLogRecPtr;
 	}
 
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 01d738f306..ffa865eb28 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1547,9 +1547,8 @@ LWLockAcquireOrWait(LWLock *lock, LWLockMode mode)
  * *result is set to true if the lock was free, and false otherwise.
  */
 static bool
-LWLockConflictsWithVar(LWLock *lock,
-					   uint64 *valptr, uint64 oldval, uint64 *newval,
-					   bool *result)
+LWLockConflictsWithVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+					   uint64 *newval, bool *result)
 {
 	bool		mustwait;
 	uint64		value;
@@ -1572,13 +1571,10 @@ LWLockConflictsWithVar(LWLock *lock,
 	*result = false;
 
 	/*
-	 * Read value using the lwlock's wait list lock, as we can't generally
-	 * rely on atomic 64 bit reads/stores.  TODO: On platforms with a way to
-	 * do atomic 64 bit reads/writes the spinlock should be optimized away.
+	 * Reading this value atomically is safe even on platforms where uint64
+	 * cannot be read without observing a torn value.
 	 */
-	LWLockWaitListLock(lock);
-	value = *valptr;
-	LWLockWaitListUnlock(lock);
+	value = pg_atomic_read_u64(valptr);
 
 	if (value != oldval)
 	{
@@ -1607,7 +1603,8 @@ LWLockConflictsWithVar(LWLock *lock,
  * in shared mode, returns 'true'.
  */
 bool
-LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
+LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
+				 uint64 *newval)
 {
 	PGPROC	   *proc = MyProc;
 	int			extraWaits = 0;
@@ -1735,29 +1732,32 @@ LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval)
  * LWLockUpdateVar - Update a variable and wake up waiters atomically
  *
  * Sets *valptr to 'val', and wakes up all processes waiting for us with
- * LWLockWaitForVar().  Setting the value and waking up the processes happen
- * atomically so that any process calling LWLockWaitForVar() on the same lock
- * is guaranteed to see the new value, and act accordingly.
+ * LWLockWaitForVar().  It first sets the value atomically and then wakes up
+ * waiting processes so that any process calling LWLockWaitForVar() on the same
+ * lock is guaranteed to see the new value, and act accordingly.
  *
  * The caller must be holding the lock in exclusive mode.
  */
 void
-LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
 	proclist_head wakeup;
 	proclist_mutable_iter iter;
 
 	PRINT_LWDEBUG("LWLockUpdateVar", lock, LW_EXCLUSIVE);
 
+	/*
+	 * Note that pg_atomic_exchange_u64 is a full barrier, so we're guaranteed
+	 * that the variable is updated before waking up waiters.
+	 */
+	pg_atomic_exchange_u64(valptr, val);
+
 	proclist_init(&wakeup);
 
 	LWLockWaitListLock(lock);
 
 	Assert(pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE);
 
-	/* Update the lock's value */
-	*valptr = val;
-
 	/*
 	 * See if there are any LW_WAIT_UNTIL_FREE waiters that need to be woken
 	 * up. They are always in the front of the queue.
@@ -1873,17 +1873,13 @@ LWLockRelease(LWLock *lock)
  * LWLockReleaseClearVar - release a previously acquired lock, reset variable
  */
 void
-LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val)
+LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
-	LWLockWaitListLock(lock);
-
 	/*
-	 * Set the variable's value before releasing the lock, that prevents race
-	 * a race condition wherein a new locker acquires the lock, but hasn't yet
-	 * set the variables value.
+	 * Note that pg_atomic_exchange_u64 is a full barrier, so we're guaranteed
+	 * that the variable is updated before releasing the lock.
 	 */
-	*valptr = val;
-	LWLockWaitListUnlock(lock);
+	pg_atomic_exchange_u64(valptr, val);
 
 	LWLockRelease(lock);
 }
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index 34169e5889..d77410bdea 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -129,14 +129,14 @@ extern bool LWLockAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockConditionalAcquire(LWLock *lock, LWLockMode mode);
 extern bool LWLockAcquireOrWait(LWLock *lock, LWLockMode mode);
 extern void LWLockRelease(LWLock *lock);
-extern void LWLockReleaseClearVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern void LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 extern void LWLockReleaseAll(void);
 extern bool LWLockHeldByMe(LWLock *lock);
 extern bool LWLockAnyHeldByMe(LWLock *lock, int nlocks, size_t stride);
 extern bool LWLockHeldByMeInMode(LWLock *lock, LWLockMode mode);
 
-extern bool LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval);
-extern void LWLockUpdateVar(LWLock *lock, uint64 *valptr, uint64 val);
+extern bool LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval, uint64 *newval);
+extern void LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val);
 
 extern Size LWLockShmemSize(void);
 extern void CreateLWLocks(void);
-- 
2.34.1

v10-0002-Improve-commets-in-and-around-LWLockWaitForVar-c.patchapplication/octet-stream; name=v10-0002-Improve-commets-in-and-around-LWLockWaitForVar-c.patchDownload
From d07a88ec8862c35617a6ea6632d7b17869af5e67 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Sat, 22 Jul 2023 07:35:01 +0000
Subject: [PATCH v10] Improve commets in and around LWLockWaitForVar call sites

This commit explains why it's safe to loop for LWLockWaitForVar
without a memory barrier or a spinlock in
WaitXLogInsertionsToFinish. In the loop we are only waiting for
insertions that started before WaitXLogInsertionsToFinish was
called. The lack of memory barriers in the loop means that we
might see locks as "unused" that have since become used, which is
fine, because they only can be for later insertions that we
wouldn't want to wait on anyway. And, not taking a lock to
acquire the current insertingAt value means that we might see
older insertingAt value. Which is also be fine, because if we read
a too old value, we will add ourselves to the queue, which
contains atomic operations.
---
 src/backend/access/transam/xlog.c | 11 +++++++++++
 src/backend/storage/lmgr/lwlock.c | 10 +++++++---
 2 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 75814ac947..2fec31688c 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1495,6 +1495,17 @@ WaitXLogInsertionsToFinish(XLogRecPtr upto)
 			 * calling LWLockUpdateVar.  But if it has to sleep, it will
 			 * advertise the insertion point with LWLockUpdateVar before
 			 * sleeping.
+			 *
+			 * In this loop we are only waiting for insertions that started
+			 * before WaitXLogInsertionsToFinish was called. The lack of memory
+			 * barriers in the loop means that we might see locks as "unused"
+			 * that have since become used, which is fine, because they only
+			 * can be for later insertions that we wouldn't want to wait on
+			 * anyway. And, not taking a lock to acquire the current
+			 * insertingAt value means that we might see older insertingAt
+			 * value. Which is also be fine, because if we read a too old
+			 * value, we will add ourselves to the queue, which contains atomic
+			 * operations.
 			 */
 			if (LWLockWaitForVar(&WALInsertLocks[i].l.lock,
 								 &WALInsertLocks[i].l.insertingAt,
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index ffa865eb28..ec7b55ffa8 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1556,9 +1556,9 @@ LWLockConflictsWithVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
 	/*
 	 * Test first to see if it the slot is free right now.
 	 *
-	 * XXX: the caller uses a spinlock before this, so we don't need a memory
-	 * barrier here as far as the current usage is concerned.  But that might
-	 * not be safe in general.
+	 * XXX: the caller (WaitXLogInsertionsToFinish) uses an implied barrier via
+	 * spinlock before this, so we don't need a memory barrier here as far as
+	 * the current usage is concerned. But that might not be safe in general.
 	 */
 	mustwait = (pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE) != 0;
 
@@ -1601,6 +1601,10 @@ LWLockConflictsWithVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
  *
  * Note: this function ignores shared lock holders; if the lock is held
  * in shared mode, returns 'true'.
+ *
+ * Be aware that LWLockConflictsWithVar() does not include a memory barrier,
+ * hence the caller of this function may want to rely on an explicit barrier or
+ * an implied barrier via spinlock or LWLock to avoid memory ordering issues.
  */
 bool
 LWLockWaitForVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 oldval,
-- 
2.34.1

v10-0003-Have-a-quick-exit-for-LWLockUpdateVar-when-there.patchapplication/octet-stream; name=v10-0003-Have-a-quick-exit-for-LWLockUpdateVar-when-there.patchDownload
From 3018eacc620e93859ed379e58197568ebe3a12c9 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Sat, 22 Jul 2023 07:35:27 +0000
Subject: [PATCH v10] Have a quick exit for LWLockUpdateVar when there are no
 waiters

---
 src/backend/storage/lmgr/lwlock.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index ec7b55ffa8..86155cc977 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1756,6 +1756,19 @@ LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 	 */
 	pg_atomic_exchange_u64(valptr, val);
 
+	/*
+	 * Quick exit when there are no waiters.
+	 *
+	 * The twice-in-a-row lock acquisition protocol used by LWLockWaitForVar is
+	 * helping us out have quick exit here when there are no waiters without
+	 * acquiring LWLock wait list lock. LWLockWaitForVar ensures that the
+	 * waiters are added to wait queue even if LWLockUpdateVar thinks that
+	 * there aren't any waiters actually. This avoids unnecessary LWLock wait
+	 * list lock acquisition and release when there are no waiters at all.
+	 */
+	if ((pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) == 0)
+		return;
+
 	proclist_init(&wakeup);
 
 	LWLockWaitListLock(lock);
-- 
2.34.1

#45Michael Paquier
michael@paquier.xyz
In reply to: Bharath Rupireddy (#44)
Re: WAL Insertion Lock Improvements

On Sat, Jul 22, 2023 at 01:08:49PM +0530, Bharath Rupireddy wrote:

Yes, it looks safe to me too.

0001 has been now applied. I have done more tests while looking at
this patch since yesterday and was surprised to see higher TPS numbers
on HEAD with the same tests as previously, and the patch was still
shining with more than 256 clients.

FWIW, 0001 essentially implements what
an existing TODO comment introduced by commit 008608b9d5106 says:

We really need to do something in terms of documentation with
something like 0002, so I'll try to look at that next. Regarding
0003, I don't know. I think that we'd better look more into cases
where it shows actual benefits for specific workloads (like workloads
with a fixed rate of read and/or write operations?).
--
Michael

#46Andres Freund
andres@anarazel.de
In reply to: Michael Paquier (#45)
Re: WAL Insertion Lock Improvements

Hi,

On 2023-07-25 16:43:16 +0900, Michael Paquier wrote:

On Sat, Jul 22, 2023 at 01:08:49PM +0530, Bharath Rupireddy wrote:

Yes, it looks safe to me too.

0001 has been now applied. I have done more tests while looking at
this patch since yesterday and was surprised to see higher TPS numbers
on HEAD with the same tests as previously, and the patch was still
shining with more than 256 clients.

FWIW, 0001 essentially implements what
an existing TODO comment introduced by commit 008608b9d5106 says:

We really need to do something in terms of documentation with
something like 0002, so I'll try to look at that next. Regarding
0003, I don't know. I think that we'd better look more into cases
where it shows actual benefits for specific workloads (like workloads
with a fixed rate of read and/or write operations?).

FWIW, I'm working on a patch that replaces WAL insert locks as a whole,
because they don't scale all that well. If there's no very clear improvements,
I'm not sure it's worth putting too much effort into polishing them all that
much.

Greetings,

Andres Freund

#47Andres Freund
andres@anarazel.de
In reply to: Michael Paquier (#45)
Re: WAL Insertion Lock Improvements

Hi,

On 2023-07-25 16:43:16 +0900, Michael Paquier wrote:

On Sat, Jul 22, 2023 at 01:08:49PM +0530, Bharath Rupireddy wrote:

Yes, it looks safe to me too.

0001 has been now applied. I have done more tests while looking at
this patch since yesterday and was surprised to see higher TPS numbers
on HEAD with the same tests as previously, and the patch was still
shining with more than 256 clients.

Just a small heads up:

I just rebased my aio tree over the commit and promptly, on the first run, saw
a hang. I did some debugging on that. Unfortunately repeated runs haven't
repeated that hang, despite quite a bit of trying.

The symptom I was seeing is that all running backends were stuck in
LWLockWaitForVar(), even though the value they're waiting for had
changed. Which obviously "shouldn't be possible".

It's of course possible that this is AIO specific, but I didn't see anything
in stacks to suggest that.

I do wonder if this possibly exposed an undocumented prior dependency on the
value update always happening under the list lock.

Greetings,

Andres Freund

#48Michael Paquier
michael@paquier.xyz
In reply to: Andres Freund (#47)
Re: WAL Insertion Lock Improvements

On Tue, Jul 25, 2023 at 12:57:37PM -0700, Andres Freund wrote:

I just rebased my aio tree over the commit and promptly, on the first run, saw
a hang. I did some debugging on that. Unfortunately repeated runs haven't
repeated that hang, despite quite a bit of trying.

The symptom I was seeing is that all running backends were stuck in
LWLockWaitForVar(), even though the value they're waiting for had
changed. Which obviously "shouldn't be possible".

Hmm. I've also spent a few days looking at this past report that made
the LWLock part what it is today, but I don't quite see immediately
how it would be possible to reach a state where all the backends are
waiting for an update that's not happening:
/messages/by-id/CAMkU=1zLztROwH3B42OXSB04r9ZMeSk3658qEn4_8+b+K3E7nQ@mail.gmail.com

All the assumptions of this code and its dependencies with
xloginsert.c are hard to come by.

It's of course possible that this is AIO specific, but I didn't see anything
in stacks to suggest that.

Or AIO handles the WAL syncs so quickly that it has more chances in
showing a race condition here?

I do wonder if this possibly exposed an undocumented prior dependency on the
value update always happening under the list lock.

I would not be surprised by that.
--
Michael

#49Michael Paquier
michael@paquier.xyz
In reply to: Andres Freund (#46)
Re: WAL Insertion Lock Improvements

On Tue, Jul 25, 2023 at 09:49:01AM -0700, Andres Freund wrote:

FWIW, I'm working on a patch that replaces WAL insert locks as a whole,
because they don't scale all that well.

What were you looking at here? Just wondering.
--
Michael

#50Andres Freund
andres@anarazel.de
In reply to: Michael Paquier (#49)
Re: WAL Insertion Lock Improvements

Hi,

On 2023-07-26 07:40:31 +0900, Michael Paquier wrote:

On Tue, Jul 25, 2023 at 09:49:01AM -0700, Andres Freund wrote:

FWIW, I'm working on a patch that replaces WAL insert locks as a whole,
because they don't scale all that well.

What were you looking at here? Just wondering.

Here's what I had written offlist a few days ago:

The basic idea is to have a ringbuffer of in-progress insertions. The
acquisition of a position in the ringbuffer is done at the same time as
advancing the reserved LSN, using a 64bit xadd. The trick that makes that
possible is to use the high bits of the atomic for the position in the
ringbuffer. The point of using the high bits is that they wrap around, without
affecting the rest of the value.

Of course, when using xadd, we can't keep the "prev" pointer in the
atomic. That's where the ringbuffer comes into play. Whenever one inserter has
determined the byte pos of its insertion, it updates the "prev byte pos" in
the *next* ringbuffer entry.

Of course that means that insertion N+1 needs to wait for N to set the prev
position - but that happens very quickly. In my very hacky prototype the
relevant path (which for now just spins) is reached very rarely, even when
massively oversubscribed. While I've not implemented that, N+1 could actually
do the first "iteration" in CopyXLogRecordToWAL() before it needs the prev
position, the COMP_CRC32C() could happen "inside" the buffer.

There's a fair bit of trickyness in turning that into something working, of
course :). Ensuring that the ring buffer of insertions doesn't wrap around is
non-trivial. Nor is trivial to ensure that the "reduced" space LSN in the
atomic can't overflow.

I do wish MAX_BACKENDS were smaller...

Until last night I thought all my schemes would continue to need something
like the existing WAL insertion locks, to implement
WALInsertLockAcquireExclusive().

But I think I came up with an idea to do away with that (not even prototyped
yet): Use one bit in the atomic that indicates that no new insertions are
allowed. Whenever the xadd finds that old value actually was locked, it
"aborts" the insertion, and instead waits for a condition variable (or
something similar). Of course that's after modifying the atomic - to deal with
that the "lock holder" reverts all modifications that have been made to the
atomic when releasing the "lock", they weren't actually successful and all
those backends will retry.

Except that this doesn't quite suffice - XLogInsertRecord() needs to be able
to "roll back", when it finds that we now need to log FPIs. I can't quite see
how to make that work with what I describe above. The only idea I have so far
is to just waste the space with a NOOP record - it should be pretty rare. At
least if we updated RedoRecPtr eagerly (or just stopped this stupid business
of having an outdated backend-local copy).

My prototype shows this idea to be promising. It's a tad slower at low
concurrency, but much better at high concurrency. I think most if not all of
the low-end overhead isn't inherent, but comes from having both old and new
infrastructure in place (as well as a lot of debugging cruft).

Greetings,

Andres Freund

#51Michael Paquier
michael@paquier.xyz
In reply to: Michael Paquier (#45)
Re: WAL Insertion Lock Improvements

On Tue, Jul 25, 2023 at 04:43:16PM +0900, Michael Paquier wrote:

We really need to do something in terms of documentation with
something like 0002, so I'll try to look at that next.

I have applied a slightly-tweaked version of 0002 as of 66d86d4 to
improve a bit the documentation of the area, and switched the CF entry
as committed.

(I got interested in what Andres has seen on his latest AIO branch, so
I have a few extra benchmarks running in the background on HEAD, but
nothing able to freeze all the backends yet waiting for a variable
update. These are still running now.)
--
Michael

#52Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Andres Freund (#47)
Re: WAL Insertion Lock Improvements

On Wed, Jul 26, 2023 at 1:27 AM Andres Freund <andres@anarazel.de> wrote:

0001 has been now applied. I have done more tests while looking at
this patch since yesterday and was surprised to see higher TPS numbers
on HEAD with the same tests as previously, and the patch was still
shining with more than 256 clients.

Just a small heads up:

I just rebased my aio tree over the commit and promptly, on the first run, saw
a hang. I did some debugging on that. Unfortunately repeated runs haven't
repeated that hang, despite quite a bit of trying.

Hm. Please share workload details, test scripts, system info and any
special settings for running in my setup.

The symptom I was seeing is that all running backends were stuck in
LWLockWaitForVar(), even though the value they're waiting for had
changed. Which obviously "shouldn't be possible".

Were the backends stuck there indefinitely? IOW, did they get into a deadlock?

It's of course possible that this is AIO specific, but I didn't see anything
in stacks to suggest that.

I do wonder if this possibly exposed an undocumented prior dependency on the
value update always happening under the list lock.

I'm going through the other thread mentioned by Michael Paquier. I'm
wondering if the deadlock issue illustrated here
/messages/by-id/55BB50D3.9000702@iki.fi is
showing up again, because 71e4cc6b8e reduced the contention on
waitlist lock and made things *a bit* faster.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#53Nathan Bossart
nathandbossart@gmail.com
In reply to: Michael Paquier (#45)
1 attachment(s)
Re: WAL Insertion Lock Improvements

On Tue, Jul 25, 2023 at 04:43:16PM +0900, Michael Paquier wrote:

0001 has been now applied. I have done more tests while looking at
this patch since yesterday and was surprised to see higher TPS numbers
on HEAD with the same tests as previously, and the patch was still
shining with more than 256 clients.

I found this code when searching for callers that use atomic exchanges as
atomic writes with barriers (for a separate thread [0]/messages/by-id/20231110205128.GB1315705@nathanxps13). Can't we use
pg_atomic_write_u64() here since the locking functions that follow should
serve as barriers?

I've attached a patch to demonstrate what I'm thinking. This might be more
performant, although maybe less so after commit 64b1fb5. Am I missing
something obvious here? If not, I might rerun the benchmarks to see
whether it makes any difference.

[0]: /messages/by-id/20231110205128.GB1315705@nathanxps13

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

Attachments:

barrier.patchtext/x-diff; charset=us-asciiDownload
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 315a78cda9..b43972bd2e 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1752,10 +1752,10 @@ LWLockUpdateVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 	PRINT_LWDEBUG("LWLockUpdateVar", lock, LW_EXCLUSIVE);
 
 	/*
-	 * Note that pg_atomic_exchange_u64 is a full barrier, so we're guaranteed
-	 * that the variable is updated before waking up waiters.
+	 * We rely on the barrier provided by LWLockWaitListLock() to ensure the
+	 * variable is updated before waking up waiters.
 	 */
-	pg_atomic_exchange_u64(valptr, val);
+	pg_atomic_write_u64(valptr, val);
 
 	proclist_init(&wakeup);
 
@@ -1881,10 +1881,10 @@ void
 LWLockReleaseClearVar(LWLock *lock, pg_atomic_uint64 *valptr, uint64 val)
 {
 	/*
-	 * Note that pg_atomic_exchange_u64 is a full barrier, so we're guaranteed
-	 * that the variable is updated before releasing the lock.
+	 * We rely on the barrier provided by LWLockRelease() to ensure the
+	 * variable is updated before releasing the lock.
 	 */
-	pg_atomic_exchange_u64(valptr, val);
+	pg_atomic_write_u64(valptr, val);
 
 	LWLockRelease(lock);
 }
#54Michael Paquier
michael@paquier.xyz
In reply to: Nathan Bossart (#53)
Re: WAL Insertion Lock Improvements

On Mon, Dec 18, 2023 at 10:00:29PM -0600, Nathan Bossart wrote:

I found this code when searching for callers that use atomic exchanges as
atomic writes with barriers (for a separate thread [0]). Can't we use
pg_atomic_write_u64() here since the locking functions that follow should
serve as barriers?

The full barrier guaranteed by pg_atomic_exchange_u64() in
LWLockUpdateVar() was also necessary for the shortcut based on
read_u32() to see if there are no waiters, but it has been discarded
in the later versions of the patch because it did not influence
performance under heavy WAL inserts.

So you mean to rely on the full barriers taken by the fetches in
LWLockRelease() and LWLockWaitListLock()? Hmm, I got the impression
that pg_atomic_exchange_u64() with its full barrier was necessary in
these two paths so as all the loads and stores are completed *before*
updating these variables. So I am not sure to get why it would be
safe to switch to a write with no barrier.

I've attached a patch to demonstrate what I'm thinking. This might be more
performant, although maybe less so after commit 64b1fb5. Am I missing
something obvious here? If not, I might rerun the benchmarks to see
whether it makes any difference.

I was wondering as well if the numbers we got upthread would go up
after what you have committed to improve the exchanges. :)
Any change in this area should be strictly benchmarked.
--
Michael