Speedup twophase transactions

Started by Stas Kelvichabout 10 years ago167 messages
#1Stas Kelvich
s.kelvich@postgrespro.ru
1 attachment(s)

Hello.

While working with cluster stuff (DTM, tsDTM) we noted that postgres 2pc transactions is approximately two times slower than an ordinary commit on workload with fast transactions — few single-row updates and COMMIT or PREPARE/COMMIT. Perf top showed that a lot of time is spent in kernel on fopen/fclose, so it worth a try to reduce file operations with 2pc tx.

Now 2PC in postgres does following:
* on prepare 2pc data (subxacts, commitrels, abortrels, invalmsgs) saved to xlog and to file, but file not is not fsynced
* on commit backend reads data from file
* if checkpoint occurs before commit, then files are fsynced during checkpoint
* if case of crash replay will move data from xlog to files

In this patch I’ve changed this procedures to following:
* on prepare backend writes data only to xlog and store pointer to the start of the xlog record
* if commit occurs before checkpoint then backend reads data from xlog by this pointer
* on checkpoint 2pc data copied to files and fsynced
* if commit happens after checkpoint then backend reads files
* in case of crash replay will move data from xlog to files (as it was before patch)

Most of that ideas was already mentioned in 2009 thread by Michael Paquier /messages/by-id/c64c5f8b0908062031k3ff48428j824a9a46f28180ac@mail.gmail.com where he suggested to store 2pc data in shared memory.
At that time patch was declined because no significant speedup were observed. Now I see performance improvements by my patch at about 60%. Probably old benchmark overall tps was lower and it was harder to hit filesystem fopen/fclose limits.

Now results of benchmark are following (dual 6-core xeon server):

Current master without 2PC: ~42 ktps
Current master with 2PC: ~22 ktps
Current master with 2PC: ~36 ktps

Benchmark done with following script:

\set naccounts 100000 * :scale
\setrandom from_aid 1 :naccounts
\setrandom to_aid 1 :naccounts
\setrandom delta 1 100
\set scale :scale+1
BEGIN;
UPDATE pgbench_accounts SET abalance = abalance - :delta WHERE aid = :from_aid;
UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :to_aid;
PREPARE TRANSACTION ':client_id.:scale';
COMMIT PREPARED ':client_id.:scale';

Attachments:

2pc_xlog.diffapplication/octet-stream; name=2pc_xlog.diffDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 8c47e0f..b1c36e3 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -51,6 +51,7 @@
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "access/xlogutils.h"
+#include "access/xlogreader.h"
 #include "catalog/pg_type.h"
 #include "catalog/storage.h"
 #include "funcapi.h"
@@ -60,6 +61,7 @@
 #include "replication/origin.h"
 #include "replication/syncrep.h"
 #include "replication/walsender.h"
+#include "replication/logicalfuncs.h"
 #include "storage/fd.h"
 #include "storage/ipc.h"
 #include "storage/predicate.h"
@@ -117,7 +119,11 @@ typedef struct GlobalTransactionData
 	int			pgprocno;		/* ID of associated dummy PGPROC */
 	BackendId	dummyBackendId; /* similar to backend id for backends */
 	TimestampTz prepared_at;	/* time of preparation */
-	XLogRecPtr	prepare_lsn;	/* XLOG offset of prepare record */
+	XLogRecPtr	prepare_lsn;	/* XLOG offset of prepare record end */
+	XLogRecPtr	prepare_xlogptr;	/* XLOG offset of prepare record start
+									 * or NULL if twophase data moved to file
+									 * after checkpoint.
+									 */
 	Oid			owner;			/* ID of user that executed the xact */
 	BackendId	locking_backend;	/* backend currently working on the xact */
 	bool		valid;			/* TRUE if PGPROC entry is in proc array */
@@ -166,6 +172,7 @@ static void ProcessRecords(char *bufptr, TransactionId xid,
 			   const TwoPhaseCallback callbacks[]);
 static void RemoveGXact(GlobalTransaction gxact);
 
+static void XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len);
 
 /*
  * Initialization of shared memory
@@ -400,6 +407,7 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 	gxact->prepared_at = prepared_at;
 	/* initialize LSN to 0 (start of WAL) */
 	gxact->prepare_lsn = 0;
+	gxact->prepare_xlogptr = 0;
 	gxact->owner = owner;
 	gxact->locking_backend = MyBackendId;
 	gxact->valid = false;
@@ -579,41 +587,6 @@ RemoveGXact(GlobalTransaction gxact)
 }
 
 /*
- * TransactionIdIsPrepared
- *		True iff transaction associated with the identifier is prepared
- *		for two-phase commit
- *
- * Note: only gxacts marked "valid" are considered; but notice we do not
- * check the locking status.
- *
- * This is not currently exported, because it is only needed internally.
- */
-static bool
-TransactionIdIsPrepared(TransactionId xid)
-{
-	bool		result = false;
-	int			i;
-
-	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
-
-	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
-	{
-		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
-		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
-
-		if (gxact->valid && pgxact->xid == xid)
-		{
-			result = true;
-			break;
-		}
-	}
-
-	LWLockRelease(TwoPhaseStateLock);
-
-	return result;
-}
-
-/*
  * Returns an array of all prepared transactions for the user-level
  * function pg_prepared_xact.
  *
@@ -1020,14 +993,8 @@ StartPrepare(GlobalTransaction gxact)
 void
 EndPrepare(GlobalTransaction gxact)
 {
-	PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
-	TransactionId xid = pgxact->xid;
 	TwoPhaseFileHeader *hdr;
-	char		path[MAXPGPATH];
 	StateFileChunk *record;
-	pg_crc32c	statefile_crc;
-	pg_crc32c	bogus_crc;
-	int			fd;
 
 	/* Add the end sentinel to the list of 2PC records */
 	RegisterTwoPhaseRecord(TWOPHASE_RM_END_ID, 0,
@@ -1048,70 +1015,7 @@ EndPrepare(GlobalTransaction gxact)
 				 errmsg("two-phase state file maximum length exceeded")));
 
 	/*
-	 * Create the 2PC state file.
-	 */
-	TwoPhaseFilePath(path, xid);
-
-	fd = OpenTransientFile(path,
-						   O_CREAT | O_EXCL | O_WRONLY | PG_BINARY,
-						   S_IRUSR | S_IWUSR);
-	if (fd < 0)
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not create two-phase state file \"%s\": %m",
-						path)));
-
-	/* Write data to file, and calculate CRC as we pass over it */
-	INIT_CRC32C(statefile_crc);
-
-	for (record = records.head; record != NULL; record = record->next)
-	{
-		COMP_CRC32C(statefile_crc, record->data, record->len);
-		if ((write(fd, record->data, record->len)) != record->len)
-		{
-			CloseTransientFile(fd);
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not write two-phase state file: %m")));
-		}
-	}
-
-	FIN_CRC32C(statefile_crc);
-
-	/*
-	 * Write a deliberately bogus CRC to the state file; this is just paranoia
-	 * to catch the case where four more bytes will run us out of disk space.
-	 */
-	bogus_crc = ~statefile_crc;
-
-	if ((write(fd, &bogus_crc, sizeof(pg_crc32c))) != sizeof(pg_crc32c))
-	{
-		CloseTransientFile(fd);
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not write two-phase state file: %m")));
-	}
-
-	/* Back up to prepare for rewriting the CRC */
-	if (lseek(fd, -((off_t) sizeof(pg_crc32c)), SEEK_CUR) < 0)
-	{
-		CloseTransientFile(fd);
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not seek in two-phase state file: %m")));
-	}
-
-	/*
-	 * The state file isn't valid yet, because we haven't written the correct
-	 * CRC yet.  Before we do that, insert entry in WAL and flush it to disk.
-	 *
-	 * Between the time we have written the WAL entry and the time we write
-	 * out the correct state file CRC, we have an inconsistency: the xact is
-	 * prepared according to WAL but not according to our on-disk state. We
-	 * use a critical section to force a PANIC if we are unable to complete
-	 * the write --- then, WAL replay should repair the inconsistency.  The
-	 * odds of a PANIC actually occurring should be very tiny given that we
-	 * were able to write the bogus CRC above.
+	 * Now writing 2PC state data to WAL.
 	 *
 	 * We have to set delayChkpt here, too; otherwise a checkpoint starting
 	 * immediately after the WAL record is inserted could complete without
@@ -1136,19 +1040,8 @@ EndPrepare(GlobalTransaction gxact)
 
 	/* If we crash now, we have prepared: WAL replay will fix things */
 
-	/* write correct CRC and close file */
-	if ((write(fd, &statefile_crc, sizeof(pg_crc32c))) != sizeof(pg_crc32c))
-	{
-		CloseTransientFile(fd);
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not write two-phase state file: %m")));
-	}
-
-	if (CloseTransientFile(fd) != 0)
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not close two-phase state file: %m")));
+	/* Store record's start location to read that later on Commit */
+	gxact->prepare_xlogptr = ProcLastRecPtr;
 
 	/*
 	 * Mark the prepared transaction as valid.  As soon as xact.c marks
@@ -1315,6 +1208,36 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
 	return buf;
 }
 
+
+/*
+ * Reads 2PC data from xlog. During checkpoint this data will be moved to
+ * twophase files and ReadTwoPhaseFile should be used instead.
+ */
+static void
+XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
+{
+	XLogRecord *record;
+	XLogReaderState *xlogreader;
+	char	   *errormsg;
+
+	xlogreader = XLogReaderAllocate(&logical_read_local_xlog_page, NULL);
+	if (xlogreader == NULL)
+		elog(ERROR, "failed to open xlogreader for reading 2PC data");
+
+	record = XLogReadRecord(xlogreader, lsn, &errormsg);
+	if (record == NULL)
+		elog(ERROR, "failed to read 2PC record from xlog");
+
+	if (len != NULL)
+		*len = XLogRecGetDataLen(xlogreader);
+
+	*buf = palloc(sizeof(char)*XLogRecGetDataLen(xlogreader));
+	memcpy(*buf, XLogRecGetData(xlogreader), sizeof(char)*XLogRecGetDataLen(xlogreader));
+
+	XLogReaderFree(xlogreader);
+}
+
+
 /*
  * Confirms an xid is prepared, during recovery
  */
@@ -1364,6 +1287,7 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	int			ndelrels;
 	SharedInvalidationMessage *invalmsgs;
 	int			i;
+	bool		file_used = false;
 
 	/*
 	 * Validate the GID, and lock the GXACT to ensure that two backends do not
@@ -1375,14 +1299,20 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	xid = pgxact->xid;
 
 	/*
-	 * Read and validate the state file
+	 * Read and validate 2PC state data.
+	 * State data can be stored in xlog or in files after xlog checkpoint.
+	 * While checkpointing we set gxact->prepare_lsn to NULL to signalize
+	 * that 2PC data is moved to files.
 	 */
-	buf = ReadTwoPhaseFile(xid, true);
-	if (buf == NULL)
-		ereport(ERROR,
-				(errcode(ERRCODE_DATA_CORRUPTED),
-				 errmsg("two-phase state file for transaction %u is corrupt",
-						xid)));
+	if (gxact->prepare_lsn)
+	{
+		XlogReadTwoPhaseData(gxact->prepare_xlogptr, &buf, NULL);
+	}
+	else
+	{
+		buf = ReadTwoPhaseFile(xid, true);
+		file_used = true;
+	}
 
 	/*
 	 * Disassemble the header area
@@ -1484,7 +1414,8 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	/*
 	 * And now we can clean up our mess.
 	 */
-	RemoveTwoPhaseFile(xid, true);
+	if (file_used)
+		RemoveTwoPhaseFile(xid, true);
 
 	RemoveGXact(gxact);
 	MyLockedGxact = NULL;
@@ -1539,7 +1470,8 @@ RemoveTwoPhaseFile(TransactionId xid, bool giveWarning)
 }
 
 /*
- * Recreates a state file. This is used in WAL replay.
+ * Recreates a state file. This is used in WAL replay and during
+ * checkpoint creation.
  *
  * Note: content and len don't include CRC.
  */
@@ -1620,85 +1552,37 @@ RecreateTwoPhaseFile(TransactionId xid, void *content, int len)
 void
 CheckPointTwoPhase(XLogRecPtr redo_horizon)
 {
-	TransactionId *xids;
-	int			nxids;
-	char		path[MAXPGPATH];
 	int			i;
+	int 		len;
+	char	   *buf;
 
-	/*
-	 * We don't want to hold the TwoPhaseStateLock while doing I/O, so we grab
-	 * it just long enough to make a list of the XIDs that require fsyncing,
-	 * and then do the I/O afterwards.
-	 *
-	 * This approach creates a race condition: someone else could delete a
-	 * GXACT between the time we release TwoPhaseStateLock and the time we try
-	 * to open its state file.  We handle this by special-casing ENOENT
-	 * failures: if we see that, we verify that the GXACT is no longer valid,
-	 * and if so ignore the failure.
-	 */
 	if (max_prepared_xacts <= 0)
 		return;					/* nothing to do */
 
 	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_START();
 
-	xids = (TransactionId *) palloc(max_prepared_xacts * sizeof(TransactionId));
-	nxids = 0;
-
+	/*
+	 * Here we doing whole I/O while holding TwoPhaseStateLock.
+	 * It's also possible to move I/O out of the lock, but on
+	 * every error we should check whether somebody commited our
+	 * transaction in different backend. Let's leave this optimisation
+	 * for future, if somebody will spot that this place cause
+	 * bottleneck.
+	 */
 	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
-
 	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
 	{
 		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
 		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 
-		if (gxact->valid &&
-			gxact->prepare_lsn <= redo_horizon)
-			xids[nxids++] = pgxact->xid;
-	}
-
-	LWLockRelease(TwoPhaseStateLock);
-
-	for (i = 0; i < nxids; i++)
-	{
-		TransactionId xid = xids[i];
-		int			fd;
-
-		TwoPhaseFilePath(path, xid);
-
-		fd = OpenTransientFile(path, O_RDWR | PG_BINARY, 0);
-		if (fd < 0)
-		{
-			if (errno == ENOENT)
-			{
-				/* OK if gxact is no longer valid */
-				if (!TransactionIdIsPrepared(xid))
-					continue;
-				/* Restore errno in case it was changed */
-				errno = ENOENT;
-			}
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not open two-phase state file \"%s\": %m",
-							path)));
-		}
-
-		if (pg_fsync(fd) != 0)
-		{
-			CloseTransientFile(fd);
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not fsync two-phase state file \"%s\": %m",
-							path)));
+		if (gxact->valid && gxact->prepare_lsn && gxact->prepare_lsn <= redo_horizon){
+			XlogReadTwoPhaseData(gxact->prepare_xlogptr, &buf, &len);
+			RecreateTwoPhaseFile(pgxact->xid, buf, len);
+			gxact->prepare_lsn = (XLogRecPtr) NULL;
+			pfree(buf);
 		}
-
-		if (CloseTransientFile(fd) != 0)
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not close two-phase state file \"%s\": %m",
-							path)));
 	}
-
-	pfree(xids);
+	LWLockRelease(TwoPhaseStateLock);
 
 	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_DONE();
 }
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 86debf4..3683785 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -321,8 +321,7 @@ static TimeLineID curFileTLI;
  * stored here.  The parallel leader advances its own copy, when necessary,
  * in WaitForParallelWorkersToFinish.
  */
-static XLogRecPtr ProcLastRecPtr = InvalidXLogRecPtr;
-
+XLogRecPtr	ProcLastRecPtr = InvalidXLogRecPtr;
 XLogRecPtr	XactLastRecEnd = InvalidXLogRecPtr;
 XLogRecPtr	XactLastCommitEnd = InvalidXLogRecPtr;
 
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 790ca66..a6d04cc 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -86,6 +86,7 @@ typedef enum
 	RECOVERY_TARGET_IMMEDIATE
 } RecoveryTargetType;
 
+extern XLogRecPtr ProcLastRecPtr;
 extern XLogRecPtr XactLastRecEnd;
 extern PGDLLIMPORT XLogRecPtr XactLastCommitEnd;
 
#2Kevin Grittner
kgrittn@gmail.com
In reply to: Stas Kelvich (#1)
Re: Speedup twophase transactions

On Wed, Dec 9, 2015 at 12:44 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Now 2PC in postgres does following:
* on prepare 2pc data (subxacts, commitrels, abortrels, invalmsgs) saved to xlog and to file, but file not is not fsynced
* on commit backend reads data from file
* if checkpoint occurs before commit, then files are fsynced during checkpoint
* if case of crash replay will move data from xlog to files

In this patch I’ve changed this procedures to following:
* on prepare backend writes data only to xlog and store pointer to the start of the xlog record
* if commit occurs before checkpoint then backend reads data from xlog by this pointer
* on checkpoint 2pc data copied to files and fsynced
* if commit happens after checkpoint then backend reads files
* in case of crash replay will move data from xlog to files (as it was before patch)

That sounds like a very good plan to me.

Now results of benchmark are following (dual 6-core xeon server):

Current master without 2PC: ~42 ktps
Current master with 2PC: ~22 ktps
Current master with 2PC: ~36 ktps

I assume that last one should have been *Patched master with 2PC"?

Please add this to the January CommitFest.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Kevin Grittner (#2)
Re: Speedup twophase transactions

Thanks, Kevin.

I assume that last one should have been *Patched master with 2PC”?

Yes, this list should look like this:

Current master without 2PC: ~42 ktps
Current master with 2PC: ~22 ktps
Patched master with 2PC: ~36 ktps

And created CommitFest entry for this patch.

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On 10 Dec 2015, at 00:37, Kevin Grittner <kgrittn@gmail.com> wrote:

On Wed, Dec 9, 2015 at 12:44 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Now 2PC in postgres does following:
* on prepare 2pc data (subxacts, commitrels, abortrels, invalmsgs) saved to xlog and to file, but file not is not fsynced
* on commit backend reads data from file
* if checkpoint occurs before commit, then files are fsynced during checkpoint
* if case of crash replay will move data from xlog to files

In this patch I’ve changed this procedures to following:
* on prepare backend writes data only to xlog and store pointer to the start of the xlog record
* if commit occurs before checkpoint then backend reads data from xlog by this pointer
* on checkpoint 2pc data copied to files and fsynced
* if commit happens after checkpoint then backend reads files
* in case of crash replay will move data from xlog to files (as it was before patch)

That sounds like a very good plan to me.

Now results of benchmark are following (dual 6-core xeon server):

Current master without 2PC: ~42 ktps
Current master with 2PC: ~22 ktps
Current master with 2PC: ~36 ktps

I assume that last one should have been *Patched master with 2PC"?

Please add this to the January CommitFest.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Michael Paquier
michael.paquier@gmail.com
In reply to: Stas Kelvich (#1)
Re: Speedup twophase transactions

On Thu, Dec 10, 2015 at 3:44 AM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Most of that ideas was already mentioned in 2009 thread by Michael Paquier /messages/by-id/c64c5f8b0908062031k3ff48428j824a9a46f28180ac@mail.gmail.com where he suggested to store 2pc data in shared memory.
At that time patch was declined because no significant speedup were observed. Now I see performance improvements by my patch at about 60%. Probably old benchmark overall tps was lower and it was harder to hit filesystem fopen/fclose limits.

Glad to see this patch is given a second life 6 years later.

Now results of benchmark are following (dual 6-core xeon server):

Current master without 2PC: ~42 ktps
Current master with 2PC: ~22 ktps
Current master with 2PC: ~36 ktps

That's nice.

+    XLogRecPtr    prepare_xlogptr;    /* XLOG offset of prepare record start
+                                     * or NULL if twophase data moved to file
+                                     * after checkpoint.
+                                     */
This has better be InvalidXLogRecPtr if unused.
+    if (gxact->prepare_lsn)
+    {
+        XlogReadTwoPhaseData(gxact->prepare_xlogptr, &buf, NULL);
+    }
Perhaps you mean prepare_xlogptr here?
-- 
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#5Jeff Janes
jeff.janes@gmail.com
In reply to: Stas Kelvich (#1)
Re: Speedup twophase transactions

On Wed, Dec 9, 2015 at 10:44 AM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Hello.

While working with cluster stuff (DTM, tsDTM) we noted that postgres 2pc transactions is approximately two times slower than an ordinary commit on workload with fast transactions — few single-row updates and COMMIT or PREPARE/COMMIT. Perf top showed that a lot of time is spent in kernel on fopen/fclose, so it worth a try to reduce file operations with 2pc tx.

I've tested this through my testing harness which forces the database
to go through endless runs of crash recovery and checks for
consistency, and so far it has survived perfectly.

...

Now results of benchmark are following (dual 6-core xeon server):

Current master without 2PC: ~42 ktps
Current master with 2PC: ~22 ktps
Current master with 2PC: ~36 ktps

Can you give the full command line? -j, -c, etc.

Benchmark done with following script:

\set naccounts 100000 * :scale
\setrandom from_aid 1 :naccounts
\setrandom to_aid 1 :naccounts
\setrandom delta 1 100
\set scale :scale+1

Why are you incrementing :scale ?

I very rapidly reach a point where most of the updates are against
tuples that don't exist, and then get integer overflow problems.

BEGIN;
UPDATE pgbench_accounts SET abalance = abalance - :delta WHERE aid = :from_aid;
UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :to_aid;
PREPARE TRANSACTION ':client_id.:scale';
COMMIT PREPARED ':client_id.:scale';

Cheers,

Jeff

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#4)
1 attachment(s)
Re: Speedup twophase transactions

Michael, Jeff thanks for reviewing and testing.

On 10 Dec 2015, at 02:16, Michael Paquier <michael.paquier@gmail.com> wrote:

This has better be InvalidXLogRecPtr if unused.

Yes, that’s better. Changed.

On 10 Dec 2015, at 02:16, Michael Paquier <michael.paquier@gmail.com> wrote:

+    if (gxact->prepare_lsn)
+    {
+        XlogReadTwoPhaseData(gxact->prepare_xlogptr, &buf, NULL);
+    }
Perhaps you mean prepare_xlogptr here?

Yes, my bad. But funnily I have this error even number of times: code in CheckPointTwoPhase also uses prepare_lsn instead of xlogptr, so overall this was working well, that’s why it survived my own tests and probably Jeff’s tests.
I think that’s a bad variable naming, for example because lsn in pg_xlogdump points to start of the record, but here start used as xloptr and end as lsn.
So changed both variables to prepare_start_lsn and prepare_end_lsn.

On 10 Dec 2015, at 09:48, Jeff Janes <jeff.janes@gmail.com> wrote:
I've tested this through my testing harness which forces the database
to go through endless runs of crash recovery and checks for
consistency, and so far it has survived perfectly.

Cool! I think that patch is most vulnerable to following type of workload: prepare transaction, do a lot of stuff with database to force checkpoints (or even recovery cycles), and commit it.

On 10 Dec 2015, at 09:48, Jeff Janes <jeff.janes@gmail.com> wrote:

Can you give the full command line? -j, -c, etc.

pgbench -h testhost -i && pgbench -h testhost -f 2pc.pgb -T 300 -P 1 -c 64 -j 16 -r

where 2pc.pgb as in previous message.

Also all this applies to hosts with uniform memory. I tried to run patched postgres on NUMA with 60 physical cores and patch didn’t change anything. Perf top shows that main bottleneck is access to gxact, but on ordinary host with 1/2 cpu’s that access even not in top ten heaviest routines.

On 10 Dec 2015, at 09:48, Jeff Janes <jeff.janes@gmail.com> wrote:

Why are you incrementing :scale ?

That’s a funny part, overall 2pc speed depends on how you will name your prepared transaction. Concretely I tried to use random numbers for gid’s and it was slower than having constantly incrementing gid. Probably that happens due to linear search by gid in gxact array on commit. So I used :scale just as a counter, bacause it is initialised on pgbench start and line like “\set scale :scale+1” works well. (may be there is a different way to do it in pgbench).

I very rapidly reach a point where most of the updates are against
tuples that don't exist, and then get integer overflow problems.

Hmm, that’s strange. Probably you set scale to big value, so that 100000*:scale is bigger that int4? But i thought that pgbench will change aid columns to bigint if scale is more than 20000.

Attachments:

2pc_xlog.v2.diffapplication/octet-stream; name=2pc_xlog.v2.diffDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 8c47e0f..e9a1ff5 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -51,6 +51,7 @@
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "access/xlogutils.h"
+#include "access/xlogreader.h"
 #include "catalog/pg_type.h"
 #include "catalog/storage.h"
 #include "funcapi.h"
@@ -60,6 +61,7 @@
 #include "replication/origin.h"
 #include "replication/syncrep.h"
 #include "replication/walsender.h"
+#include "replication/logicalfuncs.h"
 #include "storage/fd.h"
 #include "storage/ipc.h"
 #include "storage/predicate.h"
@@ -117,7 +119,11 @@ typedef struct GlobalTransactionData
 	int			pgprocno;		/* ID of associated dummy PGPROC */
 	BackendId	dummyBackendId; /* similar to backend id for backends */
 	TimestampTz prepared_at;	/* time of preparation */
-	XLogRecPtr	prepare_lsn;	/* XLOG offset of prepare record */
+	XLogRecPtr	prepare_start_lsn;	/* XLOG offset of prepare record start
+									 * or InvalidXLogRecPtr if twophase data
+									 * moved to file after checkpoint.
+									 */
+	XLogRecPtr	prepare_end_lsn;	/* XLOG offset of prepare record end */
 	Oid			owner;			/* ID of user that executed the xact */
 	BackendId	locking_backend;	/* backend currently working on the xact */
 	bool		valid;			/* TRUE if PGPROC entry is in proc array */
@@ -166,6 +172,7 @@ static void ProcessRecords(char *bufptr, TransactionId xid,
 			   const TwoPhaseCallback callbacks[]);
 static void RemoveGXact(GlobalTransaction gxact);
 
+static void XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len);
 
 /*
  * Initialization of shared memory
@@ -398,8 +405,9 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 	pgxact->nxids = 0;
 
 	gxact->prepared_at = prepared_at;
-	/* initialize LSN to 0 (start of WAL) */
-	gxact->prepare_lsn = 0;
+	/* initialize LSN to InvalidXLogRecPtr */
+	gxact->prepare_start_lsn = InvalidXLogRecPtr;
+	gxact->prepare_end_lsn = InvalidXLogRecPtr;
 	gxact->owner = owner;
 	gxact->locking_backend = MyBackendId;
 	gxact->valid = false;
@@ -579,41 +587,6 @@ RemoveGXact(GlobalTransaction gxact)
 }
 
 /*
- * TransactionIdIsPrepared
- *		True iff transaction associated with the identifier is prepared
- *		for two-phase commit
- *
- * Note: only gxacts marked "valid" are considered; but notice we do not
- * check the locking status.
- *
- * This is not currently exported, because it is only needed internally.
- */
-static bool
-TransactionIdIsPrepared(TransactionId xid)
-{
-	bool		result = false;
-	int			i;
-
-	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
-
-	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
-	{
-		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
-		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
-
-		if (gxact->valid && pgxact->xid == xid)
-		{
-			result = true;
-			break;
-		}
-	}
-
-	LWLockRelease(TwoPhaseStateLock);
-
-	return result;
-}
-
-/*
  * Returns an array of all prepared transactions for the user-level
  * function pg_prepared_xact.
  *
@@ -1020,14 +993,8 @@ StartPrepare(GlobalTransaction gxact)
 void
 EndPrepare(GlobalTransaction gxact)
 {
-	PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
-	TransactionId xid = pgxact->xid;
 	TwoPhaseFileHeader *hdr;
-	char		path[MAXPGPATH];
 	StateFileChunk *record;
-	pg_crc32c	statefile_crc;
-	pg_crc32c	bogus_crc;
-	int			fd;
 
 	/* Add the end sentinel to the list of 2PC records */
 	RegisterTwoPhaseRecord(TWOPHASE_RM_END_ID, 0,
@@ -1048,70 +1015,7 @@ EndPrepare(GlobalTransaction gxact)
 				 errmsg("two-phase state file maximum length exceeded")));
 
 	/*
-	 * Create the 2PC state file.
-	 */
-	TwoPhaseFilePath(path, xid);
-
-	fd = OpenTransientFile(path,
-						   O_CREAT | O_EXCL | O_WRONLY | PG_BINARY,
-						   S_IRUSR | S_IWUSR);
-	if (fd < 0)
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not create two-phase state file \"%s\": %m",
-						path)));
-
-	/* Write data to file, and calculate CRC as we pass over it */
-	INIT_CRC32C(statefile_crc);
-
-	for (record = records.head; record != NULL; record = record->next)
-	{
-		COMP_CRC32C(statefile_crc, record->data, record->len);
-		if ((write(fd, record->data, record->len)) != record->len)
-		{
-			CloseTransientFile(fd);
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not write two-phase state file: %m")));
-		}
-	}
-
-	FIN_CRC32C(statefile_crc);
-
-	/*
-	 * Write a deliberately bogus CRC to the state file; this is just paranoia
-	 * to catch the case where four more bytes will run us out of disk space.
-	 */
-	bogus_crc = ~statefile_crc;
-
-	if ((write(fd, &bogus_crc, sizeof(pg_crc32c))) != sizeof(pg_crc32c))
-	{
-		CloseTransientFile(fd);
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not write two-phase state file: %m")));
-	}
-
-	/* Back up to prepare for rewriting the CRC */
-	if (lseek(fd, -((off_t) sizeof(pg_crc32c)), SEEK_CUR) < 0)
-	{
-		CloseTransientFile(fd);
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not seek in two-phase state file: %m")));
-	}
-
-	/*
-	 * The state file isn't valid yet, because we haven't written the correct
-	 * CRC yet.  Before we do that, insert entry in WAL and flush it to disk.
-	 *
-	 * Between the time we have written the WAL entry and the time we write
-	 * out the correct state file CRC, we have an inconsistency: the xact is
-	 * prepared according to WAL but not according to our on-disk state. We
-	 * use a critical section to force a PANIC if we are unable to complete
-	 * the write --- then, WAL replay should repair the inconsistency.  The
-	 * odds of a PANIC actually occurring should be very tiny given that we
-	 * were able to write the bogus CRC above.
+	 * Now writing 2PC state data to WAL.
 	 *
 	 * We have to set delayChkpt here, too; otherwise a checkpoint starting
 	 * immediately after the WAL record is inserted could complete without
@@ -1131,24 +1035,13 @@ EndPrepare(GlobalTransaction gxact)
 	XLogBeginInsert();
 	for (record = records.head; record != NULL; record = record->next)
 		XLogRegisterData(record->data, record->len);
-	gxact->prepare_lsn = XLogInsert(RM_XACT_ID, XLOG_XACT_PREPARE);
-	XLogFlush(gxact->prepare_lsn);
+	gxact->prepare_end_lsn = XLogInsert(RM_XACT_ID, XLOG_XACT_PREPARE);
+	XLogFlush(gxact->prepare_end_lsn);
 
 	/* If we crash now, we have prepared: WAL replay will fix things */
 
-	/* write correct CRC and close file */
-	if ((write(fd, &statefile_crc, sizeof(pg_crc32c))) != sizeof(pg_crc32c))
-	{
-		CloseTransientFile(fd);
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not write two-phase state file: %m")));
-	}
-
-	if (CloseTransientFile(fd) != 0)
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not close two-phase state file: %m")));
+	/* Store record's start location to read that later on Commit */
+	gxact->prepare_start_lsn = ProcLastRecPtr;
 
 	/*
 	 * Mark the prepared transaction as valid.  As soon as xact.c marks
@@ -1186,7 +1079,7 @@ EndPrepare(GlobalTransaction gxact)
 	 * Note that at this stage we have marked the prepare, but still show as
 	 * running in the procarray (twice!) and continue to hold locks.
 	 */
-	SyncRepWaitForLSN(gxact->prepare_lsn);
+	SyncRepWaitForLSN(gxact->prepare_end_lsn);
 
 	records.tail = records.head = NULL;
 	records.num_chunks = 0;
@@ -1315,6 +1208,36 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
 	return buf;
 }
 
+
+/*
+ * Reads 2PC data from xlog. During checkpoint this data will be moved to
+ * twophase files and ReadTwoPhaseFile should be used instead.
+ */
+static void
+XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
+{
+	XLogRecord *record;
+	XLogReaderState *xlogreader;
+	char	   *errormsg;
+
+	xlogreader = XLogReaderAllocate(&logical_read_local_xlog_page, NULL);
+	if (xlogreader == NULL)
+		elog(ERROR, "failed to open xlogreader for reading 2PC data");
+
+	record = XLogReadRecord(xlogreader, lsn, &errormsg);
+	if (record == NULL)
+		elog(ERROR, "failed to read 2PC record from xlog");
+
+	if (len != NULL)
+		*len = XLogRecGetDataLen(xlogreader);
+
+	*buf = palloc(sizeof(char)*XLogRecGetDataLen(xlogreader));
+	memcpy(*buf, XLogRecGetData(xlogreader), sizeof(char)*XLogRecGetDataLen(xlogreader));
+
+	XLogReaderFree(xlogreader);
+}
+
+
 /*
  * Confirms an xid is prepared, during recovery
  */
@@ -1364,6 +1287,7 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	int			ndelrels;
 	SharedInvalidationMessage *invalmsgs;
 	int			i;
+	bool		file_used = false;
 
 	/*
 	 * Validate the GID, and lock the GXACT to ensure that two backends do not
@@ -1375,14 +1299,20 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	xid = pgxact->xid;
 
 	/*
-	 * Read and validate the state file
+	 * Read and validate 2PC state data.
+	 * State data can be stored in xlog or in files after xlog checkpoint.
+	 * While checkpointing we set gxact->prepare_start_lsn to NULL to signalize
+	 * that 2PC data is moved to files.
 	 */
-	buf = ReadTwoPhaseFile(xid, true);
-	if (buf == NULL)
-		ereport(ERROR,
-				(errcode(ERRCODE_DATA_CORRUPTED),
-				 errmsg("two-phase state file for transaction %u is corrupt",
-						xid)));
+	if (gxact->prepare_start_lsn)
+	{
+		XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, NULL);
+	}
+	else
+	{
+		buf = ReadTwoPhaseFile(xid, true);
+		file_used = true;
+	}
 
 	/*
 	 * Disassemble the header area
@@ -1484,7 +1414,8 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	/*
 	 * And now we can clean up our mess.
 	 */
-	RemoveTwoPhaseFile(xid, true);
+	if (file_used)
+		RemoveTwoPhaseFile(xid, true);
 
 	RemoveGXact(gxact);
 	MyLockedGxact = NULL;
@@ -1539,7 +1470,8 @@ RemoveTwoPhaseFile(TransactionId xid, bool giveWarning)
 }
 
 /*
- * Recreates a state file. This is used in WAL replay.
+ * Recreates a state file. This is used in WAL replay and during
+ * checkpoint creation.
  *
  * Note: content and len don't include CRC.
  */
@@ -1620,85 +1552,38 @@ RecreateTwoPhaseFile(TransactionId xid, void *content, int len)
 void
 CheckPointTwoPhase(XLogRecPtr redo_horizon)
 {
-	TransactionId *xids;
-	int			nxids;
-	char		path[MAXPGPATH];
 	int			i;
+	int 		len;
+	char	   *buf;
 
-	/*
-	 * We don't want to hold the TwoPhaseStateLock while doing I/O, so we grab
-	 * it just long enough to make a list of the XIDs that require fsyncing,
-	 * and then do the I/O afterwards.
-	 *
-	 * This approach creates a race condition: someone else could delete a
-	 * GXACT between the time we release TwoPhaseStateLock and the time we try
-	 * to open its state file.  We handle this by special-casing ENOENT
-	 * failures: if we see that, we verify that the GXACT is no longer valid,
-	 * and if so ignore the failure.
-	 */
 	if (max_prepared_xacts <= 0)
 		return;					/* nothing to do */
 
 	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_START();
 
-	xids = (TransactionId *) palloc(max_prepared_xacts * sizeof(TransactionId));
-	nxids = 0;
-
+	/*
+	 * Here we doing whole I/O while holding TwoPhaseStateLock.
+	 * It's also possible to move I/O out of the lock, but on
+	 * every error we should check whether somebody commited our
+	 * transaction in different backend. Let's leave this optimisation
+	 * for future, if somebody will spot that this place cause
+	 * bottleneck.
+	 */
 	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
-
 	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
 	{
 		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
 		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 
-		if (gxact->valid &&
-			gxact->prepare_lsn <= redo_horizon)
-			xids[nxids++] = pgxact->xid;
-	}
-
-	LWLockRelease(TwoPhaseStateLock);
-
-	for (i = 0; i < nxids; i++)
-	{
-		TransactionId xid = xids[i];
-		int			fd;
-
-		TwoPhaseFilePath(path, xid);
-
-		fd = OpenTransientFile(path, O_RDWR | PG_BINARY, 0);
-		if (fd < 0)
-		{
-			if (errno == ENOENT)
-			{
-				/* OK if gxact is no longer valid */
-				if (!TransactionIdIsPrepared(xid))
-					continue;
-				/* Restore errno in case it was changed */
-				errno = ENOENT;
-			}
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not open two-phase state file \"%s\": %m",
-							path)));
-		}
-
-		if (pg_fsync(fd) != 0)
-		{
-			CloseTransientFile(fd);
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not fsync two-phase state file \"%s\": %m",
-							path)));
+		if (gxact->valid && gxact->prepare_start_lsn != InvalidXLogRecPtr &&
+										gxact->prepare_end_lsn <= redo_horizon){
+			XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, &len);
+			RecreateTwoPhaseFile(pgxact->xid, buf, len);
+			gxact->prepare_start_lsn = InvalidXLogRecPtr;
+			pfree(buf);
 		}
-
-		if (CloseTransientFile(fd) != 0)
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not close two-phase state file \"%s\": %m",
-							path)));
 	}
-
-	pfree(xids);
+	LWLockRelease(TwoPhaseStateLock);
 
 	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_DONE();
 }
@@ -2030,12 +1915,8 @@ RecoverPreparedTransactions(void)
 			/*
 			 * Recreate its GXACT and dummy PGPROC
 			 *
-			 * Note: since we don't have the PREPARE record's WAL location at
-			 * hand, we leave prepare_lsn zeroes.  This means the GXACT will
-			 * be fsync'd on every future checkpoint.  We assume this
-			 * situation is infrequent enough that the performance cost is
-			 * negligible (especially since we know the state file has already
-			 * been fsynced).
+			 * MarkAsPreparing sets prepare_start_lsn to InvalidXLogRecPtr
+			 * so next checkpoint will skip that transaction.
 			 */
 			gxact = MarkAsPreparing(xid, hdr->gid,
 									hdr->prepared_at,
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 86debf4..3683785 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -321,8 +321,7 @@ static TimeLineID curFileTLI;
  * stored here.  The parallel leader advances its own copy, when necessary,
  * in WaitForParallelWorkersToFinish.
  */
-static XLogRecPtr ProcLastRecPtr = InvalidXLogRecPtr;
-
+XLogRecPtr	ProcLastRecPtr = InvalidXLogRecPtr;
 XLogRecPtr	XactLastRecEnd = InvalidXLogRecPtr;
 XLogRecPtr	XactLastCommitEnd = InvalidXLogRecPtr;
 
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 790ca66..a6d04cc 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -86,6 +86,7 @@ typedef enum
 	RECOVERY_TARGET_IMMEDIATE
 } RecoveryTargetType;
 
+extern XLogRecPtr ProcLastRecPtr;
 extern XLogRecPtr XactLastRecEnd;
 extern PGDLLIMPORT XLogRecPtr XactLastCommitEnd;
 
#7Simon Riggs
simon@2ndQuadrant.com
In reply to: Stas Kelvich (#1)
Re: Speedup twophase transactions

On 9 December 2015 at 18:44, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

In this patch I’ve changed this procedures to following:
* on prepare backend writes data only to xlog and store pointer to the
start of the xlog record
* if commit occurs before checkpoint then backend reads data from xlog by
this pointer
* on checkpoint 2pc data copied to files and fsynced
* if commit happens after checkpoint then backend reads files
* in case of crash replay will move data from xlog to files (as it was
before patch)

This looks sound to me.

I think we could do better still, but this looks like the easiest 80% and
actually removes code.

The lack of substantial comments on the patch is a problem though - the
details above should go in the patch. I'll have a go at reworking this for
you, this time.

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#8Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Simon Riggs (#7)
Re: Speedup twophase transactions

Simon Riggs wrote:

I think we could do better still, but this looks like the easiest 80% and
actually removes code.

The lack of substantial comments on the patch is a problem though - the
details above should go in the patch. I'll have a go at reworking this for
you, this time.

Is someone submitting an updated patch here?

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#9Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Alvaro Herrera (#8)
1 attachment(s)
Re: Speedup twophase transactions

Hi.

I’ve updated patch and wrote description of thighs that happens
with 2PC state data in the beginning of the file. I think now this patch is well documented,
but if somebody points me to places that probably requires more detailed description I’m ready
to extend that.

Attachments:

2pc_xlog.diffapplication/octet-stream; name=2pc_xlog.diffDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 8c47e0f..267f99b 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -28,8 +28,21 @@
  *		In order to survive crashes and shutdowns, all prepared
  *		transactions must be stored in permanent storage. This includes
  *		locking information, pending notifications etc. All that state
- *		information is written to the per-transaction state file in
- *		the pg_twophase directory.
+ *		information is written to the WAL and later can be moved to the
+ *		per-transaction state file in the pg_twophase directory. Life track of
+ *		state data is following:
+ *
+ *		* On PREPARE TRANSACTION backend writes state data only to the WAL and
+ *		  stores pointer to the start of the WAL record in
+ *		  gxact->prepare_start_lsn.
+ *		* If COMMIT occurs before checkpoint then backend reads data from WAL
+ *		  using prepare_start_lsn.
+ *		* On checkpoint state data copied to files in pg_twophase directory and
+ *		  fsynced
+ *		* If COMMIT happens after checkpoint then backend reads state data from
+ *		  files
+ *		* In case of crash replay will move data from xlog to files, if that
+ *		  hasn't happened before.
  *
  *-------------------------------------------------------------------------
  */
@@ -51,6 +64,7 @@
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "access/xlogutils.h"
+#include "access/xlogreader.h"
 #include "catalog/pg_type.h"
 #include "catalog/storage.h"
 #include "funcapi.h"
@@ -60,6 +74,7 @@
 #include "replication/origin.h"
 #include "replication/syncrep.h"
 #include "replication/walsender.h"
+#include "replication/logicalfuncs.h"
 #include "storage/fd.h"
 #include "storage/ipc.h"
 #include "storage/predicate.h"
@@ -117,7 +132,11 @@ typedef struct GlobalTransactionData
 	int			pgprocno;		/* ID of associated dummy PGPROC */
 	BackendId	dummyBackendId; /* similar to backend id for backends */
 	TimestampTz prepared_at;	/* time of preparation */
-	XLogRecPtr	prepare_lsn;	/* XLOG offset of prepare record */
+	XLogRecPtr	prepare_start_lsn;	/* XLOG offset of prepare record start
+									 * or InvalidXLogRecPtr if twophase data
+									 * moved to file after checkpoint.
+									 */
+	XLogRecPtr	prepare_end_lsn;	/* XLOG offset of prepare record end */
 	Oid			owner;			/* ID of user that executed the xact */
 	BackendId	locking_backend;	/* backend currently working on the xact */
 	bool		valid;			/* TRUE if PGPROC entry is in proc array */
@@ -166,6 +185,7 @@ static void ProcessRecords(char *bufptr, TransactionId xid,
 			   const TwoPhaseCallback callbacks[]);
 static void RemoveGXact(GlobalTransaction gxact);
 
+static void XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len);
 
 /*
  * Initialization of shared memory
@@ -398,8 +418,9 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 	pgxact->nxids = 0;
 
 	gxact->prepared_at = prepared_at;
-	/* initialize LSN to 0 (start of WAL) */
-	gxact->prepare_lsn = 0;
+	/* initialize LSN to InvalidXLogRecPtr */
+	gxact->prepare_start_lsn = InvalidXLogRecPtr;
+	gxact->prepare_end_lsn = InvalidXLogRecPtr;
 	gxact->owner = owner;
 	gxact->locking_backend = MyBackendId;
 	gxact->valid = false;
@@ -579,41 +600,6 @@ RemoveGXact(GlobalTransaction gxact)
 }
 
 /*
- * TransactionIdIsPrepared
- *		True iff transaction associated with the identifier is prepared
- *		for two-phase commit
- *
- * Note: only gxacts marked "valid" are considered; but notice we do not
- * check the locking status.
- *
- * This is not currently exported, because it is only needed internally.
- */
-static bool
-TransactionIdIsPrepared(TransactionId xid)
-{
-	bool		result = false;
-	int			i;
-
-	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
-
-	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
-	{
-		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
-		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
-
-		if (gxact->valid && pgxact->xid == xid)
-		{
-			result = true;
-			break;
-		}
-	}
-
-	LWLockRelease(TwoPhaseStateLock);
-
-	return result;
-}
-
-/*
  * Returns an array of all prepared transactions for the user-level
  * function pg_prepared_xact.
  *
@@ -1020,14 +1006,8 @@ StartPrepare(GlobalTransaction gxact)
 void
 EndPrepare(GlobalTransaction gxact)
 {
-	PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
-	TransactionId xid = pgxact->xid;
 	TwoPhaseFileHeader *hdr;
-	char		path[MAXPGPATH];
 	StateFileChunk *record;
-	pg_crc32c	statefile_crc;
-	pg_crc32c	bogus_crc;
-	int			fd;
 
 	/* Add the end sentinel to the list of 2PC records */
 	RegisterTwoPhaseRecord(TWOPHASE_RM_END_ID, 0,
@@ -1048,70 +1028,7 @@ EndPrepare(GlobalTransaction gxact)
 				 errmsg("two-phase state file maximum length exceeded")));
 
 	/*
-	 * Create the 2PC state file.
-	 */
-	TwoPhaseFilePath(path, xid);
-
-	fd = OpenTransientFile(path,
-						   O_CREAT | O_EXCL | O_WRONLY | PG_BINARY,
-						   S_IRUSR | S_IWUSR);
-	if (fd < 0)
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not create two-phase state file \"%s\": %m",
-						path)));
-
-	/* Write data to file, and calculate CRC as we pass over it */
-	INIT_CRC32C(statefile_crc);
-
-	for (record = records.head; record != NULL; record = record->next)
-	{
-		COMP_CRC32C(statefile_crc, record->data, record->len);
-		if ((write(fd, record->data, record->len)) != record->len)
-		{
-			CloseTransientFile(fd);
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not write two-phase state file: %m")));
-		}
-	}
-
-	FIN_CRC32C(statefile_crc);
-
-	/*
-	 * Write a deliberately bogus CRC to the state file; this is just paranoia
-	 * to catch the case where four more bytes will run us out of disk space.
-	 */
-	bogus_crc = ~statefile_crc;
-
-	if ((write(fd, &bogus_crc, sizeof(pg_crc32c))) != sizeof(pg_crc32c))
-	{
-		CloseTransientFile(fd);
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not write two-phase state file: %m")));
-	}
-
-	/* Back up to prepare for rewriting the CRC */
-	if (lseek(fd, -((off_t) sizeof(pg_crc32c)), SEEK_CUR) < 0)
-	{
-		CloseTransientFile(fd);
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not seek in two-phase state file: %m")));
-	}
-
-	/*
-	 * The state file isn't valid yet, because we haven't written the correct
-	 * CRC yet.  Before we do that, insert entry in WAL and flush it to disk.
-	 *
-	 * Between the time we have written the WAL entry and the time we write
-	 * out the correct state file CRC, we have an inconsistency: the xact is
-	 * prepared according to WAL but not according to our on-disk state. We
-	 * use a critical section to force a PANIC if we are unable to complete
-	 * the write --- then, WAL replay should repair the inconsistency.  The
-	 * odds of a PANIC actually occurring should be very tiny given that we
-	 * were able to write the bogus CRC above.
+	 * Now writing 2PC state data to WAL.
 	 *
 	 * We have to set delayChkpt here, too; otherwise a checkpoint starting
 	 * immediately after the WAL record is inserted could complete without
@@ -1131,24 +1048,13 @@ EndPrepare(GlobalTransaction gxact)
 	XLogBeginInsert();
 	for (record = records.head; record != NULL; record = record->next)
 		XLogRegisterData(record->data, record->len);
-	gxact->prepare_lsn = XLogInsert(RM_XACT_ID, XLOG_XACT_PREPARE);
-	XLogFlush(gxact->prepare_lsn);
+	gxact->prepare_end_lsn = XLogInsert(RM_XACT_ID, XLOG_XACT_PREPARE);
+	XLogFlush(gxact->prepare_end_lsn);
 
 	/* If we crash now, we have prepared: WAL replay will fix things */
 
-	/* write correct CRC and close file */
-	if ((write(fd, &statefile_crc, sizeof(pg_crc32c))) != sizeof(pg_crc32c))
-	{
-		CloseTransientFile(fd);
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not write two-phase state file: %m")));
-	}
-
-	if (CloseTransientFile(fd) != 0)
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not close two-phase state file: %m")));
+	/* Store record's start location to read that later on Commit */
+	gxact->prepare_start_lsn = ProcLastRecPtr;
 
 	/*
 	 * Mark the prepared transaction as valid.  As soon as xact.c marks
@@ -1186,7 +1092,7 @@ EndPrepare(GlobalTransaction gxact)
 	 * Note that at this stage we have marked the prepare, but still show as
 	 * running in the procarray (twice!) and continue to hold locks.
 	 */
-	SyncRepWaitForLSN(gxact->prepare_lsn);
+	SyncRepWaitForLSN(gxact->prepare_end_lsn);
 
 	records.tail = records.head = NULL;
 	records.num_chunks = 0;
@@ -1315,6 +1221,36 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
 	return buf;
 }
 
+
+/*
+ * Reads 2PC data from xlog. During checkpoint this data will be moved to
+ * twophase files and ReadTwoPhaseFile should be used instead.
+ */
+static void
+XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
+{
+	XLogRecord *record;
+	XLogReaderState *xlogreader;
+	char	   *errormsg;
+
+	xlogreader = XLogReaderAllocate(&logical_read_local_xlog_page, NULL);
+	if (xlogreader == NULL)
+		elog(ERROR, "failed to open xlogreader for reading 2PC data");
+
+	record = XLogReadRecord(xlogreader, lsn, &errormsg);
+	if (record == NULL)
+		elog(ERROR, "failed to read 2PC record from xlog");
+
+	if (len != NULL)
+		*len = XLogRecGetDataLen(xlogreader);
+
+	*buf = palloc(sizeof(char)*XLogRecGetDataLen(xlogreader));
+	memcpy(*buf, XLogRecGetData(xlogreader), sizeof(char)*XLogRecGetDataLen(xlogreader));
+
+	XLogReaderFree(xlogreader);
+}
+
+
 /*
  * Confirms an xid is prepared, during recovery
  */
@@ -1364,6 +1300,7 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	int			ndelrels;
 	SharedInvalidationMessage *invalmsgs;
 	int			i;
+	bool		file_used = false;
 
 	/*
 	 * Validate the GID, and lock the GXACT to ensure that two backends do not
@@ -1375,14 +1312,20 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	xid = pgxact->xid;
 
 	/*
-	 * Read and validate the state file
+	 * Read and validate 2PC state data.
+	 * State data can be stored in xlog or in files after xlog checkpoint.
+	 * While checkpointing we set gxact->prepare_start_lsn to NULL to signalize
+	 * that 2PC data is moved to files.
 	 */
-	buf = ReadTwoPhaseFile(xid, true);
-	if (buf == NULL)
-		ereport(ERROR,
-				(errcode(ERRCODE_DATA_CORRUPTED),
-				 errmsg("two-phase state file for transaction %u is corrupt",
-						xid)));
+	if (gxact->prepare_start_lsn)
+	{
+		XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, NULL);
+	}
+	else
+	{
+		buf = ReadTwoPhaseFile(xid, true);
+		file_used = true;
+	}
 
 	/*
 	 * Disassemble the header area
@@ -1484,7 +1427,8 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	/*
 	 * And now we can clean up our mess.
 	 */
-	RemoveTwoPhaseFile(xid, true);
+	if (file_used)
+		RemoveTwoPhaseFile(xid, true);
 
 	RemoveGXact(gxact);
 	MyLockedGxact = NULL;
@@ -1539,7 +1483,8 @@ RemoveTwoPhaseFile(TransactionId xid, bool giveWarning)
 }
 
 /*
- * Recreates a state file. This is used in WAL replay.
+ * Recreates a state file. This is used in WAL replay and during
+ * checkpoint creation.
  *
  * Note: content and len don't include CRC.
  */
@@ -1620,85 +1565,38 @@ RecreateTwoPhaseFile(TransactionId xid, void *content, int len)
 void
 CheckPointTwoPhase(XLogRecPtr redo_horizon)
 {
-	TransactionId *xids;
-	int			nxids;
-	char		path[MAXPGPATH];
 	int			i;
+	int 		len;
+	char	   *buf;
 
-	/*
-	 * We don't want to hold the TwoPhaseStateLock while doing I/O, so we grab
-	 * it just long enough to make a list of the XIDs that require fsyncing,
-	 * and then do the I/O afterwards.
-	 *
-	 * This approach creates a race condition: someone else could delete a
-	 * GXACT between the time we release TwoPhaseStateLock and the time we try
-	 * to open its state file.  We handle this by special-casing ENOENT
-	 * failures: if we see that, we verify that the GXACT is no longer valid,
-	 * and if so ignore the failure.
-	 */
 	if (max_prepared_xacts <= 0)
 		return;					/* nothing to do */
 
 	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_START();
 
-	xids = (TransactionId *) palloc(max_prepared_xacts * sizeof(TransactionId));
-	nxids = 0;
-
+	/*
+	 * Here we doing whole I/O while holding TwoPhaseStateLock.
+	 * It's also possible to move I/O out of the lock, but on
+	 * every error we should check whether somebody commited our
+	 * transaction in different backend. Let's leave this optimisation
+	 * for future, if somebody will spot that this place cause
+	 * bottleneck.
+	 */
 	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
-
 	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
 	{
 		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
 		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 
-		if (gxact->valid &&
-			gxact->prepare_lsn <= redo_horizon)
-			xids[nxids++] = pgxact->xid;
-	}
-
-	LWLockRelease(TwoPhaseStateLock);
-
-	for (i = 0; i < nxids; i++)
-	{
-		TransactionId xid = xids[i];
-		int			fd;
-
-		TwoPhaseFilePath(path, xid);
-
-		fd = OpenTransientFile(path, O_RDWR | PG_BINARY, 0);
-		if (fd < 0)
-		{
-			if (errno == ENOENT)
-			{
-				/* OK if gxact is no longer valid */
-				if (!TransactionIdIsPrepared(xid))
-					continue;
-				/* Restore errno in case it was changed */
-				errno = ENOENT;
-			}
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not open two-phase state file \"%s\": %m",
-							path)));
-		}
-
-		if (pg_fsync(fd) != 0)
-		{
-			CloseTransientFile(fd);
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not fsync two-phase state file \"%s\": %m",
-							path)));
+		if (gxact->valid && gxact->prepare_start_lsn != InvalidXLogRecPtr &&
+										gxact->prepare_end_lsn <= redo_horizon){
+			XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, &len);
+			RecreateTwoPhaseFile(pgxact->xid, buf, len);
+			gxact->prepare_start_lsn = InvalidXLogRecPtr;
+			pfree(buf);
 		}
-
-		if (CloseTransientFile(fd) != 0)
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not close two-phase state file \"%s\": %m",
-							path)));
 	}
-
-	pfree(xids);
+	LWLockRelease(TwoPhaseStateLock);
 
 	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_DONE();
 }
@@ -2030,12 +1928,8 @@ RecoverPreparedTransactions(void)
 			/*
 			 * Recreate its GXACT and dummy PGPROC
 			 *
-			 * Note: since we don't have the PREPARE record's WAL location at
-			 * hand, we leave prepare_lsn zeroes.  This means the GXACT will
-			 * be fsync'd on every future checkpoint.  We assume this
-			 * situation is infrequent enough that the performance cost is
-			 * negligible (especially since we know the state file has already
-			 * been fsynced).
+			 * MarkAsPreparing sets prepare_start_lsn to InvalidXLogRecPtr
+			 * so next checkpoint will skip that transaction.
 			 */
 			gxact = MarkAsPreparing(xid, hdr->gid,
 									hdr->prepared_at,
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 147fd53..598faf0 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -321,8 +321,7 @@ static TimeLineID curFileTLI;
  * stored here.  The parallel leader advances its own copy, when necessary,
  * in WaitForParallelWorkersToFinish.
  */
-static XLogRecPtr ProcLastRecPtr = InvalidXLogRecPtr;
-
+XLogRecPtr	ProcLastRecPtr = InvalidXLogRecPtr;
 XLogRecPtr	XactLastRecEnd = InvalidXLogRecPtr;
 XLogRecPtr	XactLastCommitEnd = InvalidXLogRecPtr;
 
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 790ca66..a6d04cc 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -86,6 +86,7 @@ typedef enum
 	RECOVERY_TARGET_IMMEDIATE
 } RecoveryTargetType;
 
+extern XLogRecPtr ProcLastRecPtr;
 extern XLogRecPtr XactLastRecEnd;
 extern PGDLLIMPORT XLogRecPtr XactLastCommitEnd;
 
#10Simon Riggs
simon@2ndQuadrant.com
In reply to: Stas Kelvich (#9)
1 attachment(s)
Re: Speedup twophase transactions

On 9 January 2016 at 12:26, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

I’ve updated patch and wrote description of thighs that happens
with 2PC state data in the beginning of the file. I think now this patch
is well documented,
but if somebody points me to places that probably requires more detailed
description I’m ready
to extend that.

Hmm, I was just preparing this for commit.

Please have a look at my mild edits and extended comments.

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Attachments:

2pc_optimize.v2.patchapplication/octet-stream; name=2pc_optimize.v2.patchDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 2251b02..7b8b620 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -25,11 +25,14 @@
  *		what keeps the XID considered running by TransactionIdIsInProgress.
  *		It is also convenient as a PGPROC to hook the gxact's locks to.
  *
- *		In order to survive crashes and shutdowns, all prepared
- *		transactions must be stored in permanent storage. This includes
- *		locking information, pending notifications etc. All that state
- *		information is written to the per-transaction state file in
- *		the pg_twophase directory.
+ * 		Information to recover prepared transactions in case of crash is
+ * 		now stored in WAL for the common case. In some cases there will be
+ * 		an extended period between preparing a GXACT and commit/abort, in
+ * 		which case we need to separately record prepared transaction data
+ * 		in permanent storage. This includes locking information, pending
+ * 		notifications etc. All that state information is written to the
+ * 		per-transaction state file in the pg_twophase directory.
+ * 		All prepared transactions will be written prior to shutdown.
  *
  *-------------------------------------------------------------------------
  */
@@ -51,6 +54,7 @@
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "access/xlogutils.h"
+#include "access/xlogreader.h"
 #include "catalog/pg_type.h"
 #include "catalog/storage.h"
 #include "funcapi.h"
@@ -60,6 +64,7 @@
 #include "replication/origin.h"
 #include "replication/syncrep.h"
 #include "replication/walsender.h"
+#include "replication/logicalfuncs.h"
 #include "storage/fd.h"
 #include "storage/ipc.h"
 #include "storage/predicate.h"
@@ -117,10 +122,21 @@ typedef struct GlobalTransactionData
 	int			pgprocno;		/* ID of associated dummy PGPROC */
 	BackendId	dummyBackendId; /* similar to backend id for backends */
 	TimestampTz prepared_at;	/* time of preparation */
-	XLogRecPtr	prepare_lsn;	/* XLOG offset of prepare record */
+
+	/*
+	 * Note that we need to keep track of two LSNs for each GXACT.
+	 * We keep track of the start LSN because this is the address we must
+	 * use to read state data back from WAL when committing a prepared GXACT.
+	 * We keep track of the end LSN because that is the LSN we need to wait
+	 * for prior to commit.
+	 */
+	XLogRecPtr	prepare_start_lsn;	/* XLOG offset of prepare record start */
+	XLogRecPtr	prepare_end_lsn;	/* XLOG offset of prepare record end */
+
 	Oid			owner;			/* ID of user that executed the xact */
 	BackendId	locking_backend;	/* backend currently working on the xact */
 	bool		valid;			/* TRUE if PGPROC entry is in proc array */
+	bool		ondisk;			/* TRUE if prepare state file is on disk */
 	char		gid[GIDSIZE];	/* The GID assigned to the prepared xact */
 }	GlobalTransactionData;
 
@@ -166,6 +182,7 @@ static void ProcessRecords(char *bufptr, TransactionId xid,
 			   const TwoPhaseCallback callbacks[]);
 static void RemoveGXact(GlobalTransaction gxact);
 
+static void XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len);
 
 /*
  * Initialization of shared memory
@@ -398,8 +415,9 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 	pgxact->nxids = 0;
 
 	gxact->prepared_at = prepared_at;
-	/* initialize LSN to 0 (start of WAL) */
-	gxact->prepare_lsn = 0;
+	/* initialize LSN to InvalidXLogRecPtr */
+	gxact->prepare_start_lsn = InvalidXLogRecPtr;
+	gxact->prepare_end_lsn = InvalidXLogRecPtr;
 	gxact->owner = owner;
 	gxact->locking_backend = MyBackendId;
 	gxact->valid = false;
@@ -579,41 +597,6 @@ RemoveGXact(GlobalTransaction gxact)
 }
 
 /*
- * TransactionIdIsPrepared
- *		True iff transaction associated with the identifier is prepared
- *		for two-phase commit
- *
- * Note: only gxacts marked "valid" are considered; but notice we do not
- * check the locking status.
- *
- * This is not currently exported, because it is only needed internally.
- */
-static bool
-TransactionIdIsPrepared(TransactionId xid)
-{
-	bool		result = false;
-	int			i;
-
-	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
-
-	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
-	{
-		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
-		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
-
-		if (gxact->valid && pgxact->xid == xid)
-		{
-			result = true;
-			break;
-		}
-	}
-
-	LWLockRelease(TwoPhaseStateLock);
-
-	return result;
-}
-
-/*
  * Returns an array of all prepared transactions for the user-level
  * function pg_prepared_xact.
  *
@@ -1020,14 +1003,8 @@ StartPrepare(GlobalTransaction gxact)
 void
 EndPrepare(GlobalTransaction gxact)
 {
-	PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
-	TransactionId xid = pgxact->xid;
 	TwoPhaseFileHeader *hdr;
-	char		path[MAXPGPATH];
 	StateFileChunk *record;
-	pg_crc32c	statefile_crc;
-	pg_crc32c	bogus_crc;
-	int			fd;
 
 	/* Add the end sentinel to the list of 2PC records */
 	RegisterTwoPhaseRecord(TWOPHASE_RM_END_ID, 0,
@@ -1048,70 +1025,8 @@ EndPrepare(GlobalTransaction gxact)
 				 errmsg("two-phase state file maximum length exceeded")));
 
 	/*
-	 * Create the 2PC state file.
-	 */
-	TwoPhaseFilePath(path, xid);
-
-	fd = OpenTransientFile(path,
-						   O_CREAT | O_EXCL | O_WRONLY | PG_BINARY,
-						   S_IRUSR | S_IWUSR);
-	if (fd < 0)
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not create two-phase state file \"%s\": %m",
-						path)));
-
-	/* Write data to file, and calculate CRC as we pass over it */
-	INIT_CRC32C(statefile_crc);
-
-	for (record = records.head; record != NULL; record = record->next)
-	{
-		COMP_CRC32C(statefile_crc, record->data, record->len);
-		if ((write(fd, record->data, record->len)) != record->len)
-		{
-			CloseTransientFile(fd);
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not write two-phase state file: %m")));
-		}
-	}
-
-	FIN_CRC32C(statefile_crc);
-
-	/*
-	 * Write a deliberately bogus CRC to the state file; this is just paranoia
-	 * to catch the case where four more bytes will run us out of disk space.
-	 */
-	bogus_crc = ~statefile_crc;
-
-	if ((write(fd, &bogus_crc, sizeof(pg_crc32c))) != sizeof(pg_crc32c))
-	{
-		CloseTransientFile(fd);
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not write two-phase state file: %m")));
-	}
-
-	/* Back up to prepare for rewriting the CRC */
-	if (lseek(fd, -((off_t) sizeof(pg_crc32c)), SEEK_CUR) < 0)
-	{
-		CloseTransientFile(fd);
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not seek in two-phase state file: %m")));
-	}
-
-	/*
-	 * The state file isn't valid yet, because we haven't written the correct
-	 * CRC yet.  Before we do that, insert entry in WAL and flush it to disk.
-	 *
-	 * Between the time we have written the WAL entry and the time we write
-	 * out the correct state file CRC, we have an inconsistency: the xact is
-	 * prepared according to WAL but not according to our on-disk state. We
-	 * use a critical section to force a PANIC if we are unable to complete
-	 * the write --- then, WAL replay should repair the inconsistency.  The
-	 * odds of a PANIC actually occurring should be very tiny given that we
-	 * were able to write the bogus CRC above.
+	 * Now writing 2PC state data to WAL. We let the WAL's CRC protection
+	 * cover us, so no need to calculate a separate CRC.
 	 *
 	 * We have to set delayChkpt here, too; otherwise a checkpoint starting
 	 * immediately after the WAL record is inserted could complete without
@@ -1131,24 +1046,13 @@ EndPrepare(GlobalTransaction gxact)
 	XLogBeginInsert();
 	for (record = records.head; record != NULL; record = record->next)
 		XLogRegisterData(record->data, record->len);
-	gxact->prepare_lsn = XLogInsert(RM_XACT_ID, XLOG_XACT_PREPARE);
-	XLogFlush(gxact->prepare_lsn);
+	gxact->prepare_end_lsn = XLogInsert(RM_XACT_ID, XLOG_XACT_PREPARE);
+	XLogFlush(gxact->prepare_end_lsn);
 
 	/* If we crash now, we have prepared: WAL replay will fix things */
 
-	/* write correct CRC and close file */
-	if ((write(fd, &statefile_crc, sizeof(pg_crc32c))) != sizeof(pg_crc32c))
-	{
-		CloseTransientFile(fd);
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not write two-phase state file: %m")));
-	}
-
-	if (CloseTransientFile(fd) != 0)
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not close two-phase state file: %m")));
+	/* Store record's start location to read that later on Commit */
+	gxact->prepare_start_lsn = ProcLastRecPtr;
 
 	/*
 	 * Mark the prepared transaction as valid.  As soon as xact.c marks
@@ -1186,7 +1090,7 @@ EndPrepare(GlobalTransaction gxact)
 	 * Note that at this stage we have marked the prepare, but still show as
 	 * running in the procarray (twice!) and continue to hold locks.
 	 */
-	SyncRepWaitForLSN(gxact->prepare_lsn);
+	SyncRepWaitForLSN(gxact->prepare_end_lsn);
 
 	records.tail = records.head = NULL;
 	records.num_chunks = 0;
@@ -1315,6 +1219,36 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
 	return buf;
 }
 
+
+/*
+ * Reads 2PC data from xlog. During checkpoint this data will be moved to
+ * twophase files and ReadTwoPhaseFile should be used instead.
+ */
+static void
+XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
+{
+	XLogRecord *record;
+	XLogReaderState *xlogreader;
+	char	   *errormsg;
+
+	xlogreader = XLogReaderAllocate(&logical_read_local_xlog_page, NULL);
+	if (xlogreader == NULL)
+		elog(ERROR, "failed to open xlogreader for reading 2PC data");
+
+	record = XLogReadRecord(xlogreader, lsn, &errormsg);
+	if (record == NULL)
+		elog(ERROR, "failed to read 2PC record from xlog");
+
+	if (len != NULL)
+		*len = XLogRecGetDataLen(xlogreader);
+
+	*buf = palloc(sizeof(char)*XLogRecGetDataLen(xlogreader));
+	memcpy(*buf, XLogRecGetData(xlogreader), sizeof(char)*XLogRecGetDataLen(xlogreader));
+
+	XLogReaderFree(xlogreader);
+}
+
+
 /*
  * Confirms an xid is prepared, during recovery
  */
@@ -1375,14 +1309,16 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	xid = pgxact->xid;
 
 	/*
-	 * Read and validate the state file
+	 * Read and validate 2PC state data.
+	 * State data will typically be stored in WAL files if the LSN is after the
+	 * last checkpoint record, or moved to disk if for some reason they have
+	 * lived for a long time.
 	 */
-	buf = ReadTwoPhaseFile(xid, true);
-	if (buf == NULL)
-		ereport(ERROR,
-				(errcode(ERRCODE_DATA_CORRUPTED),
-				 errmsg("two-phase state file for transaction %u is corrupt",
-						xid)));
+	if (gxact->ondisk)
+		buf = ReadTwoPhaseFile(xid, true);
+	else
+		XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, NULL);
+
 
 	/*
 	 * Disassemble the header area
@@ -1482,9 +1418,10 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	AtEOXact_PgStat(isCommit);
 
 	/*
-	 * And now we can clean up our mess.
+	 * And now we can clean up any files we may have left.
 	 */
-	RemoveTwoPhaseFile(xid, true);
+	if (gxact->ondisk)
+		RemoveTwoPhaseFile(xid, true);
 
 	RemoveGXact(gxact);
 	MyLockedGxact = NULL;
@@ -1539,7 +1476,8 @@ RemoveTwoPhaseFile(TransactionId xid, bool giveWarning)
 }
 
 /*
- * Recreates a state file. This is used in WAL replay.
+ * Recreates a state file. This is used in WAL replay and during
+ * checkpoint creation.
  *
  * Note: content and len don't include CRC.
  */
@@ -1610,97 +1548,71 @@ RecreateTwoPhaseFile(TransactionId xid, void *content, int len)
  * This is deliberately run as late as possible in the checkpoint sequence,
  * because GXACTs ordinarily have short lifespans, and so it is quite
  * possible that GXACTs that were valid at checkpoint start will no longer
- * exist if we wait a little bit.
+ * exist if we wait a little bit. With typical checkpoint settings this
+ * will be about 3 minutes for an online checkpoint, so as a result we
+ * we expect that there will be no GXACTs that need to be copied to disk.
  *
- * If a GXACT remains valid across multiple checkpoints, it'll be fsynced
- * each time.  This is considered unusual enough that we don't bother to
- * expend any extra code to avoid the redundant fsyncs.  (They should be
- * reasonably cheap anyway, since they won't cause I/O.)
+ * If a GXACT remains valid across multiple checkpoints, it will already
+ * be on disk so we don't bother to repeat that write.
  */
 void
 CheckPointTwoPhase(XLogRecPtr redo_horizon)
 {
-	TransactionId *xids;
-	int			nxids;
-	char		path[MAXPGPATH];
 	int			i;
+	int			n = 0;
 
-	/*
-	 * We don't want to hold the TwoPhaseStateLock while doing I/O, so we grab
-	 * it just long enough to make a list of the XIDs that require fsyncing,
-	 * and then do the I/O afterwards.
-	 *
-	 * This approach creates a race condition: someone else could delete a
-	 * GXACT between the time we release TwoPhaseStateLock and the time we try
-	 * to open its state file.  We handle this by special-casing ENOENT
-	 * failures: if we see that, we verify that the GXACT is no longer valid,
-	 * and if so ignore the failure.
-	 */
 	if (max_prepared_xacts <= 0)
 		return;					/* nothing to do */
 
 	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_START();
 
-	xids = (TransactionId *) palloc(max_prepared_xacts * sizeof(TransactionId));
-	nxids = 0;
-
+	/*
+	 * We are expecting there to be zero GXACTs that need to be
+	 * copied to disk, so we perform all I/O while holding
+	 * TwoPhaseStateLock for simplicity. This prevents any new xacts
+	 * from preparing while this occurs, which shouldn't be a problem
+	 * since the presence of long-lived prepared xacts indicates the
+	 * transaction manager isn't active.
+	 *
+	 * It's also possible to move I/O out of the lock, but on
+	 * every error we should check whether somebody commited our
+	 * transaction in different backend. Let's leave this optimisation
+	 * for future, if somebody will spot that this place cause
+	 * bottleneck.
+	 *
+	 * Note that it isn't possible for there to be a GXACT with
+	 * a prepare_end_lsn set prior to the last checkpoint yet
+	 * is marked invalid, because of the efforts with delayChkpt.
+	 */
 	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
-
 	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
 	{
 		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
 		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 
 		if (gxact->valid &&
-			gxact->prepare_lsn <= redo_horizon)
-			xids[nxids++] = pgxact->xid;
-	}
-
-	LWLockRelease(TwoPhaseStateLock);
-
-	for (i = 0; i < nxids; i++)
-	{
-		TransactionId xid = xids[i];
-		int			fd;
-
-		TwoPhaseFilePath(path, xid);
-
-		fd = OpenTransientFile(path, O_RDWR | PG_BINARY, 0);
-		if (fd < 0)
+			!gxact->ondisk &&
+			gxact->prepare_end_lsn <= redo_horizon)
 		{
-			if (errno == ENOENT)
-			{
-				/* OK if gxact is no longer valid */
-				if (!TransactionIdIsPrepared(xid))
-					continue;
-				/* Restore errno in case it was changed */
-				errno = ENOENT;
-			}
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not open two-phase state file \"%s\": %m",
-							path)));
-		}
+			char	   *buf;
+			int 		len;
 
-		if (pg_fsync(fd) != 0)
-		{
-			CloseTransientFile(fd);
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not fsync two-phase state file \"%s\": %m",
-							path)));
+			XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, &len);
+			RecreateTwoPhaseFile(pgxact->xid, buf, len);
+			gxact->ondisk = true;
+			pfree(buf);
+			n++;
 		}
-
-		if (CloseTransientFile(fd) != 0)
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not close two-phase state file \"%s\": %m",
-							path)));
 	}
-
-	pfree(xids);
+	LWLockRelease(TwoPhaseStateLock);
 
 	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_DONE();
+
+	if (log_checkpoints)
+		ereport(LOG,
+				(errmsg("%u two-phase state files were written "
+						"for long-running prepared transactions",
+						n)));
 }
 
 /*
@@ -2029,17 +1941,11 @@ RecoverPreparedTransactions(void)
 
 			/*
 			 * Recreate its GXACT and dummy PGPROC
-			 *
-			 * Note: since we don't have the PREPARE record's WAL location at
-			 * hand, we leave prepare_lsn zeroes.  This means the GXACT will
-			 * be fsync'd on every future checkpoint.  We assume this
-			 * situation is infrequent enough that the performance cost is
-			 * negligible (especially since we know the state file has already
-			 * been fsynced).
 			 */
 			gxact = MarkAsPreparing(xid, hdr->gid,
 									hdr->prepared_at,
 									hdr->owner, hdr->database);
+			gxact->ondisk = true;
 			GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
 			MarkAsPrepared(gxact);
 
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index aa90503..c41baa0 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -321,8 +321,7 @@ static TimeLineID curFileTLI;
  * stored here.  The parallel leader advances its own copy, when necessary,
  * in WaitForParallelWorkersToFinish.
  */
-static XLogRecPtr ProcLastRecPtr = InvalidXLogRecPtr;
-
+XLogRecPtr	ProcLastRecPtr = InvalidXLogRecPtr;
 XLogRecPtr	XactLastRecEnd = InvalidXLogRecPtr;
 XLogRecPtr	XactLastCommitEnd = InvalidXLogRecPtr;
 
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 3de337a..ecd30ce 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -86,6 +86,7 @@ typedef enum
 	RECOVERY_TARGET_IMMEDIATE
 } RecoveryTargetType;
 
+extern XLogRecPtr ProcLastRecPtr;
 extern XLogRecPtr XactLastRecEnd;
 extern PGDLLIMPORT XLogRecPtr XactLastCommitEnd;
 
#11Simon Riggs
simon@2ndQuadrant.com
In reply to: Stas Kelvich (#9)
Re: Speedup twophase transactions

On 9 January 2016 at 12:26, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

I’ve updated patch and wrote description of thighs that happens
with 2PC state data in the beginning of the file. I think now this patch
is well documented,
but if somebody points me to places that probably requires more detailed
description I’m ready
to extend that.

Your comments say

"In case of crash replay will move data from xlog to files, if
that hasn't happened before."

but I don't see that in code. Can you show me where that happens?

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#12Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Simon Riggs (#10)
Re: Speedup twophase transactions

Thanks a lot for your edits, now that patch is much more cleaner.

Your comments say

"In case of crash replay will move data from xlog to files, if that hasn't happened before."

but I don't see that in code. Can you show me where that happens?

xact.c calls RecreateTwoPhaseFile in xact_redo() function (xact.c:5596)

On 09 Jan 2016, at 18:29, Simon Riggs <simon@2ndquadrant.com> wrote:

Hmm, I was just preparing this for commit.

Please have a look at my mild edits and extended comments.

One concern that come into my mind while reading updated
patch is about creating extra bool field in GlobalTransactionData structure. While this improves readability, it
also increases size of that structure and that size have impact on performance on systems with many cores
(say like 60-80). Probably one byte will not make measurable difference, but I think it is good idea to keep
GXact as small as possible. As far as I understand the same logic was behind split of
PGPROC to PGPROC+PGXACT in 9.2 (comment in proc.h:166)

Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#13Simon Riggs
simon@2ndQuadrant.com
In reply to: Stas Kelvich (#12)
Re: Speedup twophase transactions

On 9 January 2016 at 20:28, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Thanks a lot for your edits, now that patch is much more cleaner.

Your comments say

"In case of crash replay will move data from xlog to files, if that

hasn't happened before."

but I don't see that in code. Can you show me where that happens?

xact.c calls RecreateTwoPhaseFile in xact_redo() function (xact.c:5596)

So we've only optimized half the usage? We're still going to cause
replication delays.

Sounds like we should be fixing both.

We can either

1) Skip fsyncing the RecreateTwoPhaseFile and then fsync during
restartpoints

2) Copy the contents to shmem and then write them at restartpoint as we do
for checkpoint
(preferred)

On 09 Jan 2016, at 18:29, Simon Riggs <simon@2ndquadrant.com> wrote:

Hmm, I was just preparing this for commit.

Please have a look at my mild edits and extended comments.

One concern that come into my mind while reading updated
patch is about creating extra bool field in GlobalTransactionData
structure. While this improves readability, it
also increases size of that structure and that size have impact on
performance on systems with many cores
(say like 60-80). Probably one byte will not make measurable difference,
but I think it is good idea to keep
GXact as small as possible. As far as I understand the same logic was
behind split of
PGPROC to PGPROC+PGXACT in 9.2 (comment in proc.h:166)

I think padding will negate the effects of the additional bool.

If we want to reduce the size of the array GIDSIZE is currently 200, but XA
says maximum 128 bytes.

Anybody know why that is set to 200?

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#14Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Simon Riggs (#13)
Re: Speedup twophase transactions

On 10 Jan 2016, at 12:15, Simon Riggs <simon@2ndquadrant.com> wrote:

So we've only optimized half the usage? We're still going to cause replication delays.

Yes, replica will go through old procedures of moving data to and from file.

We can either

1) Skip fsyncing the RecreateTwoPhaseFile and then fsync during restartpoints

From what i’ve seen with old 2pc code main performance bottleneck was caused by frequent creating of files. So better to avoid files if possible.

2) Copy the contents to shmem and then write them at restartpoint as we do for checkpoint
(preferred)

Problem with shared memory is that we can’t really predict size of state data, and anyway it isn’t faster then reading data from WAL
(I have tested that while preparing original patch).

We can just apply the same logic on replica that on master: do not do anything special on prepare, and just read that data from WAL.
If checkpoint occurs during recovery/replay probably existing code will handle moving data to files.

I will update patch to address this issue.

I think padding will negate the effects of the additional bool.

If we want to reduce the size of the array GIDSIZE is currently 200, but XA says maximum 128 bytes.

Anybody know why that is set to 200?

Good catch about GID size.

If we talk about further optimisations i see two ways:

1) Optimising access to GXACT. Here we can try to shrink it; introduce more granular locks,
e.g. move GIDs out of GXACT and lock GIDs array only once while checking new GID uniqueness; try to lock only part of GXACT by hash; etc.

2) Be optimistic about consequent COMMIT PREPARED. In normal workload next command after PREPARE will be COMMIT/ROLLBACK, so we can save
transaction context and release it only if next command isn’t our designated COMMIT/ROLLBACK. But that is a big amount of work and requires
changes to whole transaction pipeline in postgres.

Anyway I suggest that we should consider that as a separate task.

---
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#15Jesper Pedersen
jesper.pedersen@redhat.com
In reply to: Simon Riggs (#10)
2 attachment(s)
Re: Speedup twophase transactions

On 01/09/2016 10:29 AM, Simon Riggs wrote:

On 9 January 2016 at 12:26, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

I’ve updated patch and wrote description of thighs that happens
with 2PC state data in the beginning of the file. I think now this patch
is well documented,
but if somebody points me to places that probably requires more detailed
description I’m ready
to extend that.

Hmm, I was just preparing this for commit.

Please have a look at my mild edits and extended comments.

I have done a run with the patch and it looks really great.

Attached is the TPS graph - with a 1pc run too - and the perf profile as
a flame graph (28C/56T w/ 256Gb mem, 2 x RAID10 SSD).

Maybe

+static void
+XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)

to

+static void
+ReadTwoPhaseDataFromXlog(XLogRecPtr lsn, char **buf, int *len)

Best regards,
Jesper

Attachments:

xa.pngimage/png; name=xa.pngDownload
xa.svg.gzapplication/gzip; name=xa.svg.gzDownload
����Vxa.svg�]{s�F��[��eCZ��C��rd;��l��|��zI�@�@�����u�@��H��(1��&�3���~�Py��j���Q��H��J$/�d�iBo�$��|q���W�������$��w���7oo�t9���^�E�~��h�6��V"��(�W��r�T���f���Y0�Fa>���L�i��I�g���Id��)�]WA�)�&��FrU]"i���F�������/���.�JW7�JT�C���%�M�@@����jG�}��y���E)?��$o�`F�W�X���<_q�KP��
����D9�4���b���l0�(��L2:��$2N3������,��l�1g"����-�I�*��c�+��d�qN^��=�i��QD��D����d�"Id�j��X��Jg����y��	���8�n�?Q�l�x�S��s����
�W?4�(3*C��))�sP����A����2^$�?�����70��{z5�a9���%��+U��I���4���QR���|ln�f���3��,�O�V�}�����gAFF��8�I��a�4�pJG����0���P<�Ri&	�������p1��*Z��)�����QO$R_GY^�N�xt�c+��Fs"�2�m��7BP�!h|���W����[P6�����~�������O��g�����^��A���qZ%�%��$gH�O�%��40��/Rw��\o@S��A��@&��]�\0���I�c�C���L��b#vXd��&2_���2JF�R	F����7Q^��f=����E"��.	W2��*�������i&���)r��&�3i�8j��>��<������1��<Kgs�����%/��k�At?D���p��	l 8���83�hF�N��'��d9rA4zH���$z^�+1M&��:���������J�5�����,��"�P��
�CAR:��/I}4D����������:�@AL��iM�<x�=�f���=���|F.]0^[�U�����7{����{���N�W���.�e������m�{��=����~L�m����������%lK�1z��K��jZK����i�9n���|�^8���?�������>�c�@�E3����o���Dy6���,�#��"���ef�k�hX��- ��%����(zYc�X������QM�]dp�B�oP����?�>>�g ��
V+i�%�����r�d9g.�n�1D�b$�#�I�B�2M�(@��u�qF���<'�3M�������1Bb�s���8�m:@��K�E�8a���1���O��
��z0���������L�a���8�p�}t�o�Qw���
��~MV/���./k.�1&�f���yr��2%_s6��t")
�&��S6CI�YM����QpT�����-/��pT�
<�5�r�-C��v��Zj�d�����UF,�	Y�E�O������DV2�0�?~��>l�����*l�vWV����#T`�d�ij�<���h*�:��Z���<���0�/ ����r�R��S:��*l��lM����B�����q�P�@�.5�T��T����m���%t�����h(1�R9��2�4���o����ub+�y���!"����a.I�L�5�0��m�1m�oI��EI���2X5)����"[?2�zgB�E��<�M�K���8�M���������>`=Vi�w���|�lI�����8Z^5�Dm4AQUU51%�X$�����q-�(��AJ�A�Nx����N�����[����9���%6���(]��<�n�������s�����~��	������4+�]ERK�f0�y�k{��z�L��i�"s�m����R1�D~!:
��Xa.�4�k�xA�Li@�f�,X^��<��"1:k:��rT��q�Q�>��7�'(��#����Ijf����r-I���p�!'@�%��	���t���I�4��(���,�g�z����g�H��f��*&%h3Hf���0�G{D������	x��8���i���QX��P�h�
���N��a����Q�mq�zR����_\�bg<�/�]�l
�?�ay�j7��V�Tm��Z��%�1���q�Rwo�z`Ub�Y��S��A6Cw������g�I��^F't5�"�"��uh��#����	3��T�� ��ec@���
_�+��9�&�����wZk�>�yb�7dm]$��m`$�t���s���Y��v�/{��@����t�z5�o���s�r1�w��v�joWe��65�=y\�vC�n��A�W�)���O$���H��u�i%�6S����=���5�
���R>~�G_dH�2�s:���c�TB��������iF��N���ENG��o(��.43�]��)��o:fbQ�M�;(py�����xM��!�$����)$K!V�3XY	�v�x�)�*�����j�e�)���U�Ea�S�k�1�]�q'��fu@���V!i������h����������������/q�Q��i6���T{.>Xm7~E���G(����%e59���2�
����	�x�6�H>�<(w��N}Ws���������,���>�{������^	�8\��,��,�M�I�0^�P\Xt%[~��o
��)J�����*�Z���NrO���^fY��mb�=Z��V��'�4��[&���,���>�k�L�/�)<){i���`>E,�[8�5>�P�!�`G�|@���q�2$�P��Y<>�p�^��2�����P+;�5�I�O�~;�xP���#�)?���/z�w��|PWN�����p�R�)��a4����0-�tv���s�g��N�2ZS��h������h��i�^��[0�����B����@�I/�����F��m�J=h�mF(kAj�����7���DD����&�\�.����������R7W����N?m�G$����8?�g�v8gESU�LX��r�����<jS>D������
?�"}�������Jc��I��1NsA��y�{;�A�.;pG���)��v�z���{���D��}��w)�U��x5o��������y�?b���Y�����H_"xm�9; ��<��f�hS	Y�8���0��K<�V��8fQ���(H���5j*�y���}>@&�+����V��~<+�S^��t<���u�)
O"j��A����`{o������{, (Z���%�~+��%����@#)2��o^-��
�A����Z�e���ryt���_�����NL+
n�z`��hb6��iq�#v���/4��_�cz�7C��!(R
]�U���%Uq��� 
WY�n����l�����*���5kc�,T]��e��2��X�?�-���[�bk���(�|��
������]�|��!��S���j-(��|m�W��^�q
W�UM��.(��E�{�(>�s?����L���:������%�Q���2�2���d��5��.zn���a=u���l��^�pQ��]@���<G6���[�m���h���3*Tm]q���k�f��iv�N�RL���U�W���(��h�a�s������y0���yM�o�X�p�ckk5�7�A�����F'��LUq�mX���Uj��$kPRY���0:�Y��Z�#X���F� ��v���!���<����G�
H�S�c�L���h��>�Nv
D8����'�B�l���epO}��{�l�M0�6�i�5����QP/yv��l�se���6-]�8����m��t�fkv�kU�Bx/�WB���So����V_��K�<����N���l�qN�W~O'x=<{�L�����i�f�2��aH-�-o+�����Y�lt2w�2�����4�yA��	|� dj����
�aGj'������-�V,���#9�������=�0����Z�r�:��d�R��#02�[��<����]${��(�]V�
�)��!��
�S��<�6N����P���L��"�� �n���Nz��
�T*�F�~�i���/�h�P�9�c��m�Z��t���Up�������O"��O>U�������<g��+��UGRoCR�l����R�,�q_���&[z'_��P�N��3� U�
�Eq��e]oZ����&�������aX�4
��
�����R�<��Y)�/��4 �S��
MSc���cd����������"�k���0e����C`����'$��?�TS�ZR;@��~w��tVD3:J�0�t�q�"����a9���{+s��M�S��Y�b�L���^����XC+����tJ�YC����j�������`�S���4����2�k������_i����&dz:�J������M��gt��~�����&��-8�|�k�Zy<���Z������b��x$��o���}E(���0����1�M����'����<�����l�i��5Kq����n����9�e�V�UP7N%C��}�5��={�!4��{�4�c/�S�bTO���A2��?��V�������������SZ{
[7N��e�
��4*���u~�S��y��������{ �w
T�cV��VY/���?���L�p��G�x���E���a�fi;n�(�d�"������c�bL��?�s��T�+O�c{�*�����[����*�$�'���>>��F���Y�����j������9_j�Gd�?03�n���G4���� J������?=�����W
����

<-���;��h��������^�P�_$A�Y����u��7l�;��dI�-���m�������|��=
F�]LK��F_]k�qLW��F��7e�SqO�;A�
�J8����=������Ai'�c�8V!�Z�����uH��Gt<�}�D���<�����;Zk�H����J��U�6�n�6�*/���J�q�?Y����l�.h�QS!|;�"�"��-
`�)�~*9�����o���f�U������Fo1����s`u���P
�^�3�=��;$����/��Gh���^g���tet��.�A��V������|�����G�F�~u������y�U��������)����x���g�~����:|����&���N8��f�������%���q*�<L!-��]a�s��"O�|5�M��NM#�2�u*�f��~��5�Ji��AG�;�H�u����U�T�������[O�����w���w"�C:����8�����
�vl{Cs��;�5�~�'�1�A~�g?�� �Q�8\Y�..��(~oJ����-��NP�`����
���}S��sI�*���%U����V[-�8�,������E�T	�$/�7E(�����9�������W�t@V�d�S��@�����{O>�Vvz��������})���Hs-�5���xS��5A\W��S9n��`���(�	VQ�s�~�"���am�4]�����^G^U�[s	rP�M�8|c��.
��D�B�:�[�Uji���@"���S_��%r�8{�g�x����>��o���oi�S����dni��	�>����?���In�X�=��5����^K���X�*>�<���j��f�I�>h��ga���T��T��X��	t���E�tk/�
�{1"��K��.�]�K�t!�-��lB�f���(���$����i��D�Z�C�2&����o���`	�<���4�x�@�+��*��{�����l.+��*!hAY��41�i�4�*���P��$�J L�������F����V��8/��2���OPL3�d
5n;2����^�0E�JP�t��;L�C�&(����F*����� j��
_J�@�B�	��^Z�/�����RCWD����+���?c=�w%�B����p�8)�������?u�~��j��=�g�t'��uQL.�.�@�����/��}'��w�(���]W�L��e�k���O��J��;��4r�3�p�t���4��X��T�9�,��)�cl'T.U���������&E�F�zz���D����PX����mR�a������zj���\{��T:�Z2�Dd�	��z�5��+ar�u��k�5�������O���wJ��	 �&XR��Ihe���f0#����T���&�����~:�^��]�lEIt��Q�F
��7��G���6|wl�j��|������#u���l�9�5b8�	d��f�+9/�� �D���)�.������������Js��DgzmCg:�<���L�j/��p-�a�������N&�eAd�M�0�{$W�Q�A�}7��i���nWW���������J�����f/��]�;I�Z��|�m���_K����T7�m���v!l6����s���y��4QJ��T���44�]W�O��k�I�g�
�9��'��%r�����\���c�~k�����0&i���������8k
�
����y�N�f�[|��,�����O3��b6
~�\7��z�q:��o�L����O�0�%Q�QEvC�&o�h�u��PSm��IA��x�|>���K����t��(�;E)�gV�.z���n�v��/�}�Jv3����c���%b�V���w�������T���t��k1��q5�!h*s���ybzeOaWt����!/c���
v]AQ�;���qb���]{jrS���|6�Ho������:M��gV��5��v��t����qryE��C�fh�
Y*���b|����t���Cc��b���EF�cd��?o�������������H���4 /7��|���k���O���/Z+�C�6j�47���e���I6�E�>�0�6�m
wc���%�������\����~��\
ysXre��`P~�����}�
��J1�g@�8=Bh������.��&�4�9���������12��n���\�o~�(4�1Q���HBo�3W��������\��tN%RD��Y<�?��������K9��U�>��c����0��i��a�����Q*�EL<�]u8KfI%���X���M5Gr��O��ng�D����;8��x��b�I�,�x��(����
�A+�pl�om�t��_\GV��:�c��x$����]�<c5S#�8�{��	�B��q$$�D���
�K��������M&��8[�d2�]
�z����O2!�.��M�����>t�����������yP�,�,wW��Qfs��G��/�omW��u���U����K]3����2,�aj�� �����:� Y�o�?I�|��?��U�:��&���2�n.ve�I�}�F�����Zt�����8c�|���P�g7��� =@�x���}f�3Gs+U��g,D)_��&X��c��Z�������������?`�r�s�9���v�Z5\wj/=�f)���E��5����oC���y_�T"�6D�d�i��,���4C�	�h���wL�zw�,�hQ���s+*6?��"^�J�J�\�WO����w�����G�3����y�G�)oD�����b�d.|�_�p>�����=������xil:����]����X/Yn{&�:�Xw?�����S��a��gt���bo�N�cH��^7��<!!!�������_�S��X�����C�%.���,���H���"�F� �VS��*��A�X�\�es�C����4h'$��/u���K��_
�Vr�7��>��zwn���Mf{	A�^[�Z4��rH1`XS/��L����
�T^�t���\e����K�F�^����.����cD8������s��!�[�Ka����i����@A�T��9x�6�������������At	i[����Zs{+D�4���l�m�7�a�~�5}�7I[��w ,�'��B�\	B�+E138�8�
AUr��T����Y���6�D>��e#��k���r%����s%8x	MNX1�w�~������eS��T�D[��5��,��������R�H.+vh5�����+�G?��\E���B�����w�,]"�2h����6M7Evl��;l���A�����a��@|96������%2�}k��V��qRp������V� �������l�����Nvj*����:�.O�zf8����92�
f�O����xZ��}�4`
�F~�	��e��m��&(�e����gv#�J
I<�O�2���C$T���q���/�I/���tI9�<��q�
�`�����,���nk��L��	�@U�F��b�]���B.�����X.�h.�E�G���7e�=��&��v����$����zt�t!Q�;�<���Mo]��o�e,
{���"a�o��FJ�f�\�Q'H7}�)��q�u��1�\�a�|uX7@�2�I���%&���
�� iC"qd�Fo��'De
y����Q���5[J�!$Rj)�J����m���A��F����0�J�^Nkh(J%^������
�F������������|��}��w�Gjd\�8?aVhm�#���Pt�\
>T~�v#���K����i�c��n#A�������s��m��i���
���9���fH�m�>���`�3�\��r��~����<���F���:����0��Q���8E�=-��
3�3)����z�S�
=X���&%XF��C�t�E���.�U!%��d���F�����#8�q�v�e:��Y
��FL�~	u��
���7�,��3�COE2:	0qB"J����]�gI���\�Q�	�,Y��f�"����E���hw����1-v�X��s��� ��\9�Z#�CJ���g6�A�������O��F����g4��JB!"�+���xWN�\�Pw�m.��o��%�����rs���Azm^G��!B�\C��Oa�Jdc<5R��"���H4�
�o� � 7�����l��W	�Su���P|���X��v5�YL�������9|��@M��?�����]bA�S�����n�y������M1��{�����K����d�i�X����xThd��#p��qe�%lZ.d%�n3fA���8�X��~��.`�?*�e[���1���#t������n>)�q�g���K�Z���#H��$h2�$�����9;�M23���R���<`Dp�e���I�t��'�G�2V%��4RPC����4�������P�X���Qb��b����n�p�M������1fM�4�]<��,v�p���8���Y��b�h_���M�U_�mW6���J��th���N$����\H�a+������O�v�+7���ko)��tk#i�7/���l��hb��W��R����9���4���`��{X.�7��738���/��)cU\�`q�\uA}&��po3�
��M
��cb%i���"UK���B%�e0+��}:�On���l���a��s
f�\,H!:*QK��@��R2�������6H(�u4i?3JY�{�-�\�	�G��8���q���E����
z��6��L��7�alV���0��c�U]�=\P`�s���K�y����}�
�������� t�}e���w�pz��+�2��Tcu��u�`M��z�lp���C�[]�>����W��}���o���n��IO��A"�l:����4l8v����mECQ}����q�v��� |���i��������������I�"�.���q�����D�u�#��L�_x��'x[�"��r��m�,���������Ym��Zq
�����z8�������Z{���p/P�g`&U����l��/J��y<@L��i4���6/G
���+�M���zw�����s�V�*�T�c��2B�1)�8o����6�0�?8���S����z��\�<'��������m=(��/O��yZ����&*��)���N5�oa�y��"���"��h �h:,e�N>���,'�-8��OgPX���d��S���S�b�p�\��0��������Y��9���#d���%>��3���u0�UH�3*V��"Yw���m���8e���N"���$.��o��r�=%{����fg����,�5<�
��F���B��Q#?n� �)}9��YR�V��z���2eD�T���O
o��.����#(�������VH��b����Z
���UP���`���6G��.��KczXg���\��)rx��v=�!��{������-x�=<I�J����%ac
����&w�~>r����] _�6�z ���������8M=�zQ^p;y0y��EPy�:MFH!�tw�{�|�-H��4�cC!I���s��{� 'f��`cQ8�J~U�
�������`MD���z/����9�g�[�	8B�����x���nH�1!������o�����B_U�X��������fc�TmM�J�\J��t�� W�����p��������9B�Lenx��R����y_��
��Pk�DD�cl��J��)��cC6���l��ap=w���� &���\"���t�K��)�H.��^���~{��W�!�\���t���������
C�^`�@��Y�q#b�$�x�Gw�{Y�y[��n�q=0=ya�d�
����(��I�xjp�;5���\j���Qo�����8�x���[�dyr"2�:����S��0���'���L� "���QR��eu�����!��|U��w�n&���L�@[@�f�8�h"W��r�O���[�=��^%���68q.���Tk��H�Z$r�����j����T*-��T��y��T��AUN�|}l����]��]i���fe�5�i�bn�TD#h��3�La��������8����g.�rG��d�#���1���m��Z��������B�}r�	�3��L�x�������e+!��`j��~C*5GR��O�����}�KJ��,)C��Z'��`i���-�A��r�N(���l���q�d���d���p��E�#w~��������k���C	����j�JZH=m�b0�Q:��8�����38�gMB4ut�x�KB�T��/'��������8S
w�z`>�����}a���qd�l�Kr�����P�m���������r��3�c���C���! aO7�����g�Q��[��M r��?p�x(��y�����)����W%Gh���
�8�U��|2���c
,���4\�N�����
�yuq�^M�f��Y��n�
�.���}jE3[��5����t�H�}����;W;0�������~{hkX�� +�<@�	�4S�c�d��$��S4-d}4u�K1a3D�b���aA����>�W�������kE��a�}_.����.���	5���3��PDN�*AC�P�0�s�8���;��=�g�������U������G0V^����r^;�Ec�a����K�(N|	�h����!%�E�=l;���w�<���b3�V"N��C,f�-��O�����u����^}�!�_�pJ�K�;���;���+#�X/���k������6���DHV���f}l��O�;��������%.��~�T��]9�u��;b�h���&�;�M��]��*�@�3�����+�����Qa2
l��%�)�%�w.KX�����"L
��4P���\j�_>�����!���D��������1����!�;r�o?���z��'���y<���o���x��X^��Z����:x�������}���Vu��*V�K5�R1�YkE`�]�S�y3i�2pd��V�����J`]�����Ti�I�F?�^���H���Cec����N�������%��^���������
[P���q<9]�����S�C��M������u0=f�i��tC�e������� ���	V&Mm4�I/O4pW�`��}r������=�vs_w��6u��6�N� WqD1�������`Vz���������� �Pr�1�q��Y��y8���c��8����������B��~�F�LV�]�=�|K�%��� �
1���G�DR��s	2�n,�wR���y�R�"4�W�*klI�2%������u�p�D��YH�0bfp��[8�_�3���y�s%w���X�a2d�^���y�,,�(�����~R����(��-!���E��T'������v�6�����<��_�V��]�C��FP�Tw��f�%M��oD�]�N3� T0�m���ifF��s�gr�g0�1�����]|��d�Z�zC�N���0YQ
,���k��
�M��P�~�|���l��E�&�bV�m1�{�<������Oc�j��\KB��g�(�e��*�I�6i���U[^)�9<��b���>��)/�q���8&Z�����@f�6A�E+��Y11gn�!�����(������dk�����ZF�����R������F
�("A���8�:#hq�RD��|�����s��}+��
Z�B���r�J�R���EDH�Go��TI�3��]@vQ���pc9���7>��FD*D<V�$��C�~�c=���y���:�y���Ep^'Fh��2�Ji��.��7���C��u���6������/���U��i�T�q/@��d�������Zi�������i��E�W�8�~2>M)SP�"����4<���F��r�'���Zm��#N�
��������V�]{��pN&	�^yj�e*A��,�=|��X�-���������/�p�6a3W1k��{����9$�.�%�����|tD}���W3E���&����I��Z���v��^�#m�nPvV�e�iJ���B.��%���j4�e|92[d��~��6�	���a<��4^z+'U��#�	i��<4���B��-#V?�6�"i�|�U�.����'q�Sk"�Y4+��C�a��������1g��C?6C���������������'��$Tk�$�/�=�A'O���5}C���4}COF4~
|�a�<D����:!�K�S�"h��n8�e����x6�/F�O��E�&|�x�41�$p��=*>=�#���,�q
����`y��(Q�*���(@|������}s��0p��J�c/�JA����4w7]�Y����R00����kJ���� iglU,Is;����y�\��,���Rl*pR���s������qx�u���*�%��S ��l3-)._��/Q�X�Yt?4���{��_�O�P��<'�'!z5�!f��\8�b�Q}����o��=\�0bc�����Y�����jn���H�[8K�1�hI�K���c;v����?	2Vw�\F6���Mt�.�����|��0t�J��[��	�jS�����[����i`�*��{4<�&��M
�:���1�Q��&����?�_��wQZ��^1�_;e��)K�����X:�bl}cQ(:M?�p���]��}y��b��=S/^s���$-4t_���-��=�5�W�9kD<���vi�WI�q
G��`</��~��j���6k�U��WS��/MU	�F#C�f�j�`�O�#�m85�<Q���sI������a��/v�R��r
_�5��.S��\Js��)����k����Q2�/7h�e�]%FdA��k�'�����g����K���B���c��)��� ��H>�"�fl��X�Q�"W�m�I���;_"aSO���w�������+?E��e�L^z��,��7m���r�,��$����S����\>R�WA'DGZ���o�������W�����������<N�e���^�XC��	%}dy���W��?cmh4:WI����qYqakAK�����L���X�n�
�"R>�^�������
M�i7������{���G�w����}�mp�a����\������A�_"et:!,��L�Q�����OG�����J'���=����#�l�t�x�CMI����F���{s`��k��L��
����#�X��������76���I��|��H��!��P�c������>�h�D�T1�N��W��7�x��������F�Q}��Kq_�}����M��H{&�]�6��T]:n��q2��>v�}[�������O�v���E�E����>��=q�+����O���0����P7����2Z+������\,�jtC!�����H�;sh�6{l<�J�.%	Wbu�����&���M�>.���
I gI�s�e�2���F�I�J�:�O4�����rya��<�7��6�8n�������l|_�������Fv���^�d� ���end��kM��B)�Vq�����|��������P�G�;�r+���,7cq��q�d��5S�����Z��|+�KdFP������$r�$��x�O����$��t�1��4%y'�1���;�������/h�`�7G6�m�t��t�
��''z�PP��]�v8��(��E����.�4B�9�{G����|t.-�	
�?<w�����
�O���sq�)+����v�����������Xe�|�_�����<�WiG�fw�d)���<����+nTT�����<�'��m��+^��Y���/����\�M����(�D����� !���>p�x�}>?LY���.6Fg�H��d��s���w����ss�#�t����3�x�^���e�����n<>~�ZTT�����r}t�
�$Z����Rnbb�����n�y��������	�p��a��;_���c���y�(����z�&���S�e��QMeD��
F��X��N\I9�2)�9g�r~�w���C�X)���&��W�����h&�n|�d�N=4�(�������m�tcRz� �Ug�����p|*�����t���Y��w�X�o����E���!�u�<-������|t�9�n7/�������~{����+����2�V����h�A�z�n�~'-�a\T�q7��<�LOZG�w�p��U��~��Q,9�::\�����	�����@�>}2�
�>g:*���������9/������Z������t`�w�5��%}M`�v�+���fKw�e�e����'V���������?��������qj�)[�|!B��y�f5��*IS�"�x$���_�����C�������$xA%Y��Y��4��&b9���5�������9+*Kuq�p�����|h�
����"R�3E��DFe�%�'j+&���+;n'6X���
�L��n���m%CQ��s�]w>�0����j&Onwt�}���K���w���#�K��1C����M������? �U��T��#)U�<��E����0 ��K9����f[���P#���{��zw�����*+5�����"a�+�X-	���D�3SVX0��4��
Z�<���H���G(b��j��~5?����GWRU�9&��l��0KZ�����
���.����Vp�k��6p���=��De�o�Lm�N-���!^��T����w-��������A��5�.�Kq'SZU�'���2����\���C���8�_���~U����������8�t���9_&a��f�Zh?y)�������a�k[���:����� �J|������e"V�k�|��S��9�`,����mEZ�9���,e�q1��]E�$M��NqZ�)y�9�����R����n���:�X���"D#�PD�L=�X%��e��'�z��X�7�W,��_z�,���c9Q*�������0j(20�����O�s���|E�u�c�Z���F��d(nS���u�����1D.�Y,J��X�����U�'*���#e]��������]p�gW
PR��������>	���P���.&M��7G���f[���F|]���3;��|LS�#�q���>��q����=�O�q{z�\+�QQ����\Bw�1���;^�=�YW,���|�AZ,>����k�w6��
h�����m���?�n�����\?<v���r����&q�4�=�u��i������?�k��f�$4��X|Au��i���q*���-L����
��8dn���
����;f�j�
Lq�v������m=�e����K���q�5����4�I/��'������<��*�����mH�e����f�I����w�������y�IkK-+Hv~\�*5O����fh����V�(����4����{3��9�TO����/g�$��3O����$��c�����rx�T��'I�Y�m�dh/�G�P1�G���}j�?���[��-�M����'}"gz�i��V�)]
$����V3��8��^>�QN������i:���e�t��( .=���o��qh��4�fI�����VT����>NI�O���'4����/P��Q���wN����u�����:��_A{I]�T0B\7�_!%����&O��$LJ���������H�7]1�fq�G�1��C��<{�K;"Os��j_P���W,�e0����8d����u��?��S�52L�#6j��yH�3D�m�A*�0H y�6?wm,�����{I�"�{#����*��>��1+�����������Uh���.�o��JP@��X1Y����>������� ����oM����vo��n|���3�0���6��`m�i9��Y����9������M��2��8���}tI�1����G�-��K��3��@����XE����e�������o?<�7�����wpV�����`����`+|�K.`�?���C��3:]��������ph�"})5�k��������i���?�C�~�f��|�5>A��l�xH
$B�� �v0����n���{J���!q$������#�B��8 �R��k`��i6�y2k	������L�j�5}A�5)�q�\��G�ln�'����=���~�?�a���f�B�#_��c�
����8���eq��E�Y�uv��'�.a��;��-���,>by�������T��&K�>s��+���;M����s((������R�2����Y��MO�����
�%�"'+�h�Y
���n�b{j��l��
6_y����j[|h��!R�)�����������K!Y�-:�v��s��� �� ���m=t-��O��NHaq�a�"�hd�����lD��|=�RXZn���E�RUH-�	����Z��m����R��Q��+�"-�j�3	���[��(�]xD��M�}��6x�V�]����M���93{��[	B��v���`��j�<�U����m��
���Q#8go-���r�f�g�G?��[�2Oq�N�"dV$*�)+h�B)��
�u��
Y]�B���v���
._1�[��"_�,m����88HQ�A�������l�����/���iT"�R�e�R[�kL�2������������$}�-������	RuC9��6�*�.P�1 9�yp��.��y�b;Ex�h<,y%�`1��� �D����;�>���~h;�z�s�HZ*p��zYy�0$��)f4�"4������/���T�R�*�d�����']���_N`���+M�N���V21����!�"�����]�d5��u'�����;aK�6��C�k��5&�}n����p��lE���$"gjfL�?��Dd]
����gT��,*�1�;�>�������~����{�d@7<�S����d}ri��&��*��������HE1���s)��6�I�R�
��6����v�M��E����^MC�`!�/-xD�q})4�W^)��AV$�S�n�8��t�����,�;���m��S�������z_R������\u��~�?����*����ej�e�i_�{g>P^iM������0
|\e��}�����/�b��}�
�0&J)�I�gU�R���@���w���SU��qF���1��>m=JX�
if#�����%me,Q��w>E~�z��N������4����k�aJ�+�o8�"i��8�C��D%��rx�])'�����7&u<Q��/��[<�5!c|9�(�z�-.���~�x��0��9wI?r�Eb���G��#P�i�&n�m19��Ll��)]���+���/'�]t�*� ����4<����&��+�E
#�/DM{��9M����� O��n3'��$|:��/OF'��j���~g\;�~M"�0��n��D����f8G����R�i���3��.�`���'��4R�R����;�����B;h�4���Go��
��s0�`��,@�_����f��H9Y���!�v�z+Il.���m�������e?�rc^�I�e���TQ�a+E�� �p���D�&"�R�,����(Be�e���$����|m)�,��	�t:b��P�E4s��I�����\���n��G�g�O��G���co�3�4��x�S�t�����ZQ�H��p�������,W�W�������&�Kp&��~��=���p���������9uDU��
`S]�#�i���_��P.�H[5t��$����c����=����1�p+��\>���Z�(�^s�nU�$0��?S�]��f�7��l�a��`6����tt���=�u�.c�*X�*Z0f��o"$�����+�b�D��-�����h'>�� %�D]
�����l���A;�M�6�<���`���i���XE����(���&��<�p�X��\P�lN9�21^j�r</��#� �<����G��o���e��C�y���������g�RuV����0QXWJ���s�55��P�QL1�I�Z����
]R��K��8mh�,j^
���a�!ql���Y���3�HK�f;������Oms������Zz�6��_������qV3�
��<��T5�����D�F��I&�X�ff�F����T2#���1<�8BV�JW"�5����v�a�2��!:?��?�>=�O8���k�I�������u�-�������MW�A�u9K�I1��b�oL�^M����J#�����C���N��.H�-��H�M�*��A�-'�r�������o�?\��DAUf05t�:��c)������)e���N�~��������C�����Wv-t�2?�qF����H�;$�BSlP�^=��EZ����O�9>�2*�	�r}e�����o��}�w=6���i����`��Vg�l>;j��
��QM&#6$��dN�/�x9C���	����+�g�R�hK��ep��X�v���MF������t��i�p!H�C��]�R�zX��
�����l������@~�*����f]�����+Pr�����4��d`�ef�Vr�x������h�0��Uft����a��*��u���+Y�y~���G��C�;7=���Z�^�-Q�k��NZx����(��f���}��K�p�	�Kk]8
�f~�
e�4F�QF�����U���m�w���r�����1!d����������������h.�qpc�������5&%"O�r���[)w^h`pTF�b�]���u����H�S��X��0��S�
^U��!K�����DFn��LT*�aQ��
�^�\L[g�J>�T����9z����6�K��W���)�-.�������o�Q'�W��u���Y����~��2A��������/@_�����|�e��#�����`��t�r�q����4�����_/m�\���u��j������,�
��^��S>������e��?�����')�>
���tSi����~M����`]� �75���_���#S�������dMF8fit�o����RI�a����5���o�s���\G���=����������?�&�L+-+�[�3DG��@Uw�f���4*��PS���`�^�	�$�l6c��J���� �&.l���"��������;�N��0�
�����-�!R�F�F[g)�OP�
�)F��a��������$C����l*�R�o]?�;I���B����Vi��	38E]�x�	|����q&��F����zuw��Y���e���G���Cbvz�����:�kZRQ��@�E��KH��-��
�(e�6�|�C��G����
������ ��d�P����� ����j���i�������M��U���*(��
gEg� ���s�r���tu�,jCC�}�5E�o����&�;�BCO���06�/u7�S=�m��Jc������46s�
������������,�n���M���6�S��z�O����k`X7�19m���S��m��;�{�$�X��$�F�2��eU�k3Q�_�s�_�-E�6x�_�@&A�o���^}� ��$�rj9�����wf#�u���~��Rt�T,�$�\���n�#Y�L�/���p�������
��N�p������
<wq�@7i����3!���q��O�q|�~j{��}{���o���5�P��1W��@~�V����2y�6��@��4}���@���|9��$Xes-D��5�����6� �GE������Gb+��>(�.E�qj����u+e4@=aFE5�P���f�H)&����Gwp���I�J����[�-�^� �G
��'�p'XZ,�w�F��Y���sG �@(�+��� w���M��\����O
�n�D��zY��
���p:����
I��!'T���Ig����I���c���.i89)w��59[�MSW�2��Y(����v��v�����M�l2�M��r�}������u�94s��R�o��� �~�����^�%��|�0��oZ���P���IrGI)�s��K������2m�^J�����C�j1ugx:����E���Ss|�ozmN�aQ��&��J�����$]u�z�n.0�+��?������|�'����:��
����O���>t��Gt���d���XD��R�&��K%wL�2?��	���
%���!���}-�m����`
O�sv�/��<��9U������[R����L�CTq��r���p�^k*AF���E���G��e9�P�)��vL�^����|(O��Z.������}��HD��{.����|P����A�~>
�������V�^u�,�I�~m*e����)�T�,�Cv�����9_������8��E�`�EW���O���?I��s�F��	r�u���d�O�2����	RJ�y�x�+�\�v��g�kB��t���H�8����>u���q4�i�+U*�~K���WU`�;lA���i�w���k�oh��QY�V�M*����H���`SZ�����9�������/_l�(��Xl:��bGu)m�������� 3"+=MYs��I8:pq;*����&�>����L�M�=�n��(i(�CF�%�����+��o��<��:�v*rp���$�� �
�
�Z����o��9��R����|�u=�?&�7����0���M���B�5aA�V�~���*)�t�����iq���r^����*.{�������_�_�������Me�Qb'�����U��(p���zEa*�+|��i��f�SL$��MI���j^�F(���2LLm�LP�96��Z���R�N8�Y+M32����al�H��5n2���,/����>T��cQ�*�)���?��yE��HEP�|V�}���r�de�n�X�1�
�~�@g�bi�w��j�U��>o������a���P���d���S5����^y��(�++6F�H����N~h��>t�S|��PR�p;�;���lM��-`�<4�s���"yk���@��&A�2D+#!9%v���A���!](G2
&Y�D~g��
R�'�t�h)3z]��q�����:��\�z���90���P��{w���gK���;x�b9��VZ#E��h 
'�M��h
��3O����]P\�"��
�g����������h<'��Z����\K�k*�J��p�H�+R�
�mv�NK���~���T�������Y�����[���)&x7�����B�1"����������~�a����Ld��4�����["��0��� ��GV$����������z�g
{+�1;:���'$�l���j2X�T�����N�:E������t�u�|��sw�Ik�$I�I���X�x7;"HbXe9�;FKau@�~w�~|��O=�~*!Hr�����Z��M���Ga;L��5���v���p���^x�o����&p��;���7�:J�;���Q(�^�m
��w�����������.{���^f��I���i�u�>-*�~BFh�q����������J��!B:������fL����(����K=i�&�\U�a��!q����fx
s��`�MZs�w?������d�R�_������E$gJk��_���nQ$���\;SLd�k�{��~���y����k���j���g��D+}��.��;h����[��4p�tHV��,X>��E����QP�;��S���i�5���G9� Mx��lsP����qYUD��-����{��=��������UT�S?)�<`r��n5�G?p��B�x ?�}�1;�,��T-����J�W�%.2,������}f�����o������%l���z�v)��2�YRP�K�M]s�E:�[)�B4Wz<PpG�x�j(���K�Uz>���b/(�p4m�I@�
��a��4�g�z��:��fZ�Uh"Y���#��/g�>���	��k+����XRFE���vY'~����P��T��}r���DF:�\�]�����rp=�a����;�}s�a{��n�W�~��2���$+�����H��'���]�h�T7���~�S�.���H���������j;��8n�U��6EW�{�?�����0���[��`�F��pAa��H����<����f�����B<������$u ����A�������K��&EH�^[kn������rI�.��r���;A=��%�`p�����r3w	����B�����2�up�6�PLF{���!�#�LXU����0����OG�/�&0aE����5�:�8��g�nKTM�
R;�TH�.�e�A/�2c)$B{Is�	��u���r�������!�n�A ���Q!	�i�
�Q|��9O�{w���}?���V��p
�%*i��&���iB�r9���|�����N_�CwM���j����6w���p��`;C�8^,t���0��ol��d���
���Pj�Z�Q��C:p��ut�������
%eb�Y��c3�3g��C�����o��P�[�o	�����:b$��
����O������9u�	�;K��1�t�2\�����x�)U��S?z{8����K�g�L���Yt��L�D����2v�N�,%`�3����AhQHz;�vg\J��=��i��i�?��@�Vh�q�lVJO���������0���U�����`-�m"� �������EZ������D��3���-����	��~���r����[?w�e8:N���-H��2k��w�=jsO}��~�i�,6��`x�e`a����*�����R����vc=s
�����X��E�
�:�e��W�Zu!dXDP���pj8�z:��4���8x������T�{G�I<�!��;�F���?s�l���T9Q*����z�
��x��}�,B��"�2'�W)/��(<�ZDU��0
���2�i��������#$��A���<�y�������O���\��x,yZ�����\���[�%;cJ����~�H������l�����l"ut���]����a����bvGe)��M%��\0��m�pj*I�6��d9�>k8j��w��7;�N� �s:������?�&$VFL�����WX�~�Y�>�������+��<�+��v�
��T�_���~#���g�w�.M�����\��0�8���A"���
�����������}bK��e�X.�M�nR��J1���������7kV���HVjb�K|�.{�SPX�S��{$�����=��K��7�K��5�Y������#��k�����m��x(�������B�@�F��������k)�����d�n^j�m�CU9�K�vD�2=�����u��������g4f���|J��,�"r1[�+���Wt���H�H���C���
�s�d���F7������^�v�-�(�-�`'.��v|�����}.�~��������:^8����^�.�*6A���S������#L�L'�L��#��f�����-����b`x��"+7����2MpYlG�AD!��H����
�=��7������[2d����7�$��G���o����xb�����yh�}�����/��$�BfN<��96����Ee��NE��"F��}>������HJ\�cK�|�8��Oy����aR����-q��8�a�H�h/`�v���8�Iq�~Y���Z
6��/!Mx'����/o\�t����3����v���w��+;*��o������u�����ep�;R�h����{8��'�k������hm�7�!^i���3nX���t������������\�i���w�<�ES2,:��R����L$���xrK�f	c(j,r3��b�U�U�Cw�|�����;�g;�J��Q����wY)��@�K:��
������v<@�Z��o?�~M�MK����J_`AXd��_���/���P�~�}��h����� &���Z7u�N]Yx�u#�����\<�Z�la'J12��|�'���pb���7�yR@��G��,_�8a�SQ���9z&������]�g>��d(E���^�"Vj��}fZ�f��~�����]O���&�R>������BY��m����E��J9�F
P����M�6�v@ ��P�#���g�&mhvo��� t�y"w�oe���.����@��������4�0��6���R�%G� �wWa��3!�y������s�F#E3����O>_hi&�W�%[.���,_�U��q
��B��/����O���|>"�&�E�(�0�����V���^�
Xw���@�
�fF8-���.)�����!�x A��������:/�7i�j6s��(�RP,��H<h	
xD����w~����c��g*���0��	W^�	�g7��B�`W��nsvu��`3�t�Aw����X�$��b���]4CE@,>� ������[!�0�^r����>�G�Zg����u��*�g��0�O��RgH�M��#�z��q
`ii�
P��q<6O��2�~y>6�i>�f6jF��~<9�t����F��;�y��psV��G%�|q|�N���WN!c*a������Q�y���t�C799��o��&H���.����K.%G�����n%c!6X�vc�{YJ�X���c���Hc!��]����%����_���In�H�>��Q��(���X{��p�O�3�e_F�I����
��P�a��+B>��9b	U�e�w�=)���R�d[������'�B��7<��aR���_�&G`O��:J*��#�]�f���i��P"l�����9�>+��T���1�s�F��y%�3u_��%7�"%���!"l�����$A37bw%3���(3fm�A�������%J����^>�{�+Ei
��,* XU���b���������y{c��T�tg����->E�p�	�+�B�M$���24���N��E��e��^J����R+��CJ5���%z����;�oO��u�Jf����������7��]�^����������u����;�q^���dvD�R*�g��z]��U�H^�R6hTh�/���A�-�����&G�L�K��������w��|<�`j8��x��
jJ�1"&l�f����\�7��xPC']��cNw��S�{zq/��z����X�+f������'BjV����G����w0��_��/���7��S�4U��yEv	�2m��M�KyF��_�s>��'�$O�a�����
��WYP�Ng?�/Z$�d�	��q�8	<��>h�s&����J;�TLh�Xg��@��tI�������0S\�!gC�'��LS3�'��8b\����c{�+�`�c"6�	���Ge4��m`�^�$�����u?9@������[5���$���a/�u�����������O� L�)1d�K��j�)`!��C��
���@��Cs>���,%�8K�2�+�Y^H����N���6��������^�eq���47a�����<8�TGay-h��*��i��3~��aoxHQ�d��m�'^�I�UL"m�������m������~wy��������w�uF��* �l�3�0e��!lP`P�W���C;W�'�v~"(8�)���������-���W��4���<�
�K��UB��n��)a-U�\�:�sw��Gis�DC��ZQR�O�����d��������ZN�`k����(D�C��Om"�gH5U�h6�������y�l���'�e�@�� !��n�P�����M�L?x[��ms��r�V
ix ��]o���T�S����C�<���Mz��H��%�W3�M9qv��
)�'���v���k��������6�?��,>���(�3bVl���}����d6�^��	��n�7�M��`�(KQ��6�Z���%<���������g�|2V�V�s?�1�`{��*��*����u�Cf*j�{�3��,0c��A���b1���O������HB�x��-�����>g��)tsNN���B��dq�N������1����K���[9�O�;���v��J�/4��.���k/m��w���������7�f���*����FDU������X��}���|�	��-�qzW�'�+�����`�\I������UJ-Qz��[�#����"I�,K_������<b3BG�&o�{�V�"�3�o4-�	?x�<�\F�]�����
��6Ka�V�@��hj�E�*��;���������|}zj���nv��{�u���R���!�z>)^������>AJ{�� �j���_�a����R�J����l�uW��������M7�gn���vH��|��������Sr��]��3>������+"�'����!�j���:��8����"0b�*!�����'z���'q��a'�0��bc�����8�����n��w�\����XK���
�Y2��������];<���q6+������&E�"�uG���ZRil��A�P���
�>7�?�@=xw_��7������AC�KwSX���)l1���y��9O�2�,��Vg���zfc#�p*�[Ic7������w���O����"���������h
��p\m	�^��9�[�4@�A�bI�uF9��n�a�+���Ll-e����n�Z�_&�'B
O�b�/�����ARuU�P�C�7����w|pm��o��\��s����th�����N�*����)������/��K����$]��P�Dn�����AZ�H��j����?��K�]�(q�����e"���8$���������hC��8P�)pd�6W����� ����g7�d
@��W��;Kpl]� 5�#Sf�l%$V��+�z*9%]�F����-������;PBR�\?��FIQ�6�a$'r�D��+��1M����O�B�3�����t���C���]'a����>���ms�w����>������*�
1sOW�`�I�v����_���]s<���k��j&9�K���2���r$�������7�S�f(���"z�:t�sN�\�uqQY��p��^�M�c�����^��>�\����r+�X���*t�����$���Dbt�l��?w�swO&K�4#G
�j���S$�K3��������S����`G�2�\�Fq�jo�**R�F�����c���$@�H�&�]���sVd���:�r���wC~:���rbr-f��E/u�L�J�Q$��
�����k[_��g��"��n)s��p�$�2�L�y��@������>{��!P�y|�������6#~��[�Qe�h���y]?����\�=�o����N ��R���CLk�<���k��?�N/- bA���'�?=C51b2��f{���Ks|n?���O����S�M���_�{C�j����G��v�*i���^���i��g���-�i�lNf������j8C]�G�A��P]��3�E��`�@GNBg�X��Ip���/1����C�4x��F}"W���(aH3��b����v���G�����AQo<]?I��\p:_f6���a��7�!'}t7�����k�y����D,f��X�/��q6\�rm�[���%�oV���\����q���L�j������o����K�����S��S���&�j��C��C(�E�5��ZL��/��tu�����m�������|�r���<#*��Z;Wo��jF��nw���M�IE����"��_�Y����0�y;G�8��a�u`���{s�<#��!�p�:7[�;���F����?���uo�o�4IO?���6r���\A�*Nb��4�����us�mk��w�����3)��F}�VY��RhB�d_z��F�_v��}���C��c��v��@oY�?pL��;/���2�S����G������n��0�i�r(�j�bHqg�P�/�K��������dzO)\q}kH�`u(W��+�(�7�B���G��s�M�������i��+�[_��G3;7P�n�A$$%��(����7�{�^f$S�E�RpV�p�2���=�b,�����������i!TT����> �r�%�G����������#p�\6�������2�I���K�^�1���J��,�>��_����+,-a�3�����	�|��X=IZ�w�2�E/`]Q��X�A������N�N=��>�,i'(�.��+@u��|���������T���J���n��s��h��FH)��-�=>i1'������'�0Q)�����=��n����/?��W�7mq�m��1%�W@��f�B�2n^$��pQ�sd_{�����!ym�EF�Uz�>�N����A,��BU����'�����Q���"Xg
���"�;Y([�
h���
���
�X��[LM!���x�����t�p25��9\���w������_]�yO]�������
��oT+|�����J�FoLn�-E�r=�~������if��*o��:&r��j]�����q��Y�?�n�	IG1����:Ic	$�<5e1U'����
��S�R��"Y�L����.�[�R�m�$X�G�o�+I^A�
Rq��F���Csv���+Tqc�U<Fk��s�2^~��!����cr�z�/���`1iu��Q?�U�$�2�p��t�Jq������=u��{(�2�����l
)%<���y�&m����u
�Z��g��'gSN�k��~9ngi�N����~���Ig�u�����Z�L�8�E��P��1�#����cd��Qz��%k���	�|��4@s���i���O>�Cx�W\	�aQL����u����6GO�<�I��C�.y%�P7��I#����`�kz4%K���d@���%�#3�"���-f0��\�m��+��������z9��zsP&��&}���i�����L)MzX������\��76-���H#��UsW r ����y)����i�?9�	���71s^'�G��[��.���~�z3��b|I��a?��p�Pt�,�z��r��
T�7�����o���d_���;w�!W�>\��?�)'�!p�Ip< ��tz5�@�N��~��e�It,����"�d@��t�+{�p����R�����BFU7�Q~�,x���F��1�G�M�Q�P��YV`����V*����Pdew_cp����;o�mw&!I3	����o��H]��y)���P���9@����@�B��qS�\i�e�R����]��,_K�����>]�����u&�����~/(Y�5�R��%�ulr��sj�*dw�H8�.L\�bp����
q�����
�z���$�[I�~�D�BZ��
w��������Z�W[n�LM�c�H\xb;�,W���H�;'��h q�$��-���W�����#'s�����[�Wz�7�������d��4��\�����_�J����)��M�������G�/���?��#0��W�V%�]?4�)�����	.����;���alO�����]����G���I��GzgFbPf���4?����(�4����A+�l����o7~��^����w��IOB��m_4"K+p\DE�����g���+��e|N�1�����^�%<| �����)H�~h����G����ze��~JC�����<�RW��a�)��g�A�\�}G�d�	dS��k
{c��������9;~L������������Z��pP��EA�!
N8(gZ��Q��~���O2�Qa��0*_C����i�`�?3NJ�6]�?�������/��������DaA�_
V9I�
"M,-e5R1�v���������J�K��K����C�2"�I�����!�%���*�2��[�K�H��� �!�imhYQ;+<�7�v�D���_�csO�������r������!��7�v�kY�F��
�L�<���<��v�&
�8��<�?l�u�Al�d�u����/�q���_1N�/�����?��_#�e�7��&����������d*?��������;\5�=��8'eULf
��g�Bo����[0�)0�s�uz_��^�K�F��0�*c_��j��;j�dA�^�t�m[i.5���}����'4�����,^��G��������M�����o�������/��$&7�/L(2�����;����[u�"|���b�\*�s�p?HL���w��vc�w����cL��Z����>�gt(�
���D/���xi�L��n�'?��6��?�t���u�I sH��2��Uf�/���������L�j�D*�<�$8���u	-E�6.�GA���L*�u_2�Ww�tH�����RU
�{��mO���5��ZO��T��
�fI#�?��&B�4�*P�1&����W�|�����/����3��/��s���������}
N������w_h��3k�s�����fw��
�~����BTs��]�����t���{p���~��gW{���Rm b�*�� ���M��\���dj.6������w��P�cf�l������;Gt%WfJ�(�bn\��vL�2 ��5_E���X����������[�����7Zwm����v�V������U�i*��p�]Z����s3:y�D�H
�[���2}����?|�Z �*����y�����E212%��lqi�H���F�;p��wY\�A���=I�J�hNFB�vT8�
�7�;��������=�?���p����s�w�i���M_�p$��f�����<r�rW��uu����n����V��>���y�P%A��>���s5���g't�y�l������t:���k3�^��$$
ll��
��UT����n/���^����o$�?{=�*�S�8
O�b��H�m%H��M]�7N�*���we+�sq%~Tlp�~�����
l3�\N�cc�m��s�!b�^QG�$�E���t~y���O���s�ors�n�t���.L�T������T�|�	}D�-[�h28���_���6JtH��i���d)�~##���������J��yF#�D������lp��j�u��-�����H'�OM2\/��nk�����zG>��-]������%����2���3�a����v�T�l������Y�%�K�����J�%tZ�!��1H�5���yw��*#�D�g���m�+��Z�����������	�q��T%��0��j|��af�u)�f_�_v�|�����4�����E�w��S	�Q��������?��x�<�7?�'�~�aD2�$�����&�m-�W�������L����&s3v�23-���VK
E9�?8 �&����r	�T6�;�A�s��=�����0��,���� �U��>?^�����O�����4M�b�K�w�A:�=um~���;�
z��;��J	�F�`{��Qb��yp��� �qY��5�#��K�x��j����������n%I��l����u��B2����Q�\�J��g���*\�l�{$����Q`��bH����U<�]��'����B����h)o�������Aki��T�1�}bUI�5[�����O]��pt6v�v%������ ���c�� N�"jcU)�������X�v���GD��L.7b�;�w�� d�I����t����
L.�#�o��1�W��M]��k�/X���l*�N��r�Y����r�
�#!�K�$���R��]���=����%3�U������FC3����|����k����$���~rS9h2Te��#�Si�VrfA�RU�b�#��F�R(F���}:����TF.�"_B����I�=��W�T��[.6��"Z�z����m������*������q���I������3��2x�g�y������lX1;�Pl����o���,��m��]CC�9�!���dle%�T`�	���}h��m���R.m��g��t&�e"2;&2qdE�q�2�-�!���qYf`c��#X�aS�����o��J�+=���l��J�Eb�{q])��]S[����T_�����hp�`i��c397xO������o�������7x?� N����~IUe���"��T�<k���i��"�4���.�9�[psn�a3�������Qz�/���!i�s��J�����������b�_�\�J.inq��X<s5{�U�#�|*������x��
���odH+~Cu�M��;V�]OZ	���$�@�����n���!0�!K,+�-$9���h,b���H����N��������]��f��
����&Q1�A�R,��@;��`l��P��O�dt��G
����v,���xC�����$&����RLt"��������%��o�u*2#D�1���#>�G
D�	|������A(h��CS
kH��R����LM���}��������m����/��KE�T�*��L�u%V6�
�N�r��.{:���J/�&�!�IuIq�>��};$T��"%�f)}�.��� ���7G�iz�k%C�q��.pV�"6����<�g���6#5�~H����'M��K���p���1u1�&�E?w?�v���_�(w����dQL��h���j�D8��d�M"���9��������M��C������+���� @u� \p��9�6���Iv�a\�p� ���F�R���o������p������!�{F���#��o��	�8����By���M��m�!���@"\�>���2�	�'a	��O����\_.�����Bd���	\_b���b���s�qu\�R�E������k{m��i��"]�����P���r�����^WN���o����nd���v>G��o��p$_��rG�8����s�t��GQ�?��k��w���qH��7%>&Pb���>Z zh,����Q�W��^�,���m��}��:����F�}�o(���) n�����|�2��ZE���C.�9�N�����_X����d�����N���1&���9cq�0gl#l)�����O��q:}$���#cW4�����d� I\_�����x:����z�jP��_Lo�}�J��@N�x�K����.*-��;���-�Y����S�{	�^�Q*�H��b`���V��H�����tn�%��K�i���$���u
8��e�T!+����B:�"����$p��g8N;�E�bH��J����<Ss8RZ��2� ����;p�����p���c��;m����a�������Pw=�sU<#s�L�B�����?�?@�G��p�?����,���qw��5��%�zS�V
:��������8����g���g#$��2�;2(���	�����B�z�aV��wH�8y
�����6�b�����c�J�
�1z1����,�e��y�Q���Ar���3�f����� ba�@�;FM�:^
G7�9�������:]�4�h����rdh�&� ���Yj�z�m9���K+|��}W&@���P��*�w���C�I����`vce��v?�=�<���8�����(��^���X����2
�F�����g���_����a��7�O�C��r*3�����9h�j/C�j�T0h�R���2�V��=�
9���m���f��F��n:.��������TH:�G�g1le
2�NHh\�=$����������?��������k\��i�qFht�\���=�tEYNL.������g�	0�v�!)����������b����v�<b��KP4�x���c���&g�nnH`Y�f*��_X�k��w�S?��M�I���`�@}k�sA���*���]���q����5����)��JD��b���4�I��f]�z�t������
.a��G�;%���R�pK�a�'�4{��Y~PR����C::�����`��Un�<w��-��$��������9�lR{vJ�B
"��������o�Y�Jw��G�|������Z�����!�j��y�-X4F5�gr�fU�@�.�':����|�����^`\7&=�_NEZ"����\�	�9YLO�Gs�T��+	�y<N5��-��U����2[@�-�,Nzw#��gH
�4-�<��.���(o�E@8F����]_��������-�Gq?�@3^&�����4�#V,�6�l�t��"�\�|%���Z�<=�q
�~4�
uDx��(�!�n����y���� ���8�]��`�q�u����7��d�a�J"!x0���M�
���w��y~}x��F �XG1�����NW���B�.�J�
��L��h4O���=^N� �W��C�P������zA7J�"����k�M���<����T7v�1���4j>��x��2@ZvIW�8+�|��QS������l�|�Tr]#L<�_�,��6~9u}s���w�����#�f)�N r�LDA��rMZ���qwh��)c���7��h,�>�|/�fu	0�B�R0H��<�����6l-��w��yz���W�7����Q���M��d��2�'/�����d=����KS��%�.�~r3q��Z�|��~	pG�0�]�?��f������|�S���#�Z37��/I �q�k_7}�>�/2x5dr�$�)e�����������^�y��c���^E�s�
�kO��(�8Cb�����@��gJM�e�_Q������ W�r����������:�}b8v��N:Q����U�&� ��8��L.F��=�����_pF�#�(bb���,J�^���;'E��}��G�=b���J��t�H^�E��m�ri���r\#��R���/5$�y����6��#+6�����NId�Q6�J���<\����r��1/f�yF�0�Whu�� ������l���?>�������7����g��3ZR�W��@��?���.���n�l�v����9�w?���p�J�V��*2k�rj��1�(�=�!V��g���~���.o�dR�;�|�&�A�"2iBx[
�s�����o@i0�/(�v	���������"�&���*�H���?yoP3�
O[�S����O��9�t\����Y���aE���UmK@H0�S�'R����`��X���n������n�����
/��m�6N�3H�)s�,�m\�����]���o�����)i�����X$1V����R(/l������k[%����^H���F��@�*������F���v9����S�9J(m�jwc%C;�r��c��	6�|{�a��<h^�������CX�4�EP^`��L��5[����t��l�F�����/�+�b|>6��� ����8A -��5�,�(����w��m���6�Y��tq����B�N��0*��*�����hI��?�lc9����ELB���=G���R��&��Ub����';���"����L'E�$�����&@O��Pqf��;�U�WM|�F�
:w[��)'/�����l��
{��q�kd/��.q/'LD~Z@\�m^F�iAGQA(���7R�RJy�����}��[LJ��s5%��'���]VUz�0A@T��t��7#�K�u�00��d�v%=�����=�c0������t����89�H1��IQO��K��Z#JQ�||j'��&��3%���%�xoc��)Pc�,eO�JH@���0�����P��Kt�s���L\Wf*�%l��t�q���R���Lg��_�����Q���"��8$�0�r7G�c�7\/�(�QA�a59���U�6���:Qh9�&��/��7]��MO�1��"g&���@~��f��Y���������x�k����:*�t��������7�l����v��fIH,�u�Bx�~���^�ng�>�|�a��XXe�lHR)���U
�pf)o(1�������9�����.�m���R������&�X�n6�q��1��@=���P���@������Wu�E����+\8�+����CQ�4��s�Na}�Bfl&�F�d��W������q@����T|������o)p�~�6F���!�w����w/���?B��Z�$Y��9E��lR�#����j�������?��/���q�R�'xl �!��N�w��y"�E`E��L���o������������3p:T��]��Bpnr�%.����+����&r�d>r�F
W�a���Wn����2~9���>=���C4CDI�d����3�#�h�y����}��������+�����M�f������_S�p��9M&�o)��jCl)�s�m����� l1BFy��D2/���)�����L���1H��a|a��m��s;�|�L8Ji<�mu��FI���)e,�@7=N�=#;-_a24I6!�]M��!3�Jq��3>6��Yn�b�i�B�t�XcIw�!u-y}�
p���
q��v2�Q@�G��J]�y��F?���U����)4
�J����.,�Hh1���9z�h��I���dHNp��?F\UW��(�dx����
C�bf���$��i��O$R�G��p[��j�����!�JV���M1�f..`(r����9)(T���K�t��b3X�<P������UH�M!X�+�Az����?|gt0M{v@}�V�N�pg��bk�������?=���t#$�)[��n��!]�������Q�!�����������P�]e�2��[5��tp�R�A+�%�V��qD�8�/-<����)Y4(L���%�c�rW�1��0��-Ft���[}i����������#B�U�
P7�L�p?��������=3���(�`M��J�-��	;�H`�9�U��'w��[�s���W��_�����*��} m��`�_�s|t���Pj�����V��<=Pj�"l��@�vjpF2��
7������>���:�k������&@�
j Z��?������~� �H���iZ.���.����L1&]��@b7�?B2�c��=��H��J�r�z+d6=���`Mw�_�{�s�|y4C��rc�u�#�����k�^�>�`����������kF{?��G����4�!��!G�(m�����	�H��C�;K�t���=��MI�n��s�*|D�.�/�pBZU��N��v��	���S�F
����[�Nhr�f����.*E���
�x�����U
���������Z>%�d/-)���H����-ei����,�����dH�����	�{I����JH��f���R�"�y��]H0I��t���*���tu?����oi�f��{���*�x�*�k�+F_ul����~Y���m!�����bZ@N����h5��=������d=�����$�I����[D�4��!b*c��� o�e���8��������D�Z!��2�t2�Rm/�����@E���$�,���T�
�vE�����R35���v>����]���h��>��)2d��b"����������N4	�v����C�������k������������Kwm���	�
������/���gL�����G��D:&�z?���b�w��9<�����H�8�V�Hm�������X��"i�F�RfK�BS�w��33aE�e�h9�WR$��oh�i����e|�-��G��x���:
�4��V�����>��w��&A9�x4v`�I2�z�d��EF���c�������K��`�F�����_������ ;��������w�h�*�}��$Ho��(%��
�C~����@3��wC�^��E���������JL�i6.�N
�0:�����}�:LO�l�@����h+)C<m~x�)��E
(�Q�����:�����x&%�T�
�s�)����r�S}yj��Mw#m�����V�b�q�_7��b?_��09�4-.8�.%�;s��KR� a���B��0�p��-g��ri�L�t:�(4b:��D�� �����b���{����n���v����]���k�q�H�e�>�|?��m�b�#�����&�[�ei���oD��EegV��y2���TeF���;�b�Hy�&G9��S����AgY�{	���Z��Q��s�S�v��4��n����V����2���0�`~�DQ�vU�W`:�#����	"�������&I��t�n��E_h�p���]��a���QN9�:����F)9[r���8�u��p��)j��M�����	���%,��.��z|w��i�l,�E�&���.[�T�]���x��*���IR������df?��U��\�����\���W�^�������~�|Zl��r�L�f����x���\Z4a�$:�`�M5����n�4������HEU���[b�X�1r���=��x3}H���7�f8�yM������W0�a�����n�uB������1TSC+�8��Oq�+������v�S�>�+gO���}(�Xj��F���h��|�^R�h/��v�����}�a`:�D�U1��t����+�a����L��1]�h�=�>r���!���2���j������3l
�"�\v�F�'�&v�EB����3��A��9����z�H7���`tA�F��"���<������*����fS[��d�x��:���U
w+=�#Y.��FK�)�+[��-CvKq�"��)\�p��V��	<�	,�{.�k� ��y����V�MR��h�?���IsH����g��@�1�u��&�Z*x1����F/�5,fV��:���&��8D5D��s�>����r�nw����=n���O�F�s��r�� &3�a�ZQl6����yb�*F���
\D��|:,��1\��w
���:l{k+e�*E�X���v�$G�?����h������9g[4���m�QC�D��r)��Y�4t�n�v	'��.��(���,��%�q�||��Wdy���r�����Y>��
��;��Ak�8�o"���'���9�����L�9�A0@��`J�F|�!����H_7��H�|�+�}q�!���/T5�O��<o���\��v?3p����JR��F0���v&�U�`���kD~������(�BJ��05W��*#��`����T L���|9�'����O����GtY~<������?{�%����Xr���
D�m��m�n7O�v�w�/F!������)d�q�>��%��x�i
_9�8T0��B,z��B�"vI�BA1h[���5�j��e��+��0E�~iw�����a�_>�r�3yGM�{����N��'
�n=�6�����_z����ae�����7��4Z�q�Wz�4������9������y����IRs�#7��f�i����c���������$�QgNd&x����v��O���L����S1�������/�H��*&�����>~_n�����^�����
#k�(1��-��,d
i�*�2�j)EQO0Z����)��J=��r�& ki}�`#RA�t�+�(`�l_rh�PK@�#�^
S�p���x9���c�.R�������7��`��H$�[���O��y���	����:f��@9�})$�NG>�4�w����?���M���5���L��+;�����qjG{(�5^�M�C.Pe��a��Ys�*a>e������Z����T��nw<��5#�Mhv�� 9�S��i����x���l�Ysu�&G�H��.�����R���F��4�������W�
�>������N��ZRD�����9Z@k:Z�T�o�������_�����l���������itag�)���v�w/����>�qv��)�I]6�Wc��|�	�oF1�Z�ZU)~�l����RZ_��ps���Y$�p!���YM��]�n�����fY�Q&K��5"��B�j�	���w!�V�#����TG�ucqg��Mp1q���H�������`\�C�&V�F�xF��Z��m���-���(o1b�Sc�BK�
5�7�V�b,�N�3�W������<�����6��u8�x��CO�n�+�-4$�\1 ��w�)��8�H���<X��g������c���O[n.��t'l$�_�����i}�H����3G��k���7IZ�e���SO���%�^��<m�(����50_"9G����b���l�
\tE�i9-������w��������/�5|��	��F{��\�WV��G����T�<�`�;t����kG�9�iG�� ��y�*j�#��n�����v��Y���NK���||��3�����j�/���pZu�K���dv�����7=tM+4��1cZ����Q�lv&�OF���?>~�>���a�(-7J��m�����F��a�~�ZT��(�9�n�b�S�<��%�O�t�<���#�7�O���I��X|
���?�a�Ao��;
()��2B�7������!U�����u��_}�������'h�����~�8�������de�9�<y#��U0��Z8F����\Z�0��wgg�~��,�2���}�5U ��RA���|?����O��P��oT.I�i7T�����I�Q�_A#�������30CA��4L���'
�����Nm��:
�~�����_�s���kKL������p���iXY��
K��y�56�J-y���j����y!qi�7���IW����'��u���r����V��}F`,O��;}-�B5��)B�h��J���Y���v��m�������l��&�l�
4���SC��_���s��p:�����B���#tE���B����7�@cG�1�-l5�������E&��j��Y��9�x���5��^A�����r����.�(��j
'��B&C��BIL�����U5�����d$Ey���G��`w!1GZ�����p|����!}#�{�=e��u�WL}��s�i<#N�Hn����[�����s��quT�@��FLY"���R15��H����_4X�X��=�[��t�;�K����u�sT0�.���?����lR���)6�-,|C�r���m-�O�0E#�q����*`b���P�1D��rr���������V��r��L8meu��	q��%��Q�_A��5�Y
�F��2��.&1A<Mnt���`���[�������������	�Q?���&C���	A���?n��nv=P����nK6�ld��7S���Q_�r���\>�������z������'s,P����*���d���Ccdjq����=��S�Os�B����w�������b��<M�"i�����k���d�����O���zJ�4��5�k��E7H�OMD/��l!��[�'7������&�t���{�����	� �����>QeET�V@9�_p�20k_��_|�[##S�5���~�uj���6��FC?/kQ��g%�}�to�wJ��'F38���8������o�4�A$�S?6N7�2?#�s�l#I�J�X����B�N���{[F����B?��������Df�^,�exGz��������)�.g��$���lw�����`�@���������7�A�_&fcVW��/\wCu��N������������mp�}
"�y<w�"�u�.�E0v����������V��e��v��ff
s����~��d���O��$���Ow�LK�==ge����6��Y���s�*�Y����m��jQ6���'cCr��p�4�Q�t�����h��g\�=�F���/�P��U0b��Y��m���`A���tF��Evb��C4~4J,���on�*K��D[K��8��)zxE������
!����K�1N!����|���R5tc�n-+��w��<�xG_����-'�xW�v�y�������i���|��V�-�HYo9=�-�@�"�P���	bR��[��7���U����Kt��OT���Da5����7�BC��ny<n�/���$S5���h6��',�
E?������h'uYZ����&l*(C�d+G~���5����Knf��q��?Y�$6A3��f�M-�g (�s����������%]����L�m��d����j�!?��*�3�9���;f�KQ=�(T�Xi�T,S(���r<��&!����}w:�~��8�4Lp�Yc�d)b�t*�J)�.=����a*QU.�2>��a����L�(����p����m��v���,����kq�K@�Y:�}�u����X�}~��0�����aH�DYK��-M$ ���
]��i�F��*��
D��4E�O3}�(������}����@�D������Zj�3��J~���
��)7>�/�\�M7%�#�V,�����tX�c�K�
��e�H����s:<�D�6�*Vc���3�4����w5"��$�X�i�#���Z:����l�������(Q����J��.�s�����[�J�iT�D�+����	�3�1�4����~��;����N(A�?�O��Sfy��*~�V�h�1�8s.���[�Z����q������%�]��f'%*�A��a������/tl���$/�*����F�� ~���_���6fFH�x��������jN�����T�~��;����|>��Q��GJ��\"��1!��x,"�Q�������pw��e�e��a`p�+��e�H'�`Gb����Y�����3���#3����^B�Guy��T'�K5	�����E�w������~�:�|�k6��e�q�&Y9������X�j��~�i\��(�~���o0�8*=�3!�
����8��\��x�d���L���:��N��4Y�3f������+�%��+����X�(�����
	������U�	��/������x����9u��B����S8Zp���~-?Pz����]��/{�����n��k!����B�YO�g��0�E�3�!�Z�>O�����2��9re�;q��x��H]���e��o�Y���A��jv����w;��\I0Y���$�g���Q<1�0�k���%H�x?�3}j�TJE���Y����o�����K������$��w��n��]�����e�Y(Y�z�J	����i�=3�k!&�2�\��������C���������Sx��CE�����e�A`�M3�>�����@����I�����+��$Asl�0��&�O&�aI�G����m�������zyZ�H3���!������.��- �������5�jZ*���
>Ld�0�^�S`�r������cd#Hv��*����m�
�����ZB�M�XBq!J�<�v�Q��2�`�����-�a��$H��@bB����R6D|a��-�x|
G��a���^;G�$����Y����HC���0f�F��|C�|��k�S�k�=O����Z���\����q��K8��q��t7��Mpy�����[A��f���1��� ��e�>]6�u�9���!V������������qZD����T�hC���4����g��+��L�2=z��$��c�A������� O��������n�6Jj�xTX�j�E~���G���|TXU ��W��"8��=~�l��f�6�*p2����v�G7�
�@���m(���uwj7������n���m�[��u�G�Y��f�������Xw�	���E��p*i
���%�%1v@@�Y���=]��wT�t��}�z��"�����y��<>��@��2J�p*i�z#�&
�[Q�"`��2��R�����+�6���4��V�����`�k2����(��##�z9`Gq�$���0�"����;��([w�Qr�U�'O��sx�1	�����~�_���a���eRSHI��;^���:b���H����C4p��	���-dv��:Q()��?Y�������!+g�3MU�R��M-q�~-k�����>~��`D��N3��������ah=���O��f]��������=�KH~�;	�XM��+�#'������
�P��JL�R���W��6��������
�v�m7��������w���A\�N��y2�Z�/S,x�5�se�4%*8H�k����M���K���6	E�.FP2K$�(�c�Rg�@��y�l,5��*��'9[0\b����7V��<x~�b�'�w�O�7jT�Ll2�E��F���kxH|->���w>��];Wf�+I��S�
;j)W����i~�0S\-���\� nn7�9�<-��(1=
Fv�I��kxX����~�"�[�a�9��4�Q�"u-���X�G�\�y(�I�e���%4��w�����������xx�{�:���Y������h��[�Q�0������d-�����3���T��G���J���x��1��+������)<��k��h�w	t>��2�_Z������P��~�������L$��	?#��@
)������}*��_�m[�@��2xk�g��5�Q��
��c�	Q�3�P.���@j��_�8L9�������m[�-�0�ov��������hg0[��|��1�r��Q���u|�|��rl�tL�������;"R�2��+x(�O�n�Z�0@�<1KELsw�Y*pq��:�d���K��F�t���q����a�!���/���~��f����v��+�n#��9b�����[)�U����f�"Y��n"BH&���s������W�����G�}�q�L{�����R�T�n�T=��><TQ]�aD��bQ|�8I��yn�����#d�w������?f)V���b�B���y
�|��l}xe�nI*VE���z]E��t�[���� +Ep�+�����DgP��4��KE~MLJ�%�f��Dy*<���������O���Q���g"��E��\	����t�.�O)q)�<x�(��_�k���t������QLw�i{���c���~�y�����eNd���YX�B�_T��LT0���X1��[IK2���N��l��83��d�����.e�{z���}��Z�?�CW�~�����=�Zo������> ����=���bX`I�������
>s���B��yI#�@�_g�6�rD{����O�A�]�o�����^y�7S-��#�I� ���ZH��|�r2gq�X�].���k]	\��e��-�����{
��y����6����������
��b"9<�/��,��<Y����F��*@��+s��N�?)*p�`X�@�
(������O���{����������"8H��s�o�^Fg�����Ek/�7�	�\����f?���B�	|����6r"7%*,����\u�q4\�i��%���
��"=���c8�����U�(W�>���V��"�
V�����L�q�3AhP��#���3�Twm�i>�Z�f��g)l���#��QH�42��W&����t��=����o���2L���3�="Ra��Q������X_��o��Q�
��y�Y�A�8h�e$l�o�07[ 1,}�5�$"��u���� ��)���~|�S;|���a4�2���l�T���������[W�(SJ���n�+I��$,�Q��������q����������'�p��\�Xa�,��?�D�P-V��#������>�c�EZ_=_����G����VUJ�|}���C[����������S^$�h�Nn�'�TH�\P�d������c.���������e�M)����e��x)v
�>W���v�X�������Y�n=���Nsx�z� NT���{��X��������><<N.��dp�DX��W?�"��E����)������3�t�Ed�%�&"$J��Z�����J9�[�X���Bu�wi�Y^��Xr���L]�%y)���am�1����l���Eq�����u�I � ������	��T7���$��4*
[�
�����*%�
f(%!����u�i��*AL"�{������9�+[���� ]�b�l
}(�����r��W�)�o6�zR��L�f���tD]>5�[�$I��c\J-[�b��N��� j���e2�r����.IE%E6��z�����,��;�m7��
8U�$(��k��������,h�d��;����BOGW�����i�4���%�&H��/-���������-J�Z����
���RB���E��w)
��b������d>m����\]s�"-�6|��F���/�H��h����s�����R��vH��4�=����o7��^��zC��	�!���Eo3����@}� )������J>PD]�~�,�+�S�t$�d�-�R�D�R���U���V�����m�
���I>��>�<L������b��#X��t?�#�yl��s����84C��M7����q�+7�]Q4NW����o(I���b�����������1m7����:m������*��u�v��Ms^�����Z����~j�V� JQ��\����hp�B�F� ���#mm��<�2�M��h�	��K�����x�$DFEO���l��a�$N�J_$4��?�.�������+���}��"p�}��w��QH�`���]s��<���QNX�%���*�+o�����,���zv�aJ��5��Q��������S�2�����4]��D�&��oP�
�>V�0
�$E��%#r?���~{x���~v���#���:3�^��.������y��������q!�,\������D�K���`n�Jy��(s*�A���%�r�T�9��]���8e'�7�x�?�t��@�t����R�����
)+"2��)�-���+��h�-U����Vh������
"����h�"��R��4G�*��N�R
����i����f��v����w��#l�B�8<�[.�E����c��;-&��
���;I���6x��j"
r�~LR���k/2�s!���}+$�9pgq���J�*���tu�GXq��=��v]���p~zj;w�y��F�l��b�.@�|���Sk�p��� O����I7M�s��o���(�Y��
t�l��l�r��*"~�{�K�Q�
.h`e�o���Mw8��|MgT��
|�n�R�0C+�s:u{���m"�m"����r���4��w�u�3?x'��m�yw��;����
�'Z_O]�N�	�,��&*�T�����w�������I��qF�N�N+��Mj*8U��S��HQJ��=�\V�k�o�A�o��"���*��pmLfu���77�6w��m?��"�4r��'��AW����_��
����>7��<�Izf.
���]���FV��s*K���@^�^���[^�c��hO.4w;\k$���5�tz���X���hEH������dn�NueR�w/�#v���S�^-����}�=���iw��`���w"�E�q�J2��3xX��E&���J����l��C����	fNv���� z��UF�:�1���?���zs����}i�v�yR��=����[+d��sI��h��[�&X�4�J��t.����{��7.���M�|q,�XZ�1�K�XAx�rW�}J���k���k��g�������V�43�S��'��Yu'��V�O�x�P���������� w!���b�)�D���ub����� '��7��)����f3�3�����\��;F��0����9��(�����*y����4���"��Yn��e"�2�2#.�>����k����+�`3����9rLU��� V(#W�C�u�)����T���%�"\8�[����V0� ��[X������\p�s��h���{2����s�+����$����1}����u��r�:�*�l}�~ee)�9�6����������C�t��EHQ
s�����LK��������M�gF�&gFVN&��������F�(Z�&-����
�/��C���,�f������T_���6k�X���C,����������^M��������i��#eQ���	�����VR!;K\���
x=���\%_LJ&�9�|QB*j���)��#�G�q��]���/[�]N��VV��J�5�m��X�#��7�*�E:1��2$b�gfFDVV �����6������\�������_������$F7`E)+�sj�GFK��?���}��������[Ns�=��6��/*���J*8���Q�����K����#.Ibd����.��q�2-`�>�C������K��-a3����y�#>�"����H1FMN���A��+����y[��fgsY	�ci�dfU��P6A�mX���T��{���5u��)/�r/�(���0�c!'��u��0�����������
�Fo�?6��������� �D����!O��K�Z�bh|u��7���_{���~����!���DzD+���X����{>�o����������7�MU���;u�������n�e�vE���,�g���������f;v5�Lw5y��L����(.��9w[p����?7���>��C��n��6��2Q��%��MA���C/S(��$@)�9M1�m?��5��k�zw���>�������i��AV��e�����
h"�o���[��|>������6=�ae:��Pf�����m9@��/e��Z��i���F�^t9<�8`���.e����4r�g{���gC���\L�B3'�E���v�C�����.�so��By�����=�-���D�Q2�,r)�-���F����~:��������%�@���Y�m�����9�d~8l����x��+
��v����i����#U��b7���f��:c+��(�"e=m82b�N���3N�f����5��e�	�(�����=z�������~����r@%�.QX6%l���k��"-~6��t�H�fO������O3�v���dah=��1�{1b����K�hE��,9�����Z���RT�VP��S�2��k]��p�A�x��\9�&A���X1�����i!dU�m��^)g�|��R��onW?|9u�o
���HE�H���4_H�}��m����2
����y��AC6�������s��o[�����b7l���Pg)s)q��&	Pg�J)se��+�nI���D���Q���K��<,Jp��~����������d<K�DF�f�Y:�\H�R������+�oy���F��cQv{b&�!�RK�����NZ��41���u�m�:
(�,�%_�K.�����Y�q�k����.����d@��^V_���%��.;X����O@����q>|:?��sK�0w�L���-/�����{��YOH�u�6C[�-�P_�<K��|�Z�	�5�_���U� T�k��C�t��P��o���!���?�����4���MX6�p�(p�/&�������RK���?mw��[�q�)u��"�toYG��lJ
[UJ���[={T5<-���K�va=��$�r����a?4�M�mj������3�.!=k`��������S���u������v��ku&�����Az����C��%v�K���t��L�A ��Q�;�����RP�����v-A9	������'����*E�1N$��O$�=$
*���l�%9r�p ���Au���b�5����c���������0,��h]������/u?��g_��z������tl��P%�����	�/��<�rnvh��@%u��<��;��yz�d�D5xP�2tpa�D����V��fF�L��*����
w���������GiC&a>��X�o��)]@��N��!/,�
�����
���������R^�h�pz�+�+j�3�������9�T��{�~��������������U��/��w���y��P�W��&�}������9x�����\���c�SF���_�����Egz"vR���/�*+����Jo�*uVK���^SJ�~YR���_������~@et[�M�`�JW�r���VY�`0P����~��f���-���p01�u�
�Le�Y����-���u{����=�*>���X\�G��t�\��E���Q��^����o�A���,s4�&7q�:H!�.gS�\����{W=�S���!�f�8"&*v
���mI�.4����5.�-e�v���Gx.�I��JFhF�0N�����s5��H���'�����K�m$�7��)<�f����h��K���M{�|�JF�vt���M�Xb�-��T�o��l/@�yY`�>!`dy:C�RD�����{>)R�!�^�bVP����8G�6���]hf��i�0J�
{�3Y��K��h��AW���F�U�G���@�d�������DE	���6/c�y�x��d+��u�1�
�j������F�?F�q)=�_������"�Xb�34��s��V	���p�,\0v	�#�-�������������1�&�����o���p�4e����_^J������pbx=A.���X� 		Pj��.Z�`�^@��kN�����d���
���p���uc�	�� ���_����w^��i��yRt��H�.g����L�����%�w�u{��k���N��c����[�20R�DK���E$G�E>EK�<�������������v&�7�	���y�l�E%�d��BG�p_�)hbt��jI�����p�;iFI�s�������w���
��f=xE���{���i%�n/[��9�����KI��D�>5]'	W�UJ�dhW��Rt�bC�'��P+�K!�����lh�@��
 zP$0�hE
�
�tA���|	��|� �K����S��������P��������	�|�!%�q��
���i�~�jx�'��<�t���9|����W���=\��%n�!�JW���y\��hEd�U��4��_l���Zd_���������76����4|�r?H�6HN���/���=w;�
�P������t��$��{n,�}g�6Hy�$6�=
������h`V��p6������q
6���	>uF�*�U����������~���ni\Y���Y[�e�{��=�;�/O}�~u[7����
0Hkf<.�V,2f�m�_q`�q�<�\���~|��5-3��SZmc.~���ZV�q*��bSJ�pb�O����0M�G%�l�Nc�MJ������j�z�{�-�
�_V�YEnx	Q{���B��&;l}���.
�KS���������7�"����z�6��_�c����l�2^d&F���������(S����[��b�r�U0H`�N*U<�|��T�z.�+�$��5^D��,i��jp|N�Cr]��w��r|�&�K���`���Q��p]J:���v_{=���&J�u��K��r���F�kC.�
rF�v��!V��)�����S�0���'��$g�I��^@��$������U"�m���X�i��$w�e8SW���i\s�H�B�����k�4K5hp�����������V�����T�m`�o1/���
->����r>U.����s���B�uBSe������7W��>�#M���#/cH`���[s���l,��^�5LAv�����H�f0��G�@�l����:*�s3����Z2�o*�R���[��u����bV�WLM%�h��	I�H�>/�rE����iK����'�����>��8[F������1rp��!p��&�~�`�p�!t���t5��q��U�C��@t�
s
R%P��M)j�?���/o���8+����>��d.�r��69$m�����'�S*xi��\b��q$b����|�����y��`�������O��<�=?�|�'O)��?�*����a�������&]iy��4j6�0���(���i)7����*��GAO��l[b��P�s� \0af�~	��]��xv�:���NyA��u���"�D�������O�y�|���z��M����^\��9�"|�����RT�4�j�R�J���������fp�#R�S�)V�8�i�^_��i{g9�����D��4.�N��J�B�oO��=��QWjn�!b.(���s����z��Q����e<�60b�����$}����/$��%�R��G\�1����M��?����G��H"�TD=�����l~�q]+E�Ss�q9at���	�&����F���&.����o{���o��m���+"3*���M4��W�"���W��7���z����>��y�[�ldS2x�\��i*��a���J�3�%W7><Q$��������c�1��GXW�d)��	Zr��A:CZo�ja��������`aVL�J�E$+5?�(�k'�Lu
�.���^����`�u�-�����#��u>&����|/��L���dri�^���N�[�����|��e�\O���63�\�?��#-�d��
}f�;w5��7�����T�o'#q��0�����<�
`l<
�� � ���`Kc�t�X�9��q�RfXAQ �;l�pnrm_=� aA8�R�.�$����`f�S��9����>S�A�� H7=��6��3�u�8�x����
��,�-��eE���KC��.<G�?���Ih����r��"����'��T(19�?��i��~��O�������4L+����G�4��?�K��N�TLR�u)O�~����t�L���l�#���!��\���%�x�����X�H����}C��w���
.�/���c��s������\�	�7),��}M�&��a���]�3�)T��.U�����u�'�P�c�EF�"�*H��M����k;��aW����?V����ZC�y�X��F���KR\<LSa������5,��I�7�'wX ��i��Lq���2�Ab��H)�����f�[;���a�r���Q3�k^�E:��*�2j��Sk���R���������&b�_p���m�e*��!��*�\�O��������z��9.[��[d1��p��.w��ZT����|:{��tM��y��F��	�bS)��B�D����P�w��{!1W�G�B������u��#.q$SXrx��tL>���������w�w���&��|��c�>nb��|Y�**(��1�]Pj*p�������U�v������S>���*4�g.n.u����v����mB����c��-M	���/��F+	�C7�"���n�c�����hTFSWqY�'&mZ>9��������������>�N-��._��'O�*����Y�����H��+�At�U�����_�?��I�������\�?���LO��L�'�{
�*�!��?`�$�<���z�i����vK���x�
"IK[��Q��b**�*�z�(f������������������+Z;��b��H����$�s��
����l1�;�8����f�ZO+%d�4qF�,��.�V�����U��n����=��3^���g����B�:�/�IXn S�<������~>��ke���i�aNRQ ��fL�&a������u��ci:��m���w�9�[^������������l�5����M3��Z�hb��'\�N�����l�
����)�,$3�B"�����Y���Q�uaV��$M���N'��������{�s�\�>�k�1���O���jJl�d�������8�?�C��0We���U:�3f�)��N:J�Q�]i�R�/9�c�Z�
��>���z���6��d�%X,�~�^{K�!P���iTY�/�k��?~��(x�^�|~����
[qS��
��Z�����-$W�kN��������"�L����gTI�R��g����H�@�q3�/G
�7FQD��)eJ��{�/��f���IJ��
cT4Ymi�!y���N�o�[�j//#��o?o��V_��n6����)�j�-��������!��`.�����ygC@��~����\��<nh@�C
#���k�7��{\7�_<�#'��L�\�+�?VbM��>{���{�
�"48����d�*x;C�'�M����[�B����5��}>d�������s)��>(x���~��� ~��9]p6C:��W�-Ge��i��O����g�3���ps�Q�:s��8eI�~>�!�L��G�W�z�����5&m�'�u~@���R�ir7�_��h�������#��/�w��f�\�zr�����������a�}9�����6I����?�{����}TL)��h�
��\��|ia~,��U�<P	��b��Rj p m��������8y07d����~D>���|����V�"�u�v�'���"'����O��O���vm��+i.�
���j&�������?/��6�W�>7���:Xk�l�%g~���d���:$�������T�������?o��b^��������A�7j&rw�s=O���9�S��K��l��o|���z�bD�K[�����������`���<���0�`#�kG�
�_
W�/.; w�H�8�T�3%�"G�����`��������l�f��u�}C�������Gs��?��
�<Y�;bk�����"�&�HD��l��+`5��?�S;�{�=��:�(	�AQ&N��(N����i���9����o���p��W��t-;.?����x�5;I��9�;�X��R�9N�8��p��Y)u��^�o�I�M��@}F][���bvU������"���}>C���������������t�>����E��sF����b42e}j�c�{&�I����"s��=�9����9+X.B�o��z�6������T7�����Js��C�zV&��_z���MW���b�D�q��G'�R������;J&q��� >I��yo#*A����+6E�R#�}S�Yc-��+AKy��%���)�
���k�$�J��*#J�w]��b�<$�\�H�����%������c;m12�����@2��H�>�������x@W!��
�<�%�b �������i�H�YDS�V��&�D����w<���i5�2A>4o/�"?����L�#e����v
�mn�&#2Y�����i��t���|>�d�S����{S���K�f����1w08���;��Z!���j^
���l��p���AV��������&�����8��l$�K�`�R����5u�������&�������hGn�Q���������o��o����\��m-%p�)G8+9��M/���SU��� 1]�!���}�v����t�8�6+�Ky��f�����CXX����L ���4�S	[�h�B�l1h�v��&��~h�G�#��SwW��AT#�Y����-�����/�a��:oN.2�EwBr�
8a��C��<��K������H)��cQN����5��H������d=1
���/@0����� ���G��c�D��lE���X�&1.@-p�6`���!x�q=��S�[��(2�V�����������0F��E�^�XN,�)�h6!+�X�C�?�w�����b���A!0?�%�F�R
�W��pT�boX�Q�����^���+k��g��(�z����@����=t���ir!�4D��\�����a���i�&
h2o��a����?���HE�-$53�]�@��~���z���4<���Qa����;Hcq�'V��(�����v������Q�M�h����^���zV��T1��B�����a�����!��\��/R3I��#�����tj�ZL!��Vk&�lf�%���ZLkY0�+�S���uw'�����g������Z�6�q�8���Y@�b�a��c{�����"��,�����PK��qya�pc����������a����F	'���M�=��bT	���g)����Dtj��B�Xp3Y������*u�G�c��;v'���n���|0iHAY�������{�����c�W=�g����Z��*��t�?�Ow?I��-E�r�v�ut��T/�!��:E����+6�������7�s�~�Qf����[fc�p�p3�T"r���'[��nw��l���z��Lc��e�����u7B"�H��T{�{�Cz*���S��%i62�����������!����S�����q0$�\WF_=p��Q]�Q!���T-�r`*���M���%�j��[l����R���C��+
�2���D��	��7F�MJ�����`���%1��}�u�*�����p�&1���!�+UU=�"�=�h�����[HTfn�@c�$��l*_������BFk_�c��!�����O�SO��a�z��ix���Kr9�����`v�	M��gy�����J�"�����������Vh��O��g�#��	8�7��!���R�|W����M.��c��E�:P�>�V��e�E�/G+������|��mX�.\GZ��������O��J[
�"�Y���58vkW��=.,�!I���aI����@���!"����\���=y(/����"-13D��d;���jTB�����cQ5���k�iOC^���C�R���@e�$�5$K���`�Y�J�x����M_������g��dq���X�ED�(7�\�ucw`���s���o��'�h�$�Pb�?��9.3^�4��A��$������5���������Q5���d�NrQ$.+%��t (IVJ''�B����E3y�aS��L^6f��/ J���B��{R����$g������|O<��I�Y1
��C3���)�T4D����S�	�R�����k�S���#�;�N���PR��4��.I�Z"�2\O!^����#y:v[��5��S��K���.e��O�5���X����PBQ�l��$x�s����I6��K�9l����s��9<{B�dn��F�xl2�q~����_�����q]wOC�=�?�R<���M�:�	���j@9�G��z���]?4������2W��D��& �B ������	�%�R���`
Q�E���xy����(�j.�y~
Xm���E���tJSY72�	:����BT��t'h�l���:�% ��a�]������F������e��zJ!;w�&:V���v�����sM�5�#_*n���F��RJO]��:pv��
���Xl�C��vQ�ydUA��9��;n8w?I����z�<��>A������d�7�����u=!*Y�R����9l�~S:z n�o����9�,���\�����E��Ar�-)`��"T��-��"
E��/K����.�N%��Fd�o������O�����I�&p��`��hqA�NVp�	x
^��<��u��"b��&��a(�����
P9�C)x��)G������@U�d�$��/��
�{��-M�0��W�����~�5��u��qx��U0��l���m����/�Bl_��dz;�Z���YU���&��e�\��t%A�����+�jT��n�=7��W���8�_������`j�N�P&8�~������!��-�*������c�7a�1�������,9�
Q�!���$�7��v�������"�������M��D�����t4���J�w���#�����������d�1�_�o��A6���I<y�K�uw�O�fW��!�K�t0<%,����?�
MF���Y�l%����yw/u�����SI��TTM���&wK��W�B�T�!��Z�~H���.�2�I�cx�?j����\��X�K#�
&J�d�	[`����1�r�sx�x���LU��HI�k�l);�	����y��_/�8C�sW9�������E)����5����HCp��;�x���Sw%=rY������}9�����[�<��'���*�87���5�~��V2+oz`0>�0�p�o��C��.:M����E�0�!����S������3���MF�e(M]+d�G��^p���bh�7��MS��>��2���U1�����2`��HFx�ks���G9h��)u����v�t0�\�a�s�-���-�vpU���L:*��^i�]{h��_���k����t=^��a�Nf�V������H'�]}���W��MwE���Fg�1�r�6��������^���4:���(�z�Zwj%2�F���Q�v��t0G^�����\?7��������]RQ.�Kakp�w�����Y:D�]������X{�]�D���rbl2������N���42������&�=�I
Q�cg)h%q�h��,�EzE�X%�|�C�����J�����J1����^�ca��}��9�|��t�`d������v=V)�h�	8���GO����b�s%�]x)\��:��O���$����k��V}8���lF�
����h�)��9E�T�(e;����V?u 4l��O�*ol�F����R)��n�_����C�=��w��)*�t�!-�����.�1T`���&[�^�b�\/�u����"�{�k���9I���8'h��{5p���LI�R_J����*�tq�D�d�{����N��b�([N���Fo�Z��e~��>?S��Yy9���n����e
���P�����X[� ^��������������I��UV�q(�&m:���J8d�rz��S����_��v/fF��|�����`j �'���[���;�������o�BwEF�6Q�
n	IF�n�^��|������Lr{�������6��qzAe���<*DFx��[�=��%"��{4b�{,�`���G��xS�����#��\i1���������y�~��	z�U�QN|E/����������@/��H�S�u	y#�H������S2�h,nIix���,�����T8���!��*E��wJ.3S3o�������AYtv�.�,c��T.ot�yHn��X����iOQ���aM�a\�fo��DCi+j2��v1������_�}�������c^����	fK��qt���������������v�zW��O��P��w%����@ou3���4�n4.���O��F�I�$$6�R������MD��ta9��l��
R^\�����Y�
Vp��k��Yo�8o�tr���p��y!�`?�x�d0���(������n��Bo7�<�'�q.+�j���s�
X�'�;���`~���EqZIQq��	�S�`��_@�FRO�R��\���H\XOGP�[�(N)]U2H�=����S\�H�r�z�	\��DJ�t��3�#>�
Xl�t����~q���\o�-g���[�@C�^��9�fI�Kx<S��%�j)���y�U��S���)	,E?L�*%������A��e�����t�?�]_����v�����T'�TJG	H����Y"7��r�_����}�k����`��&>��������ZB�F`�eJ�� M����F����R�OpVo6#&h��c}8��R6#^kuO!K�d =$��
���b����c�U����=s%�oy=�A��g��+e��o���I��vz6:)�����4��"�$�*��>��%;)b�������b,&�=�k�]w4��r���{��UT��S�5=q��_������2o ����
|���s�CXM�t�0���p���th�u{zvGz���n�O��lO��r�N���Ki��N�+��T%Hh2Hn�4�'-r������4�����>�������_��gDF@�s0[���]�qc7��l�_o����\�#�"]�������1��M*���X��'���0c���R�����_�O�+$��4�nZ]��`�Yth!�%s�|����uv���N+�������"���9��
2,�:KyR0���zw�GB�*��P@w9�K�Un������*f��O�����i4������SV���Ii7��&j� yvu��wr��A)^���� 9��P���~Ho�.E|�]��9y����
T�OD�!l�'�f�������ys;1-�$��	�����S��s[�D��?���xz;�[��q�p��$w9O������&vzWV_|j��S?E\�LJ�w������2D�/�Y�����;���*4�Z�'p��.�8��P[Q��4��0?���	��^B�Ie��V����G�Y|�xp:��8�.7��b�g���P�t��Dy���H�>z���x#S���`�nH&rRL�I��v�+�Xm�k�8����l�q���
\L��R����0S�������t(4Tf���j��C){p�A�;n�������~�G9�������M���H�4;!�5�0j�|.��|c�C+I��0c�c$w~S���E��|��sr��~���Di�:�@	�@�&���qm�)�J�f��Z���|h�*%�Zk�7<+�zZ�w��z����J@�5�(KKU �&e)������R����R�"V�;{���R6S�3zl�	�cA+���Q�}VF3���2���
eK�"����o�\I6���/v-�u:��];P;G�!4��E�������(
$*K��S��f�a��`�sr�`����S��^���XJ�{?��"%�����W�m��x�����3If0<�������*���5r�%X?�v��B<I��\� s])�D���^���p�q�p�%������P�F��a=�������,���g(���
a��{�-����v��r������g����'Z�]>��?�22��I[�s5����^�
��8�%��h�8�_�v��+z�K��:��0�8���NP6����P�K� dh�7f���I��^��]��O5��A���P15��v�6=|��R�5��\��Ej�����q]P2�7������m\w��8����K��R�A�N7�C�+/f���tkv'7j6`��*�4?�������vI&%FJ�r�����!K�.���J�x���r���=�@�����c�����
�V�,fK~;e ����\�G��eBv��e����5�aw�����S���v�~�?c��f����)h-�s�m�"}(�k�5���M?�������R{t��}�2���{j���G�rc*	�:�Uz�J�������_���{�chln�>�����
�V�a�3��b�\��f�X��D��6�?�D����1�v��VB�Ac�fC��3�b���
��r�6��n�%!���cL��H�X�)��3%��Z�_������7,���G��1�H�j6\������/�!W���p��������c�/-���%�ht���v7�[��&C��y�?�}:7]�[�{�?O��G1�aiJ��
9�P	�	�,Y7w�R���~6��8��Eq��8��S�KY��|��O/����`��������Eu�B��,	�92s�h�)��]=e�^��<��3�)���"��!�����:YK7��R�]q:6��X:��n���5E����t;Nt��O��ds����H{�3��G!9��9A����A����k�NOm}n��nhC���6Ce�g��K�;N�L�`Rm/Erp�^2��FF��&�s�N~��@�tC�ua}<xz�:������z��%
N�Q<��;+�x�bl�]�4dh��~Pb���0l�h]*�c_'�8Jd�Sv�y1��{<w��=4 |"]���h*�2�(�+�Day6nb�Y��s��Q�>�9���]p�s��I-�G��R���2Sp�Q�	���m=G+��0��*��t�����o/�������/94�
����E��F���Ph�N���')+�ru�rCgI�L<4��*A>Hp�=�YN��<K���K7>���5�e�8���
/F�������q�����`�O!�hq�y��2x�S���m�������������?m���� !���4�>�6�������43>����?��^��b�a3\{�p����Z{���Q�H]�@�i��MK)B�9�]~mGk�wp�2#�������\p���R,����^��n_?��m[��c^!U�C�4W�%Ye+�L(�^,\@-���S�f�?4����vW�h���g���#�k2��k��|�����U1Rg����:��������S2��w_��h������2�n������!~��{�R��v
�n�D.$��R&Lw��M��B�{Up���wJB����|P�#���/�9!��~�	A%�S�~�6G_��"���	2g���L�����I+���.���G������f����'3����#
DUL��0�(
Gh�9ot�����b)�B���l:��q�����^f�9\z��e-��ADV�"=���?��^@)���v���:�e�^� �tI�yJd6���$���&�,I&�;h�#gl�I�'��U�4s��sr�S4�a�s��c��}��z�)���\�8`������gfvK��vp��$��v
�X�C7��nx?�����N���)���� �l1l���M;�,UIXj\'���#
g����-�O�q����2��.i���fC�`6i�RiZ�*LdR��b>��Sa�K�7}=~�3�CetIT)4��$A�0;T<O�\?�c��wf
�4���+zk��F�6Xx"'M�~�Dj.�����o����[{��"�hP�� 3\��������/�����������A6��������W�}������f���QI��i�/!���T���H�4!��B.���������J������/q^p���R>�@�E:�D	�o�q��e��-���#���p��$��4\��_$g����:�sN7��W���o�9�	��M������d{�Nt�����_��<���GI(������� ql����[����l��q�u�3�D<]y�
v�Q�R��>���z��7��7�(�&}������;w���x�s��.e�>�����ew��I_~��&]PX���t�*E��_��n)�r�����}w��J2�~'	��q�S�J�3���ol1e;.�DM�/���w��d���v�;������D���G���m.$]�a��\�j��c�?���#�y�.^
5�Q���5v��Dm�,���v�ZOAl���V��~����'H��:`�����hd��`���
��~�!I��S"���	FrA�����$�����zd���6=�t�A�[E���|63_?��� lk. J�K������6J1���$.Q�mt,�����<��"4aA��9�}��3_���~��OPK9���zx�Z1Q������v����?��,%�,��#�T�oUn,�a����fr�(@�VCd���������c+$����V���$t%��<��L�.����_��
��1n����#��y�G�)�HQJw��&��u{�;(��H����,���P4$��rs �������7w�������%Z�L%2��67����&^�4�f��N]����@����v1d��p,^�t���V�t��n6��cq ����������`��NR��@�w7>7�6�6I�ZMc+���0|�����~�\��W]��!�����\�!X�rl��f����]�i&CLN�V4���3;~�=��#��X5{*�_�zo�}�^��	|���_�d����
�o��t."�"��NZLd�}<����-l�a�>����<�����7��
F*-�=<8���!�%������`�,JZP��-��������mW����t���G�ZF���S	dX0i-k����~���^�$�)Y;uJ���4eO�(�rs2���������0DG*h�H��5�v�G���a����.����~���#.�
Y;�"��u�QHGXkg�fE��nV�W���Jig�HD�F�p�		�(�~�e��t*Y �2��2Bk8�)E)���k�g������v���2�9����:�3�{�E6��5��������B�x�����3�)��r�#NZ�X��_�b�E)K��gX�y�>�k��b�f���c�O�S������),�N��t%�������g�����VoO{���J��$#�2��q���f�3�
�,���=@��%��W�6�`���7>LP�\���������=�@�~����>4��!���&�WA�ll��
�����j��]lX���(����6��J�V�DzgM���A}��|����V� �8z��|�������b`��?����4�E�|)yF!�<$l�fc��;�_^r����j���@����� �����+����G��!���<��='��#RQ�&���o6��~�m|"�e��$�R�~��Z�cI���R�w\���I���4Y6�ij����,�\�h��~�N��0������w��e($�(�����f6�+������p�:7�d��	<e�Yf
4���Ol�A��O�qD9����!a����	i��d�O���	����k�@�^����c�6�������wM�����,��3�nT��G��5$���Q7�[Z
���gs:��h`��+\'E�
0��Hl�j�@�R�I������K�����c��2�� �S|�Mwf0�$;�o������%�$'��<���\l�g�X�
L��t\U,^���w?�/�8_m(���)���Kiu|���YL�980	���'�"��G�����Ao�R`!�7���|����E���>0)���
�!���R^��2l���>�o�:�LA)�0r+��w�}��4�&h�7����K���Mu:��g���T�N�����7V)�l@���m����:3�J9�����*�����0�����o"�t�8��c��J����t��/��xy~z�}r���s�b]�c�(GzkC�������Z�;3�Pc�b3����v
Vb0�N4\l�dA���^�R�;�Ph5���i��#4�O/�6Q�g�A��.��0w�����|Z��_��Vn:�e �g#v�v�X��R�����G}�����r6��S�n�l]@d�),CZS�]����.}�<�e��k�]�t�&�r�l�e<G�*�zK9^�*���8��m��|�(�#���?����������H�� !��$D9v��~���P������O=�s��G��8�����(:$R��iQ"
�`�!�I���2v����G���O8f	�b2��C8SXi��"�������9��t%�����ws�?�v�z6�S�E�o�	r�aod/hc��(�>�L�l���d��f5��D{�W4�����.�
Z�.v�	Rf���M�D�� ,9��w��x<~�����n�
S35>�Yc/Ox�Q`���U6������������-�����qd�CK�ia�H�D��HY����'���g�W
���r�lVN�X��8� H<Y.��E	*��(��,+����a��,[��l�5������\m�������/�����?��"�'�K��0�����@u����{:����T� �aI�����QS���������H�0~����������+������.m��m_fC����hA���2"���]F.m%�:�����w:���������z�\1.���p��
��`H��������B
���e�j����>�L�ON����$G�����(��.:����G\B�N�I������Ww$t��|z��F�\���yg��7��o�����~�`�rjJw�'�m{���F�\D����������m���t�����`�V�|]��8����5��<>�6@ y�����	y�Ah'��^�[
��"�Bj/k����N;u<������WQ�/���j�eY}���x�c�\[&?��ml���=��������_�K/�0�8������99��!~��������g~oGHZL����
p��Dv1��GF(��\8����}r�@���zpj�i�`V�]k������F�������^���z$W�hY�5��3^��H7�������sz/m�Myg��
B��8����k��^|�y����{3~i�������,�%!N��bj�o	b��
�����6����!��"q�b9������N�&�N���J�p�������l1��{��>�5Q,�����'�=�R��Wg��T���f��g��R�Q����2��w]B�qL�|���l��%) ,�����\��}��n��o���������<�����F#S��+i6��9��_���#�T��)(�g�#�]�VG�`�+�6��n[�]{�^����z3�w�+'��o��#[T�<K����|����#ul_��m�������8�������}j���.��}@��C�|��2��4�?���]�U����"6��e��������1
���s$��BD�v����&)�� E�Y���� p&����6��#���L�4�$8I
Y���j�gf ������6���6��=������P/g'9��[]}��C�
K)�
�4�YJ����[4����[9����Ue�l�X�}����]g�Oj��r�<����|���2�,��=#6�e�1��3G$-���]
��s��U�����:�����N#wpA��)Im���i�A8\���&�`Z����
�t?�Ar���|� ��i*1�p�O�8�)�S}�?�H���<�:���V�9��P��\�Ac�\��`�bX\�F�cT3�L�H��j���������2n[jh Gx�����T>t �+���^�,��,�l4Q�����v���R��g"ud������?��c����v��?|��/�;�������p���0C��Fj{'+�Ht�����������y�pdOA�\��4H�8P��l�� ����=*��S��1������"�m�ht.m���r�Z����ze�x���^^~k��s&,��M�s1���j��K{�����PTrtu_�J���	r�&�I�f#������o�av8��L�io�����i�s������20��;���Q����/�A������dMD��0�e+��/"�$�R}���.�)�k�K�~#�T*���N��e!p�@%��%������5?�\wn���9�����FU*����(���B����V�T��T�_I����.e*9���M]B����m�2����#��g	0���P�MO	�����I����0U5�y��S��6�9�,�dj�.�H'HN���+=�]���� ���a|q��-6��+��Q|Cp��Ks50��h��}
.�������
8UO�/���?8��rP\USz����Vs��\�$�����&`���g�����*�����K�41��~��e:�F��f��-Hz���Te��dT��W�x���p��i�&#����R�x3����Y�p�(+%�p&��dc����U���N����@HF����js���`W�����w��W�$��=��W��g�e`_4��Y�`�>4�k���q���~��A/y�e-�]/�S>�{99[�c0�H��n�M�
�����n���eB,%t.+��7{AR]bI������������r�/o��Q5��=�*74?�;z��9=5=�~{�����03��K��d��W�4*@Pd�0>�������������?t�b���#���G�4v�l.��� 3~�A'�?U�Aw�u���T���L��-�B*&m�D=q=�������~�9��L��|�"&��MR�S�2x���qlN��zY*�P�'��1��p�hD�����NM��04��@���1h(.��Z��Ia#l��6L��=h�*�?�kF�}nH��hI����d�]������{�L.���{s����[�_������(�l8/�@�1e���
[��l��}�}j����q�]M����A�(�5GB�ZM&�gke�]�G������*}x�r*1
9�U��3�$���D(���?�>��������W����2%�/�I3�7�Q��$���U�?���z���,}��	�
��10T��{�W�"�e�v�8v�����(D�\5�f��BQme��
K��FH��3Y����K�|��2Fz
1��@�"
��,���0����}�O��1���a`}���O�J=�	��]���c��L�7�m��=8��7�T��@<���p������`����y8��&�4��3�T�7�>�)��i<�T/�yx:���B�
�ZG�x�?�
sk����V�;H��;s?���&A*A}j���X&��)4�H�]���X[�������������7}��H�AV2�QY*GY�L
Q���h����Nd���&&4�5��-�#�8��M��)�1��f��Z����8�����XjC)�;�����p�e=�nx6�@�^�!��5	/�� �Jh!�����,~m���Ks��V?!{����I��2Y(��n ���2��e��K]������_������]P���8���/�`����!Q+�2���LW��q�idn���a����%t<�'��\&05	�F8C�p��7�����U��~����nb����1��$S����
��M+Y��g��>5i9�\�:���A^���"�i$x]_N��C�T��F	f��U$��pmy�
����*��SAI.����������f�l&���c���\����k���C���7�B�6
G��~H�K�mK��	��}s��{��
+�2�Skr{u�w`�^G�UA�@U��y=0h������}����rjIWl	^��/�=u����H�(�
��a�J��
�����u������i�HUi)dV/�����6������VT�~�"���Jp���"�Dm���4�@�_U}�;����i�c�}I�&	b�2l�vkJ���h���������o�qJ�f>)]4E��+�\3���3�s&R��_�C�����m�X��a����,��������;��6Z�$�R{$}��_��r,k�Gioh��������?WW���nw�f
�P��a���O���I�,�E�������a��Tc�o}�����d#fa;I�N�[��`�l���2�E<>��>��"����cL��/:�<�|��k=%�!,L*���7�V��s��-/�|����B��0����x�iJ��#U&X[c�Z
p��\�dH0%�#2�q���ruBFa8���p�,���|��kZ l����=����"��s�k��n���%:������tCw�St$	v�<X��N,����	�F����_��w��%O�=
FU��Av5���T�	���2(�PTM�;
��B��w�^8�6Brhp���@�����`�"���M�s=��4Y��6��f�����3����-W��Zp�"��t��JR��|�Y.�%��a]�^����U_�����' �9y�����hU0��3z�\v��	J�)�1�)�&W��e�e*������Y�})�W'
�Bv��w�_��#!��!Mw��(�����<�C=TO��q���a.M�=����-���4����d�f� ��B������]�*�j8<W�v��eL
���x�Nzkk	��m��w'��x3����~n~��>�����i��q[N���n"�;i���C$��������R����
a�c�m�6$��@�Y&�o<ch2�D��4�+!��2��f��^��c�$�LM���	�U����C
�C��.��]Q�/��;W�B�[Q������2���4�����2�N��Q~T�n�S��UJ3��3�=����K�����ie���'\���ix�e�=�^e��`��DH��a�t0qa��T����dv_��T�������hFh�`��*��%k��oE���ePXn�9Yu��������������������F �w��>[2������B�����������"��[�0e���vhk�A}�R�O.[��E�����[}�X��C("$M��b�m�\j���s3N�sg�z�A�}]JJ��s�l"�����X����Tk�Y�)����
���*Yl��$2�\�G��'�=��|G!y� ��j�c�jK'��x�,eU���y�
��,�7� �*��@D^��'A��fVA��DC}8@��#/l
���Fx��j���f����������	����0�z����p{=�����ex�`p�����$|�H����)!�Ba���+�����ye�o��Fhh�u-�w&���)20V�N�M;�?��������a�O�0]hd�-��>t7�`m�����2	���[���Y����N�/������C[�G2%�5�>g�=~f$�4Uw��c�����B�� L6�P5�^������&O9�ju[m���6��z|R��^������Wp�t����c��'@���r��@��P�;����3�BO`B��QXI�q~n��7�M�����.M�
9�&�T��G���i����T�m}p���Bf�e.����Ffq�.����=j�Bn���?�}D�z����sv�,���A���U dq��B�>\.�y�5��a�q���Y��������y��q������w��S�)�.F�A�[�g�`��yK������Rq���P��oO��=ZGz��k9��������)+(��>�R04�������`���T�?v���`k����5�?,	�������Ic�@`����a>7�� W��V��}gB��7��d�QaW�Kl�����Q5��cvQ�M1:T��}�;g�!E��O�Zt9�XU�	W�"
� ?�3�����Mu:�c�mv3`� c�;������)��q�=�&$��������j�KW_+^���Ab�M.�����@����wF��#��A�rT�*P���m�)6��P����}������4�$�����JB�s����^�* ���>��8�r�ta������r���/�Bq$�`&���\s���r������df�K��g�S��UiU�yb�����PN�j6�p8�&4�<��}��
� ����>j!���}��N%J?f D�0�_��g�`��e�����RG!��Xe�
���./�n������������|���������(�@<��C[�}xQR�U�`�3�� �������?7��8~��$$���g�j�O��%�����	e?er��%R�����J��:����x+UzK�F���.��(��T������6�$�W|��Q��Q����FhL�xf/ �$���!��~+]M4X���8]�9x"H�zd~�=F�����P��UMB���S���JTrj4Nc�����x�����LWVa�nL�PN�����p(��*2�3xq35�}��� �^!L��N�����05���A��U�P�Q���\^�h�*C9yg�N��B���d
7��>����s���"��Xb�j��N��!�tS���t��$�q6aHFP�!�"��U�R#7�� 9f,�%S��?������
���n���z\#'Q�f�	,�w�[�>����������4���������X���W����{���y*��6j
Xr,Y(�'r��b���)$�&!�����5��G_�������p��w'�q�tD�c�9�%��tHH�������|�(����^o���7+W�����0�
���c1_����#)H���<[~����c�
��W��7�S��_`���\�b�{����t����QSq������bv��f������}���Ve`yyQ������%6"f>�e�=#��sK_���C�}���/X[�Y�,�nwPIu�0�i����X�l�#�+�4�cn5]���EmG��&�N{��]7�Ag9�hG�&9�E-��j�t��`:P�^����dIc��*$0�����	����+RLQ�=����jz^�_�����_p�G��IR�;�|�p���v�7�~��]Az���s���pE�C��l�K������5aT�U��[|�1��~j����>g$9
g����,-&���o�rL�����f���6�L&[%�ksj�<�=���D������n����>���k�W�f}\�����[�Y6K���[��-dB�ip��%*��	���CD�P��0�H���� �k�3�B����[��k����X��*f�2�X�i�
�=�6+��;�s�3/e��9�M����8��Jd��
}�gw'h�*�"�e��<�u���+r�<n=yS\���4Xw���
�R�^������i���g�CkQ��\@������Oo��u��N���9,�V�Y����,m������}~�u��}s����7�VM��X��������K��1�J��&�Jy�o�}{=$���AH�uZ��UH�B�}g��+�������������y�;���=��t�����j�YiQ�����Jw�+c/ %���]�������o`�Q$�+CJYGp�~��9��������9T��!	\*���_�E���_����������	�i����)�[�5�o*������@^++���i�=����O��?������M$�+j@�����":(�)|������(��{?���y�`��di,jkk}$���D��_��B�*�i��gW��S�yU�<D�+�M�M��K�C���������\�@����:�o������YE�il2�D������k�(i��;M_���W��&���������C�#b�%{�W�H�����������v?5z���1+��Z�����^ ��V�P�@�����n��������SQ�\�)nA�*����ey.w����$�lP!*2]V�s���=����������Vv�W���������$�:CBH�@�|#s�<i��L��,�J�:����tJ������������K���~�����z��iz��b�l&�F@l2eHZ���r���=SM�z��`����9��
FG8i�eC��������}q�Y���f�G���)�@�2�X$����<���((��Hs�`k��}����+�#y��9D�xo�e/�,��������s���G4VJ,�h�P�Kg��T�W�%\�|bHL��qB`=������������_��<�3�T���F;�2�1����6 �.&��\,���~��~i��D*D4�i��(�A��m�
*���R��Cc�m�%oM�D�����\�G�Y :c&�wu$�\&i�6��H$"�p��H�Y)��E�2IC�F .?>ZJ�|y�i����_���G����M��$O�,>��V���\�[MP�P]��J�6��2<=�tYr�O�A�\��v_>�~Q�����������#r����)q-%X��0��,t����)��W���2���g~A��(`\�#�X	^J��Z��L-�������<�N]Q�*���@�%K:�����]�
�R;:�����.F�E��\	S�%	��/�G0Y�9��n�dx�)&�"��2]�;� g?���B���Yr���&]��K�}q ��!EX���\��*��P��0�<��o�����r����{S���@�z9�_�OD-|9/l���?�{����r�6��:��Ya~m]E
/ s�#:j	~�\����X2��D�x�k��K�"ul��W�JK��Es����`�~jAK�~���f�=DF�2���2%���>P�B|F���//�\p�\D+Gc���5C� �R+M@���p����c���H.�lxf��b
��U0H.<3����d�����sO+f������\�l*u�`3�$��|*����iw��~n6���n���C��iO��Ae�y�^>�~5�a��N�Q3?�Y�����g4��;�Q��yx)�O����{t��a���G��sW����|ncJ���}�+�c����e��M:�,J���,���F6�L�p��LS�I[�R�#�p�=��4��E�*��S�� ��"2�����e���1re���^B��R�'�JI ��= �U��w��''������c�6��o�����=zH���6��f0��l6����3�����
�{�zit\i���%��ls�  h���l�0��7��}�~�Q���H���)H<��1w�+���+&J�����m�6�/���Q�3��FZM��_���ejl��������Bu�[���1�5(	�&����<���D�����>������i{HvI���w��`lsu��4�g�K���E�m�nT�#}�������;����a���W����>�nA�(Qn�-9
!�����;C>B��>i�!$��zsg�v�����s��j��Z����9}yf����mJg�hU�����5
Lw��{f)I>>��[N�Y��86`U�����gL���I/%�Q/�Y]��	m8�AMHV->v;�7��#�������D1z�M��!�L��8l�~��ms<�}������}��!�Q��h2<B���f����
��D��+T@�j}����`
~�Q�v0��N�����"�t�OV���GT�m�T��8�z�~���C������;�Z8,�W��R�s�F�G���\�4J��R4���e��Jc3p����b�P<-��Is�&A��b���������)��d�2�H�M:�h����V�zVd���*�����A��/����p�7F]�V�a�����2���i@*��t��o6��"�0� �u��c=�=���'\���	���R�fm�	�s0�nXH�/��6+��2M�t��'dCl5�����s�{���Wv>6�G9���z��4�#�T���k��+�Cq&��Bq&K���?���^��p�^�!��������U%���nW�����` ��&��x�)1[�K��%9�E0�;x�����������q��'r�A�-�Bz�L���oT�V���j&������W��1��oQQQ�p2iN�W�����;`A�T���5(��(�`��t�r�<$c����������;�_wH��HY�(^��9r
�p��Xn���3�
%C�
�j*jy%p�1R��-�����Go��dF����a�n/b�����.@
:���r�u��^��f��9]�M6�U�!>��Yi]�
��Q���R��Z�9�1n"R�DqI!�GLp?�HP���~2?���~4�$R��PL��4�X!8B
��Y��rD���[�tF�;���Q�}�d��X�Qw��ng��J���}��;Z�@L�-�Q4�e���A ��F�1km�����<'w�F���7%���*s�m
��&F�l�x=�#F1	��;��7
d�b�b������wMa��������x kH��9�t��@"D�=8��\~�S�"�]G����*h��?)!#![J�m�9V�B\��D)R����� #X���|*�N�����������"m�)dx���l����w�J$+`����C�
�p��:���U�{��l�]�im)��i��I/��,J4�u��>�!!�k�����fn'�u�!������6������\#M��A{Z@g~�o���&����*<���W�\m����[��Z���z��s�	?��]X����s�J�j%��+�JY��,`9A{�
%���k&uT�7bB*G�R��oe�<;����&�"m|
	��MH-�8j�*8FJ���9P����#`o����/�;�q�$jnKB�/������}jO�t�<���NkP�P�rg����yAaDqsq*���R����;�6�yt4������Q
W��W�� ����x�Kp[�]��������%�+������Rq��-���?��}���/6�-X���M���,8n,;�2���7�	il����9���S�E����[��|������ +��RT�]�s��[����~Xo�2����FIl��``t����@�:�F8� �+CKQI�s��LT�w��	�&Q�L�F��E��d��M	x�
���*2�8h�A�Q���CE�a��7/@��������������1R�Q����:i���N+5�PY�U�B����W������y}t���l�kmF�NB
�T�2;�����%]�9�V^�~�����<g�i2�Q��@�k��gj$���W�AV5z�	=E�c�u	�WS���CB���Rvp���f��7m��oA�+)&�7��!��b4)�"QY���1B�,-E��6~|�������M&��Er��r�$���B�DiB��?}���M
vz[q4v�\R�,Z������.f�o7Gn��?��q�65<������N�W�y���F���TX��d��Y���E.e���]K"V�[JB�~Q,����V���`�&��Z�}��Y�����s;�b��7)��Se�J@%��id+��WTy��Kc��:b�<��{�	���NK�������� ��
��m����mH�x����i� ��
���3��xr�?x�3-���`e����BA��Z#�W
K<����p��u�w��R?x���DiH����R�}2�8�nm��I�����?r'I��s��#t��)&����C��)��yj6��������k<���M���d
=|~����4#W�S�_�}:�CM?�K��1S�BJ��)(?w�z'>�<��G$���y0����a����o���P`3���aqt6Ys�/�����Q��Py�=�R�	'��� �+�K)�v� 5�h��#���f��Ec�T���'�@d����*������������������aj$/�q�����zMp����,��cV�q5v�����r�gY����"Q;f]�KJ�n6������!�H�1}V�"�L��]�O+�0�t��A'DZ�jUx���u�r�%R����������wX����Y��f�N�`��8���Q��MD�9Yj��D�j
xf�Z����D�o�,0�,���O���3��K�8��L���l���?���J��6�T�
)!8�`�m�B����.�w���t�;��s�n��
<�)�9;bcg-:jcZ�V�c�{���3&�����%K;�r�JK$)NX��J��K�2>G��^���}��+�<$M����y��
�� �W�0�����XA�$�65�H��!%VD�t� ���]����:��r���,.wOj��#f�I)}M�Q����@���BJ0ic�@"[��H�����>����t��c�
�<�}6C!Z��f�T���u��a_��{�YZl��&2,�ea��c�����7���e���i�B������=QK"%�F���7�u��n��������Q+	J��5h��}E��U�����*3t����c��~M4CW�����,��������h&��qV��+
�4d\�����!�������S�T�����?y_��n1b�q�v9�p��@8HD����.���>�>�w^��i����U���Z�u,���eTP	�fPV��$��l7��y�>��C�) @��~��[l���S�3�V�o�s>���T`pkG�����/]H��2#�\��:Q�K����Y\M�m)Z��~j����i>����i���`3E*g[F��S#����f1@��*ha2�
mr<����t$W��rKB������#\��N�$�M8��N��>�$�J`G9��P0}������4�a������|5��s-��(�d	D���1M\�4xO��s�#Q�F�Q5�`N�k���#s���d��U"o}���(J��W���(f$��y)�q�Ii�QF��
�sqr�0S�0�����u�(!2�!&�Q���9�k\b<���}��[t�U��|��f����)"w$F}���Ky��r*3��������w�fncR���P�������������?�������_N��~EQw2�"���d��gEh�]�bRT��d�O���8nc����~��|L*L�����3��$�l�K���Hv���//e�,#�"���� ��B�)�s�=��w<0�����02�������B)��jttv����&���p� �`i�����4$����7���8`7����I���YJ��
I.[��������������v���Bj^�<� ^7�~U�l~m*[���<�u�^Uz�o2�������"����YC�\���?]{8�]�B^S��������^��4��
w?��+���y����k@&���{���m��x��t�`�	�����C���	��:�7@JXRX(}t�d��)���	Y���s�)��f[�P=������y��pn��~S=�/nZ	$7�=�`~����tL�F��=X6�[�_����c�i��	7%,�_�I�����d/���9�_^��;6D�Dk�Or=7i�%�r���&s�=o�GI�i�����K��������*j&���o. ���6?����sY��j�U���a<���X����"����47����|$����>��j3)��P�`������E5c,�h���KL���s�[$��l������2��=t���C}^:�4K���H���
�������h6���XM�����@
_T��Ba
������:4�������u��T����c�Z
Rj?�H7��0�Gf��4�>20��r���?#�s�n��'�b���`H;
���U���.U6���F�"�U�����$��3�W���-�����
�}=�����=���|t��&��f�xay����E��m���N�g�O~�������Q����&����x��BwL`������%;dUm���e�@������E�VcF������{����ms������u')������(��v,�C���8������`~<�^K��A��K��Aq!�+u�����|�^Tn����^m��>�2�Y�I���W�a<l�X.��K	����	����,(QX�&���\�������}2�����1=��;[S�}/�nE�$$�h3��Ca(�r1+j����T�	�~�w_ A����
�  p-zo�	��J��	�E��=F�������^L�n�D����=Dd.b�&{��'*�����Y&'��y��ilt'�+#�y7����q
�0SH21��~��?E�����l��4�i�nh\>@�g&�����^F�c�Y����n\<��g��.�ksC��E'{I���c�%+Csa�|{�uw���������W��{��6�B�F�%~,�J��H
�WL�Bvw[�����>�%/=dO'v��h
�#v����l��Nk^������`;����hp�)���zXL1;x��
7��f������������T2�'W�S��Ul@4��,��������;]b������DM|>$jp,[<QsaC�M�I����Y@�z�w;���v�}�2~�\H�Lw.c���*�0w��0f���hQR-�TK/]��>�T���M����\Fo4��n=R#�� ��1�-�r�<d������-����������b���d��o�y�8������F�! +��tA>�s?��(2__A$�R!����r���_����8��\n'����98R[���\LJ�d�T����y@6r�[8�@Qg"�CRd�l&{�21�_�����&�t��+,�i������d����u��R�\�P�����}���W
����?���_Q�'\���#�|��\�&n��$�W ��Q%��8pS��A\XL�_���^G�6��^��������,�qg��K��\z�����u{����'�s���6&��K���s���%�7*����������9����M�
�$�q"
�K��.MR��L��A���:~
�����S:�����$�&��@)_:L�;K.��g���zN����zw�4����%�u{8��f�,��t"h'{Iq��<��w��^�2��.���%�rCF�H�����YvX�$4�E
��.���z�s�n�]W�]�(J�-�I��'�������/���\����K��|�vl�9��j�;A��wq�G�>����������?���h>�,$�sMY"�6�D��JQ���W'����cs�|�~��Z�������"�e�������w��,'C�/��HG&)4Pz0�b\����������Ru.sr��*�[r����R��a&$ic
�9a+"r�Z�)��'��������F3].�f:���WU`n{w������z�&��UY���XQ��&��
E�Bk~�
�A+�i�9s7��(X���=P�r�����`�W���T�l$����[����H��c1tr�����@~	u����C;�|��c�7��Ek���@�(��~�B�p �We\�%�=�H	��s��g�����Z�E��G1	��K2����k�=Omp�+a�Q�M���[�����=�T�?|�w���
��lD�ITmtq\_����lEj�~�s��l.���R"����
��*?X��,N��[k�N�������~6���p�����������f����G08%��:X��2�{���r��������85�M8����D�����=t�����a4��&.���_�J$�hl�-�p���u\�"���G?o~~>|��m?�u�����	�'��aW
$����+Y��������:��O�\�v���������j���Kx>����5'~\��+�wa,��%���&��n�pU^�n�����2nR�E�s��<�H��nn�i���\&������?6���/r��I��M��50�_�[��	�	Fa�����_z�wc�������i�K�j~�]sD��C��\���%���RD�W�KZwp�9�O��@2���Z_&(�9>��B2���v>x����/M�<���������iB��Vy�����d�C�F�Vy�MQU
��_�����(����R�c����xQ�i�(E:�B��,�����}S�
pD����5�N�t��Xj?��~?5�9��7�����z���%"�TJX(9
�{�p���@�C���l��p��u#G�
���c&����:O����qd?�(���:������Xva�?4���Wr-��� �n��
��UP4eI��yfP�l!j[7�w�Sd����)�`^�q��_�����&�T��9��3z���<<t��T���i%/9I��@��A���-G�����w��<"�d�B	��JDE�������-������7������c}j���F7�4����
k#�l�G*/W��B���l[�R~�1Pe��UN ~&�X���E�z�h�o ���-����g��I�siKO� ��,y�p$ZB�oE4���j�n}_�v�a�������T>}y~1��r[��A���i?���f��8v���0�*9Bm\�)��^�$�+��t�ezK��d3 /�Ri\-�w�qL�;��^cq<G�)���������8k{GOG�,�_Wl��|&	��P�|*�'�����<�/�k3N������N/cT	�_OB��l��,7*�J	w��:���?lx�i_����z����v������JB{:�,�vyb�L�j����\��O�~���qYl_Ow�^T�k_��aR���O��J�������k���@� qY�����b-s$q��Y1d����S�{�5����������-�'��/+���}J���$���V����z�l��P	>���>zwh��zL�� 8K�r]�ut��[_�N������H�<!,��B1�^��R�\4�*.����S�SsIJUP����g�%zCHO��4
��k�:�����O����`��.zH��P����b��X�t����"�����1����=��scN�-2�����c��t� u��oO�N	7ct���t�}	��x+����@K��+Mr!%9Q}xu �Af9y����)D��O�E������^[0S���6xB"++��r$��Ab���J�b���������O#B���P���.�t]�5VV@%����eN�+���P
�x(
O�w4>|��@�����S��#����	07��*�����M3��U:������`o��_���]�������
W����h����K������%'�wH'���B�O�}����	^_FM�� U�
E}�FO�:�uw|���?pF���������;�@E��0��J�a~�?�:���u^��$�'�sR��J�����\.$�1����{�����h<���e�����G��FUAp�����;e0��.�����pV���j ��|*����AQ�w������|:�O����q�*h���;yAJ��Q��2�c����?�,���<�E���dKS��n�K����T.��[�S\B�����~�\e���d�C����Dt�����1�9J0��n��!�C$�w�^3���H��x��N�y���&��m,��63J� ��@6]�I��������L�A3���q�>��/C�K���-�:TiO#���%�8g�
?6�����L'�&M��������7lUBP�/��r�����������?W�vrv����dA_�%Q�(���Rz���$��?���f8�N�Zc7�X��Dy���H�6�X 
&i��}���z�� 9�j���<���J`�io�4-��!��!v:!����}})�L�o�!�.-=��t�#-
,��'�83��n+\�k��P�L�'�>�,�8�R��U�e.����K%j��'��08v��#[8���ua������Q�H�d!R=8�}2/�$�X���Y�)��Nc�.%Y�2n�������9`��4���<e��[����{G�Z:V`��L����>���~i6�<��	1	��H�����d�%��������e�![��m}|��ahX����)��PCC��L�
�-�(�����z"&&k��������8�AJ���t����n��������>�n����V���+NlA"q1�(�e��X
�8��A� ��)d�b>�M}j����_����4n-Rr�\s@|$�����3��<3Xs|</�8U������C�]�����`q��v��e�����n�	�f^�H��3*���5��>�E���]�8?��H����Ab"�|�����vC[�)�+�r)�3&�3x�������wI7���0�����@�n��m�U<�[��r����"y[������>��|��|m�u�9'pb7�QX������\Y��F=-�����"U�o��I�c��V�vW��(���*=�d�& `������@FsY?����A
��;��#�O�_�!�t�kRb��,e.^
�i:��7��A�r�J�{��jm06��eC�e.�t���]_O�u��D�����n���*��i�{>Qzk�iV��zP��(���enU�Ch^�)�j�N�1E�#���0��g�o^�(��)�]f�=�^g-����Y��[`����?��SSo~���n�w�y��C>�{{�9�_:M�h������U��d�SR)B��HwT��k�_ZLv�bfp5�>��m���z�!"�#'2��Y,va%e���M��y'|��4�� Dy����-�iQ<���h��2���,�MQ�3X|!>�af����3ct3��?g%��nby�CJ�E�^}�[U��+��Y��I2O�3�4��@&�P�<C!����:�@�!H_ �qR{_��2�<�����
*�����M��u������;4�J�����q�d.�=����1�\�\P&����mG��O����=/��u�48�S�A�^^����!��!�i�R+V����C�����@LAd��XyJR�M4�-���$XR�"0���i�]<8��~���^��d�i"w�s���`b�Pj"�C
�6��O�n�_���o:�[��,�N~@I!Kd�)�����>{�}mh����]�H����k����xND��#<�p�3�+-��>4H2���A�t�$d�5X�fS�/������/���2�;#��s�&SJ�������=;`�B��C,<E�c�-�$P
��n$G��z��R���������=���0���,`�y�	E�Uz�+�Y�I$��k��J�\d��������������}/���r��^8)t�M./�I�A�3;T���_o��������"B�/M.��Gw3_��z��t��i��j������Dt��
�jl�M.�H(yl�����{�d�f>�����1!��E�0s$�3<��43p�h�����E@�g(Mf5�I�;�SdV54��q�a��v{x8�}s8���a�\�(�W����RE�4{Eq
�-�,�+����q8���������Tq#i�!H'�����,k,��&��s�m�u�s��j���~�����y��NO�O��3RH��"�L���r��|}�T�O�o�z�d_�i����d!�m�(���lo.7U��M�|��
�f	���{��Ot��W�1H�+�d�X.'z5m�����_����md9��9��`�����,�=��V<?���Ht7�4�_��"
M�K(�gB����1A���d~�-�����������%�E�������.Z�
��)��a�4����|���������=~_�?����
�������j�T���������+��H��6��`d�J<R��;��w7
�F�ue��z��-�}��^T:�T��@%nJO�*��/�����X���������o�Jmp��dv��[I�[Lx�z�%N��w�C���K��?4�1��cY���4��!kR
��,���'���&V0J��EfiOq`�.t��������MV���e��B�N�m��e�d����8M*cd�)��l��������c����<�3��i�
TDJ�<R��<��R���H��������g�gG0:x�._�Y�����%��A��6�����*�[��?[���m���u�1���jn�b��x�����
i��(���\`����U��3#E���4���@��9��J2H�\��(!4�S�k��f�����cvQ	�Vx�����_T?�m����>��x�$�Q�XH��((,q����#������X�m-�����c�kMh4�ir���������\�����F�[P�zF;��:q�y`��,�����z�5�}� i�kX����?\T������l!���\%)c�-38�]�3��O
3A����m���?�2�}�k�F�'-�5�K����$z�t_a���L�o1_����{��\m��'(��������W7$�h1�C��X!������3�uk\���aj�u
���u���<U6h�F����(4A���ODr��Y���{�+h7e�2Ld��O��� �3�
��^�v�6o/����Ti�<U2���$p�q�(NK���hs�tl��=��lF�����#�����}" /=�%��<�KH���S��tn;x�^w�TL��3�U�����_#)��oqe����_,�-=!�-����S�5J
!�
's�g�s���4����&fHb(
�����������<��3`�o���;zi��-����5�s�����cs($��9�^��X���#J�r�Z�i�;��a�
&iF�%���m�w�{�19��as�u0�S�sx�T.����7n!����7�������>�f��zn���>�cB���WT����'�4/+O,��"!y���N{��d��L!���[s���A�0X��x�	WQw�I������q���~3��<V�n.��C�["�l+&zq���"��Mfe�	���g�
��=�8)�v�l�p�?�����4d8��y���f�yh�dL��1l�����S�r��F�-�{xh6._o����u��34�8K8��Fy���'|��t��q%����w����K5C�a�`d>���z�H��[��9��)C�X	>���E�f�x�$��5���.J���J��m{���/G����K��|�G#D�w�L��c3��)�@��44�G�(��V��8W����<�����R5@�w*�����'e���n����
�X�*G������Ra�4���e!zl������Gs	��J&m��+&�KUP����}j��w/�HS	]g�h ��nF0f�&(�F�&h�Z���X��@�!���xOxe��������bB�9�Gb=G����RX�z���laM��	�X=�!d(���ZZ�g��w�Ct;BPe�QC��W�}l;��n!�9���>�M�o"2b�=@r
:P;sS
	�o��U2`_�6�3�^n]����=��i�5e�L[���r����!�~���_�e�C,��,DZ�SN����}��+����c�����N]uz��m�]�X��h�c9��@�D
'T�D������6��r��ik%�D7jL���qIc�1�S�����=�n;�r������m�l�u��z�v�lHY��Y���L#�������=��U��=Vkol��/�VQ���\{�:�
�oN6�F?4�@?�O>�+W�W��J�������^Q����-~{�e��������_�%d�E�����)�I1�����
l����Ne��?�	��X�g�XR�(�>D�(z-�N�y��.�AS����#]fd./�VM�K��������c��Mw"�h9Y����\{��f�2s�4!���}�^	��4:��.g���g�=y���� q0A������)�4� ����l6��gb�b@K�[�����o�O�����X�7iV��J���\���@�G���T��S�i���M��asZ\@��h�fo���������������[F�
���%��_��l�u����_L6�	���p�.yx��}@�����q?��ZV�'��DU�#;�Mzf���
���Pv�T.z�V���x��9WJEC�&$��S�M	��M�����cu����"c�e�/����ZL�����V�R2�2��X�yL���=~�B�����[UN���>H�:������f.J��uy�>��>X��A����6Z_NjL]})9�_���\��^�v:��Ak��s?a��a}~��P�-F2x/��q����^?��� 
���j���`,�f���#gk��!�Lg��r��ho�i�8"�)��a�t5������D�Pe��_����O��v.��_�)z0�����������Z)��]y���a��d���<�)Fy����>o��{Im�n�[/���4���(��Z�Pn�������L7@�
]$�m�#A
�*�)��;<���ho��[Zp��GZ#���l����B6��w�tWd�	�����)����>%�������+�-6gE�'���Kw�(���>?�q��(�����M�T�3����T�r��I�X�b_�����M��3��,8����&DM������a2��|W/f	�/U���j�p���X���`���C�gBH|9�L�D��Y��n���@��g}����Zc���6]���	>���B&�{�wr,�7^}��I���l�i14Q�A�D�}&�ec�n����|��n�ig�>��'�AHA�*���Gs��k->1����;w�n����"V��A����U�!��W��3����
&���k�9+8j)����l#Y�V���<=����)GP�M�E;��lD��"�qU���)��|!/��@3�KR�giQ�
+A�e�*����J�n��c�(�H
�<�P��J*;�����
.Y);��g�DI��q]�/f��vp�J??��U���..]��1-���9#���<��
��������W���;?����kMF-[1������9�Q��8�SWv��w/�N�9���d��M*��?�8�LCi6��C��h�����'L���j�FS�`�&"7��$��39��0U��E
��o\7�z���}�5^����%�:"�����pE��D�$�AT��s��y�5�������}�����z�v��&y�rn�1���t^`�	��`J�r�?��M�`��)�7�Q*I�a�FM���d�>�1dTd�r��!d�s�](��N���#ir��
j9��!

��`He���M�U�8������q��'�xdk6�w����{�=j�ri9�B�m����y�<���2�m�di[����z���]�o���H3 �E�q����S#�����2X�gl��$�x*16��Y�\�@����|��'����v��t����������k���<w5�����Pa+.s���N:��56r���5������(��<�5�S�X���w�M��Q��G��I`�������F�&g�XUG�a0 "��������}���/l�S����+d���:)�S�ag�
�(�n��)4���p��%S��_�M&I0�{��[�����%�6������	��Xp�<���A�t����������D!:����&:{��&2���H�	r�t6��������t^we��}����M8��=hG������k�k:)
gc�=��j�)7��m����H���&%���S���R��2��=1���������g��M�E�PEHO���<�y�����@������/��x�����K���"S
	N�\�7%(���1�8*������zx>Y,�'#k%iV�/�����`��C�4U49�t�r�)��P�)���R��	�M������,��c~��%3��xrF	7�D+�D�
�'���fn��>�w�
����?#BB�|��mL���J�@2�<=���:��``��q�����-Q��#9z�	������wO��n�o��-�o����X_�FB8�����$x�)8���'�����|d�<�1�62g�:���6'�%���r�&�"��sWu��<�XcchG������5,a+-r;>T�����.\��z�y��D%J�GH���)���T��6�"���C
�68��F��7Ep,nM�:�|�}_u�g��o�0���/���������IQ�<��5e#O�)u�t�:/$.������l���7����r!Ea|]������u���=��w@��k�>�D���)C+4�������q�'Cr��!����:}|������z�������5o|L��i����i9�W^$����gPwzV��X�����M�Y�Q�H�%X/N=��d�\�w�f�������P�������,H�Hh�rTF�U�x:�kP�
�~��e�4��
	4��� ��v���C	�|����?s�����e�48�R!t����Q����=�O���/����~�M�
�9m:��\f�����-�Q���!V5���w�>���������D942P��|Z�)�.(�D#E�����>����?��^�H�6�F���vW\���!��a�e�n�����l��+���J�s��#��H��%�4���|�1�Y���`e�"����H��^��\�J��o��tQ]_lh����Rvv#e�w-IX�%aR��
$u[�� ��Z}Vp���
�XT�AL����M�x�A������L��}W�dQ!g���c�|������1�fF���;o0�oD�	��X�e�H�:`����>�k��h��k2��{A�$�a�"�������w���+�Or��5X�]�/����wZ���R ���,L�f\I��8e�[PAr�[T����&���b��f�U�{t��7H���`���77=���k�Q�$I"O�dC������UG���%N��Z�������A�`-%���O����o�?�<h�X�M��G�����u�H6��P����O���W��j@�7��F�+^5�	@Qq�uag%�����MtB�.�D�0
�3`9���9������Y%�O��������{5~�
B��b������d��*y�5���������F���n�ZuO��4m��W��4�H"Ym
�*�l.����uY��6������vpJ�z�����/!�J������>5��
(�^n�l��H����i�)v��R��V���e��o�m�~]n��������0�$�����f���o�V���A��j�v�$������1�1f�(����H��|e�Y�-���f�AP�gI2L���8��
�����x.����V�.e�s�����T�x����x����E��5+�
x��
�p�O�c�����7�MqV(�|�����0�Y������3VR�x"c"
�N��&%��|A��~��y$�Y��6���#w�l��9IE�PB4I�@T���i�����0�}yL�!����3��\��B�j"�d:��������<��C��('W�9N�����=���=G1�;'��z�����x�a���+]FK�b�;�!Z���I�$	�b��k�����I�o�bX�)��I�{��#'�_���8���mz�*��~:v�~9�5Y���s���K�r$H���g�����;0�u���_I9#*G��l�/�R�5t��^�s��k�m}M%�5<�q�Z�XFM���Z��Tr��1���������g<6��E���,1�����L#�.���C�����e�h���(\���G�����.4-Pc^��u�������Mfi�[�{��~�l0%R�"����8�M
dV����
�'��jh=�bH��&9�`����2�<�m�V��,�{#����y%��h����e���B���
?�t��NO����Q��e8�-%)�%N<�g@2����4������0n�'8�d��B�: �si�o���L#���X�(�mep��`�/������Wyc�E���j�l�)8A.#��Z��#���|�<U���W/���R�k�3SN{!�:��H�
Z�?�b�_�ml�o�,I;��!�e�$U�"G2��	�(R��*�z�Z5�����4��Z}���!�(8\��${��g�|�O�%����.���3�1�Dh +�rw'j"i� �0�[��B���(���%4C����j�*�����C���y���0'����� ��R+�sa��?���y_��c�13�����9������,�}t�G�d@{��_�m�v������q�$m���VH=+��]�����r;�|.��\k�~s��,N2���/��i��<|�2���Ts?��3�����)��d�>X1�s�s����c����P�u��#������ x�<:���/�x�I�*�t����j���><��.U���z��l��n��9�;�eh`���������t#o�)L������sg������v�Q�b�g�n����ISqQ)�cI9�p[�(�?���������,G	\��?Yq���A�����$���/��Uj	��E���YN��I�� ���!����O�W��.<�?����t��:�� ���L����
���)�C�~�k�*�J3t��KgH�e`�P����k�t^�K#�/9����p�5@�����O�{Cm�y�R�Y���0B�
���Sq��^���y#/�<�V�1I��u��"-w�`s,��[��V��v��-�����<%��Ky+��h?��D�������#�DN������{K(aK��������������S���#���4 �E�I�T>-���i1K
��]�s�-�R��sX
���Z�~����\B��g�9��6�x��/��.8a�D7��
��	n%�If���),��uE��G2�ly\G|95��*���Rk_��S��F���H��<�Z�9;my�I^��^G����(�?�y�}���'R�d�;.5��L���\��j���n�1%�_���;�{��'Q��$#��?���2(!�_�����S���7�����k�����T�:�#"H�Mk5{^�NX�����a�<��e�Y8c�-��y@+F�1��ym�O����������Tq�s���H&M+��k���Hi�F�R���Z6^����u�i�brwG���A6�l�S=���*�����]@�^�������
����Vy��>��J=�+�C$�tK�Vr:�@(���T�������y'���~��/nn�$���;*E`�s�?t�_���ms����1��Wj��crI�!�w��W���l$*
�'
���:
���oe���u��rm�/s3	q'W���4"<��'�����v"#	�������!������C������Q&�Dr\s������(�8/��h����E������� C���1�TXr���������������f���=�@��vw!�qm��5��o@��.�TDm��B;%��.m !��|izW����4������{�i�0�
������~&#����d]����h�.�(�dE�����T��wRi��i%fn�����_����j�C;W�~����1�8mU���fI��c�n(T�b��lt�	�}W�c�(���HS�5^�����PV�2u6�{j?�������
,����Zb
��g�����S�)����k���
�Av�}?�����B���{��=����?C��b�^]o��'�C�j�������w>��*�#����Q.
y���"!�L7���
y`������8�������=6[J1�t,R�������lcY)u����{\~��1�R����W�e�g��@���{����������h7I��dJ�IF3�I����oM��L6�,gB���~�5�Hn�n��
��2K�Gj��#�4���+����g.���r��`���52f��c9uU��T���
����������a�wgw�,O�c��	��j�!��X���������w�S�4�?�~��KE����c�\KL�����Q���a���$�U�k���|+���8
�����o�_\�y��n��g��������d��TD��wd�"��V�
�</��?����}{x�m����v���v������q �F�Rh5n�#�?������O���R��Y@��)�M����y�������i��S�|����;���������(O��`9s����_��H��vS�.�|k<��]���n=yC���Q}�"����{&���,/U��K�$����T��^��y����5$�y'w���Q'w3;���K����2[�H����{���0R�i�e�$����d
���.������>�NO ������H4��R�A�v��tAr=��&|Z��9>��tl����oy��@��w�������L�@���8���(��\,�X%��CQ����d�vJ=����&�J��Va2Cb����+�];�>����]�g��j��\8&�o��PH�q��<=�����m��!g"���as�=���(/=������o��f���`5������
:��v���.�����(*%_7���41��?����0���4}��?~Cy����e/%W:o���y%(22�;��Rm�i�WM���������,H��g[b��/��(��������z�l_��r.�[����6�5�����\:��UwK#��|R������Vy��R4���4�S�1�6]�5<���_���_T^��w��[�i���{*���e��H�D�5�zu�Dk2�q�b�H������2���)|~<�DA��-d�|�B���W�@�Y��O ��b�������[��v�^��N���~�������+N�W_�Fh�]�:�u�z,���$*JA;�����w�_��$����mw�&�wq��5��x�]�kH��z(]R�4�����"���*u@��r����c�KV��=�������|ha��&�@)�
�^��s�PJn/eI�k������4�N�lYRDDk|�'I�|d�X��IK)�7q���!���8�+��_$�"oT9Q��s�kz��v2c*G�q�q����B3����cs�1H�����3S��m���J�@�*�B����|���W:<��s�@�op��0��
�y�Vs�=����<��Y��\#	~�����jC��#�a���OS*W��\�BI{�R�r=+F+��]�I��B�7�n���a������<�\�K3����������5�Q���o��i�LL*�q$�f-vG��<?�Hn���\��n��I�|5���v�X�5��b4���4Z)��8g:p`�s���b��{'���PP�~�R�/��`��$��j�Q��d�:�����mD1��%f�9%6��g�����88��
/&���������0fZ������T��Y/��;I��!�����������hEh��}�������x�\"	���z������;�����C����=�
���%��@
�C
�L��8?_�����G��i�`��y�R�'�]�������x5�c����N�j�W�����W�������c��������V���5
6<�����)� KMg��&(*�������������{�}wz>��~x���;�A!����d�����������JV��	!V���|��/�px����yb�O���2%�x���J�:&�"�(Ls"������t��0�>T�A}���v��CEu����-F79;A����`��	$���7�8"��Iz�[||�����O7`^���������q�z+-'����A��d�G
f���o0T�����*P9%S�D�[@O���a8�>>����p(��r�v��������F�@~%�XF�����;���vW���Pw�s}���/���Q*\�<ye�^�Z�[�Z�~�����'<)B�2��M0��AsPx��|�t�?��-+���'�m����bTVG�����P��JX��5W���U��V��Z��Fs3�]�o�]������|6%,l����SY���!���l{>~���X��
z���\���=�
��Q�?�j��R�� eT�;�(@=I�6�F�?������uhO;A-�G�h����s2r�"���jQ���f#�A;�$�};H ��z~GN��s��@uu �k��A���S�t��s_�/M�.���*�xf-+Nh?��,����FG����/�o�U��i����j3k�L(?{���@�,�,�|�%���%1!j�C�,�X��n������3e��G�������]Q���[hR�
#EiT��<x��s���B��W����������z�
/&Zm��T�&�^
����K��'k�S��[�$���D8KLtw�*7nne$.R��Y����d�K����bdM��	��z��W8;Wf�QL�����7$�K���QZ�>PI�a�3,~�~/���o��F����F�UD-Q��}3�����Zz�K�
P$�w=���Ot��s�&�OD=����Bx%P�]
Ns�Y_���A}������b���L��GT��\�%l�jCd)Q����M?��X�f��o�������z�����l!�x���.�8h\���A�vGA��������6���b(/ms��W����o���Nr�6I�L�<R!Uj�����I�=� ��CC�Jv�
�~(�)�3�*��Gh�2Z�PRi���,�s�fF�oI�������4n  ��������+�G����V��W`��M��s������^�m[hOm�����K[o��RO[�c�<�Hs�f�B�UtC*��6�{�M)��=�C���9�����!ab��0$Z��47R����A<XP�^�?�B�YY�YD,���I_q1y����l-�M
ta%���i��PB!2��I����}�O��k��/T	���]B\-��lD1����b|�b<�=�yn�F���B�d��y|����w����73��ss3�s��_�A�����p/_}���e�x�$��TF�F��^��S��vfx��]P2Ep#Ix�R
����\����]��������m
��E�����q�:N��Y
���w�DC���N�K�e2#�vsf�+e�sz�_���u8�
�Vb�?3���U���"A���m1&����Z��u0��8>\��ec�G��"p��h�JJVqvkE��3�<~��s:�g@5�
������E5 �S2$�����o���s_:�b�#� %�k2'P��U�@1���q�6SL�M��{��>��]3���7I��$��7s=��I�������{4�:�G���m���$��UZ`�j
	i�F�������e]��/�4����f����J:�lw2��m���&��u��L�#au�4����j�����'I�;w_�����.��+.+W�1/��H�p���l�'��|Iy�=��S�kg�-��=-�FI�����B�����b�Fc���v�V��/�.��M�W���"��h^����b<�2P�U��kc���}e_/@���0�#��+���E��{������F�������aG&v��_�����o�\Y��t^��6��@-�h#�y�[��o����+L��lD�����|���~r(+`<o�t������m��9|l�Hu�=Rr����b��zS%�[�Pt��nw�9 ��[!V:�0��)����Q\i����A]�%��71?���2��-�d
o��-�����9��;�3�a���i������x�$��X��u���uH���i��U�������-���E�������)Ez�4�����0����zF�7��/����*C��f�5B�����$CBU�cx��Szn����V�:I���J�Z"+��J|�%�������y�"���d'�;F�%�������n�	2j��2���:�_3�:~6F���
�
��u?�O�������z�
x��}��V�Fm\�) ���I�\Rfa\����ER�@EP�S(y�+�!q@��c�~������.u%�g��������3��]d)2Bn��q*����~�<^�f8w�&�;3�X��D����,-3�U���tB��������5� �3\T�Wl�#I6)p�AZ�Sw���B:t��}��]��H�#!�B�����
k
Te�k-K��#U���~�*�<j��'���$����J"-("�������|�s��kiV,7h��B\M��H{���b��P0o����`J������Q,�����RL�����	s�j,]�KJ��������'=>W����M��Y� �Xe�<V�Y
{pjG]M�K���a��%~���������7����
�d���U�"���x��
��n�4���h��g��Ti��HQ�����l����RX��J�_��C�	"��=���7��[=�����k��J
����v���O�(5 ���2r1��(yA��N$������RmL1��J�Xfe��"�I�������ZW��C�����~r�|�U&�L"���/�
�u��(�[���s�_��i���r�"7���"QPV�����~������4����w R�'�H3g��������C2��PiD��bE�7%sJC��+2��:���k�?7�W��+R�A<�68$�!Y����� 
'��K�f��/]��o}&������^#� ���z}��bXD�� ,��t�8j�����a��d��:���\'�����&��T�����Fh�����������aw
��XW�RX������X1�������:+���
�3���K��Y�Y��9��m��LAM�)?���E�e�w���R�&����i;�h�<n.5���p_:Wdv�|R��=�����'w�?��	���F�"K���A L��zd�D&XoX���8+� ���/]s��xZ�;JX4���c���w���G������w]<O!���(�M�����9�
kv�v$��Z�50t��?�8��C�JP��;����>�y�����QwmMr�����Q�)�<*.�Q;��*9�����rw'�[H����$��,�@�.���q��,����.���Xo�Py�vfW%��d�Nl��_�M�AY���ZO�L��.���wi��|�����E����������+�P��8�t�X�����������C4[h(^��^cd�$��l�Z���8�9��>������#��8t�������<�w���s�ph
,�����&].C�q�\Ch�8�g0j�K�R<*��>I��4 $���{���?�}�=<8��O����7������j.f��K��D,���3mV��r����p�����OC^X�����I��1���G�� �����q�}����4>��%���!��\��aK$p���<�fQ���H��vFc��H%��
�%��)�O�� E�[�0��!bs������v�Q�2�2���cIQ^x��KQ�S�|��H�b����9�3��r}7P���W:�bn�w�7���l�N��NZ��90lR�k��b�5�u��������'��e����,��+��w|YK��12�[��Q����.e/A���h�4�3���1�=>��pn �����)���1M�eB�n�#��
y.=���/�\FDB�I����:�`.��M�����St��QF��!y>�������`���2���q��[����x���L�=�G���Mp��=���o�2h6�t�K�����W�F�������E����d�%d|���0]6��'��i�T��f}��y�(���$!*"�����@�	�n�d(�����"��]��
�g��8;�3���9���$��h8]�l
� �r� �~@=V�������;K�K<���D_A���T+�r9X�����l��=\�yH�i������
�K��@��3n�I�������������a��6��m����o� 'Q")��9V���3��Kn e/���C�^�r�#e�J(OJ���|k_��SuW��C[������6�];s:-�lOrYf{��^*d>^���7��~h�naZ�<�3�s���E ���)�J�pO���O|��;���{xFi!/��Kw69���@L$�@#��m�[�l�����p����lo�]�����[�a%����������D�3YA}����j���-���?���]u<%�$\�nn��H	c��G��v��d0�+U�?UP��5�	�yDx44==��B ����S�/	�4�E�h�4�����|���
0�?e`��|�g	H��7s^�o&@Zy��:*�������l/o~{h�`�����:9I`�I$@"8���?Z�~���<�:M���(
�PE�|*��gf����N�������67��'Dr�F���E��2���13����;��u����i���@L�������w0!%D��I� ��xU#���v��}��C#�)1��k:����q2g�-av����`�`����vcG��yx����r����6/�,B��o���E��iNap�P�[��xNs�������#t�s�i����<�������mBq�p���T��������V��j:[������')v~�*���rk�U'��S��l��+���Kyr��������a���%�&��*�������j��RjED.��Cc��f]��T��0Rp�5���2�Rb$P��2�l�(/q�o;�z������#��M�~|��\$�����v����8A #H=�0�������J�z����)�7
9�Ll�^H�5���dr)L?��X��I����U���:�����E�aY�R�+E����\�%��P��q��r��0���@��\�@�r$��U\�4����� ��#H��rf��Cl���gB������!=�c�H7��������h����EtY��%����&s�����*���^��2��0�������;��4Jri���g���~������/���G���"��B�>r�C!���������lm����d��n�h���ci���t>ug}�w}{Z��?�}��
�.�(�)=�n�r.�����N\X�����r:.����������	�Fq���2�Z��*����P�x*mge���O:������a���
�l�\4�/'�B�F�6���<��?�=���	�!J|"���)4�
_a-�ra������zx� <K�E%��E
��8+:��p�����8��z�t�z��Z�M[ooz���u;Z�����A�������G�|AP��5L$H^��3�
����_9!�����
\����A���>�� ��\��d\�8�tH��Ng~��-N���0��L�VjbgO��m���]����61q�1���@��������g���Jf#�h��6��lV�>����E}��[���c��t;Q��G����*{@_nO�i����rdxI���;�s�
%���������S�
~���bv#���& ���R~�v�e�+���+�������98��k����=�����k��o;7Z��hI�	�.���;gX�0���g��!};������P�_���	��2~�6k�<+�PuS�����x���H�����cU�
�����}�]����p�������=�����E!
/�'1t�|<��89-�j�l��		0F����5���Wt�I���+�#`h!X<���)�uni���G�!v.�����v�6��IB�-�����a!�G�m�.�U�	��vu���upfu�ia�kT��%��Mgn�f� ���L����(x��@��r�a�A��d���l���[�u�S�+��#�~�6��G����
���"D��� ����JyU5m�?@H���~���3�H%�TG*{-���1�Yh$Jr�w���s���4��������H�h;�D�"]OK��������n�T����/�j6���Q����=���:�|���ny�J�tHo�1�!�6�(p�6�2���C{X7]��~��@?D
)�9����{�5��JC�"�D��!��)/��o�C�'�b)��:�yzG����A+���2�R4�m�HzuX[�����A"t��|��n��������1F'���|;��S�%�l����_���d�h���>)���g`
�8{/a�5���Kt8����e�����t8�M����sv�o�h/Y��[��e��!�FR�>�s$�)D�����7m�CA�u���� t��FpbVh���I���2�����]uO�s�o�=�#�lO�w�c��/���:�' ���Zw��n������m���3g�?��a�"+<�4��\���1������Ou�fv��IL�[Wvr���*�0] �F6������<f��� Q�����|�$�]�,��R�J��{���D|��xA������%�r��'��%���0���=�?��������$C,�����D"g��<���f�����k��=��Y�rI�RM�� -�R��R;S��I^EZ;#����-��!�	�%�g��'�FDCb�o��	l+1ZFg����_O���x����!aL����<� aij)r��7G�vD���� M/E�@CJ]����k6Xy �OMWf<������n?�����#����.�<h��^Na�x��t�n�@�� ���7�w����w�����v��pH[f|���Pd�n�~	�h:���e��]v%�����P�7�����V*����c����#����d�?��������H��������|��Y������f]�?��6�g��?����G��iv^� <�v���!II���06|�h{�1E���q��*�2��%3Xn�����w�{����W�\?,W���i�6r�$z��_���%
��/&dh�HV-EF�+��^���n]Z�`��\��['��3vLW�7td�����/	�:%u�v��3v�u�!�>��()�M��=�^7����K���9��oM���
�7��W�K3�������>��WD��,���}W}���\Sw������A*�<�1���BGt�4;���M��������G�U�HA��$���3�aH?9���4��%&�q&�Va?��u�m�5���#��{��`y��f���j����h�\�x��&"h�9q��]�d�)p�����E����j������:H��]��Q�b!y7e ���Et������Qn��T�����?��x���N}����H���l���c��Rr�`�A���{���ww���j^�������Y�p���Ml�/M.
�$��<�������������G�s('�EE�EQM�*�d[��	�S�E�\N�u�,r���i������U�;T�raN��L	��HP�)252S��)��D��v��t�(��m��X����v��q����Q�����5�E��>����������P��H(!��;�s1����8���t��,yZ��$LL��$��?�R_:��2��d.����BM�s����)ia��JW*��{���z����dPP��B�6�s��i�c�n�+�ZN��zl�cu:�9{IZ�2�0M�`Z�Rl�����@���V���.�V	����i����F��P���g�~Qfh�<2�L����\�|��O��?U�a��Ek���o��xOH���&G3��Pn��iv"��D���h����H��5�m��m�Kd��KVs��$���Ym�����|k���6�4F��e�I�Y4J���U{��5p�	�����t��%� ��B��{���@������i���(B�C�gd�Zy�}	1�X����W�|6����1�X������G/	���e�9��#X��+������H�8=LK�����t���:R-'+�������}�6o����M{x��<�!T'-i����#���8�.���g��x�bt\�*���&�z8�e�P-����J������c�f��&I"��\�
��d���\�6]wj�o��Cu�=���(�����`"3�I���R,'��$��2�j��Uw���������/��y�a���<��/��,�d	��8���6��`I�q�.9�\��R�*Y��p�<���GN>I%��k���m�n�[`���K#t��H����@������R��[�i4����Y��h)��x�K�	���*�tE������1{�=�1Q��������k��z2��j���O����z�������-D���(���e���|iJ;a�&+�����+�^&0�I�$����x�'9�3Pd>��%�;�1����L�]B�Yr�Rox|�����YW���V;�������nC��K[�VM��=��"xL%��>�t*1h4�Y���9��\z���n��o�����M��8X�8�X*9�Cz����'.�@j�KzS�[�C��SW��&��k��@��8�k���	��m�t6�*��Mg��}}������{'3H`�q��i��X,F�s�X�����"�F;��~zW��=�w���Pdj:�W����V�#�%���\����{���������r�K�=
E�E�����D�
�!5�EJ=��_?[Dp�G7�)�r.�z��I���e�TQ�$�II.�L>~z����qvj��Ph�����2?�"�	�-p=�z����~p�9d��o��{()9n���R(���������.���L0�T2�/u	&��HQ�LN���fM�;��{�H|\���(2����Ci����{����������M������B�)��$�h����lgR�����&:�n*,A�j��"���X�Q���g0�,��taX���L.?�t��~� %����{r\;�
����'`s���D9��e�!��d���c��D6QD����w��1����]h�E$x0��{���e�(���DqTbk�"���\��+�����et�~�L<�nk5����br8��=��as1�����	|���;���=N����`@e;"Sh}�� +�����O��mX��\�T��S�I�~��ho�����}��z�t�z
lw�T
����e������a�����A��[�p��"��}Q�O
���$���\��
(*�=�v"�`pX�e)ui��4��e=�F5`���C�����'^m��^�Z<C=��AA�
3������z^P�=��}��20�xID@
j�������Hb+�VqjK;��l`��dJu���zO����/�x��� i!�<���g��J
F;3�P?�L�������>6�
I���N�&m0&j�);`+�i�ws�"��[|d��\��zzr�Uj6��H �>7F��6�i
�Qo�(}��AWIJ�����.i��k�
������d������~����y�$��C,�����=�H����R����m����)����*�Z�{����,#�zH�A��jJ
����P
��F �W,�NJm�����	�����s,$���|����������&~���UMA��M'r����8�������[|���������]�9�>C�K��(��T�Sz��{�D�OY�������m����0�,�������>�����D=k�x0�hj��H��.���Pw-M�G�����Q��q4i'V�������@��=�I�"����o&�B��2�`� ���*3���Ru�����U���]s�����9�:�Mwv��7�1dI���m1��V�o#M���H��_O����8
M����ul��yH���0������X���+�Jq����o���~��{,���������'z���31fN*V7�Q�:�-�8[b�������[�b��k��9!O��(Q��l�+r���Un�dT~���6JT��u��kraf'�I5��I��C�uY��<�0����y&�f���

�h|li\`|�k�M��1���Iyb<G���Q�����W]���Me�-zT$���_�einl%Iw�����mT�������i�S!ky��/��|�<�.��/v�{w�t��[
���}����#�D�����!D�WL�w?4�q���~_?�-��tG�,�|�I7rI����!������g�x$e������yeW�x$�h�O���R�|1~��-O�v������.fe�P0�������/�BJW�U��$����(��^������y5C7Z��7����������3uc��������L����9?�R��*��������q;���Jz��2��Q$}�H�U�)pN�����g��w��O�#��t���V���;��#c��Dj�p%!.m�����9�J�����B,�c�"s��xC�3�U+�K��g���c{	�7=Q
��b�f{�%�#��LZ����mwf��������O�V�&\�HC5���l1������]���A:���q�>N�hy��m�mq2��/�����9�u�oX<���b4f]�R��)h�f4�P�U4ym��^MO
P��d��k��Q9�>|��u���1}?aa�����	.��T���,�0j���:N[�6�(�kn��P���lJ�/c?��m�n�����~������9����O�'yz��k���	0<�w���s������n�S}}�d���31���N=�K9��I���&���3b���&�B=�H�]�@0*�%m��~��a�����������7�I�F���hxEU)��|����i���Q���`������o�b��*���@�&axc]�j\5�	�W��n�v��}�P���9��Z%�h�C%���h������d��W^7������Q��"c�������0�VV���x+Mcs�T.0�x�4��W��K
]B9A"��W��5��������J���pf*g��	�h�%�/��
V������LT��L������$a��
�B�~�w��f3�&�l�]�i��E�n*mU���m;�w�	�g4"v�'����T���{�������zT����a������Y�h�p����j�/�>����k}�*����@�������&-�~�Z�b���(���[)��9���LA9�sD?7���E��"#�P�����)�Y�2V�S�%j��D�4�H
p���m���t���k������k(K�������M�Ct\W2j�$T������LkL)����6]}l��#g��>����2���omQ|�Jn��$>
|����'��6Og���?�~>!?I�4h,y@@����eIT�W�s�@D,�4�>��'���}eP�s��x�����V�cZ�;����vx7V&�����\�r�G�&���b�-����zw@^l��\X��� �bD��@�sV
S�Vg�:��>�z��*��*b�D3(��$�n�W���b<������N.�=t��`;��y1w���T�4���0j&�]d�91��
�1J�"��b{>:@2���V��F�s`]%-
�������<���;{ij�6�a���A�"s/+I�h�2���xs�Je,������M$��3�=����c�{x��nD��������|n��5��x������<���������{Lk@t]
���yi�y������R��zqy���o�Q��n����S�yg��v��ys��v�]����LfZ���G4t������.eoqn@�BUy3+�Q#w�
sAF���vB��}�v-�����8�Q
EY��Fk�"K�V7�#�Zm^��`L����9`^�q�������S&�{
)�g^��L���?�u�o�(5T���::qvY��+����ZO$��Tr�-��JoA7������S(��
�N��),�����Az��������u��#�0��.�~����������U�{�	�3���-�,����kAM@=���\�S���Ck&�ls���-�	���0�c&�J�	"h��=5*���|)���	~E�6]
����(�Q�6D�����[4T^�J��M��~j�D��I*4I�B7: ��m��Dv���������^�1���,�j�l�������}i���D����]�RUc�[I���qt=\&PN>.xi�8��~|�������v�?����I���up1�!����wq����M�?�u��W0��"C��9�����G~���>Hb�})��h��9����g���<��C<N&���Ar���r��<(+gK��������>�^?7�9y��U�)
0;g�Y� ��&�����y�5[�W�����?�����2���Ip���(�B�����;4�V��)���K��o��U��-���+���Mk<%n�D)��Ds�������z,�M"���&>��p�G����L��Y:�J�;�����R �Ac*�	�+���P������ �ro�������P��L����T��k?��3�����C3|��[[�y�*���B��5[,C�}��p��������|��I��`p
��������������)����mu3��;m�yB+/L�%Ah�ZN�����bPf�����~Z�k;�><��	��B8���kux2a^'��.��DI��#���y��,5��^K����L}��6�GS�t���W~N��J��ug���b��B�
'M���������o4�u��	�)e���	t�|r�*9��YyUz��_�~:���q���6����	N���Z�E�b" �Yt����KWL��������q`����!i.������&wu����fZ9�&B���>6�f���a�uZ�����mO"A�#�D8%�����(=o��7h������C{�P%���`'w���3ND><4H��Z]�������t$���u�"��Co�5�8AP���}�����7�c�GBq?��9�j���tU��2!�1�^I]
gWB;�	��G��6s1A��e'�(ZM�����I�v���5l�<0�4�d���F�s/��i6.�in�\r�k-<��q9�_�6�r�xl���T��]Y_��#�2���l�����Cd�hQ80:��(���@�m���Ql�P��"F��������[Js9o��C�g�5e���(����Q����-��1_�A[��KseLT��bi*"^����6��e�?o��JU�Y��I'���MD�p)��@?�������=L���f@�����!�"Zw��d#�$��,��\���'�����s���4��3������fzK�,=�m��o:h��v��}����������������.4QHm�.��N���`�������MW�/���j�>X������'+Q#D�1d��R���K��{�}�'��^{6��v�r7B�~!��"��]��~i;���u�������5<��|�8Lr�,�ap���Q��vh��a7L�=��D���K5�Am��+�n��������J2�-�}�oOYK�H0�kG�V*��{O������7�3Q_�	���������#���:�
�^��=�.:����_����9�&�F�/e^���~�\���k���0���%�Me�m�\��
	����$)'��,`������To���P��]����N>Z��:p���$'��k�V��5������k�)���h��i�m��+#��i����{��q3��L���������~5�m�<0fm��4��r����.��.K!��#$�R�P$j��&��l�����ISS��q
��e��q����G~��h=G�����JK0�E���������*E��q�\�1��W�O�;5�Z���T��W�
���=�a*�v��L[2�A$8Q"���+&������Jdk�.3|>��#%�d��I��G��t%'�a��:�\M|wc�8gBj�����s�����e�� 
U��RVEc��sW7������7�6=\Z5A�������Dc�����l�������%�Xq����VQ���Ml����H�>S���${dxzx�"��4��UZ���!��u���X����A�oO�Jr&m����"�	���Wg+yA��?#�y�r�1�y����-�sf�`�� eV�^�w)q��
�e�r��#54�m�nq��:�<p���1?���46*��m1���5G�����C�\�; ���8��KQ��J�T�i���b8J����Ps. f+�a��#ZI��"5z���\�M����R��(�&����wh.�
"���
���
������]�����A��O?��8]�lY�	�4K	���ip�����������iS��s��,���=qy}���Y8���2]o�>s��D:�$v�RN6n�z�s^����x���%��,
�J�(%o���R�M�n7���zsX������\���|%�M���d�&��0�HU�������o������JW�ZdV:Z�6�m����	kn���n��z�/R��?�aW�����e��pJ�uU\��0?�n(N��
���KX@���E�N;i:e/�o�:b����L
�w1V��}�y����E-��=Z��T�`d�
����.�Ebt���Z�V^�r��������#p��3�������zrf�|�gIM\�k�W�6L���O{x$���F% �G�dD�+��,�i����������A^��:��)��e�|���^���q��7K�3����E���<n���mp��2|_����<8�*E�[dhm�K�������mw��^������gYa,.�x��6�6@�������]�tm�)E%���1;��1��e�]y|I��q��R�����|h��u��zQ[b�O{m*R1�G�5�x1U1D�7����mt��_�'Xu��^*J3���.�	h]������`�N>m� ������Q��&�,���o����:��#��).n3����
��t�,*������J����o��i,�U(
��b��0������g6|������+��b5��=��k������6���it��:�V�����}5QX(i���j-	w������_�B�9������������K8����4a��H
<���s���b�����(� �o��2���+��[�R����1�:��K<K_�������qL��a����=W>�?wNl�8<��B���c���X�>>=��/�K�04�	-
�O>��Q�|�td$���{NMQn������Dp�X*/b��dN��&��t��O��O��a!z?������!�vO�QsC��JLp����g���b�e4J�0��-f��6J*��Xa��z���F��j������4���hf��Zl?10�Z��T�*����H��<#�Ap�P�
CH�M�Q.7�^�����Ex��m
��u�����-���c;�vx��s0c�k���h`����w�����,��Ou�����R���	1����R��J"�S�O������M�G�n����>�>����=
���3@R�Em<�QX)l%I��,����x���~��}��p��K��)����g�[�R.�QI���4�����i`���[���x%�3��c_�>X�Q���d[���z�Jg�v(����Of��9Z����F���Z$�Nf�%��A(�r�"q���l6�f��Z�1b�Y��Ee<�������2�TF
IiI0c�U�����2+.Jy)/�Pl\�����-�"��7�s����o���>y�g���Ur+�[�������4J��+W��;Jl�3?jN'A��d���P�U)��[�ye�.�Q��y%������1J���	f��.J�M�P�?��J�0�X��L���nZ�Pf�E���hl�����nq�I�����xSq���6�������J�R
��?������1A+��k�W��Sg����*oe��Y,�G�Yh�	�����R������2����<O?��F�<4�o]�=D���`���K
�k�
�Z����s��h|
iEH��6��T�������f������y��;l����i ]���'{��eEe�K���s���ahp3��X?oO�g|1�M�tL`!'���w	AW���$���\ ����������G8�/��	Ev.���#�CU}q�?�R��.�C�a���CE���M?�$����h%M}�gT����P�m�I'�
<��x�����r����gPO���i�H���s���
i &�#�l��(��K�[�~jFR�x�3�G�����Gn�I��}��8��5�f���]`/��[3�������=���E�r��Y@4O���d�U���1��j9!;D �7P	�����x�&�����%j)�7��o� 'JNBK\��T�������T��-B|Hw�@����	�i�����76#>��@	^S/-&#I���q�����1��/8��,�U�nB��z�������"=d�w��2�,>�<M�"�����	�M�J���%]lqo
~�4z�L����*�'j���3������I,���;��v��'���_�:�d�>RjT�����By�nUF�
Iu�����m�6f���-�T��>i�xg� �M���p�E�E�_@�Ep���=��1�N��]-Fg�|	�F894Y��X���up���������Y���h-'&��'���wh���w��l��]�����K�n��u�^ER��.r}�7�m����:L�����\����)����W��8����O'�����~�
��>h{�����q{,Y��^�sW\����qp?��!��S!���gHO':��x	���=��P
�5(����=�t3����v����9'�������~l�?�w��������@3R��,�k[J����2�b�n��R6�Z�����6�������8i��?������Z�x�4�6�RE\c�v0s-�B�:�}�E�tv{7��7�+��`*
	�%�b
�m�v��T�BuH���G������;�i�����-����#7�R�EJ~�K
HV:t>��e����e���3.����-��yrvm��V=��j1c*3��/��ro�����F�������K9���7C[w�v��n�|n��������$0����!d�n���K�g����{�CK~g$�w������$�x����>�������|q�C;���W"����rB"w�
��@S!JYf���9�~����tYK��5���o�,*�.�[���3����z���!K-��m�Q��3YY�a�Q@e���4-���Q(���:&�����s�"����l����4*��P��b��+c*����V1Q��*�S�BZ'*��H���J�f~hwm����������;����v��n�3&������x����Y�(���XG������S��tx~:X��9�q�x�f^�ItS��w������&)�������0x������u�g�j��@U^{����i������/����c����� �I(�b��,7�a\��z%������L�O��=�q�����8��t�c���H����#��`�QNt@�����+%V����BB"�����R7R"q9j����������IT����4~j{��ek)f�e�W��9����f���'�
~kMSd��U����p�_������C�JzZ��

}����T�����H6�N�Rz���m�O�������C�s(���In������!�-��(`H��?���d Y�5�D�Bt�
�dk[Z@X]UqB�%
��l{?�u3�}w���.�TZ���Su��y��������yZ������$Ur0�����n�h�7�@R�		����m�)YF���1����M�w�v�U7S���?��S���t��f����c�\�!�2P�sI�R/��������0�����[���������f)l"H���77}��?��/w]�}<�c���O��&�8/m��N����1!}�������v�}�A���9������w:Vz��
�	�SZi�K���������G��/B�/��i����k�\���H�
���B��jp�,���:������7��@!+I��U�d0 .�-W�W�����sW_�_w��(}���c^v�b�����`E��;�Ki:��������o�}[��hW�u��*�Q�F�����tq�)�#Q�=�+D������O�F�"���1�@���e!=�H~1�U,�����l.�F��H$�OD�iF]����Z��f�8������+#�R[&+2~�|gk�R�r���!�������>v�$E_�
�ZPi�kM���� �Rc�/����k�H�]CI�"d9����j�M�DSr�������E����^z�3.=����2;�D��i����������~�6H}>��6X��t�i���#�������Qc���}���~j��������~*��fE��j��Wm��o����:��=R���3���u�@tYD�2a���"7!C����PS�Vcxz�'D�U��c_�1	3�V����d�Ya�`����45����	�<���J�@.7\��t�p�K����x]�����&�*���7�!Y�5=��CYx���������.+������oVo�;L���toq6����`	�$t�dD��
	UA�A�����)(�E����RWV1����$�m������A7���S����p��?�������>��29�����s
%��=�:�[�`�w���Tw~�v��P0A�X]�������f>��#x����w��}:�Q�����_n��6	��������R�9(��L8��`��7�u��}%h,)2�t4�y:��@+�*�7�I�Kx�
N��
�x��!3�Q<\�������0������:�<u��)��I���=)&��i�;Wq���r�l�~d"�:�ns�1^���-0�X)X�+�#�4�C�<�m__������}��"$��U���@f��&��RXH��tN�4����.�uI����4)E�r�k�ou�c�D�p��)�I�8��I���Rn��K���@!,��
����7&-,���j���}�����ov�����L�~�G?:*�v��<�1J�Y���X������1��u��b��I�B�����V�!�H�����9>�+�c�=�_��j&�'Xj6�����eYUg&�z�y�,@N
;���n|���
��F���Z:�(F<�)��������f�OOa��V��5fC���%b���qo���Z8a��sz��}3a���?�]�m�	u��@������f��Z/�j�M��/5� ���5�S
��?2��Y��L]��0��1k6m�k������ �u�	����|������a���cbamd��	��B�'��&���s~>��M&�/����;>�&�(�ym��p|v�����#w>9+����>�����?���)��4�.�
�&�=���q���Q6t�L����{c�Lc�D�������Y��������Lx�\ciGH+Mx&I�DZT����?Q��(&�~�?�!$lh�]�	��i�E��T"y�L���Z���>O�p>4�68��(/��gu�7E��y��r%���^���?j�M�V�Ib�H��^�D�)0	g���}����C���������)����]�qm'��d���4��c�R��oL�����$��Y������;�I�c1��x��`����pW�p�����\��76����	�=�G�_�t)���83��r]	���B��3���w����IU��iU�t|=�o�������PO��'w�����������tW������)6m�����*
i��Q�D����f������+��h�Q$|Q��ar�"U��W>[�������/�_Ej.�
�&�i����0HV�G��!��"�}n����ssx?u�&���:�Y��������`�iL)��x���i+8��\����fOd�R'����"T��S}�����;5�����7Z�$�NU����Ml�x�0�����w`�[�?��������y�������F5����"�u~��|�_t2(�w�}����,c2S�����f6�n�T�C�Zv&C�4�7�n^�0uF�q3y�Tok��P�����!l�o�D�5����T��_�t`%��V��3��&��*��x$�W{����������T�U��Jd;k�
Z�FJ2�������i�G;NV��q4�<a%s\����J9�MX�����&�{�?���)�<'JL���r.���Kj	R��A9u����#Yz�1����� �u*g�k��	���b&Ih�??M~Za�v�8��ImC��~��"��i���,,+J~�����iR�4d$���K+�d���A��~V�+��~j�v��z����
$�f�8L������r� ���]<8
x5�����n�O�������}�C�eH�������lP��%-��lK��]+�2��2�����1$S���K�B�a�����X{
G���2Q����R������J�;�K�7X�>��������f�}���*��H�%�N&�&�m]�����?<C�#r0��^���F���,`�s~tr�\�H��2���r�E4Yzv�BW�)���N�r����'�{�{������y[�%�jL�������E������I�*�B��	�Mw�f"��}uZY�J����o����!���������~�]D��I�B����ZRnz@�JZ��G������?}�&�zs�bx&,��+��o�I2�;5����}���m���=A�K+B��:�I2"��0Uj?��	`�(�
7�i�LGSA����kK7ibe��M�<�mo~��6��[��i�5�����7���;�	!+J��L/�5�X��#�l>�T�n6�g�oG�������.���^�`��2Z�|Xn��H�fv��$�m����8!�
�D�ad�*f{F'�F(2_H)0�f�w��
���T���_���6��Vn��v0����m����9)7e"��W�	�?PT-�8v���f��G��!t�5E^H�^�,Qdvu�aV�x7�����(�>�������O�e�h�N�R���{v��w3�5�����P@Z�?���0��J�UAbqKg1�d�8�{7�AN�n�4�����������L����>����[������g����{1?������RI�5M��jw�_#Mvs.�1��iw��*����$�����UO>Q�:����L�RP��f	�����������V�-��� X��������X���z8wG�����7B�$t��r#;BQ?���$���bL���?`���=�
�IA�l)�n2��LUQ0��
������~����2��	�TSo��r&�����n��������&�	Q#��Q�\�C���g�l��n���z���������`�K�G1�r�d`\���Q��wo���]��N��������P���S��czz�y���6[���T���4!Tg�VuT���h^���J����2�C�.B�|�����T��s����fv�����O)��};[!Y�~���v��r��+���;(���d����LwN]k���a6���A"��;Q�y{�@��\��d�25�;��K�����]r���(�q�T�l�_="�3\{�A� �����+T��f�9�8���0v�G�iT�.�t6U����0R7�T��)����M�����^\�K�
����a�s�7fT���:N	|N��p����O��?���������K��U�M��I�3�V`n����X�H���_z�.� ��	�4���i;C���$��A����a&��~�^���n�}�G�U�>��h�|BME�dcE�X��P��(��<�l`_�����B���*������������B��L]�4/cY�4K0X�!�$>h��<{��"3����1������6�)c!cZ�",������]�_������o-"Mz�+\u�H%�M�@"�I��)@
�?���a��?� �n�/LS�*���<�����)��g}V��O�X7�t~�c��UZ�_8j�>i
�o��X7,`�h�����>cd|v��~>�'i5��76������*d&+^ql:
/ig��FRVb��MD��Dc�6���#�z@5���]���=4P��7}��_��}���FDz�)X��k�s���:������bP���a���W���>}DW�J���)��U�g�
=h�@f��������b?C ?5�����?��;	�Og/z�(�H�hsjhp1/`���S�26-��R1��4��lc��N���B8k�o�7�_)�]�� ye2�4�[u
�����l��n������;hE-Ys������H�X���O�I��,�b��������0���cs>� <Pe0m�
��|&<ox��dm�o�^>5/�`�G��OO�l	[�7o7q]!����0�������0�c���KA�O�Dh��y��I�hT%��|Vn��x�GE��5J�.���N�[+��D1+��n���X���k��IX LZ�N�Y����.��`�(%�,��������>6��c�:3������Z�����/�3��N�R�k)��0������aE�
�5����
E���z�Lf_�~��� ������$��%������
6X��;�KR�~nz/]���D�q�Ui�+i����u��dg�K-����R
�?���C��������a�a�H���%y;�T`7�z?����w�������wo,MAZ��x�TAx�N����}CY�7��
�����c�H��������-�
g�\���]K��6��+>�����q�5��"<c���N����b����H�����$	v���B[8N����@d���������}eRHY�dyQ�f����O�N����������H6�q����Q_�(�� K�-4��>��*��D+;��k����mdv	x��������A�B�xc��p�wF����K#9
;�K����qP�����_�v�k�����X�9�+k��K������E����4�r����:�~�\��Q�{5]��t���y��d�j��.xA���a����xh��.�����fP�_6�e��u�����6_�2'�	V������� ��acx.����;]�m���c��>�a���njL����G��)s�v��f��LY@�/���N	"�L���5
:u��J#�$��7P�`,*���C�$���(��1
�+�Z[������,��u�4����NAy���Dp�t#����Q�������+�1���c�w�i��m���t��Y���Yt��{w��4�wWn������P�/3`�O��J����!�|i<�{]���A�m:���lh���A��)�H1����#i"��C9�U��N�\�;_W���Vw),r{.X!����,��T+���
�������~���Y.�E=�������
�LcF�(%�w�K�{4)lL��&1>�(��rB/��CW}�v]�?z�T �$�G7���@���S�����gS��#��{� � 8�wJ8�������z�����(��+���7�CX����H���[2�Bp��C�r #���C�:��CR�X�>a���`��+��c�uL�e_���$�z)�S������i���9�&�F�VJT���3�g�d.�V<��X�2���T�W��!Qj��J��
�d�`;�� ����Q�A.a��"����0�jTB�k����j����`����L����@����P��W����0CsO�P���}n:0E)G�r��S�y�v�C��7��.������pR�SX|UI �����d@��a�Q_���x����B�Or��� d>��R�_Z���OO#��UqY�h�l��B�M�q
�
�����w������>.��\�N`������yO4���@���P�}9vH�}�h��!��1\=����Q����uS6M���Y.�h�o����Le�1��u����w�dx�H���)t&X���A���vz������$O��V� �5� ����@�U2��Ao��]��<�	A���F{�]�-��7��
�Q�D��xt�����7�,h:H[$�E=�t�(��
�-oqx����nW�4��Z��]�)2'Xx�j/��`�G�t����u������&x���!+_������xxj���x*��{�m/��8�y1C_7�<H�}��s�s�6*V�+��q�uW�pM�vM?�������'���l<WUP�1�5x��I!�N`�<�����).��1�0���z��������zp��������}�u�P?5C��
1�f��0��/�������l�!jH��Z(��J2�aD|���q�(U(�Kw`����W�����[,O�}
f�+KJ��Z��F{%��	��w��l����D�?����� ���&b���!�>��E6	�����x	���
<�[�#t�b��ZfgD-La%��%��9� :����]���j�C
�0������2��M��q��5Ft��F�	T�����l�?UM_��c�h�u����%lv��!�pM��tJ�xks��e��0�e[wu���4][m8��^.�{�ZIB�g���l
��(���d�f�����=�%,O�N	� �y>z9����$nj��]��}�3iz7O����`?
�������
_���&SCV���Z��kk��C.����a2}�@N��m����M���[��>Dp�&�@5HF�;���O��t8�^s���S��H�C@����mf"����r�y�����Usr������-�x�.U�3�������wL��<�nl6oc��+7����	�I�k�#�3����mO��T��R�]S[X$!�zsy�s�t��+40��2�����8��	�dp�v��R�`�����\�$P�=�'x$U��}p�@<�(�A�p`ylh.3��v�-eib�K��jb�P�������3�rw�{ �"����>H`u|2u���&11N�%G��}��'9[U}��Y����1bp�t1;��U�����D��`Z�fG�
�d��g���}s�|.���X�\������Y
�Ws^i!�[K���l�����8\z���`s���4h����`���R��[�K�lZ����lX�
��QH��R��n"7��G�Q����4J�iO�E�\�Rb�4L.��Ig|Xce��=���@?l?\����_�����
1g�������M��F�*=�/e��x���!l�qO���v�?�����s���O6������94��w��t"5s�v"�k�=`�(�xgJ�D���Mzs��?^�������r�W�1/�`�"����<��t�������>{�j��A��p���~;i���R1A^}�#�3�kr����M���|��R�r<Tgp/\��/�*,$|\����g�<�"�������T��,�kR�p�I������7+�p����q��!]^-w}�����l��U-�`�?���`����&%t�S\�$
�M��1����|p��?���������/r?������8���R&
�ZL��G�b�t�Z���s��]K�u�&���d�>�r��n��\��N#4����� �"#�_ed�Fx�>�j����9������e�0�\��Zh��BQ�s�C����<jw*��#�zM���k3H)U�l�8%3���� �A��m��(Js��f������q��� ��8-�2�7Pk~��u������5����~~�X&r/�c����&��	tCf7��f@���%`���1A�+)gw�I��Q��d�d
�J�<I�~����V3����������bZTl?~��vEA^��,l�N��,��E�9���� <���qMQn6�x�������^�H����l�]z�0j��`�=�����Om�c���*^�h�	���������y:�:R�����Czph�9��SY��]s|*���t�z�����fJ��r]h,.L����u��C���i0L�+�F�%����!�a�I%��&��%�
z����$��������k)���c������@�����u!�������3y�5�J��) ��c
���2�y�,�O�4v�]	�F�P8t���B�2��e!U:�Ri�$���E�����!f�3��[h���5�!q��-Y<��V^��'R�>j��-�����Px�<:x�WR����M���D)�iU�pM
�Z�]��R_������Y�Vgq��T *����Y��Bn�M�
������������EM; �:�"A��R-S����zO�Y�s��a����`9� /�x��j�YT��z���?G�+�*��pdf��+�A��9H$�t"e�������|{���m=08�}:���:���,q�PS��V-X�q�-1����uX���f~�#
d[��_P4��F�1r�k1A7�k���\�*P�0��qP����h�����#nP�6*�R��y��{��������}���m��s�`'��8������e��{8I�w�9	:s��R�Y8u��|#E.X�?\5o�'o�9���=��o��5,|Y��*���g��'T	'��{��C���OZ�I��k�W���e#��U�mAq5�}e`[�S5���	���KD]��+�����O6&� �L;�l|�|-re��xIy��m�����}[K���,��[wx_��$q��������$������?�M��I�q�`G$V�g���zbJH�����S���M�B�81M����/��,��Y���J	!H��"l#�W��~K��4�K��v�B�%5;�0r]��
&������F����H+�&��(��
�G�3(He������J����r6�k��`�L�KI�q���0Z������a�7*9�33���)�]M!�$$a�FgC�i�]�y�7>Z��7�n���E���
H^H���6�|6�����<��rt">�{�(MBu�{���68���`����3A|`X����|l���|�
\�($~C|G����N�T��|9�;L��6������'�"M�����	���2���A�����������x�dA�L��/��M&���P�Z��T�r���������9�����4M�8�����^9�X�M������_�Z�V�����������*q+�g{{�rA����"�d��J�v���5@��b-�J�l���y��t}��@����4�Y���
���'�4z"m2{���P��G1j������!E��	����1z���Z#{t�5OQ�L�m}p�t�^W����]i�p�aO����Ku�.�F�\h�c������^��}o
�4+e�3��x��^��<���n��3N^6+�����u?,.�-$��7&���z�H���;�Sr��P�g/a��=?�)��y�P�M��-����G!��RM��'\�Y�z�|r�r���]��6-���W����Q�Bj$���2?�-;P;_���m����Z�"\��`�r�9A�D��w�X�o��o,9��O�v��������pm��p=0�D.����-�����B*E����	fRGK��A�@�����m����0n�C
�@yc�QkB|a_D��n`7�i������t�������.��=���a�	�q@6q����R� �v�����
F=��tu'�F��m��Y�N�e3p��:N=/�oBE���~�&�,�����V��_�����Z���i[���	�=���5�y_=�O����*��5����
�!�� ����&($�3� ���ia�Z
u�����P�����CC�si42�iQ�����������D����������T���^������k	��
���kI�H��lT�A�~��68��������/������h�u�W.�ri�\6����1[���P�N���3m�(��e�/r�|��*����tY�" ���K�*�Xc.��O���f����i�r'J_��'BW�zp���ru��(,���5�S����_�������<+^�����������S�/.`��
�&`�%D^��7n9�q%�v&�J0���,�O���"�&ML`����z��G
�6�����������������yjX�l#��j	��$�@K@fU
1���1��/�5��}�Q�6}�vvw^����* ���-�;g�N����]�A�7������n]��2��	�ke'r��=����fp�_$��	7���@n$7
��S��^�8!M"��{
��z���!�i&��:3h��b3%7�@��tC�&�����}�	NO���}�����hanjS�0��?��5Yd�7V�6
�?T�ZW��@��s_w#eN,����(W3en�k�Pwn�U%.��MAV�l����<��#�2-��Lq���VS8u������<�=�u����?������J�^��?�e�Z�4]Mj�K�K�_���{]�g�DI.GL�������������7n�Q�����f-a��fx�	���:,�
�z<a�
��n��������9��i-v-�H�nt�8O�7t����/� ������s�����4�7<�R~���B�N������������eo�����L���MS4�S����Bgf�������eX�	���Z�E��-�;S��0���vsV0�W�r�I��+dC��`������Xz1��adpCoM'�-�#�B�do�xZ0�.����+��G9(>�g�*���S[O6\,���~�yGg��
�kw_}9�����V������	E��v���T��������R��A�uk�0�2Y)�U�<�����J�pa8����k�P���h��w����q��Kp���Su���'M;]�k7�8�9����X����?9��wi�gWz�Ym��S`3T�~I��d|�=��U����ua�iG(4��\38Eh�>�[����_�p!�e\_�8C�})d��N�l�D��w��'@7��[a{4G.3K����aY�Sq�R��k���j���b�I1�n����QF�R�C��&3��#��N
��W�{�e���t������������q�s�����\q�Y/�j��Z��EK>�����Wx�HS
2����G>\�rW���.�50����:w�h�q
�2����������|��I`v�^`~f��m��Z4�JW����s��:S��S�����\��{j�������{�$�X�
������?�GH�����K��p1��{-,�.S�A���(�o���Q��S��x��\����K3����*������v��]�B��,����<����t������_`u��Yx���5)������K|3�������
}76	�%���~��P�$�d�����)3����<�Q�J����7r}�	
��_���Ir�H�}`T��8���B^k<3�zOv��b�%�5��_o$�`�!y+\l�,u[����{���Ra-Y@������n���zh����y�rv���tSf���EId���n�=YdN:Qz/`�q5����5��!l��L���{#��������=N��o��ak��>|:)
�k�0{9����v�=)�������e"�; 9�����8�-�R���|'G�i�f*��n'�7���� w~�ou?4�?7�i���_�M{@M� ({Z��UV��4���)eZ�U��J�AG�u�a��5P��b��{n��y����>]Ut#����#��dJ�O�Du��
���Q�i���v�������UF�+C���X��Txj&��
m�,e�l�����A4�i+�n0���sV)d[�~�]�rx�1���_����.{7�|�.n^(�������E��TE��D����!E���W��.���i��'wp�f
�F��?w�a`/6H����@��.J�_��+d/�U�4i�["R�7k\WJ��qAh�l�W������������������F�/���=f-�pc��B�@�>F���? �m�5�G����i�b����{;q�K
B�Z�eT������+D�%�&6���n��v���b �����G���:7�<�����j4��=�I��zs6������Jr�F����5�-"Hc��������<���W0>z��V3]��CF/!�������:U������a�J��^����Uf^��uN+7��xLe����H�����x.�%�a��^,�L����|.#�kY8Hc�o�Bm����rl^�����6� %��B�R�\@I\=g�����P?m������>��EsX�����v&'����O/��s��Y����!t����FG�W��
;c�������syW7���|2B�����/c�qvps�>�������
f<��[���x�r]�
FfN>���+���*`��>|�~�5�xeM�����AR���rMB[��wPX�������#�
x���w�k7!�VUj�Ic���`N|;�i[#��������"�K�3Rl8J���R b]�\�$�����MA����u����ksz���>=�����ro�F2`�������n������z
x���t�J�����2aY0��{x,2a�%�b��`X��D�����k����fw����*Y�9�rFf �R�wW���(qn&���O���#���~���'�;f��oJ��P��@Vr�>�MA��
	pk��PZ�;�Rl��GoKC�+T�"IK��Y)��,����,m�_|,�i1����
�����Rf���&�O�����7�{�2bU�cJ��.����%P���\���X�������]�o�*�&d�R���f�v���Ms�5u��M>����H�<)v�~�X]
BR������K�|��m��/_�I��Q%�����J�h�y)P���gs������{?]�=���F�q� /�	pJ��#"$��@������������qS!"�H�.����D�SL��~i/��?M,E�mz�LUQ�y��P�PwT0���+Fs��U&Y��d}8�2��w`r�q>�CU������,#<��
-�n�@�X�_X1���zE�w��-l�C��(�!*�������;�~��g|v�:*q�&���?�"���"������-���N��wMQ�w ��c2����$���_�����m��|e���}Z��~�F��D� A��uA���Z�|�$;�c�#|��Q�!/����z������/5fc<~����J�t�P��'�vW�
2�R>������N�Px��D�P@�%C�9-[�b\�������wf�*��;�V�o�����Xb �:���~x�mJ<�:�*0��u�u�u���%q��7W���_�~��u�-,���E��t&KW���;��.��� ��;�=��N�Vo}}����S��n�$I)�>����J�@4UcI��}/�-���Vbq��R���f��K�|-5�Q�"G�"�55#!_�]]�]w8�D�Q� ��8m��P����o#��0�v�!$�������c�����4Oc��c�X�+��b�� �����L)U:���Z��r3���@M��$���?�����!T������|��(��,6���Y=9~�`�D;���-z�g�^��y6�H�z�Ht���'�']�L�JO��l!&��%Ha�24�%��p?�=�5������m~�{���u+�J��nP.vK�V���S�L��G�o�3�8+tTk��a�dU��$���P���;nw������=��e�����v���lJ0�C:q�$���?���c�yoX�����E��Cu��<����Z���D���x^��/X�U����#]�O��	�����v���C�+�z/-w#k��>����u���}�
����`f���x�k.���O"A2�	Qa�,����?�1l������$3�#g$h����8��x1�����V}�s�M@N�fwCq����8�����5����n���|8��������f�g���r���Q��K��M@���`�y8��_���H��
��6Fv�$�Yiq|����6n�P��~��Z4~��.��@H(�Q$������]�h>���=�����j0��hJ�M�q4Xzc0��g	t���khz���dQi,�*;
�V
���zqy7�~��R�����`T�ihm��#]Y��X+�ERC��x��t��o�����q9X["��1�{�����)�U�.�>F�6�r����5�p:6c���Y������{2��<�VV����N~?6�����������VB�bX���t�]���b���'xQ]�����k&}c���u������5�)�>	$',��/���[K�z%���#�����9��6PRAJ0/D��}�����j�H�R
����s�'O��,m�X`~�=%f��Hm�������O��/o��������_C���+��������Asy!�b-���Q�5�a���w���a�$���-��AU��I�&W�R��}�}}�������dh���zr������X���Va\���������0���O�����%�[��Q>	��%k�c���R��Y��_S���v����������|�PO(��-��}������<:���x�2��������!���$�a(�����>������h7�o�|����%�wQ������6�eQ���DU�n����YD������h	O����)���68�0t���R_2��F�����iI�/$��e���F��v����Y�t`� &"$LS�aXB���g����K�;6�zs|�u\��P��M���R���&T���r�=��{�4�>N����*5����v�`+���pE�����������2 ���]��X��J�E>��4���ucB���>�����[a������]UBb�^���*E�5to�p���}s��w��.�:�6fX�\�����uF��.��?���Mp�r����=,�����1IS���&s��8K
O�+��|����K���n�1�D�����TQ��i�r��|q�����m����MZ����(�|�,%��g�3I~�t�eZ\��T�|���:q�>�M�Q�&0i��S��wz���Brp�������o��D.�:NNd�|�%,�$�P�t=�)Fc�=��7��q���]��S\�mRWeAJU5�&3>�1B���k!����[����rN��n�y4�d�n���u�i5��<~����t�5B�1!E`�������BH=e�>�9��/��u��>�}��ee>�Q
H������9g���GW��ao�������:-��)'��r��gpa����9�~�qn J�u7i,�����=aI����*.l)R��P�"1B�o%��(��m44�d��
X�~�C���|���>��8�P���&$��n����Xr��#�d3�<��O)�X",��3[�[�����)��6M\=&&�����
�.v�D��"��Kb��'K�J��_^��6}�K;���f
ll�!�psH�9�����SG�Q�VT��l������[��+)f�:�9�Yf#��X6�&��H|�r6�������P_�j/e�kZ�,B��"9�D�����<7���P��u�)��<*{tA�+��f��G=�|{	
����]���^���K�K}�t�<��GH��BX��v��Q:
��/&��~J*,�~�������;�����o8JQ�yNu��������b	�X���KC��{��}z�������K���AG\�����{)�D� Q�&�b@j�����q�#j�Y0�.i�]�[l.	T�7����Q�����"�yS��QeI�v��O�KW���LelHa+`���9�b��7j�Q�vvATTU�4�~Y�v���dX����'������HTS]�z
����roD>`W����T�/-�W�Jq|K�sGpG�� /����@]����CB��A���x���C��J�/���|v�{����VdD��)	P�J{|�8��zT@��n\�S���~?��v$2�$RW�C��=		UY���DO���Qn�y�x�Y���L�|�[���5��P$�����H����HZ~J�J���E�fBvF�����=��_]=�-�!Q���t!%H�F!�_�
Ia�G��l�����[iE�l�����-=�����4@p�S���t1���K�����~A��I4����p16$��p�����3P�wO�VEUw	�QS	N��f7#K���S0��O@�`EAr�vB���E��
��:+Yn�~��/�M*���oQ�)��S���~��w�""gR�$]�)'gRmK�m�][��!JA�<����t���^2)Q�
!�H�����S����i]����}>>.������(`E�?=��U���1�V������n$*���w�2�APw�F�e��U�H&Dh�M@8�����������o����&%��\_�L4d�@���,x�pW���4�����6]���b����k7�FR����x?~mO|��"<m��\k�#<����@r��Ka&|mA�����1�����+n��%�Y�1�*<U��.�����~nv��?l�!��Uv>!�(��(��7��H���oi�@�^1QJ��=)7#6u���_���eH2$zK?v�S"�V�x��{�9������|�4C{I�J��(���
K�<���!�22�)o�'u}���v��Ue,������(S��Z�
R1fO#`��	���-��qS�Jc�	-a+.J!�_��������f;�]sx��U�w���"��4%���}��-�Pat��{�����E3����|�����
kKj��gs��c74��4�Cp���\��&>�hj9���^���h��J��1��7������u��-���a{ ���d�����E-a����*�&�R�m���^?�u���.)W��gL��C�<�wq8r��T%P_�d2��d�~j���[���������^��?�o�S��NWI�����G�.b�������nv(����ZW�_G/
k����/��c����!u�Gd	�v��z���^2�`
9z5f��QQQpJ����
�L���vL���y���=2�y��bH�6�S�|S�?�m���M %= �DP��^���+%V�.���(^w�jk3s%Y�������<�"�(��A�_��}����X��4�F9��:jX���E�0��+V
�yh_���b��v��@���)�6_�A�G�t5��3�0M ���;�����������r������e~��r@���I�b^��5Z�����#W1���}7������'p�{�M�����^�tv�$�q5�.M�P����������q���t1�K�'wn�f
�R"�D	��8L��TL�@\��s�����Z�T
p��8�[�'Y��U~jw�f�G���o�5����si��s��E�H*�>	����,�vP��za����#K�����t�rWi�ar�&��jG}��o�G�kHs}&uEw��#�&�<nirr���*P�"�������4����/D��A6�k�Y��i�=�-d��HC)k�CR���
����<�[D�S��Xo�e�	VQ@�Qp��~���|y��:����<n&g[zC%l���2���{��B
�`�X�x0�D<�����������0	&3��6��v���Q*:XZ;�[��F��&��G�d�@p[��Og7��@�)h~!wg����������\��v.�_)2���D���N2��QX�d���t�N��{W39��1��U}u;�e{H��^���3�|%0�h���l�m�9B�Z�TV�)�����I�	��>�w����{��Um2�B����b~��X
0_q[�����>�c�4��Afx���uKD�/oFv�,�U��W��z3{P�w���r�c�����b��t�����(�H|)��B�,���
���x��Z.HBU���O
�V1IL?�)����=N��9hG�(��	35��E
(X���R�h���b������z$7��_���B���j4^��0=^�O���]V]"Y=���I&��N1���\`-���I13"��W�\�Y�R�T����^@Z����;���urA�L����E��0rn���_"�?USf�"��!�,>��p�����z����=|#��Z�������w�m2\p�����W��(�������m��P�M�y�1�*�\[<T�[���/J��e�\4(�+�v��2�m�3����1i$�b�k�|������\���D�b�����a�6�S3���s����T����4V��6��/��1#��E����es��e`�`�`e�v�1M�26�,�F*0|������u�����Or��M.���|qB���H�������w"��.��ew�<}����U���T����6S����8�
������}6Su��S�*+n#�b���6��3��4L���z?%l��"���-hA4~�����|�K0�38����T������p���i�;2}�V�ha_L5>��N���l�:X"�m�������pgg��SR�wTL���[@��,J�Ma�g�����O�]�!������$�9T/"t����O��)���BR}CUp��5����"�!.�"H��*T���f�~�������+~��LC���HD�rRH��4)����'x���e��"H����Dn��,����1�\/��>���*g���76�2���Ib��Z����Hl����a�BT��;7_��qj�b}tl�N�z%:Naix�]f Do;0���1���2(B3P^E�}�����S���}���M��dP��������������/g_�	O�����������` �����Ab`�s�'c�
P�O�JZ���s��}D5j&��������2p��k�m�����}��;b�z����tz^�cu��*UZ(%��V&��p�j�)�
�LL�������
�&^u�g:��V[�JR0������R6��`.xgY����/?�B�Mu�2$�����P�d�����\�w:]�������q�A%X`�C�(��s��"�����V����6�������x�#Z�5r$�Y}����@bgu�Y�
f0�/R�]���Gv����Mc*#�Ky��a��/����c;"�X�H�^)���N�"WD�� ���W�/?�/���|�&�=T��}�T�����Y�.�:�t��P7n�������S�U���'����85&�
ZT0�5H��ksx��������l�"�����0d���1N��n,�9��e��/eY���r{8%^��;w�����`�1�n�����K���"�d��v�&�.�<)'���1�a�~�e(�5IT�Fv;�;�~�5�s���Tc���9Z�8	����kri��M
'�����
Zy��B����PJ
:�<�D��i��~�����Hap�8�
D�0����DU��09�e���n<8Wxj�� "}�b���0��:=n���/��w�:2�n�IR	�Z����h�p)CJ m.�N�G����uWu]�y��r����2����T���>��ky�����!x�L����>���}��� �a��/�So"�oU���A��8A�H�
��{��2d���,u6�8q�%� �s��W�������;P�jC�
3�t����}�Z�@��z<���������bt��9�-j\o4R�N����#���|���K����S�O����$9��+@:T.�vn�/p17i��4�?3,�L���@���1!:0���?�r~�rj���Zo���;}X2�2�N\���U�u������o��/����������|���6��A?�-��;�����0��e�/�B�k�����J98�~t���[u�>��^�s����a�O/5��n,?���,c*F�-�T�0P����3�6��a�``�~:��������x����l���9�����R *�Ari�q��t�x����6s�=�m��������l�X�_�����N��=X�(��_�����Mb�.�F�7)3�3����WU6�W�[��k�.������mSp.}H���2w���l�$I���Q��"pDI���t8�����9y��7�Og+z�y���9�:C��#�^(�l�?=uu���??Cn��G�6�������F.�qV(����$#�H����EW_��z���] ����"ni��b,w����o+�l\�z��������'��Xq�S���Q��$�G�?���5B�0���f[���'��=w��FV�D� �p�)�"H��������
�U[>m������([A��*����[0BJ$��	y6���3�.��R�`7q��i4z*�q�T��b���(�����v?_��\������&I:�#Cf�Q�2���Cf�I��Q�x�2��?<����O1��
������gSi�I5J�=�H:�������q��g��?��n��b��t������}�$]!
9%�+�\�����M��A�<G�Y��0E.�����td�����T����������f�
J��85�Q�����=�
K�����=�����}�C}r��GH����wc��z+��@�Jp�
���?����@�_o�8����#I�Bd��>O"Y2�p���B��!��I��}N�	M�e(�a��/�c��2)��L�_Fe��X]\����?U��C��I������HS�T`O���7���J�s�P��7���t��i�������	e�o�T��F�)�q��4%�L�>�~�GsnA��(�(��n2V3�}��u{�Qg��x<�6�6=��%�4
��q=�V�+RP�q����B0(������en�'�K���W�R��u���u��D���8t9�
b.�r=���9�'Q�!"Y��fQ�EY�rOd�\����M�K�����[Ji�6����L���	�	�DK*�@z��<h��=�N��S�IT�>���Db����5����dI�����}�^���0qN�J� ��`3F�d��Gc��8��������dz��G�}#�����/Bn_u �O����uGY��o
�Ks�O9���
V`��p�g�A������l��
�S�
\���d:xH?���vM�����eN��nt�1��]�kT�vfj���MBGD��B��1d����J�9�
�������E��9G��=���2L���p�A�K8��H{�H&�1H�$�����o/w�_�����(��U.G< ��F8�N"!8��M��4k@��g�����~2�pD�*���c����'�A����A�4�n����t��kI;��R~~���3��~��!����)U����f���� �`���W�t$d��*��s���q������������;����|t�\�P^PB�c��8��B�o��#6:'�?Jd�F%[M#����X\\h����u5��eq��a�+I�F��YE_t9R������e.���8���������h��'��9�b�����
�2�
���t�����<>�]S�'l�8)�F����[X�d���7tp-�w������
<���NM&s����Gx��-��kR��d�IN�I[�6�:�u:b������3d�/d�
�x����������u�@ Vt����h�� ����N��rq>�Q�����]P������(B,G��	���@{i�D�v�
��������YOc�s-e��'�v������D"���f�U.����I���q�����\�_+8���Uo����������S!���BO
z�E���k>M��}7u
������m�����%nV|�v��_���t��
�[�������_�C]���-����E��U�[M�9������H����V@s����
h1��|�����G){&*<ld-1j��8/.i�FO5�������|)���	�"Mq�Pq�:��&�}ec�/>�����$� )	p����i0��:���s}%������t����c�>
$�I�e�@t�@���*�����
�O]���������
T������R�
�*r)F��{�y6�O>�����i�cK��/�$���H��������i��/���ku��<a��u��a��vA5�S������o��wo���*�FG��o���s����m45�"�az�vL���$!p�E�K�����e�%��*l��g����|��>��T/F	��!��<�Qnz�}��g���X��	3�r�����4�.�{s���++������0�Q�)F��������e�5J�6�:DW������x
N�������Y�7�^�����������}&||�����S��� �h8C�i��[\��O(�-�4aN�J��4^�������;��A���ct�{��e�qu����e��mW.�z��L�M���W���J �p�6�u�M?_�|M��O� �,��,F���0�	��Q�b�NH.����xn�>����Iu�k����i�
�}�u_��@�����������������@��
�*����DC�{�z2�

���)}���m{�-r�|n�� o3B�jj0���!���PV���������z��s��X��>���\85+��gaV������0�C�c����x	C�����#�����������$Y�^%,t��rz
��wr2*�3@8��f��������pn�rOH}�9��O�a�F�V����`�ZQp��D���m�]����M�����N�6xj�6���pW|�d�r����c�`������'31�f���<� ����KL��w�e�{ �!&zw>�����|��G� Y�!�N�V���Q�v�E�3h���^��S�t:-@��}\$VX]����g`W����0�i�R��P��(����9@zn}aX���O���}�0f�~�w�@8�KtEdTo��DW^���@r�:�%x?��������yj���3]+V@`��M-\�Esi2K�������#��O!�4=H�����K�$3�gy�X�F����������q�JqM;O[�!G�G�2��$$A7���E�������1���v1�y�%.���V.r!wvBVO-n*pV�7���o����7�bQN2F�6W`�[���b- &��7���N�~uy�5e,mkj�k)V{H���R����Q*�V�.6����x�dx�Z��kC�D5�3eB�C���,?�h�l9��m[��=K�\9�����t�o�h�v�C�Y61����n�[���e����,n�
Q"%d�dNJ�2��KTb�P���F�\�~��r��0=���O�?����� s���W6mh�T��U�����k��6��������1�4�a�1�����	�XF����b�d.�,�{����5o_N�n_�o�_����$�Bv����8�3���&�nB�k�������uNi�^�=��v�w�������H��\v����~L����$a����J��3�������\��u�cR#�S�M}�����FO�����q���$��%���3�P�<��;gi�*E�0dM.�<��^���������(�DD�WH6
����-P��2���[��������V����+z������h��>T,3f]����D@Cn|m���9���M���X�tb�r�TX(�X��j��|�����}�eif�4,��j~���( ?N��2(��f�����Aj��W_�y:�������$��e���������[�{���k����9��S��N�t2�W"�I��H��`����xl��>�n���he*-���E����Fs�*�s`�������MeN�_$'�G�Pd�)���Hv�q�WD~_����� �K@D�M/8)6�������,�H���FP���+�l�[(�;�[��xM7�m�b��n��y�J�u��G��{�U�C��A��o����?*D�-)d1�����;���
������|�;��W"-*�2`H�E��9BW"+G)�8�]���9	��8�c�HX�Kt�["a��Z���&���=�T�E{��&i�*��tp5kG�{�Y$jx7����k��l!l�����O�e
��}��R�>����`}{�2�b�n_��|m���������� �2�K�n[8���� ,�������q%
�:��������t��ZKttQ�x��Hg*K�F�\VEe�B��Jf�a����n3��,	
}\�o�Z�{y�����"��V7uZ�1"���xu?������xZ���w��`�ct?H*H����|���f����|�7,���Y��oe�����b�WZO���t��i(�tu��w�%��_f '��_��^��+d���G�����H+rw�#;xw�� F!�<�<�����o�=7$�a�Dn�Z-,�N*F�1
���L`%�$Y����t
�6#-�(���m,!arA���X������3�M��-x(>�D7^p�)�F��;��������c����t�^�!���K���n$ '\�n���U��|<V��'0���/�mS����@�Lo�D���;P-^w7IR�d���`ctFT7����Z��#����j�)��ddr��K�=�USzU���t��E)v��
�Fs\~���q���6`z���<h�/�����t�����g����BQ�m`�mR�������k�q�����~(�_�L&�]{m�`0/�T��Z-)�j�����U�V��'3D N���/�|����^Q���9$��ei������6�R����e��@��3.��b����i��B�NjA�����Qv���9}1����D�R��}��F6Shn�:=���a7����8]V��(����9p�,+�JV�e�4���C��������(�������M�,p�'�B�<P^N/O��W�c��W�2j����X�p�������]���*X�1�����I9���7q5���(�|S?����Z�j��O$�^-���"��!	���cf�L6�(�>os�t���p����������C�����H�<���R��n����g����m���&����gY pI�X�h����/z����������[o�������rL��g����(�iZ�v'G��Q���j*�I������;�� F��'
r�����<�'w'q�8R�N����qNQ���<F��F�EAdc�|�_��)M�xA� T�F���f�����M�e�y�yIr��j~S�h1�|��0�T��\�L���Rt��iS8S*f�����0����0%O.�Y�Mi�{yL-cr�,?�8u������+~�4�/�R�T�D�$��R��t7��&,���pa�Q�@���+����lE�A����x8�� 7�����z�e�CO)��L����"C����;n�0z���I�7���|�b.Z�$Z���!Nce^tA���t�v��U��� I�BF�Z�K���y��^��hj�g�������0����RC�
���	2��e�o���|�|�ou8��,�;<x�C��K�j���'�5�)�������@��:wD5R,�eq�����w �TB
�E]����A�z���i��W�.0~�(*$$�Xc5.�A�G����w@��
��c?��H��`�%Md����`��=�0�3����>��fr���V_�o�H�0����@0	����/��6��h�.�y���r?0������>v����������1�.���|�Y����3�-�'�\uK�3�8�I�W���k�v�aw�?�e�Z �agCnA��G���T���w)h���_��G_�'�wA��&�W�^r��[*�8+�)W/����J��W�T�Y�e�6�=�������;�aTYLx��{�p�t�e-��5*W�VJ��,�G3c5\��I0$�@;�\�3B�@���j1l�:*�@ �p�����O��������K)5��H�=���Z$��
����$��i8�����b�� 4��x�3��F��DT�d3������kfGEr���mN�V/��o���j$�>(�3�^�>��k/v%	�z2��v������*/�*E4+��d�;���F
-u-�p]J�� ��Qc����G*S\�cIE$����A�pl���58�J��C����e�,��A����$�e1G�h�8?�$����h������pE����9��W0�	�o�
�?��oNq�8M�pk���T���$����T�e��N�gO��n'*�����@�f������<��6��I���I]z����d���=�A1u�U�u������Hc3IM�4L[�B�d �hl��GxZK)4u9���%��-��.JK��%��uAcIRvo	XB��$-����������;x��v��~^�E��j]X���N'�1XGB	s�j�]>����P]Y�tX�w?FS�$U��3��$��4\��H���e�3'~�`6�����,Z	�S�V�P�Cq�3%IP�����g�KN����vv=(���n,���������P�?���s7xd�����4�;�&)��q�������W�b�R�6|i����V�id������jP���v�i-����|:@B���WS]���VSpu�I�,(H�	2J���+�k�+_f�/�N�+�=M�6s-aHj�#:i��G��I�W�!��O�S��3]�n�/��a�����/����|�	�~�]�N�]lKfb�9T^�<�������js|�~^���e�E<%��������L�����P�����4��7%��OW����T���@�D1@#y����P�������3i���� 5I}w(�,���LU�4%�����F3�|�bw�"��I��B{�\�������B�y�\8l�����
	���r]	W��R��\���I�����*������x4�����SK��o��~wx��s�]�C�]�$�����WIl���N��O/|�4'>v��0bt���,`o�f��3d5B�r��%�5�J�{��I�y���,mS	���1@{��F)	�O[�T�3�`%�Av���`+Q�J���_�����^�Sn�����f�����^9���>�p��
��V��}�
�Z|�qI~0W�����~p��}:m}e.
�P���zbY��@A)ps��E5qo_d��{�!���m�y�<JM<�*U��QC�9�at�_kp������A���/rEgK�R�=4\b���
(k�4���6�?���HW�ie�K|$��7�M���������K��������0"+K�8���"��*������xh������H������f�E�L0��X���&wx-�f<nIw��]�K�OT��?J=�Y2����y������������Jiu����%����vS0��CB]K��t^�C��X�WdS����j]=D�k����;_����W.����d
�"�1+]M��=0�����ykt>���[j
C�V$��jx���`��e}��� ���w�����o�������$	���������S�3I[
���%����k��^��4�!b+�1��Wei���
R�� �M�G��/�_b
s�xn��-8��u=(�
 7+mXOBn���^*X�c���@������yki��S6�F��2	��8_!�D���Q�k��u���@c�2��k���;�A���*
�
����������*���$+�4�.�&pW��=����s�|�K(���w>��a���D��+��}�i�����M�u+�g��"i��T�?��Ok��9y`���a�\�2��W%��p9A���P$��}��+�s�9�C���(�U��ta�~�������F�(��-���@������+��g���CV�(�(��?������6wj�+8�c���XW%��~�S�<r��HZ^�t�-J�l��k�c������*��iX�f�����B��L�ZGa�	�Q����[fz�*���9`n���o�a
�M�]K�b��'	������e�����~�g�>�	�����(�$��H��n_r�������
���q7T������G�EI�I6ICr����Z�E���|�����m�4����_�n�a
Y7�������Fs���`+S�Q��+�R.Q�@&����7��WH���v��}}������������m�
.���#�.���k9�WA�����j^�e��l�igE"V����r�@��fp�G�1�=�h\rV��
G�C��B^
�L�u�[`+L�f�$�^}I�LV��;�2����ubjyz�\�%�����	)^:G��.Ji-����
����`G��6�z��:Tjuf��<�=�|����������gd�~^����s'���yF��������HJMWG��x��=��t�9A�r�
��c�neI���b�\���'�r'"H�r�@�������u�j���q5��F�A,���F!Y ��8�����x>��i�w���s�O�_����Y�\�x��D�F\,�q��]����C��������x��S p��k������������?.X_p������v#}U���7�`
�q&�fEI-���v�v��*���J�U�/KM#Pk���Hv>��{��s���5Xt�G!u4�!��H�����B�;j-����Wc�~�!&��ga� �����������y%��W
y����M���+�c�9��jkY�k{(�X����L��0P�:*���n^M��[����<�{��v'���}�e��/` �E�:�[�z�����k�Z�eK��x_Rn�H��lY-�r2j�Q;�������&{Q[F
��K�MO"u#(�qX*9{b�>s�����,O�r��'�q�H��|�R>���U������37�y[XI���F��]6?Q���!�|I�H-'p�5E�GJ��_�#��S�tg��(��*������%N> ��V���������_��lPs�y��Pv�����t�5U*q���Zn���d�'��k��
�Z�n�%�85�p��M���{��X8�������]���QHx��� ����`���3�[�z���2���	h����n�7�9���d2�g���q��1���P
�D�A��@d%� �����as$������j�����!K�#+6�6	0M�����D�y]�j����KQ�&��x�g���p�caZQc���w�����:�n�k]���M�u���/��F��IX�OL�HBx�&j�JBV�K�WV�����<��g�j����3VU��?���;H
���lh8��/�d r���YWzZ�
�2���o0��c;s7\~�!�p��L5�B5���+�7��\�q^Nr�Fe
p��|:W��*�Yw7�j��_���]���E�+��\�~.}��[E���(kW��}{l��?8���aL��c��� ��R��{m5=d�z��hP�������hs��N2�'����I����gX'$&�����i��~}���^��J��G'J=_�K(5���}������|��@���/$Q<z_�RR��!C�����l�]�uZ
W=��o���2���1%�Os���X
?%����?d}��xlpm#Lh�B����49�&o@N"1�nGrR��J?t-��F������6/2�s�^���}`/��`������6_`�9��XZ�J��P&���Z&���{�dZ��Z�v��S5���%�0��'��~��`�����{����f�?��?3�#��K.��=v�M�}~f{7��������-��,�a�U���K%@��B����-QMF��{s��+���<��PbY07���1�%'�"ae�TT h���GXK�4�u^�G�	��R��z�� -Q����9��Y@��;�`/l�Lc�;b�]�y���>�L{�d�UQV�0l�@���[��[t��xD��LeQ���>C#�e-ZT{c�S�� ��!@���r*�!�)a���^9��p�`�c��R��lq�8����������z�{}����W���`�`��;s�x���w��T9b��n��>������H7�q/���&�����*�4��+9^�|���o�D���u�%� $-������A|q���O�/�A��S��w ;w�\��!��F�d��5�X"%k�q���S��z�~/e��KJe|��<������U���?��TO��^��%
�LB����P�=7
��Ax oT@i�����7���^r��<>�����A�����X`U��
`j�([�����������~��EA���qs���F�W���q��=�����T��J����R�$Zi�$j��W��]?m]s~�q���yW����3!��!�@F���e_���S����b�{�v�d`��p���\;P��|���
��]7��&2���M�1�����t~�����G1H������P�t�i�����,��3�awUT��������������"��;�pY�*������K�������m�����Q�H��,���:�	,������z�T��H
8<x5���9�&��J���@���
�}4�@�|�n_f�=��rm��;� we��1[MM"���������s����MW�`K��{����/nU��K�������T Y�@�������3\���+�����d�5p#�WD�"�X�}���������PA����3��4�"]��J ����l��[�I�����!�t���M�Qc�)�X�[�w�9��b�sRT)���FH�Q7!t�n����������6O=p	!�p�#c��42�UI!�d*�$�vr��`w/���P�{*G���rVc��5�h-�Q���@�y��W�j����|�@��(�Hy���'@
2����"��Z��w���0r�N;�c�t���,��yE�6�� ��,AwQ����zNr��nS�7o��9(��kJw
�|M���V����o��;�_����3�AJ����P�A=������`Ly�6g{<���`���a*/P�����K=�������j6��}��~���7�C����T*��s�g�X���CK*�~���3�pl�
�%lw��NzEQ��#E�f�H-������h�=�@\�%d�Nlr/U"�W@�zE���V�CW��Q�L&S����h*����8g���v�������h��z�@����6[��A��gY�d%�k�L�SD��
�r�?�]�x��S�zv�����n(��y?R��~1����rhz���Q9�!�wa0�L���U;����Z1U�����O�}�x����2��I���r��
��q%��kQ[k1��c�u������06�"�������y��KO��E���������v�z����K�Q��$���1�iz&k����5e���/;������&�m%�W�8F��(��S����4�����Z5�M,�����	v�"�~a�/�Z�Ad���rX����$7�z�\��#S�6���b�3�/���u�	������;��!�WH���T���8>u���a`v���NGS��6�3��^����������<����U.c����ui�J�!Sfi��d�_�'j{%�
�g(�0�PM���b�^D!+���&���}z������iT)��y2���{4+3�PF]�!v����]�!Y�������G\1Q��.����i�
(]&��q>���+)�+`pd,%8����4��	�PJ0�i-�5D �{)q(R�1�o���>T�"wGcm��=�
�dmBK��H��"���yO���t�q�2:�u�y,Q-��	~f���c���K����@]��q:vW�>�.���R�=6����g�X��bs������A�*���u�S���!��1�Lcjk"��y-�1��%tdX������e@)����?C���c��^������|0�Fq�2���$~�U���=�Ab�+AH�=�Bd�����
{�������#(�t�v��2���?6���||:7��[���	�<��Pth���T�C�
M��)`��t�~s��J9��4$$f/, M��P�t}l1������5h������
K5�!Z�3�����,���U�k���NSd��F�|�Hr�:�����f1��x �N�U|�46���r���~m�
���C.C��m*���z���jQ����[t������kJ�,�����y�)X	��>/���������e��������d@7
�,s��G�?�X_>�>m�$u�����u���
@p��[>C���*�(�1��s����v��2�q�-W�JfH���Z!�[���a��^?>���+���$��q�il�u6K��s����dVb8X�xP�.hR���12�m.����a�q%�H>X~o���
T+G������]h���}�z�nH����6�|eE)_��2�V���5��`c���9����GB����0��3{��q���;i�&Q��P~��I����J��8
�~��q�F�Y� ��k6o�w3q80p���|@��F��:`���q!��Q���"�f�j�����ar�W���.��B^��7Wb�����R��u	�z�?�x�h��w��0���Qj"fjv�KT��&���a�����3Z�����yT�h�!�������c,p�x���,>@r����"j��@0�E��s�W��7�7��i�;>���/��p��;4�X|�t����=���aW�4�8�~�)����o���q��s����c���'y��'������#a�����X-/7l%�_B1��������%i��U6��O���`gt���W+�[��I���[����{7�Y�L��aH��5�y��������o�v��7�w#g		R����{��
�G���������1���<F���IF#��q�GJ#���F����u~���v������RX�Jr��QP�;�0�����*��ko�����lx��_,�Yv�AZ���e��][~�t^�=� i�27�G��,��zY�U�[�R��h���k#�;5o4��E:���3����}������'�a�v�����sR��F�2��H[� �P��������g�?]2�;��u6�I+�9�e�(��o�j_d������ms���g�����Q�)3������h� ��V�����H�	t��V6�ZS�#Un�_�bd}��=�/<�|]F=C��5�JPl4k��z!����� S�4����������	�i��0]���)�����s^�dEe��v���bf�~n�3o�W��Cs
���v?��(���yR�rE����W@5�`��u2�7��B��b����0_-?���i���x������h����i|��i�r�9)@P�������F����b[JO�GZf2Z���S��t1�~���I��
a�~=G5��I����9^����f���6>��f���u��m�������i�N�����c���w����c��{��==AX�����?�q��4�������b��{�r�'/��>����J H���J�����#&��;�s��1`��H]L|�s$w�=�b#s8[1�m�$R���C�h)5 O��{�~YA+b��^AYh�l�a��&)�7��#WWth��_�;&�v��dY�b��&����e�|�
8�W�������|�������.����T��h�B��SK*��Y�'x$���U�N��@a�Y���Z�2Id7��+�j���J���(A��D:���)��������]�:#����8���RV����S,�/���!B	������o���E�*�H��,�Z�B<��������������iw��F}�g��[��y��FS�&K�o~����=2]��S���n��d�y��(���`dT�-�'����r��m�k{��Z7n����&)���n�u����T�{m�-��M��VL�7�����8���d1����c[���N��xa����J��8�hFJmq�#N��Z>W��S�IL��L�/�
�E|�f�����w�i�+�����9�s�5k`��+N��b
u����r.tg������:��h:�~l.�vw�N�F0���g����eq#
2�9_;�Z�z���e��A)�> �|���A�]1�G�k�&�������l�����j��|�.���;��rs�|�0���R�G�0lk�r7v
a�^����������e�"���`�[�=>�E�	���I����?m��z}�m�c!4�]���3�1��%5D�UA�y��i\m����_;�����e�d�u����m�c���� N����%�~}>�HO�A��
V���t��5M��A�!��sQ�)Gc�I���/zv���:�(��9 ���Rz�����9�����������v��4����_|�y1��JL]&��-����swR�g��IAV�RL������e%e��<��t2D�y�(����)]��n���w�z�e����r
gs�\G@���JWqx)����O�zwt��_�H{M�;R����2Q���#�H����c�������R?\�>��(����3,+���u��8�W�t�+5�B���kw�5��nw�����3t�|��f���	�H#�Lv]��������/MVf9zTI�]O�3]]��)�a~,@�
�����7�����tEXM�������L�t��N��IC�r��wtsj���i�|����>J��$�J�Fs&}}c����O��+��qce�v��������r��������/�d-5�!�//mI�P���I��7���nH
&���'���zw��hw]�sj5���(�1��vx�������i1�sRC�{]���9?��K���q|�B4�����gww����2��L"R�u�z��������;�<^�4�Lf����\3
W�A��32��!��-��������yQ�iS��F��i�g"�RQ���r-))h
���E��v����kM+2%.D��e�����J"o��r�#.���tUx:�G�R����MG����t��v*�BZ@8����}��&QP�X�$���������`Jbgy�=�����jE�����Q8�+eo,�:J��#mJ9��/BT�^�F�E������#%�4z'���k�b���w%'S�A��
K�aR�T�>��sb���)������t� �y)X|g�l��-�v3EN������e����3��-�S|����	��$U���U*8#`�_�����6�&����9�X���e������d1ko����yE����uZ��v���aO.*�Q�H����2�������K�$�s������rg�$[q���RbE�� ~��*����DEn	�*;�{����b<���|��0���Di�X`y�V������[�W���v���a���S����.UA�q[��lO@��$S�<QJ%��q0���6�k��0X��w�w�R�����k0����v}�������m����@�3i�����I�r��{:�\�b�wO� i��b���=�+�w�zee)6�7�
Z�
7��9�c
�����NQ�G��RdCC���C��MWs��G�Tp9K�f�us9���'[�����~hw�Co��������/R2[���nA����h����/Mv|?\?}�f�V�LO�BEy��K_QI�\qa�f����m|w�q�>6������a4����}�22�'�sT0�F]���kh4GSt�n>����t73+�F�f=;3�u�����I)�	���!��,��L��4�����G���7�7�q�����#�;�{.G:�Z2��
XY�^w�����>��������������]o��	��pj����B���M���!k��Eeo��m�A�P�22�5	�0CP�$��r�C�Z	�	������0�x���j:o���&�&��������)��@L+y�sF	T
_��%��T�n2H�]��������798Ff�%E�CxzHh��S����*e8�����E�����sd�4g�G��hhAW��w7X���#?\>���G�2jY��\X�O����$���4�s��)eX��~���B���3���*�e
��D~�^KS��r<x����)
��:�D����d^QBv�'�� ?o�k������&��A%48$q��t=�b����A|���|]7m{<y��$�wZM��U
��X�w�U+a��x���I�r��q�����U�\��y)P���=e�)rLa�����)����$�C�
(��U�������<?F��k�
k[)Y�@��S*��F���|#0��X�]p����8tO��<����d �!G����:?X��?��$YYV�h��z��q����;>B��N���D3+���"���IY�Ino��/1����J���`F��)	�a��K��E�*�~J�^�b��>�����p�|�����s��4c�W�����P`��)g�3���rm�M�;oOnmZA-UX���f�
����f��(h3G�*&�������{@E�9��F�n���
�$(P�@��?����|�L�*{�[jX�"
g"��U���EKi8���m���~�������,��"<�������O��+|Ao�|CN@�.���z������=��k��Ka������)��5CZ�P
�{[t�_��;8�[d�����;#���4�B��V�F�ZQZJ���jH�
�1����jTV�c��92�L���8}���Y����J�4�	���XFUYlt��&�80!��t���� n����!��o��`�P�����okHd�����Rv�(�8��Fx4l�������n��N��yS��~����	� fK!��9<�t)����	���/�����NY4Q �������Lx/}\�2�#��I�z�}������H�e���2f����z������8kW�--���6n��d��D�Fc'
�@��D)��?QS	����
4������7��0F�U�N'�h�7a�1�0g������J"�\���A����`B��|�|���\��$rI���,��!�Rf�D�Z���(T�����T1^�1��O�0G��L�.,��`H��o��H��:.���������.^Y�v��)��'�zB�1��G�Nd&�U6xz0QF%mB��r�C�m>����*$���X	U
�������]�������yQ:Y�(1&���]|�*�n�A�+��pz�����.��,�(�'��]*��a�#<�?U��`O�^�	5��l�w�V!�,<$����[�WtHF\-�M.��J��ZfW��r'X�����]�����������n���P)��V��.���~;��z���?���&���Zu)���W����|}:���q���u��m{�Eo?q��M��A��&*%�,������\�h/���n����E�D������}���FO�D�������1�-O��i�A
3^��p�(�a2/@�6��X���DA���H1��M�5�^
2!�~��F���{8�7�����]?�c7p�T���6?�E���pUoL���t	�8�����C��|S�w�������t����&+��?��E���7�>��-����~��_�MW�����&��)�,�;�x�),�A������!�!�K� W2�4�"#�������JA6o�k"� CE���K��I���#LHp(�����v��w��J'�+�i&	2�U�m
bi��;>����2������v�WF"ioB��\@{{tCt���{�Y���
�6�>���!����*m�+���l_.��a8�\q�@.��b.���7�H�FY,�X�K�b��,X���#[~�x����|<]n�i	�6�Q����k0��s�K@�	��hZ[J��Cx3�sV�[�:_���L���YB
�r�g3������&��n!�O����6_��zs:^��4�K}��Q���Tj�k�����\��_��v���&�`#J��Ki��+��$G�NI�i�l0�}:f(D�bm����(GO�q�%/TU�@H�jXR�@i�n��=:�%J�������kir�6�EG*x?n+OX�"d�<3��=UT��=\��bQR��E��j ���<�����{�1��
$��?�9R�:V�X���E6�N'�����4���KZ��t�
�����f�P��?�|���&���t���
����j���8�����3���TO���V���~�>�����o}{���0�V��EYSc#��bo����/"X0hcLA���%ZfLMiC������)�����}9��t7i���(�������!�2,�K����+)B�_��f;<��e8�<���@�JCRu9YI�F�7GV�*�+ +]QyH(�4�~g�K�f�������D	�����T�����_�)k������1�rY���>���8��_Ea�������G��e��N`z[.ne���yd���!�
.�R0tg
���_���ts;��AC�0�&��&����,��y��N"�w�3K�R#5.������
8����|>����Q��_LKLt��,���X������O�s�i����e���a��UV��oWP'}���XCW�@)z;Y#�&�$�'Ab'���)$Y���(��XZ�H�~����f�)ID�I�1
5+&%���$���^$�=y>�%x��Zd��N�H����&�?�C����]�����t������F����2@��k�n~=�W-��z&k�bZ��rU��Z�������]���������a�_����/F���$��G*�Y^��(p���dp��b%�Q�M9l��������o+nIQ'mnB_y�:r�v�$�Bq�����FZ������yw��~<�}7�����|��%�����M���
��3E����`�}?N�������	��x���M*zW�L�i�w7��gg;Q�H>|��)J����@G�6|���,)�����o1I�*?�SBX�L��]g$4:J������mm�����"�F��G���Fiu��*�$�����wn�l�w��������o���"#���$}|�Bp���&<��Q�����a�M�\",�J�:d4�"<����c�����
�w����wx{�,��7�\�����8O���^K5o��qr#�������L���`(E=g���`B��f3kWdZ���o9���/���.�~�O��t=���8�I�G�Ta	��������l�L�N
M/fq�I��M��A]���&��k�	%B"�f���b�Y�(b����P����
��M%�D
�S�PP��q ���ZA���=t;���������%�X��7�5Jf>��������0��_�Y�]���|���??��U���a�w��8�����F��i�H�4���Q/*�f�d+��y���\IR� �`�)�?(������-������������x��r���$K�ek,�*)���s��;A��t-��O����b&R2�2u,�E�j�hBK�C����A����e���n��K�.�9e#������u��=�B�r=�8�]���g8����#�63�XHi�wn G��9�����'!�Cw8?�= �A;|.h��MnBO�^9�z=������
�	K,k�
che!Y���.����l����~������w���X	�H)1[��*�${(�C�]f��kY�����T��3uf_I-��k�V���hN�����<��.l�w�/#�����H�����x���6m���x��Z���v�{�c$�'���R������?B������'�u����,�]S���#5^b�����$V���C�U�n�2[��Q�V6IN0A�_Ct�~��'����4x�b��]�<p�LY�Q���c������30� ��E9�f�PBT|O������w���O����WpI	W%}�4j$C�BtE��W�*:#C@����\^�&�R� ��YwU��������>_�\�,������p#E)����K�J��"�b��^�W��>���R`�O�
��(�P��@����|�g�="��%D�=�m1inKNrUF2X2,��0�����T�|B���L�V�>o���3�b!��;�~��?�����]�|;���R4jii�Q���
{��&x���#�s�'��V#��9�4�1�B��f �1� ��� U��Z47>��Y�m%�0a|,�+���i�s S���r6���Dt�&"t�4-SW�f����Ap�"�Q�*��`!9U���,�a���FV��F�?���T-��^�q����(�O`���|v�������M�����~�&a�o&Z��I��[����_�wDl�r�R#���s���u�K��`����%x��j�@���
(����l7C"�""mX�\����*�W�������}�>]�������������t�5�
P�IW���-9S��$���J�o��G�&B�O����zJ_��R�;N��A{q�?�%]���.��
�&��H|��"���������q��e>QtxW�FqDd�E�tM��1����}��D�����Csd��YQJ��QT8I���Q��y#�����n����������������������
Y��H��|)������*�Rxj&rc.lE�B�_��
=��w�/C�o���������� ���<��U-op��������>z��������"���"p&i@�RxY��������/0��������ie�C��]r�y�M���Z
cN��(S���������(%y�������PI�=��8=���Y5C?�_�����4:������!��<+j�EW_�_/��s��DY���� ���q72`��p��2d���5��L��0��we(s;u�e8�P[SK��u���=�JX(�A�w��HkX���
t�W+�jYk��r�b�U��t�X,�7u)��j5���j1���{��s8����v�����6#w!
��yz���F[�t��Dw��t3T��� ����_�#T��G��D
������r3��
G��4��*(������~�aS�Nc��H�vwR2$��hM���������������=�6�F���	�����k���jjy���/��U���7���j{��0�����fv!'���'���)!���~�V����cw?�O�<^���*�G�;�p���.������)Xq�D����~De*T��K	t�\�����vZ�����	-�}�G���-��d��[K�2��#CkA6g���nx����6X��
/c(��a��U��,$��4+B��a�^�/�b24J>Lw��x���4bE����Aq�����=��Z&"3zzb�QHZ�e:��5�����[8: |�wBxogu��Y}�[-�6���(.������N�������yG5r�E�N3�y������WWC\=��V���P�>J6�.��O�d1C��O��69U�|��o(s���Hd�O;�������K��������y�G���!������h�4��n�fJ�2�h�TkdG�&���<��p8����_�
���2��F�4*���~��!��&�"7�\Y7�����3��4vtI��z����D������A�$��O1G�B� Z��DC~L����o�e��1�v��d����}?~�8�����]��+���k%:������hH4��D�@��k����t��RcM���}a����N��J�����5?���v|�f�S?��������9�!���b�7�s���t��<|�[�?�����M��=����	Q���>�����e���k��7��v
�K6i�n{In�+>��f!��3.��n���4�qp>�[IT%��9/��$
�X�W0��.���C�W�������d���xlu����iZl�}$:6��U���m;t7[�����.[�|0 ��:��b��������2���ZZ��CPp=���&J�+Y��������M�����oWI�8�����T{l�n���T5C�a��8�\�A�X�R�M���d���G`x�]/��]��a _�@f����Ly9��@n�sBCH��ARC^%�)�sVn&e��f��\���;'\U��6o�6�O�wL������k�w��BF@s������<o��X*Jk����)d:��'2d$(=t5�Lh~u�g�|�6���d{�h@z�=�zG|���'_�
��A�&?�3r�����$A�
;���TC����E_{��t��{�Jg/����U�������oa
�H�>t�����"J�L�J��B��
\W����r��-�*�O��t�r�����Q����~��4`��Dl�B�'tV��i�}�c*��p���j�<&=����Ql\#������6&V�@���L�$�NC���Y��Ws�q���t��OV�wA�^�r�����j������6����S,@�tR��Zj��E����>�k�4��*�����@!l��K�%i\�����q����s���ek����`��i�dW�T�L���W&��D)�������p��$��G6�y��F�e��1���A��<��r=�BF|J�V������</G��|�<�<��z�b+�Y��_�O��=���Z��FhH�b��t�wm����<��"�v^��Y>]	"�(����O$�w��/1V~������.�*�q�ZF�cr{�|�p��{?���B����w�|� �#�k}�i2G��>F���q���i��Anl����f���It�$B�/!+�k�����l�c��I?.�l�I
�������4���[�����J���&[6
�R�_MHw]T�o�~Af�#�Jm�V��u�g�_LN�xX���-��ac�����^�����������K���4��l��vt��6�Z��.�:w��	 � �T���r�0�
�I�z�m�P�n�$\g���G����H1J�O��/8�HS
������O�R������iO���9Uit^fa9V�F��J�o8��{����;m�]xo*RTD��d^���TV ky3����2����0�A�6�e1%����*�He�c"���g��U�$y���'��gy�I	4UH:��E�Og
@\�f����l-`�e#���0r��LS�1����a��k6���^����y�-53��={&c�5X\`��Z���$5�����l����`�67K�@����`���mOf���>w�R�Z�������z�-��S�����^����0���"���������l~E)U�
��bX#S��&������E���?w����0.C���cOMD�����C�p5
?N�C�(] ���YnL�Ls���
NC?
�������1�{!���~OI�Da�1�%��R��)h���m��g]�e�WN����$�=�{1Ql-;�����O��?��y��?���R��N���V`%�JTe���|��}�<�R��ADx���n	a�o���S��
���(���3�`*�X�LjYE�U��7�.[��e�m�������<��iz\���?>yL%~;\���������S�	<��tH��{��\E[�oG��o��� �RZ��rA�7�R* &�Z.��I���O���z���}x���#��]
��f6�o_O���3Ie���@��x�����|���_O�w�q���o���%�b�tncA�>*,i6a������Y�r�>}
>2����X���/��/���}���?R�qm<�7fe��|=���oY�i����_��rZRp'�
u'N��������;yj_���Q�8��</��"L���V�M(n�	A0��X�9d�/�o�,��������FHyMP�X,�Ss��a�����ib��e�`�b";�$[�e���e��z��;�l�P1���(��d�
&�@����K�k�Ow������zT�a�o��?�e#q�6�z��b��i��<2���=�� _w������f�]Klt�2n<������~j_D��M.�g�E�F�x��k�~zG���������>���za��f�|?��S� ��WO�G[AEz����WkO�v�#!�*"#m��L��H#D�Q�����������w�����2�K�:�<����=����"v�����V�m�%�x��5LU�.�����[z"���*RTib����T�|�p��jP|5v�w����yx=i��Z7T1�B���<T��4�����sw��e�k�Ly��Kj4\aWH_a��T�wq�0����D���v��dw=�l���97���zR�7�8
�Pd�t-���>�q��{���"?!��N�����(�/8��jRfw���E$��8%>	��F5������Yx��s�^Z {����Y�l����L
���H|��v�K�pc'=[>���N{K�K�s#��H�g�N�jf�������K�}QL�����Bc��_
+�$`������<>iS��*fq�j��f�R$�F�e�R����jd����?��eQ��d�\����BUDfo������5��$��2y����o�_[��wPh�����i3{�/ju��4�}q"@7��j��MS�<�����%�=��a*2���L-��n�e�
�c7\`���f\�`0^l�G���H:UzE�1E�jx����bF-�'o�����Cg
��O���������(��G����m$�_��} 
y�<z4�����d�BTW�[����lY�_�I&�X�b2��Q90�a��3#����AK.��[Jo���L�N�D?
�gT`4��M����H���ST(�4�1��O�s[44�D7��z?����o��,�@�67�d]gPd��e��?NM����(*�����r7g�c)�K3C��!2d��D����%6�R.P����P�����n����b<�����Y(�&��k�W��xd���"c�R;�8�;��Rs�T�����������+H�vk%cE~kF�m:E��$���J��Q6Q�7:��i��-&x����)/��]��a�������+e�I�mQ��#)���)�e�����y���7^
��A%���
<KvS����a��*l����'���w���q�6���y}A����_U�rI��6{�aJ��k*��4D����8�?�4�JW�?�)��j�
&���Av�+�����+�����W�'�w�q�lqi�w�S6
�k�U���*P
�z��?��_����L,xB���%����k�&G��*������^�w� ����c�\./g��y��c1p����*���5�be+��S~k�3����~�����..��&�W6:+��;����M�����N�`����p������b����d��|X�t"��a�["�H:=�����l �W8�d~a��I�l_d�o��Cst�aPp���:�"O>��G���4�>@`]z�k�EY�-e�����-A�ve��0�p��<�� �ve�$�����b�&L<=���.���X��s��S�����WaUA�Mq��]n��r���~���Q�Qx�o��S;I�\�,'���}y������R&�dg���3��QI��\�Q�g�����F��A.�E���f&q���b�9z���~�G�i}|n`����`��m�@��[!3���~�Yr���1(���(~�E����������cl8T�����Z�C*���*��������#5�.�d0��I"��)�C1*�_B�.����]�k��+�0R~�1��Y�b����0��O�4�`7U��22�$IH������)*�y��s}�8�\�)�S����L^
P�m}����3���@L�$�=�"��
���K�����K��MJ��o����5��n�����xU����w�aZ�������'C�N�B&�sa���4�p��T2_P�������M�Y�H-�����A3#��~�/��um��s����X5����.����J9��A����I�L�7|�yy����F����\��;j;�+��ysT�xM��Gq�H�����xe��������������G�CA�������W�;�RS"�Nv����b���3��P��r�W+#J�)O�o��1���u�[X�V� N:DxC��-��wA��M���|���4�����~���i��I�O�����c��s�[4'W�`�-�rM����U,�	�H��ffb.�C'f~�1}��r
��A��c}��0�v��������+<dc&��c��9(�'�����c{=,{���M�l�,���aT����8q�#$	Y�|�K�1j{����8=K�l��NJ��j���QDc2$��b���e�{��w����.T�.o���z��K��)�,osj��;����l+;G'�B�S�VS�O��'b�������iq�fo�k�L��A#rx_���Z(9Xh��-Q
*��*�����nd<�$U�gh�zoQ8��+�?��N����c�c�����'\{L"2��/��$�~���6�mp=�����3el�X@��TdG3�[�>m�o�E�<	{i��k>�;�*n9�d��Hs-���I�}�|�=�u:K���0���O8���r]}�TC��8��"�<����8�AS��CUN�[4�g��U��;@�_
��iw��_������6_���eO��hq2����Y1AuN����D��lo�0��.v�;�d0#�bh�!}$g��s�
��_�sQ������d�
���om��_�A1G�kZ���_:�[��:��_������S1�xq����S}�`~���!`�$���m�s=C��Z�8�LG�&��'Nbr���"��[�HX��W���������	.�j��.�
�T�1��;I��&�����$z�e������ g�#���V-������@��7�g�l���)���~p�#5m���X)S��"�m�^��5m}�t�`���G������8�R��`���8��MH���$�"��;���-�Q�����}��_1�n��son����{	`����^GB���)�4�u��b��P26��I�6����}�:L����_��:�R�5:��Q�w��z���6���v$%b���T�S�U�8�(E���1f(
���rY��=��m�T�F�	�tt�p��"6�J���]m�S������I���Zq�l�fW�I�D5���b?���w��o���~}��c��aD�(��p�����~\������Z��,p�����h���|�:��y�\`�b��DS��L<7�x������B\�w{��5;��Q�S�c�K���hc����`�u����(���:�tI��YC8]iF�+�
\)����|	����{W�(� Fu��*�d�)���l��|��3��V�R���[��<�����
o��C����| (�R+��:Y1Y��N��2
��"!�����_D��Y�x?���K��e�����M�#z����P�����|����i����Ble�Uwob���\�����4U&J�`	�Y�r�W��K>M���)&�u��������z�{���=z�����HJrK��5��Z@��-����\�W�����'����p]�DV���4��HQ(���0N�&L��/0��.����x��<4��'T;��<��S��mw�G����r���G�C����IM�q�&��fcR���T�q�
���\���]?��Y�f�<^�:jBZ�FD,N;�R����F�|��]�	b)��`����V�%�P~�����+?�|��J����������(�t�E��=����������r����q�"����r�A�'�)BQ��
�7��7�CY#���2�A���W��
TV:�Q6��Z�8r�����6F'V��t"�n��
��U�
�	�t�-W�.�l�i���2~E��RyeL>{�3)b%�T������J����V�1A]���p���Q5��H���h�15��EA}���?�^���n�|����G0����cJ�����\yE����,'�������:�������������	��N4%���v�j�L(�������H-u�QR,1Pr��zM|3��#�y����
�9d"h���$�nR�
�+���h�5��``�.�RUf*eK���@��.�����$�����J����K��>�<�����t�K����,�$�e������G�b�F��r��os9��������m�2S���up�F .�2����-��`��D�e�>M����A�^����q}n����M�
���}��^=oAnd��X��%��p�I#{ ��b��np9�+?5=�i���q%&����p��ak���t)j\h�t�����:�5]��-�MRf��z$����,��������_~�g���2	��/5����e`�G���4��|�������������?.���a(��;
t3�5���+M��S0M)����&W�iAI��HP�n���L�	5HPb����%���7�'����W�<YQ��n.�
�y�B��NY�d�:��������}��]���Ss��g����Y� FL�V'�����8�������N�J,�g�
���M�BCOO�zA���D;��S�����Sp��S�)���]"-=c��{b�3
g{����q�;6a��4����;>��,h������K\���(: ��8og1��������['�m�`cx$�,&�`���d�G�G�d]�7ze�#�����$Ki��s]YF�-���|1����p��~��$�I����o���=��01qq���1 �l�w�,3U�2x�=��3���8S
wFN$�"""^��)���Ceu�����6_���<w�m?�{(���r��'��R^W�X�S���0�`c��(�*�:(�
w���@��8(�Z�K����%K��f[y^1����� ���5���.��������N��3��4	���V��<5���h9��GO�K�����e�6if�������n�rJ++M�����6G�a:���1$��w)����tER�8��k"�l��;��GSJ7����B�=���C�p ��_+�2N�Tw���|H t�����(>������"�X����s�8]�D�c�t�
�:��	���|�u���!�H�B�9Fl8�b�T)�����n��h)�=�S��;L���"�@�k��Z!l�_
8\�/��p}��R�]�s%m������C������?�4��%x�y~���@�YLSR*��S�[�A�~�D%��Og����������$3��Mv����������h��0���v+�"���zZ���M{^��H������cmQo@��B��(�<��������r����j�!�����Ba�D�Cd~�"�d������oX�[b5R�R��o�\P@9�8�>���|���|A���I�w&���i�?&�J�l��<x�s\�r�~�xK\��R��T���O���8=�H�|2���'��|���I����_C������J���C��c��'U��X�GwcA��[���c��}�����C&��EOe����h�m���iL��!r��Gu���������o(D������a�����Dz��_.4`P�%@�h	oJA�o,���Qg��GG�}�rM�P}H��Q�vk�]�A���B�����	��[[�����YSb�=�H�L��Y�b��B>,4>�O� y�N:����K^A�C�~x�5
0o����&.+U�9O�<��~�Z�W4�0�b�������i7A�i����1/�hC����sL�z�q5d�AZ���XkQ1�W���J��9e�V��
��p+�K)J?V�������cV�-*���t��_�V�������.��v�'	^���.IG��(���F?���m�F�����T��\�.�d"{Fk"�C9oz	����k�f�����v�u�2WY5)F2���m�����1�LC�NZ��nV��~�x�cy%��vD;������)1������k7��v��;��<�]���Xz3<[�|e��x1r����,x)��|nm)E���?��fT���x:����������>��8�~���%A����(�J~����h��mF�N�P��l���g=��f9�1Q�=*et���P���RoO5.��������P�/��
�N�����H�&<1D����h9����`�z�����3^�`S�tx�����F�W%��qr��{���0<U�qO[��yu1�yy
H���m.(��a
���-.P��c�"5N���~���=���K��^�t�k�4����2�aQ��Iu�W��P�y��k�������0-u*�^��/�$k�HA��0�_r.�/	}��^������b����HG�������M���ri����C�vp�X�%�C�zt��B�s�����T{��QGT�<n�N�����} jA�(g*-��m���x��#���tu�m���O:0���k8O��E�3��p�,���-�3'V�����F�����g���wx
�fJ���I�p>.�
�z$�����s�
8�7��s��RgT���[8���t�n�O��u�YC���m��+��M3g�9��&8�:y1��C����o7�"%10l�������a�I��9F'�b�p_v�m}�������=Ud�L�H�|A���9"����:�>y1�^�p��|�D�%;y>RfC�h��*�B$�(_�H�]���v��oL�s���Xe��	Z���������fA�`�������8TW4$����u|��������>����x���x1f�L6Es���SSp�x�����������M�#u{���=Ez$'y�S2HZP_N����H?.��b�	7�8�Y����gmj6��[�����l�0H����i�a��\�IO���K
���"��5������;A	�s!�m�G���</�������MX�S��U��iCU�����
���t���6����R�WO�
*���y-��|")�� 4��p�T���uk����1iP�?�v�U��u_�����o4Y��K�q97��4�`o������b�����i�u�j����v#�����fCx"���������&��z)�C������qX����GsNSm0���$�����Ag]>�Y�����k�S�y,P��7�Z.Xc���S�i-FH�x���������y�����-���5�K��
�U8MI"wig��W���r�o����~
�$zE�%�b���V�ad���a����
�V��m����^ �z����@\�H����.F������$��H"!_�_5��#I�R��-�B�i���1|������,�6�)
��%�C�m�����Q�[}p���0S\X#LR���{Fl�4}��K���EWvA�������9���>I��$ZW`��R���#^�H��y����Sw-MnI�����Q����3�q�x�����D�[�
����oe�&�%��p�v�e�n��2���1���sZ!	�`��M;b�X���2�u��V�F�I����J��!)A"/�]���|���w��x{�H�#�&#�?%Y6�Q��D���o���|�+H�xK=I��jo^(��
��lD����}�D/A����5�m�9�z����bcP��i.�������> �.���>��
�b�)����/����Gytc��5���=���U��B��?g���>�H�/�5G��������������LGg��$�����c=�7����ck����{?�P���J���Nk-U���3��RP�W���V)�����>{<y�S2*����*H���T���"$���<�=w��������:�!#x��Z$�	1)7B�"�0��|8��r�x+�V�G�is���
��Z_�I�a9B���]_���P��a�a[6��)�|n��b�^8�#]��4Gl��CA��`�.)��}kT�m����M�	�bEp��4#8<_�.�<t��jk�;�$w�����P`��IVD�������K@U���~�Ou������e�#��+B���4�c�Z���e�K�n�#n*��s'�����L�.���P�v���z5�������2(@*��'XF����|�6�5�	0�u�I{M��Z!��\��O��O�������Al��R������)�k�;�X�r���F�\������i��u��T���M�����+Ny����r�\�q
�������z������?����M`����9=��G�������x�z�>|���k�����l|e����=u�c_������i]�F��z�U*-�F�\N�7<:Ea�l�.��c���5�X,���;����9S��}$ze�6�p��msve_����z"d�~e�P��(��l2��M�<����HB�1��Q������D
����`gtk�}��U��>��F��k"6��!�C�-�O�<�2@��$���s��w���!��	D��t�u���)s���[����ua�Tx�����O�w�(��^d��x�^��i3�&ZY�{��g[� '�7t�{��D�aO������i�h)Y�V�����W���e�������{D�Q>���/h��`)�����dryA�z{i��s�?o����?���L�����f�}O�tE��sy?��GwF����������u�����\���yG3fG��6s�:�F�����4�D)��xIg4a����g���a�G�����
����b�
D�mh���9�����lC�\�@�r��������[�k�$7�D}O������-�h�#Ps�t���s���/Il����[xZ�GpyNKQ
9E����
��p��U��Xn�x�N������utZ*���B�i$�`}��jwM��!��x��Z�|���f�$�Ba�O�R��1����v�����I�A�ca6���S��������=$��"��q��B�4r=���[�C��,�`�@p�+���u��h �AZ�����hX1df����0�gfq�1	A����h_�0�Z#�*r�[�q���p��|��U�O�
�����=�*M^����4����Q���8��/� �>*�?�N����"�������-��m;��� �0�������#,7�L�:��v*~�H��(����K��nr����+�h�?�	��f�.�D��<NTh�����?����oUW�>��]�tL�f2\��������R �nfJ����g����� ����}�V���5"m3��!-���H
pOBZ�3wF�g��?��������R���aR����RG�B�����,U:��8�p��<j
�d�����?��!�U�+h��F'���<.���eF�bRD�p����b�Gq���8��e6��~��gw����A���b��w��a0}H�2�5�����t�S[w?V{Xb��B]Wzk��\��t��.1�(HA�������]q\�������Bq����7o�Jo����#���Hb�q6�R����!�d�Q]��Y4.U��v]���
���UQ�H�%eP�G_����QW�H��A��$v�g��[�$�$2�i�� ��i�2�,j���8�/	�#��\E2,������
���~�$�a������L���r�|����k��`ZA76j����c����5=�#P�"L�#��yH#�>�i
����2mF	���3y�r��i�2�5�k+H_���H�hPsb8)1?[�s����'�|���#�h^Y��������k����B}Y���k�^�z�l�-���vC����6�p����C�I�e��-Kp:o/�A�����%��T��LJ��,�����<Q�L�0<%��n�Em�Wc���9j��z�l�����P���'-�7�1���7iuT���P�Rs�l�\���������G�����
q�O�F� ���)K��h�T���7:��o�7Ta�n"�J���>'��E����;�]���
`r�hW.�]wq��������
/0��FK��Sp��}Qi]���@���	���^N8�>��J&�n������+����gsq�*�c}������j^~����r�n�9����k����#���L����[����
������0�-
��w��9�q2Y.d +�k�E����GWE�� �k���M;f5r&r
��&�}�y����{��o�j����6D?I��I0���5��jR��/A������^�bT�}���q��GI);6&��$���G���+;�8J����}e�����qc%v�t���`{<���Qn�_���V�-�=�L^ 8�?�a���<��J�]������;P�ju�52���Z"�	��?K.�:z�_���nX	����;"G=|�&a
�����98��dtS�j�H�n����9{���v��ea=_��Wte�p�������l���7v��rV�M"��{�g�
n��P\�I(Ai?1������&��
�s/��F����������[{B�!�G�z/i�sh��fU�
��G���*���$.gY���Wj���&���$���ap�%7�cTs�fq�%s���T��
!U����~�������'YB���Y�Q�@�6f��^!������	��+���;�L�2�����|�S��������,�Vw�����tJ������R���G6�����Ame������eW_�m}tB����������z�m[H���K���q�L��`
���o����YW����/�^1����)��f����3�-���?���UX��0�`*�1���P��N����)��S[{����+�64�D_������B�G!&"��w��&=�P��CcF�����vB�f�@no��l����:��g��`W�
rC\�`�m#c�PI�������;���2����c��/P�)���n:�f@Hy�X��E.��{"M'��X�Kg	����lNn�]����� D��`��	Sz��v�\
A���%*^����`����Xb��4��}v������/}�����m>�i�#�B��>*�
7�P�S�����|�N;�p~�<�a>��F,�b
�"h[��=t���B����sa����"�(�Xo�$hwq��-GEt0�����il�S�BE����w�`����Pw,���!W@nF����&)+
6�%���/z�d>�[x<A������%�"��d���\j��d@������#U:��@����k/��g�/UH�����S�qc��,n�������8�-ACru��h
�����8��i�^�jEsu�	a�X��V"h�2�7�������uw9����i&��6bp�|L���D�f�6��2}��.�
���(!"���\���
����q���\T>?��>Vg��_=�p����"����lu���,w�;f�/��C
-��J�f�|��,�F�u�����RM8���k�������;'9O
^X�"MZH����SA��n���A�$T����&��kK��5����>�
���}[�<����+!�	�j8w����Pg�����bc��C{)=��S��?
�sq�"��b�S�����"���'���y�V�1�"�W7���0d��,�4eo���M>{�������Ve����<�QU?�0�@1\95�r����+��������*�P�K����Z[	�)��"������y����
�Zjm���5K�	�U8�
�{ w����2���@v�"��"b)��k�)e-d8�f�.�����k��s��2������ ~���h��
�����]�H�|��a�PiI2��l�e������&�7��iM2��_�k	i�o���(s�-"��t�n�f�����~����-MO�D���$��[t���7(P�]�1;$M�C[�V��K�>�������9�0i��e������bE����Q;M��������_Gg�1�������&����m�p�U ��Iv���j�,��=m����j� lE�>��(�&�D��GC��1i��=�I�&�[�R�����i�����i\e��O�L�>������H0sL���;�,Q@���Zwu�?G�[Qv��OI+39�"���	Z!!/�����6�#��Y!W�!`�U����r6%d��s/'p�kN��Y�������Di�K�`�n84X�
��t���C�t���m}�x��i��5�#�A��^�g���3rg~h������m�����S�>��r��>D��3j($)���`)�����F:a�� ��a��|E��
��)p��E�`���j[3���j�&�>4�'��<x����c�9���{x>��afuh5s���q������t�	�UC~�������0Q���B[�Ex w���_�}�J��a�Z�����j�9�k{�Q���,���:�W���iw��w�U#�m�ab�oj����#�&�P�ci�6��O���5J��v?����n���|�����J������r����ME�2�����<�oqR�� 6���������1�����y=�E|1b��o��?��%P�p�"7��r������+���B��������,����Pz=K��w�-��Y.u�|9��U{j�I��T:C����5��9�Y����[��AD	iJ�c��d#{s��6��$u&-�����`8!��&50)@��fmN
����&d�!�1F�������M)4=J
��m5�X���\M��������g(�9���VK4��K�p5�j3q���;�!�]}�����f���Ec��X�����x
�d�Ck���:����*(�� sw���c�^��l���g�"?�v��������V<mp��	h0]�Lb+�����������)��,2v	Q�M�(C	��t�|��y����}�=������y�%x���i��;����P���y�t���x�������o������:%4����&	k-Fx�s)#tb�e`or������;N��;�X�a<\����!K����������|�	���������������7�F)�I~������A����i�����)�c�����>�,�\�q19���k4w�)�;��U.'t����@�3�+]A�S�d�:�p��Q|#����\�v��lv���������#��0b"b*p�J�
G������7��_�xuUw9x��P�n>JV$�����������r��f������_�-(.D��D.�\hH�{5`��pxR�?�|��)s�/�����M		Tr5��hD }w��
,�r9O�O���c��]��e@���j(}a���<��l!��f7\����{�m�nT+�������
�c�	]�A��60����gs������T�����B�>��X-*��$J�]�`��n5/��O��_�����T��.���B���w�D��~�+� ~�b�����Y0:��-�����%pA�f�u.z5����:�Y���"�
�@p^�k�`�`��F���cl�
�x��S�>�M��n�A���g�� ��Z�p_�H����'���������tR��/@Rx%����l*P�������a����/���9o�x��6(�����2��$��GJ�Fw����/�-�>V����/��- ��N�a���`y�UH}�#�16�����~����*n��8]�~���=�pUy����T��zu���$�'����O��r��J>������n�N��&����^��:@#�w�,��}��g���4�QBC���3.]m�����1F���w�F,o!I��j��y�^�Q����V�eP����
+�^�1IF\x}�32
�
m���C����4�q$��{J�h��a�~�X���$��|S��H���!�������y	8V6�A
�p{(ua,k��{��9��#-��Q}��V���:f`�#������������DYp����(9���S�s
������X�`���p���.�B�s��aD���f�r%��O�\0�W=;�5$U�L:8��U��������&��
��o6�����*M������	l@��F"��k}y2���j����tH��?���Ir�H�?��Q��x����D���Z��Fu��R]K���_�H��.VCRr(
�����d����ny�����0����h� �T��[.J!�6]�k����8j����SH�
�����2%���U��W�f�	���	�k�m	�u���)&���
l��{����Bb@tDD'����E'G3����P$3M4���#'��f����*&U�V���t�����c��H�q5��q��b��( ���Y���J(�_��$r%!��o�1`��Y����/r��r�����V<������.�4��$��d,lx
h'�z���_��#���s���+�����wq���qX����	�����:�WG	Z�^LZ�#��� �� � �^�`��������9�p��)�9?6/M�7���I!t3IA�$�-�k1@�k�d��41�txd�Cs�7{�,��#�����.���@��	���8���D��������s�E�[d����G�bO/���X�p8���
hH�zh�����_�5���\��uT�+
�D6A[{m5n��D��g�?����x������t����l�W��
i	1W�����X����O����,����'�Y��_����3@����d�Iq
B�T�r��X�����Z�h��U�����xl�a��XB�Y�"E
���u���
����%r�����?_�(����N�&�f��
��"�,��Y�}Y������j��:mO`��M����p��d��#�����:�����������i������z�7���<��k���s�m�2{�
0��E��_HM����$�On3�mjR����k��
���~��?~���}{�|����>7�����]N�a|iH���`���G�?��}So����s#���S�;������Z	$�AR���������q����E���?{@S!��}��L�������:������0�+���5h��}�@�A���+Z���j��N�~a�5 P����k|L�b�N���w�=l|��G&h�������D�����b"��K\��B��7�U\�����3X��L�}[��P�a���������������7���y9��@�#^7��kdo�� r�lR�J������
���7�����bs��"��Hs��28WD)u�+��l�{h���,~8����8�.��d���K��H�x�r"G��z�������P�q*�!�����L���n������
�f��q�q
[��3�%fr]*��)x%q�b����8���������x@�wJ��g��
�|�S4���;�U����k��z����b��,f����b*�M�S�L��o:�|a�?}�u��9��u'r�Z�T�1'iH)64��\�t0%2��'���$���vT�F��V�����Q>������F��"V����^�j����� &��V������>J���b�����`�Y?���9��#�y��\D5y%�v�[�b�{��i��W&��������6���"��k����	�oY4��t��������##��v��z�
��*`n���L��y<��w����#]�#{�.��VF!Qb��(���OC���P�J�^Sw��Q�)���
i|�m�p����D1f�7�����O�����w�d�����E_�9�0����)�RH�/���k�h+'TF�b4�I�H�4���=�������Ur�M�8�6��hs���6H�2n�_1{�4?|��i}N�a��Xe����F�1v�v.e�1�b;Of������x��n<!h�Fz���@#��!B�-��
������a�I�4���F>�SX��p*"�%^� t�v]�_N�fL�a,��s�#�)	g9^�J���8u�P1A�Wg�ah��o]��;+/�mK�
��)X�7��
g(�����������>��"�j�n��"��,<�~^��x!��$'���;�~�4��t�
����>ou��i�y���
Dx�����{���=�`�y���v}aI��T�M����.�4A��\\L����1�p���7e��:O��jJ��]��A:�
p�C��y)��>@=��N���k50{�
)$��MK�@:�9�k&V	�fsp�B�D��VQ�u��L���6�`����������fg��U�.g(� ��#3ii�$��B#���(H)RL�jl���Rm�k���e��'
io�J�r����m����oX}9��>�u�\'&���u~4���52FP(����==��crG�M}����L���*���a?t����oTzi
66�U�����o�2��N����m�r��k\��r��5�����@��@��8d�����FJ�b`iRPl��X)NJ�/�#?a����g�G��Jy$Nl9����cYH.�M)�f���m����p���k�����\�dJ�$Q�H
Adt(�|��?�r�6wH�����I,����n���8��[�&�b xX\>�)�k�����!����7�Bf��69��\0���*.�/d�\@/_C^[H�����C�Q���7w����g�B�sEe���+	���xpR�30�����s��&1���rq���)`�+#J���S?��{Y������������p��"���
2)��H�������w���nS���s"�j����;�+G��f������(j�k4��4'���KVF�z,z�
��yU�/�sS,o�-�Ur������%�������Oe�����YS-&�a�z���\�����*M���D��v�x$���q��]~}���<�������,�2[P��L�S�S0"�4F�Cs���i��f^��Y���c�	3C����S���� ��q���2j���
����5�4+]���{�4���a�.�h"��G��j��"<���{o�	�={�s�3MK:���6%-�r_(��p�V�b���]��x�_v;X�	�>Ji��H*U�X`i���JQV���&�"���I	��!����\W[��7����2������l+
Q���(�M	����7w.2��D�^p���c��C8�&������/A��O������q�NWr�P�bl�Ieu�>`�[;�;�zK7U�R���3�~GGw����c��u-�z�D��l�'�qZw�)f�v����6����"�����{�2i�M)���r�1�J�Y�r��������G0�]���t�6��x������9�c�R�|�����;�<�5C��l[�z���wT4���)(��nD�{$]�6�r/�)F	��?�v#A������E���}�l�K$B!�t��GIJ�(?�&�U��5��_��ym`AAm�M�{��E�*�`p$��"����S�m�H|=|;7���yX�'���c_�pb�G.#���$D����;�+����,�TS����k��6�5q����;�0��2���K4N+SI\@���6��w����i/)*D���vx�nyq�h/�����j�_�H��g�LWL�dGQ�oO�./!A]�����mK9����w#cqm%C�7|B������	��O�x-�������t��;��in�\~����-H���jA�j��L�~�������H_t������[S3�����7_)�5��{w��
�����$
��7m��F������j�+)�����q��)���u���)�|��.�Ll5]�)-�`�\���9�+�����N������~��Uug����u_���r���l�����	KC��;KAY�e��+W�p�5�~�*�pt��U�*pM��y�l�vXX3����c�������#f��7s6[tG�s���4/�<��_F����������7���to�^d$��8��TA��wX�4*M��q�"K�.�y����L� 
��_�3u�D	\�B��!
�}�!H��\���i��|�f
���\����]S�J8C��������;WM�S��yX�	����m|_[�J�J�X2�����|�b�J�o��K�������<�
�c�����O���pw���H]w�g�=*�4��vM�Gs.�X=?y)[����f��_���*�2!&d.;�A�.����Q1t(G�?z�� w�y8��i�C(:�����o`���.4��ET�,m������>Q�5��6,_���~��m��]w���`R~7a*$���"m���)e)@R��������\�y��c�#R��S�|�g�>Q#�M���\�X��ox	:#R��x�@��@��a\N����l6�N����)#u}�(����8�P�]��
(����c�9��k�3�_;P.d�4��)9	�op�D9��NJ ����p������s
(xH�A�q�V|E�h�`=L��gW����s�7J����y���1�7#/�m��`��!/��2}t����7����iX{�|��1������/EC��3�}���@y)�������f���x���aQ�nq������`K^L�~~��u��������{5��&�k���#l���6-�P����������Ism������O��"��^��1��\�:�y@��uic����w�L������U�%U��6K����������<�Ys���."d�U h�	0����Y��]����O��<y�9���IsU���H}gj�D)���|n�[�y�ACY��%�hN��~%��� �>g�.m>�
����mKQ�&�O����o�n$�������?
�m�����qS��A��OA}3\���u������(�RR"!>���Dx�g��^��������J������Fo:U�!]n,��H�8��-'���?o�TpP�^�kO��:��"y`p�#������L������3��.
��Rg���H��=�u����e"�}�����"��<��W���7�3���S���/���e>'��HI��4�@H��<g<�����H{&H��f(;��J/���GR�����Y�����QF��=!�$���b�S���H+����y�������./}MX����X)$WN�a_#Y�~��c�!�2jSQ�j���}��sg��C��s�����?&��L��xe=��^���i�c��X�{���2v�j�97+bKA)��yX��z��])2R}!�mX2:�j^�J���(� ��_�\:f��d3�-	����,u��T� ��|��):u?6g��f�O�}r���iJ���`��d8�v:d
�������w�@Q;��{a���a5AZ��)e��t�.��N�=U�\�4��;o�FD�b��p�A���w�:]k�`BGH��t7[)�����D`=>�Vb��t����������>�z���S$�_����E_8Sy�0��3����?������K�i����tWf�W�����J��)J�_�ID�K����L"�����*=/�A���b�a2���.	T�IG�t��{LY)����t�����?>H[����&=Q���[ ��&0ih0�^����0����j�GV4p4�����#�u���_X�a~���O�"��r�8M#���e�q&A:�-��T�=���AG�+#����D�O�VV##����
�I����L��}�/�����\L��� ��Z�R��r�~8n��O>|�U�;�Q4�!L���rq�v��8�	�<�����iA*u=����E� �^�k�t1G�S��k���#L:G��Pp������G�ZTPW�E1	�
*��1���R���&���j��p\\7SvE�=1����Z���p�7�L���_�U�s��So�
e����j�o�z<a������9�������]�k���4>�!3�nF�����
]'��jET)�s��e�+I������r�'"���Xt��x��4�Lz������=}9���uA�RiCF�;:�?�STR#qFlX�����9\��u?�`4��A�\��8��'"w����\a�.7
���8�]�L��{��'��q}�#�+��TMz������n������T�{xJ�\�O��]�E�Hly������b�0���|����?�A��Q��%f6�'p��\��E�1��7����@o]!$-r��8���cp�
���D��X�����z�l��bwK�h���0���+j+��f�B��-�1�����]3x����x|��R���abc�
��������G���kZ�q����w�2<g����������~��G��g��?���E�u�$���;$��������W���Ow��������k|0�@��D�$������&���w�D[��MFL�{�L_Z��l��Y�b�!w�m�~9a��	����NJ����r�VwRYq3C�hln����Y�e�	V)��<�;*�0}
����_��S�����=�r|����#�O�K$��2��������N��������&�[#s�W~<�^��Yd��~�.e6z����Y~ �K���6Q��a
��y��.��������k�ou{����s'`w`�������l�X`�q��(#���*�}����,}���g�m��o����������^��,�W��B�}yLg������4���%�{9�(�����O��W��,L�Q=	`��\�9���
����y����k�#������mK!�H���b�(�6�6�bD�=��qM��lL3h-=�Vq(_'}y8�7�g� r6�T��&%��',���K���%f8�X�Kzce,W2��cA���-��� "��d�W~�&
K��3_q5���2�.
!.�uH	)Q�2��0���[C��/����xx��mR���AN��n]uAq)D�d#8{�}�-9��	�(����3%�g����$N�����~���fwM�����j���b��XAl@�������E�����DS���V/$�nc"�4���\���J�.l��8���y��o�M��@c#>KCbW�j��	P"�33P�������Ys|��`�2}�_���ZW1��G�,�W�.V#��P1�S���Sw��z�m��~��r�6jjM.��%�����^�G��\%��B�\6�
@����]�EQ���p����e���
i������%�J]]���7�%+yfC�2�@����=��!I_�Ni������:y��/��)y�}A+�E����+�s��`kWi�l4��v/4�>��w?����4N�Q�-��������q$7�K
�PN����}�OuW�[_�`'��`�����`���)7��rYz1��j����W���n���Q�n��y��&��M��z ���?U�P���0�B^_�Q�5�:����$Hg��B����0�����FEL���p�M����2���
��>���k������C�����-�����#�5@	Af_&������dR2+8Y�uLAoN�&�|�i7>�'}�T�I���#�)�i�~�:�����D�D�� ����t���YsNc[���q���\�P�{!\��r�.�_��]�(��)U���(}Wq&
�1���s���T��A������;~�o��z6:Y�[wcZ��]
�"��K�F�\z�����%+&cU�\���j/*����A[���>������a�w���r8� b�p�����h.����\������I��b��|#�	����KW*�^V�c&����e)'�TZ�V_��^�G���U�i
�d.h����#�K��a��VhQ�J@�"pM�$+sY���>���i/.�M���>�?�^��cIL4�n?���*.P��
7�H``��VJ�n�4�t�|�p���7T���:��zD8K�����%	&�k�8�5����t���f�����tn}m+]��#�!����%�,��I����t�^�An����$��_��		4D�]����<p�3����w�����"R"�MJ��^$t�1��b��6������������h�R����sZA��>V����H����cA7"�m���6Lo;��*��������N��N>�I�qD��v����PT�NX�Wh�e�
�����	o��g�I����	�����u������mf�HD,M��Tu�������
*�����x&���cw�f�?T����t�	�q�x1��@=���3(]�����$��� "`fmh(��9�e�����>���q�~i��3���;"�D�"��gNsK�6�a��'�!	"�k����<�fTl��Vy����dC�Q%���4��tOt��&!y���CK9�g�A������5��������
d������4�������tKz���*=Q����?w'�H�h�k����+�C�K*�s�~��`��1�s�A�F�����I�����BoM��[R�Rt\q��a$yI0��/�.����K��k�.��	0��I�5��f0��g����;��omu�_��y�I;����x�g����*p�D���`.��[^��3�KD$@���-zE�/�p��\J��yl����C�Xp����	k$��Xr@�Z���G����H��Xl��*�@dp�/6)�m?���8Q���>��+BC�P,Rh)MW��A�����sw<���X-��R����vdA���L�x�{�j�<�t���Hwhc�����y���s8��{@L���`0p5(N�&\��M�����������7���z��D�:�=��B��S�M.����h� �.3J��T)�qa���m����G�R����i��l��fZa�%A�w)e0�s �@)�\�Q���s��R,I�_zl��<�oqLbO���3p/����U����b�����7��o�eL��^zW��-��]N�-S��O���=Q�������/I#�����P�o�/���]o�K���R�{�`��.�^a�)$
�>3�����r�,F�?��A�:�v{�B7V�!�}�;�9��$,s�C��T\��<.���V�l��y�/?U���c��?�<�,XBu$kxY>(]wK4����M�H�������'�sT>��K;yN]���Xj32`�MK4�1���;|������mN��>����n��8~�C�T�iw{������������M��[%�/]�������� ���|��
u3��EM��mV$-�`�c��$��\� �y��/�������o�L�3��Zl�RZ�Y!x�q�vWs�cr�~�����A��KdW���p�6�EF�h���:m���	l��l����h��E���O�����?���n��b��e�m}�=��8��2���\/��1��49�a�r�O��QN8����&����f���U���Ce�>�f��\{�~G���%��K�*��1%���u��N�U��C+6�l���������q�"�n'�G�.f����S�VW�5?�JV��49�p_2�r�~��������Ae<B�JQ�/Gh@[�d��z,9��e�#
f�/��l�!%,#�q��%:��x�F�G���0������ZJ-V:�
L��78sim,�f�������-��)�d������j6�)��C~s���M^NP��g��%r:��`����Kz(6�9AO4��;�����)�$�C)������O!W]���>���@�SX+�>Ls\�s{��:Z�jO���{����1�Y���7f��=�������o�
x����� �#R�J�$z|IA��}y:C4O7��
�����T"��Vo��ec�����i���
)��_�ywG�_WE������>\X}������k��I�:t���I���
�2
rd.\��F{��������9�����gG<,��X"%�������&~L}y�Q�\����m���+�K�����j����z��>�Ao��%L�/8�D6��f��-�E���e��!P�e6&r�j����vH��di�%vjyV�Y���#�E��\������z_����+��%�6��	w5s��n�_v�����������	1����i/�5��$#���\���z_��p�=�@�t��(�u��s���z>E���$w�X���umt.=P�~<�����C�T5y���%Gg��>]k�8�N��d�C�p74��.����� ��M�,�!�%l�9N��%���?�7���=
	���]�������TSP`���"o�F ���0���N�g�t��i�O��������������x�Q����R8�3��-�v�3��;j�^������?N������W�?{�H�!���f�D�N�(8vI$����3�Qz�~����p���+~Y�o:��X�`;�������1j�
w"BZ��#I�c���������	$b��lH����j�1�P6�V�'��U�:���!��Z�<��e�q��N����F�vq��&�]�hC��I�B,������cF�W�6�M���������F}��#h��@d��[����{���P��O}���T�������Kw%C��ZK�rK���il��~x��.$���cVc����������?�E9��lX6��V��_��+88��p3�m������U����	�u��;���������kW���M�o�'�����U:��/z�CA��&��BZ{ee<:,���[A�St�����i:� ,�4����������|��x���B�=�XLtnm�����e�	�A*���:��,O�e��Z/�pk�~�yr��z��w�%��������A���9�����,����B�59���Y���aYp�D��._�b�y�����+�#8�W�Sol���$����"+�M��[tG�7�����~����hWW���*!������k�p �S]��\���Q�!��!"5�>*����+;{�e�=�tR
]Q>��5�'����h������\�qcPF]����:y�r`�9 �h!�L���pa&����0G���_�\�)��F*��48��w��[�r��������!PTJ6WT�c��Uu=�L�[�m)���sav�c1`�0/���	sps)��5D��ZP���l<�'�s�?U���m���Y���&���!P��I����:E.W��f�_�j��}w���
�0�r��p [�#�v�%�������eL|�J�w%����$�P��"~��)WH�����y]�G�_[���yIR.W!+��D�Z
pBj-E�}K�7�&�����"��&�d1�����&�����3�?c����5���B���m����g�*NO�������3;I3�-�w�{rqx�
J�/��n������Q���-Ya���4��NhQW)=S7�����cA�W��T�S�������f�:�z�/q��B�����\��D8�5���J�'�ZH����T5�p_�OmYu���{���T�FR�@'F��T���#-\����0^���/Ouwp��P�����qa';�Y��"6l ;I�j��e[������{�L��0�%w����hb�y��+�|4����:�O a��5C]vU�lJ
-f��9I�#|���0�E��Lb�4�[_��2��efE���C`kv/��R��m����
�����m�����c�����{5u����<�E��L�
]�E��@���l��.��1���1�H�E��I�;tdC�z���^��C�����;��t�/P�/�'�k���)�+�pg��#���KPx����c;t_��g�4����4Z�1�fPw~�r<���T��Y��n=I{���H�]=
�CK�j�x`�ZU����|r-�������}y^�������k�!l�oN�Z��C�h.wg,���q��##�h9��^�y�8�iL�1�5�R�8�C������.���fx)���t���ck��r�J=vc��6�Q%���nD6,������7@H�}g6��_�9��w�l0 �-����}�o��*i<��S�U�+�wY(��k����'��5��C�x6��s��Q���=D(�y�Ncn�J��;�/��@��n���9^����;��cQiV��<�j5+	LT�)�
6����c	���=�U:(�C(@d�X&��y�P=���r����r����Z�=��'����U8�� �c�����a���p�0UQ�u�VZ�67h�QG�g8�������:�M�$�J�H(/�^yUB��+�pg�=��9����2��^GV�
�=�8���k~PPx���f��\��������K
[R�HTdTG*��������m|�r�a��K�����m�i]0i�t��aabQX����J����?>���t��_�v���]���-"�u�a����nM�\Vj��DzH) wE3�������W�v_JH�4}Wm�����G���e�CR�*$�"b��q�wt�<��{����W/��2=#��?"iJ������5���`Ia9^P��Lg6��W��D�-aH�1g�s�?{�;���~<�����L���M�]��=�N�	�����C��]��������n��?5m���A(�r.g �p�lJ m��;�!|��a��o
����4�IDR���ZM��j'$�?. �D�	tqK9K#�,KR��#I_��rq�:h��B->?�Q�XM������X1@s��6�C�Qz|Wm�Ly��T��Q�q�-����v�!��*��]���������C�i���'�C�s+��:.�f��9C��s�CJ8"�������������]��\r����JNw�E'���<8~�2r(eIs�g��C�\w��q��
�n4�&���$�+�l	��|N�����Q������y�����kJB`���L�Q�f^��l��2�����0���wS�@#"	�0���EH�e�Tq=��h�I��t`-DP]��t���N���z�7:o�������6��)/������ag^����s��"O��>���I��E:jL�	���]��u
+o���\&p?�Qolwo����O7����&1�P�s\x��r���?���C���-��5�������P��	�`TApyX�C:��d��^h���� ��j4�'=����|�������*����|�x���V������6An��9u.��s##?l����(����U��$?\����ym�
~�kgpR�?�D�^�2��s�w�����x��TD�ou���������:Tp�~m����7bl)v�VQMV��f))���"�B5�
���	b.3���0
������y�3wM��1q�w�!Fj��s3;��O��o�u�?!����4<3{����{9	��a�������k��|>�7?�S&���Cn��R2�dw3]Bbt~
R��$����x��xNU@���$*u�"�N��
(r
��k����/�9)�v���O1�Wv�����q���]�P ���2�_g�NC={�:�]����w9=~s�����
~�	���5�15!x�r&!�@B�W�����c�k�*��RF���(���"I4�sQM����#��e�`4��M��O�#�����g����9�]e�\��)������q��OG�j��O�H�51'l�������}\O�{A���
#z��2�Dj��O��o���V�Q��t�����S=���Nn� �e���j��^3�~�D��L�����LqK���[a
������N��]���x1[��=�W@�wo��P�
��Xs
�l�Hn�O��8gyn,��B�sc-H�N�����|�_�7Ct
���u_����� }�_�Y�������4X��u���Y��j��~~��!nlc
��9��\L05�M�\�U
�>��j��,�"}Q��D\��K�d%	�e}������'v���_��
��u�--�����LX�;����]M
�$����=����K��Gi5H^Np����Y�M��u��>�F�����N�]�{�.��Hxx�vz�^?�����a�������~�`���!y��RX��b��^M"��H��MjQc���������Tun���l4�>9��FS$!�r���xn��Rv���WqF
z<b��#��ht�_�|������!3�'2^����KV�������.�q�_�+9;����i�B�J�0�T�(?6p#��#�wy	�1o%e1��K�u��+K+RZu���?�����	+(�D�����24�����h�`�����$�H���)�n��3�����
�H���>�t�~�1��G�`�S�.�����D��
�8hU~:?>�������,qZ<���u��nHH6�0�#�����1��"�?n�R���R$b������
DjK�W�������E�N�jk$��T���y���w�P����@[�����,�NJ������R+CkY����i����n��oU�gS���o�=��K �2%��X���5�G�I|t���L5� �~��mt+8�%,D���e\���2i!C=�
�]�\�Z6�S�����-�@�&(L��uh���������m5i��Sy�6_�c����7j��lD�K���Dy���D������|to�������3����:<��l Y1���`��������&��,��z��8����kT�n��H`�������:o��J�F�dd�L,�2W�_G�D��d5>O�������_��vxga��a���������FC�
z�7&\��7�;[�$���q!B�������7���]?�S,����?C�d���y�a+&g��U�4Q-7�X��Y8ow�jP{!���L7cI�����"U����g]txh�}}�\��s����������JWu�X�p���c=�c�sGt���L5�H��W��T��(��kHU��q��x=i��=�{�r!���R��7���L/.,�W��$ ���2��������"{'�z=x�������9�8���O��}�����TgqvJg
�b����Zt����j&���o$a�7;���	��/gHm���=+K�!�M�K�f��y_�U=-�q����p��+Wae��Qa�H�TE*w'��
<wO��q�D�|�Ss�
/�*#^n������pq�oc����-�)Q��e�8+��)Y��Q����(5�
�n�!��dv�0�	RR;RG>���9z���d��
#�SF�l��8�3������)<��`q=����}�m����6��u ����T�(I3cC�����������<���x����Pi���k������\�MvC\09�����2#a�l&����w�@�����yb�f2��f���SpiL���
nh��H�v���;�)�(GQg=9�e��$O�r��
�'"�,�k���\;�o>�a�<	�'��?��S����-����K��!=+I�b��L���?��0����J.��GI����L���>n����t�v��
��+��������1I*VF�����R��w��
���* v�`��;�DX ?��G�R,0�*�;�c������%$Q�z�I����"&��Z��tk,Ay�x������/�����|�ph�k#�a�\d�?����~l����=����-����w4�����I�~���OaN�h�(!�K*!������h�J��^]��z!H.\�D�Z�j��C�##^9����G~xg���>��~��W���_�h4�.i5��m7{�~���b(7--��1'"��W��trxB���QLtp�����pj���:(�x�n���
����i��?4v/�i06��(g�2}�� ����e�h�r��|����>�0����'�;$%`��j����@�{�O��zt���6[��
T%j�a&��-q2��\���M��1��Aay&o&�cQ>V3%J���G+��b�D�a�����Z("�������w�y1��H�v�_��yKN����5x�0,��PxAa��#x���8n��K�$�r�?���.q��A��q�
����P�J��i��!h-��I���uW�x�y�{j��O*e�J?���
�)��S9/D*�#�[wp���YGC�Y�eX������@���=��|��M����~����''�H���qa��C�i$Cz/P%���
���P���XI���0�
D��X!���eD-����6�.E��@�D(B�rRD�Z'��)���������6�q�7r�>�O>�A�d<��{nf���N&	��FQ��r
9�E���8,<%�Ic7���TK^���Vu����%U5�3UlDg����+�>@�D�$��"�@Q�T�`�PA��x�r�������]f3F�Q�S�������\FdI���T5�C��zi
6vJF}���y
$���H5�2V`C����n���a
��T��r&5b���sH[�p��d�mG��o� u�v������3=�����fu�!]��������x��RE����l�,2�������=�[�O������.g���u��M_�	�de���H�B$��E��c���+����u�^B��i�2,1+0�����x~���}n��X�c�3��X�z���.J�a`��j�����{��O�m����dA��Wl�Zt�#
l��
P��u|�=��B�^t?*���4de����i�Hx����3�+���,�R���G��/N�|�9��|��+��]��-zw:A�|A�����ar3�/!���J7��3~����~t8����`����hd/����d�H3�����9��vg]�<��%	q��H����p�WZ��>�P�xCnlc�8fU�W(�|����x��MNh��G���m��G��O�������S�2���_G�be���Q���W�����<��a;~��#���' jI��2M��'{���01���_ �����z�������z�\]pI��������<BgeL�������y}xi�������>������N#���t*	s�x-��Cw����
����&#����]��R�67L��rL7�.e�1� ����/I��6���p�d`jT�b�Hn?�u8z��D#��1�Z�CdJ� ��J�R����v�6��5D�2g�C��}�H�X���x-���c\9o��k�<U�
fJ���m��(|E��W�r��/���B<`?��u�	�A~n}y�?�3$9K2o������N�Y�v�0�k�?M���_q�-0��!�n	�g����W����*��}*�����L �Z�%���.�).�����+�`pe��A����h!B���<��R�j8R�"x^��������:dE�;(�����V%��������	�'�4��&�@K2�k'X+C+�������\��>5���<�$�R7�p$Krq���4�6p�����W�I^H)�����'S�����2�G�j��Z����ZN��%�3ka7��=�����mb�c��?��������]�+���������wm��d�R����gI�I�����|	��U-��`X.�u)Ex3��u*���(R�Qb������?}��2<��6D|�P&j�,M�%nP7�t?��9��Rw{�n��0����z��I�X1w=f�I_u�<-8�l6�T�.aj��6�-���u/���W��@@XZ"$u|�}���E+�j�K������?Z{8n�����S������@�O�\�
��Z�������
�����F/A�7�Q�V�+�Le�(����5��|�k��a}�~?@j��y���l���w�������ZN��M���u7W=O$d�F��I��];���]�$���������?��;�<��\W��& ��-���|&��%��ods�g;"�UW����r�{��F���i;f������"m_��4�?���7�l�R�U�z�������M�2��ZQ&��p�er9}J���`�^��ELC�V	:��U��o������!�`.�>4:DZ0au})S���?C�3�L-
��_���w������5���n��9�E��2Hx+%Vz���
<�UO*M��|��H01����HK����g�"���O����0����ur�j^�G��y�rl~�U���'����dM������Z����_�UK�.X��0%��I1
krb�>1Ga�����
(�o����j*V�����@�"���1��r�q������<v��'��F�����2ZqR�\��I�7�;�zv��?y����g�s%�	�9cak�������Z��m7����a��(����K+�x���H*��?����L���K��n����y^Lh���X����$��j,���2R1�m�������@sc��%
xd���_A�c��wl!d�|h��j��y{,H���(�*`��J�!R>���_4K{X~��9������6�������@�� *�
N�����[�V��-15D�����q�F�F�J9%!+���/�7g��x���%%A�����,A
��8��� >'Q���}������}�]��M�9w����L!��gA7kB����,��*}��ST>;=�����`+�6�����)�b��K�F�j�$TI�z.���ql����2;c�t��'W���k"��{]�d�p���kw[O1(X=����be>��C��@�rpX����V��|>u����{��O4V���Oua�����y�d>�
��y�=�/#����0L�H������Y��Z��G�\�2*.�B��YJ��P�z�Ww���bP�W�?x�T��i9��X���j����v
���W���{B��
�mtU��26KZ6�(��D���^����w<�R����"�t
o��O|
�m���GQ��
r�M�w��w�o-qP���+Q
{��9|U�������!i���F����u��Pl�(z���
��_\!���n�qE3����B���������N0��+	��~��.�RJ��/J�A�I
�(89�R3�����l!�O��'��x�H��-�����jT{yS�������UQ��F�B�6�V����Dw�n�>t;X�^|�X�j+��(Uh`�&H{"ELH���Jf��F���F�'���8��������P�+8�q���?�������~�K}k�6�^�)"���*u�(�~�YqS��t3�~���syA)7
�WH0_��z�G<"O�D2O
F���k�?�a���/����HAD���"be��g�.c����"���b`�{���KS��#<�������~�����]�Y��K�?��IG(�<�Ru�9���8]I#�Ndn�3
y�����5���5i���Q#��Z��w��U#�u�kB�&�,7([�jp[�������=��x����S��0d\.�4�� ��P�A?����
�vu}�w������y�tG���x<
n;�O	*�!6?QKB8+�S�{F���M	��c����������W�L����0�/����&4�K���
��o����_��?�����
#16Simon Riggs
simon@2ndQuadrant.com
In reply to: Stas Kelvich (#14)
Re: Speedup twophase transactions

On 11 January 2016 at 12:58, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

On 10 Jan 2016, at 12:15, Simon Riggs <simon@2ndquadrant.com> wrote:

So we've only optimized half the usage? We're still going to cause

replication delays.

Yes, replica will go through old procedures of moving data to and from
file.

We can either

1) Skip fsyncing the RecreateTwoPhaseFile and then fsync during

restartpoints

From what i’ve seen with old 2pc code main performance bottleneck was
caused by frequent creating of files. So better to avoid files if possible.

2) Copy the contents to shmem and then write them at restartpoint as we

do for checkpoint

(preferred)

Problem with shared memory is that we can’t really predict size of state
data, and anyway it isn’t faster then reading data from WAL
(I have tested that while preparing original patch).

We can just apply the same logic on replica that on master: do not do
anything special on prepare, and just read that data from WAL.
If checkpoint occurs during recovery/replay probably existing code will
handle moving data to files.

I will update patch to address this issue.

I'm looking to commit what we have now, so lets do that as a separate but
necessary patch please.

I think padding will negate the effects of the additional bool.

If we want to reduce the size of the array GIDSIZE is currently 200, but

XA says maximum 128 bytes.

Anybody know why that is set to 200?

Good catch about GID size.

I'll apply that as a separate patch also.

If we talk about further optimisations i see two ways:

1) Optimising access to GXACT. Here we can try to shrink it; introduce
more granular locks,
e.g. move GIDs out of GXACT and lock GIDs array only once while checking
new GID uniqueness; try to lock only part of GXACT by hash; etc.

Have you measured lwlocking as a problem?

2) Be optimistic about consequent COMMIT PREPARED. In normal workload next
command after PREPARE will be COMMIT/ROLLBACK, so we can save
transaction context and release it only if next command isn’t our
designated COMMIT/ROLLBACK. But that is a big amount of work and requires
changes to whole transaction pipeline in postgres.

We'd need some way to force session pools to use that correctly, but yes,
agreed.

Anyway I suggest that we should consider that as a separate task.

Definitely. From the numbers, I can see there is still considerable
performance gain to be had.

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#17Jesper Pedersen
jesper.pedersen@redhat.com
In reply to: Simon Riggs (#13)
Re: Speedup twophase transactions

On 01/10/2016 04:15 AM, Simon Riggs wrote:

One concern that come into my mind while reading updated
patch is about creating extra bool field in GlobalTransactionData
structure. While this improves readability, it
also increases size of that structure and that size have impact on
performance on systems with many cores
(say like 60-80). Probably one byte will not make measurable difference,
but I think it is good idea to keep
GXact as small as possible. As far as I understand the same logic was
behind split of
PGPROC to PGPROC+PGXACT in 9.2 (comment in proc.h:166)

I think padding will negate the effects of the additional bool.

If we want to reduce the size of the array GIDSIZE is currently 200, but XA
says maximum 128 bytes.

Anybody know why that is set to 200?

Even though GlobalTransactionId and BranchQualifer have a maximum of 64
each, external clients may choose to encode the information, and thereby
need more space,

https://github.com/pgjdbc/pgjdbc/blob/master/pgjdbc/src/main/java/org/postgresql/xa/RecoveredXid.java#L66-L70

http://docs.oracle.com/javaee/7/api/javax/transaction/xa/Xid.html

which in this case adds up to a maximum of 189 characters.

Best regards,
Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#18Andres Freund
andres@anarazel.de
In reply to: Simon Riggs (#10)
Re: Speedup twophase transactions

Hi,

On 2016-01-09 15:29:11 +0000, Simon Riggs wrote:

Hmm, I was just preparing this for commit.

Just read downthread that you want to commit this soon. Please hold of
for a while, this doesn't really look ready to me. I don't have time for
a real review right now, but I'll try to get to it asap.

+
+/*
+ * Reads 2PC data from xlog. During checkpoint this data will be moved to
+ * twophase files and ReadTwoPhaseFile should be used instead.
+ */
+static void
+XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
+{
+	XLogRecord *record;
+	XLogReaderState *xlogreader;
+	char	   *errormsg;
+
+	xlogreader = XLogReaderAllocate(&logical_read_local_xlog_page,
NULL);

logical_read_local_xlog_page isn't really suitable for the use
here. Besides the naming issue, at the very least it'll be wrong during
WAL replay in the presence of promotions on an upstream node - it
doesn't dealwith timelines.

More generally, I'm doubtful that the approach of reading data from WAL
as proposed here is a very good idea. It seems better to "just" dump the
entire 2pc state into *one* file at checkpoint time.

Greetings,

Andres Freund

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#19Andres Freund
andres@anarazel.de
In reply to: Andres Freund (#18)
Re: Speedup twophase transactions

On 2016-01-11 20:03:18 +0100, Andres Freund wrote:

More generally, I'm doubtful that the approach of reading data from WAL
as proposed here is a very good idea. It seems better to "just" dump the
entire 2pc state into *one* file at checkpoint time.

Or better: After determining the checkpoint redo location, insert a WAL
record representing the entire 2PC state as of that moment. That way it
can easily restored during WAL replay and nothing special has to be done
on a standby. This way we'll need no extra wal flushes and fsyncs.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#20Simon Riggs
simon@2ndQuadrant.com
In reply to: Andres Freund (#18)
Re: Speedup twophase transactions

On 11 January 2016 at 19:03, Andres Freund <andres@anarazel.de> wrote:

Hi,

On 2016-01-09 15:29:11 +0000, Simon Riggs wrote:

Hmm, I was just preparing this for commit.

Just read downthread that you want to commit this soon. Please hold of
for a while, this doesn't really look ready to me. I don't have time for
a real review right now, but I'll try to get to it asap.

"A real review"? Huh.

+
+/*
+ * Reads 2PC data from xlog. During checkpoint this data will be moved

to

+ * twophase files and ReadTwoPhaseFile should be used instead.
+ */
+static void
+XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
+{
+     XLogRecord *record;
+     XLogReaderState *xlogreader;
+     char       *errormsg;
+
+     xlogreader = XLogReaderAllocate(&logical_read_local_xlog_page,
NULL);

logical_read_local_xlog_page isn't really suitable for the use
here. Besides the naming issue, at the very least it'll be wrong during
WAL replay in the presence of promotions on an upstream node - it
doesn't dealwith timelines.

I'm aware of that, though note that it isn't being used in that way here.

More generally, I'm doubtful that the approach of reading data from WAL
as proposed here is a very good idea. It seems better to "just" dump the
entire 2pc state into *one* file at checkpoint time.

I think you misunderstand the proposed approach. This isn't just to do
with reading things back at checkpoint.

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#21Simon Riggs
simon@2ndQuadrant.com
In reply to: Andres Freund (#19)
Re: Speedup twophase transactions

On 11 January 2016 at 19:07, Andres Freund <andres@anarazel.de> wrote:

On 2016-01-11 20:03:18 +0100, Andres Freund wrote:

More generally, I'm doubtful that the approach of reading data from WAL
as proposed here is a very good idea. It seems better to "just" dump the
entire 2pc state into *one* file at checkpoint time.

Or better: After determining the checkpoint redo location, insert a WAL
record representing the entire 2PC state as of that moment. That way it
can easily restored during WAL replay and nothing special has to be done
on a standby. This way we'll need no extra wal flushes and fsyncs.

Feel free to submit a patch that does that.

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#22Andres Freund
andres@anarazel.de
In reply to: Simon Riggs (#20)
Re: Speedup twophase transactions

On 2016-01-11 19:15:23 +0000, Simon Riggs wrote:

On 11 January 2016 at 19:03, Andres Freund <andres@anarazel.de> wrote:

Hi,

On 2016-01-09 15:29:11 +0000, Simon Riggs wrote:

Hmm, I was just preparing this for commit.

Just read downthread that you want to commit this soon. Please hold of
for a while, this doesn't really look ready to me. I don't have time for
a real review right now, but I'll try to get to it asap.

"A real review"? Huh.

All I meant was that my email didn't consist out of a real review, but
just was a quick scan

More generally, I'm doubtful that the approach of reading data from WAL
as proposed here is a very good idea. It seems better to "just" dump the
entire 2pc state into *one* file at checkpoint time.

I think you misunderstand the proposed approach. This isn't just to do
with reading things back at checkpoint.

Sure, the main purpose is not to write 2pc state files in the common
path - or is that not the main purpose?

Greetings,

Andres Freund

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#23Simon Riggs
simon@2ndQuadrant.com
In reply to: Jesper Pedersen (#17)
Re: Speedup twophase transactions

On 11 January 2016 at 18:51, Jesper Pedersen <jesper.pedersen@redhat.com>
wrote:

On 01/10/2016 04:15 AM, Simon Riggs wrote:

One concern that come into my mind while reading updated

patch is about creating extra bool field in GlobalTransactionData
structure. While this improves readability, it
also increases size of that structure and that size have impact on
performance on systems with many cores
(say like 60-80). Probably one byte will not make measurable difference,
but I think it is good idea to keep
GXact as small as possible. As far as I understand the same logic was
behind split of
PGPROC to PGPROC+PGXACT in 9.2 (comment in proc.h:166)

I think padding will negate the effects of the additional bool.

If we want to reduce the size of the array GIDSIZE is currently 200, but
XA
says maximum 128 bytes.

Anybody know why that is set to 200?

Even though GlobalTransactionId and BranchQualifer have a maximum of 64
each, external clients may choose to encode the information, and thereby
need more space,

https://github.com/pgjdbc/pgjdbc/blob/master/pgjdbc/src/main/java/org/postgresql/xa/RecoveredXid.java#L66-L70

http://docs.oracle.com/javaee/7/api/javax/transaction/xa/Xid.html

which in this case adds up to a maximum of 189 characters.

OK, thanks for those references.

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#24Simon Riggs
simon@2ndQuadrant.com
In reply to: Andres Freund (#22)
Re: Speedup twophase transactions

On 11 January 2016 at 19:18, Andres Freund <andres@anarazel.de> wrote:

More generally, I'm doubtful that the approach of reading data from WAL

as proposed here is a very good idea. It seems better to "just" dump

the

entire 2pc state into *one* file at checkpoint time.

I think you misunderstand the proposed approach. This isn't just to do
with reading things back at checkpoint.

Sure, the main purpose is not to write 2pc state files in the common
path - or is that not the main purpose?

Yes, that is the main purpose, but that's not what you were talking about.

Currently, the patch reuses all of the code related to reading/write state
files, so it is the minimal patch that can implement the important things
for performance. The current patch succeeds in its goal to improve
performance, so I personally see no further need for code churn.

As you suggest, we could also completely redesign the state file mechanism
and/or put it in WAL at checkpoint. That's all very nice but is much more
code and doesn't anything more for performance, since the current mainline
path writes ZERO files at checkpoint. If you want that for some other
reason or refactoring, I won't stop you, but its a separate patch for a
separate purpose.

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#25Simon Riggs
simon@2ndQuadrant.com
In reply to: Simon Riggs (#16)
1 attachment(s)
Re: Speedup twophase transactions

On 11 January 2016 at 18:43, Simon Riggs <simon@2ndquadrant.com> wrote:

I'm looking to commit what we have now.

Here is the patch in its "final" state after my minor additions, edits and
review.

Performance tests for me show that the patch is effective; my results match
Jesper's roughly in relative numbers.

My robustness review is that the approach and implementation are safe.

It's clear there are various additional tuning opportunities, but the
objective of the current patch to improve performance is very, very clearly
met, so I'm aiming to commit *this* patch soon.

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Attachments:

2pc_optimize.v4.patchapplication/octet-stream; name=2pc_optimize.v4.patchDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 2251b02..80aba8c 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -25,11 +25,28 @@
  *		what keeps the XID considered running by TransactionIdIsInProgress.
  *		It is also convenient as a PGPROC to hook the gxact's locks to.
  *
- *		In order to survive crashes and shutdowns, all prepared
- *		transactions must be stored in permanent storage. This includes
- *		locking information, pending notifications etc. All that state
- *		information is written to the per-transaction state file in
- *		the pg_twophase directory.
+ * 		Information to recover prepared transactions in case of crash is
+ * 		now stored in WAL for the common case. In some cases there will be
+ * 		an extended period between preparing a GXACT and commit/abort, in
+ * 		which case we need to separately record prepared transaction data
+ * 		in permanent storage. This includes locking information, pending
+ * 		notifications etc. All that state information is written to the
+ * 		per-transaction state file in the pg_twophase directory.
+ * 		All prepared transactions will be written prior to shutdown.
+ *
+ *		Life track of state data is following:
+ *
+ *		* On PREPARE TRANSACTION backend writes state data only to the WAL and
+ *		  stores pointer to the start of the WAL record in
+ *		  gxact->prepare_start_lsn.
+ *		* If COMMIT occurs before checkpoint then backend reads data from WAL
+ *		  using prepare_start_lsn.
+ *		* On checkpoint state data copied to files in pg_twophase directory and
+ *		  fsynced
+ *		* If COMMIT happens after checkpoint then backend reads state data from
+ *		  files
+ *		* In case of crash replay will move data from xlog to files, if that
+ *		  hasn't happened before. XXX TODO - move to shmem in replay also
  *
  *-------------------------------------------------------------------------
  */
@@ -51,6 +68,7 @@
 #include "access/xlog.h"
 #include "access/xloginsert.h"
 #include "access/xlogutils.h"
+#include "access/xlogreader.h"
 #include "catalog/pg_type.h"
 #include "catalog/storage.h"
 #include "funcapi.h"
@@ -60,6 +78,7 @@
 #include "replication/origin.h"
 #include "replication/syncrep.h"
 #include "replication/walsender.h"
+#include "replication/logicalfuncs.h"
 #include "storage/fd.h"
 #include "storage/ipc.h"
 #include "storage/predicate.h"
@@ -117,10 +136,21 @@ typedef struct GlobalTransactionData
 	int			pgprocno;		/* ID of associated dummy PGPROC */
 	BackendId	dummyBackendId; /* similar to backend id for backends */
 	TimestampTz prepared_at;	/* time of preparation */
-	XLogRecPtr	prepare_lsn;	/* XLOG offset of prepare record */
+
+	/*
+	 * Note that we need to keep track of two LSNs for each GXACT.
+	 * We keep track of the start LSN because this is the address we must
+	 * use to read state data back from WAL when committing a prepared GXACT.
+	 * We keep track of the end LSN because that is the LSN we need to wait
+	 * for prior to commit.
+	 */
+	XLogRecPtr	prepare_start_lsn;	/* XLOG offset of prepare record start */
+	XLogRecPtr	prepare_end_lsn;	/* XLOG offset of prepare record end */
+
 	Oid			owner;			/* ID of user that executed the xact */
 	BackendId	locking_backend;	/* backend currently working on the xact */
 	bool		valid;			/* TRUE if PGPROC entry is in proc array */
+	bool		ondisk;			/* TRUE if prepare state file is on disk */
 	char		gid[GIDSIZE];	/* The GID assigned to the prepared xact */
 }	GlobalTransactionData;
 
@@ -166,6 +196,7 @@ static void ProcessRecords(char *bufptr, TransactionId xid,
 			   const TwoPhaseCallback callbacks[]);
 static void RemoveGXact(GlobalTransaction gxact);
 
+static void XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len);
 
 /*
  * Initialization of shared memory
@@ -398,8 +429,9 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 	pgxact->nxids = 0;
 
 	gxact->prepared_at = prepared_at;
-	/* initialize LSN to 0 (start of WAL) */
-	gxact->prepare_lsn = 0;
+	/* initialize LSN to InvalidXLogRecPtr */
+	gxact->prepare_start_lsn = InvalidXLogRecPtr;
+	gxact->prepare_end_lsn = InvalidXLogRecPtr;
 	gxact->owner = owner;
 	gxact->locking_backend = MyBackendId;
 	gxact->valid = false;
@@ -579,41 +611,6 @@ RemoveGXact(GlobalTransaction gxact)
 }
 
 /*
- * TransactionIdIsPrepared
- *		True iff transaction associated with the identifier is prepared
- *		for two-phase commit
- *
- * Note: only gxacts marked "valid" are considered; but notice we do not
- * check the locking status.
- *
- * This is not currently exported, because it is only needed internally.
- */
-static bool
-TransactionIdIsPrepared(TransactionId xid)
-{
-	bool		result = false;
-	int			i;
-
-	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
-
-	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
-	{
-		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
-		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
-
-		if (gxact->valid && pgxact->xid == xid)
-		{
-			result = true;
-			break;
-		}
-	}
-
-	LWLockRelease(TwoPhaseStateLock);
-
-	return result;
-}
-
-/*
  * Returns an array of all prepared transactions for the user-level
  * function pg_prepared_xact.
  *
@@ -772,7 +769,7 @@ TwoPhaseGetGXact(TransactionId xid)
 	 * During a recovery, COMMIT PREPARED, or ABORT PREPARED, we'll be called
 	 * repeatedly for the same XID.  We can save work with a simple cache.
 	 */
-	if (xid == cached_xid)
+	if (xid == cached_xid && cached_gxact)
 		return cached_gxact;
 
 	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
@@ -1020,14 +1017,8 @@ StartPrepare(GlobalTransaction gxact)
 void
 EndPrepare(GlobalTransaction gxact)
 {
-	PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
-	TransactionId xid = pgxact->xid;
 	TwoPhaseFileHeader *hdr;
-	char		path[MAXPGPATH];
 	StateFileChunk *record;
-	pg_crc32c	statefile_crc;
-	pg_crc32c	bogus_crc;
-	int			fd;
 
 	/* Add the end sentinel to the list of 2PC records */
 	RegisterTwoPhaseRecord(TWOPHASE_RM_END_ID, 0,
@@ -1048,70 +1039,8 @@ EndPrepare(GlobalTransaction gxact)
 				 errmsg("two-phase state file maximum length exceeded")));
 
 	/*
-	 * Create the 2PC state file.
-	 */
-	TwoPhaseFilePath(path, xid);
-
-	fd = OpenTransientFile(path,
-						   O_CREAT | O_EXCL | O_WRONLY | PG_BINARY,
-						   S_IRUSR | S_IWUSR);
-	if (fd < 0)
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not create two-phase state file \"%s\": %m",
-						path)));
-
-	/* Write data to file, and calculate CRC as we pass over it */
-	INIT_CRC32C(statefile_crc);
-
-	for (record = records.head; record != NULL; record = record->next)
-	{
-		COMP_CRC32C(statefile_crc, record->data, record->len);
-		if ((write(fd, record->data, record->len)) != record->len)
-		{
-			CloseTransientFile(fd);
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not write two-phase state file: %m")));
-		}
-	}
-
-	FIN_CRC32C(statefile_crc);
-
-	/*
-	 * Write a deliberately bogus CRC to the state file; this is just paranoia
-	 * to catch the case where four more bytes will run us out of disk space.
-	 */
-	bogus_crc = ~statefile_crc;
-
-	if ((write(fd, &bogus_crc, sizeof(pg_crc32c))) != sizeof(pg_crc32c))
-	{
-		CloseTransientFile(fd);
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not write two-phase state file: %m")));
-	}
-
-	/* Back up to prepare for rewriting the CRC */
-	if (lseek(fd, -((off_t) sizeof(pg_crc32c)), SEEK_CUR) < 0)
-	{
-		CloseTransientFile(fd);
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not seek in two-phase state file: %m")));
-	}
-
-	/*
-	 * The state file isn't valid yet, because we haven't written the correct
-	 * CRC yet.  Before we do that, insert entry in WAL and flush it to disk.
-	 *
-	 * Between the time we have written the WAL entry and the time we write
-	 * out the correct state file CRC, we have an inconsistency: the xact is
-	 * prepared according to WAL but not according to our on-disk state. We
-	 * use a critical section to force a PANIC if we are unable to complete
-	 * the write --- then, WAL replay should repair the inconsistency.  The
-	 * odds of a PANIC actually occurring should be very tiny given that we
-	 * were able to write the bogus CRC above.
+	 * Now writing 2PC state data to WAL. We let the WAL's CRC protection
+	 * cover us, so no need to calculate a separate CRC.
 	 *
 	 * We have to set delayChkpt here, too; otherwise a checkpoint starting
 	 * immediately after the WAL record is inserted could complete without
@@ -1131,24 +1060,13 @@ EndPrepare(GlobalTransaction gxact)
 	XLogBeginInsert();
 	for (record = records.head; record != NULL; record = record->next)
 		XLogRegisterData(record->data, record->len);
-	gxact->prepare_lsn = XLogInsert(RM_XACT_ID, XLOG_XACT_PREPARE);
-	XLogFlush(gxact->prepare_lsn);
+	gxact->prepare_end_lsn = XLogInsert(RM_XACT_ID, XLOG_XACT_PREPARE);
+	XLogFlush(gxact->prepare_end_lsn);
 
 	/* If we crash now, we have prepared: WAL replay will fix things */
 
-	/* write correct CRC and close file */
-	if ((write(fd, &statefile_crc, sizeof(pg_crc32c))) != sizeof(pg_crc32c))
-	{
-		CloseTransientFile(fd);
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not write two-phase state file: %m")));
-	}
-
-	if (CloseTransientFile(fd) != 0)
-		ereport(ERROR,
-				(errcode_for_file_access(),
-				 errmsg("could not close two-phase state file: %m")));
+	/* Store record's start location to read that later on Commit */
+	gxact->prepare_start_lsn = ProcLastRecPtr;
 
 	/*
 	 * Mark the prepared transaction as valid.  As soon as xact.c marks
@@ -1186,7 +1104,7 @@ EndPrepare(GlobalTransaction gxact)
 	 * Note that at this stage we have marked the prepare, but still show as
 	 * running in the procarray (twice!) and continue to hold locks.
 	 */
-	SyncRepWaitForLSN(gxact->prepare_lsn);
+	SyncRepWaitForLSN(gxact->prepare_end_lsn);
 
 	records.tail = records.head = NULL;
 	records.num_chunks = 0;
@@ -1315,6 +1233,45 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
 	return buf;
 }
 
+
+/*
+ * Reads 2PC data from xlog. During checkpoint this data will be moved to
+ * twophase files and ReadTwoPhaseFile should be used instead.
+ */
+static void
+XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
+{
+	XLogRecord *record;
+	XLogReaderState *xlogreader;
+	char	   *errormsg;
+
+	xlogreader = XLogReaderAllocate(&logical_read_local_xlog_page, NULL);
+	if (!xlogreader)
+		ereport(ERROR,
+				(errcode(ERRCODE_OUT_OF_MEMORY),
+				 errmsg("out of memory"),
+				 errdetail("Failed while allocating an XLog reading processor.")));
+
+	record = XLogReadRecord(xlogreader, lsn, &errormsg);
+	if (record == NULL ||
+		XLogRecGetRmid(xlogreader) != RM_XACT_ID ||
+		(XLogRecGetInfo(xlogreader) & XLOG_XACT_OPMASK) != XLOG_XACT_PREPARE)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not read two-phase state from xlog at %X/%X",
+							(uint32) (lsn >> 32),
+							(uint32) lsn)));
+
+	if (len != NULL)
+		*len = XLogRecGetDataLen(xlogreader);
+
+	*buf = palloc(sizeof(char)*XLogRecGetDataLen(xlogreader));
+	memcpy(*buf, XLogRecGetData(xlogreader), sizeof(char)*XLogRecGetDataLen(xlogreader));
+
+	XLogReaderFree(xlogreader);
+}
+
+
 /*
  * Confirms an xid is prepared, during recovery
  */
@@ -1375,14 +1332,16 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	xid = pgxact->xid;
 
 	/*
-	 * Read and validate the state file
+	 * Read and validate 2PC state data.
+	 * State data will typically be stored in WAL files if the LSN is after the
+	 * last checkpoint record, or moved to disk if for some reason they have
+	 * lived for a long time.
 	 */
-	buf = ReadTwoPhaseFile(xid, true);
-	if (buf == NULL)
-		ereport(ERROR,
-				(errcode(ERRCODE_DATA_CORRUPTED),
-				 errmsg("two-phase state file for transaction %u is corrupt",
-						xid)));
+	if (gxact->ondisk)
+		buf = ReadTwoPhaseFile(xid, true);
+	else
+		XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, NULL);
+
 
 	/*
 	 * Disassemble the header area
@@ -1482,9 +1441,10 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	AtEOXact_PgStat(isCommit);
 
 	/*
-	 * And now we can clean up our mess.
+	 * And now we can clean up any files we may have left.
 	 */
-	RemoveTwoPhaseFile(xid, true);
+	if (gxact->ondisk)
+		RemoveTwoPhaseFile(xid, true);
 
 	RemoveGXact(gxact);
 	MyLockedGxact = NULL;
@@ -1539,7 +1499,8 @@ RemoveTwoPhaseFile(TransactionId xid, bool giveWarning)
 }
 
 /*
- * Recreates a state file. This is used in WAL replay.
+ * Recreates a state file. This is used in WAL replay and during
+ * checkpoint creation.
  *
  * Note: content and len don't include CRC.
  */
@@ -1610,97 +1571,71 @@ RecreateTwoPhaseFile(TransactionId xid, void *content, int len)
  * This is deliberately run as late as possible in the checkpoint sequence,
  * because GXACTs ordinarily have short lifespans, and so it is quite
  * possible that GXACTs that were valid at checkpoint start will no longer
- * exist if we wait a little bit.
+ * exist if we wait a little bit. With typical checkpoint settings this
+ * will be about 3 minutes for an online checkpoint, so as a result we
+ * we expect that there will be no GXACTs that need to be copied to disk.
  *
- * If a GXACT remains valid across multiple checkpoints, it'll be fsynced
- * each time.  This is considered unusual enough that we don't bother to
- * expend any extra code to avoid the redundant fsyncs.  (They should be
- * reasonably cheap anyway, since they won't cause I/O.)
+ * If a GXACT remains valid across multiple checkpoints, it will already
+ * be on disk so we don't bother to repeat that write.
  */
 void
 CheckPointTwoPhase(XLogRecPtr redo_horizon)
 {
-	TransactionId *xids;
-	int			nxids;
-	char		path[MAXPGPATH];
 	int			i;
+	int			n = 0;
 
-	/*
-	 * We don't want to hold the TwoPhaseStateLock while doing I/O, so we grab
-	 * it just long enough to make a list of the XIDs that require fsyncing,
-	 * and then do the I/O afterwards.
-	 *
-	 * This approach creates a race condition: someone else could delete a
-	 * GXACT between the time we release TwoPhaseStateLock and the time we try
-	 * to open its state file.  We handle this by special-casing ENOENT
-	 * failures: if we see that, we verify that the GXACT is no longer valid,
-	 * and if so ignore the failure.
-	 */
 	if (max_prepared_xacts <= 0)
 		return;					/* nothing to do */
 
 	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_START();
 
-	xids = (TransactionId *) palloc(max_prepared_xacts * sizeof(TransactionId));
-	nxids = 0;
-
+	/*
+	 * We are expecting there to be zero GXACTs that need to be
+	 * copied to disk, so we perform all I/O while holding
+	 * TwoPhaseStateLock for simplicity. This prevents any new xacts
+	 * from preparing while this occurs, which shouldn't be a problem
+	 * since the presence of long-lived prepared xacts indicates the
+	 * transaction manager isn't active.
+	 *
+	 * It's also possible to move I/O out of the lock, but on
+	 * every error we should check whether somebody commited our
+	 * transaction in different backend. Let's leave this optimisation
+	 * for future, if somebody will spot that this place cause
+	 * bottleneck.
+	 *
+	 * Note that it isn't possible for there to be a GXACT with
+	 * a prepare_end_lsn set prior to the last checkpoint yet
+	 * is marked invalid, because of the efforts with delayChkpt.
+	 */
 	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
-
 	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
 	{
 		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
 		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 
 		if (gxact->valid &&
-			gxact->prepare_lsn <= redo_horizon)
-			xids[nxids++] = pgxact->xid;
-	}
-
-	LWLockRelease(TwoPhaseStateLock);
-
-	for (i = 0; i < nxids; i++)
-	{
-		TransactionId xid = xids[i];
-		int			fd;
-
-		TwoPhaseFilePath(path, xid);
-
-		fd = OpenTransientFile(path, O_RDWR | PG_BINARY, 0);
-		if (fd < 0)
+			!gxact->ondisk &&
+			gxact->prepare_end_lsn <= redo_horizon)
 		{
-			if (errno == ENOENT)
-			{
-				/* OK if gxact is no longer valid */
-				if (!TransactionIdIsPrepared(xid))
-					continue;
-				/* Restore errno in case it was changed */
-				errno = ENOENT;
-			}
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not open two-phase state file \"%s\": %m",
-							path)));
-		}
+			char	   *buf;
+			int 		len;
 
-		if (pg_fsync(fd) != 0)
-		{
-			CloseTransientFile(fd);
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not fsync two-phase state file \"%s\": %m",
-							path)));
+			XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, &len);
+			RecreateTwoPhaseFile(pgxact->xid, buf, len);
+			gxact->ondisk = true;
+			pfree(buf);
+			n++;
 		}
-
-		if (CloseTransientFile(fd) != 0)
-			ereport(ERROR,
-					(errcode_for_file_access(),
-					 errmsg("could not close two-phase state file \"%s\": %m",
-							path)));
 	}
-
-	pfree(xids);
+	LWLockRelease(TwoPhaseStateLock);
 
 	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_DONE();
+
+	if (log_checkpoints && n > 0)
+		ereport(LOG,
+				(errmsg("%u two-phase state files were written "
+						"for long-running prepared transactions",
+						n)));
 }
 
 /*
@@ -2029,17 +1964,11 @@ RecoverPreparedTransactions(void)
 
 			/*
 			 * Recreate its GXACT and dummy PGPROC
-			 *
-			 * Note: since we don't have the PREPARE record's WAL location at
-			 * hand, we leave prepare_lsn zeroes.  This means the GXACT will
-			 * be fsync'd on every future checkpoint.  We assume this
-			 * situation is infrequent enough that the performance cost is
-			 * negligible (especially since we know the state file has already
-			 * been fsynced).
 			 */
 			gxact = MarkAsPreparing(xid, hdr->gid,
 									hdr->prepared_at,
 									hdr->owner, hdr->database);
+			gxact->ondisk = true;
 			GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
 			MarkAsPrepared(gxact);
 
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index aa90503..c41baa0 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -321,8 +321,7 @@ static TimeLineID curFileTLI;
  * stored here.  The parallel leader advances its own copy, when necessary,
  * in WaitForParallelWorkersToFinish.
  */
-static XLogRecPtr ProcLastRecPtr = InvalidXLogRecPtr;
-
+XLogRecPtr	ProcLastRecPtr = InvalidXLogRecPtr;
 XLogRecPtr	XactLastRecEnd = InvalidXLogRecPtr;
 XLogRecPtr	XactLastCommitEnd = InvalidXLogRecPtr;
 
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 3de337a..ecd30ce 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -86,6 +86,7 @@ typedef enum
 	RECOVERY_TARGET_IMMEDIATE
 } RecoveryTargetType;
 
+extern XLogRecPtr ProcLastRecPtr;
 extern XLogRecPtr XactLastRecEnd;
 extern PGDLLIMPORT XLogRecPtr XactLastCommitEnd;
 
#26Andres Freund
andres@anarazel.de
In reply to: Simon Riggs (#25)
Re: Speedup twophase transactions

On January 11, 2016 8:57:58 PM GMT+01:00, Simon Riggs <simon@2ndQuadrant.com> wrote:

On 11 January 2016 at 18:43, Simon Riggs <simon@2ndquadrant.com> wrote:

It's clear there are various additional tuning opportunities, but the
objective of the current patch to improve performance is very, very
clearly
met, so I'm aiming to commit *this* patch soon.

Again, the WAL read routine used doesn't deal with timeline changes. So no, it's bit ready to be committed.

--- 
Please excuse brevity and formatting - I am writing this on my mobile phone.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#27Simon Riggs
simon@2ndQuadrant.com
In reply to: Andres Freund (#26)
Re: Speedup twophase transactions

On 11 January 2016 at 20:10, Andres Freund <andres@anarazel.de> wrote:

On January 11, 2016 8:57:58 PM GMT+01:00, Simon Riggs
<simon@2ndQuadrant.com> wrote:

On 11 January 2016 at 18:43, Simon Riggs <simon@2ndquadrant.com> wrote:

It's clear there are various additional tuning opportunities, but the
objective of the current patch to improve performance is very, very
clearly
met, so I'm aiming to commit *this* patch soon.

Again, the WAL read routine used doesn't deal with timeline changes.

Not relevant: The direct WAL read routine is never used during replay, so
your comment is not relevant since we don't change timelines on the master.

So no, it's bit ready to be committed.

I will update the comment on that function to explain its usage and its
limitations for future usage, to make that clearer.

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#28Andres Freund
andres@anarazel.de
In reply to: Simon Riggs (#27)
Re: Speedup twophase transactions

Hi,

On January 11, 2016 10:46:01 PM GMT+01:00, Simon Riggs <simon@2ndQuadrant.com> wrote:

On 11 January 2016 at 20:10, Andres Freund <andres@anarazel.de> wrote:

On January 11, 2016 8:57:58 PM GMT+01:00, Simon Riggs
<simon@2ndQuadrant.com> wrote:

On 11 January 2016 at 18:43, Simon Riggs <simon@2ndquadrant.com>

wrote:

It's clear there are various additional tuning opportunities, but

the

objective of the current patch to improve performance is very, very
clearly
met, so I'm aiming to commit *this* patch soon.

Again, the WAL read routine used doesn't deal with timeline changes.

Not relevant: The direct WAL read routine is never used during replay,
so
your comment is not relevant since we don't change timelines on the
master.

Hm, OK. But, isn't this actually a bad sign? Currently recovery of 2pc often already is a bigger bottleneck than the workload on the master, because replay has to execute the fsyncs implied by statefile re-creation serially, whereas on the master they'll usually be executed in parallel. So, if I understand correctly this patch would widen that gap?

Anyway, as evidenced here, review on a phone isn't efficient, and that's all i have access to right now. Please wait till at least tomorrow evening, so I can have a meaningful look.

Andres

--- 
Please excuse brevity and formatting - I am writing this on my mobile phone.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#29Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Jesper Pedersen (#15)
Re: Speedup twophase transactions

On 11 Jan 2016, at 21:40, Jesper Pedersen <jesper.pedersen@redhat.com> wrote:

I have done a run with the patch and it looks really great.

Attached is the TPS graph - with a 1pc run too - and the perf profile as a flame graph (28C/56T w/ 256Gb mem, 2 x RAID10 SSD).

Thanks for testing and especially for the flame graph. That is somewhat in between the cases that I have tested. On commodity server with dual Xeon (6C each) 2pc speed is about 80% of 1pc speed, but on 60C/120T system that patch didn’t make significant difference because main bottleneck changes from file access to locks on array of running global transactions.

How did you generated names for your PREPARE’s? One funny thing that I’ve spotted that tx rate increased when i was using incrementing counter as GID instead of random string.

And can you also share flame graph for 1pc workload?

On 11 Jan 2016, at 21:43, Simon Riggs <simon@2ndquadrant.com> wrote:

Have you measured lwlocking as a problem?

Yes. GXACT locks that wasn’t even in perf top 10 on dual Xeon moves to the first places when running on 60 core system. But Jesper’s flame graph on 24 core system shows different picture.

On 12 Jan 2016, at 01:24, Andres Freund <andres@anarazel.de> wrote:

Currently recovery of 2pc often already is a bigger bottleneck than the workload on the master, because replay has to execute the fsyncs implied by statefile re-creation serially, whereas on the master they'll usually be executed in parallel.

That’s interesting observation. Simon already pointed me to this problem in 2pc replay, but I didn’t thought that it is so slow. I’m now working on that.

Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#30Simon Riggs
simon@2ndQuadrant.com
In reply to: Andres Freund (#28)
Re: Speedup twophase transactions

On 11 January 2016 at 22:24, Andres Freund <andres@anarazel.de> wrote:

Please wait till at least tomorrow evening, so I can have a meaningful
look.

No problem, make sure to look at 2pc_optimize.v4.patch

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#31Simon Riggs
simon@2ndQuadrant.com
In reply to: Stas Kelvich (#29)
1 attachment(s)
Re: Speedup twophase transactions

On 11 January 2016 at 23:11, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

On 11 Jan 2016, at 21:43, Simon Riggs <simon@2ndquadrant.com> wrote:

Have you measured lwlocking as a problem?

Yes. GXACT locks that wasn’t even in perf top 10 on dual Xeon moves to the
first places when running on 60 core system. But Jesper’s flame graph on 24
core system shows different picture.

I think we can use a shmem hash table to identify the GID by name during
LockGxact and avoid duplicates during prepare. Hashing on the first 16
bytes of the GID should be sufficient in most cases; the worst case would
be the same as it is now, all depending on how people use the GID name
field. The hash value can be calculated outside of the lock. We can also
partition the lock without risk, just adds a little extra code.

We can also optimize final removal (sketch of how to do that attached).

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Attachments:

2pc_remove_prepXacts.v1.patchapplication/octet-stream; name=2pc_remove_prepXacts.v1.patchDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 2251b02..e196a55 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -114,6 +114,7 @@ int			max_prepared_xacts = 0;
 typedef struct GlobalTransactionData
 {
 	GlobalTransaction next;		/* list link for free list */
+	int			prepXactId		/* array offset within prepXacts */
 	int			pgprocno;		/* ID of associated dummy PGPROC */
 	BackendId	dummyBackendId; /* similar to backend id for backends */
 	TimestampTz prepared_at;	/* time of preparation */
@@ -407,7 +408,8 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 
 	/* And insert it into the active array */
 	Assert(TwoPhaseState->numPrepXacts < max_prepared_xacts);
-	TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts++] = gxact;
+	TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts] = gxact;
+	gxact->prepXactId = TwoPhaseState->numPrepXacts++;
 
 	/*
 	 * Remember that we have this GlobalTransaction entry locked for us. If we
@@ -551,31 +553,25 @@ LockGXact(const char *gid, Oid user)
 static void
 RemoveGXact(GlobalTransaction gxact)
 {
-	int			i;
-
-	LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+	int			i = gxact->prepXactId;
+	GlobalTransaction	last_gxact;
 
-	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
-	{
-		if (gxact == TwoPhaseState->prepXacts[i])
-		{
-			/* remove from the active array */
-			TwoPhaseState->numPrepXacts--;
-			TwoPhaseState->prepXacts[i] = TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts];
+	Assert(gxact->prepXactId >= 0);
 
-			/* and put it back in the freelist */
-			gxact->next = TwoPhaseState->freeGXacts;
-			TwoPhaseState->freeGXacts = gxact;
+	LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
 
-			LWLockRelease(TwoPhaseStateLock);
+	/* remove from the active array */
+	TwoPhaseState->numPrepXacts--;
+	last_gxact = TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts];
+	TwoPhaseState->prepXacts[i] = last_gxact;
+	last_gxact->prepXactId = i;
+	gxact->prepXactId = -1;
 
-			return;
-		}
-	}
+	/* and put it back in the freelist */
+	gxact->next = TwoPhaseState->freeGXacts;
+	TwoPhaseState->freeGXacts = gxact;
 
 	LWLockRelease(TwoPhaseStateLock);
-
-	elog(ERROR, "failed to find %p in GlobalTransaction array", gxact);
 }
 
 /*
#32Michael Paquier
michael.paquier@gmail.com
In reply to: Simon Riggs (#25)
Re: Speedup twophase transactions

On Tue, Jan 12, 2016 at 4:57 AM, Simon Riggs <simon@2ndquadrant.com> wrote:

Performance tests for me show that the patch is effective; my results match
Jesper's roughly in relative numbers.

My robustness review is that the approach and implementation are safe.

It's clear there are various additional tuning opportunities, but the
objective of the current patch to improve performance is very, very clearly
met, so I'm aiming to commit *this* patch soon.

-       /* initialize LSN to 0 (start of WAL) */
-       gxact->prepare_lsn = 0;
+       /* initialize LSN to InvalidXLogRecPtr */
+       gxact->prepare_start_lsn = InvalidXLogRecPtr;
+       gxact->prepare_end_lsn = InvalidXLogRecPtr;

I think that it would be better to explicitly initialize gxact->ondisk
to false here.

+       xlogreader = XLogReaderAllocate(&logical_read_local_xlog_page, NULL);
+       if (!xlogreader)
+               ereport(ERROR,
+                               (errcode(ERRCODE_OUT_OF_MEMORY),
+                                errmsg("out of memory"),
+                                errdetail("Failed while allocating an
XLog reading processor.")));
Depending on something that is part of logical decoding to decode WAL
is not a good idea. If you want to move on with this approach, you
should have a dedicated function. Even better, it would be nice to
come up with a generic function used by both 2PC and logical decoding.
-- 
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#33Michael Paquier
michael.paquier@gmail.com
In reply to: Michael Paquier (#32)
Re: Speedup twophase transactions

On Tue, Jan 12, 2016 at 3:35 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:

On Tue, Jan 12, 2016 at 4:57 AM, Simon Riggs <simon@2ndquadrant.com> wrote:

Performance tests for me show that the patch is effective; my results match
Jesper's roughly in relative numbers.

My robustness review is that the approach and implementation are safe.

It's clear there are various additional tuning opportunities, but the
objective of the current patch to improve performance is very, very clearly
met, so I'm aiming to commit *this* patch soon.

-       /* initialize LSN to 0 (start of WAL) */
-       gxact->prepare_lsn = 0;
+       /* initialize LSN to InvalidXLogRecPtr */
+       gxact->prepare_start_lsn = InvalidXLogRecPtr;
+       gxact->prepare_end_lsn = InvalidXLogRecPtr;

I think that it would be better to explicitly initialize gxact->ondisk
to false here.

+       xlogreader = XLogReaderAllocate(&logical_read_local_xlog_page, NULL);
+       if (!xlogreader)
+               ereport(ERROR,
+                               (errcode(ERRCODE_OUT_OF_MEMORY),
+                                errmsg("out of memory"),
+                                errdetail("Failed while allocating an
XLog reading processor.")));
Depending on something that is part of logical decoding to decode WAL
is not a good idea. If you want to move on with this approach, you
should have a dedicated function. Even better, it would be nice to
come up with a generic function used by both 2PC and logical decoding.
+       if (log_checkpoints && n > 0)
+               ereport(LOG,
+                               (errmsg("%u two-phase state files were written "
+                                               "for long-running
prepared transactions",
+                                               n)));
This would be better as an independent change. That looks useful for
debugging, and I guess that's why you added it.
-- 
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#34Simon Riggs
simon@2ndQuadrant.com
In reply to: Michael Paquier (#32)
Re: Speedup twophase transactions

On 12 January 2016 at 06:35, Michael Paquier <michael.paquier@gmail.com>
wrote:

On Tue, Jan 12, 2016 at 4:57 AM, Simon Riggs <simon@2ndquadrant.com>
wrote:

Performance tests for me show that the patch is effective; my results

match

Jesper's roughly in relative numbers.

My robustness review is that the approach and implementation are safe.

It's clear there are various additional tuning opportunities, but the
objective of the current patch to improve performance is very, very

clearly

met, so I'm aiming to commit *this* patch soon.

-       /* initialize LSN to 0 (start of WAL) */
-       gxact->prepare_lsn = 0;
+       /* initialize LSN to InvalidXLogRecPtr */
+       gxact->prepare_start_lsn = InvalidXLogRecPtr;
+       gxact->prepare_end_lsn = InvalidXLogRecPtr;

OK

I think that it would be better to explicitly initialize gxact->ondisk
to false here.

+       xlogreader = XLogReaderAllocate(&logical_read_local_xlog_page,
NULL);
+       if (!xlogreader)
+               ereport(ERROR,
+                               (errcode(ERRCODE_OUT_OF_MEMORY),
+                                errmsg("out of memory"),
+                                errdetail("Failed while allocating an
XLog reading processor.")));
Depending on something that is part of logical decoding to decode WAL
is not a good idea.

Well, if you put it like that, it sounds wrong, clearly; that's not how I
saw it, when reviewed.

I think any fuss can be avoided simply by renaming
logical_read_local_xlog_page() to read_local_xlog_page()

If you want to move on with this approach, you
should have a dedicated function.

The code is exactly what we need, apart from the point that the LSN is
always known flushed by the time we execute it, for 2PC.

Even better, it would be nice to
come up with a generic function used by both 2PC and logical decoding.

Surely that is exactly what has been done?

A specific function could have been written, which would simply have
duplicated about 160 lines of code. Reusing the existing code makes the
code generic. So lets just rename the function, as mentioned above.

Should we just move the code somewhere just to imply it is generic? Seems
pointless refactoring to me.

The code is clearly due for refactoring once we can share elog with client
programs, as described in comments on the functions.

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#35Simon Riggs
simon@2ndQuadrant.com
In reply to: Michael Paquier (#33)
Re: Speedup twophase transactions

On 12 January 2016 at 06:41, Michael Paquier <michael.paquier@gmail.com>
wrote:

+       if (log_checkpoints && n > 0)
+               ereport(LOG,
+                               (errmsg("%u two-phase state files were
written "
+                                               "for long-running
prepared transactions",
+                                               n)));
This would be better as an independent change. That looks useful for
debugging, and I guess that's why you added it.

The typical case is that no LOG message would be written at all, since that
only happens minutes after a prepared transaction is created and then not
terminated. Restarting a transaction manager likely won't take that long,
so it implies a crash or emergency shutdown of the transaction manager.

I think it is sensible and useful to be notified of this as a condition the
operator would wish to know about. The message doesn't recur every
checkpoint, it occurs only once at the point the files are created, so its
not log spam either.

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#36Michael Paquier
michael.paquier@gmail.com
In reply to: Simon Riggs (#35)
Re: Speedup twophase transactions

On Tue, Jan 12, 2016 at 5:26 PM, Simon Riggs <simon@2ndquadrant.com> wrote:

On 12 January 2016 at 06:41, Michael Paquier <michael.paquier@gmail.com>
wrote:

+       if (log_checkpoints && n > 0)
+               ereport(LOG,
+                               (errmsg("%u two-phase state files were
written "
+                                               "for long-running
prepared transactions",
+                                               n)));
This would be better as an independent change. That looks useful for
debugging, and I guess that's why you added it.

The typical case is that no LOG message would be written at all, since that
only happens minutes after a prepared transaction is created and then not
terminated. Restarting a transaction manager likely won't take that long, so
it implies a crash or emergency shutdown of the transaction manager.

Thanks for the detailed explanation.

I think it is sensible and useful to be notified of this as a condition the
operator would wish to know about. The message doesn't recur every
checkpoint, it occurs only once at the point the files are created, so its
not log spam either.

Well, I am not saying that this is bad, quite the contrary actually.
It is just that this seems unrelated to this patch and would still be
useful even now with CheckPointTwoPhase.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#37Michael Paquier
michael.paquier@gmail.com
In reply to: Simon Riggs (#34)
Re: Speedup twophase transactions

On Tue, Jan 12, 2016 at 5:21 PM, Simon Riggs <simon@2ndquadrant.com> wrote:

Should we just move the code somewhere just to imply it is generic? Seems
pointless refactoring to me.

Er, why not xlogutils.c? Having the 2PC code depending directly on
something that is within logicalfuncs.c is weird.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#38Jesper Pedersen
jesper.pedersen@redhat.com
In reply to: Stas Kelvich (#29)
2 attachment(s)
Re: Speedup twophase transactions

On 01/11/2016 06:11 PM, Stas Kelvich wrote:

On 11 Jan 2016, at 21:40, Jesper Pedersen <jesper.pedersen@redhat.com> wrote:
I have done a run with the patch and it looks really great.

Attached is the TPS graph - with a 1pc run too - and the perf profile as a flame graph (28C/56T w/ 256Gb mem, 2 x RAID10 SSD).

Thanks for testing and especially for the flame graph. That is somewhat in between the cases that I have tested. On commodity server with dual Xeon (6C each) 2pc speed is about 80% of 1pc speed, but on 60C/120T system that patch didn’t make significant difference because main bottleneck changes from file access to locks on array of running global transactions.

How did you generated names for your PREPARE’s? One funny thing that I’ve spotted that tx rate increased when i was using incrementing counter as GID instead of random string.

I'm using https://github.com/jesperpedersen/postgres/tree/pgbench_xa -
so just the client_id.

The strcmp() in MarkAsPreparing() is under the exclusive lock, so maybe
that is what you are seeing, as shorter gid's are faster.

And can you also share flame graph for 1pc workload?

Attached with a new 2pc, as the server runs Linux 4.4.0 now, both using
-F 497 over a 6 min run.

Best regards,
Jesper

Attachments:

1pc.svg.gzapplication/gzip; name=1pc.svg.gzDownload
���V1pc.svg�\mw����,��	�iH1x�$:�����N�c�I����C0(�i����H�$�M���&pg��g��u��j���A�%��I3/ya�������o�z�����}������/���%�e���~����������
��^� i�e��^o�\*K]��I����O?�a	aP&�Te#	X��9�D��(���5���`2���m��0�F})������D�|���JT�C���Ei�A �PQ���j�}!u]�����_]^�7�7cd�+B��{`�.�0���@z��1Fp�f��t1T�x�&p�&	�Lz|�
�		���Y�L`HE��P>��sRe���$l���!Kry	���8%���n@x�%�l�(# 7�'I��FYS@
��k�C|Zi�/~v�f����K?��/����`#������_K���v�nh����WD�����:d$[�A3��z~��g�x����4��?�Q���a���?]r��R����$���yDK��`��F~��*��<��o���W/>��t~��%d�2/S�� �?f�I3/��l��2
����C �f��K>���(�3X�2a����?_���:RN"u�q����4G�8�d�o� �� ��F�}#r*1���6}���M~e`-�O���R� �a*<�r~~���Y�H�%�T� �P�N�@r���4^�LH�LY<�]4��|��.�Io����D.8�}���p��A�G&��|������
Y,��lD�x�x�����]�f,bIG�gkxIr���K�����
��>���J
���S��?lM��GR'���}��y���+6�a�� �H�I<���`�_�B�B����D��?����3�
��e�`��)�5�!����,E.�F	��zM���^	Y4�����E.9.�x�)��dBc`}�qi��-� _�	���Q�|K��� .�~�~V��]U�r�I�4 N�� �X'_�L`��UXHN.]p^[�������=��s�=Bv�+�qL��x��	'V��V����Z�cr?&�������'l������=��%~~-��IY����7�|}�Q/�m����\�s������W��"������o��uzDy����,�#��"���e&�kL����8
 ��������:Im�x�����K��v���6�X	��\�_;����������/7��ZI-yT��F�.�9w�w3/!ZdS/�AJ��7F�q�MBf��3�3��,�
��Q��w7w?YUl�w���"�m��B�7�(�	.��������<�����Y�t���y�����;�;��S����.�u9�Nu���^j�d�������Rc���`�>L/����)�b���0d�IQ0�84_������8�U�?���6���E��j�
�A��S��E�����G��Z*�������E6!���IV��Y��J&	����x��
4��V�������:3{�
L���U���t��b�`�*a:�,�q�U���|.���8��Z5�;�#@���65�Vw9{Kjj���2jh�%UEzL�������\������-,J��T�^��}���>�������.��}�(iH^yz[3��b&�
e�������� _����z�@����|���X���p�3��"�~�&�%���pz��&����/W��8��x��WZ��=xw�#[��6�3VDT
"QMPGTEUU�O�/��.DB�
o|�������*���pA<�V��:�aa�@�����:�K�)�%�[:"�o������j�C��D�oS�s+����&N
cW�����9�s����x��9�!a�����~S^��+�_�NM`�V�q���>^��W����������T2O�����G~�E�o�!�	Jr�0{�^���~��K��(����H� ��M��mI:
RH�sI�8b�(���(�g�z��V�����f��J&h3Hf��Enf�����*<���&�Q
x{�l��]g\�FY`�����/��&���}�)4�<c�^�\#��-APM��"\���Q�L��E��D�O1�+X��M���9U�,�b+���#i�������X�}U�!�)E� �����k������m'a��cG�O�:4^��L����
3]�,�EA�c��@��9T��[jf�	��t�6��J����7yC��A�;�F�Mx�:W��UzlG��7-
�� }`����x{��`�\����8�;hW��vU�Mj]����un7���~��i���~K��A�L�� ]��VR�f*����������^�qfS&�����Y&�|�F;D�����a"l���s��1�S0�l��Q��������a�(�K�����Y��(�m���;(pE����pM��!��K���)x$�!V�XY�v�x�)�J����j#d�(�R�U�E��Sw+�1�]q'��fu@���R!i������h�����������������+	�Q��i6d���{����n��!���8�N�*��`��>�$�@C�S$P�������K��
wHw�����T�t���f�����C��s�z��B�g�d�s��MBxX��b�����(��Co��&*��pL��I��rI�O�U�{�FW�"I�ugC���s+��D�z����}����s�z��)�E:�'E/
c}�d���KVNb�OTgH8�����r\��Q^�B��?>�Y���2��d2eO���.�v��k���@���f��A�;���������n��������r2-�6��O������q�����<'<������bN�a��K��	�3���,as���2�|�q(?���;�������T�-��e%Hm�|����&�����/7�l���P��l��
��(-���������/���Rwr�8?�gv8���*y�[H�t�����E�"�b�1)j#�~*Y�&X�Q�v��Jc�$���� ��������E���L��Zo;S=�{U��qI"��>���*���c���{��^G�n�R�]�1���Ev���(��^���$��R$ }i�F!�����	.�A���`e���,��O,y�Wp���AME>�����2id�Y�����uV����2Lq���L����g^x��.t�x�
��_l�p���cAE���8,����9�d2�{h$�Cn0���EV�;�V��U�5,��.WD�6K�Wu��S��	��A������:~m��^0����_.�c�|���X����\����C���dl~j_wA>�4hj�����;fi���m,������f�5dj�`	������l�Y�RGprM��CK��h�&��3��
f�(�3o2�1�]�6�i;N�iT��{��-�RYk��R�����������d���W��T�t\zX3u�*:_���#5S�dj��Vp����9'J�:�b.�
D�;wd��U8u�N��[,���i���C��l�m�4K�5��}�p��p�No=p��c�U0����T�!��7��8����l�RM�E����O�	^o�d�P������~�sj��k�f�GzN
{����5�-�S��<�� 
�� ���A� ���N�p\�Fi�����/��e�f+@�l]n��G���	K��������1[�������""Y��J<����
S���gN������������E>=����V!5�H],_������E�
����
J.uL����>uD����,d^�(=��T��;y�>j��k��na����'�����
�Q�i0����e�Z5@�@
��5S����^�te���
�����)�G4�}L� ��D[=�-H]r���#��i���2����"�"�~�F�z�����,������0�!�cYJ:�E�x�p�m�d�zK�L�8���G��h��E�<��T�P����6E"J-E�����F���|�|�U�{*��|���8"if����q�$��G���"�9���rHr�8��A�A��xKM;��a�����c�%X���L�
�*:�}��2�&���B�c��1����E�h@���*�f�JZ9����L!
Y�_<�]�`�q�Q�B���m@�
*i��4.�=O.�'o������$~����;�y��i������cS�]�M�h�N�������`�)k�v�3g�E#I������nCM��{�����O���$���~���y���9i8��.<`��������������4�T(=��q���,����6H���q�tK�]�N�<�vA�s�&�g�������C
����Ior��Q���r��q�&O9u���tlESOE9Y����@�c9��\Iw���iZ���;�H�vg�
��>����m�1��/�s;��q8��)m8d;��3eS�u����q�glG �@���i��5���t �w�Sia	�w�A�(�L[S����f����n���'�������V�i��R����m�AKmY�c�c��P~��?�,y;�z�p�1m�8�9�7(�b���	�g���:P@��a�t���`{��:2�\�h�&!�t��9��X��x�|0!�����%9U�&5��uU��R6[����Q,;��=ye��ri���vL��b�v�����LU���Qu�h�Tn��&���G���X��b)^�s��Z�>��ih���Z��G��Jcsop��'k�Q}���~��(l�~��OGujMnw���9�34l����� ���6�>��4l�u�U)��z�5O���{6�����y���:E�����g��c�=��N�o3�'�4��EUY3ki��t�k�z�V�\��tH���	������o�:���s�M%$+������ ��4o���
A�>�C��q�y��E�=X{�^���A�i�%z=���������W@�nX��lb�9��9Dr���z��t���vq�V`��|�uP+w
�r��T����c��.L��e-�,o����Tn�)MCq�S��l<|����������������n�]�}��Z�n��c����'�)�q}���=��mZ�=zS�����=���	�M�V����";:����^�d��2��.�f-�Xf��������L��{R�A)	��h��m�E-y�f�xQ�__y
)'�i��^�p�)�����f�����k������E!��N���h���P\BH��q�gjf����>�����}djN�������`5@s`E�)�d���a�Q��35c������*9�a:����z���^����"c���������*^�myZ�7d4C��5�TMS�~*�n���������iy�������-"��������U�h��v�5�
��]���;������k+�}*���x��6���V�H�!�4eN��(��W\��&m�MPM�8�/l[;=|0l�z����u�����l��J*03���?y��_�xY�f^��UC��q�u��:��{�{-���e���68O��$�|���d��w����A�)N4���M�U���k�q�����~(�_3�d6@f6k;,�BY�nk�[J��=�~y�b�JM�uf������u���}5���0>U0q��]>B-5�'���)�G�?=R� 	�b&�fL~B��	�N�)]�{�q��V�1������n<h��J���KD���>�;U��
�5=vdWZ�Zv�|	���s�m��{���z#
o�)��������o&	�&qK�c�d<��T
�Y�������;�" ����i�:���@jD���x\��������/W���X>�k���
t�S�����N��R�$��N�5����w�$W�����I�a�{@�x%i��������
c�J��@����y�@�Q �����b3hW��$���\�
;��.�[U��?�dTo���sw���'dS+vye 5T@e���K]J�e����P�v6"��p�=��0�,`�4fvV��+(5���U��t��(P�&���RXWp?4X;���8�Y(�6!	M�Ix��by���_[��.�x�6hD��v�9����n��E��2����#A�2����;A.�f(�q�!fs�le(R��i���������(����"i��C�����|=j}������R�P��ZWy�3W�zF�3h���i��U3���06jB.�Pn���3��������kw�VW6/
i��
$$�KR���]yNL)s���>�_�����f�X)����E���q:(��T�o�G�|��C)fh�B���|�*�*�����}
l���M�g�����e�z�����l�2`�,��f��Z]��%��\��Z�
6�",3�
B2�������a^��f{��7��UkG������op/:!PG�Z��
�������3m7�e����9r��Z��(f
�6������H3]s��`�t�3����Er��+E��a��-��������5-�*DJ9���"�3�%��e {"ct��P~���������nv; �^g����S������S*�a�j�Q�Fi�V�_�P�{%����_{#�2��^�K+,���_��A��n�1���vZ����0�!�Q\cL*e�r)�����I������5�Zf�[
)@3�>����t����
���/]�W@~�������bH�%U���qQ%����u����v�����4C����d����dK�qP�� z�Y)���~���7�������Q�{��K@���lex)�#�����
��[_i��?
��"��T�!WD��TT�4-fs�����j���W��x5�j��\��dRH�g�Ir�e)���>N���?������H��2�Ng��@e�.�0�+j�MI���Ib1mFT������:�GH]b�z%Ky9a��']�_7��������tx<�'��U�\�����3#�"�;���
�eYJ�	����u��]Z�R��	a=�4�TGL����~}��W�����&������r���S�<��=�fo�\j��/B�^�bvm�����O��{���_t��/f���W���(9�.�Q�d�a�����=������Pka0"e��	<,���iE46��+7Z�Mc�������B���V��]�W����}F��KO*'7�	�����g�T�%����`���=�����t*�7nn�1X�b�!��b�-�9z�t<��+��}��s��{wk�|���~5�K���
@ �>��i)C���(�<��W�f���w��"T��&(���o'���
"Y�:���k`�@�Q-y��E��
�[����p1�������53�wJC��A����\Q1�'/�T���]sxj}�>��r���S1R�4�*��1J`���I��9|���������[�l�����4^H����c!U�Ey�)���{@^xE��3����3���y����5�8"�Z�R��z��i�=���h:J�2�a�W ��K�F����C}n��v�n�'�&����A������k&������	d|����
�{�Ta
����(�4!�d��C_`�y9��+]t����$Gg��I�8��{���M�O�$�����$����Q����$��^]�GH����5����u�Nx��������,��;9!k�t&���_1�8�5�j�o���H�'�����;�����L�d�'���P6�',�~�V��i�aC��Z��3e�Ep��K��[|D���������
����I4eL:RH��I�.!+�f��IB��'�OF����)a�1�&%KN^Q�����'Z>8)�psYI�I� �H#�dR"��3����v��Y>fx�"��U�}u�daj-�9@%�Q��-�E��=/����/���h�f�	�n��>����8���+�������c�	�+�3���F����d��
��w�Y����z(���������a�\_Z	��TJ�:��7�������{�9wB���}8^�+�u��_V�D��7f$[1U�D��j��}����u���]}����@na�@� `���s/�X��f�b��C�=l���?��S�[�����H��U��R2I�@�����+big�e��<6��')n:�SCr�f�@88��C���QCX�q�!��
���m�B�^���{?X�(&��FA���m�M����c��&���RNO/�Z�Y�[���f�f�$w�x*o��K=���������o8�f��7�� �mg��tT�3K
Z�\�0�,rj�F�/1�
�"KL~u{����Rj���s����{x(g���X���
"����,b���;�HV�+������i]�T�6�@'�A���bH�N��\>I�Q	.���y%i�7H+��0�k�YYY�#	��g���u9.Q������R
w�P���X����k������1������g��I���X#rx�`x�oN�>�VB`Gg���1x[L�r���^�$��%uQ��.����0/=t����m�������=x��1������c�����(9��7f�{a.��l��Q��������b�y1�b
3G�-H����#Y���~�vgm�5Q�e2��,*���SB��de�k~�]�������{�]f��I�K�$~��"�]�D�yC]��I���n��n�F_I���Ah��	�����rj;��w����e�3p�t����/��;����2��A��}t[��}[����f��3s��d�\Rd�n�.���x���J��jJ���d���+���Pr��P�i�:�1�|H##�$��t,�������w���@t���<�]���t�ncNx�0���Yl�]���N8���=����/�
d��b)4�6*���;�P�f��8������$E�~a�9�W�^��Jz�����'suH2�8te�	�t��������_6���;w�����(H�g|Jb�g�a�^*��QI�^��	]��w���Y��f��d��!�o�D��y&U�.`�vC������\�:�S���t�yxPg�	�47��!J	�������8Rk����}H�kB���P������[;��B��N>�
3�)p;)Ic��	�� \fh���S�&�>�f�w�A��rX�a��>�8R���O�f�n�I���+�=�LAs�g+%�J���I�b��6�&�����.����N7���l������2Y�C�e��+v�Q�������2_�����j/Gv1B��Y������fw<��xZ�X�����;�������kl��X�������X�$��t&W�n�o�]�~��z���y5? ��F"I�23$�
�0��JC�\?�u
�G��}��`��O��s�!u$��R �{���L2��w�%&M�H������R��/�
TN�K�y���X�U��>�IT"�,��w����Ko���9�U���	�(���Y�����(��K����rg]�^��/�+A�o��?�ykf8a����3d�b�~U�b
;s��Q���n�?��s+����1�������w���)q��������4>�Xx�w���c���F�b9(��wA����r�XY��l�������H��F�F�1&b]Bi�Hr'�jp�L���q2x�V�tK'�B�_B��$����t)��A0�E+��2��wF�K@2M�:����E��(�~���((�]%+�������Sf��l0�}A�Hy���N�R$We)Oh"[��g0B8��4��E�e�p!,F�f!���r�'�BHt�98A^A�'}����~"��.����9P��VG1E^J��?��L�EKVa��>����+��}x���g��'�9C$�xP���a��K*�`�L�"?������ok�?�����w��l�D	���WnB����&��h�K�k�9�yY:#�w�D@J9��G�N�{G����AJ�������;�Sa>���:������2��������:e��SH���F���K��^E�{]o��d?�kp;�������]�}�����k""����m��U�������#�a�U�C���g���!!Z<
�[��N�6� �`�7v�o���LM�itI�eH�7d�\m�BG�&�H�.f~�Jj��9���yB�$"
aH���o8QG�%#�L?��9���'�_e�i��{������������y�*)�D ���b8�u�����K���UD�Q���<W$��Q���+�Vr�������s��������9�<�����2k��:n���J9�wd��e���Q�����K!���R�q�����Ss���7�?<���H�+"0��1���H�k����p����.���.h�gRV�����bN������(��5�����6]�������)��A������~Z�[$�����O]�:"�JL�
(�<��KN�/99�Q��\��Re�S��q]������c�������1	���E�KC ��H)q�������n��*o�*�7mU��Dt��8����vuf1����v��r������H�'��a��'�+^�*���|�x������E�c���8�$�9Z�b�z]7�D�����.qD���cLpX�
�u�A�x������_�����M�%x������LTK!X�M�)DrH+���!�V��Y��~�?%������\�������3��M��s��W���%s�+!�4B?|�]9t�t�s��f<�*�S�B	�S���{<5)��P�u���%���E��x���!��Z�bv@��pu�S���x��-T(��	Y��@f$n<$���)����=����������J]�&$�#%)����3���0	������
4a�j�L'�\��z0��p���X|�]�Z1��F��.�����rw����\��aR'M�K�����]P�{���J�I)Oc�b�U�DJ�%� �����I��L��p�����3p],H�L�3-��qa+���.>���|F��pGt�)�o���)�r
���;Np�����R0=4}�;>yF���O���W��w���u����:���PsG��'�p	J���-�W0�9_��F�7�����4LX\oU��O+^����?�3�P�tb�3�TV �L�t� �����>������������S��B�F���^$P��	�����x$'s_|�O!a�<q�I������d��P�
��b72?���o�����
�
Z�}c�N+���0q"����C%��@�=��+[�������q�s;W�0d������0���(%�^i���.H-�7M�p��2bF����Y�^	[�2����nv��Y�����+��:l^YE(1���L��"��	�J�}G���?�B���y��}���EY.L��0y�5�(T#_X��,�s	J���?�R;��F�������	��D����o8a��p<=�wA�0�(y�����P!����Q���z�3n%���r.?��S��@z~�V��#��b���)]��77reX)3����C����T��|Ltd����M�y�S)`V�����e����M�WH��=
������J`�!cD�PsN�J��SF���\O�-�;c7
�,R�v�f�����V��������m�g�o�~�N�����2�j�yB-���-�[��A�]?��3:;#q��p{Nn���J�I�xg���?�NM�����$�&'/�I"�)I}sa��dAA�����F) �7����a�<UC���i�M� (R���ek��+.�YG��yK
�?�yX����qJA��"P�O$���2������l�����SG*�DU�"���a�.��qHa��McGB2[�[�!�i,a+VL������~<v��e�Q�����	(A"M�L�����jOTRb$ep�'���E����p.'/eR6�%Z��r�H�D���QS���}f&��}s��2'���S�W�z� BW�l�q����I`A�-�3*������O{���=�-O�����W�?I:��l�!�����Q)3�}Fs�QYn�.�E%�c��<wkE63,���a���d�B���r7bX����D�v/r�<K+6G�����y�����5*�����\>�k4���������RR��Yd���>+��-��������d?i&�d*��X��
S(&F���?���	8���e '��t��������d�W��B`w�����5n�����
g��F�I���_�Zl���a�nC����'�ivU�::J�K���i��g����N�����������zH�Qb�:�I�g�xYHGeB[d�\�x���-L�7P����v��T���-]��04K�p�$���Y��8t�`k��G$�>�]s
N�QL�h���B�B�Pn�����#�+�z���
�3y��M�/��0�3�[G
3#]N��J�\����?��uM��k����H]����!��4[�p��)1b�
!�C��������]������m�|9��]�%-7�h���&�+�sYCq�����6�K��4s�gs?b>$�orLV2x6������z��Y�%������zF���E="�)m��������-�0@.�)�RD?k<GS(9Y�X]���gBw6�b����H�W���uO�_1,s�5�Q����1�g�t���IG�1�Y��@~=�Z=���]���{^��N�T��N	hK���D>�����E%t9�l`��6��|=EO.
s�OxE9�PI�T�k^��������C��~�M}{�p���Ko��\G�M��x�ie�<�B�c����7�Y��4m-�9�=�A�)m�U|3<��Q<�w��}�+ci�DfBb/K`��2����Y�N�����@��`���?j�����w��N��A�D�B$t�:!��'�������/�I��������D'T��-����mp��Xo1sA�0Q��g�,�&������z��A7��w���+z����Q�E�%���<�
�����B��9�+D����~���I[>{Mf@�k����[?�9���p������a��G��$��wZ������]�8�����Tw��i�	 ]|$|��2>�f'���)����dfU��I��q��4e h/8�G��C���*�l��ow���0�q�/�2��	!��%�?�1*�V��dfV�:��� _F��)�����xK��!��t�1�����
?�4����-���0D2j�=r\g\�Y����}����q_�?cLI��k ���X�����$5�����������1�������S�(�DN�F�j�x���z�{����u��-0�*�4�e��9j�=�J�6�y(!	FrR�f� -#�^�=�l9��P��(�.���Q������,�v���P���$Y��`b��3���n��vsN�n�����A����;9=��`��������g�it_��cg"�cg�����=��Kv�n?����O��������]_!�Y5_��@v\AL��Q�4��:�=o���;Gs���z#�a
T�����zN�����4��
��-/9�@�sB/X�(�M+�8zP��$��/���f|)�V�lfaY@�(�6����V�l6�I�0��_~rG�W{�0����x�KE������ �&�����Bi�'������$q�s��](�R���$�`6;����o���W�xJ�������d��������vt����|m&b �<�X��6�%0��+��
�����������TX�c�j`pMS`f��E��.��`.�A
�K�N���2��0����iBO����\p4[�������n�������n�{�{�����nw�G��tk
cX��x<�Rm���c�zfS�FB�C�'`������q�gt��{cp�D:MY���4U����k�}
��Lg�3&����8�Y���.$m7������l�i�n��T���'����~@�N)�2l�b���Dd
M�)�������.�42<�U���9MsOR~K�xz=����36�����
�x(�^4H�dr6:W\�?���b��a�>��Z!��8�{q�\����)�8g��\`W��{��phvw�q��rl�C��i��~�1��E�fW���4�9�Cos����-���S_BA������x7I�v-YF
������������ww�D.N��D����$4]�y��o�g#
|%*K��`!q	�9��<�y	V�<���^�Z�����tf�_��f����S�(+�w!��L���P7��JSM
!dz�_?��b�q��o5F���7;�Z��u��@�u6X|>��M
N��2�7���L<"���	�����#���>d)�F��K�Z�#�}FAK�K%��9��b���u���V�e�7I��i��YX���m���!��{��J�x���������FRW����0���O ����;p}T�)�+�z*5�GO������h���v���J:l��n�?�P4���/��0�O���N�wN$$�G����R����H}\Q�Tu���[4�R�e��d\�q���
r���T
��4���e�	&�5�a</��#@��%�^�+���G
2�I����<��Ap}���z�6��S�AY����������/-��b>�:�lhz8�p<F����D�ec���D�b�
���h�d����k����IU�M�� B]�)����0�02�e���`e`�d������\�*����&�����re.]=�V����6��m�#`�DZ�����|F>��#�����8��������a��x"Y�]9���	�	oYQ�C�v�D�3�������5�u#�n�Y/�����M��c2���=u�%�ci3n�\h4����(���������^}G�i�T���w�M��y��T������h���+��E|Um7�k���;������Q|9��g3���_�6�����??<t}����]_y��SF+fl(95=��O��ID8I"'�tA�������m6���P-��0�)������B�@���|E�i�`/&��}9u�%�����1B��������:��Z�CS�8q#�����'n�Q\�rGA��/�)�r�/Y��n%�f�c��dj�c?��>4�] s��������&�����@V�4���t����y������;,��+��h�
G���Y��(��&#�Sd'���������(/b�b!�F����5�V�k��������?
/�E�S�IA�Ni�D���.$�Sb���H��v[}�����K��)��� �~�s5]`�U��IR�E���w�4��
*�Y�f=���Y?VF���W*3D��#����M����i��;��J8r�Y}��&!(B4�9)��-s�19�������D��YIP������,�4EA$z*�	C�{$��}6�U�~�l�X����|�$F���������2`"F�5��P`DK�C�w@R"H,i7������������fW}*s�i��M�����e���1�������4�1�$9:�_��L8��sa�z����z��5����8��b�8�UU�BZ���+]���`%����q������*�q.��<|����{��v�{��	?\N
��#2�k�aE�����		z'](�%�6H��'��������e�0F�&e@�hANe���DF��+�G�y����}�YI�9}�(_`3k��9���<R�h����j�K��K���d��������{,H%Oo7/���Y`^z;
R\�\6���D;�C�������H������!_�%�V�?���'�����HXX &*Y��M�i�,��D��x�a���;M�����we����"A$+Kc)�V,8��m�������`�d�4$i�0���	x�{m7d�@'����w�K����{��B�E�w9�k�r�*�o�~�]��, $'���x������4��e�st���<6����rt�L���S���r���q�~5<�����a��M��
�T�Y"�P���F��D21�![����C����o����g��X��������y�r${��f�d�s�������<�7�M_�y��?���I^O�#%��0���`�=����")X��v����?��m/m�]��-���+�����j8�� INKz���lp���$���%f�;�
�f"U�,�t8��#�a�*�B���4"h�l6iD�J8K���S�p���7��G�")&)��o���LO>�a�r�]!*���#�r�5n��9��YaM��O�� Bt�U�b20+��]P�]?�����09.��2����Gl�UI�y�0A��������j�6Dpz'�����F�6s��l���lu�C�SZ!�)�����@s+r���L�t�J�`Q�A����2�3��$N;L /���Fo�����0��]N�89��	_�hfR��Z��qp�?4}?�!����R$�H	�#�h��t������sO���q�n@�u��*$����eG����q^������J�4,u�@X]�F�%�F$����a����z���j�����i�iY�}��Y��#I�����������@�2lIj�^zZ� ���#3�{	m����rl�'�����Y��������+��T�sg�yc8xA/�t�]q���l���}�F�i��<X�baZ#���u�R��"����[�7�g/c��W�Y���@����7�:���]���\(f��Hd2iC\�����3�n���MqA����13b�K������al��d@�u#�����4Y��h�w�s�I4��ZxU���T]���������B�w+������8��&`r,��w�k�mu�����7;��=�����E�����}d 1m��x6���C��������"GC8Z����<��6cY!����������?x�|��K�3�8���T�C���[�]�q�����>������������h%�i����8�W��'������$BVS
���M�����o�I�+���+����7'I3uG'5�s��Z'�@�����G���Q���8bZ5��]?�9��SE9���s�U�LO,;�^+�S��y6�7�d�.�,:e������������p�o�_��3�P
��g�j]io%���r��S�k�LND���(�lB��$�������a�t�h����-�74f-��.���e�)�P4�
� x<e.GT@�h�w��������y��I���'�R�di�h��V�����D]����������A(
>�+���pB���Bz��.�-v����g4��^�������B��[�m�����^s|�/�E�'�Kj�(sYj�FLR�SS���N�!���O�@#`��������\>��������MH������q��~|~�����x���"�rBc3���h��r��9>S1�-A�������L���3�5T'���NK0\������G����0p�x�e�/]��I��X)U�U��9Z�)�,~�]��#���K�\Xwh3�>���T{S���*�Q`:p��mzl��_.���2k�.:���E������2�d����h�"����DgX��t0&��t��UHi�^������� N�/h=z>g��x�g��'Ad�Gy�}���/�i�8grq����������pa�U������D�������
�/u���������q�1
^{�Y�pC�=�hmf��Y����.)�6�MO�B�!�X�M���j��_�}T���)g�3���1y�$b��$�MhF�j�����5��>��}�v]xi������AbLb4��q��<�����9�qWw-��@��%�A*c�^���\6~������7����o�Yw�~;m�K����td��c�!�cX�CBm#�9����H*���/��
�?{O�0����Uu����W�������O�p�t��^O��E���.s!����S����Z�h�l-���Zj�MFa��������U�>@w)��9:cQ��P�����`
���y�����u����};���������z��������td������u����|��&���5K.�,)�"����:A<W���Z�gM��o=����[t���H��;hWXd�;kp�O
v7�� ���@_n5����It|K��%����$��*�r��	������oQrpB�TMR,*{�E��	�&���G|,*X!�=�f�qb�E^j�G�����\ ���w��m�S����1S:�����E��`��A��>l�>�}���m���]v�fL\uK�D�B�EF����8��N����3����?3&s	>7'�$q�+d�/W��|�!H�s��)��L�3�+q1<���L��	j�*�z��I�h��������?U��o�Y����z�-�����M�>�������$������v�l��6_}���i�,+K��X�3��I��~��*�����n�5mc������4�[�f�-��'1������3j��^o��Is���������F��^��w6~�9�v�����P�.��c�X�W��w�'�
��D�����`���tU=6�'����m�*a�����\n
�4��h�����0����������d�/0�D�a8g�lNDI�e��7Q��#)�;�k<I�A��E7�pixD,<�� ���F��/�e��yF@�I�m��Y��(�%�/
�s�;g��ww7M�Z�����v:E/�*E������04�A�
���p�6���K�V��aN�p	��4TM+�q�\�p���������i
�'�\�-�����|��f�tf�<�p'� ������C���������h��Z�4�iw4��/��#}���.���������?����m#I��9�������l�h�3��e��^�n�"	
e����Xh�*�*w'fQ��a��2�2����m��m�(M@�j�'/��l���yAe�p U�e"�
I�l+��GzL`.�b�f�fh�G��q^����a�l�S��yc��@v�c`	�3BoE/C|���,����H����,�C��|���bQbY0�n<_^�<��E���Po?���u���@y��������H%��O��y�m������>S�~zK��+`@h���]��&�PAsO>��;7��F�}�6���%�{�Z�mz��s�aHQ�b#$�F�
f?����
���&�^V���
��\�*K��bc
����y�L��"����# ��#�
��A���h��0�0��p���&Y�Ia��d�A�(0)��L�|nG����>�6=}��!L�fp���f�����b�M�u5��H�,�gr1��o�D;$%~{��3�MrW����t���n��H�u��]�r��G��X�V(�T��PP�\NpF*�������R�B�9S;(z���[%��I�����w������>x��B�C^���j����oF�-��Q�����{�q���w��#�(��7�w��)��<���&C�/���`C�jH�r��r��=���.��AA�6�0����a���9����#���5U�_����=���N�w�J7���/j��n�d��.~�;�����.��2	�B]B�#&����G=,Q�@�/�{�	��#��A��X�T����;`#X�]Y�H�R��*�l�@R*�~�,��>��1�A����Bk�^�-`s�k6R�����I�����8���;K�a����@0�znf0K>�O�rw9�����iAL$�D(_���,�u$M�� ;�T�_�2^������T�?��	�����I���T���~p��`K�fs���u"�7]���j����� uZg$�����i�5Pd�aa~��-���~9Q/.��aW��S������`
z}�v?����H4�����+��Yz�E2/��� gK���d#�����O���[oI�C���?�Y�x�I!�iMq����a���I����4��w����n����[Ll|\��Y�����qzJ��T�����i����������s
��'N^�rV`��S�������a�����''���/&
�����#�')�����%��X��i���<�`����4�'I��-�{5&�.>uSD>1n��,~'n�
����G�Xc���\&����/��;���t�6��h��K�����H�z`Te���Y $��'0�xl����}��K;h��}�����x��[�F����p��J�����q���d
�����M���(p�#'S��?�c�����0��*;�0]�	��p�%��u�K.`���H��$�X�����O;7���<�o�w0��~p/�e����u:��(��b���GW�)�v��9
��2n�����JE�����A$����[*@{�����./�J�	f��`�M��$��8�uO�������>�������.�m�Bs������_�D�/
x�����.P����z�|V���t��F����9N�'��s^� 35x�ce�jb*�t�*�,��a���L��0@���D�'����K�����}������\��H��4x��������r������T�jW��-�|Q�CDky��.�`da����E,8��\��oda��=g~�ZzJ�w���,D�B �&��NsqZxq=����!A}��-[�.�F��cM������W}���"d��a���e�-,��G��2_<j��3��ht��k��).����?���P�����4�#����?%�<%�S������M�n����F�y7�E�����lt��������V����W����>�A�����FTD�;Kd�����\z�1~���T��Wqd�X���"��IL���
������\��m����Ku(�������4Z�q�������K�bN
[�>N�����Q�pb�fV��Nd*����".�W%]��%���K�k�8����*_s������0�p�"3�~}K�
�N\Y���g�S���RXTS[�1����:T�\��p:Tx����\��2q�u3T��m/���w�!$Iq�-Y��yNb����"���sN <<���i��u�7�U�Ul���]O�7`���MN���x;��+���'�l�?4���vf���d>�p�)^z�`3a�%d.���K�>�M[~����)��q��{}��dau��:��	�
��iHp)����l��f�hg�B�%~jR|�_����w7�_��:�x�IV&��u����/��W_��/���W�D��s��
��p������F�><�G�!s7ad�����X���o�rW��4u�6���������3(J`uq>T��Cs�
9e�K_��zW��C��>�#@��F��*�+_Q�I���sy<p�L<,Bk�k�0������eEs��e���+�h����U�C��OK�����)�S"����-8��������H0I������|G�(����.����EJ6�]�@�#(�"��Z"��l��Wq����$��>��gK���f^X,���������i���{P������
 K1�!%�X�	�Y��d|/l8
b���	�.|dI�����kPl��g��.n�`c�:\���CD�g\%��l�\\�@�E���F�������@0���K�S�>]�K]�U��|���d��	��<9K���or�2E�P�
��K�xC.���N�-$�WX��!��
��������c��$qg}a����L#GD���#�1�wI���S�����T@t'�i+��'���m�f���<y)��E���RZ��n�
IU�Lv~��������~�|�����=_ND,�a���~�q"�e7���C_��q9
w\��/
�!B�2�X�?������[����t��!
%B��R��Zf���2���iB�	���� ��c y���d�:3�����B��x-�m+�b���"F���NA�tB��r��e?&�2U���!Hz
�J�Lz�1�d���fc}���K�S}�[�:���������T����`�MU\Em���D�P�:#��i��Il.���O�7��Xi���C������jj#t.m��c�p7�hD2a�ht@G�l���A��h����\*QhI&�2tm�)2f�5�����T`D.cO��Q����{r���������H�0`���������|���������9��xz�~���w��9���Y�c���?�;�m����������1}�m_�!�7���Z����aff�W���>�;�jP�b&��dq/��f����5�S�������f[w�C���$^��*JiS�67�����	f�B!X��9?�V��O|�cx��h�F�5��q2 ��d��~N�W����mP�/�f<,N��k)��Fd��I�����4�)���v8n��H-�� Uu���h�o����;���HOwc���gPa�D�{��������f�3{@���n�b�y7w&qn����j�W�]Wc����9�/}���v��I��Q"�����##IFw?Io���wu]��-�/%�<<<9q@Ta�ia>�:{���t{h�aRLX�0� ]���|xq�"���2 ^�=iAh<���s��_�x�@i��������k.��BZ�?���9����9��<���Oz�wO
���t��5�1����:����{p��=����j{�]�d��<T�T�����W�����g#�z�M���s.�e�(Z���vD�<��ZvDp�*��37��a]���,��SW����v+3�N�d���Q�T���4���`%=�L`������-��|F��c���z!uP��h��h��Y@P!��!<�2Ya���:J�c�n1�H���x��;Q�r��>�"4����e=�41SH1O��4��XN��	
;��
_���`��>6�����~�C�����JIq�.�������<Rn������`8��u��:��N���;Z{��a(:��U����4�lLy���T��� �+�����sQ������8;��#wDJ)���I�4n	�$
X/.�%E��J�	Y��yo	)Y|�4L�O�M��1d$'��4���<���q2��rJ^N�"��9qr�&������1��t��U���4���JE�Dhz�h�q�H�R��g�~@�{�����K{*�C������VqN�������q
��T*=Y��?��.�E���+N��vKx������"�|����:�pW���9����������d��$�Y�����C���[��y���NU��������m�����r�}x��5���)���M����`������Y�(%D
?"�	�K������a�����AW6N���XZ���u�X�������,m��.����|��A?��B��~H+|�m`1��P�P��v5��\�
W"����0D�>��hO)E�!e�������N��r"3�+xg�<DPX���K�(��P�[���Re�J~�-��X�.;�KHP�Pm����T�\h7��u��zG=����q7����IpH���Kv�p��q�.���H.[�o���B_��w;E��M��rw�E6�/=��C�|t���0�+I�[�;�3ap(��Z\	"S:S�q����\=�Sj���2�������,D��a�o��Q�T�f��f�^�����5���*���V����o����n��h�a�"��(�8	I�&0���h��W��}F��bB�I��X�h��G8:�0S�P����������<!�	���$�������/����l�����C���M*���������Xw}(�C�QP��N\���P;���q�'��otp��L�
B[h���Qa��K}��Z�
����7C�:��y���5�8�F���f%i)[������[Ffc	{~�z����i���;����!�!!�EO���L����Ef+�^#5�L�!d��
�H.����)���ef�W��7oK����r���G]}��l}��SI`u��a���]��Y�3:6:n���F�C�5HR2'����c��+7=������U-i���3?p"�b�k>�y����4��O��(�p70A�%�T��y��rr|,��&��H_����:�L�����G�w?�YJ��a08�f�^�s&X���
�����/�CY����3�0��n��y����^i�%OH��R������`@��;@���`�������kIg�&~��}��-�3������<�H���#c����r��`��4��_M%3����_?6O�/��#'���Y27�\��,���!��C����VW�������.�}_z�g=re
�����{�d�w����>����
��V<-/��'3�	����=C`H�XR7���$�}��d�:����=�H�!��W��_������]�^��vp�!G|zT0�Z���c�jTW|�X.f_�AUB�*�8��(�����sW�Y6�$��,O����_i��_�k"�_�$2"|-,I����_�_�v�Q��r$Y��Q�U����������v��T���&Vq�	�
�%X�'��r������6��3�^i[�a�	i�����xWO	�^k�^�m�)���~��_j�}��,O�nd/�xv��S�K���
�'�FYW���|s�hz<�l���0�M���Sd4�|�	Y#?@��2
��C����*�������,y3x�3������l�F7�j�r�����w0����������=��-B��MGK��>�1�\so���}��4���=�s���H]�������;�g`�����o����Ms.�U[o��I����������MF���v���m���N]5�P?�7�������&	o���saw!,u���*6�O�7�]���0�>|D!�|���xx�*��������oA'�L4��9�(��	������/�8���z��L��s�;{r]p�,/�����.�=��kAg;�Z9��7�]��(MFG���\p��ghu�|�y�~�?
;<�kI�\��M��p�O�@}���w����z7���#�hA��G5��Sg�!��_s[p=��5A�����e����
�fr�����a��?�,�5��@r�U�JR�J
;E�j��BV�cJ��y*w��N�r�����|�\4��Cj+��r���\��_�j7~���9�<eqg��%��{���t���+��J���f�
�������I`��I
��LF��:��c�F�}oT#�(��~�L?Gn�:.p������I���rx6�������g��1�����O���N/�}����`D?6O?�=8v���Y�N��6E|��l�Y*��q���_n����?�C���boU1�0r�a��L]+�m%gl�����#�ZA� �)�����_�h��Ea)&G�E�
��D�E��i�R��=���]����Y����I����c�N�����?�]K��6��+>��� ��Q��E�c[�x�{a����Z��$K��� A�]�����%�-�\������{�������=���Dn�:A� $P�G2��E�*�����=��:���W�����������	F~��-K�.���c����\Jv�f��n_m����gH�a��@��&�8�%�><��(����5�`�~pW R���d*7�;���d:�gb����r�����~��/ozBCi����LfK'��4W{FG��a���yw��0W����6�w���F`����,==.��
g���������2����\)v��!�x�-��������(�����u���������p�w6T7��!n�,IP[����-�'	{�<#q����{s����$Q�����lz��Y�����@(�x\��m��d��OJv���m��a���m��/���I>�0��=6���m��l�~wp3_@z�v4Tx�st���W���(gY}$����E<3�.gm�
��5��ab�
��[J�l�2����k��}s��-��d����Lm.��E�r
\���r�1�&i`�b��&R����E��*7����\����,H����L<q����as�����<B���<���q��0�|wH`��3��m�tI�{{��#����O���}}��R��H��K#*��2�|���I�D����d�YU�������7/�������lR��R8>��g��+���xJ����0KN��]�lx��Nr�$_���
,#g}�������l�E�%2��k[�g��2G�BB��e4�4�	L�<
���h�I�|�����f��Cx�}D
�X����BI9�j�������T�N����G�2l�n�)\�-��$��Y��*���c���/|=
6W� �f��vd��K���nN>L������qN����H����GZ�R*6&��F@`hw���>>	dlC<n�b��8�JCaj��	��`�a������$h�e��LFV���FO�r�U���k�@_�W��2�<�XY�,Q/��w��������2�Gb�_��0`��A���|-��zb.q�?R�$X�����W�s�a��YO�!9%|������&Jpfq���#%&��/�!iQj���x��L(���*���L�����h���{H��~�{�����HkW���|���P�
E�t���	�:t��dW!�-�>�t����a�=9�u���"Y.�},�����R�)���J��s= �]���2 ��.{��x�����F���Mhp����7\�&U�p���L\$��v���|q�C�����d&������k�����{(��	iA���7��,.����\6�s#��i�������{��g��{0�2���!m��'f�=Kp)�����a�t�1�c�J!� ��>o$����6�����T�Ww�Jzt�����
�$	zt,�����b�7�K���!���0A� �ciY���6q��U.���������A��{�m�������&O�e���6�=��
�����y���V]��
z�b��s��
z���4K����@y���+��)���M�z��������J]4Vw!J:���v�xh��x�a5E�"'Z��zr��Ko��1��%���<;9�g�P�����`��.�
�T�Fi�Ofp�����W��#���3�yP�/<T|}����t� ���V��!T����������,1�7|���}%�C�q�Ie�R����/e���>�W���"�n/�n�����%���������he?w����W@��Q�"��%8�`Hz�%(Q�t#at2����
�/������~��������l�Q����Q�"�d�	*X��Th$oA4���D�"I��Z&�F�1���4aWz��������c�6���V�{���
�x/��^�,��3S(��+3Yif�L>e)�m����������w�y��s�u��u]t��He��D�}��l����h�5� ���Z� z�)e��N�\�����d2�&����OdK�
�
�yrw	t����N�Q�$TW��>M��d�%`
�L'�8��`��b��l��]���+�I
�4����,M�R���~Kfp�o�@i�����D&
[��-��f������U�\C{Mf��3f�3�fir���6
�2�1mEt����/}���aE��5h�l(� `�l��-6
��p����9��*����=��d��-��N��t*��V`�����&�*��W���p���`\�	��}������o��XJ����.)$���)�"���d�����:.�f2=J��a�8�0O�G�_�����'� N���6g0`@{Br�QE]��N���!_>�M;���>��ri�1��g(Y&0M��:�D�_6	�2��+�\m�����V����FA���N�_��Y�)��z�����������08A��5�<�]&�n����\������?�
PS��ag����F��<K<f�J$�2c6z\��oo���u�hI q���1:G�_�����(�K�c}scA���LV��&�(��Y*���bi!�p�M��3���0�B%xGG6�������1#nR�~|��d������������T�@!0&����9@I���'��t�T_��|q�AYau����7B�Rn�����<���sucO��=��"���]����" iC�H�@�5���<�������wv��y���������s��!IK������������Qw^
G��q��]���S�Jz�U�����I�n����&�rm�tp��uo2D�B���$X�?z��>��;�� 3[��b�2�?���C����!4L��2���1fH�0
�&�D���<%�k�����T�'0��C��6���!&y*�
"�l{9�O�����Sc�u8�w�����x����Hp����x&�t����&��9������C�k]���7H!��;�"�(Ivor�rrI��O	�su�������ZSmK&��/
����F�5��h6�a`�����&�S���h�w_�t��Q��)���\:���/����>w�zo���;��d�%�9�H��"G>d8��Y��+������}��=����t�P��'D ��Eb��L���c�������Smo�������7GF�`��8��H�$#����^E������8,/��ot��\A����?���������pj�����+'��5��(�p��)w	����j�M����_'��>I�c�V���������?J?iW��VQ�{�����m���gp+'�hc���EF�=�F"[vS��
	�a�g��j{:�~lK�)�����L�)���a%Y%�g;�9f_?:?(���I��q�dZ�� m{�Pp��7�����
H[*Z����Jr��w���h&!8^Z{Ng�����Y���������e����+n�},>�alr��lcY"�HR(���A�>*.���o�;���M�
B��u%J/���:h�8CK��Zg���:m��;�=���:���O�+@=�S'DL��s�����v��?(H�H�#
2`}��b�I�O/y�M+��%p�f�/�r.�j	�����M���U���{P�
���[V�s�9�4Y`���\?F����j�?A[)MB	�&�[�"p�42$P�'�[���:!���KL.��^���S�72;����]s��������w���*!k���o��R�������5^x(<����Aj�?���$���� 6�~l�uIqf������Y^�9�;���(|-:)���^��=�����{y@���d�(O���z�R�M��T�@��F�w��^��st.Xr�X@�I/LzK�� ,��l��~�����Z�������U.��	U���)6$�nG��_����(e�]��z�`����`s�����&��}���g���t�x�?��Phc.>��L���>n^����V3I�0�B�$��1�l.�h����!3����B���nB^�+_a�T���Bck�r*0�_m��G5t�O�*JN���<�/!N�?��Y�+���*p�6.����"H����������������&�e����� �dg[��a;L5���Q8T��.��8����
�T����#"q;"
��R8�dY��
������������}y���m�K{�*���9�5����Fz�A�#�.��h|>O��v-:�	� ���j��T]@8z���������k(,����H�%��
����,%����42#���rw����Pd����#��h�q>��@��	f��L(�L�������1��v�oG�R
���G��|?Z���<���:������c�������E��H��Dt2nq��a����*��Tc�-�%�L��?�5l9c��6-����[��EaR����
�@w��j��[�:E+-J��z#�uQVFH��}67���J
�Xt	IIE�mQ��]Xn?���7&�!T�wu����:��]��������F�M~�xlN��`�� �f���~{��:���T�S'DM�9�%��@�#���l��s1�����<�"��&�-�r�Y�"������@�x�����m���N<dw*�!�,�S��������>�c%C
����f���tIZ������H��N���8�����~��&���������Im������&x��>�����U{9�P����`����T�
�B�U9Q���-���?����w::�����1�3D�IA�E�p���\���c��O�7/�F�r��[A�2���-r��]u4��h�^����+)��	��Ph�9Er�����\h�H�o?��m��������}����J�*�E��rdC�jR��V�\��G�q6y' x5�q�r@�4�W�6�� ��t���=��Z���xs�k+���6g�!������d�XQ|��%����;��,	�I�a��������}��&a�#����by�\�dx�}\�-�?�������K���y�9?9�`9C�F�,�	gb�#Q0Z�L.�f #x[���P7������e��<��@`e�K���k)��2��H�4�������t��;U>���Tp�T�}q��o���Z��|U�6����p�N���w��_�y��qB�c��Ef\��J�G
�
�����z�_�i�����p�2!��j�.��p]H���C��h[�W������\M�����A��n2�V
h����~	SL��}R%r	�g(b��;~lv2X�w���}8�o���;EeIt��l��05=,��B���=lIIV��t����xpYWR%���1��jh�
g
� yG:0����C/����O�s[��80bl��U�w�!h�-��R����O���1Iq�����807[jE�=8cwn;T��p����y�����H����w�S�yg�t�+���@2�d.f3:{���;�}s���#�~��~���DFw+��U����v����
�����������V~�>���[�� ���$��������eQ�%����xoyu:~_���w��e>��U|�;�')�Co����_���9���^h5�LC.(�{S�tF$6������9L��
J��d��nrS��,�4���X`L�{��w���8bD�7�H�GL�pTB��g��p9�_T�}���C�B�nlS=��	�1@6i�4���y��j2��������]����!$US�e��[��0�����^��/���������VE�(�nA������l9���� �J	U�
��#�U�����g7�\�����9G�XZKtf�CZd6�����O������� p��d^���4���Y��t��!�\�I����}���>��@uA���#�$��\b�7]s�q�(����<�N~�=���kXuW�B<!����M�����l��X-��7����v��P��\�B0n���\�j�����`�d��������K9E]�%A�7�#f7}y���/�4�����D��<%\e0�z^����C��z��]��;����s���RG�.K��j3���6\�����R�:�@�t�I)�j�v3�4��M?	�"q���M��\���|y�-�t6��In~�+��jP,/��v7"M����l�p�Ak^���E����9cSSz��t#������f�\��g
��j�=��WK������n���)/��-��n��9�]I�;N�v��8V�m�o��y�y�@PhA�W�]�E�+�R<�W�sA�n�tR�.{��d<o,:��B+�;6S� z���D���~!/\����6e0��h
�+�a��)K0��rx�I�o?\mO����PR��������F�kN6\��]Vw#p��F��r�5����J��aO��/B��~tI����l(_�v^�~?�O]L���IC�:h���Xls���
N1���`s������<8����z��^��H����������z�t�K�m~������{��xh�m��\��8|)'N����%7�$�+}�!
�rvi��M2���6�4*YD�����q HT0#�$� �TdEg��?K�F �$i�^�R�j������vp�=�FZk7�3M��eJ���c*Y��>fZ�$^��)-�@������*�8�8i=������4J�� L����	�q���P�IxX4��F�.���Y�Xy�����u�c>I)�B;�j5V��Y:]��������������#y�����:��H���q�
z��?��<o�6�a�T-�>������z��1@�9�;N�*�����u��������Z�gv�A^����4��c��?R�	9�I:��T@{�}?1Iq\����S������^7�	���d]-Nm{��p�{LPw���!s�o��M�*�jo	/������{<.WOo�^oVKh��?�n���������&E�Hq7�	]�ka ��{��Zi���������=4K��T|��'y������:����;N����py����e�`Zo�?BP�\�����{�4�:�D��P�}0#��r.y�2���}�MV�q�����i)�&xl#5��0�*���^^Mi?
�k����G@87*�P:y�M�j!��9��gn\����<�������2C�Mq��9'!�e��,I�d{���C?��?��"�D)�w���)��!���4Q�Fg�U ����o���^Gsg1�Qg"����
=�����/�^c(�n����$IW��k������u���x	�g���~&������C
_|�s����N'��g�^2��-����;j�C��������,���2i9����������
���?������mW��E�:t�'�N�H|9��%�:c1S�Y�<��T�� b��f���O�1:�����Y�H��]�j��B�w����>����@&iJ"�&��*xC-��I1����vI��[�����?w�G�\�����	�q�W#c�����ST�����_N��AV4�F2�B?���}L��D�1Y�5�h��� ���*���G��`+#�<gVGi4M�$� +��&�An�d-c�������$��m&�E��e�T3�ji�>����c�L��;*�8��E���� ]cq.���s��4��P�D�a��$L2�a��Q���0���j���y�������Z�kOh��H�EX��Gw��"	�t�=��j���kiy~$��K�}3tei	���+���p���GK���D���[O�Nx�:�i�	���7�8�j�����Y=u�b��F��L5�b�9���T�o'�'�M�P
Nqg��{>�~�(�,On����3�8q�,�����������D��3�������n����l)�]�uW��*I]J�b�A+��*:g������O]{�
�&f���d��&�zNg��,)I��(����t�������Q�W)&@�N�Jq
TbtC,v2�b:a�!�h���� �/z����Z��E~	�^�v�.�.�i���S26x�WP�'V��7��`8��m4|��)���n��&N����OS���A7�8?��h���)
q�+`.�g����V���14+IR����
H�`+cF��w�	U����?b��!���%�WJ
j�Q)���3a��eY!\$X-�|�Knw�f���|�*��l�5�S�+|��k���|.�>��$���`��f��&����|6l���..��"�xU
�H5��$i�Ig
P���C��	+_	���Ki�a�LUN�)��C��t0Uc/�������_~�b�U��:��[_�b&�U�t�7t��"��J��<]�
�0��Z��~��~������{WF�x�������z�0��j��=f�|���?��������n�0X�3��-|���<��]$z~0�"C�O��f�_n�'��Q��E�*cT���{
�ALPLF��&��up�������qj�'���D�+��f�x����R�h�S��`�}i�<��K���9��b^B���4���C���N�&����F�}�vs�K���	�~I$���r�;o~�c��u}���l4]�$U���
�MN��MJE$�.���`tk��t��]y�H�B���xE�"�}�	B0�w�����v�9��Jb@���G�L�m&e��{*�jb����U��V+����IN�����B�&��.we��o��V��T�!M4�&+R1M�����}>��v���		q|�A&	qb �
#�	q��[F47����b������,�iJ.r�NS6��M��Kn�����W�o��;�*O�����/�h��,�h5��������S��������L���\&�v
0&�?]s+��Hx��]NFz��x�f��%��!��u7�3il�aD��`�P
��|���s�����Z>]�������JF�x�:�������F?��R��]��R���Is�$�o��Tk�5/�I~f�qG�g����������n����$e�SE5�rv����,w�-�6��bD�AXP(����S��Iu�6i�
o3�eZ:�0�"��4�pj�P�F����&�4""���W�]W���|?��������wO�#�V%Vo|r7R��%!���MfM�7@a=�Q����q��)���6!�v�k�C�7�#Ul��	xy����s�~������hF7�;���x 0���\��>�"H0/R��I���4��F5
��2�����E�
>P���h�q��G���E��Kb�wb���u�X�2���6�&A���4E�c,m���L������I�?�������/�y�.��sn��'��I��	|cY� }Z-������w�>N��
��d�>Q�l�����u�4��j}C�����������o�~_v��:��B��	<���6Z�UP�V����<�	�j?�*��r�r4�C�!T��3<�N&]�	�Z��e<���zFR���3S�
'���&d�.���HBx�3���S��5q���<��R0��s���Z��?[{n~�o"7�����=t�I����F_�ey����� ��''�Yb�K�{=����~�z���_��t���/��������O��"
�����5Kx�K������NR�������H'{S���;d+ar#rl���y�R��v|�)��L<��}����c]��hB��%�����^�7!�����90g5������r�
��:5	S
��
��h	�������������__��	��
���(����{�0����s1������{~����V��g�ajy���Lu�=����Q>�T���n������Z���p�7��5���}�Mv�0�3"���L��������Tv���j�\:��3}��E0�O�v�p��:�5�q,�s�y�U�}A���ZYn��^��SD���vfM-�����p�>�B�5	�d���o����E-�:�Y�~\�Q�6������Mz�M^t.:����6n���yE�h�,��#����+�M���9��y����j�LU��F��vw���xa�;:R�CN-�MW/��5�Pj���+�����UG�M�V\����o��N�]����� f�o?�"���!��>�
�����$�}!#V$fCE"$�v���L���!Hr���-;��s%	�p����'�f|z����q���L��%}���?;�W0`~�s��c��3{:�#�<"�L��h(�����	z+Xn�GN�<,���1��y<<l����e~|G7�T|���n�c�U"�UE���0=��	���T�q�)aX�k�x���V���0"YO3�_��`�T���}�����/y��@�P��S���*��n�\����8%�������
IA��b^�����rN���^�J��(�������_��-��{��Q�d��JPr����j����#�.'���y�^��X����!uI;O������[������������G#��P��;��L��QD3%]Y�����[���^������+t�t
��'1��Hd��2���t��r�L]_���
��_y����F�SR�=�����������{{6�}8eajS�v�!�/�����FC6��������/���Y��e��LK�� �p
����f���.�����8_��g��8��d��H�2�9c�����N��D�+q�����S����o�J��z~�j���9��m����>�#�������/G������w���n}�/���r����]�Et^����Q03���|�0.��`��?u��c�o�P���V�/%�p��zA�%��
���&����*3j�Dj�tI���r:�m�%5��I���_�]���Y�U�eO�ZMRD���x�����T�Z~�yA%R^T�]L=v9���D#@�\�nV��������f���{��&��i��]7��A� *��d��P�e9/a��9e�H��s����$�d�A�@;L�kF/��������M{Xm��+N�y��P�D�s]����Nc�{F���k����a���)2n\�*�Lz�N6���%���y^M� D@�Sp���i��c�|7�q��w&��}��������rZe�����/{N���yA���'3�U{��o�����G�{��TC[T.-��h#�.QP��S�-�Wj�/+���@Z�$� x������t���>ri������u~��+���8C�l"�t6X��hsC�+�����s����'Y�HB=������sl��/t<2�{�U�2u��m����!�(-�&-��B��e71zb��I�\'���4����!�����R���}�&���N�i�/$�F�T����$#FAx'��K�����W��U���D�&y����	��o0����������+��P�����7���&�M7�&:7�Pt.G�Y�\5�1%�������������c������r��Q�
��J��t�r��T��hV��N4�x���n�F� ������U�D����T��)?EK��uh~�k9��
�y�6��S���l�U4�&��
< ��Y�);c��O]����#����b���<�)P~�nU���pfi��E�[[8)�'.�����!�"�s<_N���	�K�����`�D�x��t��/`+�$�4u�Pt�'����*j��������5����d�i���4� �[V���"���a�y��u����!�p0���!���1��=,��y���Ti[X
������'Q�.����_�������"]S�X!X��Q�Nz�x�JsU��A��Z��o����y���#M���4<��81\{K�\�Q�����8��?����5�L�����$ gC�+��k#���Q"G��2{�J�dL���j����<�(������n�r�R��L����p��DKY�&:U�������m��-��;55����a��#2d����l�������s�z��Q�fT����8��R�� ��W"1m���g�~�Z@y?�Z�OB�7��7A����e���#���e��uZ�����d���n�u��c����K$�e�*3��5��������:��pWB\@��k��� �12	�X�s�3���|�?��H
�y�����E�����������K����i�+J��YA�BOw���us���G�Jc����G�����vW���{���EQ�����*�����U�/�*x���.�T�y�]I6�PN�`���&�#���c�'����[�0;g�1���bs.����r4-�,x(���j@<���=?ZLGW%��a����82I�w��������y�;�=�c�T�)m*�#��%���������q������*|t1���&��(����Yr�����#�2��n�.����(����U&6��H��J��p���{Bk�G4����9���3��y�72��Lb(���"��-��j�"�U��s�}
�������d;!O��e��xZ��j��o�%��;a��c��_���iYKB�F�m�u�������I�����
gT�Ny6s�@��.D@�<}�FVW��
�qHE"=�����gD��`��V����:J3���)OsT�JC�X��%���>�N#��
���3�p�9�����q�<�#SNW`��e4/s4'a`����(V�S��<l��6��_&\mE
y��h_
�.|��>�f!l���I��,�G_��N �OP)���L-��-UEyT�Z�\�O�����n�h�~I�2�����3�Ld���DiV�j�?,7O����>nWO���.7-������by�m�?�b��0�vD�>#bd(���y3-���[)���&�D:��j&m-�����������c(���]$��S���]r)N
��Wc<�!n�����A�����
6b���m���T�D{��/'Z�/9�p���x�u5��+�����q�Dm���\��� �cI�=:s�%W�����/�}�Ps����$��v��T��v�
����vx����`Z��G��'���e�{��X:��
������6��������R_��H�T�Odq+~��r��y��aV������ ����T,B��I� ���l��6�*���0�`?N����uf���J���#��T<:�������#c2����G�]7��������r���5��C7��w!�%qr��u=#EWDoC��1����~kM.FuK�Qz��]Y�G��+~�C���x���G������K��]���M���4��7"�Y��R�;��$���N���whO�2�U1���N}�o�u������hx4� 4����J�jw#�f}�-��m�k��"��M0��Rf��b�����g��.xM�����R��1��d�Qo����M"��+U��)�����o%�QC��b��GX�����������M���S��O��4;������gl�\�>i��L�����}���ZS����F����J���q�
<��K[P��K���F��=��l1��,vC{��w��Q���a�<���a��*_yi�Q�,�m&��L���Uz(�w�u�w�=rZa=M>�A�
vn����%r���jT��n	���9�j�f�|�_�����|�h�T��6)�e����/-�_mN�i�w[42�C�(��C��5���2����C/�r����}-��)y8�;���y���	���������u������\B�f�\�&��9s*��/nQ���lx�h~]�Qv%X)����B�|K��"1l���Te���(D1���h?�����4���GG�3�.WrN9�������UT�����[`��5�U����z�?O��E��*K��\F���dh6l��[�i����(e4��yl��������a*9P����{h9��Q�{�g<�!��.=n������P}fpC���I���O���?��)������Ih�eb�6�urL�D�v�R�2ak�f���nPM���9���5�%�5YF)aA��)������.���M�!@"��a���O-wy���e�2������W��	�5�2o?���"���_�c�(��[jcj��&�hT�-�)�Vm���������K,�����I�aK]P�$��������~|�sW2��X�#Kvq��=���F�+��xx�w�S{<4��n��PN}��Y���LE��������(���R�/����MwK���N"���Q?}����q����zN�ztY���Y;�w���\��t��
��w�a!N���� u��K,�a��\~{�`M!�T�P�����u��(8���;�q��9�F�/>���U��{:�b���o��u�h�yR���'r�(N�`�O��k6��nA�ZJ��z2z.����������}(
�<�����Xe��g^%�6ga%��5���O��+��<�������1n�--���CM����gg3���E�����kt9����c7�"y_.'�)�%T���@�G���CJw�:���w�_���yC*�J?�G����	�������[�>�G�&���f���^><��JT������@~��GB=?������1$
�,�+�����e)�k�8�o}���C�-}8��H�!*Yy$|�*���t����hE1+v?����'�l��?�K����1������_\%�!��P��1v`���:u��YC��Y�Rz4r���_"'|e�dQu�9�#(wiUaf�
ne��i�&T�����������(-@K�%��1��RS����u�4��-��6��!��>4�dkY�K��X�����/0wha��Kn8qsz�V��%�����D�L�1�,|r��F������H4f�N��W��y B��_���`%��3�n����9b*/��yw����V���u���X!y��f�~�6}��D��qw^%~'Dfx�">��/+#i����0��C����K�3T�!;�W�*#��{���yn��uZ�
���D����m�V�4yV����z�<0W$A�h^�����in?y���`�ler�b����1p�Ckn�A�w
>>u ����fW�t-�&a�F�6o�����)2K<�&��\��*�\{����|w:�S69�=�	W'�.��$fV��a��v5�5~�������9���*���������;�f���4���,�Av�����vwEGJZo{��[���T�m���\�b������[��<l�����e/���hJA�=���D�������/����zL�_6���;'��z�$��k4b>_v<�N��0�V�=��Z�:l�=��Py3D�\2�a�;]@
Zr�O.&������
g}��(�^��|������#!��K������������,?P�g/=��J��n�3��5uF��SL"w>����Kvf�9�	Jw5�sR�1d��N�����Iz���#�u�����?��9�zA�eP���rtE�5BU�,���I4u�����P��o��
�fA�dl,o�!���6&�������0$�M�j_lH1;F��S�x�a*�J��]{�@%�n���S�������8�
~�?Fk����<T/���k���rNe�
��hj�.!����LA���r���}�)t�n�����C������������ndEb� e!����&
�/���
�{n���K���3k�������C�Fr7�q���b^N������'G��&������;t���G�m���v/3b��l:�E#��w��4@^��F��q�n��k]�CC#�B�����]P@U�R��O;q��V�(���Z�*��J�J8�P�T�E��a��i�F���8BI��mWF����%Y
�����"Y���.FgM'�SL,�h�N�x1���w:�g���������X6�L0� x�bE/L�� .DX���]�,/]�\�3����a���]�fe����m��A��z�"���jxR�au���3j\%�BVaG��c�\�BM��5�z�D:
��)���;������}���MPh���9>j�F
��!\�P^�K�	m��6�r9���}���`.!�YX��Ei��H�)�`_s�A�G=�$er�.F���eU�

�`P�a�)%���o��~�������}���:�#��5m�
0��T���T9ic/q�B��"k"�}y�������5&�Y��|�q����H�&��2s_.1�IC��Kg|����l��51��'�9INiMu�W.�����k�:���|���j�X��o��#��%VG�%m1��fw�m�����9m��y�]E��4�e���
O:I9����*`�v8m�8N��/�=�6��1	o��)\�,q�����W��yD���X��	��+5p2�a�H���}��g��b��OM������������|��{/���F7������;F��g�"�Y�5��9.m/����b�b���D�!v�N����]_?�W�%@���&���F�C��l�*l#�t��R����>�$�x��wIhc�gis&�����M�v����[��3��6������������l��t4E��=��3p����J���Y�#���`�0)q���]�|���rL�� q���
��D���+���8���p�S��r�����t��r<0�������m)��~_�h	�	\8g�\�E@CNGdx���Z@�Y����~���p,x%��^�v�P�$��sn�Pq"��8�@��b�Gx���
.d��	tS�S�z��7	���B���D�H�}4D+�AGQ���C�S�z���s�(�PQ����%�2�)\v�~���%<#N3�4�H�������}��m��zR"�T
oe�4���G����yi���R^\Fh,�m�=��v��i����C�< N�.s�1�4��$�sR@�%J��lQy���,��d��\N��2���Y�;l)�����z<��fqb_]b������&a��Ze�b:k�X��%�H���s���F�G�k����2��$	������i���	<6C4�t(��a�|k���<�����$U^;d�N�W3)�FE���8�(@��e��?m��a�y���cWY���I�?f��M�!��^�1_��o6�z�b��u���L�8�4��4:�b�V
hjJ��?B,mJ\�^Z�p���`���Z��c��I����Et���-�T]O�L��u-�^�h_@�>H3����$�P���5���7�]�|�$M����B�C��P��v����u���08�qGUZ#���R*{TZ���r;xx��KI���@�I���f���2R9m��^)I�.kb��zw��e+��i���8�������gx�����b����{8Y	�P	�J�����������zD5��������|\�����(2���dl���~#y��R��S�H?~��M��_x\'\�3H����oF��7����o�d#�������1xFn��9�'v*��;�{q�qs�Ggl��C�z`S~���!����3u�7��,�f9[Q�r%I���j/��|���F�{��?=�w�^�����I�m� �7�t���n��	�{����}��_�]�f��~��y�1����i&�4��Ok4w��f7JL�"m(`T�
G�8����������o�����",���1_��3�&���t��0�>f��J�$�83b�`�]B�~n}
(����������%��rv�Bn!�57�Ux1�Q��k�-��?���oO������o1��,��W��hw���(Q�%������B��j���E�H6�1��461{9���=�><���v���6��Y)L��b�j���^��[o��B�i~1���f?zA���2F$ ��&�[C4-��D�@�{F��J8�&��g��
�9+�KA�S,������=�:���#�n"i��P�j%�q��
�P����4��E��z��vi���R�������lm�.�v�@IN�Md�J9/oa��HO����R�?M_�o���\�����*7����I��l�YU�f4(^�������z�����:m���7}R{�mLx�5����P������6��:�����p�\	�)���G��s1�Q]�����#��8�2��#���f���U��$-���������y�
: �m+�c�`���R������+Y��'�ZH"~�l��U�.�
h��`�r*!���������v��8�F�!M�#�D�(x���D;���(4��������c�	!���)�c��e�U�b8x��a�p���r��)�&�|��qt��r�64���\1l������e�����e��������Q�t)�����O>�=�,_%����l1��*�<���	hX�O���ox�����6�}��y]%|2�a��� ����_���m����h����t������$�}�\��c�����	�������������?D�Wlj�������K{�����S}������Q�q,��s}<[Y�5��A�� �J�t�l���Bhid!������������!�%���
��`���/��#�"�4�fi
�	-;�	�7�j��TQ��Vw9���������vdT���{"�c"����L�j ���]
q^r����w(�>�z�>����|o�� S��@(�$�����cm�+Y3�.HjaR&Y	9��Q��L��V�:>������OBes����=��dj��twu&�����I�9�4�M�c0�}���c
|��\?�����+����	�������_o����J�6J)D�C�����',
3G�U�������������M�Q���'���!����s�VX}�V��(�(���������_b?��/�r:y�3�#������(,�d���{�d��U��n�8d}�<Vc��R���KO�/�R�xJ����^����:�������@�WLNfo"�*��A�_���y�pV�"S������L�w��05��������d|e4{���r��y�������7X��;4I����~����
�7�Rn���Q��D�����V�T|��������S��}���85�`il~~$y������`q�^��]���u�������C���W����7��&�9m5�4�")��<n=�@/�7�Wq1P���I�a`���X���Cu�Kow}����N�e�!���9x�-�j��^��R����v��>4]�i�u����y�W{���\�������T y������~�u~kb��:��;�OS�huQ xD�?�{��X��c�,���?r�oLT9G$2H&�$�����s�=���0����@k��u�E��!�YC��A�;[��}���v����L�{s����vW-�d��E?g�:\X���p���o���������k������N����Y
T����*4G�u,J�0�?�J�.�]��nk|�G����$��C��oEn�4��&���[1����������'~���6�I���1����K���y�1*iV<�`����5����l�I0G��%C���.e�-�T
Q�9?@,%f5�KxQ�+�j2��$�4G�zpS���u�b<����3��Zn���p�7$���H������������k�~���/��8y���������3�Ab|>[L����������n������O��4�VBF����h��X"�d��
������qs��2����������� vi.~��^^4B
�����h������]��#����&��MB�<�p��wXSM���>�:���Qw-Mr�H�������m��iG!i<�'U�nqU/�����/$�d5D0m��p����(����8��CD�^#��M`�)��v�%�������]�]=z[5P�����L�������`����^��v~KM�B&�R�5���1h�g�;�)���+h�.����K#��(�@���n�{L(�J8�'�~�A/v��	�_�{��^��!c����*Z�+�*4�?�h��
}����R�>K\Ic�
�>���>��c	�j4�����o���\��N���8f9��J��Q���z��k+
	�3H�E�#��O�X��-�(Fr���x6����n�"�������Q�Jt�O�h����87I(��L�^��9����\{�D)��E7�	�����&�dF{��l���R���}Y}�"4�<O=l����m_��~�a�w��D.�&��>l�}��VyD���)u���ZR��)�t�1��n~ ������m�#.��"2���]��	�����p�FL�H$�Fd�d���������c[��t�������Dl�8]��V e����I����7�������?Ecs����k)�Ff�p;w�v��wD����[���8`p@����A�������:'�����d���*i�TilP U�;���b1��4sFp��[����j�cb�!m?�U�!�E<��0qR,O�.��T���s�S��h[�i%w,J�!�~J��&}G5CN�J�Q�r��.���O-���Pu_����5g10���a��!�E�N�CH���<t��!B��p9���\���q4>T
���?M��X$C0���4�)g`P����?�LG�ZlN�^�%	M)�/������������#��R]@��\JO3t��H;PQ*l;VcA��A�5]7�B�]<��p�?��i�G��UXF���m�+��	����tE	�$�Zm�<����	5iI�RW����L.B
X
M�����i�I.(:y����+�Ww#:��+)�w���������X��
�Y��la�{"�X��%P9o���5]�}�����{xF�������Cjm��&�����e��8H���������[�e!-�
8����b�/���{�eI1:��>�����4uEc�P���5I�B	l��G�J�`�m��(�KK������yFk�)~Mj�v��1��3����(�5��0y�7�k���sP�k��T.HH}�v�h5e!4Zg9��g�:���u��O�)�$�"H7I��	$�]�q�f3�.}�F%q=v�b.��/"Fz��0�g�b���4�~}hZ�W��K�;\�h�G�b�sIr2<������.�/���4���5�2J�$6�'�����t����$v�����1p5���
>nt�|�rD���5�jU	6>5MJ�����_�m}��nB�����u�h�iAMZ�
�i�9�9V��.0�GD��������W��X�����m����ns,��f��7�:�L�/G%F��>S:���e��m}yh�����qvE\����6��i� bG9$1N�0��w�@�x��n�>���������!' ���f����U{f���i���Pu��'�������D�z�����oaayh�����;��|�R���I]� ��B+��9W�>d�A*cWw�����Q(��Gs]X�U
W���z_W�>k���/�s���5Mre���2W������&HwI.��h4�;Fs��~P�T���(�Ki���4�Q�`�\3�����}���X}��Z�Y*#{Z
H�����fR�������7oJ%�������g����������0e���]��)�E�J���o�]{�=�)c�*�-�$���Y@�R���o�p�2��%��{��9��K��fG+m�ZW3d� ����@���������\���������b��3��q�����2at$V�����9!1"�2X�w�������Zv������v��6�w���� k�{i�j(�*Y]����J�#2���]}�_)�D���81i.�4Z]��0�K��V��Q6E����(���	�����z���f�����Q�}�@	/�;���ij'6<>�GG�I���$Mi����p��l�������B5��Z[CI
k�dl1�������POz=�������c�bh�CG�
��
�&8�
g s���A�Py�~o� ��g�KJGo�y�K/���^�'s���b�L�"�RR]^,�)�p�����l���*W��o)�&sg�b#�\E2$��Y�]���\?��&�����Y<6PJ������E��l�����w��h��Kuo��e�Y��+Nvu���c��\���)���T�I�2Ha���csk](BV�
6��y���	rt@�P�_���=����iM�h�Qo1���N&�����m�{����������A������4�]� ��������wh���������r�Cui��������Uo����p�M7��`�����jX��l�Z��s}����]���t�S&�A� =��G���������1��J�d�+����\D��b���zM��������\����K�/��5{�=�V���F��K���r�!�����@�
�����jH�4I�`��`&�u�Rj��������|8�����u��Q��D���S"L�i����rn8�M"i�ig�,���*WH��e��E��$���o6���oN���D���KLN����r�e1��S�kB=-"�M��~s9�KjE��$����n�������=����_���>"II-d�vs�#�05��X��90hn_�Q�ve�J�:��j]�g���uGJ!7��x�s�����[��x���t�:R��-9r�dLn����L�O��<�����������G���h��t������[�Y�!<���rW{��e.��0�/�� ��J5r�f�
3{�<0?,I+������+W��V2���������J��Ay����jZ"c&����g|����%lYc��ER�.��B���e.�y�}�V�.��t�f������P� ���`�6{`"e0�������7��K�y�3�����9����a���UZX��0�Da��hZ�.���h��)m.�����V%X"���_h�����X���Akr�./3�O�\�"��<��	Fc��az�3Y��Xk8#7�?�L��o{G��U{__�4���?����O�R2p�V[$d�$=[)3>���l��F��0��Yb��K�P�u~�����j���~�f����4`d�Ws7;"W�������K�l)WtD�����oA7Dc�]�����}���������S�����E�o������2b&��2�����������j������������!�o(�,���0�����8!�7��� ]��$I�\���"����owa��/��;��d�&�y���	Rs
�/K���	��8���A(���>@�T�b�����P5�T+���r�f!��/(��d6Y
	��Z�`��&^����<���nn�f���Pu�.�`�Z�(������8\���:Z�+|8�rj3V��C'�K�J���N��o�?W�]�]��<��rZ�>�����%��w�
u�1����:{�����R]����K�������}�
��'&�$g�q�����u�2$�x.gR�%�X�B$GS�Q<%��	�Y�Q�����dW���%������g�M��O�]w<]��G�%	��o*�n�#�h�\�1�#} )1nD��4��`C8MCO��3A�"��
���Sw���S
����+Ly�;���"����r��HI/W�\�u�l�;���n�����
O���3d���z�i.�������*��t����6Mn��
$4��[���l�	%%������3�&��9��m��	qPfs�7�=��A�$l���b
�#�����qQ��6H��D�����g0�8���������$oR�9����q"��d��� 9\��P:����;1-�
�3)9�L�>(�H�4L6���B��!��������:(�i��A85�N�=�uVU��I��}���!va>2�X4#A�:��ry�Pr�gDBM_u���I�&+F����"��A����c���A�����h��u�H����0��<D��])s��P���SI���u\`�c�D�v#Gl�Q2s��g�{���������r����/�kqfW�F�:�+�W�%c��:�i�s\�����P��]m/�z���cW�����,����	_v�K\��L��M8<��ie.���P�
2?���T[�9iv�oj�t����z�8��~�+���PQ��0��+����������?���:vw��0x����������Nl�P��I(���,����)6
�Z���"�Wg`[	�fZm6|������uwzh��?����0�?�s���G�����r
[\��S �\(��g��������V�?\�7%+�����i�S������+e7J
�����^��oxu8cV^����+O�.�XzA7J�R�������t(A/ �{Ee����`t:��L���g�C�ht�A���>�z�l����}��4��ukY!��Gjb�Z7h�2��]5�B�1P�m��/VxZh�|3����E���\������w��~���<��q���km��������|Xa���1j��4:-��mR���'XUfsg��/�=H2G�C��+����"B�N����$��r9f8e@�(�ay\���p���]�L��)"*n_*D�C��������*����]Wzv�:-X�
�[��@'�u��vT���@�m��UW���zvN�_2�g':��H�I��%pza��l��Cs�s��/u/<HG�>g3�����+;Na4�d0RB�����a�
�0�h�0������d���P�+X`nh�K���u��Y����M{�=u?3��7�����g���c��Cpo��4-�$h3�������hZ���<�^�gs�GM�8e�s��}_��"m��G�vy� ��G1��4N�{W������y@+�a	^���r�-�8��gms����&�C}���e+��4�:����~�Id��#�3���C]����3���8
	�f<)vE~�:�eh�{�W>��`L����^5�����F�R�|}�O(L�i�1t��8�ED�O�F���M:�����i��
�xr&D�^���%Lv
�QNH����}�ny�)����Ps#m��l�����E���	�39r�2�+�6��`�F�v��!!�X������������1n�������}���P~�����
]Tc9{1�F�
)\��/<4��ziW4El�V�2A��H�.�.����5q��p�.'O���$Gs�>H}�I���\Zk�yW���o��f�C�iK
�#9rs&Eo��j��g~6!��i�����{�1;5��Q��F�`�^J:raf�]Ox��N��(m�cH�H���]
U�|���s_w������ei��	��,~ER(��<��������ssb ��`_�}\+8�)��T��y�R���B�(�(���@lH
�"����-+�UDXl~4�Fg��"��$�n��hq�"���V�g�@��H�����W���YG�����!�;!�S��vE��!�o=�N)�\P����|���H��B�=nzE��GZ%���{IX8�@D�����%�D�0��c
�k�.��3����UX:"����%�~�{�JN�
su�
 ��J��KS&���gn�_��p,�W|Q��\�<E���(�+^����s�:��_����U� ����������������i�_d�r<i����P?������wU�����g�� XM5��U�Z���0
�P��>�KJ�+d���u���d��z�ED6h���y��C�������`oT]]>>���sW�=������C����������`�:�4;S�m��������?�@����V.��F��E�C�"u}\��@�A;��K���s��rR^�Ir��,���Kk5�z�#&�J<��U��������Ri���<��d���> ��)���#x.�Go s��t84�����������Xlu!I���,
ShW�q�<��g�����K��7w�e���;�z�hJ��?�b52�E�M�{T���� ��������4����b����������j������n���S�@����t����������N�a%0�@:��_���l|����?m/����'��#i���.G��X�Wu��f�s�z��ffe�#6�6�1bJ�`��&qK�8p.f��\��cn_# |	&���m�y�^�
f�1li��J�&��E{�M����O]����������\�����S��=�x��r���g�L�H7e������Z��3�}������m��_���`wTm������+���BMr�9�uI/;D
W���=$�<����}|�V�_@�fhZrNY���'������*<3�
�W���Dp��n��YE����
���fi������
����������iT+�(�
��Q�7u�	�����~�2W
�(��hx(����R+���`bcl.]��fHW�X��e�,n����k�q�����,�����&� @r}�����
AK����PTb��9��b��2�	��+�Z�U��=�%���tFG���H#y��[�}�],�
�P�h�������`����6����6����BI��>�G2#Xa�G�k����(�*��_�<R�a������7�-@o�hDO��Q��H�G(�r��5F��<�@�|�q'S��U��I5����Q�f{a�R���&�w���[��]}���Z��M}��SN���D�eS��X�4����b�XZ7�n�����(�h�P�n�wq!T@�b��r�t�&
�LY�IOP!��|^:�T�T��8��q^�`�"��
�j�Pa���b�+p9�E
�.��W1����X Rv������o���cmT����AL����M����%:#n��������&�����������v�}����%��R\y��N�P���Z����|B]��9���^����3N��T���@-�$>���IM�{�'�������p�PD5}���d����<���Y"M�H�@)���O��CM���p�Ur54+��*��C9���2~���C��������Z� D��(yRj�����H�K��T�<tG�j��f�yh�Om��z����C���.�	u����M��)��Wy����%��kT#.�����+�\>���?d���Lx�4��`,���u���l�����o��<��#���
u)Zx�p���W���,��#]�S��+)J��t�w�Q��y�1��j�X���vC(i���[	
}���7�x�`�1�*��qkIk��f�d�`�o���[���./��=��bl��#:T{�l�1�;�?��z�Y���w�_�q��$��� ��Sd� ���p�&r�h��M���-�V�}:��2��O+��$N�� ��I��v��|�Dz�Kl�f�I*��4z�����i�_��Z���tr�r\b������vV��
U��Q���}�1�3l���T_���+7_Ht�"���E��b��/�UX��\���GG��p8QiMc�Z�F������o��"��+?���M��*a- �13�4��~t*`��8q�;b��a7t<��J�0�Spb�]��;���|N����G$R(�/k�����L!��
�7�T�i"r��Q^��w�~�a���y�����2~2-�)������5v^��B��	���*6��52������tje�!�����f�/�����d���46NM��r>>?u�v�x�y�
\!�TQ i9�7!r��Qj��xv�D>)Y�#K-�����U��cZmc�t��������������r��Y���n95���D��7�#��?����<�K������[$UYZAa1����*$�JP8�w�����NWU�0�r���N��>sn������hOy���C�l�J���6��</�M��+���W�P&�PI|�2x��������d����:L'�lW���[C&S��B2y'9��������p��HH� %���d��\��MD�:T��r����q)�|���(J��J�
�k��px�o��Fm�����������B�3��@)G|�����IC�y����s�1f*�����*!�$��Pu����'z�������
8$��r*���\j�sW�����R�j�}��!~��g&���8��"�2>��4B�`8�3�@�&v��!����_��:B�iM�H�~���&��}���T��[8�m��[�����������/�����eV��Le=
'�<��	E���C�����Y�-"6��>6���\X6AkDsH��Y)z�DZr�(��/'�Ws�Q���y�b�s��Wx1_�{~@
�������lr�*��*�d���W���P�����.�!����_����I�#aRz&��Py�D�9�G�["�W�����������-����T��rA���M8��y�� ��e��q�����S���C�}��K�d�[N�\�.-�3�M�'&xW3*#�?2���������_@s~�A����
W��Rz]7�����*�������g�X����!���\C�Y�(c���b�
r��������||�y����yy�W�,5deO#F���
���5��w}��l�_�x��Q����(O���j� O��Z�������O}����]�J52�lLq�Y��T?p����/����k��] _���B.��>t�%kN�G�[� ���?��{>n������
���	�%��Gz/�~�a�e�i�EA�O�������b�*c�kqH'WZ�jf��t�F6�zT
Z������Q�C�k,��
2V�"��33'���@��Jy��m�|������o�F�������&�#t��Z��L&
F!�G����q����f��
�\VjJoU�j��^�����^���X@�-HN�b��)�(��'�w
&�![sM��P���������%'N����|2�F$%�sB�������\�[u����`9���W=/V!`���O�Q���)�-�IT��,��0��r��$�
ZY��05*+��SG9T����	A�C��\L�brn#�j
�f���R��V�C�����-�
J.��k�@��:t�.���r�}+h\�-E��R�G@�]  ��]�r\�
bZ���6��
GX�<�����WK���\F��)ice��D���<�f%�!m�k����*�H������R�SK���t�^tUI?.���&�y)�R�o����~v�(C��+���a�u4����EN	�H����<�-�/���>��&�k��i�T�e���n4��F��D%/� 3.�������n���s�v����{j#���}�(���Q�O��fQ72�K�^]{�*���Eq������UWNMb�L���H,��l��2��(|��E{����U��MWH>b���&��V��i+���J�G��S��(:o���x��U��=��	8 R��</.�������}���whf�&���z|��s����PS����PW���w����wh���6JD�
^��}�`��H!S��y[
������Nu4X ���,9U)T��RK��t�����P���'��J�=5i��*��@������J&9]��C�6�'JK+��
x������)hi�p�T��:I|�F�8d*�������
�7�����s��S����j+�V������J�RV�o���e����}���g���yH���u�\>R������\1��#Y��f@2Xz���L��+�+I
�^�Q���wn(����������:{g0s�$��������54v	�]�����9TWD�A����RZ�M��9��m���;Z0}wq����t-�*���q\9"��F���}
9�H�J��s,�_�G>�_����|v����R"�S���d�a�2�(�+t�X��N��$�$�M��GP����.��/���~��������X<� �!���h|�J�?_Do��������b��g+M�K{7���?H!3;�W5�nl�_Ll1�K]@1����d�����v}�Bnw(EJ0��5�5#��0�#�9���y7��.��^E�������~t0�������R��
�W����N���w��Kb}��2��w��	�x���f���Q.m�,/:	;�!�r���d�3t�}h�����k���%�8����$t�2]����z���d^	�	;JL�R��!��u��Ywmp���u�93��q�;���p��a&3�uc��D��\3�-79�&�K4��j����I~%.���oL��ktb�Zi���#�KS�W�~4q#T�6��h�N��J�R�<Z_��2k���'�Hn�����������qQb.s�����>����G����F�%���;D
�����I����($����TO��_$LM���-�f�yu���fi!-�gcgy�u*���6�f��vN\UB����F�����i�c�=o,B���w�1����UU�0�9��_b���)Q��r+��5���1XN���&�4U+-����{����XzQ������&�����^M��q�������/r�L�_i)�g8�R�����F��b��i;u�
/J�**�@���sT�%K�d(�����pl\�`)P.?���p�<6��v�(���3/3��C�M���4���K��s��k��|�|��s�]�lR7P 	�G��U����d�.�o��6�~lN���7�����T�%b�`�H,)#WB���D�"�B���(�w�*g&�����E5�W5+���F];�WL��_xh�m�w��kq���>�]�_���{D���^b�N��l�$*���m�Y)WJ@%O�}������E�,j�MI��(������������n�\^��j>������Z�3��"M��� {|��m��c��8�]0A�0�����N\�W���3T
���9D�3�m������y��U�|i90�M���U�t�`�a�i+w�@\�FP����y���e-t������+
��2�I(7S���y1�1%����r��~�w��b���o����\s�}���qJ����)t~"�H5�����
;$��G���SbKZ���������P'��������+�[�,�>�@�7�x�g�j����"A�����+gJ9����~�����a�s������O^ Nw�����k_�<���r3�4?�& 
2����������<,4)��
q����!�2��n��r�v����5b2�����E��MA4_�b�u]{q.�7���@�K�����9��!��X������U�������+�H/�%xg�%n�����N���(���$���H���>�Z���S�QC��.|QA|Q9Uf�U�U����]jX�,������q��+���2&2"�j��m��mK�4c���
9M���P[��K���O����O�`O"��J�d{����+J�z(x���9��T���D\�O��d�]����Bd)��/L��.��>�y�U��kR	d��H�?h�������/p
�����j��q��REm/���u�����)��4��������f�N��V�-���7s
�Z��3���5���oI�����8�+�?_����uD$G���T`_&;t�a���R��^cv��[��F�����8x��1yM&���,��a�F��d&.��xN��C�M�O����bsBH+ 4P;��)������S�LJ�����Ms�����L��2��T)g�������>���#��^��!�[�$v��X�^uD��2�.�h�$��H��-�����8�cwX��S�g�)�S$a��'���lNB<
�D�F�E�05Fq����\��S�������a�S�ARQx��l��Z�vZD�*B���7�K�EN�����J�A^r�M��Q4�������v��y��D�R��[r�n"��z*2�oG�mO�76�R0������>%<8M��i�E�V�T�Ko#������gP��y��d.��>?a�f�fu�����r���M.���G7K�2W�
HBz��e��	��\tAjI���"�sA�+���m��*E<~~�o����R�gO96U�@K�����(�+���hV�������zH��D��Q�cy9�>9R��(���_������kFV}FY������U��{h]�S�����S&��\��t##x1]0�F�x����.�����nd������XQ���"���^����;}����6]�����>'_|�
�$����G���q������������Qs	a]E���zA���i��>���Qon����c����P}n;�i���K�6����\/y����'_���N�\v5���������� 
 ���^��}B?��~�t��_�d��~�s���@�������9�q�N�J2q;T�"�}7��p�
��D����&'��'�Y�
�di�
FDS������n��u��p)q��][f�������i9+�N�h���
�����S[����
��������L1����Dr��D87G���{�0������+��s8i�/M��R���Q������HR����6����^�����3�c��2������;���(�
�8���oWn&/�c5O3�?����V+�J�'����oAw�����w�J���6F�$�n�b"R	��I�
����RR���wy#</L��i�Le��k/|4�(�h��?��6���N1%�?W�h ���!�Q��ZF��g0:���c��h�B:C����m�+!�O��wbA�F����c~�;
8U�,<���cR���
�c��U�<�	dq&�H������jem)������� )O�-#([��Q����-�����?@�Ds��J���F�0s��(8hc%���w���]�n�}{�w���>6]�|���}�P���7��\k�M�\��P0i�Sj��1����w��
&�c��N7^�ng��@p�����b�-/�W���mJ��l��Q"��j����+S�%���&�n��W�R�x
D������=�%
O�)�Z�+�����_e���Q�	��k���	wt��$�N�����C�D�a���h�C_������O��4�m$����F�����G���G����	us�&i�������	�����p�
+v�,�#�������r�����p��Qd�h)�K�A���|�E�nd�-'�����H�s}9a���~?������\��'���R$���X��]�fH2��H����<W)Hd�r��T
�x��,�8��1����c���`���t�	����^���4���R�#�"���_�g}��B���A4jo�I^N����!�,��Tn�?,���fwx������x��4��o����f!RBd$L4��������2��G���/����g��&K&U�UEOS�4)��D��*�c�!Y_��[�9m�(�2�U���	4`t��"k&���W��(�>x~lcNO�X�?`�k��+
�X���<H�?7��u�k���W���(
#	m�m�X�5���0�W��D�Oo�:��Q]��3x�P1��Cs�x�)�X�*�Ne�����?N�Oo���g����'D2�1f,�+P�+�@�/�F�L�k��:���EZ�eLH`E$ex~�hh�\$Jyz���R��(�������+�U�l�R����Od�t�t�mf���(��i@�d)��6�����42�=�o�R�8"!������`/s8/���S�=:e�z=J���I�z$�YM����K7q��GB�����3��U��7��F|��`�#6��2XIp==��o�q?�,f���+��,i�@A����/���D��^	[
���lP
�msX�Pb�3]ac*�D�����,�h���'���^�;T7]�x���T��+��#���[�q�MG\��U��L�lr
�k8:-e��\2���t�}�F�1��Q?>5���|��0�lf��feE)FR7��$�_�%c$��6��LT��v�.�t�N�:��"3�Zx�x���H_.�k���������x��>>�,'�S��O#m(��4GvAT�I��p������aw��������6��<fsF��L�^$��
\�
pD����
����g��4���	�l���	���4D��A�1��5WbD�VD�����h\�d��R���~����w��N�J�Qbdh���T�MIKy�o�K��#��Jl�D!���;���f�Z���/����O~T���J��x{r��D"����9G) �w�]g+S�~��$�t�E���y/�4��h1�g3��wm�@)N���[V3[�	���pw�[+F9�m[Z��M_�E�e�|)=���qe�5_������l����}���G���=)�#����*�����
@���>��Pq9�d��2(����$�����x
P)����ie�h�7��4
�~�+���y�_�a��$�z[�Q���n�!v(��<��!�&�;JR1����_l���-��#��v��F"=�D�p?f�n
S�xE�p�W����_[D~��e�N"bA��������@��X0�=�D����G�n�p�2�
3 �u������}}~pn�����G�-Nx�b�
[W��b�tS��&�����]X����(�l��KY�9I������x��<��vx����`O�3�mB���)(��[�\pH)����ns:����������yX�"���u��,�tO����F�)z�|��?{/X�$H:_�����b�s"s)	�+�H���O������};��#�D�a�Y��+��S`# �uWn2������4@B�+wkC�3���Pi
���5�R������Xrs��~�s�4��\��!";t)b�[hK���(������t>]�r����&Fx_�@{��#�&�������OD	]�����&�-S�TE8��9<��
 �y��i����NX������B0��t���Z���F|��&���r?���	�k2�-'�����,r`��g��H�4)����<v���[_�����&@=J�5�l����OUQ��(��I1��>pd�����9|���u��g�1l�zBt+r*$�������k9�Uro
N�N������YQ�Zr�n�R^�������6"��%���RG�l��3��C�#"�HAs���������u��O����u���I6H�x�hB�(���O�*�G��������}�J�P�m4��F���X\9���Pwwc�p���e�D�jlM�\�{oa��F��#g�������s�2��)5r|���h�
>��5&<D������h��J;{��D.�%�.wYJ���<@���?��w���;�:�0�E���$R�j�^=,���~S7�;���c�U^Z�;r�X�f��D�	r�o�Q�Zo�a��~}z���faL�z'S0ts����5!+�:�I.�nM)SC�E�����aj?5;oN,��N��mi|��A�.���e4�-�q����_�]���?l����D���L��\v�p�(8�3�W\�����5��/oa���Q^l�1S��-$u�:NgI]��Y)&�	������dCe�h���\����6����-wOW*�=9���D_.�aGI����*��r-u�����5���:�}y�cn���
�HC�B�&��p����S;������o_����k0�N�b�R�Z���O�&�����f�q;s;���!!��a
S���8&0F6��0������o��t����������g<*��k�X����3OF��\PtE��ji��s���;*������
%y.f��X�s}6��O�<�^?���{����6Dx�=�-�(��l+�O��HR�r�s�����WO����<��wm�
�Z�{j��;9��D�RA.S8�������<;�n�?<X�����>����O?�����fo�Q$���RqH%-&�����C�Vws6�J��7��b��/ADe���`�bKy�g��W]��KR�:������cq=!��W�9&��t�/R���g��?f� ������>���B{'I���p/��Z�ku�KJ����0��m��t��������Y��y���=������k,���tV�/M��Z^Pl:�5AaD���Oh���v+�����%A��V��K���b���P����m7�o:�_{�
�z��l$�Q�����q�t��-��������q��_�d�%S�������N��9��� �IV:x�P=�}�T�������Y��+=��h���\F�'�<�tM���x$����5�#��K��}�����������C"+kq� ��i\w��X
�H�;�MO��b���p�=�<P��/}}��*G`(�@�(}�pD+����s�MW*�f3]\�����3�]�R�?�{���m?��,s����I��I����L �x�E%%�tE�0�+@"����:���!�)�waV1���pn���[�LA.���3��*���<P�N��-�M�H���J[$��Ke�b���ng���J���+�H�A�	Q.)�9.K�LYZ�K__�L��c��GS���c5�`T��p��E2���0g���R��Mf�66�\�l]I��CB�Y�T�g6����y��`�����a		q%���g�gd�w����>v��	���Q�t
���.R]�
9,vg������K���U�*��DYv����A��i�/����?
���~�e��J8��I:��
*%�
��������3�2!E'�w��|S?~^D�������}M�0�&�s?���GU����b0��cH~�� ����:1i��q�d,McQ��k�t�5�AK�~<v��q����f������1�dAI*������h1����$M�AZ��k�(�;����d�s���xC�|���}oD�0�� �T7�T<d�Pd�Y(��e0W����F�^���j�z���W��;�����V+!Kaw��j���o���4��_����,�������?E�?����]�t����+I�����.�n���N���N��O�zt�r-KS��?��
n�F#a�YS��q��_Om��c��A��N�����o&����8�
�*�)
�%!�d�<��Wi���(r�P�E�q�:r�&/�R�&��d*����	��@/��IE�@�X-����7��B�1�������Kg��JPd�.9[Q[������
w�
U������a�H�����	G�EC��.��M��p|�������q^-�I����1��W_��A�r�5"�3�R�����A���������;w���������8�4K���KM4�%���2��P�RP���J���P$�([�T8���+C���t���v�F��gH1t�a����#\+�u�p�J�Tc�#:�����i�����-�E:#o��6Z����<��*�el���8���S��>tm��R�_���	�o|8T0�v���EX��&��CX����J8���9L���wk���-QA�ru�i\y�-����b`=H��{�Pp�v�)fMO���@XI����\����������o��)���-Er^�������0�����P����.���&�Y���E���'���F��(e��a���~>��v�eg�F�	�/;��@r	���4/��0*���?����+2����,���JI�'���R
�����u��\���1�u
�Ed��(O7�	�Jcd\w��-dd�������7�v���fEu���P�����G�)���d/������?�\+Ql��,Su����y�ht�>����\,��@�h\�)�^	^J{4����s3���1US.I}����nMw]����9a���u�*�]o�T�?�|�>����\��R�|C�
����c,<�m�L2}c�XeZ�Z�x�B���o��!(������;� �b8����B�����������=%O�PImx|�� � �
G����)�
g�����,5����,C��-g��������r����|q��������L>J�+����������r��|vM��5�BD�5��-n�j�@wf�-]
��@cj�+-ME�q�:j�%���u�c�s��[�+��{��)��I�]�.����83�u�'�[���jJ�������zbmr8F/*�y,�zeeqA����b���f�7���������\�9������=���$4�~��������#,hT�;|�������t��sf0xw�y���(q�br/���8{c��Dw�����U�+�.6��������$w������~�����&�7��M>W[D6��P ����%F�js��a�2i|�qu�M�T�F
��]*�Lv3���������%]�b	�z���,M��_�����u�m�XZO-
�fX�"�
�� !����j~��Q{�R`|M�.����VF��QN�9�+������WG�����vX?��TW3��d������U����3pN@��^v.��5}�Z�	��S{}y^����J]���|Q������1����r��)�	���?ea����G�n����LO*�����cs��3W~
�!��mg������_?3�ZT���I���:y��b0�_�|�����������Q���H���$�!�$My��t���w���S*t�4�P���Y����'������n9rWZ.�@����C��3��p�7m]�L�E�<b���K$0��\�6�H
�����"������)w�cH9����=�|xN�4j�@'�� U���������/A������}�L��o��	���8��c�I)��_��k��>��5K���h�MH�q].��
U��fr�insQ��4,�,�e�U�`��L�q�x*���y�-�i��f�����C���h�#��G��2���0/"�$k�����������,aW5i�j�����x,���3#/�Hj5��O�B��g�����S��3o~���s���J���[��A@f��e^�U34����E^�����K�?�v�%A���<���i����d��U`�����S�{�[����8�`N����O�po�� ���8���}���MST�*U4�(e���*`���4J
P�N���[_TF�ya�Q��+GD��L�HRD	=��;"�k�-��OM�n��?C�����&�Z$7)lA�
�xc��)]�VL�5T�g��!�����oBn���*���;����L=�a��W��8�+��7%$�))Q���H��*2��Y��=�������G���cL'h0X��+��SS�a�\'�+B�K��|/nO#���@O���DYD��vG��a��~vr"f�eT��_O��
�RE8N�#n��.o�K�G�6D�(��8@�\1$��}���k��_��c&�M��ds]���}��C��YW@��i�����<BaH����xr�V���N������R���9K�d(	0�,jlqjo��s�����?i��k�~�5{�<��`���9��Z7?������g=���R����([g�)Q����]T��pG�V�+9j���/j�
����%~Q�RC-��o�9
�� '�����������_�����~B�W����w����3L�8�K��L\p�������??v{oI�{�6!�?�("��F�r���2�_WJ��_L��������k���<�+�
I�?u�������W��/
D��>`�f�����bQ����i���b���oDVe�E�3+�A�y�����C��~Q��(���.eH8�A�9 �J�������s�C�)�DA�u.t�P�@�8��4:+0�������N71m��Xn_"U��|n�UQ/_��C���X�9����dTM�>[�G@��j��Wv�Fx
��m��UV*r2o,�*�]i�	P�|=�r�z����������7�1���a�ng>e��o����!:q���T����DZ.���IfYHQ�_�P,~�����(J0��E.�O��i�r���<����`#�d#
5�*�di�i������;�z���t���\���-��h1W+�.R�i�J6yFT�Vh�A��0�+�q�O��|�^��N�F+\{+[�u�q�jN--a���H)9l$c|Q� �HR=sY�����O��V�]���}{����j]MYW�N��
Z�d�5�D���k��C����!`r�lx��Iv���y�eV���s>�
�w�1�1�l�hU���$����"���l���z�����K���"�bil��8Z�}���E������?v��C7J��5^<zD�o��d��T���!���[��
++h���E�'�`3���E�����L���$����\�Rp�%Q��>rN��
&��S��;)�c3{�s�M�dZ%�I����w�G����~���|�";�k��z��|��i��|��������]�GF����`{������'��������C�4�!Z�I������:an��i��%,nLUf��`���@,�|d�a�k��`SQ��K���4|.'���t"��*�,�eIZ�#&hR��.����@0��B����[n�Z�j�G���)�8�����D37�8'w��D�
$:R�R������<�����(�M��J�P"d��A
��VF�"Dh��->H�#����3�������R��[
���4f���~�|G�A`HCF�S7�U�%� S4�����SPq��pO=��������d�����r����s����������_������ah�n����3�5e��7�f�����������Z���u���b����v�=��S���T3:��oRv�5��V�h����'�_�O]{=�x`����v�����L��)A�t�+ J��JaHP��g�'�n|H6�N��4x�h����)Yn �h�{�)�S���%����~h
���_��d<9S�M:����	G!��<~�NWl���t,�;'��7�S�J�������f�^v/�����i��y[�.q�����l�M�9|* ���8n��W�@��S{���!�,w�F��/�P\�m�NF�chX�F)����{n�,�s��
r�VXO�M�H=�=�����1�tul�r�MK5��j�l���2<��(�J�Z��a6�S=e�������(�k�2�4kBmm,&W���.��fw�JVFt*������f����Z�x����{��/�����eJ�=�~�.��;�i!�����������		����R�r���80����6��y����A�����/�u�#�|A�.m�y,'qr
%#�"m�V��������>�S�)��OFF�=	W0v�h�D��zW����_��'\J�t4�zl�=ya&
���6n����?����}��{�����m9+������8z!�,�������>Xi��Q����c�s�f��1�f�>�$�j����{��}�f�q���0h�3��s�v�wj��khb'-����D-��DI1
0S^�2�'��r��|oi��*%m�-rO��w��YK��I'g�fz�5��38}�Q���)������~-g��$ �q���1�M��r�lhR���=�rey-|��t;�3w�2�f���������	8�����3��i��g� ~�[�T�d}l��rv���
u�	�j�K�^~��\FM�X���lp�+�2Oh����{��.��<D��8B^y�_��$�HE5���bV4����$}dD�� Y�1��i�No"n�%H�J�r��v)��\�t�='Z����U�T���~:����+YV�Q��<�7�q�2KS�`R����w���K�>�CZ�P���K�b�C=2
���"�?k��5^�������P�[��g�w9�O��NBko@J'���v���A@w}����w��=O�����#.6�s�>FD�G�p��2�������_��C_����$J��FDEn��H����(���<�y|�	�Cxp<��]#v���������_������
[BJ�"����`�Ou����j���f�b4s�����]��m$9TAi��U4/������%��}��\Z9����oB�v*7K���x�r�Bp�����'�������(���������o���3�F��i!�'k=!L��`�N�����������v��(��M�6�&�	��!#%���?	j��>�m��=m&����']5��9":�����T�r���]"�&�t���	#�)�dV�9�J�r|�qK���A�u��Lq���	��<�1h�&7��_)Q�8	�3bZ�w�����*�\uC���D�;�b��u��-�bl	�d6dYBi,��#�`�������k������k��^:�>��|	��YkHri�����u����9��!����w��dHi���<����h�����8���?��g��� �V��.��F���U�G����ZT���(��rBz]Nw�JK��{�r�M�q����������:��e����^�-�3��#�y��
�����%��V���X����sF$�����
�3^MC�3���nq$�Q����UHF�����ms���g
zj��;SiU,d����B!����|2a}|il���p������� #�����>�FwG>9��c�>���Z���Ei7M��V\��[�zwh?q��]�<�M�o����#�p��D�T�
/=)��VT�D��*���,nP�&1���rwn�%b
��r��+��q�����sI�O�gi
Q��9�jQt��������$�p���U����X���=~������u�?��a,�ns�P���|}�9��w����="�����9V��s*�I��j�xwl^'�����Jm"���w����;�����6M�qB�W����C�fB������������7Bi*�r�A?�_��:<��-�F�����$4&.�-m���L��!G���0�zV���w�p�>���.1J�F����zih]����T�'��>l!b?����7�������M�-���1P�����$����m>��~
n��
��c5��-�}���H��
	�2��]-����p��$�,F`�o���`
60i}�7J_E-!8��,G@�4���Y��n�%����U��������������7��L��0G�n��<��0sA[M��j�
~�	�|���u�X�y�m��Q���c�
�{4+����4$���GTseBp9x�}�eXK���K���i��Bj����R�P�,���`i(g�Jb��|x�L���$�z��*�,��l�G�b��3���w���C�Ns�f�N�c�_B*LX_rqoK���.Q�D4v����}ZXX���w�|�D�.�\�=�mCZXkXlUP��_�{}��w�z��	���:	�e���F�h����J�S�7��������j� %��%�&�N���D�
�D5|�zZ�7�%a���z��fI�l���1iW-W�������9]1���W�0��P�J�C�� �e�+T��_���u�W�������ex9��
"�S����
`[9\����m��](or�`�K9���{h�8�L�d���������p-��f�7���N�b"P��xG�@5����������_��Oat�����k�Z.6��
~.ZI�y����[o��vs|}
P}��9�S&)�p����5�g���+b�.����,y�R4���,�UB�5������3bb+�'��+Y���������"��9�C�$4��U.o�Gsp���+^��?���[��X��E����m�)E�y�4�������
;�r�*��wdx���$�e�B#;V���sX���R��K���y}/�r���^����l��0)�XN�I���'�0{�PR���X:h����;4L��Z�������r��F(��Hxa���Z����_���;�}`v�����S�#�t$����Y�'���>�Or+7���O���}���R�O�
"%t�!-�W"J�*�-}�����}w���N!�S��d7NGj�,r�2�	�c�C����\b��b ���2�1<�����Q��5I2�C_�m�;<CA�g��gN$�)�����"[A��<�F,����'��e0���R��^L-���&[����+�������j�(c��o�t,��.�$�N���k�~��
7����5�����2�2:�-q����$�j�l-#n���#H�������s�&z�L���@���1|���E�R� P��[+@���y�3{���E$+����T=���5��i]�t�����2�F!a%��F��������������kS��	��I�eH�H�`���j�47l>�XW�%����+d���B�1m�7��*�@#;�_��9���{F|Y��ylm,�9k�&��kXNV
��>��r��i�"������OvZNg��cwX��	A
���f����<ys�c�64x�����_���*�'��
m����3W2p�Wl��gu�+=���)��*8�#u�)�) �gpe sqqv����5i�E9
�+��O=�?Nk��Q�T\&��,R4�,����q�������>��}`���V���*a�.�4�%��
}���F��������c��|�:���)�����|Q5�����Or����rH�	��ewx�w}�x5��Dqr`��dc"�8K,n8�XGk�
��t�^&����d�Nw	aB���-��O��/L]cC
1�~n|�Y��[7�7n��h5�d�h#���$��9���p���8
\��R����y@�=�bKF�*)V����$���
��������N���z*�W �9�u���=n����6L]��X�'���a��1�B�q���T��`��-�������>`��6v����f0�����1_�s$]�;����������3#����K�?]��Q�s�G����b�V�k3� �������RA�<���-d���k���g]���C����
�N�.���F��r�<J��d���%l#�np��
�
�^�[_��K��:x����CI��U���4�_s������/T)iKT�{HDG�Ru�7������6bP�B�\n��d&��d�2�jtJ	`�ve�`'xb"�L������w�^�N���]��m4�ep�6�=�]
O���pq������DO���!�w��Oo��A2���$%�8L����K<!���S���\�2}���x��/_���
y�6��p��k��t��g�ux�h8 k��p����8p������2��Z��B>�2��'���9	�N-z��&�W+]`�{�JuUT�������[3��v������w	���U���lR�_n$���,�qI���X�6��
������@�����1��s�KO���P'�Z��O2+_����x���?���7�A�m����_�^�����|,A�#�\�	��b���YAD��C������-�T7��a��&�_J�$�� @c���i�>��P
m�]�.}:��&f�M�D��w�"f�O]��a}��v�\��Pa���J�E�����W^�=�Ux�hJZ��4�U�+����.�G���0����*��-G�k�Qq���|5���!.�`�e�)l�"��V��7�F�jH��p>�*�<�N�]��~w��w��x�5������1���=����-�W����c��TN��^�A2U���������_���(�D'��1`�w������)fp��8��y0"�R�Q$&��"�����0��
Po]�*��G%xbcJT���������}8lw��7���DmGg�eV/;bH'�9�d�w&�xY�U�J���f�,!�
W���^�i-_Q[�pu�<�`��?����r��4���/S
���~���0��Z�W@�H_�|��/s�w����:jc�{����w}�z|~�&�*�{8�Q����e�hC����HJ�������z~�S��hq>��.�"JO���M�8�^�,v������E�����BW��PDE��DG�2�G�0Z�[m~�� 9�q��h�&ZB�Z83%-H���J���^���=�2����oD�
�V$!?�/����1�{N$�)�c-��	���Z���/�8�i���&$���jiW=@�%��#�\��_��K��l7����r��9��{k26�<���e����eMx��P��u��c��s����|4rx��inB��Z>_v��0��z��n���=��.�|�l���@��~�:kn-�4����a�s�J�������]���S{MLf4���\|�z����-\��������~��!R�)��0I���E6�D)�(A8Q�����/���SJT�B����f�����������?u>>��_/�q��s���_4z.�M���$��5�(Lm@�N;���lf������+�^ E�#��Sw.Kn�X~/��`�~Y�e�[��[Rt��2��,��b��E��~��,&N��8��W-Z�������jf��;���\(Q��f��Z�h�a	_Og�	'g���^<��g*��s4��f�{ ��6���Fe��
��'9��g�m~����[7\A�gg�h��uP��E�D/q�6*(�fm �h<R����������`�N�(�\����<�K�����v%D31�u������:�j9o��rP�S5��K���s�rW���+:��[��z��5*��q��U����c3Y�VH������p]/c�
D W4�W�����g��)�!���=����3S��cl�o$jd�� �xs�lA��H�!�q#����������Y^�$�LfL�
d�h]xa*���������T�r�G��r����JWAG�j�@��)j��T�Y�.�:���^,\����7����=x�;3��ur~�\�_���}���@8b��s|��t\m�
? ]���@���a�u����qk@��:��q��iz8;�_��B�;|0��t���#�D��4�������o���������Z��a���Dl|�/��
���zs��gN�?�p��@���M\�RDq�T%T�����[Y��d�@��A�+M��5��_������k��S��Wo����z�_���}D��2�Y%�R<�X=�������9^�bu�*�?��7��G
M�J�p�r��=�<IO]@����dM��h���3���I�bR&���g�k9�K5=t�)h�i����Ia!3��U�>���px~D��a�-��2�.�A4��f2;���G�#L2pQ�B�b!�Qy���S��~8^��o=������Kp���r��������
�m��>�	�f�NV3c-K���kf�k�'J�����L�0��&���Y��� g��o#6�a�X-<���K���>�)���$�������<w�#
�-�����v�V��(�����z�~x
��E�b��3���8�
w�
����G�q�*?�I���]7�P��B,���'H����F�]<����|�A�.�D��f<~!+h
_{�[�W�p�c�S��W�����TA~���z�Q@���O�h����U.�}5���Q�X�Z^���9��-�J+^gGA���z]��j�
$�h��������9T�K�X��D�����6�~� QG��ER�+fE��85�s���6��9��e,nQ�
�)]���������5?y���m��F"0ui�M��"�2w�/:l����^�Q���g��r<�4A�J����e5W2��4����6TA�P����h�$KG��nc���$�L-�c���Z^y�{��S������h6g?���m��x�5dDcYf��nK�����
�����R�������<i>w?Q�
�KFJ�"���o��K=k�D����CO{a�XTN��9W[���������z��P��;���N��P��I��@~UK�#<s(�/��J�����(8(�|�mB"=q�#����9� fu�.�<N~����m��������q[�����-J��o����C�^A0�)g9D�7�s%�F-G+U7/�Z���Tt��J�
C�J�����V�|I�9}�����~��f����v��
�y��k_`�6[u":�O��zv�O���$]I'Y,�+�BWG��q�m��n��j/_�T�	���r=���T_`Usu�\qq*/�R��>�� p\NS���3~�I�'A������XL������=	�2E���'$j^\�1o�
�p�]W��H�����r)��f��Phx��������k�}�Qa	��T��S���������B�� z�����=���������������V�5�m��)�L,��6��
jCI��r�:b<�_w�02��m����ej�.���b������["	��	����]�V{����u�w�<�6p�������s-��^T�B�k@�M
��`�B�?����_b�����7HF�u��PR|] R���z�d�����D�R��j�}@"���s�#��|�	w���8��P�(���i�)�p�w�����T���+���������d�Iv�����EdM���3U������8���/j�yb8<���
�b�R|�j�jf��@":YM������e��4��)�{-��/�}�����_8$�KN$5�sg�`i����5�L1_�X��9�H%���8��z�b�������Y�������Pt��~����Q��dj���Ls_K�=:�������ZHGg3P8�u���h�'� '���'{�B�XM2�a.���Y_�R�}��=��C{�P@��������ocQz;]���}vgOm����}�{�+g\1���7�b���o�"H�h�wx5�]d�mW����+.��.����Bk��	����k���v����c�t9����,*��
Aq�����{H%t-4����}���-d�O��2&?��&����W.8���O;��p�t�����Zh���BOV:��{<�
�_�9������]����=`���e^J�87�CI���XEE�������G�� �R�qv$T������Drd�\�f�1]�#��Z�1E��js�������f~+��6�1�r.�-$�F������/�~����mJ�H�B�E���k�����0�t=r�.Gy��-���{���R�M�F�yo�pgc�L��G5���$���ij����eE#��4�tz+��[��\�,N�{�GugUn�/���kS�,�H�|2�%���<��1�W C��}���������p@�G~�v�j���� �'�����E6{�Vw��	e*{9&�������!���
?N��?�<|~@�@���+��"�������a�/$`<'��JQ�u��U��BO'�dr'���\c��_8S�	�v� ?>#���I>����cU����������ZR�Q7�����o�G3p[�D����0�`����HK���	9�]����dD\�-�����u���
�V-$�ez���~[H--�����F��K�#�\-j����-�)�8�,��R6�GM���)������"�L��,�72l��od���X��������.}Z|pFB%���>j���F�*F�^��{k���S����G��j����L����Y����%���}�*x����nX����y��i��zv��JU�����4p �U���q;������35���D�AL9q�@���^A_n�}��{���V�C�����M[|>�[���D�}���b�O�5zTi�z1��������������"-�t.:�U~^v�vX/�4��r���D�n�im�����>a�������u����h0r�C��}������m��\��f�o8�m���s�h��f�a���n�v+�g$�&�d�.\C��TKt��4�w��\�v8��AK%��Z����"��i�_���D��as���C�?|Zc������PL0�C���3�����{�X���� m�K@R%�fi�Jq�c��S�`�~}<\�!(��On�2��%�c6bc�1�x��K&2`+x���H���0���������l���Y���~��k��s4Q��Q��OEX���	O�~��	��[:�W�\�Q)8��������-l|w�l�O����f�~-�q�`�kXW\m��y��i�"F$�V�����+je�g�1����r
�#�����
�QkY�@����qH�D�6�d�v�D-�8������o�6U�V)tTjw:�@��p����7:���)pqP�$
�g��i��7.��W�������M��@f�d63��EI��fp�K�A07�8%�_��|V�j���|Z?,�[U�nu:�������-�?cv�����R��w��w[�������P�,����z4m���t��~I���
zp=32�;m�J��et1[q����c1[��/gTr�z���!�.�O��&J��L� E�[��CF��f4g�&���h��pDG&�$;����h�?�v��+O�)�@r5����5hB���iBX���O�����DM���_B7{��W�~���	���~��)�������<�����+W�b`��� ����[M-�3#���lt�.���L�65�l�+��@9h����/������������a������j�z\eJ�������o��P�j
��)�t'TB=���CR���C���+���� D}����8l����x�I���Ddvr��L�}yo]hV��!�(�
*%��r�LSwhR�������K��\����U3�+�VT�����������������������7���q&\���rtma����t������n��_-�����R~�3�L^��z^_�l����	�S
�4��>���� ���|[I
3%V�;%�x��#)���ZZ 	��e�a�6q���4V41y��Y�i������X��27����g���F���
2.�����~�C}���9���:9W,�0���XC��g������O����)�,�$��E �'n2w.�I��HF��H�:8��������A�%]^��c,���e�J�|�G8�g��6�_RT[���������5V����5��vtg��
��������G�W���=�@B�|�r�v�X<W!���
7?T-��������,�-���1����kX�(���������n�-E^�����<6�h{��9o��U���Q�������_�}�����,��/]��?��\����#���6�f��qv$��?{��-?��/�KM��� ��Q�4nd���S�M�%�p����(����s����F��$��
N{�(=��TF����$�i�S�~��@\#]��g�ZN��jt��-5am��Zg��F���Tc���.[2�hKe�ie_Hc��E�Y`>B��$�xA~��ONt�����J���Flq��~�~9�'����
�6��{:��$��7/f/��#���
�ET���������?���v�s ���:$�2mtX����w2Q?��d�k�����B����pq�qRg�� ����
#����b��������;��>t��z�
H
�
���zi��~A$�5�q���5����/���m��������@$��)��lb>��z�Q��
�HQ�l=��T��-/���,]���"��������,S���O�MgGq��K�suj���%������'X��)�O&m�d�Q���
��i3M4�Q����3��UI���S��2I����d�5DQ�����Y�@�T���,������RB��<W�F6]��&d���@���
�sv$�'�$w���-4=
��i��:a1:5�c8a��Z>���z8�=mW��� ���^�.nt�b_1+Q�3x00;������7��Q!{&�����+~���"-���V.��eQ�l���������������.�z��g��o��$	W2V����_{:�$�s>v�'��Y��:�?H��E\��@�yN/����C���F��C���b$���p��jF��"�J-�v�C�.�/:�~�T	_�y��u��D*
-�������7�r�3M%��/�m�T����I_�M=��"U.5�a���n�F�{r����w���l4cPf�������]�P��v�}w�����2�e��R���BXYK0:����o���H1�`q��)�2<�2�U����KR��Z8Q� �F�T��w:-��ky���������.|4���=���>\F���V�G�E�O}�����c�.�����m}�uk�
H�����rg'zj�]�������*O�u�F�l�e��-k�"8Qp��,��t���X���7C]r��i�=)Q�N���U�4/Q�u^D:�Y0SK���?lO�e����������1�/kpr���=��@	�W5C����z�3��ib]?|1�����n�)��������J�/�V*iL��}H�v���Et��"}������W^�vpb�A�S�$���nV��������'c�G0���
U���B:�jY#�& ^:qxk�R.���BN�f��*�&�^���
�PZ�����v��-~6e����X>��W->�\v9M�.!�t��G?<nQ�����h�l�p�Xp04,�#V���@!%�0�y��17���b�C�I���E�_K4�h�~m"o�I�8�Y���R]6����r��q����c��.h<����-<A��K�
b}�}��Hh`���LnN�-����X�#�mB���	��(D-�*��Z������l���R4�x�)�	H,3h����?%�o���g��z}�E���~^�W��K�wxj��������"�u~�y|��$~vd%hq�6z���zu�xW��;o����@!����r�c��I'�u�|TPW���p>��}0]��n�8��%�#�����2�h�\������D��S����e�o@^�`$����9����G4��|��Ss�2�.�ly�|���d�!}=�1��k������ye��2n	����d
�b���h��������Xk�Dk�
� 1B��B�?�������a��������e>�p�D�Q-n�+���\���_���InI�}`To��������Z�l�����b�jT/�����	�b5L g6&
>�b�&��_~�"4��4�+��W�~O~&�Z%�i�@5]U^��>�����J}(�D.��*Y�B�_��Jc�t��~%@�!��q�.[Fb�D�>OTD[q�I�~^W_��:5�~PV�h�!��A�V��P"����$+�
���
�	j�+����t�Tb.��nH���r��
�3r���������gF���Zi��m�����,[)�#7��D�����ckO���s
��.	��z�����DUb<CX������>}<�t���5�V�2A.�T���R�������*���8fn����!���zQ���e,Dq�H�]�	�����mF�S{�����4�.-�U�&���8&	P���H�Nc�$
n����g4<)�f#2uR������q?�J�������
2
��S�����CAL\��d�4 �^l�L�K[�p�;��\d��Y����Z70$	'���>q&��D"�>��#�8+�m��c�DX�l��7�KOYg}�m/
���\���)d�k�zG.jt�j��i�=H�������m|��f����c��	����s����)
�<2
Oz;�������'�j��bTNl��L|����n�L��_� }��
W%��9��|���&�Q
;���0d���!�'�9��3�GO������#Weq�K���~)h�M�f���*?��x.��i{j������*����I(�:���Ws�������PU�f�>}���kX�P�M��,X�8c3m�H.�?�/S�*^�\:��~����J��fhr��m�)Gz>C��( `��g������GRB��61yH����]����"�H�\:�wM�"�������c&�����s�u�q�b'.��q.�+yq��>3��j.f
Z	��/	Z`�I�	��O���?K�D��:a�����N���.�'�qc������_^���4��(�<����e����;�-��RK���>K
I�&~��i�@�6:NT]_�U� ���~�U[�aB=������O���.rWa�'o�����#P�����LM.#P���v��1h��s�1�(����
���8�
a��_MKo�����ex�����<mh`�
�z����0��~�,�.���p�@|�E�l{e����6�g�|9{�����W���?�[���g��A`�a�dY�$H�H�fh�[��C����$�K)u���zi
A�*L����t��t;���������w�������v�\9��&yp����b�m,7Z�f���a�������Dd��K8��2|%�����8�2��AHgC6M�����P���e��t/"%e�E��l����+t��l���$Y��c"L��-���a�d�LR/�X:��o��O����8���c\\�!��d�E��C�9VBr��"�Z,9�9L�b���>w��!��\����B��O��M���V������}e6����\mm�^m�Y�������
l~�rc�r������'���w�v�kR��e�����nnT|��au<nP]�:�N���)�1;HH�>Y�U���Gj
m��9�����T��;I��$����t#o�AB�T��ftMtM�nE�n���8���*s�n^�������=�,l��"%�fNK�=�0�V��7]�}:�������������o�/�����Q��bd�O���b�q�Ps8�`c�]���W���k�� ?vV�Bp��
��p��:��dr)��vIn�}jv0��k�Y���0!TU(������N+���I.}{���:�}�a/8�A�4D�������;�b�AZ!�u���fs��i�����y��m�V�g!�#7�4����������k:��o����}h��E��O��_��p��^g!-5�����h�����(PTE"��g'�};��mk������� ��z������.E0���7��d�K]z��w�i�����myq���gh�Q&���H1F�e�D�k��o��%;l�<�"� ����a1Y�LJg����3p��#������\��$�G�����c�|�
�S	���L��g���?���w�����.(��MOF���)t�s9�rZ���e��f�l���/u�l��k����t�H�*��!W��fH��Ti�GP�4rMH`qBU:��*$c�j��Q�X������mZHn �a���=��=T�J�����0��P�E��K|�>���?��P����|J����xi�>�b�\�	�@<���M�=l�?���\%e����u6�8]��I�������c:|�n��":rB#

]9!�MI��>��:�����a#7���B��[9/�����*�[��Q^��������a��B���A�?	��"��0k���6����Y��|��I>�������J!���)�!Nf.p���8�:/}�ep]@EDt�ld��d%�����S��3�%�������Ge��RB�@c�n�!�
���S�3h�������D�����Y�^�.a��ArZ��e���o>�?tM����S�k���,a!LK�����c�
�$��7v����ck�!�f��D{�r��i��Rgj��7���ix���G��&]3;��^��w�������>g`���Q�{P`�#6�~��z����HX)����)���j��d�r�u�Y���y"�����2HSB&S�����p<l������o���<P�M�I��k���;=��B����r�/���-u!q��3����,Kl�n�r�M���|������v��D~F�s�GO��S0)2z���$�o���mOd��B0L����;��K��7���K4�,z�8����&�X���G�f~l�����v����M�{Xf��S?l//��G4}{	99H�/u1�e{	��M�z#wKad�x%������[]�����#
8��h�NF����h&��L��#�3J���4u����(���$X���x\��d9����� E�\>����&x�����@�X>��I��;��F&/j�������|�id|r�m����;�Z�LC��,s!�rj��	���W�(�L�lT�����.��fk��yX�������/
�B���v�C>��=�!�?��Mha-T�1��f�
}ij�g	�2����
����'$���.��]>E� ��J�\��\��>,��3��}�au��J��:B�K$}N��\f gu��Y���c�p��J����{4.���79A�v�5�Z�n����_�^�����w�/��4��R�g-�"�	.�id�������������m���%��aq����K*���������)C�+Rr,BK�`�
*����Ar��n���	��������F�Eq�.9�t����4K%r&�L���H2�LT��S'T���g�4Z�T���3;.��4:��i�5�6�����v��;9��0�����cF.��OS#)\�"*��%��I| "%�����c�� H�����+�R�&�#A{��>�u�H67+Cr)>�j�t����n�0����=_[��,d����C/)�\?p�qF�Y=���v�t���p��H��D�
����\��4r�4��7�� W��b���G��W��}���!2Lc�nE.����t|��Y��[3���C�m6����;���i�~Ya�'���{"��!�Dq,�I��x��?�W�p�K:!Cb2�(�F]A3�#i��|���ax��Y��k�M��9���\p�%�]� �&�5arEh.������M����w�g�
[��~�6	,����
��'S�~�#d~�M�����
�,=n�-����8N�+�_�e�U}�i��
"Z*�����Mu�lU/��*>}�s�_�w�i,w��+�M�o#���
B`�o�R%f�$6�R���Z"�P3�	}�/&h��������M��uI"(A���\�\&��!�Y8(��9N��[������*�����Ou������q�3j���=<X�3d�W��=2�F���Z�6���w�=������E������"O�%A�R��O�2���+�������}���}����;��u������/{!b?)=Y�����(�9dX��+���������q����X�b�xz�%g��E��$�3�$%�[������~��w���+���Z�[���P�M ��(����!�������<Z��QC�JO/$��1���w������}M�LL-x�[1�M��|�
]"�	�t.��+�3I���f��8�f��Zd��9]Q���^���������zpV=i�Jx�q������v�(Y�i�~��t�}�}����?���5�p��!�CXj1��$��8���+�R+�uz�����A�>3�W�����I���Dah\A����)\`�O9�`������ ���wgp��������g*L����b�!|���
��8��
r��j@�~�4��+�� (����!�wj�\�\`���\n�8=����R=��B��)c^����R����E�@�K����s,b��+�
&��\`�s%C�(�4�C����W����/�L��!vH���������J�w; ��������3,Tw������fh�����+��H�I	4C�m_����}p��$�Bm|��JF����@*TW���^�k.wj����}J!����d%Ce����H���W(�i�Jg�Z������F���L�Q�_��n3�;��o[Hn�a��4�����=	�\��QG$T�Y������2��0���bn����:u_�H��G\� ����Kp��r; "lc��s}��6���n�th6E��R��t�s������qN{�����}�y�������d�(��{�T1:�v`�1��+"rQt�S��7��?�I1���;�\�����������t�,N��2x����LUR�3���)R��]�����R�)��x���h����yW�(�lg��/���~I���KE����A�����%��0k����^��9�Y�^t�7Qh1��Hec���u�W�c�z�}/�\��C�1!�7�r�=_�@�.V�k��#R���S����~{��V��S�<�=8�� MzR\",8m�'7�knOU���������OY��C����N /��<��?��!Z��vn�M�9
>n�m}�X�"8} 7��A����@Y"/<�(�S
����z2[�����guS=��l���YC��`qTDq��y��?���2�Iq=������=5#!��W����^*�����ssQ����B��Q+��lUA,���������K��|}���(������P�����i��M��{@R������$��d����M����,���o*��H+�����7fB9�K���0�r�����vCJ�E�*A���`���L�3�b���*�y�|CJ����=FM�G��p�v2?}w�G&nz�4	*�e�bXg��J�m��
Qal/�3H���Tqn��������X���<u38���]s�4_��f��-=��~�m;�
��n�������8q��,�aj.4�Y���������=��2�4]�>���{e���M��v���K��Qc���D��l��+
����3e. Q����*��[�Xo��Zs���y��|���l���30�a�2�@_��|����c{w����qf��Ps)K��#��
�AHi���l�!��q����:������x�c^�+:$$��(�I���%����d6�0��y��,�{Q�o_Q��m�6rE�)MoGV�������~�	�FE �&�gB�/p=<�~������������7��N�k�\��P����S�^P&�T�����leh.��M�X�w��Q����pE��e9\<��Y��p��P����
}r�J�l9��f�����y9�Yu�����O�>���o;0=���o�
���8��>�a~�$�(�j�r�M.�B���|���	��S<����l�$	2q@��3ep�[�������-��3)���%`X����� ��e���#oy�q����D�I0aI�����J�r�K���G��"	6%�v\�%��I�	�%����������p"ri���������lY>�d}��tK$�I�������w����~���s�}zj�����O���� q>���R{pr6�@��Q`�~��0g�v��`��.�#�b�bDCU]�����+\���t��U����S�O���>7�_����?<���������m���,����_�V���3���
�N�j���LdJ�����,�Y��Z%��sq��Y��@&N�G��v4��`#p�}6����}�����%�B��{':����v]������y��S<�<a�N�O�*G��C�dP���*�<e�~�y��/�`Z�2�n2�M������	�!�Lc����/�������%H�8u<o�5�H������:N�\��!��[�y��������&7n��W���.���xe��*%�h{+/]=��1��n6%M~}p�D���$���r�B������������i�����-x�,�N�d�31��t����d9��x0@�q�g�-��"x�W$H%c>�&Sr$����p����'����G�o�?�<
�
`y*���b��p�N��y*���jvp��8�p�n���J��T�=��U{�=U�w���~���>������F����L���+�@e���I���a�!m��U������t��i�(rU$!�����|�u��k=���><�;��_���eM�b`hV`��v�N��\��%����^>~�6�<H�Yr�L���l��)=�j5�l�t�"e<+�����I�������w(_�@Z
4��U����H����|�c�tv�P�����%E����J���h��
���/�v���=-���q�#
�KjR�k�"��-��6�?� $�� B�t5D�����$
�xv�q��r������bAN���KX�ggM`��P������� �����\b����UP`��:W&LA�� ���]������T2�c��I���@�!%>�>H.o_�?�1�3II5t����T��v)0���,�WN)��
�W7�e�?>����n�)8��FT@�M���_K��Xv/���~��q��<��C�c�b-B��28Y�������BM��m�a��!����t�����o3�0$���FId�fE�[�����;�6��,�)��bIH��+#A#�k+�y���������
PeJJy��)����:J\;I�	\�D��������� ��4��u�^��X�e��;��K{<��=��`�a���������;�2����,$j^
�o���
����N�6�Dm�z�,^\�Fb
b��A+�z�)�m7�n�mpm�O�N��\#_���Q�lr�f���w
<�����^U�)��:�e{%K=m��HG9kf��
v�`��6ko�2�����
V������PX��n'f�|�(U ,
(��X0��l�E����#�������y�����</�������<l9��_��Z@�5�g\�������������!��}��������v����9�����B_���V�1�X��0�����O�/���������m�������(X�@���tF�����F�K���f��o�\�-HSQip�H%�V��XFk�"���Z��������[%���8��k���3��<�3Vj�'|����~�9��?��~\���N����{�L�|9R4pc�)4�-������0h�@��7��m?z����� '��*IWIw�	���Zh-������� ���K�>[�3"C��p�`���\`����	��8}��]�Z�������e���n�Dm�$@���v�=��*���i�q��]�_��n4vY���A.���m�D ��`�V�&��a�EQ��p9E��|}�F���@��`$���n�l\Q��*^`�'�y�\n�S�
�7h�Y���,�_Is���$���N��n�����a�j��wt���<��B�06|(t!���o��?��>����k�X&	�g�@�#���<B2�ER�����������[��m�|�w5�GR4K�f�iHG �h����ia�=���|>��J~B�"�c-�{I�
�� �{	|\ZK����]ls�C�n�\��l0��+}�(����H<�o��������]&�x(����(_����ym�r�g�������yku��Fj���|�3!
�&P��mv�xK����,0m�;z�s�p��C����G���<����@�&"M���M�_;e��@c��^�te����:�5����k�x�N��
��.�c�NXJ��Tp�m-�x�c
�1�]���IY��2�j�uC�������i\�x���&�.MT�=:<���U�kqQ�H|QW��(�����-g�d8V2DDE��` ��>
��[��)�
��(��@q��;�Br?I����?�$2�<f���H/��1��"dt5�,.W����*��d�-�$m�42�Q���fE��������V�h$^r��
��@`hT���������o�_�6dh�!���1�������9���db��zN����HY�DE��{�G��qv�7�V4Ip����D7B��E������8��������7� �fQ�x��&^��Vm��N�I�[�<t��o������s�Jz5�N����	t�� e�:�b�V��D1��\R��}=m���*U�&MOu�m�q�w�E-�W��$y�!������������	��
��%�nN�S�������6������t���{a���K�!|�LKh��l9��7,�OE���H���C0���#���v�m�r��>?�ga5�]�����PwJL�q/<?Dxw��k�8�����o����5���'$n2�Y�1d4�4�C��Jg��M��������l&:�O.����<?g�/)3I�/	�L��y�q���I������9�d�$�+�1���QR����}gle�Y��Y���W�u�4�$�v�(V����@u���`�}{����5�`W�CD	U�R
�p�M
N�N��b�.��~������e,8�[-�W8%�\W&j1[������
�B�Y���jLnt���$���=L)d�!W����Wp�Xd��l$v��uH
��_UDB�w��2�$^����c��6FI�T`L�/���?��,*;"4�`�6�q<!@\�	1���R�MM�
b�� s����Vt7��v[��[��{�{�Z����G���]>��q���E-�����j��+�F������Z�&sWn�W���/S�\�y7 	���K�P2����#�9�8/4�*����>���a��������$?12!"������I�����Z���,M��G	�F�nn+���t�E-��s��������u%���I#\��j�j��3�QW32,����4��GC��R��1���l,�Y	[K��l-���c����Lj���OCuA��Y�z��e�v�����@�=��u�'sh�����Ms����ZQ]�5����#kB�1W�\B
�]�d�zGTC&�T.�(�����Y����.#����9`��Or�_C#����|�k���2���C�V�h8��K��F���g�����D>�y{��_Nz��l�T#p�G���Z.��TZ ^Q*����H�(�
�og_�
�������������V������V�Q��jN���l
�����V�+3�(�KH�J�{�Jw%�$��ue+Ej����~�}M��y9��&L#U�V�p�@��<��������|�U���T�p1�A\[D���YCu��F;t_��a���?�������O,�m�(�"��d)��E�z��!}��V{?�Y0���������;J7cGR�M�W�8�;�&����-x�M��:����2����!3&��L���s7�0jfi���n�g�=����:w�
�Ya�\f�'w�dQ�T)hp�� j~��u�dW:��4��9D�
�f&�9G�9
>�6�{�J�����n��F#��+��z�D|t��'�P�X�� ���n>����QIo�r��\ )���
g�JQ����|����T��o,���)S��h%��v����d��"a���)��%��v?���,�O����gp9�B���'��Y�q����1���S���m [��2��|�F��)�%�����i����*6�V�<W��4�g�?�l8
��nS^���o����M�~�[��$�M�2��r��6��p�q;�v�����hDvb���h�|jd��#���jn�EP������S�[%���������<����������C��~w����������S��.h��
����=I.G���O��=X��~x:�����>�_�V�`$�vNa��#9���
C�F��6A�]���d�:�?�D�xc�����?S��	D������F��.bq�X��������e���gLu���o���C=r_���K)lH](��A:��.�RX5A��������w=�R��K��H�mu���2�Ma������c���.���(���{��yw������~��EE7t��r*��i-��R��En}�gg�C����z�<���.���)���W�(oJ�e��Mq��T�K��<��|���
��������eI5�?�
B�|�<�������V��}=N�����?Dc���	1u�S������`��Wr�b���t���H������>��>��C��n��?�3��G]���A:EJ�Q!�;J����g�\��y�Gz<�O��`[���hC4���.������	�|&d�9���E��������5���_ O>~�{��Y���
y�4�G�<��	�&P4<d`������S�/t~ 5�����3�c�M!�jEI-k��S{���m7����dUXT����pj�'$%��<�
��t'�\���%������tw~���|N��w���
��`��gjc�f{�����8���,�T0�B��=4�������W��
5yCw#�5\��;Wr�d$l�k�x|�����!�+��@JD(H�:� t�q?I�����5A�g���[c��9fd ��l*aX� �����y6{xJ����Q�E	%���
J����?@��j�J@���o���0y��GuW�d�'�UT���������L�mE
�Sw�7u�0OP�E6�X)���~�pNgi�h�!`;��C�d%���4q�84�g�
"��=�0%�����q&��UoF��D����Z z�z���p7v����U��?��(�G��F���n8�������N������:v^ u�:��'9`��,��JA���O����%�!���Kp>��`0����j>��
�=���L^�1��ess).��n�`3���������V����d�|����qk�6���(;u)aC$-��7'�����>H���m4C��OYrO$n�|�j>��q��������\�JR
`����2�X����7�m���H)i$[�DFb(�	R~z�GK�=���5��t���N��>`��xT��wA,�������|4�����9o��������<?w��wGpg�k��{�N+Utf���������txpm����9�!���J��t�+7���9���[�2,�����w�H�C�"���#��UC��e���%�(I�����
�����c|��C�8�`����_��.�����M�.���G'p�<F�6���s��|0�;C�W�\�� b�f �����u�4��i���>��������vQ�i,�J�7�%M;������5���e6�������������e��B�e=������=�_��nw�I54u���a&����v/)nN�deI-h4��5��X+�J�Y����s��$*w-��u��5�����3�1����VzTX �.���\��}������x<���� e���g��)!�F�$�zV������D-�+� 0QF���� Pi��L��
p]��|v���~��,��Q�+#3y�_R^��)��n��������u�������r�g?n��BA�$e��`�S����k��	�
����Gw��w��"�2MyI��C��!��Y�RAi����*�m��s+���Qy=��,r��z����KT>�I������q��2�q��h5W0�<�!���?m�|��	������k��B������y7q�U4f	����'�	E���L��B��
)g�"�i3y��z2T�D�^��}89m�������B�uc��?y������4������\��&J��1�8���������?
���/�~x��v�S��L���?'��<c�[MrJ��0�j�e����U�6^xvZ[�H=G?,;�X�h6h�R=k�3��<OVRRF�|r�$\���#*7(�j���5��������s��'bY����8/�����E���*�W�����.��b6�m/�WI:�P(���L�������f�SD`���%��[W����������P�M-�R(�?��X$����QKS\��&E��)����+�����8��nw��# MJ,�����r��4i�����V���q������h+�����Fw�;�1:��U��/����:cd��
na��u�*��t����H�l:�^0��YWH�)*I(9��3������(�F�(%$.��`�X�p����w�HR�, u������~=
]��CJ����D|,�,�����`0�J8�9�{�`#�k�/����C~R%0aQ��������d-��~�j"��S�>v��*�|�*��!a)��p�������*Q_��`����}}���$T.	X�x��E�@��P�:e&�{��x�1j7��s�b������&�7Eye(q���!�EF�d��l�Dr �V3�/�����#��,;V��=�����m�_|^9(%�	�)���z��5�0�T���~<]%�yn�+	!d���q�V�W���S�{������S��<�u��#������>�|?nkkdC�06$����������e�-���o�dK)&���h03j;�����{�����-�:U��1����{��v#!��"IEI������/�_��%<��C��QX�`b��m��>�0�����������`0�p������n/��'}�Wk���B�-C����A��t����N�����a��aVIZg�d0�_-s�)�)��6";e��8;�#d�m�5��^�3F��C��V�a�"~�/�"�������+"IH�������O��l~����p���4@��c�P�]-KT;�Q68)�����p�o�\��0�P�H��NQQ"�W�������t��}�T]�8h6V0�D ��s4J������ E~����m?q������V�#W=�����^�&>���1wN�FZ�V_:F)
'I��&$�f� a?�w�qI#��qC�`0��B���������KN��b`���P~�W��	C�cu�i��B.xu\�6�V����������\>��F�Ct��|nZ�&o��$���}&�Znl6��[�b�MXcT)�����/J�4
9�^L�s�{Ln?��j�;�1�����
���t���aH�Y��e<�3L��,��f�����0�B��S��T�`{�"/�U&@��Vg��7b��U�C���\A��m���/_k� ��K��C]%�$l�M]>?������C����=i���i���X�����F?4j�C�����]��U~����I��������{�W��� ���v��o��jt,�w�RD�F�Nx-9��kT)�\>���l�U�MN�N�W��}�]_\e����$��x��8u6��o��T����ln�b��1.(��K�>�*yI��f;����0����/���s�1_C��p���3�`�"n��G%��=�3���9�{~�?j���s���&����No��@g`�]~��KS_����j���� ����E�N39���7���|mlnUP���
�cE#�O��g�1�Q��X��C�������4�>�M���`h����J�a-4o����L�I��z���C���7�}�:���97E�O��Dm��s�z�,������n19������q���EA�~�'��fI�s1���Y~���#����O]a<�}TQ���f}4��Yt�(
���<�O�������H�_ECRq?�����!%��K��+��2�{�
(\Z�qm@!��cqG���������d�vh��LoW8I*A�5�(�#���������1,���F}fL4�&� kO�k�����z�!"����ji"	c����f����
�'����K��fLF��������$4=�Z��G5}��)|���S�e�b0�*�$^T�d&����+t4����Z���i��>c��!!K��B�$YFan���"�%�l�"�J�@�Yh��o�?��LF��('+z'�Vs��slq]��	Sgk��)su���5�	�]S��`xu.&XKc�0
�p��y�w���M��'gjjI�Z�,"LxGiTY��dq|�e�=^�����8�c�[W7��x�a��]�[�!��|CQ� t�������_6u�5��V��}�8F{��x8�V1��"-����$����1�g3D����?��x�2�Y��@�Xx^h+��5Hb����k	w������v�]�x���l��h0��iCT�*�����i��,c�R��Y.'��$���6�����(�<k^O�v���U��JV�u��v��Ibl��]�����^pr���2���e��Bh�L�
=��������0��\��_���3�P�&�;��Gr
��P��q��|<V�~����h����k��B�u��ey�E���yz��f�}�����&���C��*�������v�
#>�]�����/�[�]����Qk/���)aP!
��^J�����?���B����	�!#��DDZ�!��:*-�>U�
'Jhl���O@w�����%�������
��)%��l��7��,��������DB�7��	�qE�����-�i��&s�}~��7vb����Ep>�@{y,���Ojd���<�&����1f��kc]�(�&
�<���b�G�<���������-�*���T��z{>sm��6"�YH#�W�;�\�z�S��|�6v�C�qs���o6��R�rD�Q>#��
���
YxVCL:S�7N�2@��C{:w��gT��������3�j�Zm�R�
��*�]�c�p�A;�����T���!��=�d&��>�1�]J��q��������p��vpt�������"�fvKEL/����r6�/�!�QBO���������
}���/�0s(��E���-��q������&�
gy��������b����#�r�����&�������.��2�:�O��|�N���;���od�W}�'�"��Y`�������e�9x@����g���t���&\@k�16�G������!��`a���,<"�M|�J?q4`a&Il�
�N��YN���@"	p	}lhJC��q�`"����|x:b�0Wi9�����nuz\[��E�%L71����jR�3��zH��k��r ��b�j�;��y�q�H���;#t��*�%H3�K�Zj���b�[g�eP	�B�7����*���#N���=m� d0�����\ZVpG������l��q���G�k
����3;�'�!��8�9���x*�i<;��ex\�]��9����#���Q��	�s"���ANF���FN�������q��9�%'��gs�(�/�)8�����@&������ss�la�c��"�k(�,c��(=D��Q����1$���'B�i����l2�#,�a2�X���!� :�-��J���KOc*��f	��'��C������+�S���]��S�����w�wx�G�.�Xi���w��&<�������h�hcu8��$}c�+���S�8��Y���1F���p�&�����7b.m��i��h=�v��p�C:xF\.k��p�&�K5=&�KD�����^sP�������S���r���2�b�&�X���m��������(��gSy3%B�?��_�����NO����J�����k�����������s)A	v�(�M�C��n���iU2��F H3����1�5x���'8���s7����R�Sf�j���s�]@�)�=4�����3������
�r+���`����tM�;Z��r�C�5��|�{�.O�����W���^�8��h�,4�4u��A���|�Z�y(9��3�Gb��x�i�dLX0�l8��`��|Zvd�d�����2�����2�J���tl��&�����U����8Q�6<�����Y&��1zJ�^��	���\T����2-�����{��sYOJb���'�u@>�����C"�X�)�949��+�����i,�g�����%��i����,Z��K.�i����^%7�r)H�}����%�-�u���U�[���y��d�FGM�=�P��_��1?F:b~��2���������F��^�fC�+	q����1"J�����������9�5�P��>JaD��W����
M����]���OLD+�����&B�;������M@���:����B����.pi�wC�:W$3F������K��v�������t|,7Z�E���WS�8�=��2��v�4�ALym��������4M�f����t��.��T6�4E���='o���{i�����P�.�l�H���l��e=�b���4(-�M�neB,�h_�:Xq8#i���aP���]|��pW����%�+�n\8��$��\Ys\���g0M�������K���=ytOds���(��/�����9#����|N�P�h@go��
l|��-�{�X��z�^��
�A[E�3�q�"��}��<4u=���@#����M�A6��wQ^o|FJ`�������x�&|��
\o��_ah�8�-�l�O/���#��pt(�+3��W��{�c�(W��^.;�IN�2tu����?��<�n��E�@�^K,�9�7-S���M��?�('���X@���"��&2*�7p4��� ���C���A��7�;0p#���JA{.s�~"�E~��T.8���0�#�'���WM]u�8��;���Q����	�������/	���ZA��%��fcp<u�V��1�D�w�u�
�4���z��f��M�p&$��1�p��2�sg���*����iZ��?����d��6C���r�5#uA���m��4{���~���{t����@�ZZ��W�3!��t�+4��]�_��}�[���}[}<����Aa��T��� ���K��4���6�5\,�yg!�MK�,C�������E���N��h���Q����������:
�N���7H7��	9�,	/z� US���������:av��@�TaE��c=���������w�i�<Oa���:���n4�M�B�������}W_���w���������J�1L
5ADV��e��%+�����lx17|{�<�����ojM4Vl���v#
���#��w���s3L��L��uT,����!�����������I"���^	�;��
����
�,��Kh�6\�Rj�.h�a��f����6l�e�a��q�T��5)r�0���u�������5����:z1l���14��bj���zY�%�]����Oj��SW�����u��^����k�p��D\�y�B�����}n��>��'�I����"�X�pb���:����~���+6�uW_0C���*�8M�Y"�I�BF�)i���Y�
?m~�A�m�q��#���$U (L���sYE�8�p������x�r�#a� ���Z�R��{����M�g��3�#f�I�'��^�
��a����h�zDn�i,������Sp+�%.t��
�P]���
��~�q��h��/>�84j�n��������Zw�������n�������{y}��Q!|�����a������������a6@�X72�{�+��e���r����
x��
9��Y_�o,ab�V"�a��+5\JI���o��2`����e�����(E���3��L���S���?�M�������T�����}�ic���DU��5A>�
�m3��T~>�t�P����-�T�?�|�[V(��4O
Z����2��
��{z�AA�e�)��#��c��Y�j(�4����y������)<�n�a�WQ"�"�	��"J����5�����9��m��
�����&0e�u��(�u��D����a#�������A��t���:I� @����V'�a �\�t��j����n�:��)�����E��+PEE�������|��q��`����$��w�^��kZ���3���y����_�7��7I����xf]���[s,j�qz��%.2F>B��G��,��r�/���sf�&��I��m:2j���6��t��P5�u���QZ��1<�	����a������b�m.�q�%[�x�F����7n>y����r�}.,�S�����Pnk�c�Jv�Z�����h7�
���6.lI�~�����&�h�2���5k����[	G	;-������<��i{��O����Z��c��*
KF]���Z?gL\���{�^:l���?�}� \�.
'K����
���	IO�tSZr���	1��s�G�F�����Q:[xC���IS�\�f�,��s<��^\����<1�AM��_��b�Vp�0�"��x��pT<��x6x�,�������Az����t��9q�����qk�������0�>p.z�.��s7��0�����/����@��R�\�V���i������N��01������6*�g��
���l��)�l�'��F��Yeg�������Z&�����M��b�#�����v??=��|=������<*��*�����6�
GUU����0�����������c�@:���is�*�?����"37
����|8>6���p.�_;���O��{���!I�`s6_e��������-!3�e^��	~w���z�����I�H�""�(�a����H�b� ����Zw�+���U�\b+�Xq^�UY�[b_XA4����fe�,��9�K�;���
��+������?c> b���?�l��E�>�[�Ga��_�������{|�B�?#4,����8����S����-���P#���'(��_�(��T{RF�G|��cl��
%��<?��ey9�==���RO_\N�|����rt�'}�����\�~��I3@=�z��B�0QG��	V�m�]m{*m��F�
\��f'7yH�6�R�O��H��b���q��e8KWh7�>:J��bR#\�wf��(SI����`��|kBAP�"�J�H@Xb&��EOs��Rl���J������|+��A�����7�G����ncU.�.:=��.��s,hf���}03h�C�+=z"�\h�-���+O��!�%�O����|F_���CC���O�!�2�}���������\,DD��S�g�R�b�7J�r1��<^����)��iR�c.B�OuQ�
�g�L�n��G�����][��6��+y��P������d{7�x��`�IPW��5��U*'����*�K�4��F�0p�	����|��{��9�����	P�l�|W�}���4$�9,�������i���}��
w���,�T4w�]��%�iJ�{3��	][�b�t����? i�3����������Sd��b^�}�������H��,�<���4��M��2y����+���a�{~��s
���i��`K���Ozn���m����a�F,���0s�H�������k������]�Lk�%�LX��v�t��@�EL�1��_���I_���*##t��A@5���(���Y�op^����9v�<����#3?���,����6S��?�?<.��h_�FC����&3}���R$4�p�<;����u?z��a�N+��T��\=��92��J�`�XU�u��=y����?���� �����g�hJ:L������xq���S��Y���?m���>��?�g�]1����$vLc�]U(���BF�����B8J�"�&�4Y������R������z�a�Yu�*)�u����d0,w�U���7���������������|�F!���D�a�J?�,'��������v��������*eL�;c
k�K�G�U�	�e��*�<��}_7}�>�N ��429�2gO�`�&��{�H��0w~n:�
�����hFC
I�I�%�{�Y��O�a�^g���$_�����f�1AY$�wRK1*�nd���[��df������fo0�}4	yR�b�Q�������JM�y@��k�%��%�2X`��2��1
������z�Q�GW�O}����k�������
�At��3���aY$PP��l�b|<W��9��}8��`��@�����f�	)��I��9�r+J���rvV��D�hD��x
"X9TU���X���5���,�;�Y��O;����S�Q��0�R��	:28y��)�gs�.�E����d����UzjX��)���2(4�S�k(E���\�5~��!��'��6��C����f��Y��i�b�"�V:{)������P�h�_P�O��s�:����T������)�����t0�=�O����U!�<�D�d���c�����T)����Q0�5_e���U����}�Z/e����y�Yj����\���*����!���9i(����`�
�K�����M���O|�&��qQ}���%�=V<B�Os��4i=g�sYo�N~Q���.O#/��&r���	w�����������zH�1���wN*����~I�/`�����z�����#�F�N���)tu��4Xq_n4���'�������U�B����������]���TfN�����)�V��R�y��Rn���M���
���f4�T�����H,l14%����V���"W��X��"���f��`w����T����D��������J"��D���^��z��MR�v����Dy�����������Y�ia��x���I��$I���LU��{��A�T�-[��[$�`����z��ae�2"�$e�B�	|�;G��,-F[p���;u4�aF��3����O�zP�k�,XRc������K/{Q�����W��XM?��h���n��
[�@�\L�q��
���M8�.�)�#�>!"%] �������RxH��������)�@��u��&aDIr�c��� �. ����m���w�_����<������I��/W�����X�q���Q����r����>�!b�Q0J_�
[^J5_������}�������,�/�w���C5@PU��1:��!5�}7��0_����8����*0+�:��V>�*0����S��i]�|W��f�����?zCU����������.)��>�����y]l�Y)�������f�a���'v�_�[����}I�����W*�\9���4�����5rL��)�m/�� g"�v@��23��V��1{e�g�"I������y�������M�"��'��`8�$�]����(�Qo�r�][�A��?7�<7�4�RS���x��5Eg���H������>�������$��y��q�0�jJP��Y��l���]Y�����pS�)�5:CBv�}UqX����BbE�u��������sB�
���]w��(:��������@���$sGh4],��s[���D�5��)�������Z�#�z��C�z��i)���2���~�~R��(��b�j�(�������@zM��.G)r�U���7�5|��7|0��/)(u�Wl����I�
17}�^K��.p���$���=,|b�@s�tF�:!�S�Z���q	{�������n����!���?Ba��gT)��2�B^)�,En�XhZ
J���m����g��������,��-[q�r����(���M���*D~UA		Q���'��pm���e��A���R�kmY����Jf����$�X�#N������}�[n���
���W���r.]�[�'�tWp�En'�k6dt�+������%���<l����2;r�NZA�B�G> �o�������#|�FOI��#\D�0h��U@��~t0�-�P��3��;�	�U�+2R�'r������J�:IEdp7*�Yt���o�c�[�{`��G.�kI�-����"}��{,	)���6�{��_���-�-��+���HY�Vl����������\G���A�M^�~2n�Ur7B����R�?K�8u#���������L�"���	��!��Q��11��\��c��-*��B�@�*��|��R:
)��L���
%OQ����rx����pD~���K#q�3 u���HD���x��f��j��[ow��`���7�����^^<�<m	�+f���6r������;�5��u��;���w�'��	!��(�MH�i�4:�����$<d�P�������w~gl$�������������&�����������r2�wg&����(g��
����1�����d1�.'���j�/x��q&�X�[��p�����{D~�E���F_��k�~����������$�%�RT�%4?CTVb�%dK��0���~�{�r]<��2����cG��6��R�v�6���~��a����Q�V����8S@'4��Fz��%3y�^�U��o��mAy���T�w�e=�p���^UA`G}��2R(�����K����0jD��,�,�;�5P���~6�5xE��S����;R�fk��k�)r�F-�����������0h���8 ���cl��Ca�W�����/.�E����}�j�����`'�O��{��	y������R���n�i���~lN��"~%��5�L��|*i��<�I.�"�
�O�o�c��_�!<��'���� "�l��W���ygJ�����g����������������������$��������w���Y����;�?���9k�SX���n
��5:(-
�=�+7	�|,(%$H'�!��k*������$�~)�A�v�V{�G���X��g��!OuG��tH-���i��������M���t�XA���8L:�F
	����������"��~����Iy~������i���A�X��TX�/��bp�Fg����F
~Q�Z��N7_.���:l|��5��6����_��!�3PY
���m@vq�������[J�e������E8aW��N�z�B�;����g~<=��a��vC�4�<��$O/v#C�T�

��������k���>�qJ����ZE�ssgI-N_�]_$��jgB�����]���8	�������� H)��R��t��}��`�S�8�fR�.�W<g��6�E0�0�����l�i�p����������<�L�C��m��^d�#�1PT�R�EW��
�:��r-�����Q�)C��1�w�{Y�s�t�h@|�n�I����a�wPnv��T��C�8��rO�(�]��T��yh�m����l���_��=��V���x�o�S�tMNM��@#�7%]pR�z�]V�Vf�) ��H���J�>{a�B��-)D���R��4;F\!������tAh)+�LSj�O�QE���S���u������T|X'��zS/����K�J�N	�A����y_@������xX�j�c������J�(L����Hom
��b2��^����u���
.��<���i���)���s���w}��������U_�����M����H��!y)��������q��M�\���c��h���J)B��{/�!~��6��rQ$l�f�t`����3�P�
8������u��������>"o�)F����&�)���"����X)&5m�y�U�U�P������r��H��Axd:��G�^Mw��U�����j�����BDuT-��O�$3C�iv�Qrg�tGSa��w�����Mp��=#���n��k6UvF���	��EG*�������QC��������Ic:_��'d�0%_R��8���mh��nH�N����=�V=#����GD��T����?-���r��rF��������<$�S�\��)�x�_�[V��3z~�>I����\��A�R����<G`xa2{�A�7B�K�&����x��n]��v�����k�����4<:o
p���RJ�Mt%���@���w���xf*����e��=��G����{&^�������Oy&JV��6Y���m7�_iN�D��I�SU���O!��#���<t�n	�.>��d�ru	��p�Q�w�~Y���C����i>����6Tuw?�M\��/�z��������!�����.��h���������e>s^1�7��,������JD+��K��5L��s�usZ��:������p]i����+4i���+6�\�,��/�������U�n�y������p�� '$�&����~V�9|�Z�)�6j�I	��Z����[�4����,�{i�fay{5.U�:>=q[i�4�����c��}�j?
u]��j�>�A2���H�2��9�1�~��G�]�@�"t�K*�v5�pC8[�lW�E<�_x"O�9����7���\&���&*�s��5g���Pi+l��$	0q�7X@����1X|��3���1���->S���bEk���$��0�m�f��:���������?�1��]�3�DxtI�O�2Hs.HH6(�G�~'���p8��S�xs5VO�2>�sI!Ti�"�(�����$YP�Cn��3�m =�������`�&���@�_Uz����u�e�$)���k�Jry1�05}�|~���9gLhH;@P�x�j��}�O��iR���w�����\�wt.���|/��� ��W[E>�kb_���Q+�`��T�g 5D�R����Oa���o���q��8�z���~u�IG��H$2'�=o�<�z<����>�:��Y���	���
f��^�������r��v�w�5��~��/��cs���V�+x)�,	2c�|�T�� eB"�1�k,�}B����`�x��ag*!�T���]��e%/��Ih���H`1@���m������k[���UvYl�A�
Nk�uf
Bf�2XC@��Da��5pl�>OF�)��DE[����Qd�e4�����>�7�?X���p���Q�	Ti����x�I�c?q�����R�O��>��C�6j��4$���}���W�P��������Y]�p���GI����8����H�Rp�ms8���Uy5�����N��J+d,�d!������ gwZ�����u��
a����K������NC����E�va�W9��71��		���$9n�	�|�
�^J������6�������j{|Jz]����n�>��jv�����v�j�@�����������DL#%<i9�=D���"&��
^��W�=$O�zh��j�����p�H�X9fr���W0�����0��f�m��A;�y�T��e,�qD���` QC��@�`J�� ���R�v���-?s�� h�`�=�|d�v�C��I���z����������1���P�X�	����r�`����Z��������f�\e^�Bv�	�^F�L���S�H���=|���|C�?:�6���4b�8P��5��NAO��N1t������0.I>�3,;g��J��X��J���Z���������o����Q'4���������E(t�e����>����[�r������1-'|�L�a���	?���U�;�vS�l��)g��(�n���p���i���k��b#I43�?�>�)��<�`P������j��6���H��F��
����f����'�Y9y$�.>�%R����1 �#)�h]@���fw|�w�!����S��Tu��9P�YDF�t���"T��wZL4D�&�NH��.�+�(��X����'E���(�+7��{�K�\>���HJ���N~�!!H?I%���RJ:�+k����v?�l��I:��3�����m$K��9��b_�5���z���B�Zb����T$�*��y/�`�r����h(.�@I��x��oq��y���k��p?������W�������VB��(C��%��%�X����E��P�NZ���]��9�����T��Wy�L�4&#��D�S��
�������:lA�?�]h��
x�R�^���x����&pay.���$�??\2*����d�E2
��f����AS@��� �W�G�;2�}^=����[[�Bao8�D_4Y��(]���I���7$�qv\��CO(�I5h0v,F���W[G��x8�����\�mtm��R4ka����\���@�q�#�m?�_������\	���� eMl�+E�t/�4�|�	EO���a�?^�qW�����M�u^j#�����U�m���^@�#�{���:��7@�X���\���IKP���1������x@�.����}���p���&[]3��4����������H���C?���*��2�/�&���>���������,���R���;*��#�����Y��9�{
=��Mu(�+�p���-���09�������������gv��P'����/'��kH��O�Ky@�	������M�s�O����@�v��2-�nA�;��o���r�Z�^;TB�;���F�m�,�U��ym��#�/��^XN�6r�O���>S.B:���&Zajo��}}���N�B��|�������	�p35�Z�����q
�v
��
I��U��I��0_�f�� R=tm[��~��)�c�;���bm��}i���$M��Z��
z�tE'!���L�
S�g�L�	%����hV}o�dy)����q-2!���BX�H�]����4�x+��o�/%�n���|
���Rg�G�	�r3:�"��_��I���E���l��8�#c�97��V+�C��5�)��*zDm?~=tgyix�Nw*��	���$���z�Jm�/�?��;m��K��������H.��Dx��r
�.45)HiE�~�f���p(|h��q���H�_N���c<�b���ueQ+�3���0�%��
�J>]������1
�������zh��{����%R%�d#S�,	�������-:����2�8������f|S�P�����5�>;��>
vY�r�q�d����W7��_�{����H�S�`�T&�������Y�x���e{]������zF��pyqh'#�{=a]�<M	
Pn�a����
��+h/��+�W\b+����bR������>ZEoK0�A_N\Yh
W�[�{������Of�5� B�B�g>����.P����G��z���#�I��F������o������C*a{G�R��\2b������!��M��Os����_��K�oA��.9���_�sp&�!s��K�F)&ogHCV��F�2F�d��7
��
K���b2�~lu��e��5�7����{���1�7V8D"����X�������9�����>�mw9������L�����j�{$��bxmu
�x<���y�cB�y���(����"bFN+������*��rf������,VL�0��#��)7�c��q����?������j������'_�y��`S	M�n��b9�Q��7�J��N/`����Xom��^��Z�'h��N<�������;�V~�.Y��*�> �I�<Dy��~�y�]�)*�X�:&i��(��hWA�+�RS�R���6l
�JI�����������HDJ� w������� #�����Pp��*];�#93�"=gF�4����o/�J�D��<3_G����^�2"��;mA�~������;��yh���������2�p4�=s���1�
R�$���j#F&����DE�& 9�v��?�p��1���fj$	�u)\Q��F��"}�2���}�o�Gt��R��eQ����b�t1�i��6��T�s��������q���pO���������7��ro�z�|��5�yh����*����G�Q���X�7�@��!u �I*M��{f
�z}4��eq�Q@�^��3�^��i��n�A������,�\����+YYO��r�b�{�]��}��Z?����w��tW	�w�Q���;~/���SxWC���;�"��3���
Nq�I���|F���������<(��f$�����zQ��c��R�P�R���������1W���	�x��~yP�6b�h��M�X�0��s{8�&'�4Jf!q|������M�B2M����A_@����t��h
��K�.>��S�F�tK�B�P.�b�V�_���j�9rn����Yd,d��y-�Mjc�e�y|��������u��OU�0e6��7��%��~ �S��%�0�`�f
;�T�V��oDru�"��&���YJD�+"��q�����8��fE���_-��]%�N�&>��U�����K"�I*x�6�WX�)"�x<��+7����m����?��.�������
���R��qT+N�7�����6��z���S
XN�����m{Ej��?��27[�b�
��E�W,�@�����7�J�
A�����*=������G��U�J�d^oL1�����������
(�x���Q���'R�G�X�C�=��R=|�y������"[����D5l~�)���2���I��2���fc������w�0�/|��2�<��5�1�]i$��*��%��q�^@e���o�����j��Ut���b��/��Y�%H���Q�����J7G��������������}
���#-�+	n1������s�r��P����%\-����B�7H��V��J���r&�B�zm
a��@.&M$��-3b�C�w�|��i��?7�������Ga���.��P�Sy,^�I����O]���$���B�����:�fh�������t�N����?�s��4@����	��.E����\�,���&����(V
u��p����:~���+�*3'����N��X-�I��B�����y����Vz����x��Gm����{N���9�7j�x)����jl� �Ue����o�!��2��(QEM��C��~�=�S4�E ��������b���8q3A�X�����61����F9m���
�HfO��w6�I9|e��r�1�2�C����Q�������C���;���p���a�'���1�o[u�������866B�!|���8 ��~�N�u@�Q�+��&y����b���.�.���c���������y%���j`���gD�Z�(FQJ+�Me�{|����.l�B��6���X�n���\�_����������#��\����b[in���e����F���V�{}�\����y�����.��5[������RR���r]���.eA�P�O�c�����9-0������lf�rYq(4i�b���j	x�&r��Y�`�%��'�:p�i�k��Z��Z=��e\��VDd�,�3�.GK��H`+ ��G():���n������kS�F���L�����������o���
�\����_<JSyC�-�sW����_�W4���(���K����9�
w�&;�'Y��}7\��?��p-��rwh���P
"O��,.���A��_����Z"
��R��%U�[�&Lg�����TN%�h/�.e�t#�S��>����+���{�����W���?��������wy`"�Q�pF,�X�!���Q���R�7qr#��H�O��~{���W
"��V�Jp����TbNHN=��L���#��8FR�i���������?_�SWR��$�2	NXJ���M�s��,FX����w��t/����Y^OM�8q���C����A4f�&�[�����������|V�2�W���7�����F���j�])�|��f��mW��@jC�y?I�.:Av]D����D��7�n����W��W&��C�F���������Q����$����+�od|8��� �5�I���U�}z����.��\���]�M�
m�!��v`e�x8����t9����Izk�XF�Tz�c�Y@��� }#�j������G�m�3'�em�yKH�:��'T2T:���BGN�Y+��O�e������X��QH2��+�C�W@��������jD+���. �`0�������I�Y����K�[�:pP��$��8G1��4��P?���9<Q���<
]����b����q�$�'Ji�~��6�7oJf����|v\���\��r�;$����n�,����?�M��q��+^�y�~fW��!��jW����f�u�6;��a&�/'����Z��!��#";��(f����	��������;��x��.g�����|%�����Rr:6<������>w(�0�Q,.��Y���RO���5��@���<��Q�5}�?��~�[��f�XQ�s�\�m����i&7XH�b�0�w������/��"�N�������iy���Rb=��4a��f�iX��U$��S*|:iv�����.q��=�� ������.�P./�q��U���3�������KQI���:��s���*S��M�`���JQ~�>�m��r�w�1����E<���(|b*g�3�]/�2[G�����xZx8�+nqY|Y;SI���qY��Cw�uPy���P��w���v��2���;�m=q���QJO�p�����3�K{|�5����H�I7w�Y*n}P���R������~�����oe��Af��;�`*�KI'&�9q	N�
�(�Y��s{�������_�������_�2?��F&�e���4D�ykL|T�S��/�<�a�vQ,�^�'�?�e�k�E�L6m����b�S���z������9���99:�\�[g��A���
�O�9�#��g�{�3���nJHsp���W"���ZB>�h����O+V����X%����&Z�J�"��D�����n9����$T7�f>Y�*E
o:%!G���R�C��	�o��������Q\_�������r�^1������������6�D������+�]K,=������866D�3��d.��%����I%M���sn-�
y���v&��=�d��>:�?��w��li�(5z�JI�8/`�t#c�z��s���#?�D��q��>�d��&M��l�+e?c�CBU���?�b�Y�gC���D$���L)������:����n����n��?�e�\}��
�61WC����=����q45n/�=�������?6������Y��Eb���
/��I�X����b4>������4�K�����["��1�>���s�7N��dCFK�<�������W�Em��}<O��Q�E�Uy����4��5�mh��6��1"z�Z�JL>3�>^�9/^�$Sc�M��5D,��nc���6�h��}�4N����!I��CC=!U)��m{��_��~�����}��`��Erz�����PN����A�F�����}u��BR��f�V���q�)+�9y2���]�;�#w���+x��:�J�����+���I���������7x667"H��.!��|�d�)3>��>~f�s:��|"����@oC���8)��<C�c>��u�{��wQUv��T��V����B�����'����b���C{��(�����GF��3����z�|5>6��$��A���0���~j�-��|n�v7VEo�+�p��c��#yl���$�P!>n�
��������<"��U%R0��!D�	C��������������o;��I�'6Q/NuY4�wHA$6����
(��/85�j&����\�Ci�Y,�+%���,��-�ml,��4��(��=?]�������W�]���>�<W�<���Q)YLlD1?�"!Mx;���l$'v�M#)�O���"��������\>�W�w�&�rY>N��8 %s�YAG	U��_���������yRVXbD���*L=)�xq�b�b����m����=�q��+���qp�WSq�GT.��MC���?uCs��U�w�^�`�<����i�W�������8�@A�*��~��(��>�������Q]a~53b�X%��O�'�������Z7��k������`��L�.f#�h&� %o
R5��w��>���e����Z�e�7-'�������,*��o??����S����2���S��6������X�������
��|
���rL'mJ�p�����'��H��e1n�O���������F�$)���$�m�������x�4%V�Z�����^����F��v�_�����s�{��W,�W8�iw�t�{O��JC%Z����i1��z��P��o�/E9cj�S���AAM�:��L�T �gj����n�z�nx&�m�I&��*J���+EZ�[�
J���~<���H�>�C��xRWZ+"R���t��(Bg?���.�
�/�bH���.$���J-�F��V�Xo��3���zv��W����79�����_�����s
����^��
C9qN�+(�3xNT	�C�,d�o}m�"0g"�
���O���/O(��7�M���*��-��&^O;����7�����4'�`q�2�{v���������kkr���%�y`�p'�6��z��d����'[b�U�D��l��P`�j������C�*��0.�9����?���?j�S�<�8��L����9���2���j��s|(��s���A�"$���Tjr�B�������b'��H�"�f����0�/�)��n{��:��'�{��GC��T���NU2?I��@���Zz�hz���?����2a�\
5�W�����$�� ]q�z�V`'����O���
'�i�l%{��t`%��R�L'G�`���C��_��<�N	se5��K�ag���������W��?���������S�=��=~��O��'��k���������������3t�h���DJ�����XX����T��3���o��P�A���t
=i��J���%�s��[��M�JO����|�S`���W�O�����]R��X�p`��[wn��N�����?��O5#����:X#-�'�h�HU&��Fz�?�C}_�������f��j.-(����\��&-�1�B�R(�H��N���w�W@M�-��~M���#�3���x���1EN)��8)�GP�nj�g�[u��/�������&xDf�>���8
8D>E���Nq�}l2��������r&!�2Y�t���Dd�tL]�P���F�I^z:�d2pL=>V���2`}&���u�N��N�d�;����hg�z����O���<�������a�)9�����c�Y+i���D�����lg��2�Es�&H6�dA�P*H0B��������-��n�*��@{��{�z��Yo��n�m�R#���Z���>������������X��k}|�������vR��l�M~4�������u��;��y=c�DK���H�+���{I��d��<��yw�l�,��#�.VF���	�&[���M�q�E�����z�����v�_�z����S���W����`��������U��r�x���[S�x���I1�)
��C��1��G��[��;:�'�wE9#�T�E�+�j	�|���s���v]��B���x.OiA�
P�����B�8&�MgW��a�#}����M���9O�DB�rOg�C���� �v(�u�*s�J/�L����4l���J���I�r��e���_rB������������`*���{9~��8v-�
�l^�C�������u4iX����HdB^Qe�a����
�5�M\q��:�.�#b�9A>�
�c��yt���LqE��A�:E����X-�@%��vV(��e.zX��{
��1������������!/���gz��J�&�[t�Wh��_���=T��=��W��3����>� $����`��"�V%��@����`e��>@����1�mhY���.$v
�[����n�@����s�?�u��c�g�e-Bg^���^���Rb>6������tl^6���9���DG�`����"mEt��N��eC
k�������-,_������n�4y���.����EI1j$�'�"�Uy������?��k����>�;I{� G������6���\��`�X��O&{C�.�Lb)������ �������d��r	f�H_��s����)gp��%d*�\�s���M����T	���9_�!������CJ��e$�������������r�
B�Q:)=r�.����h5����������X��1�(D���
�%G����X�T#&���T�@���������_�>M *I�4#�M���}�kW��*���m�/��JGy���R
�yU	�E�l����>�����~}|���oxu_�GWE�@����J�TWE�+
���p�r9��������^z�r&��`� �������#��?���u�O�8OW<�H�M�#U�2Yh��#��l.���������&n���O���/�Z���a\�	���\�CW��T�B&��O}b,u�p/����"�q��s��	�������5�IC"Nyi��-��eL�R�V&K�s�Jd!�9����d/�����wZ�q5�.�����w%X��jP����l�DA��W�,�����v�����!i�\r5Y85Bh�]/�����RL��M�������w���Nt$AM` _���M������}K��'g��[��t�4���9&��!����i�.�lB�n~�"��`\�!��KR�g�����q��y`�R�����z �H�C���?���H�2���m��]����h��FT�u�H��	b�P��bw��ov����\��~�6�y:�a�����yw�8���
� Y����K+KJ��"��3���2�O�3Ce�'��c'��!�P(\!�����nTq����[7�~={Ww��8������n3s�-�$���-���t���x&�C��1�$���x{�V~i���/E�����G��~��M�����	6?�
��,6��q�����l�
������V�_�gkDU����k�K�iP �t�ML��T��q]���WC[]0����2qO'��E��Dk#���������>��}����w����lw3Y���N�#�46>h�$���G����GrF��!���:�*I��V�`8�x�A��m���\��j�G�S�7�Fe��I��A�H��Dq��K�*:�o��\���M�nv;�K�I�IQ4MY<�%��q�>�~
�F�vg���������s}��!5�<O��LZ.���� ��R����2��������|R�������N>��C`���sX<�H�����8k��k6�u=4c�#���y�dO�&�����������M��f��X�'�Y������2]`	�n�����2x�w�}����_��SA�E������c?��\��v�0qp��	���W0#�R�������(�?N�F(��}`���m�������E���E�AZ<g�]�N�,[�-��Ve��y3��=���c����0qBJlTr��-�2��yq��H@0�������^���������~|��G���1S	BU��u�b��J�*p%����`���k��~�S�&���>LZ!��TbUfS#�������})r)�tk��
[��Z^H�E��7��(�s&|�xH���fE�`7p�=3@t�"���D�`�[�����/K��	���9���W��{&��������|����g+h�MR�����IY�K���&|�H�/��ztD����I�����il������a/�c���z�����0_���s�&/���B�����W�SM'yI�)k=�B�v�PHIc�@�~�v�#��I�qH!�e�=B�e�P�Ll����@-�V{ ��xz6Rc��6#s �l����%�����p��p��G\i(	����PV��F�9�gp�7!�Jd�J=7 Fwac���x�]���\Oq��4Q���D#OF�}����o}"@]:��4(g!��!/�n��:��I��))[�2���:Z�Y�2�BL[`��\��ta���qP�P��k9�Z���xh�k8
:����@���Z������.�g�����;wK�82g�BG�k�+QK����z�{������?�� �n�=6������Z�l��?��`*���*�*\���Q�.$b*�b�dY���p����m�����@.���a��EZvi�:E���0]������fZ�g�q�&��T��0�s�(t��ND����v�"�p�M��
��<�>2N���@'8/���I���`6{�����A9w�����m|����#��
,��K���\�;�}l]�R��l�1i��QD����lJ.{�W�W��-9���2���>���L�g
�`EV�����r��l��2�,�\�j�V(���/#n�r�*U�+�
{����D������c,-�U}�*LeB����AU@t:�R��\Z���'�d5[Jr��N��e�<8b+�j"^�~�s�w�Ey��BML��G�-��$a�#�#E�'af���AA��N����*��O���q�h=��
9��Ac)t0�aq��C�w%��t�T��N����i�1ea�H4���yN���u��mw�(��������]��&A
$�29D�+X�=p�ik��G���e�Q��S�|z��=����H!x6�!w�����������/����a�@1X3����2(��$�U}<wn:i���D���f��%Clh�)@��J���^�C>�;�d�����?�y����L�d�y��6^�j1�=������[�M99*^j���rLJ��k��x6V�����EE{�al�#���w{f�{s�c���k���c��>:���t?���yz��vbs�.�Y�l\�@
�<�V�E��x��%CV���8�S�[����3�]4�H�����s���B�*��������zd�2J��FSKr�V\�(,N����"p��Hq�V��)M�l�<�5���ET��Er<S	|���0Y&v1]��
Ub\�������b�"�O�8{���[�&�->��g�������T{`�W��WJTu?�������!"[��|�(��D�r�������]Ar[��H��'h0�m��� 4�,e���<��(�� GY��$����P��GW��n������]����F���RK�dW�]7�� �5�W"�����V�C����HiF1�8�"Hzc���T,>H����jUd�[�\��FZ}>����b�vh7v��9��4~�ffd�KG�N
��h.��3n����i���M��[o��H�I���FoZu���?n���������;��>3��S������i�3��ZT0�(�!���}�~�e\&��"����"��8���e�?�#�B�,���2<B����Bs�Eu�XQ����z��w��l7���_M�����4$ZI���H���(dH���Oz)r��/�OV�2�N2�������5�-^�UG�������Px���+��(��'��X��]�s:K�'U�e��@z��P�����*���LA��2A��&�7K��J�:W������j�El(��m�cJ�xBa���R)m�A���z8��6����48D������Y�B� #��F||�����dH_�!�����A��%Zd��X"B�2r��C.Y�]�p+��WC!����J,��S�}���X��)3J���hB�M:���NW�&�#g�A�<��
�z��ye�����)�v���k^��f���������C��	�E,R'h��D�9
Z��vA�(8n��`M�������'��(I}������*��a�V��@�%K�$��
c�J��8��6Lze���W��^Iy��[4��&�Je�&�e��t�HK8E|�_��YX����s������xP�(��B$���=l�2~�h�����8�A.q�����Lu��*�|Y��
j���M1`���a�]b�]n�z���#j�R���qgD�lQ
��mP��Bu!r���lw�-�F�B\�*(>:���J#C������	��������o���f}����m�U
���.�*��z�������m.u��f���?�C
����2���h8;��]$��X,���������d����r���ic_��[����_�Gl��\�A���L���1]l~�L�pvXK��w�\�c��������IQ�S`a��K�������*�<�#��?�gb��)d�i��#v���@�v�d���������8���5ID�����B��H��0������u}�@�nK]�z�OM{K��s�l�J���,��V��b�r�������P\Ei	��>�"��|�Q���4�N����1=�����v{ ������sY��L.���c�I�p$$EV��Q�?~hTU.h�[�\�4��f��<X:��\L�-�g�E��C��}a*��]�-�<��R�x�Ly��3�pz!���"v����2��	n�������X�q���f���l�b3��F)}��<rvi��!G��K�e�\��vS�B��%K�2��4��#���Y�5����
4(��_t������������H�|�#���n���_\I-���Do/��i��OO��-���g�!�����c@6�_��g�Q/�<"(�'�B�C��cS";�Th{��2���w'���a�&]�E�roX�BR�p�(���2��#�������rB�2@8�L &�Ei6�JW���|�N@;2<�����8���[�n/LT�C���d;��f��k���~�m�C�Z����"��K��d����T����C������z;8f!��R�%�!���JP��c�G���d�z�%O��C�=<�s-#{M���.��K^����P�.Jdt���w���}������.6Q���,H�;�{}@2{�+���}��������v��^��j0�$)Z���j-��������ecJ|i�|u]��r>�{��
) "{�Q;R�e?��0w�x
��J����{
�[rq7��,��F�J�-zn����q���v]��F��sO�J�Ls1F�"�����l8��!���t<��jo���M����qd�������~y��&�
�I��	��A�-�1����m���s���*w���C��tTU��]��a�b��iw���\������z���&H�}{i<T���i�0jV
<��S�a��j~<_���~�d�����Q������D��1���RFr�f�l���nM�m��2��8h�������Z01h����.������|Mp:?2����n~��h�w)n�+%+�<����#H����GI�$"�e�q&�����M�953��C�%p����������o�A����+Z��9O��E6~�q�$}&�������I����
]�=��_����6����QXV�TsZ�pC[
�=��CH������t� ��p"�*��m����Y����9�7��@.

�:?T�|E=�1Q�_Q�wR�Z�D�y�^���+��������f�2�'��?
����(����J�;�"ZS��_@j���x����d���&^y��-F�#�S�7���#&��;��'��>W"���zg������m^��Y�4��P�;Fl<����������[x6���������RqU<\k���C�����R*/�k����)�6�c:�<m�F"��:�XB��I�)��u*N���Q>>q��IC�������F�7����7�}m���kf{���:�g*�R���x<�L#f���-=������K��hYh��=����u���)m�Q�5<��*
w�S-�6a��	.V�(b��0G	���P�+�$���w�0cgt�G
���bz��	��_OF�8���m��ZmJ���I��2<�)��4,�EU�1)ND*J��>.�L1�^8�
�x�.�s�0&�A�.�c
����DEh�����v+�������x�&1�R'o�L���!��R"���������?f�0H�DTO����.���&TAi{�6��~;����y��0,�p@U����"�t��L�'
��R��z����U��6�\�
E�5����/K3N�����}�����.�Th��'I����t{FY	K����]J���u����`��/�'�h�{7�;D�f�<��zJ)Fk#�*d��H�FK�?�V~��3���z{T���_J�A�@��DZ+���w���``��8��.��h��pQ`���^�����Yc��>;��#����!k|jpw���f���M��N��	N���
O���A1�S���Td�0��_b��v���l�.P����;K����G���w���(S����=ps�v�v�?l�m�@�kN�4���g'qJ��
���&\|��6���������P�c����6�~�
�$+��
������P,�l��\�R��/�����gRVF
&�{cV�`��#;�� Q��8�@�W���������e��@N$p�����h�W{�t�!����q�p:??2����2!e�,l�8Q
�����a<m!�.����t$S�C9���\B��������c�/�>X8��=O2@�&*2�
#
��)h����j��
Jw9�w~�'-���$��4T�6��q�,��8���������-T�&������
��<`H�q�����c3�tT�B����U$k�K��/'9���t���H��x���5��r��S�M�t��G��BrH��]������%�ga�s��o�����!����2��OZ�2��&��e���K�I3�?
h'k�>��:��3|�OM�����A������u�=���S9���<��)�G�i�[>�P�V��[����T^��.�6T�r�9Z�T����N�������w��Cpv��bC�Z�t��6!�o��R��h���L;�����2���k$���25���7��}���8'f8�$��~;��b������%+x��.D$YO�Gh�7!z������=�BR��T�K6������:E�b���a���������������E���e�8�cY���82F�~��Y��o�:";~m���o������<dF@X��C*�3NQ-��F����9J��	�
\��;��C�N�"����_>���RfY�:�6r-��O����S���������.6��Q?�-3'}q�����H�D����T�^��)�A��Y��<����9HF�T0I4���D�P���q�x����yi{�:���]�&?$���(��OA�G��[/�%�6��H�n�Gb-������v��:����������U���l.��z�R��yt1g"�6Dh��Sd�$�����������~(�����Tm�D�����>�g�'�6��~�Wx�^�7�W(���e���f��������d����c�@9���WR;����J]��������;^��5�r��E���E�'���!T%��JU����7��������_��M�c������]�@��\D���K�
,}_[*%�.�c� ��Z��J+�K���{������}�����
�/���1��i�z��P�{��>�CSQJ�9@&��Y5��������V6��1�6����� [_
� �y-B�ys$�=�&O���18�>&^W���s������y��:����(L=
��!t���
�q���(V��>N�����R�� ��.�
s���.��4��.;��6Z6�N���-�K?���:����
��i��\u5Y�������t��(��M�@�z\
	>��Rz3\�D��Y
�$��"lA]���[�.��\��Xcq�V���{��DnG��wo*I�WG�*e ��\��\C�~r
��#Op�>bd�T���4�����������R]����D,�'S��C�m����� ��G�8QiBX_�u����p�������?X�����o�S�y)
D)D�?����'MbJX��+��x�J^�?)���`�7���u)���!����4�g������������q�p��
���e��W�4=�4te.�I��K�N8�s���FH:P��uq��#�}�������{tk�}�����)���?���m����J�q�>�]}4�~���]o�y�n�(��r�$�AK@R��v[@K�@wA8M>��LT4�?��i��H_����9w�B�����������3
���:�i�����D�k�c_T@��TA�B����9�����A��9M%xa�L�Q�
�)��
��������PN�|3dt�$���@YS����L�~E�����5w/��T,����`dO��s�����tt?����<���v��8���e�����9*%�U��������z�z�M���f��N��rm|��E(�H�S��JIC�f����US�����'*��Y(�S�%��@���PT/o�,e�����l���=�LIOq���gJ��&BL%�;`��������WD�p|e�M)�����~�@�"Q������y|T~������{��t���7�V�V��H�������c:�0�wE��������_�-����	S%��>G�qW>�d�:c�U:qJ?��T���_�wp��?����'\jn�y7�b^�v�a�i�����8��5��D��\x"hP��)�kV�.��Uf�B��	=�*�W���*4KZe^4����[���%+;�L
Ti������(�5���^m������^��uW���Me���=��i��o�(|���}��6mQ���'=Z���r�u�V����G��;
�+���ry�M��2n0]2�I��	�};��'+C\����@����.;�L�D���C��.>I� �H����G	?t>I�B�n��(�����n9p���O�DA[��7���[�������� ������$-�k�*��K������n��L��v,
���cO�lA�(!F�Dv)���qW�G7�j�k�O���K|q#���2��Da���?Oq�<�(���6�f�
����N?�����=��fd,1�����e��v���2l0@v������Q�q�t������/\�%Q�����n�Gs-��yQs)|r�9J��zV���m4�����i���ne��x��#N�OB���4)�
����?�o�9(:y%m���%����,U�3��\)�����Ks��_�����q�e=m�����a�4�����(UC���y���������n��q�4?Y�^z=%���
�CJ�(��-Q�B!����C�k�'[�������F�����9�������n�-ee<������H��O\Y����H]|�d!"�9������������gu���]��E���a��~F�*s��7<�X�.����k�=m�	���Tu�R���_8�e�QSF�sO�j:�������L^�J������*����492��Dl����R���l�VT�^<����ST�pb{���x�/��T����BoJ��j�,�F�~<�s��-�gU��a	��C��	n��_M��QJ4g��%J����!_'����
�!���8�?w�\���M�C��|)�����;�1c"6�h���&���q�l�|�J�
KDx	_���Ts{���
��f}��n�a�n��\o��?}��[�d��n��
��~�����
?�7��u}��dvw���
��r�������	~�RM�X��CI�Qj�
�W���^�E~t?6���*����*�����dUT	W�JN�h
��(P�H�w�;��#2,���y�x��i�#��	����l�z���0����Z5�%<��d��+O��.���R���/1<��W��SltR�k��R4�Gh�N��}�l��E���������;��i���kKI����e���c��7�������8F^g��]���L�*��g�]�s;�,�K��mg�&m72:��-�1�8,J�Q%��m���KX�;;d�����778_��h�,$��.y�u������Y���b�����������ZWo�`y����/��z��o��|��wyin�\�1���8dHC��H�.��/e���
h��aC�f�����#^Hr���7�D��`��K��.� �M�>�|�(��%"g��R0��^���x���<���(�9���X9"<���::3�n?�of�l�/\�������/�d\��<K)�f�,#"j<�7���Y4�<�������Z��Q5�/#u�����F)T!����+���~��]v<oD��h��
}�&�0
�w�J��O���eV�������SI�V�*i�O�E��������z�w��<S��r�T�D����G�KY�?5{m�����k�����7(�~r���)bg�]\`N��}d��#��Y���	|"%���t�f�L�
�x1R ��.�?����������"K���FQ�@b]��O���n�3����\�+�|�5V��&'��
��\����������=i�����%��@s�|)���x����}:R�x����1o�83\V�>6RD�|R��)`_��6�z�m�>�Ob)�M!�����>��a%�Oa���������4)z{�#a����*a�*��Q�XA�9-
����������������R2Z����A�����i�#��q��o7���}�<�M7y=#�=�d3Qa*A��0�������Kk
����,[��A#�|~4�{����0d������p0���GD��0��q�O�����&��
���q�t>I�`��r�����i?��,��N�u�$<�����'m��%�
/�!��f{��	<�����WL��6,��M��
C�e@�W�4B����3���y!{9�Q Kz<�TQ�����@Rf����d5.��������$�8
k�G����l*�<D��sp��)/����VCw�.]ur�)uHTe��f�b\�PW���,����s��#u���s�d~���mTm��Y�I����F�R�nu��]]#@k��g6�2�6
�	7z�R�PT��������z�?�|�PG�`���!�����LH1q��P���[�-u��^7(T���v���SHa������n�G�n����E��\�������h��K�	W��h#�����"�����(��lW��:�����������:���s�"�`nk�"����Z�od1�Lq���<L��0>��/�����Q�	�L�X8
��1x�%}��v����M������4�f)��yipHGU����*PH�1-]���?6����?v�=�C0k�S83gq�7>-1#\B:�q(1��H���eq�h��K����R��M���N��/����'�f'�_d)���3��Y5_f��n��G���BU�'���MNE�"�ue���%�"S<�/:�,[HI��2������.!E����Jc�c,$�2�`N���L��Q�/Fg�q�.��a5�6](����N">�I��Q	���F��7��%�iP�gP'�>h6�T������mKi-������h��d\��l7s(@d��.��"�!y��Cv�(�q
TOK���~~��_��b���#�P�<�R��]�����D�.U)�
��1s�����*�T�P{�������@dlB���-����	�,:b�LzeU �N�lN�=qX���T�������+�W���Q���2rla��+yc���n��QO�M�e���:E�9*�xv��0�f���9��.�������������;�.���vSF�f��1��Y/GRc�;-s�u��X���T���m�������|����z�Y,��Q���E`^���@E�E����g������?�U�NS���NgWx���K�:�H�Kvz����������mw��N���H����E����"A������=(n�8��w��=��� ������d*�eIp���e����8�?7;�r�K@Mzh:�q
]�e�d�+��}e��w	�
R�I�|���"�'W�Y�J:��E�kcR��q	�����5w�d�W�6�6T"�T�O�F���"���=�*8$����w_���U��)W�>�>���b�Y5~��}$&J�b�����A2��UAf5
��$ah��c'��k��
�IG�	���u]��Zr�"�y9���+���o���C����+J�Z7���;�eRXq�Ps��_���3��B�������i��]������2��.JW�q�&���l�{��>T�wW����3����$\�`n��
��#�K��W��|�x#u^�
d1�$����A�q�y�N�\��C[��Gw������Q`�����F�����073��2�����v���=����m �����N����QcC��2�eH��C���������V�2�k��;���!o�����#3�����XFB����}����Tx�S���\j�q�����
I^�~�>�/C�C��Gu����I:v�3$�������&�6��)��u�D�-���D�s������l�=<c��FA`��=D7���%�!�������n}�U<XP�(�d�`g����0�2�8���F2@�f�8��m��&��5����P��N_�t;���qo�K:��Y���aP
-S�]
�dT����/�Hj�i2>�5	�X6����W���n�����Z0x5�Z�ob����������5����7)�Nk�x.��/�������y<�����	�6,D��y^F����2����<UO��+��eK���Yu|����-1ZX���R�`"��3
��o���x���N�<���1]*�mj�7�m�Z+�������!)�.	�����5���0pX,@<��7G����4^��?M��������M]��o<��M� d��&�����/p�Kf� d��KR?t����7����<Q�0��d��B	6,��&��cs�w�z��{��2��E�&d\�u|}Lw��2�b������f��s@���d�h��h�>W�dX�3l6&W����a6'��/�����}_���)��ZJ�X�k��:��q)Lp�h�e�@j_(d���s\�O�Mk����0�b|�����L6_�Ny���0�0zJ�&i�FD�������1r�����k�F�9e��>Q��K�#�P�~��F�.#��<4��;����-�
<�K;o9yP�������SJ���y���������������U�gbr����h�Bj��Q�A��S���[
�:aSF"��U$�� ��@��o�x3���
��GwT�g�E�%����C���{`3i��[�v����>4���X�rP&�mV�VDt���Z��f�����:�d���>���*m�$�����<0�����v��ygiv���~���nrf�:��*,BL1^�p�%��su�D������4�O�t8����*WL���
^#A��P�@��Ez�k���R�^S�_�4�\V�(j�>"\X���~�a��	�^���8+�����o���K�\��(,����hv��-O!�~��Ssp��s1+EU Z��~�����:6�h���\�a����u9���}����,m@o����05�O#�N�\>��`���G�_|#��UpT����J�`\�A3�ss����J�(�ROb���@�������}��D� &d0��`1W��]����t�o�H`$��m��E���fD��2_�������w�������|����2-�kx�@�C5����c1��}����<b7$rpi{�&�sHN@���}�f�Xo����{Xt��,p�0��n~�)����dO)aQ���M��h�Z"w��"�#��� ���|��� d6^HV�Np�@7@����9�k����Zu���Z���
�R��GW��jp��t�m�a��d������aH1uH�B������������/I���.��h+�B!�����B GM.��s��Z�/~�n�Og��������P���?O���A���B��ymr�:�A
�*�=�Y��T�L|���D
g�
�o��,�������i����4XA���{wH�4�L�E��;�a���7���,n���cr��B������F�"�~�<�V�
��bv�&�a9w��A��{$
�����V�Ir$�F�fsq�	c@F6r���A��5�
���C �y�EW��t��\����V���!�n�����]����k���r��W
���/����m������A�����4��o�nx��9�-����.x��]{6�q���w�}����~�uI�����G%"(��9�+HI��9~kj����$�V�0��k�Hh|�m6!�0@/�fJ�M�t����%�;A�A_j	������G)U�Y�*2]��!�(��H�!u��
R�����O}��e�t��w�Rq�Ol-��B@n�~��m���������<=�03
6��4��T��K��w����j��������#c�m8��B��8pdL>}P5|,����vF�t^�#x[`6)q�P�����b��{�XOn�4k������v�l�\���\�{���}��7_��@��l`��]��W�ms41���U����l�jWS�#��x�f�}���7�G��������`�y���� �������!r��/�����m?]���i��f<f��X���'�p�u�gRL����B ����@�f:��_��GH7�S+��
�7�7���k���C}>��N��)\�$�9�������GE�
jJ@X������{�^M\��b��n I�,[�uV������M��5GS���T���a���k��O����TTn!��g�����ik�~�V����������9���6���M��v��g[���P�6�&�~�Y��kTO/y�4c�������X$�HU��2���V�&SV,?I�*Fq7�f��^���������w�&�����������9x�/��6�u�����{ ���c������k��Xn����fn�>�u�C0G�>q����A[�����F�v���K��Hd��Q"q��
���o0_�Dn�Q����^����5M�Nd�q;�
<(��L������l�{����p��?mri����~������k^��9���W^r��#��b;�9�#�a!�~���Wa$fC��{������\��u�$V��H��|���������T�-�� �5�
w;x��%]e��W3�+��;������K����`����Tw��<���a�,b�H�z9�+�&����W�Sy��i�:�!v�}@:��i�-��j��q����:�����_l����aEIe���W�V�� ��Jjv:�	�+�}7���#n0a#���\v������x����������`���(r���38��H^wm���P_e�(�^_���J������zOc��L$1{������6������or�$!�gy�~�<|�K����N�,�:"#6�l�4���p|amfP��my.R�=��
�S3�����a����'N�iQ/%LG���ab
�����������^��}P+M��As+I����`���:J��O.���cu>���8��l�;�H@����u�R��$2�l�Ru�������B����cH�"6�B5�rs�|��n*�B���[�;oz����������*m�~����������Z���r������p<���F�Ig=;[�n6f��HXK�Nf����Pu�:���v�/'`vmPK��T9jJ< ������&�T�Q�=�VV�ucAVs�����n,"$�b�!�yz��3�}��.y���M	��������x���o�.�R�'�s��X`�m�%�>],c2(9�������������@rvk��x2��� 8r����a���;�����hSp��rEc����kklj�%���i
�f2)n�:!�Q��[��������*��	n.Eap������Q��)�����2�M�i��!��\^��Y�u>����uh.���]�������c��v���G�t*�[z�S,BQ���9Wy3�	��R�VlF��T�����T+�s���Y�E�86iu�O�:�CR�KUO�����f���L�m��N/�g�������]���o�fWg�_u�����a�+(���'�������<�6�@�\f`�6��6����A�������~���=�rO��9aL�����X���������������P|pu����/�v���0��
��&�\�Fm������f$|�#�W��������K��+�~�|GU.���o/���	8s���; '�A��oADA/���Z
��U�=�a)�<L�u6�������tw���4�S~�kk��D��R]����O�F[�D����l���{1i|f�Jl���'���9G�j���y��7��t/��8��|�'Jy:4�S��V�5UA
A&9p�>w�� z�
f2��"
�ql����Y��F���V��N��k�	���|�G�gbn;�F�u�q�����x��/}�������>)�0��#Y:B��������@!��u4�,��>=����X��
n�V� �j��>����p���|i3���Bq��H��eD�Ih�����`��4���yv��>��'�E��rfp�^k�+�P�������N���x1!��u7!��7e20)�fDzj;/����B������v�4HZ�A��p�RU��Q��A��/��Ct�����,���/*]gl����M�����JL����(����% t�i�&#�8^:��B���S0�����r��b��/w���Ls�������VC�-8�ZK�4��cu�o�q� 
+�1�AG����a"�|�(�����1��?^�],�@�$BF)I���n��XJ���vmz	����k>{�]�
.r/������B�eB��s*��C�����	iC��Z�U���>�c����\�4��'�����z�`��/A�;Q53@�_��j�^�	b9��G�pV3�hO9�#�������nZ��3���"��&�B�����D�����WE
����W=63-��{\g�-R��lt�7��P�xsN�n$C>�����&s�N��j?�����/�Dbadh�7W�
��:5����W�������
�DD��V�L;.(P�&'zV����<(��$��>[���.��hh���}��)��D���!7<���=��]���i��]���
�:
qPbHt[�����;�������%�}yY���e������e���
�_0��6OB��eH�`&��2��fj�������%��z�[�{H�������#���
��-A��C	��f�[������D����g!,$�<-�Tr��ov?�0$�����l��W�?]sz|wzhA2�!W��v�`��E��n��2��?K�!�I�e�T�]��R6�������,�,(Ez2k�3X���v�f�����|���i��ye�~(v�`��<����G�����'s�����Tw��?��4z�i���D�s��U�-��?38��j����W�������?���%��������]
�V�EEF�=MW���������((��sU>�i4�lP	e�xB�/���M�
ua�8������!����t.@]�:�JC�B���*E�t�0�|E.z�����K�	>����!w���`����~_�~����J���s�K[����X��|!���8�H�3�?j�tz�P����u�
�B��Q"X����6�+�Z�y2�����?���������;�7���/r�D��JG>Y���R
I��T�@L���]��7�@H	-H�[�q���WK5[����`T�����p��|k_{%���)�,u��^��>����t�R��n_z����z0��@dP6���� ��Pfr�S������bM�B"EiHl����T
�
����\�|�D<
�}}<~�������e�j���S$<��T�3Y�����:�t��������a�e�8\]�"�l����oD��~��Kul���wo��lZ��x��o���((E����V=����K}�enN%�}����c���*��7P���<���A�E���>�t*��ls:�N�T�*.� d�'�h��UAR�����
\j�
��*R`������J��y)�(�j`����rD����U��;$�,y�:�,"v�f��3^���;@�K=o�D���%'#wU#al��NQ(@O�	��Av�}uo���>�/u��#�����<�A(�~y���d��"^�3���B����5�VU$�;��~y���T�El�ru�4�s�w�K����C��+�n��������Q�:M�L��~P��k��e��i�V��Y����j71�]
l��Iv|�'yN��$���D�P�����!���!���-q���'���'�P��xi+����}>
z�8.@Z
o�������GRV|i1�������\�����>��V���o���~=L '�����>o,b0�F���	i#i�U_ ������4�qy%I�+�$$7-C.WM�A_����D������{�R}�<Q��I�!�IVs�g��l��o�@Ze����c�};�n�)����5�*�-��	�&���+(C����YUJ����_E�Ewz���F�^7�s�`��0�f�������wp�9�]�	N�zY,���FC���n�'�`�+i�����t�`
D�+]!�������B�{	���?DY����IR��/��=��:�����
��F����p�R
��<5��z�u�����Fd$��^S�&?pC�2�#YW�4/���C�����?�-�3��]���K�d�f���B����N����R(�q�����Z:�E��,�c����:����EZs<$�$��������>�1�k?��0<LBb��CR�.���X;��V
�e��{�=x���~������uCv5���48��$���P�4k\f���r�u�/��������!	dI�l
}A�tEM�{(���1.2�%����
.T�����|��t��F�J����c.Iv��[�FH�\��S�`_��?��������v�wV�x��CG��U��\?�4���*�Z�dv���t[��^���tD��D����*���GqH���l/y�V6����k3��2l&��
��m����8D�we��h�@����f8��2Y� /�(���g��*Waz7u�#�aE|��r~�G�(R��M���Wr��^��G������(�,t�s�O){ Q��N����?�>�
$�y����Zu��i�GT�J1�{��n�t���?�������BYb�2K��f���HwF���Z ���peu7�+�����F�B��\��{�i�!m	$���RL��L0I!I^��$P����4g"$�pU
lw��*I#����D�']O�XrBb4�C
�����]�i�]-r^:5!UtILL9	pl6�t#D)����P7������;�����t���j0P���r���H����m��{���������~����Ul�"�Y���w�B��43K�qV�l����?���P�6��^i���q:F�H��:O�ZF"����G��'���v��q6��V�k,�����	��nX�7�-tK��"#�Z������)�@�-���HY��~J��� i��t��"������>����%��+�&^�D95�����]�t	�����;1��)5��}B�	�&U��]<)�a7�e6-�x�r�N	�\]�!~+)Z��3�������?�8�:"2����)f����O�`x���\��th,�6�b��]���\�,�XV�f8��M{ N����^)��@���*�ev2Eu�q��)j�uwLTX�����2�\R�2|S���������J��.�v���VF!nn�:A���7��y���ol!>�U����PFE%���,]:�{J�������\������o�mF4�
��|f�d=�[Vh��`��}��j��9�dp<�	s�f� �����w�-�Cy���v�� ��&��K5Vt��*�	��U�C���H�^)�����5�/�Y�����Gn�j2>��^�
��n�i_h2R�����M� S)������D��b����y7>,�)`+�^O��*k��%*�&!�����
����
�}��5�g(t��sv� ��[I(pH�}jE�Y��|8
�o���}j��C��;$m���1/�@p��� N='iw�B�h���ph���.��h�(�e��S���2q���<���������&4G��H'b����p�z���*h!�XM��,�q}��������c��:�c�6�r�6V���1�a�@$+����;�Eu;�]xE.�������k�r�T�aK��qO�����L��Wuk:��*��PU�$��{d�r���\$��V28�z=e���5
�`V
���$�nq�����}6)����h������c3���)��-i�is���dP}�3��6�����$h�����vw7���jZ���A~*��&��������_t��(��@b�������ug��v�*��'�������otJ\���C,^���3�*[���-#q�B�>.��^�vu����v7�(����[�QbX��8Y�X%���r^l\�J��Cj*�T������Q���&]}���N�f
������Y�U�����r�epJ�*�NKP�*7�@� ��`�[�sho���|���_������\`������nlGar�nB�����������4���jSi����K�1�@�*J�@��wg����>N	�����+)c�K9����4U��H���%G���P�����O�����`��NcV����zo+N����\���:4D���H����@��=Q���t_uF�[6&=��c"��d�o���F����}7��6`����6 ��e�v):���|	�.����)������2���h�����s�z�|�+�U�������t���O��?������}�n+!b�H'���������,��*�Y�P�����������?t�kk��'��an��Xe'q���"{1;����i��>S$m �<���3;�0,q�ne)��C�m�����k����q����������q���8��p�c�9)�7SZN3�|��<v\�,P��%C>����s�)@Ge�,���3t�������,��n~E�&*�*�*���@M:�J"��{
`�E-RO#!V��KW��%-.8������pO�l���s�[��0�����;�,�		(��c��K���O���[
4�>����F�`t�,.\�?4VP"I����'���i�3����`�����p!\m�)�����~r���]�����2�c:�:�w���!��������_����g�A�P������>�Jd��T��Z������@5VF$\����2�����>��`�$<��l���	W}�v"��rMOI��X�pA�1������I�[��S�O�����7�v��nn�����u��-O�.)	�]�R:��7����{�#��	����q����)���3j��	�@H��|`� *d���E�����������p����@�8gx��4��`U%p^�F�z��W�'��%I��*X�gA0�:p��*i��v�I>��IB��tC/����;
J7d��pO�R����xs
��+C�"����?"��z�pk �m�#�(���������7S%���c6�#��)�+�Y����~��8�d��
��zWq��U���K$�K��.�-.fp1�E����XSn���k_L�*�����l-E>�&������J;�E�{wHE��aI�p~�������K���~h���J-��50�����y��7���.)������o�|��N� �k�Ed������Y�$c7��R�����M�����}�����-�"��w�;�]c�L$���F:M���S6�?;��	=��<�w�I�22�AN!m��!�����f�n�}������pcq��)9�����GVs����.���n����q��=���^��t\=w�V �	LU����;���*����a�,MA��$����#]]a@��1H{X�%	@\	������<[��g7������P^),�K�?��y��������
���{��e%b��@&"^hl�~���\�	'}`J���Ge����F�����(�����7^�,y�s���0�c)��I*%������B%�������k�s�t���;��	^S��(&$6$��/*�M��dI���vch)���w�~���8��Ri�����=��r2��q��6!���{:�/����!kp8���la�t�Q�6�R���J )�(�\���*m����8�������� �X� (���F�f��(���F\wu*�uF����b�������������ar������	
���uBB��+
d.BL�#���;�������G���m�y3��k����7�cm:@������4M��n�K���|
��������g���������q�-b�����E�"�����!Q*�E_[����Xy5QvCs������06c��v#���zN���+�Q>���A%���$�������X�_�^&N��J��/f?t:���!d�fd�s�J`awYa�$u�.u��#ZI����Uas�t:\�?�R��+6������?���/�]����]�9��R��b��g�U3���I����:��5	Y���]K�Td�C�v�n�����bV@��_�/Cw�w�x�-D���k�#���7����$�0���o������1p�l�����i��5���L�\QkBJ ��VhA��i��M�B��"�56����
��dB*�@9��j�>�/M��c�^���\p5�V�:�^�x����i��G� ��FvD���b6�E��/�	]cuIZ@��Ji6s����s;����}�lr�H�;NfDt}��i�u�����b�l7rG�93�%yT���\���6r5<PT� c��2�����p�?���iO}I���R��Z� ��.%��5��/l��=����#�!� {�$��+��{a�V����o�����������v��a�	�n�$	�?R�K*�F�dV�)mw���>G��_�S����$��G>U�\c�@���u��WU�,|Qu)��C3����;>��+�y"v��W�j���s�
~}��z��QJ����vhG�@>��@%M��2Z�V�Q�3�u���On�|��#�n�kzR�l�+�u;� r���K���,���g��cJ������T�)���q�gbn����.��gm��2���n�D}'9�����^�:A� �<�����xu��p���������N��g��	H�($D�t���l�p����Ib�x����{$�<�$8a%��J��_E���2$�(EM(�L���'�xG�v�L]��~;�q�����
���e�n%��q�&,.L���(2~��fR��������Er���Ug�I
�#-Q��\��������������S��~��2|v]-����P�����\-?M�t��{<l�k�<����������DHr1'bCx)���f7�k���_���n%w/r��Mv���
��)�o��&���@p�����>���c2des�v%y�g����Xa�le�#w��V�o���o�O/����9���Y1f���������$�v�]=p�+���F�1�M���?e-������z:��;����0xo_m����qBA	�'R�c��g�P���t���"2��:j�X!�7����x2����y7>��|i����<�F��}�zB��8Z�Ay)���>v���s�?��Hgc�@Y����p��p'��n3o~�����=�f��`!�A��������-!S]#Y�F���,@@4�S����B�6��o6T{K�,k��|]�N��!�\5������p�����U����$-C���l� ����`�Q������z�����7�CiZ�2b�������!�=�0NJ����~���SV�H�k�a�����'.+�����XP�����@��^�2�O3:c�����a��>����8�tM�C-�F)t���G�.*���������W�H)3��_ocv�2t�	r�_���T����n���8-�;6@�2�qv���_a
�)�5�l0\W���C����q:SHP6����<����*fK� ���T�����{��}�U���dAL�4�L��V`I��m�,��p�th!���0N{.uqSr�I����*���s�j4!�^�?�\�n����U��������v��������aN�����^?c�n�?��n�������B�f�*���@Kt=�	x�BbK���K��<�'�z}�v�e����,�Vi�#�PeJ�|d
������[<�����7o�l�}�BL9�(F��"x�p�wO��������C��������.��E��1���SS�����z�}�WcB4�0��N~�u��+��k����g�
��`�v������dm,O/7�=U�%����%���%�,	��Ig���`�b*k`'�1@�
�����X'&���N�L� �(���	"��L��XR�������zf
���+
��G��}�	�<�}C��-p����LW�Lzd^�O��o��)������$�b��T�����@U�H�$����#�n;0������������������ss�&][ZR�5�.���!����
��f����'��_����D
��J�4	�S*��S�������O`��^"���}>��������c�W��y ��_���,;�w ���J�YeuK�^���"�Q����C�`F�Oo������w���g����l5G�/���������^�>P��s�}�JWV��FN���	��c&9ej?-�y6�.�b���a���RjQ�^>������d��7�R�1�Is�~�iQ9G�@ZzSLNV]��O���5�W.�l'�X���$��m^��!����x�����g�a��.6��b�*����U!���n8��fJ���s6�,����>B_aP5�0Vp�������g�7bs�(��b�I�W�esN�*��f�ZsS
�8����p��5�B�m}������1��x=y��6:�Xb����P���������=�g9��|�����;f��{*�
�M_}h�=�WgT)�tT�_��a�2+�BGm:EHp�"�EO^��2�h���PY�9{@}������M�����H���2@�
��Z�`9c}���nR��m������
�,F��k.;��QRu�g \���y_�@���DO
.�J�R����<4��Hm��;!���-���{54.GU���9�b(A�Q���q��X�v��zt��4���t�LgIO���.8 I~�GY���t\�[Q_�n�t@
��f��������
�����R���;����F�0�v=���b�L������G/��%WN��m4�
����g���q�(�������O��W�d���kG�sn�1����1>3"�]���r��
��zN�ip'�T@j�[�\�P�s�l����<�3�TH�3������$��f$+��4%^�*
�4,1K2�
(F���>���m�9����e�e�pr���%����Q	3��Ky<o[yWY5[���K�`X>���-{N�w.a2����0�q|��[�6}��+�g�]��8��+�M����LC���j8��0g��s���lF��w��g�����!�|���v���L�w��E����<;AP��� 
1�����H�l!��e��k�����)�M�X��P>?��������-b�(�����]��� �|����2��s�%
sab�����@Ll��6����3�_
�iM�sS79���g�MR�Aq�7��[q�y���48��(�v�^n*B�X�ar���*��08��sW���n�4�^ 3t�E1��S����V����@����y�9��<nj���6��"�/�?v�9��?���_����D|5�Ut���G��#fp
��J���|�����T�v"�<�k��K0vcZ-�%,�%0����v�%��T:��Z�6��d`E�� ����f}�?7��*�
3�e��*�l�$&
���T�O�&���8w����$��P�>�K,u�&nx�ID7oES�J<����|����}jO�n����'�q���z�,���m+����"�z����`�9�;�t`>�<�R�n�g�7?#.�t������z��@_�������m�|�m����la6�U*���E"��[�������(�^8M<���� ���(����-�$!���"Iox�KR��Yb�H�~*f�b^�%�rQ���WC���+�n)7�K����p������B�F��nl����(�����
��&�h�������������k�����azKzk�U�����#�ey5�Rw�
�`�]?��>6�d�*
rJo�$�{�-�Rz<&
���~�����>O� �#��$U���	5�M��^Q���\IQ�,?���r:>�_d���{��1��
�\3�C�����i"��p1G�Z��_��B���f����8+����n�H���4�<��������2�i7�#*mg�&4?*C�>��t���qP*���-������7�6(/7����!��4�0#'��2����J�9�V���y\�B�5|���-��R���t!��d�*���M���^@;�]�����D�����`D+�~%L)�]�^(*{��[�)&#����oh�Q�As�S�L�����o:�A�=g��-u!<}��Y���T�w��i�����g����{���r����l�M<K���I.$���������d���{����K��NqO�I~����s�&�i��?��>��z�p
��yw�+N���$�C��=�&��W�A��Y���5'[�IWQ]Q4rMt){��TY19_���!R�m���(�T�Km��a�
�Q��IB<n ��[7�a�G��`w����,���x4-�U)Sy]7���C��5|�u��W����=�i�D���2!V��v4w|�I������Mp��7�-���%�om��:Z���#�c�_��S����c������+�U�3�������g�j��� ����R��[�6-�S:"��	EB!d�\-�{z���/PCV����Y�L�4z�A�W���l�}�4bg�g0"�����0���z��k������O/)�gP�T�y.X�z,O�y�W��/F��6�����o�LK���������
��!b)��u������p�����C��(�0�,!������8�s���G���C��x�����*��E�M�"Ld���e�*wP��!w|�BN���Dg�����j�m7"��B����<y_�3Q����J��u�wU�v����^��"_�{P�B#��c'q��V��3D��m��7i�����;B�d\���#Q�!�0i�`4�t�4���4���Id��}�1����z%��4��8����
q&�8:��v=����-<h�Lz�����x���G�|�u=��MSQF��&B�KLE��KMdR0��������y���Y����_,�Fp5��-��O������Z����O�/���W��ir0z�QH�*���v�����/+���-DV�F�3��"��;
E}����B�x1�t}������������(D�+����X<�>�RS���[�b�7�.e3��s�<�&ab
�I��y%Vn�A��{��_[�F�u��'�}��LMl5��������IPn�u��a`�k��r�b�9���[N���=��OP����C�|�t��q���V �3�\p����-hW�+7"'�w�J��u�t9����
���F9S������"G������Y�A�-�W�:�3�%7T��u�7�K�3���}:�����V���D��k������T�i��.4�L�1��i�
�IN,�0YE,?8O�b����kv�����a0W���
&^���c�R�dg|�E �T����]Y��Ja�����j������3����d;X��&N��VN�b�q��"_q�c��pA�6���ST���m~v`��f���\V�&�2F��L����a�%V�D�>G
d0m�b�6�o����]s�\?6[�Wf8���*�HK����x9����T��v����������w��/���#Vx)|�
������5�X���6/Cq�9I�CGt��`eM)�C��^���}��:x>=��������T6.�-��Y���:6���F��@t�}���W]��H��J��d������'�1bBV��
����+���}N�*�dC�rG8���(�,_tC���%[$(s��b�$+��m|&JGj���RU���'�s�ud���������?�W��������Zd�-��8bj���bD.J�~��?�i�m������.�V����BP��4BR��Y�0/�L�M����^���y���[*�M>��2KK�u���b��y�����=�&����M��L38EP��6���}��Q8a0�������m��6[a=�I�7�X+L�M�W����e�*'�,N���W��b�����'�jx�f���HN���H/"vr(MDk|Je)��P�������-���zM-�
)��\6a�������"�C������g�m*�8�aT;�C�A�GL���[<����S�kU�X�������
���3(�N�D�q����N?^����LS�v�f9:����3j�'&���WS�R�R�`�']W�����W����|\�7�#\�f�(�WS��y�(�[&���f�q�q�D��`��������L��,o��:�,��|�2�ld=$S��`�Q_P�?c�r:��3��-Z~��b�g0�I]J����>]������~���nio4\Bx�Y
�(bf�l�����_����@\������jd�����^�g�q}$��v���_�o��l�=���f~�,��v���M�����h~���	����������O�f�E�(�|n����J<��?v�AfrU����w����03�����Y��phC%q
���u�Dc��5A����*N
[d�C�D��
^��I���iL���n��L	m'� J���%�������{�^/���R�T&h@�yCcK��������7���F�i����w���lT�]��*�1�W|"��AK���?'"U���/��>5_���-�]�HC��e�5�&q=k��(-eTH��t�?�,Om�zl����_��qf��W������������SCp�vms�<�BW6"��5������������l��~��R]o�����mO�
u=#�����1W�.paD����,������`��V���0���N��o,c���I��\)�
��:�u����D�F����
�r��/�a0`�"m9��k)J���q��=<����}����A*��h��vbx�r���#����V�i3�����,f�c�&i�^��]����8��WN���_J�l��D��r�z�R�����ZRV2�J��"�IK]��W�O������CzW�?a���Yq#��f��!�mV�������p�V��R��{9t�}���F����3����6VIb�=W+1�rp;��2Omw9����s��_h����<���yD��t��.�$M��Jd�� �|��<J4q�0�`�{�h1����Vw�|n�J�������/�]��9�Crp�]r��y#�i����W��E�iX&�����"_��$�1%��D����?	�b&���]�������|��yR�=�5�BD����������m�#56-j�lJ^����]��4	^����f�t:��#�'I!��?����Xo��8"���������r�g���)��v���.����LZv��P&�f"���|������|y�u��
����e�O�Z��E.(W���Z7��a�nv��W������
%4<i/}6�w�PR]%����0b�r��a��qU���1��z��%��*"8�]��jy�t��PMU�f�����s�g�=�~�#p��������������xbB�����+�k
����[���q�[�W�*������
C���(��M�N�!�/���<7���b�(�TL�?<�>jK��P����3����S��%rG�u��t�Z�
�����H�0w�5������	��<� �U�S���c28}.�z�K�����<��&i��?�
�Q>3=�<���rZ��?1n����������V�\UuWcy�������"�Hc���2�����
��������B�g^�����TD%�I0��Rx
��z�����/���H�I�~}C'xh0b//-[S������ynO�6�OH�6�L�yJ�THI���nbz�P���k����{������6����d��KM\�a^!s�|�c{������E�%���H���b�l)3���Z��M����� z�-N�rp��i��Z1]JWtm2'T��#�����L�
�	2�d��E��c���7Np�E�!=O��(���>�����]�
fI4�A��y�F,��j���8h���E3���2���R���|�.�q0A�������sZ_F��o}�/�id�#M�RIF�=����$l�
����6�.pv��n{~��������S�x:�A1�$,�*�~�W�d��	��~j�����s��o^b��u�q,%�tLe,�z������������q~[w
E]'8����N�l��y��CC{���Y���[�?�w�/����xwx<���+1�Q	���l�A�3M���2k/����
>����
�0�n��3N��i����xN#:�z�R��M� 2�tC�4K&�Y�������_���}��i*{OkK��������}�5���r3�G�����%'�p��l��+A��������{������
������GK������*�8����v�wGdk��n"����T"�C��qg��p��x�l�D����}F`�K~h��j+=#2�g!&�]�"����E
|��/q��*
m#$�X�����;�;���������/�m�g���^;�]���V��&���R�Bk��i����_�];1��W��J���h�=�d���	�cq��6����!���x�|I��Q�����3D�	=T=/�:��|�c��m����F�m���!�DHy*�=?�PQc2�&x��,��)
&�F��M<�Xr�O��{��w�ah�3�=�����Z{����h�����4�K�:���6?�]�������H�	
�a���8PIt.7�b�mO��=m���f�n�o���P�+�7��x^���x���i����g����[���3J��]�"k��&�4K>�mX��z�u���#�o(��8��C�U��dmSY������>�u���6�����@h���c&��"�.�F���(���x�VQjZ��l� �N.ql����=���Fh�\���yj��(���P��<=�z�3Oh�O�+���l�'v<���N�X�����_C�b��g��1"�R���nE<����E�?�3�Q*�V��rC���Pm;,������A����fZW��=�(,�B�����%��B�[I��N�:�
%�J�*���Z=I�AO9�����||�������e+mSY��]Yo�fS��>!q�e�]�J�1�c<�����F�&Hh9F45f��/i.o��K���X���>����N��}�M"�:���Og��=����������G���RE�,�~Q�@b��S�������NS�|���LTXhiY*'	
T�w%��uUl�rW�����
-���~���R,�"���?�=��M���|�����?��ctc����Z
����A���{����f-C��:��9������4�R�<�k�o�E��L�8
����.������0
�d��9�������aU}���x<o9�&�%TD^s4�+����hSJ�*��&����]E�������3��d%�=�d����BA�%+�����c�k���E�s�?�W,@o#������q�gdL4^Pe���5���<4��f��_�.�>�[���}�5i|���������M��Q+�a���!�Mm��`�Rd���6I4>���k���m0��c�������m���������N���|+�<r;����lx+P����U����L���Gs���8�����%�����)���T?�m�������)�����NV���7H�i)��������+�������S<V�~w��d�d�PYnl�w�T
&��E�J�}h6US|;�m�����wtP���dW�{�
�`%_�d�\��.>|2(IL�(��� �|��\*9���Zb��%�n����A��Z�2�B1��*u}2EJL�����c�gs���..G�<��}�����_e�����=M`����C�/�V������ �����K�r������@d���&��������{��n��H����z������&��%��"(`>"��g�����b���^F�4�d������j��P@����X&E|����.�����M�����|�|�Gy��Y\��H��M�>
]/�H�R&��y�CVq����!b;d�H�8�t�D��r<�EJ.��+����&��e�%�8z��.z�O{_6_�S6�dU|��������`���O�*k[B�������Qyi��t\D��������#e����r�������"i�M9�^X���d[������������1Bi-���:_I��tN�'im<�=%���m���b�����vp��r��W�0.�I�y)�����Z\d�^��)x������x�����F�����v�iN�\�Xv_�#����*�B�q]��(��v"�"
�b,g�j*x����m8��U��P����������dj����uBf<���5�L������g�����x��'�u l:4����\2T�wMY��|�����GMam`�9�����E��\E?^�����*�[�6�<nv��h���4�Tj6]@1��8|�j�N�H�,�L�^6�������R�������?V[���n�0Y?|z�3����HiRI9g6v�����:���nR��L�������Mj����#�t���Q�����A�?�91��
�>��L�Fz����vx��E:�U��������1��hFWp{B����xMm�hll�VV���_tt��� ��@�Hh� �
��J1�X]/*���Au�F���8@�@��3-�<q���A�������o{����-=Q��W?��D�� :�
������/{!/�jg���X�����(�TIFL��dD�_P�b,sj��x0#�L��K��f")�<!�D��}���f��[���0��.�=�����\�����[�y��9���TO�l�P!���2���*�lwM��jw��1l;�������bJ�,��(c��p#�����k49y��C�e�-�'���4�Lh�\4��x�wm��m��_��\�P�Wn�|�
;�gCa�;������qGE�i���?����z����Ml���rr���;h�W|���+��s'
@
��0C�q��be\*�J��l�W�f��7��b�j4�NV���]F#\�2��B�������m�����$.������� nJ���rE�)x�2��Y�
\��H��3@	�y)i>��t&8Q�-�He���������=���hs�vx�!:[���i���2.��q[Y�������VP����]���n���;���es���%�����k/2-D�K�'���@r�8M�Y/�(����r}|,���DqW�{������u4������J��h��R�fFI���hF��C����r��2����'A���������y��O>FR����wj�J��@�O���mG+6L9F���)m&�~W<�O=�,t�/��w��b������?���?����W��i��z�)���1�V,�Z�CS��Q��S]���#"z�BX�d�s�5�:��7,p.-Ri��U����y�9cd����4��'��2�BI�+�6�^��gS����[���1,!��"���hM�$�e����_[�J�4��9���a�����m H�I�7?�����eV������wL&r����ert.�<���vD�v��2	�
#������&U��	�?O��>?��we�n�v�l���PN�)���UO�O��C��_��T�up�>y3M��#R��ne]*��(�OU����aVov���+�&��Ca	8.��[�@�|p�Y~N���?��B����4�aU�8��!�cD,*~�<�A�����a]�m�DJ�
�l��G�*�R���<�$L�� ��p�h����@	��pj�U*~[?��}u�c��Z������&W���$QN�����[}3�����Fv���$PeW������C|���C(q������V���H�E�E�'N�_����������?v��c��/1�ft�
����4�tJ~���_�i��
/_�G!P7��!4�<���-�LM���<I�-�u@m����n���(/2R���]��fC,�n����WE��Rm��C�0�\�0��0���$
�
!hJ|Lj�I@�J�����K��:8����o��������3w��4N	���C��C���o�v���;���P���d�P�w����R�\"Si�_�F�8����=��B�"jU�2y���~w��b����+>#/��I��M�P�"J�r���������w����V
���
2pc.svg.gzapplication/gzip; name=2pc.svg.gzDownload
���V2pc.svg�\{w����[���!-��IX�s�N�:�O���u]\��@�E2�����]��D^G�i%�������w�g_�f1�cY���B5]!y$� Nv�$�����g���������"����������"��`��y5�|��\��5�^}�eZ����r�����f���Y0�Fa>���LF�6.�
���krP�,�q1��
���h2-.���&q�/�(���+�
�����tu��D'8�pJX]�_�:
 I���(�m#���
�S����s�:f�LpE�Ux��EG?H��k�N��L���.FZ������$c���O�Q!7iF��`yQ��R��>��u����6-`q�
f���9?Y��&'�OO���,���8bIA"@nbO�t������2k�C|Z�?y������0���R����F���v��\!��#}C����M�@����P�cF��T�`�b�9���,���<����0��[�t�r.��s�7Ou�� �"����<���e���s#�0��E�	g���o����/���pzrdd�� �s�� q8�I���l\�

����C �f���M.�8
3X�6a�����_���{�$Q��M����4��8�b�k� R� ��F�]#$����J��~L�e`-����J�(�a:|����d0 �t��s�9��Qr��@�{I:f�	���t	3!
L2����]4�4��� ^0 S^�����q���I���C��L��b#vXd��k.�X���2J��R��Ww���(/X���r��� Q��c��+��?5x~|���%��"��/~������&MW���'L�gy�d7�".z��?���\~�����B�5n@t?D���p��	l 8���3�hN�A��o��r��h��0��/H����b�L��Etv&%���O?D�Bh��3�#�I��E`���K����l�'_��h$�F���������u"���tK�,��yp�zr�*�)�{Tc���\9������?N�_� �[��7��4b�8�i�"_��{
��*��������}�|H��d��9]]�����X����g��������4����5����/N7����-\�S�~n��s8�!Y�1�]I��`}s6�7 ���%�g�I����!��[fv�1���cC������E/kl����>97�iw���R���
������_��O���r�E��J�h����6��qYN����q���	��r�G?3�L�/
��Dh]`��{�$����O �{��u�1F(�qn���`�MH\)^�'l�P?��d9�y�?
����G�!�z1��&���!�t�����.�M9�Nuu��=7.��98���y��>�duf���0�H.ZW���Q����NM��$��|j��c(���i:��3~�3���6���E���
�A���Bg�'�����Z*����1<p��lBVm!��*�n��2��J2L�_�}6�Fx�X6'�WV����T`�d��z�<��P]L���Z����<���04�/ ���r�R��S:��*l��lM����B�����	�P�@���^*�C*'�}��~��:��o�X4������F�{Tb����o���]�,i��<�m�KR3�~
�*��z�z�D��[��fQ�F�j�VMJ>�Lp���L��7��"�~�&�%���p��&����OO��iv���������A�$��lb�&ZQ5�Dm<A�5]���,�O],��V4����� �
X'<l}�fg���x"��i�E�������S�X�.BSJ�tD���`{����}%��x��[��B�k��4vmH/
��<=���7�A�s�	��Y���j�K9��R�e��4`���l(���[�q��.���`y����J������/�Qi�_Di���?B��$�����'9����/?�kIR�o��92<�N�����(�4\J��	S6@�LWe?�#>��?�D�t6gU1)A�A2���=�b�%�x�l�M���v���O��q�U����W��6� �>�Za�ix+pn5�� �'�q�����(v"R���U������5,Y��z����m^���?���r������[=�*1��jC�)E� �����+������i/c��cG�O�:4]��J��)� f�}�*�EA�c����w	>�W�Ys�M@k�M����}@��&o��� Hz-��H���T�:�C�*��X�����	�����j.�^t7$b�r��F��������Ioj��a������a�y���~+��A����(]�{�$�f�����������^�qS&������Yf�|��;D���@�06[y�9��
�S0�l��q���
�����>Q��z��73��(�m�&�5�	��"S��wI�&K�s�d��	��+o2XY	�v�x�(�*�����j#d�)�R�U�E��S�k�1�]�q'��f�G���V!i������h������Q���g��������Q��i6b��T{.?Xm7~e��?G(N���%e59���2�
���	�|�6�L>�<�p��^}Ws������t��,���>�{������^	�8\��,��,�M�I�0^�Q\Xt%_~��o
��)J��_�;)PU�i��e���������`����[�t	`����h� �n���R�X�n�B�q2m����M�K�X�f�)r)���I����~�-9�W�"A"U������N��2�@%#Q��]���Qe���:��i�o'�_��3�O��8�����u�� ���i�$�1��}
�xM&x�>J�"��/���T�� �S�����fc�i�=6�3G0g���E��r1~9e���4P~��v4ov�da�VZ|��Z���-�����M`A�)�7�l���P�3�l��
�<�,���������O���i�;�,N����yD��N�HT��B��\D
 �j!�(���6��O�H_G+6����Ai�7�D��4gD�\������w��i�b���mg��y��>,I�y���x�B\e��x8��1x�g��u���h��5�C�������6E�
�k��	�8	��2����)������� �����r���`������$(�aX���#�����gd�����'�t����Y�e���8���n/�#OYxY�_��x�
���������y�A�0b���o%��d2�{h$���`��G��zv���s��kY.��\��,���Dy=���K����B4/����oZx���>�:jD���������=�p ���\*6?��� R��Q����
�2j]36M��A���R��Z���@����v�V�Y�=�LO�1��y�`�{!:��~(.FH�5@� =��\�>��k@�@RG�:U�.@z�Ps��Y�������a�h�����Qck4��n:�^��!��e�����i�] �f�����!}���4�}��"�4�GU{�6�tZ��uCF�������T��j�]��u��83�b���q�~=DM���v�i��w���T���V@-�,��e?z@g�-���A�A5]G����f�m���5a~��V`�~�����������tL
��Sw��J5��U6�����2���m����i�����S���f'Dm��L��i�:D��?�����4B?��e ��^@Mpf�_�m��]5-Cs�c	�/�8
�Y���m�Nj�"���M[���vr��2���0M����������z#uw�tj�!�k+�;�#PH�z'#���j�~,A��A�����2�I�Q=�������mi��hd�{�P�Mb�ivBU�]��~�h�G��8D�Yp����[��:�����-&�����R��T����c�yT�p������A�RY5��N{wU�A�it��B�eKT��D3��g
����K�6�ue�v�h�<5����n��P	����1L���e�	+��CCU�
�4����e�N�����9T�@�:i�fMe	R�G�������8��Q����V�����>��3�P�#��:%H�1�\�X��Oo���,��4��s��������'��`N���i[*�n���c���:�8��8]B�>^��^��U\�gHC����}O3I4?�N�9�p
�1�������V3�lM�\0>��s
������;5�\�j�L2��^1�!�Mp��WtE[��J������U�S��6� �;��	W����)}��F��A(���r�n/�`���!�]�n$-T2�nFzi����XYB����K�����#����s5�R\w����9Q�a���y^d��Rx�ydi�%��Q�=�������� tl�*;����)o�����i��������K�T��[�P�SW5;������������,����� ����:��:������xh��gZP�����a����_%E�~(������-�����5w&J�j�C�D��lG������3�����	 :I�E�����P��IR�`}W�"���(B����7l�fk�
�J�i��e��[���"�XdZxp�4������`C����x������W~�fE4c��<T2�^m��f�M������^����������]%(�;��[�X�e)��I��g�z.��,���}�6�.���������c�
��_��}�STQ�}Z��]]�
��EE��'�C�dv��l�r�X�y~;��=�GF��n�{3{{h�]���=����������������}L�_X0f���#���S{��.��n���v4���>��o�����y��6C�����h�����S8���M]�;���M���!����%H����:(;
C�����
�ci�-�h���of����py���B"��]F��O������Tj��XR��-�8��mx{����_tU�v����P����2�<x�S_�J
����hu��0�]PZ���oo��Q�p�y_E��� ���t6_�/�x��������\O�i�C�w��x[���2�4�G,��Ip!=wh�����vn:��=d}�R�5�4<G3�����@5o�Y�5�1�^�yho����;i��h=��|�R�����S�z��y+ig�_��x��e��.%��|S���o��NK��u��WIt<�������-�4���C�-��u�������u���^39����{���^g����7�ni�����
AS��K�����T5;�S��w2��gA2a�������O�T��
{�	��zan��J-�c����G�FA�u����	�
Y��rr�'�4��W��z�3<��K4w<���"�}���^4mS6����;o.���ju+~l���Z���,���{�s�+%���[���Xw�;DU���)u
J�FxG��:f��
��2=����g��+�H�w�m��m����Cz4I{��Q2f+�������V���n3,|���V�C�q�c��E����s��"��z���F�{-{o�x�����4�����=��E���	�^%1��q��|��mX�:�>��Swm;r#��W�QD!��7k������Z���F���-Zu�����Y��bw��0�r�`����q"�\<O.z�@</���>�W+�Jy/���
R2?m�"�W�}v�i+*,��E�a�gm��=��u{:�
z�������~�7���\<7�2�"�A����)@u�9������_��3���&V��`G����)����2.e�W^���Xn;�mP���J��7���B�zn�uX�������.�����u���|��+���;?��T���K�!��v=��� Y���o��'��SxFI��%��H%|�v���6��>u[t�|# z�y��S�
��$��_yQ��s�JDmJ�$�h���CsIB��@+K�=�����jt�{�����d����[;
�&��9V�JGE����,FUpj���~hZixC��v'������1O�qUt�Y���i���"��of12��
D��d����K��7�V>���j=Q��O�RT�h����!��C$\xN��1Sq���,	>�6��:�3���yL�����d$�*C=����"*���;E�6p�������,KN�FE��gt�=��a�F{rk�M�q�Q�oO�S��S�}����9* r�����0	����b\W�9��n<��v�#C��KB���11����0],+�A��&U()U7�d�W���T]�
>"iMd�k���=�u�VGr��E�#�f9��pi>]�e)��X=�������0yJ�a6�����:!"f7��oK��'t�?����H3N����	Sy���<O�
��![�����RdbD<w5\k����g�����g xU�Q�7��)��5�~e"���r��D��5� %�%��]3�N2��+�����t���4�S��y�U����7�|�N-���"=2o���q�4��	���[(�����~E�o���m������0"!{]���/��j�d���NR���P'��
sA^���l84b��D����H
�(_=:�1�jL�jr]i#Ca�����cuW���c��U��j��#����E�b��K�-��]�4Y����P�m�Mj��Q[.0���	��Aw2�|
c�b4�����������d�s��9�|����.|JO}7��t�j����A�{
�~c*�&�a�D�Q:}k�!f�L	���J�
<���R�;6m��[��oN_�T��^��$�dI5������Z�sx�U)C#c���oc�1Vt.��T�����	��+eJy1��������\��Y�:��:4D4�!3�.��a&|��f���bk		m����d��N��rx�������l�����s9_��9�@��R��L�	P�8�r3j����,�7/|��6(�'�2j@���n2���(E��3�.l�����p��Sb
�^
Q
h�>���~<��z8q�-��{��G$�����I1Bed#���8����d�m	������L^��]���Y����!��6��>e1~2��Qy��4&1}��"x09Q�!�����V���|�N�Q�)�'�������Q�}�!J|o��
���	���=�j����������+
/�#*���Hr(��R��?�e������W{Qs��(ntw_���DF�sz%T)%���|�3$���wOOm�m��"��Up��$A����
�s+W�I�y��GB�1���y������0��dxP�h��
_V�=�G�����A:`�����B�T�)L�U[���D�@&f1Z��?�i�2�/�R��5�T>i*3{�e����X�b4�zE��;�j6�/��Pp�%"KV�b�y^z�[9��I��2I3�Y�5|Jb���C��*`��2�[,h��
�Y
����JPc�Q�R�3��_���B��3�b��'<�)�4h��)�Ydu$��!������!%�n������KKV��������������P�e��&�X��b��r)� �%;��pw�_�i.�Ba���B���
����i�VV�R�����7�������>�j�Q�3�����Q���;.����_h�����{���z��� Tr� L�#�ld�4���.�%R���?)W
?�F"�*35�����P�B����J:��p>i���.��2��?�u��7�PG	/�wQ@���q��yj����
�\C�Q�_�d�KLd^{�2�r���I�.��p]��2�c}�k]���F ���'��������J�u�7�]��Sp4W�1Fs�IgS(���t�m�Py����?>�A��3V��kH�C~
��	5]C���?����k�v�"u�������Y����+����Pz�.\��������=_��c��z���W5O(Vf��,ph}�F�2Y
$B��MH��5��v����>v��{�7.��2�++�������X�r���	�\UFL�:�V�'2���m�����HIo{nuq���n�C#�8�@�����Vs��6�1�$;���_�`������� �2��3I����d7�I0�d�{�"���y<���SB�#b��c��Q�D�������?��C�F��iEo��>a�E���:�Pu_�,��!���#Zshx�Y1��3^F�.H�2����|�=����D)%�c����[����`[�l�Nz��
?N:�V��<^��^]����hOM��w����1��!�B�
Qwn��Lc~b��t^���N����)���:x�+��?���[������N3,�08�XZ�>����?�9�����>}n�6����LL:"vW&h+h"^TL�bX[����s�CI��o��&K|��������=�|��������I���Mz:��X)`�v���0`{��xX�G/B���^�WQ��n./�A���Db��~����C�������!���Gtp0�3��vG��'��+��\�":�QR����P�����O&L�YgWY�s�b+�J�!���v������<Y&���W]ch$��NJ1>�`���`xd��'[I?�����<#L��1VDI���0�����6Z���RL
����W���S����p���es�ad2A�;���
z�5f�(�w�w��I�Bo;_��W2�,M�Ff�w��&�x�J4��������x��s�����9,�B�N��.��x>_R�!x%�e��QiS��	�N��O>�6�#��_�/��"$�+��
��c�{?H��p�t��R
[��OJi�S�L�9�i�s���R�p!�n}���������f8����xR?q@1���������dXj � ���d|��{;�o�����w��[�!���Y���Z�i�������[�:�Ju�BMg��� �W�hV���+��_p7�6��=���Q�W�z���
\�*�%v+�������s|����{�i+��i��4
�q(�QQ��e�vm������c��FC��|PmS�:��o���m01G��Xf����������]�}�m��q���<����?�VJ=%B��)�l?�N�c�b~��G������Y!��t =��b��#�r��2`��-T�[J�lq
�(<��!�X������x+���~�r,��\�S3�g��~Z��z������\�w���s���5��P���8���#�I-udq ^9t��+���WfF��K��i�[M�47)P��[�����B�}d"�#_wp<�r*�����K���������������l	wP�)q���B��4MT��VN�2:��~k�f��0c[��g,*,'���Zn,/��������yZ/`����8������h��������f=��u����I���h��<�v�4�7��Twm1��P���
R�R"�Ye���m-��+Q�L�7h����C�ihBH\A�E��%���y�$-#
O�����k0��+=
���=�)�7���Y%�f�Y3�)����<��A����7�NyY��%.rl%it"D��^��5��R%��\e�:
��&
~���`J�w�-b�%�4?V�MMF�~s��<�ZW�������R�t�����Q�<�B���_:��^QZ���S]�&?����{;|�o�����B�<�Ult�|��
��.��3�����?�4_�D1��qJ��z�*jt
7��q�p������f[?n������s����g`y��4�
�����+�����}{l����a�1�T�q�rQ}2��fC�M3�P^�</�c^y���m�E\.�L$��2����,���:P~�	�����q�\r���!���������#������mY>8��0$��J��q��.�O�����^���&�g6;�
��
�Y1�����������<L���Z,�E{"#�������sq~�~=7��[���l��].�B�oZ��f�M)������������7�]���+�1���_,������
3
q�x�_4��]4�H���(y\����u�(��3�p����N�����������n-�IT��g����k1TP���3�����E��� ��i�F������wp�Y������CsQAJ� ���
�b��Z+]a�\�	��1����n��5�?���������t9�&{X9�69����`�SD�Q������w4y^��Dr�)D��-� �K�k
�V���o���!����S;�}�iO���l4=5OJ�Cn��c��l�p�4N2�R�DX��������%�On*��]#��S�����~��;�3?��m��w8�C����|������@�� $4e03��Q8���n��:���:����qXa�ed���&AU��O-%
X?�m�k�w����_��"eD�|��c0 %N�|�k�d#P�����O���w���x�8�M��9��;fJ����G�����'��S��k^��{X��S�~��2��x�:���Q5���w+�K��]��:��_i�����"-�Fu�������Ay��2ih��*�i�^��=e�7�w�D����n3'�G���	t�����ee��"���|��'��
�m�[@w�p���z����}l������cK�ey��g
t���Qk[9W
1g�����e��K�f0~��s���f�
4��/-!�>��Y�����T�����U����)����1��q$U8�u����[�M�:r-
3���m�x�sI�Ge�)`l�8��9�(��)�����'����K>�=5�l�y���o*����@���k���]�����������������g��M�h�b6��h���w���?�0/e/�zi��8����Y�����+��j����N�c��?��M�S-�2��Do��W�2(j%��q�WL
�>���_��/�u�!�Y��M�J�v������TX������ ��P1����Y��hTrE���� F���6�<����&BU��b�d8�S1TU-
��MZ���+qu��w�&Uh!��;;!�xZ��D��9��IW�(sr�?
��|�O%��O�R)�����98�^��>���Ib�:���%�8*��Q���3U����vW_���bx��^;�~�'�I��H;������nU�g������cd�rk���J��Yy[�yF��]~���J��g<p<Gi�P��Iux�u�C���?���vaB5��hj���F���A���F�GJ���ihpr<�C��U�D���;������u���aE�;J����D1+�����>����G1�$6��FqsD�+.F~��������:r��g����6���w��
ns�f=�9r5:�����*)o.6_������&����J�J�^L�����2�iA���Q�M�h���C9����Y,�J�:�k�`���}<����e�����\;�$&�P�h�~��6Bo�/��K����y��-��A�]�d �r$�]�/��K�'�H�I���q���':�DR���J�`�_��~��a�����D��9Q�a�(��|j���������Gc%��S��Lw��:����yN?<�����~��-�}�Ml�����[C=�z�(����������w�,�2���]���:Z�����N�����v��p���i
X�*��+��X�	~.�*r~,������c�����2������a�F��O:+�����ihz<�fA����'^�<q�����I����/Gd�j�w��^FY�$zz��a��m.��n+]
o;"��������@L�z*O��8s����5a{A;�������gyl�q����~Lgx`N��9��F�/��tF�����t{���c�����n�[h#�=�3���G��>O�c���9&�?1a��t�z���T*?&R�%��c"���$&f(���������{P
3�'k��&�8�F���:�21Zm����udX
�k��-���3#n���[yAt����fU����L�����@��|iy�_L[���f�h�e�XIYJ����o�����������0�E(L�����	&	�,���>�������J���|-�w���+=��NR��H�W����''��1+��7����a���t�u�\~f�"?f9�.�V�e������m#I��9������r������J��! ����RR����
MP5�����uy��*�*3��B�h1T����^��.���yD��8E��(��ZjU�n�nw�[�������-���J��u�U����������n�� �j��`
S7��o�����������;���f�\�z���T6j������T�w�u���a���;w��f�F��������&��/�6%��J���$���#M��U77*�XK�S����W���j��7���\�V�������eO�K!�5\��D��H>=
��������m����-���S*'��V]B4�A����p�-��9[��@��H]�������^-H����X����M����.�UtV����F��9���t#'.*��W/��_���hET:c������)@&��

u�G1��rK��
��K\�7TT���[g�����1�FW"}�r�������
1[�.�c��S��T�{��n�}��/�Y�dQn�r����A�
��b|��]�J���1�*{#K��d/)T��,G�o-�T@�����v��S���s�;���h��(U���K�O�L��,MK��y���`Y]XSm��U���'�D0� ��]�ED)7=J&v
�<����R"����j7�X��E��P��y
_����j�T��E>��aS�t���h�s����y���������Z��ZP^[l^��+aK!��&�}�D��6��b|�,bQ��QS��JN��	�~���E)��+M^z�1<��LH��+�a�"ph���%��O�z�\q�i����l��L�[�
�� ��?��e��J��]wa'��|'9��
$L�~z��l6�]{����/c��f@r������#-�v�w75\��*��xlF= O��������i_�jb��	=7�������|��o���@���1���4\���(�w�H�}M�T�oo�_z�������������FI��l�N�'�m���V�F�L�(b6�R��ir��|	0��Bl;Wf��������D��������|��M}S��r�k!�H%�"c�VC��������y"�>SRJ�\��� i+j(V��M@��d�_�>7��a���/�{F�f���e;���'�eTj�$���'�+��+q������}c$�?_��.�F�F}���9%�U_9Az���:���=��h1=�x����~��\�n��l����4"C�A��{�����\=�9���{�~A������t��R�$�n��"���5�+8�#Y���i�^b�9�f�=
�Yj:�7�B���L��VE>��pl��g7M�"Q13� �q�.�t��R�NRbA:6�tV��~��_��������U#qw����0��Dyz�xO�rJ�pu��HKc�����t�X�ilN���P�8�Uc4/�����u���������4����`$����P�'j����}{��O���K��~�1NnN�&�ia`����N����p�����r������19�2���	��!]� ���8�r���J�"�?�a�^{���f�9C��%��c��u�FUl���E���z����0�?����7����P9�{�!��������?�E����J�7U�3�7������(�
/_�n��>��2��.�{��[J5B[��uE��Df������*s�2�9�K�(��N+�&�$d�@Q����r$��H�i�@�I�
�G�'��� I����s[@t80f������?�!�z�pW���UH,�����	EUzdZ
p\�!��;�	H��Z�D
��<y���#�����>�7�,=�?�������r<���z�.�&��F'���r�c��p���5g�p&C]dYtv_����#%p9���������{b���~�q��C�-��.�
s^�yE\K�����(+��['��(��%>!�:w+Lle6FTNU��7+%��r_��q�����e������}rw�6��D�/�][�����5,�`z���w�������0-�J��&D��z��O�����{iL��>:�'��A�o�l����ow���7?���?��8C������%��Sa��O����"�q����U��.��`��FI��K��N���#�w��������|<���l����+�hF��R&���)��+*�G���{9U1^���i��B�D�L,b+$J��(�bH	u��,�R�'@kQ��`����g���'�t� ���2��V�,{�+9`��b��W�"���l,<'��������)et)z�����C+�Wn���B��(�3��s�L!�H!�6�7��Q��	��!Ct���)J�:�����G��"Y�
�R�������l���b1����gB�-}3���t��b��y��iX��{�`�]Y:�>4�y&�fX�a3�t@�
��������1�EX�5�BD��;#!�y
N#���RV�oN?�
��:���oS����
����D`o���������"l�kO�n��p�����5x���)&F�����Zx��i������v��Wpx����6��o����IS2��Y�l�'����?��:��/6����S��^S�4�A��]���a&X���0* Tk�P]6�Tl�t�!"w�X�]��HD��)J����!J+�g����",�$+9�K3��b��oH\UR�5K��l�*�&�,����r�J����v*�W��r�]��>�����(�c6?fs�p��dr�s�>|t��;�����"&��%�� �T$��b���
���P�!C��O����v�����nMe/�w�^���RN�T��~���e���:)�N#_]��s���o��J�e���sG,���'S���2�@�(/�.k53�cH����+�����V�WS�iz*��6���6�g���?���[q.�r28/jd?um;��Lf���@@��������,)�s?���G��LJ�8�|B��E�����B*9��V�R�={�7-��0���pw��tfo�W�8��3i��U��-�oRU�������gYC����R�}��z�<vc���O����X�J	}8�A���|�m)��I�&u�#2:���?J�3tT�������r}��w���~o<���J.�������_���D�����S�������g$���:��\��{ A�D8������40��R:L?�v���}?p;\�0s��bpjrO�p���$NR��TJ�$/��Wq�n�����`+���-��p�^K
_?)����w=����v�um �N���#��	\.m��I����P��������q������F(G�x���__h;y$�>�h�
���f;�k���
�
K�K�mQl�A�IL%[iY���":���]�n�[wx�#o0�<����5��>��u�R��P���W�\��3�cy�C�m�"���+�Il����/��`Lf����I.$���G���
���^�7y��MZH_;�|�]�a"���3*��<|�o����i��f�<<��r.���2��L=}
Q���M�h�B�j~��$�56B)�J_��<�1�drz�^���[a'p~�l�we2������F�+�m|�	�<�`��������=VU�^�l:�k�(��$�2��X�E�Fyp���;��o�]la�g7�n�z��;�%��,���,���D��ou9�V��P����b������~�f���p�B���?.�u&T[R�qU�U@�����,��y������4��2�7 ����e�+���U��p�c������Z����9Jd.�<F��0�K���_k�j	��WmEi��T�D����T�&���������5����Q���;N+�~p+���(��D�Y���v
��&-�p�V�E����7���=������g8�PW������Gsw��a7��$��{l{>� R�;x}M�������r�<����aZ�D���9�o���[��~�����]�=��gw�k�5x��t,b3����=�;�Wt)��o| y<6a�Qs�|�C��`���A��	���Y��g5�[sPW��sT������
�l4���@���q��r*��2�<^	D�O���#���ri2�@�QhW��X
��MG�P]��x�]I�j�w������|B9V��SK�D�I���(I)u���A)
�S���t�|�o�n�'I�Z�R�������hH��P�*�e7���.�������R���{oi��W<�1�!�����@�z�ldm)d�N�E2��������H�YvC1G*���RArQ�������u���v��Ms��
R���b�@JWY�e7��,������mK�x6#Mz<������e��!nUA���"�����������y��u1�������������q��J6#�����b��
oM6�K��
������us����=����}�EF����?���H����R�eo��-m�`%���$�3�V"�����c�9	������Ps�����;�vn��c����;5�;7G�����y�1�8�f�����8H��x��+��-��� }���9L��W��1+#�Fi(�L�aE	�8��k�e1a7����s�rz5�I�TB3��#����PZn�GI���>O?{��5!�M��wl��x#]W������������O����8g�@��+�?K�'E�@��I$y�+������o�3dN�
������J��!BC�j|HKf*�S����\��/�
��IBB�BWz�z,�"��GLL��A.�=�G��F�iEW��r�W�V�yl�'I�����	qb"&Q~����R��/������_�o�I�`���k���1�p�&V�c	��������L$�N��|�l��P�|T�dAB��x�r�l^F����}��Fkx��\R�"�i5�`e��Y�
`�������$?���J\|6���&����R��G0_wofs:u���z�|�,���WQ�s!j-1��K�
��;�*�����Ou��1����
<�9B�t��\w��$S^�RJX��������C����I����5�fB
�x���3�2�f�}��P�e���y��q,2�E%2����R9���������N���?Q�{Hc�<��-]I�����9�^�6Qg����		@�����afe���}��4���N���b�� <�2��o���sA�J�R�YWYJ_B	�u��Q$�&\�P�y��s	P.�����(���[��`��9���F����ug7��:w��'{�~k5-WQ��"G*��j�)��n����!�b�?��3MZ����R�d�I�:���xX����t�v��v�����(�A����|�X�V*
�c����}�z���r��X�0k������������9N������j"�=�4�:�n���B�s0��K_�$z��1R��H��SJy�����a�K���M*��
x1�����h�="���1��"I�����@�S?
h1�LKe�s$"|��wS i�T�)����������4�<z �,���H?0`���A2x�`���S����v�s}&IK'��Q��������Pqs��,�"D-Mx��3X����E3B�w��#&���W5H�4)�)��L�
jR �1�r?f�����b�S���j�����B�4��4"B�^��V�G��|%�IM�m��}=J�aJ�~�9Fr�f���a��pq�"���������{lO�M��K��l�l�{`Zl��[U
��v]j:��G�HS�LX_�S;�9c��R��	��v��l6�����e0<���nuw
'NW�+H���O��������I*
mR��!iK5����������)��2�_[��o�N+��$V��QC��Z����p�����!��{#!����\��8��!'����zp��2�e%��7B	�p����\��M�ph%� ��n��T�����T��S>�9��t��u�Rc����t����n�|t?F��ICXt��8��.�b�@D�����><���a��He/�a����Lq%c*��g,r��� ���o�����8pb�_�����|��_���Nc�)^�w-f�#��&�S��Tp�1w[�����L������$������ �����y?���y�����4w�����T����}�}I�qw~�����/�HSA���]��QY����!���<�y�����S;�
��J�k8"�I��D�������]�9+s5�J��{*���N�es8`�k�����M1�)�]��1�<��X���0l]�t�<q}%Wz�X��8jO�e�p~x���&���<��+�bAB6�:t+J)A�`���7�u��e$U���C�$I:��K�V�rY)���qiW���������}����4C�<�
�`��{%AbE ����x�gyb����O�*��*�l8�WU�R��k77��|t��J:���+S���NdH�_�%���V�":�/�+AK������w��[B^i-b"T@�b������G��Z��Pu�Jv���u+F��H�D�1����,����6�!#���tw�!�G�4iX0��N�������7n,�W�������1k'�$���d�X��jEc���J���OI�V�4�������c���]��~�u�J����a���C��-s��"<���2�(�4�
�=|�?0������h�n���I�^���� ����e ���%��-+���L5K����~i�8��)���q�a��� ���2B�\>i�������R��9�3�+F�����B�$=/R�x�����n��QI��������������2gm�uP 3�D�w�RH'D.^���,a��f���w�j��M%�]
)�R�=���R!���G�mT�n�������%�
� �cl=V0�s��9�B��I����a���c����`����O������k]��6�A�Y�s0����(���l�Tq�|V�����{$(�5�8X����A��T
Uvn�\Nfh��>�|�@�����{����=}����Z�b�M�y�0g*��q;
��-g�-4+��<��-������~��cd���{���nX��|n�����bb����q��Fk�;U�����]�q��I��r ��^�V�T�'�q�d���s�S:;�<h�b�����G-$q��W��4�3�
�`���Y���`���zs��H�Q	�HQJ�����YTKTF�F4�9PZ�l,�� �H~�_T1`���|X6������A1vZ�g{�v�8� ��R��K#���B�l�e�!���/%�,L1V��nbyx���oy���t��k�2(�������H�r%���R������m��A:���u�� �������LN���#B�2 �s��X�H�A �3[���Y.��n������^�{rz��
��U����zL�z�DS"_�b��_���aX9�1���h4�*��
��&\�r�\��=��v�� ��.�����X���;�S��K@��t��~����Q�����'ie�G�u���eGI�-�,ea�t?L���?�P�o��8T���=������I��b]���r�K
���uy} =�n����F���Ugl�*���FT����������tT_z"���I�d��x�^�Iq��6��������a�m���[Z���I��]�0��w���Y%�x����������v�L.��a�$�=V(���I�,�������~5]Y���4����_�i��]Z�^��-�p���
F��T<hjE��^I�������I�	����3�f�R���9����O'�1�z������8�2��wU�Y��9�E+B��JFMF���L�*����j�)8/N���'JVzXG~	hA!}P���P�k����S���P������������X{Il%���?�+����a�P#{��P����X	��I��t*�>;�I�)e%�������[���u��kxG��&�2�	��hb���0i��c��k�cX*��}�����������E.v:]}D���Z]�������������e�������E�������5������
�O������>�2�}tu;����,,)�L�l����d3�`���\�Rg���h��#������F����r7���L��cg��#A
'%������D��]�]�
��O�����Xc�g3�r�z>P�s�<�t ���K~��5[s�,�F_`E��'��r��.����%�{����q� ��x&�e��i���A�t��,�]�)�����d�X�.��T �)�(7���-%�����w���x!�A�L��CeD�� Uz�����t �VA.P������z��/�`�otZ��IC�f��� ��_D$��4�����W�������S�O�\}0�f4N
1#�L����`���~���I���7�y����JB����t Z1�}�u�����<Z�8"HD�$����i$�XJ�)A%l&_����n�%�@�dD�^_b����.��8
��o@��'p�=��q�<�����

���<���N��a�'8�2����X{�t�y��������p��f���D	�3�/�.�l����>��<�?��� ���j�8��[��t���%W���M���"��z(���p�/�I�����qYo�������nfBy���B��'�������qN�`�b�I_n�$�f�
��l���H)�
P��b��n @?\��o\x��=t���c��W����C��^T��^Y2Q_�D����?>\���#t�QJ�\Lp�+]b-�F7f���Q.^����0���������<9��>%�;	����	$�����F��A�.@�����L�Z	>�M�\S1�z�q�aD����|rE���Z�4�Zj_���b��H��S��X�Ii�1� ��V��$�K0�]7w����L��S�f�-���9���(�
6bc���K�����Kz���%��N
#�b���~$������%4g	����h/!p��.��,���������c�u?[���f��V����Z���re*����4/�9��9)���Q���Qo+Z�]Z(Q���P����w�����n�"k��;`�n}P�LG����h�STib��^L��������"�m�0���y�!����e^`������}n���5�BYH�D8��;���`��z�����E����c��l ��4��o��d�
���[Gz�����������O������Fv���;!=�Ud����T{��=���;��yU������	�Y�/���������L7�}�f��P������CW$5�P�\��U���g?V�6x=�vU7�k5���6�
���b�L���r^�R�T���w��0�?A��+�����$����h���\?���������p��.+#��2"��5�WF�=�)�/]g$����������y�����\!1�Tq�`Y@�9�<������lfWEv�.�"�\j��7+8K�<<�if��V���U>���Hh�0hhR�R�"=2�\�9�E�w-�
�X�
N`[�d�B�)�A�}M�BU�����F��Ee�8|>�����,��`��������O�����������4��sB�I�fu�|'Vn�1!����L���R2�����#�A�LC�7r�I��t�[�����0.6s���A��(p�W�"�v��A-l��0���tm�mZ��mJ���\5 ���6���M�]Q�j��8�����)��U�;�I��P�R��o�����cT>��HP����[x�i��O������R��i��?�v�����]��nS�!����P��fja�q�W��y �dX�zh�\�"
��#e���'��,~m��G��J��g[(�p.��4
9�np��X�S� �&��p$�f��{e�9j!�l4�P�T\�G����_�_�
��5�!~�ns�i�D��6a���) �,���~~����M�9�:�m���_�(P���.�2������Tz��4���f�?��3W�/��l��:�J���XR�E�����;���4���}.����M�*R?�HR�����Y�b�=HJ��Ig����9%<�����]�.���*0#J�z9���sutS�S�~uHY���
�'�T{fV�4Bv!7t��O��)��nrTw�	�{>������?@���~�k�b2L'AS��[�G�+���w��g�`~��8�1{W��J��t����;X
�D����|�{�kQ��1�1"��_w6��\���/ ��J���6[J8��l����������X���Fb4G��������~�}#���:���Ri�*���U���=w�bL����}������Z�4]�R�5���o�<��[9���]�;C7Oaj��B��n�qj0w]<����'�L.������Z�
f�G���t�UJ�{�F�b�q�2)�U�v��F n����.������}*�qD��)a&v��[B���aSb��e�\
YI+��
��(��hf�T�.�n���(�H��1r%�?��p�n�?�c����[S��+3'b�C�3O�&�������\���������������m34�w�q,f2�C�\�|k�}%mzi(�$���/�����6��`�c�����7O��5TGB���%>���i���uH%��$����~6�s�����.Etq�Aw7���=�>�h����-�k���Kp���J��I ��f�!(dZQ>c������N��r(.�������B���*�N�/	!A[��9�����1q�&+�����pE�����
�RZ���
��x���'^��������8�����pww�)�Jr�H�n��^��<����p�ox�i��0
��U�0��g��9.&}��/I:��y��\�7R+$�pd�e�����v����|)�y������

��i��'�����������3M�����$S� �K���������o�������-����=���PM��f�L)a��kw�()W_�YdjP�`;�iVAY�R�2X/�qX��W���v��G��i������2��uE��P�$w%�-��f%<�gqWD3�����\<�D��*��=�������~�D���w�2�4�'�P��I�d^���[;�?���N�����;�$M�9�?&,�`&��$h��b���M�q�h��
w(UzB=H��-z�m�
v;�/�4��z��H3�1y5U>��C8����-���P|��D�������5u�w���C�#��K$��"�J����>1�C�J�J��.��R��e �D-�te�L E'f	����<��:����L0MdG��et��W������n�W��q*t�\�R"��fO�a����VWQRYJHa�{6��4��4������x�;;������0SYz������%>�_o�h�w�B�9����b6gu����CJ�9�V�+���aZ(Y&(.�U	�2��~�L-G)����`����KR��E�IS����U�f��UPi�������N&������Sf��d�@2��G����x*��=O~<��Y��f��:�Xpn��� *c�D��v�/,�n4s�#K&��	q����U]@*�GZ���Nf���O��
���sm�h4A��������i�q�����wMC��hm	P~T)����`��8���a������9{����#�����!�R�PS��h��6�a&���L�o���p����x���b�q.-Gw�w��q���D+�R9fp^��XrJ=�������}}�Ow/�N�<�{��\�0���Tn��.E�J�1�7��l��Q���~w8
�?����9�Z��Pt���r6#��+�D����J�@*��u����f	�
]I2�C�`�1|�D_��rxD
R��U���(t#��g���`�����z^@�8������������f�)D�b���dW�8�R<�- #���-�_�o����v�w�I���������\���L$���^l��y�����6�����1mW=Q����	��ct)�(p�\QE�,m��C��\�D�;LE&y�p��s�iY@��h��ae�G"��x������Is'�M�f���-��A��������t�o%Lz�9��
�$w��#��]�����6�0����7���������^��^C��v��������0e����u1*���Vc�>��{��p����v�]��=���g�sG���\���_U9Z���7*?���'���f��\R�r�	�����QS�l����
������~j?}�f��y7jQ�������t6P��S�����T%�����,�^�V>�`�"\��
�,���-��:�*%�����5S�����IPB��������
��=o�7����$(#2N������4�N��b������7�J�����J�+Oa�d���0������iw�z�����@I_���!UH@%�x7d+�tp��K�$�.����e��k�v���t�����k/]8�����t��	�p���Q��7.j������]l@���D�G�fo��y��S1����Ff!l){�_���7�0������	:D������L	����������5�	}����wU�;�J�i^�P�!�1U��1B��
 ���#�G1`����M;^1A.����*w�-{�\7tq,v�5���)�hx��K4E���p�x9!<���N;]�r�s�$�I)a�XB��i��`��K	b��6c�)i�B9��6��D��	��R����E�������	��KL�G�n)�q�#%�0t���Un�����4UP���TT|��^����������%������I�_�67�vA�g��,���J����$�s�u��=K'M���D�c���4�t3yg
)q&�d�\@#�|q�n���o6��FJl$m�&�����q����o'D"Wi2�S��/��*��qJ������Y����������B��q5$����R��g.������At)����R�I\� �%�4mu#��|Y��O�����5�A���n����[������S�s_�~��f�����t"F�O�0���,`�q:���,�����9B
��7u����q����> �~�-�vb���{V�� 0$�B�)=�������+��u���5�.*3����%���P*B��5E��9L��qR!6,���!o��
��z�1$8��!U08��s��f��E~���\gZ��������8��k������m��z���>"�x��Q��8c��UF:MUe1Sxo�])d��\��'O�z�����(8�L*3��gK �T��w��i3��D�,��=L~�aF��Ws6E.�{EaR`m�@�<��?U���^�j�����1=���X5��� I.��U�M���
g@R�[��UNte��%F�����;�O��0]�G�8�������tIw6���nk'A�������:���"���Xy}�,*
�uu9������IZ����`U���z����R��#���tCd����`�g����������C��!���u�vAd4Gq��h�[�V��;�������M��l����������d:���`��!�u������|n�����7�]Y���L`���a�Iy.�����z��m�����E�-xCmy9t����z_]�K9�'������q�����9��4�9�zQl-�2I6l�7��;��/�������&���"������M���:I��N��9o�<y���H*&aU���K��I��t6���U�J(��N{��M������u�������iw���]/�*~��;)R��?!��Tp(f��O���7������|�
��$���G��+dv�fA�F���_c1�'��k�]���m��Wf%��&J�Y�P�.��>�A#]7�k/��eU��;����Y���Ww��������8�5�s�\�J���/���� �}��������4����(�.n)	��*�3mZ�n���<�����A����	Y,}Q�B
F�z ��bl.������zBE/?6����X��������!�>�&����������i���]M+�i�J���Q�(��1T���
�@����f[������;�B����h�V�2�N���F�dfc�������
��� fs�h"��i���2��MH�8����\z������D+�%!�L�h��q����� 7��(M8����E�k��U�-r������9v�%����:eX��y3��[�*Y�
�dBxP���U.�_�j
Mf`��[I�#\��\��"-y$Q��9�o<�Bt*�r�RJI���E�kJ\�<dbd@����0J�������L����F4�GFC�N�g���l,J��yiA~�*l���ck����
d��)U���n�J�rZT0���O]i��'n���9���P-��e�&���$T���SbZ(%!��
Y��;��M��bN��0�����GX�����3�l��A�2��[��(�x)6����h����o����>���������JJX����/��#P�k�w6�����B�����>4�'j���W]'���&��@o��V2��=��p��g	��V>�)����E�g�r���{$��4���b���$�+���o�b��%l�Is��5���������}�Wf`=��jP�5�^X�=�������L#g[C��r�*������
����s�������EH����}��><�G�Wg�W#u�t�l����k�M��b �}�{JQ#�}�Y��n�7���r�WK���_�����;}�x�H1�����q4�j�G�J�������kf)����q?S�y��!v�/�Xt�C,�)���\��:���y�+�B�J�����-���M	��,�N����������\��N���sw��s��O6
�l��>��z�1�(d����5���8��M�N��~@v��B���u.���ti�����Ss����T���������gR�N���NS�X`�M�*����B�U�7�4.uO��s4Xb7$�8�����]��.��w�{���s+��O\B�{7YF������%���9l���(���6�|�,"�������'$����c�+��L8�M��a���8�gC�Fe�����'*��z(�:9������� ��n]�l���U�u����]�����]��Pe����QP�S��D�\�!�^�����9m�e'��weu� HW	�c�7�&d�t��I�������*�\�p5�t5Y*
�6C��V!X!�`��������u�!Crm�79(�pCB���f�tc�����?n�n��~�����DY�"��<Ym����X6za�=~������������8M���i<�)H�&3�x�����������������������%U�2����C�>%���\x�� ����b��Ul����R����v�Pd�I�k������I�)�=����7�)��x��%�jW���1�E2��*���7�?����v ���*yB��B�Tbs;Y�g0"����n7��7��(����{� �����`�P�rCa2!���O�vIL�u!Q��pM��&�r���S�+��w�X�a������9��� �	(� �������\�)����t��."L�`���"��A���!�{��s��{���
����|�H�a����D�D+2�����M���Cr]>�H8�9X�1C@���������H �`�L�n�o;����#=�4�����Cu>�1#�m�,#Q������]F���u����q�$@��h��;��h=�NM���PX�!����:���w�����>�D�����(!Ep����M���
X�?Q��O��r
%	O�&T��r%����^Jjw/������:8+��XiB��5^�����B"�6Bp��S�f<�!w����n+^�&��8)!l�x.���BN�r���[_uc������|q���l�T\��wJM�c=��k�]
����s����~8������GHu�]��@�.��a����l|i�������#��J
�\I������q�lL��r������m�KWm��$[��Jf��$|�����S��9���� 	1�j��O�:���%p�-��Kr�=�s�zK��|�dH-��hD<qUG�b�=	`K��B�����>���Kyy�x�{����p)u�<iq��B��Vv�2����I��@��;������Y?R`%������-V<$��`r9R0�{�����k�O���D���0`��QH G,��|wy��Run:��J��gu09�V,�P��I`r��V��Ks������j �z{m��N��N7��oZO��Gv�2���K������w�S�����O�
\��
A�_��.M�Q�`�!kU���?x9�����7���/[��^w4�S����t� 8vt��v�x�����x�B���f&�{��K��x��L4S�`L�-������[��	�."�����c�-�������l��AvU^����W���� ���&��J�zr�FM9	Q�h#���^����0��M�����j��5�Dx�b�/6)f��n��eD����\�3^JT0�L�dYh������1�a���>�OU��.Wij~+i��F�,������i�����l��E2T��!1�l�t�.P���l����n��ia���l���g����+f������@�\���ch���+�U�Q���Z6��]}�O�k��v��BF�kD$��e�$�-j�
/�l�K��c�(������8����������T�G�k��
���������Qre�J)��g�q�K"Bs��a9�S��m�/]�k�
���B�/�c��^>��Q��������x,���������=j��etp_���*���fCl.s�����S�� ���Kj�@P�dd]P���D��.il�����@twY��z{��p�B�Wv!BO�Ny��3�qdBL9:�k�]��^���40��3Z���T��:����K��:��^yc�������B����Q9���Z&����-�\��"�z�������m����+�%���/+I`�&�nN2I��������OZ�u/P�|��N�Y,J���A�J7!�����*\h��@�
	�X`B�\ ��6���||y~�;��e	
,��#"o�W���9XJ��8tc��yB�5�<����Q�*N��~��(Q��,�^�Z7f�%'�!�c@�ar`y�]8���C�t8y�}f�j�)��X.��LY������+]n.�&�	E�L�]3I����K���i��8�E��s���	����g��k��8EZxP+��T����~������M��w��l��w!��rl=�Z�����2�!��#8���-\$\K�<��c]&��#u�"�f����������o�
.slcQ��]�A��T*��3@6�<�y�C���5��y��_������#|����V5���e0G�$����_�w/�u�����k�^���DsO���/��Q,L����
�������<7�'�[���X�\Z$����d���/=M��X���:��v?�=��bj����47�Cz���V\���������t�
�h�������X��*38q�i���g0�7�������k��DF��=*�9�����2�'X�&H��P�$��rRR(���n8��t�6b��)
�/s��Vk)��K��O\��ti����������K����Y��3XL�(����c}��!u���K�e�;J.�T��0���N=��}c��������s6BA��s�D�<R�/j.�F �A�W�&���DX��d�c�v��������'o�-,�����������su��Cs�����U�!�����}
Ez�K-�8`��V��xB~�����k��:���y����1��S��U����$�����z�e.�g��n__��S6������-��L\$,���,]4��j��P���'oJ��U���G�;6`yVd�K]���c�L^0����1	�l���N�z��x�A}LN�f�
,�N�[>���K3_������F:�,`d��{��"
��K*���?a����'i�OR��9���t���TX��2*�$���0�p��0~�q��'s;9�]�1�������FF�B������U_N{�	z8�/}
��_�Bn�����A�6`��������N������z�A�&�f�;�b�����H����R������\F"��8�z0Dq�����#`���g�
X�wSV��
'�p�5a�����o��"�����/�{v�"���L�M��{�U���
��O�����j�w�!Pl�t}S���&�+��FY�1,��uNR����������	X>�j4�t���m3E��e4��;�e�.V����
�3$����������P��yh�X)�k(A'	������V�+G	�����M'�;�����P�zi�~}�?������U����-���:�2Gr����t�����|�
8�R'�;�q�9��.��D!�V]��6>�����]�e�~>�N��$/!�l=r������4�nIA�������X����m��<4���T���ZHT���y���m3�,f@����g�������[�4%[����Q�sq"2�bHwX�UF�� G|�������%�
K�y�"u����j�.d���k��t��A��V��J�@����TA�[K�?�n�������Su��o�O��zF��A���9R�!�zM����@���?��1����ZF�2�z���d-�m�����k���^���������M��T��i��	�����E�+uz��&�U��n0���L�y�b�W�"qMw�7����~��+�?j|h<�Vp2C7L���@�+�tW~��]	>���OB����vW�������M)���e�����t���E��`�����`�4��Uw?�������Y6��it�t�F3N�5�w<���u.���c���z�a"$��������3G�/��01�����<�\)5���\Hck_nT����N�sK��n���)��}���������k�:�=��&� J��I�/��X@��������!A�l�s�xGm=��F�%��aW�/�������.�M��?�y���gB\���%J:��;c��]*��3�����������ns�G��\���Y����:����G �� ]
����#��r�<N�RByV��3yG���Q�a��MB�w�/����A3*&:8�]���yITFc�K���l�Rn��n������6�n$b
�"Fig0� ��G�r����XA�g�k�X��c���1Q�r
��&Y��#O>�Oe���� �~-A����M�m��J�����������hz.L�������k��"I��n_���}�������`�qu�R��c��,_�F>�-�3�������\#]���[��"���/a�LE�o��C������&�J���+��	l�]�T
�\�d�	��!���o����7I"<���)��zv��8;8����a�M��N���A�8Z�9T���62:��(@F�,lwu6�(�U�u#��"�����"���p�"��b>��f	��/z�t-�A
��S����^J��Q�W��\]T�6���6�B7���K3�;�dv�*��Qwu=r������B��"��3k'�X_{nr�A����������=�>,JT�fR��f^	31M��N����s��e.��)U`���R�Z	�3��U�C�"���W��C
x�X�#���:T+��u���G0.�-�9����C`� ��d6n'Q��`��T�Q�vRp�������g	�58�������A[����~�HH0���H}OG�������n��t�
�X��A8�T�p+sy�����>n�����U����@�7��C����Dcp
5�(3������S�O�j�`�g�HgI���L���(��������T���^��>MI�7z�	W�	e�8R!
��F�"\���Y`i&E0�&�b�� �K���?�E�?�_����
�&I�'a��D��YpV�����d/�C�������c�n���x:�I��PH0���~_�\�G�!)`�\�������}�u��O#��V\�
�Iy=��%-�b ]�{� ���\���i�����Y`Tv>$����8�y� ����k��������\�m�o�0��P[$���"�b$wO��V�����g�*H�TL{��\�-jN\�8��F��k����Qi�Rn����u��(������3��\�����~o��`�+�fL���/U�TYT7 �C���J��l�����_����p��m�4���� .���y������t�I�~$�~��v��O;iB��S
6�r��R�����@�7H���EzED.C���6��s�����G`��d)�f	z�iB�%�R�zX�reT.f	`!�n��;���ID)$F1�'JM�����~�c�n�z�[?6����W%����]�m��R�������}���0p����y[f���B�_JK���3���Iq�8�����,�)���d1�]*���K'��^)/�������
��I�6@�����y���*QH��)�������V���oNmko�o?�NA`?����C�X��
g���
"��c�WdP����v�k��_��X�R�c|��'^�����������=IR�y�ebur�I��H��'
�I�I[xi��������~S?��q��y�x���Wd�)�QHD��D�������!�kO.�n���}�>��-�@���~m�����0c���v�Wp�=Ww}e�f{pF�/�f�}�-��9^��4Xa�Y�W�$3����>m�}��d@�a�f�\Z������Z��#��������d#c����?�"��=����~�
+���x6I�`������C{s���/w:��"T�K��v2�Q���%���L���������c����_�������&S>���3��}59N�.R"�q�%�KL�]�rI�����������6��7�_=`G��3h)eTP�5H��Q 3A�1+��9�1^�[���7��#�B�]`e�(Ax�����t1��A��^lL.���xe.d��Au��	@����9�>I(����47u�2��k���z��P(��86��-���$t� ^����<�r�=ME2��;'7v�A!n`#-�����}���G����Nw�k�4���%Jh���������$�>������\m����/j��!	I�6JBbB�L����{��1�v�o���A���]��$4�|��R|#�����!{��KK�6���{����\I
�u���;74�1�`��YZp�.�_��!=�����m�AvI`Y�<������n{����C��`���!� ~4��c9�3�R�V���Ob�2=�+S�|��zM;���%!�����eR�r	!v��?�������|.T.%��l��td��=���s�hH�b�����G2��C�<!�����T�$0]����t����:�h[	����X�����;�K��%����� 5-�;�\d������&�{6���P�
�0�����:n!o��t�'f�b&�va���Qz���+�[7���P����ty7���vv��`��B�Q��Y�bh2{Ex�2�����i"TL�3�E�Z����Ag�(��B�4�S
0?�p4�1E:��P�b���
�������z�/f���%Kap/��$e�s0�����E�p���8�e������� ~Zz�C�����
d��(���#��
09�+3pH���I�{1n�?�C����3c>�2����\l%���� 
�(�aH��E�R�?v7u_�g��aH��j21X�}DRc�+�
�>w������+���s�#�
���@�"K��%�P�V$��������S��@��L�4N��*������?����p�R �T`���q/}�:c�PJ=��|��Z�4%FS?If����7�����On(%�	��BQE����Dz#1�f�M�G����v������v�3�����G��:��7F�0���<��q���t6d��6A���#i��� ALSN�2�C��q���
�+��@�/��I	���3�u4h@Lc#>���#����0����Z�`�	-��;I ����i��$o��d;��7�R���fS��C����.WQ��</��}�A�5L�����R�e.���W��������^��D�.8��xn��0;9�~	I��(������:=��H�e��\�(���z����w���jp'������"������qY8����l�#����7�'��7�;��8�Z�K���Eq�1�,�ui�mf01���<Z)Xh03o�B����j��HV(S$DO/�������F����g<D�������n���������u�����8����e��JX���PZ	�!��iV:c���G���?8�_OK�
�x$S^ls��G�S�>�������,9IL����YRFT�
C�q����q��2������6���H����i�6k������2�����U6S����e�!��G�lj(��x�S�Hmy$�'��u_lO��M�%�^�Q���;k�3��a����"$�UWd���~ivM�JPzBq�W��Xu�K��i�'�gp����]�4����.e�z���<]����=��X�����`�9����u�\�~��A2��/p��p7\��#�@F���\��N��s7���(8guis���;��>"@���D�J�\*���M����n[���V���I*��	�15#�>�9����6T��Le �I����S��2O���b|� ��{
)�*�*i-�	��}����@�M
�X��N�XRT��n��uNe�����>�
�!�b�2Vp�K������P���������������+�K�0��r��?�"8^�	@�����������[e�Hg:����7�Y�9p<u.���zl���\xh{���<��LG0��)D�\N�[w[03=g�,�)?g2����H(d6=U�#���������wnC���>S��d8��F�
4|�2&tE7���i���2��I\EU�w21P�X6���\e��4�-�� ��#?�
�'��)p����������U�#xu};5����i���S����J.���^j��_�����7oK�v�d��M&����DR#%LOw�>��~������������&���*��G`�A&d
	����v�����c�x�!������WW������3Q%	�Be._<L��_�l�6���;P�a�{��{�(x��Xs�yt3�kj�t��O�
��n4vDW�D9�c�&����p�fex.#���^�^���z$�^�H?�_:+��^�--2^���������;��u����=������P���F�Hp������p����x.��=�l�����YG�rB?.�$mC!b�v��iy^�]������g'���Y�4�������~C� ��`[e�#}�����4�7��kK�N�;�y}-�;k2��������k[z�����?bGB:�Ep|��]�J"J�"8�(`&�� H���Ga���fAx����L#PN�
�#�*����x��	('$�=5OwM�6� ���P0��,�vF-b���p4n@;}���y|p���4`����*�jF����FT��^�ld����������]���,H�;��b�8 *$Kh����W���������}����U	�8���s��X����!�u)���@g�����Mk�����(=�U%���4;N�pN���K��c3x-_��2���mF~�������sO�J-�q���7@��>�iX���.[J���)U��)���f��dT�Ni�5K��$�3�8=�h�e"]9�_`������rr�3������D=�JDC^0�	�7n�3(�3n���8������V�G*'�Ua�V5��Q������{�~����=Cu1c��734�p�����jD<[~6n�!��.J�?d�\�yW����7�����I��]:h2����N�	^�����0�+_4�A ��%��)��e���&����]}<6��K�n?u���
l�i4������*E(3�z����y^�Lh�r��XmH���*s��'�����S���}����x��y9Ar������\$�r��>��m�S������ v3���)J������m����h@8����6���k����.0������96%���P*���e�--�����?��z=v19-43V���~��D�-Hj�Q
Q*���H��r��A���h�����5_w[��H����L��\ �Z �3��^�2��:��A�2po?��2���2q%���:�m,���$�e`e9�+��eh��&�X-�N� ����Q�����C��Y������Z�e<h�'y�#�m�(,,� �����OF�w�o�-xL���_%]�P0�!r$^.�(�kVm�N�gi�^����<��LXB����B�8�m�����}�$�~h}h����r~�w#1����:�����.uC,u�(���:���z���
�M�n��P�5�_��V�!X9�n��,�<{7�[iA�]�l]���
K�X�^�S����������9pK����MO�������j�]��A'�ia�X�5��	��^\T����X�N;��/�Sh�?��=�B�yk^���� "(g�m/`�d���
{x\�2[�f�]_�]�&��(���,T���/����vT�H,����9|�3��dZ*��Z����1
�����|D�h��k�7sJRt"�1�%;1�0��o%�a�k�����(g��(�N'	��m#F����?�u���X��f�\�{L3���^s����0��\�dc�	<X0�������K�>82�};Ef_<
��������D�UH��|)�����YIsn"�U��Af�������!eGM�>Xr����@�����:s9[�L�����s{��Q�������g�~#���6��=����v�8�Y����Fn4�b+�Mt��?�F7�%�g��R���l�#!�fS�\@���wt��UM�&g���-��y��LQ
�><E���J�f�m�=�?�,��J��))�o�{&q�H�8_R�����k
s��8����M�e��L�5�6�]���h2���|���d�;���7u_C^�K��s�W0)z%�
�����i�
��7��~x���y�������J���2o����e���l
\��Q���2��\���"��L�&M���|W�R�}&����h�9������t}j]S_�.K���xWnA��
dD��#�9T�S���p�o����7��%~�
������$�(�\�53i���vc������O6h4E�5xU ]���.r�M���U���o%�c���<�6�/�d���\�%��O���
.��.�����&=G�����hoOR"�E@�#e.t������ol7w��^��4X���e���{N�y���y ��\hJ@�����c�i����>:����+���-�����C����C i�L&���rD��������R�$�V��3���7��l�����-��!#|	��S��|�eM@t�Y �/��&�?���7x_�S�i�#����-e:��>~~�Y�8�xQ
����h�3X��]������v���������{�f#����)�����f��L�T.�gUu����:Br=I��R���b^��Sd*�,��d����6i"e�b���Qs9>d�!a&3RW�Z	9�C|�_6��4�i�XM&�m�T)�mw��4�D.��+ ��@���)�������(����&�������^?�j�-���z>z�)�=N�\I�K��������}��f���
�Q��m�������u�l	B�)0*V$���c����������,	B}�$����j�
�`f��������T�b���&KI�{BB&^I���GiQV6^������i���"e=85Y�/��9+��#���M3
2�5p���vQ�$����]	���R�!('����|����v�BNI��l�!V�,���8i�8��9�t���d���������ZH\����#$���^5���u8�?���n�}��!e����r�����+j��J�\Z�!���4���q��|������-.�`
�]�����������c�W����������PbG���,w5��Dr$��?]=���l�����C8�w������rn���W����5A��O�.%��H�^��d��d���S2`��$�/MV-g�djA:�����D'�H�m�;�e8-�����'�(
�������(�81S5-fP�h�)��"��,8Q����Rw�5������y��sJ���U��"�(y�N�a���-���� %���5�e���`�j�lH���y)�u�����}hS~���&W'�����'`%S c��Fz�n�a��8��Vn*e�r`ND-H)+���p^��Q0�Lj���@��R��>L��"
����+�����c���/��xJ��
b��)�����������g���e���Wq��m���/�Rj��N]v0�bh+}�
2�������&�:h�bV���Aj�<-���]������=���;JM�Gi�m>.Kc����U1D���v��G����JZ���$�,�E���j,'�;����������->r��w�2hIy�o7��s-�'��GT_���U�������d������k��/&�F�8�j������4���B@��������)qOy��!��s��r�����&���l���4
���r�T��������*	���+N�T{��v��n���yFJ���3$�������&p9
8Coh������xite/���f�����t�.�>�.�����V�f��0w��$�	>�n��]3�]7e9���'J�0�Fg�T!sg��.e�����=r����������������H������Ld��R���Y�:���R$V������g����p������H��X��04�M����W������s�v^��t;��� �_��y'K'to�n�s�����n.�������G������#�]
�~5�����Q~;L����N����m)��&���������o�����g^��g�rD�d^�
�B��V�0���u^�+L�e���N���QHI�I�"����d����� ��,f0x-q�\	>��o��9��5E��y�t�1��d����H�8���b������,Ss��I��l�o���=����4	
D�������_s�-��l1���s{�q�]��U���!RE�,'`�F�aC���r��f���n�_�Q�x^���7��'�q1���)H�<�<���zJ_%j
�d"���#����5�4� q�.�iCm&*��D%[3Z
��=$*s� ��K�9�#�
:��!����U���������v�����8%���1�PH�g�����o��%��:m�a���3�d���IL��"�z+�f5/&�m���}�<_����z����hP[-�x���7�N
�Tg��&���P��@Vq;6c@E��%G��;4�� c�5����p������pzy��%���r��+AX=��E|
��}=e,,<5Uv���D����JY\z�usO9Tb�c����������~��X�A�k�9q�A���8.M�N�f����
8��z��Q��o]!��lw�4��JW�&=�����9�t1Y#�{�I�,�0d$��dzh^���H�9�gK�<g�Zu.@���������>��Rf��&������y��'M�uS��y+h��)��4�*O�-5�_}
p�g8�<=��I5�X�*����a���!JQ��������7A��t^����.�����J���HA6��p����5��Y�C7�x)X>L�B�����=�W�R.}�c���a�{�R��LS��r�g��[���r]J�����<�	~1�L����������G���KuP���~�C�_�7!����CN�e,s�C)���k���5l�eu�zF���h�Y��s�Ac�-���&��&>ff����7W�������-[�
	��bz��K�(�|d%l��0��q���p�I8
0g0�������G�u���%FE[����jZ	\1�����=V��]��"D��.�#�
$���i�{����Z�)d��|�	���U�7�M��B����/�}l��xUHP� x	���Xze:H[
�&&]
?��n|%�W��x.���9�(����s�H@��i����\VI����G��d��T�8�C�����3���������07K��M��E@9��h4iz�!*�hrqu����T��>����YX�H&Nxx���aB��p��RO+�����������K�����7���2��T�s�h� ��"�O�������y
Nnv����t?)5[��	�"�3�^���G�`�3�x���`h�t����� ��1��MgrED|X��TOV����������<�$�_b2��,��y�3�a-��`�-�}>l�OV��"l�
-7������7�R���v����
���-��V���(�0a1���
��m#�+������������A���Iv4V��V�3����U�����R�o�O��>@����Y}s�����We�������X����aE��`'�&�����m��!]uE'�1|����kkY1��_���n���������M�#��mw$�-���R��gu�>t�7���
	K�2@�������b�2@�$��<~q�+�t�w�a��aWi����#�P�|�/9�<��TXdN�(������iiX�2��`0����,�!)���~��\1�����kh��x�v��l�|.>lK�����d�1���.��I1��:H�������$C�vp��.��)e2��57C�}R�MNl��a���<49C�PG�p��RL�����-��O�D��h��3�*��"TMu)C����2*�U�
���fG��������bhv
�:t��k��o{�����)�f���N�a�
���/J�'h��0���� "���f�[���K<�e'���6��K��7+�
e��G �
�m��c�l�w>L��M�Q����4������	q��l������G���2�����9Y�����^�1�>?dY���$��l���QO�B����t�o�{|���7	$*������PtaI����};�]������C-q� �a��p�"78���KQ����_!���~����ET��s��xry$@y��rn'����A�9ow�/^��L�B����Lq�`w+�QjX��C}�e�-�_Q$��k%������@�^�V��`@�����0q�O��9s��b��y1������O@%$���{)�+qA"A���L�*,�Z��|)���.-KX��e�(�/����z>��0i�]�I������9J��xO�T���#E��M�����
���.����<04��~12�b�j���2�A(Z^
���1�G*�2�<����2$��\�1���`���<��u�Y�?���Q1��N����.��:l!K������ H.B��r����OJ	�X)�����5���alG/��W~�1�W����)(�x����,-�R�n_?\��q�4~���!l4��3���!����8�:��I�R�0~�_����b�E&U"4��`�t#��&*�"7_�A���[���G�Q�?�r&#��
"�^^�rv6U7���x*+HT��l�x3Ty�n�����@-:�k���+�����i��*$S�5�+�pa4��,/:�����x��[b"��Y�?7H��u��[��n�`���Q����>l�����X������%��Ma�q��G����<O�aQ�7-"_&OG���2�l&4���e����)���|��&P%���R�]�6I ����EW�����������g�(��n���$d��m��*��L��lS]Vq���	�����i����*���/\�����v���;E���an���3i��=7Or7N�b�a����_oC}h���$����0n�Fu�U)+
�����W�04����7�W�����RS��vw�aH&s�\3�<������g����O��[s���/��	�4-f
��0��i�1{?�R5����'D��}2�����c��C�
���z��#���Y��z�K���7#���Y�{%��F���|�g��W��&�a#��^r��HF�x\�)�k9I)�	��_\~9�r��Ld�n@����������������5L��Iov�e���=t��nKS�D�<U�0�+K��@
���5���+K����I�O��s��S�8�V����=-�8'=���h�Y���z,���@�o��+���d!�M_@6�g�i�\�A��B�
8��`�}��Hq�7W�:�vJ�e�K�u<9��<@*%�;KVRw?ul`�~��}�jq=�%����7��(�7��3a6�i��������v�	�	���������~�W���T�B"���o�������{I|u��]�q9z������-l:�H4X'l��xH��o)������h�j�n������@���
Y�lN��^&���7��67����))�=R��:1�T������������a���}
I5Wz�Xe?}?^�;�
;�@qG�O
�� w���@�jx��A
So��P��d
���E[��_�m��I���_�]��;o�^�9W���h�����%Y�T%�)���@�*`��K��\�#����-x�S��by��� ��� ��s�3��k�(J���~V������X~��4�T����n��l���v�7	���{�������48����8l�A�j�s?��RU�������i� �g�K�BF<

'A2�$W5+�����pl��s���,I1�,-�����eH7^�w��+;�����
h4��������EDQ�6!&z�s3��@�|J'��_��/�Z�vw�|m�c��`�8�����WK5M�TT3d����	i������������&���A�n��GJ�������"��<J��W.`NjW+5O���u?��H�xu�S��Y����)�������"�|2�����3����@�u�Z���T��,�o��[I��I�&�	 �	dR�����u��?o��S^q����&Q�B�){�Aq�&��V%������y�8DqA���2�|��r�L�J�<fH!6��	��:ATp~>���q�>ON����d�+i�����-�%��0������n(��t�@����K�*8P��a����^JA�G����Ry6<Dg�#gU�>M�^�R
h��f��W�qg�M��,��.N\q�$�8�8���Jq���W�=x�J����QS�����1��tA�����R������E��K7�`��p(�qm�e�<��F�����{W��X�
�S �Ld�1��c#N:Q���������hWc��v�
�v��W�����dM��F�e���R\g [�Wpn_w��������J��Q��-���G���M�1ajMJ��db������cy��3����X�2:�`���x�rp�e��+5��U0��tgI�����q�%k��^7��Mf����`y��e����AkZ7���
���eV,�,��Y�FY�h4)�A>JZ���{��x�8x?�/��#v����dr�d@�CzD����gg�q�����������&s}������s�+��&s}Q�K�7�nF�"-��JjT��^���%qQ�����2#W���oP��Y0�`�2���
U]Q�n�Xq\�����t�~���'<�s�C-���3tFp0�p
.��`-D����;$�3�{H��b5=���"�<�%��'������*#��\Fb�!0����%>
�T�{�TN�&�C.[b�J��7VZ�3Y)&s��T"cE$��%��Up����g������w����72��W�^���c��v�%�fx���JI�k�����@o���/Q��o���~����	����VJ�F%���N����a�'K^]9F7���<<��sH�8�&t2�����-�U%��B'������j�$�i�8e�b��>/���w�.Hc����-+Cf�%��s����rb�O���s��LuR�t�j����3�&�8<����0�\\���F6L��r�=/`��� 22��Q"�p��5�B�/��nz(�%�L�G���h�}�@�h��WTW���3�#��v��~�5�����f|<t������V�5�FA�JuC���9�j���+��7'a����T�9q����=�����)��<Yr"��\�'�{�@�
��%F>��B3��'�����nZ���o~��P ���x)L�[M����)�
������c4��|��m�Ds��i���u������{��������y��Cd�����S�	���k�sM������P#'3*\��4g!��H���3�����(A��{7���
I��?�_�X�?,���1�N������\TrpQ��v�;$Y�~?�~~���F�u���o�or?��,��`��R��Bw��uK8_h7lRSJy��I�_�a0�*G��W�����$'k��?���jH����,j��D���%U)V1���\�	����;:@p������m#K��<���~y��c���8�����m�)J&)�{��)�������Y��O����:�|���S����U�����H��I�#Q��s�������~{�l�G�x�QB-/�Bt�#��Ba=M%]q��Zh�d����w����	��{�&�&��]G���d�������`,Y������^�V���})i�H#x�Y��u!y!Q��*,�3X��^��|��������+�WT���Z�U�ID��S��SB����E�n�-��\�U�Ih�\��RS�����&�`������q���:���t)������l��ZM[P$���~��j���r;x�P��!�~���#Q�#a�Z��2�~�����
����c�8u7�wg����y�	lt��%"?��s�}������,��Pc�q:�RUe���n��)��|��*�;�e�?��t\�Tc��bB"X\/�r�m.+Lh����jO��������}��/�����
��p$�1����l�<��}�;Ga��*��s����L�@�Q��{8��_��9[�B$q.��-7 ���N_�o�<��%���y����Z��ax��t�
��_�T��g�^;�^���XB�Pr�ec>�
`M{�?��?���2Cw\����-S����r�NW6���\���r�R��PX]0�����B����
������u���#a�94�0��s	������_Gq��T��R
��r&�K E�8��|�����3������l���R`-�]�%��)#(�v!�&�G��#�����I�SbE.�C��������������i��T��_^%�]�nu!p�g���gV��w�|�����m��jz���R�r\�h���!�����T6�E�5�`�v��a�NI���[�������b~��j:�������s0�����������8 ���IHl�`�&���L���	o�"�L'6��^r]H����n�b�T��a]��5�2��%�D�!�\�Y7���v����Q4�B��"}Uv���?� ��-)	�a����z��'������3[�/ e��0�tr�����6�3��{����C�����a}��E�����)�@jV9�j�\����~]���z��6�f��%\d�
�Q�~��`�Al���
�@�]���8��%^�������a\4sZf=���.a���H6���U�l�?��%��2g���F:��EW6cO�>�����H���F�{]�"X0�w:���B�T@������'��e���g=m���
�C"�t�IY�S��i3q.E�6���c���0�]pBJ/���"GHa
�N(��
��ht�����3,}���a�6{��jtA	E�s�"��&C����u��?{����f���hN%S�.Fq�Z�z�A����0��&��j����g',���I��@�p-�����"���o?J�U������uk�rS7�3`o����dl���
��^�r� ec���s����
����3Z]�C:v�)�,B�`��%p��*8���7��������������C���gg/�L��q$�	Wv���H����������c=��]]
C��}�f����Ht�<#\Ql!�=	^}p��{F��)���y�/'i���t�FL�	K}���O	���v;�9����~�Y��iU��������'��)�m��e�t���-��������P���tb����(����R6?���
$G����u���klI�O��9c�fH�O���ecj��{!�����~}We��zR�u���G��E}��.]��#����?M6�Wk7���q!\��t���C�1m��{�����p�����a<Z�	v����A�=�� ��#�'l0?��������l1��-lx2Q+Ms1����_&�J�F�����HPJ$	1.�`A�h�u�y�l������/���������M�Lp�,�%7�%�����%��������+��z�{���o`u���@�mTt5;��Bqv3nb����>������^�!3M���b�]z4�\W%s��}C��������`��U�vU\�<T��z��_��*�T(08�"{%�J�>�xEo�0�G��w�w�?��$_������4n�����iZ�+T*��'������6H�>�����\����2��K�%��0W��h}�&Z��� m}�++s����}W�h��E��\�Y��YPK���f/���7����F��0�+.�lQ��M�}f�tE�^l�K��m��p:�C��t/��l���n-�n�H&m�HY~{��8�(�t��Q�51#�s�f�55��-r5r��>�\v�S��m�x�[�8�"�\���S�������l�8l>>�+�����<�4n�(\����N����w
�A��ci.C'd�����d���t:������^��9�d �����dW���y�n��Y�41LPI�J�����P��
F*�����O�h�p����V��}_���6G����d7;f	����������k�uR.ie���O���c3~�����\P�$���3��'$2�|o�=��/x��gR�!&��.��4m�h��C���#���������)3m�el]L�(��j��|���M��`@���)�M��5D���nX��@����B���2PH.�[7W�	`�ou!�:���4�;��'��ih�b,���~������������]rp�/���@a^P�rN��L���;i3U��H�`�)�a�bAy-E�nj�`�8?��gbOB��<$�5E)�v?����^�f��]W����p���~�<��1]�Fo2��(w�Dx����i���Y�5�n����:��
~�5�'���C��8?I��b%��lL?����V��M���������������K&���G0n�e�X��������\�H�%�'�!!�<��>�@���H������C;7�B�?#��- W���*e0�c��/��F�p�b��Vz+FR���	&��nnhD��4�d&9�j���L!�<���9xq�oD!���O��}��7��f�D�����i�����==�nx���H�}W���e��ey|x��X 92�F�5���U�)[�lw�]��L��
�S�����P$���.6�kW%��v
�7���������d����Dz��b�r$i��%#�����}��e�*�������|IQ��%���r���s���%u_��$Ke�Fg����%htT0d�46�2�E��{7��~��TC��� �hl��DrE�s� A�#�m(�H��e�����s�����J��t�������'7A��D�2�]=���G��Y��c�A�a$��`\�;�%f@����rg�Vhn�%s%�T��d��9xa�v3�B�2��y2F���S�#&���Ec4�'��!7���k��(/6E��v�\��8�q��!S���j6_���NI��[0���S��wJ�����?�U��`�h:�������j$t���X��|�������'���p���x�S�����0�/�
�����)����������	������Y�����(W���4f��U�#�8Z�o��2��jX��\-N�x�f�/sP���b"$�K$��?X|�>�1�����K����J��n��p�iv��`�=���J�GU�%'������Y�|4�X�F	�*  B
���A�����aXW�d	�Q�0@�1�zq�-������d�"���Um ���~W�k����mA`��a��!�A�a��)IpJ�`�6�md:}4!�A�2PJ����c�[:�20PJh��"�������w�;lh�O.w�}���p�T����}9��.��1�W��>@�E�������BW��]L:�8���^��J�1���z$��$Y*�Je��(}*�����"=�(G_�rI�n���L�d��]�������u�J
�������l.`\�����h�����w6}��!��"�]����5Ic+��V�/A���n�Y���q?��@Mi���\V��^��t��)x��L�A���q��`F��:�����2=�0A�:u����eV�j���>������c�&��
/	&~��8����V�I7b�l,(o�8t����n�^�,�.��DtB���#��T���r��z��m �0zJ�����u�K��V�`���tLh,3XO�[������9���u���8��~�0�47>��p��eH66%�f��v�|)�c��<4���m>�����\�	�R�r�3��q�T�*�=8��[-K�v������d GG��JbV�f�]�j�4����dNKcDV��v-Q�|�����;��(�N�D���USnA��S5�9B��)
u��#���'��
aS_!o@<�i��H�����Nx8�,p�H�s����?��������&-�d����_M
�$6��Yw�U.���m{�^n��Pw���S.�����k�����7��!ze�I5��y�^�'D�^������nDy�i��R��{B��/X�j�S*���d��X���(��}�c�~���*���o��O����q
�����f�HS���t�o��?��>-�4]4���l$�)�-h
�
��l%�=��k��r�+fk�h�1�cN��w@������
�g�D��Q�����(�XD	������.%2D]g�v������Y�}_����]��*H�~�,OP9�k��h���$�eY>t���/��/u	Ta��\�9ESY�u��[��{�pW�Ba�����|�����`$<�E^v�����l��O���W��"jy�H��.�9�+�U��t'2�e�>�1+	R4�^�
E���:�Z	�0���~x��<P�E����
��P0(�T;/Y�����)8[�l��U��Q���,�&t����{/�BbX6d��������4?�J�{+-E��������Yj�����:<������7��Q��������8���A� ���'�����CW�iKTT����x
�[���2)�tLU�}�c�����}q��WK�"H@��eu�Z�k�[��}�:�+�]�|�&�m���ktN<�I�*pY�����_����m�!\��&p�t0��zWn�O��`���6�vo���{t_�gs*��W�B0W�Q����������U���r�[��5L����������|�� 0���m��7�<M��*X�NA��3�xF�[��J�\@�f���}NZ�*���[Pu�&���1Q��_�u�?v������>��zw_w��06!�lX�B��������H@1Z�Z������_�����pH_�H,��9bf��+�tV0��J�;=�����y��Z/"

ji���t���hb�����w�]5T����l�.�n���Q�5�}�������P�3'd�Q��Dp�d�*������z��	��8���a���3*�U�<��z3����QF"�s����`�]�}9�8��m���*mC��g8��&��J�\(���P>l;�0���1��(QB&	ta/S�^�e��m�m�C�?�4U0*Kw@��85�T,���� ��Vb��b���]S_����s9+�0�����CBn�`~��k����gO~c\'YJ��E���s,b��Wu�V+���p��P������n;<�_�U"�.�FR�����;6)�v��>�[����|���KZt���j�M;\t���L��M.3�+��`i#��8}K���B�V��`�����l�wI��M����'��"]$������Up��%4�#0dL�U�S�?����}Wo�kw������i��5���O����(3���+t)3�K�ACr��e�S���C�����T�^q>���+�a�k^]�����I���b4��+�~�
U���~��!�r�����Q?;o0�}6E���� 7�@'pe���8����}"+�R�TTI�W���i��o���q�{��Z���;����.[?�����8`�U%w����w�t����N����~�-���$�����.%��U8�G+_T��?\z�������dZ����<�!\���M��|� �h���

<�>U�c=�i��f��MQ�O�N5V*�Y�������.���h��N;�p=j�K����� ��8�x.���3�=��oe=���HoJ�V�b�^��.�Bz5v��]|������l��`7Na��v��A���n�&�1WR��,���_KC�S�"���dQ(������D.���R����l�>;��ED�P:�����:��*���?�6� .��?��_��:���wLl~�C>2|S�p:y�rY@��i.�����\��+��3p��X�z����
3���(���y����If�/J�mP�@�	5���zH��6��D?�s�F��(�@��A�}E�J�\&�]5�?��v�
�����xb��wR $��.��<�y�����:��������������M��?rCb�
�m����+���H�E�z 
yo+�h��w�x!�U����bY����$�]�N1�B[	�
�a��33����C�����\^�}���O�� J����X��Z��OI�V3��_>I��g#k'��h��|���i�.p�b���7�]�	���^�4(����O�R���ZC1A43�O<�)�pt�����c�F��k��4J'�1�tM�:��`��HT�q~���][�O@�����},�s��bM*&W��}��Wz^}��#��QX�vC;����h��_{h��<e�F��V������A�s�Ycqe�������u����@����K���NPU��(��ltR�}�q�,��(C"�;�L���������u���u/oQ���`��|�t���b�uGb�dSd���������b,YM1�����l78����������*t��-�!���h&�	�q���-y%d6���u�h�e#�`�>P@2�$a������(I�Ku4����a\!�g�|�o���};~��]�}�!2���FN���yv
������1�����3l������x�+����<�L��
������C)��$+eF��b�����1`��������p�|l�]�t��B��QJBP�eQ����J!E����Y.eU�����a]�N���+8�]�E��q�F����b=J�����A��,Q�Jf���=����\@��c�MNs���y2$]�>�gH�v�Y��3 �m��<�}y���f���������/�.)�dc�Kn�6����`�uU��8p�mn���������7�8�RI gc����G�(
�@F�3Y`�sl�%�����?�DmC���-$�k�����G�W�y����^���W���*��k)s�!]Q@������{6�(�����������u.�a�uj;��dc�:wYV������{�~��Sd�lF����]t1���=�?��n}6Z�d0�o����]	5Y�.��s�A��`�����f�-?~*O���8�h]J�!�7!t0����U���U���y��<����Gh���c"���T@�a�\�����(�W<�<V��D`�|�>[
UK���E�������-?�MW�K��a�sN��i����\���E#2�����S��~}��{��4�A	i0qe�/NP�'�~�^�l����G7��S���N���De�	t������L��^��m����^,�������K*H^�����lX�-�����:<I8%�&�y�`�t�c���|Y���&(���|0[Gv�g�RU�>����l6����t������jwZ�Ea�E{Q07�S�V�2���/�^��U:�W����+t`��&�q���t��zwh��v�I�'y��������y���Z�.2f6������������<�
�'���h�u��"2�'��Xad|QO6+�����
�i%A������:z�L�����i�����w�'���qT��No�Yr F�9��"�8��t��]z��
�d���p?j$�}�=����c�~�pe�����))��/���') ����)\����me]_����:5���\@��B�j�,����@��\
��	J�[N�B��N\����B�m���K��������|�����t�y2���Q<��&���������*����O�w��%5M^�z�Qbcy���h��	M�}�#����s�z���_J����X����O��hA��ck��JPZQ�����:����@�EW���:��x�U
1.��#�����f�y�����uU!B.�y��xc	������Pi�I�:U��6���W�����Wo��J��@M�������S'*��^X��� t�U�GJd30=+,���r�U*y��9WTw��6H��W$����yJ�����1S2�t,�����O�!�:�{u����}Wo���R�4.*�k$��5+8
b��i���b��|B���5Z�������T@��������r�w���s�L=�}w#g��-Pg�0��s���|�����:��p��'�� �_�|_��qc�p���=f�0��V�e��3���P���K��+��a�D������.^�����l�OOI\��a�A�~�R�l�)T:$�f@���p��w�G��	m!�&�<6"1a����o��x�
��������Pv��o8K��-z�0���8}2���7�E�)����>y�d���&��^n7U����������f��[��>��{�.w���G�����;�hD��������K�V������4�D���s����d�u:�j^TL�aQ�^6<L��$:�vw7��8-���lB����}��}�����������~y5Qs��?�f���-X�$mDQ@�������U������&���!���F�g��p����s�z�����e��������{�h�z_G���(
�(�!�`�or��}��2��+�iE0���-Ub-/9���<�]g`K�m�}��[�DA���g���i��K8�m�T�D�����!K����o^���D�h3"x"

j������BN}�R_��E����jc��m�����p	�N.�:������M�	%iJM��P�}��q�w�
���Z1��~}��u]v_��o���TdVft��K^�������:���\���(��A/�
zI�3(�X��B�j^��qR�2�aq���
���J�^�Yb���2�0q�m�,2��R��3$�\���:��U!��XO(�Q_��d
((G'���&_��t�Qgd��[Z�LB�tGU���&�E�]e{�������k<K�������h�������
��s�h@�a��w��������9	{�5r�5�d����/�e���r��$���X��a��/�FCI���F��z�U�5�Q�5_��Qo�k�{f��9��k��M���S�Q��\-jx���\y6S;�p�U�;���Z�n��FV4�\�FD���������8�(�4���y����{��7W�-�"�LD�<���.W��<w[�s���z�{Of�Y0�KZ������(U�g|z����C��d��all
?��U�����fd=}a8��06L��E�A�_k�6�]H�iB�h�z������:�G���X���+�����
�b������Br�p�<������O���q�"�y
��c\�$�ds���G�*Z?���������6�����)d�~�2�#�Iw�����?=�����`~�����4�a����F����'��Z7��S�&��;�~�&Lz�'<�w����+��� ���E�L������X������A`�H�Ke��F�sWu�5�W��qB��S{�YB�(�=F����U�,���1���t�G_������8)H;�Jf0UF���Lo�)
���]�@�A�������U�GO�(�5��py���@|D�:��L</f�^�;��x�n�������-���\�J���t_=��JP5�$��c1��uM���	��O��)�������{L���D�
�$K�x��`��nEVfx�n?�'��������������������G��L	\������;c4�����n�zw����w
D��2���)�&A����##O�|�o�Z�4M�
LC�F�s`�O.���	
&����c���I��`L��
���o�[���88�y.����\B����R��V6+����8w��s�������]�����k��w�m,�HvG��L��V������H�!'h�x����w?��`e�d:��Y�������e8�?;zSf��
)�������2��!���q.�*se4��Bax��g0��Q��:o�KE�����JX��� G��cbs�������IQYL�!z����S�k��,J�n9_��B�B*�f���f��=<�~��eze��������
$9�}�4+�r�P;(�����Q��!z��#U�)"�b��rG�>�Oc�
L9@H8��3Y.j�6����][���\|�/^R#�!���J���������y�dY��a�����d�1���������Y��!����!�y�dX�f.���Q]���01��o�8�$�����M)l��Hg�����\���n��sF��1���N�������:=iJI����</[(��lHWC���������V#����F0�4�	�8Z7����!�1�]��.�H��lC

�0�������j�wh��rs�E���2n��T��$�&�r	��H(��@��|��)$��?��|��$`}��J`1?��x���c�����?����"H]2xw���<u����>�i/iy�Qa�������6��J�\�����v(���Wn�vPD���f�$�YP�h�����e.������y��������&��t���"�D�?�\�ioO��=��\jg�Ry���z��A��8��i�?���6*y}
.��=zgh����1�`Z�Y�����[hW6�K7o5b�4e�cMu�L9w�����WwW�l��w���u��U?�s����Q3�����)������&d�����5��R�O���*}�t���p����!���H�WZ�(BDM$*�e�����@���vA���S������H���hq���t����L��mR�_�w��H��	���\F���B�U�2����p��J��`�E��%W�\����
�4�3O�Z�Nx[zA�=���qM��� �O��/U�w�j��s���~l����
#9��	�c�0�Ah�	N���\(!����
��}\tAaL�(�T��NR	G[��Y����8
Uq,MFn�F����+�[7��L�9M[K�����D��9���&�����8G^�T&��4��Z���2�
(�~4��t6A�w�\�u��W]���R��:s�G���.<ge�ua�N�	!$��A��������d�Nr���p�9l�R51J�G���w1���jU����	:`AT���$���q,B&D@(��O�����zN7�.O��]V20��2	G#�pZ^����r�������s��Z�u�r�K�t��A������^
��n����,J�$A���������@&e�#�8�|�M�b�`����M����Q����Z$�U*��tB	��%2^�T��[N-p_(G�����s��S�#8�i��e�F�q7���/�2�b2����hi�N�=T��jI��k ��*�������@S����d��UnG��b�B�*J%�`H���>�C���l8�t�������9�K`�0�|K��
�Mx�K-^C��h�+�D�l6{���� �E��D���s�{���<�l6�u3[��`���1;d�q��E`R�FT�����8�����X������5�G�<H��Q%���^��q�uH]��h�w����_���JW1t�t��B`���c��6
W��W1,*��������Jx���,�6�RFERs%��n(n& ���r��%�����#�5����Yu�$��}���0�����U������qqq��J�5)#����4v����0
�~����^�O���(�C?�����H�5A=�]��)d(f0y�����U���;�nf�59p�c����`8�\���dH��|���
\��du�����-8k�i�9��ar�m.l�uu�M3��y��,��('���3>Z�^�p\��/����Ius2�d���(��e�$��`���Wo��J������/~�J.~/�x�R�9bq����<�?���O���d_G��CK���I�g�	��`�{�/��$?uU���#���e)!�;��"��B^�}U}�+������f�[���S�����l���$����E��>��2{�B���n��]�w�0��R�!y\G���Ru.iY���S�����pA��A����B����������7�N)�
�Eo����+����N���Y�(��� o�h�r�>����C�U[Y+��$O���������%�s)�9�&*�����U���6�������t[������QF�!�
z����9�5MT�$���T����� JL�3�\4�m]��g���d��Hx�H��t�5��	���>?��?�'����=�	a���~r��p�.��x0��z?�f�������46g�H�^,�@7�����X�E��6�������{!M�:�	��:�:��������V"6RYV�M������n��K�-��eBE��
0UX�v�x.�w7?����� �����$]��a����p���7����rH�`��g%�>�=C'������1iiy�
��&�y��X.�fPO
����2�9�$�3m�R�4����m;-���N	1Qq�,)�����P���4�W	��*�!��|��������z�9no��7���p�[��|h;�	�9$��G��w|.=��[�]b����k�K��n$Y���l*��J�8��b����@m���j��/R_��b�r�
�G� 3�����T��>�������S�Z�����+��n�@y�C<2g���]�i)c��������GO&z��3{%Q������&�m$�Wt��QA�����=��g�j��FLL0�U�V��%�%���/,����\;�����h�@f~�=�������d�si1�Q�X����'�%:�O��'$:����`�c����\lO���u
����_�fi1�)��U�Ir��@��w._|�a�������c�&EK���q����6g�&]�'�/R�z����G��$%���g�'�p�g}O!f��R���lf���l�DU�`�k_R�e�~��Z��&�Y����j��.��|"��h�s����}��AyPk����M����W��H�J���F�����I�&3u���a)a�	���7�����)���I�����0iWa@)�
����FMMT�v4L�D$�����ZdD����������<#)���v8a�*��7Xw������
<"X:T�p~H���s-9N���\)���jVH��m�D��W��S�m<!����FKt��i�,��I<����
�m��A�)q�D�����bV02��<��y�KQ�K)���zwm��x6r��7o�V���k}3�4��k����	
����n��6�Z�|~BWi@��7�����&7�']L�wB�f;v�|DIW��IH�.+"g�08�U��,[j�d����(�8��\i���IG�����l�M[oS��+����zFG���$rl,3�8��9'V���J�;J	j�u��/� Fw}�9��#Y��]��Q���s��D�fK�Q,s�z4��c�n*&
l�
V���l��!�N%�Ni��8g��VE8��4z��-�xw���)�����:$�r��AQ��|���.��@9,(��a��&��v���Mw���;�)����w.}\�����K��e&y �"w��p��`����/�AM�����}����[da��lV��[����X������b�s��H���=�P@6���3-u�U7z�����rEV���5�����F7j��Q@��_�q{p�0�Q:��-�p���r~xh��t���d@J������:���EIy��!����C��R{U�tg����
�����P��w���N����O��6���%9)f��ZJ��r�(�IoC����9�/-����;��\b����s��lRY��������W����\���g�9���q�H+>'"���=6���n
#�����m
m)������(���X@�PH�&�J�9�O_��+�7H�l�$�aFo�iA�U!f<�.���!���w���'����Hqvo�'`�HDKQ��0������6��o���.)���2�6>��]'8�j�.sj�ou.&M)&%���}m�
T�i}��?�G���`j���#�}�^+�c��6�{�������M����h��n��M��4b/#l����-N`���b�k.|zX��M�oN���)�v�	������p�S���������Q?�����
����r������&_@c��Q�$.������}���q��)��)4��1FJ}4=�~�E��P ������rAf�����f A����wE�`(��m�D�H�.=Yf~�I���ykRz�����I�q#����B'�Q�A�T�/$U.��V5��o���}������|�jM��I��j"[H#WC���ga����i!�T�RX-18q�����0qKJ�Y�|�
�+=��I�n,=���1rd�E��sV�|���Cx.�0���hs{�j��mC���."<?nj�et�".
�Kp�����5!!��|����>�
�*6����@e�N�B�!~`d�L)�k��}�h���G����"-�')�����{7M��	���X��Vq8�~�/������|g j�������#��tH�O�x��\�?�{@6�(q�w����������-�Y��l0��B��3�n�a�A�"��G�Om�:[u�a��P7���V{j�@C�*��+��ce������8�fi��2�S=mOMu:�v�a�#(�,4{����`8) �G���.��^7�8E�p�7>����H���S7���"����aS���m���c{-��3:w�����8�O��%1X�*�=�%J(�pU=��@8�a�L��4-�pp��`
�C35\i�����k���i1C��N�f��)�������d>��0���:R�]���qASP���8�)��2���6A�S�n���O��z�D�
�<���>O�u���D.��5������?��`�s��EY��D"����-�m,mEG���5��6����7��9�h�G4��
�s.7���� �����X����\��Kr�$ei�Jp��
�Cl5��P\��i���j
����n{[?����dn;��uT���j/������]JY<J����_eV�5#�x�Q>�=&�~���	�Q�����i0��
SA 3������:H��������pl�?}=�����UJ�$F$������W)�p �~��/�H_f��2�J��y��;��J$��P������4�����7J��2�E��(��1B�3�0������T���-��;w��gs,�`un��)n�Lt�g�?�W��B��vK���b||-��
���zw|td�7vP?��_���?��$6'JH�`�o��4���C��O2������rO'H9��MVX�#0+���cPl{5��j�	�D]z��AgY����9�`r�^��_%���vH�%�w_�������N��0����x'��K\TIm�}ld��}��&��ovM�~8�v�3,<hY^z���"V�KQ`����=�`w��Ss��w�t��&����*t��Nxb�AK";%x-}��)�Wi.�v�������N#�%��'�f�5�Q��9z�^>���^AH���+��b�T���R�l�}�K~�+�`�;�	�����8��0�����-�������6	*R�F)]1��(�_����D�9]��g|�yK2w�AM)qo$������>6`����q�5����u$D7��$C4N��$��qJ���l��sxgP��nR�������O�l�|����g����R��&��h������4��*���l�1�@2��TS%=t���G�����{���I{?C����2(�M�Zsc�]6���������H:3�7U���ah��TuSe��`�D/��U8���z�k�r�!`'���dX����-�S���N��Gw2P����G�+��4��&���t%S+�������Xu� �Y�L)�AkN�w:��c{�Rr�$���Dy�vx��71i���^(f1��Wxp#&�fy����������~<owxU��yr"!~�w����`AP)z��lE������t�7|9>�x�:@F��h���������N]|cd�t�\�;?����'�&3J�I��0HT�?�2��������s���5m�c�pl�is�M�0!��������+!�(Z�aBu�0���p
0�����6~<��H�&N�a��	^~9H���}�L����(���FE�.�~qFio��A����W����S�Po��6�$����2>->Q�(QC��C��$��������i�7h��Ij~=�	]�H�4+��38�����/���n�(gD�����'W"=U�[L*}�C�\���/CUzWaJ��
Q4x@�)���^n�KC��WM��GQ�`b/fP�m*eA�@�e�O�+cr9��p�9�����$�K{�Q�\I�}��_E0��Z������XX�_d<��t��#e��z�k�/M�������O	��5�M�FB,��~��W�8;L��M�f�8����+�2����[a������h�a�TH
��W�����v�&�;9�n]6�pm�r�a�Po�5	�D\6���i��5�vP�v���?�W�
KK��H���	�Ri[�)Gn�!�*��n�
�"�g��A�h���@�
�h��r\b3�h��������pv0P2l�A�l��~(\�Zv����?]H%����t���o�������E�,��������
�vwO]����
�+�L6::���"SH|�=[:f�6������a���JR���-�@Zt��%�@���� %���?�?��o����q4.���G��$���%��(;p�\(.�Tn���^��8��&�\�2�V���k����yy�VM�Al-��G
�#.����Mb���O�o�k�W����%C\�vQ�P\��;�H�a���s��A�6����t~]��{r�<-IS!`#�?*���pm�)����EYs�c�B�s��.���k�O��2x_�H�_�}������y�6?}�h�l�:�h��]�ny����7D)�?�6ze.���n���,���7�g(M
������"�n��K/�nkd�
�`*��hZ���i��ECgT"w����)ZmgB2%xz8�T �$��A��B���H��>lvMu����>��G�����E��	^l�
`����f�O��X=tO�����]���+�����:�CS����Mbo���~�]����]oApv���C������j�0cH�i��lF����`����S��Y�3����=�����Zyfu�������s���/���e�#M;kYz�����v;���ph{�Y6,��yw|X����S�#��d4d�,�j}n+9E�$�r�q,�v����������l
�ib5)���6D��]j�@Y���:��N� {>�����G@�
2��Ih@g2�`�%�����k��������7��3|����%�TL����%���dr�x���=B}���t6Y���s�|�:�frc�:F%����(�>��,��TR��E U�5���Tu]����^1�����5S�����S���m|��wq��4���iBHbZ�O�nG�� �\c
"���y���F�N���q�����)�RM�L����L9�>�w��_q����>��!.�u���v2����_��$���p�gdJ����$0�y1,��[�	)���g["�����g�.��|�]�^�m���!v2��1�QX1������w����s��4�4v$G�"�
���g[W��R������V��O+����<X�x )1�}e��H��P�
�?�_���q����%&�W���7���N�\����d~����yK�6�m��m��CN[��6������8��}@]������8���=tK�:$iq��hM�)�A����w��'P�P�W����RP��=5��z�dUHI�Q����Y7���X�����K��n~�9 �t����o;����.��tJ)}�tM'`s�
�3s�f�"A�6�������N����5��dz�?`3�����aR1������/�?L*�O����K!Z3�$C��Th<;V���,�J��@s�.�o��4,2T�#�jy)B��\v�X
�f6��������O�
r~�rvHWX��.	�T�S=��1�N���.����\;��@N���\�����B��T�v�?7��[3uu�]k6�pWt�2#7Ij6�[d�>w1!�T���[�B0�*���o\�B���K��l���`��������>l�vSyy�]_;��*�aKrEC�8��^����&SQd_N�^��	�w�.b���vP����%(;�sy>������#��������hJ}M%��p�!=�����3���Z���x�qp�=�0��1P���dm'XEMxL�� �c�l���nm_��o���Y5mu�Xw
�(-O�\^J:�NW
���i��Tt�|���:;Jn���`?�4�Q�4:D���	��������c����~����!>��I��$>9�o����(������+.��B7��G�����Zv��`�^
��3I����M��4!��\zF6{���mj'�Q��P(#|=8�}q	!�O%<�q���M��}���g��<��4��)V��b�ILA�BIP�p�C�Jqf$}}��
��+K}�}gS)�7M���\��

��6=�D���kJ���qq1����-u9	��"��*��c�m�n�!��QJc����-���n���S
��2��������,C&�����#�U����~���%��a�O6��7)���!5J�'��*H���L��
���^��������s��7
�A��*�l��3%C�X��r�J��,��p��5�����#<�Xk3F��%��EU�����6)�V��.mOg|����v�
�#���g�f@���@22g�	X@&?H����{����^"�~�[�@n��v�D�M��pl���}�b ng����Qw��vAM���{��2
S��:M;��bG��P���I ��h�1�Nu�5`�	�}���CKs���3��&+
���
���.������=`�}i�����%����8D���*(j#i��Hf`��7Uw��`��v���E{�tI��������[Y(�����EwP��;v�)2�~3���I�(	!i��N��d��$UwO�ER�^"�]�t�kL��`
\�n�S��5VU�w�|���z�a�a3�����S�p���=�c.�d�d.4a~���Wu����5C���	��'2� sU�yIG��\.��M�s�Qh:M�	���@4J�c�0����J������2��Z�����Y�bE��H��=
�3���R�����������9�T���k����k5���5�iO�4�m-�7��p9���`�IR���gp�2�k����Qwm=nY���1�P���:����$���f��`KT[�R(����[�b�E����
0c q'eV�s��]=�q���?�	��x&�
nv��C�hn2���)b6w�,�����{(%�;�F��J����/������n��(��-�i(�N)�����
�����l�`�x�:�&���i�w�"q����xtef��e�x|�������k���{��^�����aZ$�@S7��\>�_����H�lK��&X�'G a����7�����S���������J��f���,�����\��f�/��`������Z&0������Z������w�������Q�8A�"�H�	Xy�nJ���������?�I��8P�2��F���T���j���+Ws4�s�]
r"�������O�Rr�l���b]��E��4&����38Hh�?u;X���A����O���8��m#w�?������8k8��9��B�M�D��H|����Y�e83S���35�]����2�S(m��=%��r�C�������E���h��P�s���H,M��c���r�����1�����|��S&P	(�L������)����$�������NW7U@��y{I�a�'3��da��
-�V����C��1�r�B���mo�A��J���S����PI-�N���i~����$����*u'��
�e�]������F��8��\;.����R!3y9�+�
�q-�&�9���$���W��'U.F/���'�Vr5�$V�wR�B�Z�����]�nZL�`f�����TK�Ff�s��u���Xw�?�����?������Jg�G�x��`v?Fx\(����~��O���"b���N��=�"�5����J{.z�F�F>4|g&�@�?��GHJ
���~#r��9�{#���u�o`lL0}q�A��(��"�(�6���������^����B�N�A)��H����1o`0����������������!2z���d~��G��2��A�G�&�����G����D���mr���o��/�h�x�rq��z���Xu���8�f����P��k�l-�f��2!�zx2�#L��W�
+�����k. e��f2$Q�d�F����G���p���BT@K��R�ZG�ZZ�����y���x$)�K
������� A��M�G������k����>X��x��6�����Es]<�S��Lb���mx��I��|H,M�<��R�
��~���X�������!�����a�l��$	.)��a�[����p�!���o�MlB0�L�C��6�t�����_�]�����.���s����~���0	�\�>)����G*2�>�\@�������=U�Yz�nq9�#:��L��Q�B�Y�J�FY����xh�����E���14����D��w�*�O1���� �V���~����bv����<��2�
���P8y����>�/~���H�$��Pl�KI&����b*�����Q�t����wp�	�$W��Rw�r"]Q�y,SnW�d�H��OW�/�@G�FgJ`O!)��u�X���}V�0�gf�,������b��rJ86�C������z����������IJ
�FY��VX����$VD��^�����M�����nn�s]���4>y3��	`R��a����4HpS�DL1�����A1E�|2��K?F�������\��!�4�1�]��92��z�^�#�If&��K�H�������4���~���4��7��_����B�i��q�-P6��/G�q��r�����
�6�gP
J���;�����W��p���H�����=���W�RMJ0�V�1��'iy>�f��v/eI�2!��h�d��>��q?���&��������b�E��,���)���-^ob�4����?sy-Ap����7w�u<I
��>��)r�9�sq����]�=A3>�3�=���0�1�9�0�����������-g"~�FH_z�c��IT�p��n�4&�S���Q�?DP"s���FJ(��!�
���X�_�m�X7[w�*�N$��<����^D�X�F�zH"r��c���&5D	1��<hRO�������E'V�}h���l�k��(��������:�[u.�.�;�j����
�bJz�~����8��O���n����*�-x�Y�������s<7M�Ru``}8M��B3�Zk����f����)!��ig��5��8L���7�\AF�d��u�5Z�
�$�s$q���+���#����u�*��Zw��"V��;k���
��sq�_YZ�0I
����(���rDu���`)��,3i���r8W�����$��G�v �����/ab�x�2��}����+��d��(�]���`_�SV�\%Y�������"��H�+��&�S��������{>�����
rn�����Fv�lb��Nb��b���l�����q���$�mF������?QO-����;�Uw��1,�m
F�Z���7XZ$�H0������,���4<�k���g�\a�V[pT�a������[���!�o��D��)$��?������)p����\��<�&��K��we�~t��fh4�S����A�{C�������t������#8��,���,�O�����t�SUq�-�Ipb_zC����b+����``_��j��/��Ni��L��Pqv+z�N]����[7��\�5u%+n��!H0�/��������B�"y�4+�������cy��]K����<���������cj����r���c��F�9g���W����tvM@[*?���C���:����s��|L����%j ��8�����f�ykx��{9P��I������Q�Lf�I
o�i�d<C��im<�8�z5����*��l���[B[���l.��m����^jc���l����
��u��uo�e�|/��|,G��?�r&��g��6[f�p����)r�-���8��8j����^�N�#�p��D��P�]����py�������
&U�A��K��Ld��2���.E�6D�K@e�:�8�\)��FnSV>�y�����?�~��{��N�z���H�8t��(�����������7�fn;e-�M�u���
D��2p��5h���������J�������d���aMO�o�����N���6��G7[��j�*O����-�s��s����F*C�
,��9��
y!(����2�4x�)d���Je�o��!�lt�1\���.
�zq�R0��l�L�j0��������������3��f�lp�D�nK����"C����+x�����<�����}�$/��u�R��Xi� �N���
�0����(���zp/&{8/�
��4�I��+E���;L3n��?	����[N`Aa��%L��a^����:tJ�{�u.��q�� jSq��P�[:��HsC�bJs��Gn_�<������]_���\��B�jO8��S_{�v�����	pH.��(����5�����n�<�*8���I�"�E+����S����������������L�f�l*8T"TU��l��Xw:�����D;NI��j����(�qM��((�A�p�lf�p�a$>��J7��rd(%���P=����\7����
�@M0>=>����`���\�/�������5�����4��)}�qa(�J��b�'��J����k�;o���>������ejn%d���
�*�w����;'��]�V��2Ak���:��) ��@���s�`�k�j_�����C�
c&�C��w���r�� w�����t�;�X.�f��	�l��sQkgm������?7��y�����������z�Z�+w������a�{@wcFr���?	����r}M�8�+d"��
�����N�%MC���r�
E�����Q�+�Q ��x��@���5;�A�M�PPQ������T9\��Q'�Y�g�w��a
x�k@D()�<4]u�B�x�b�wM�����
�R4�zT���G~l�'4���bw�.��Ai��N)P�U.
� �8��{)����
�JR�'w�h�`�X���i$.lY�IS�A:Z��=�D6����+���x>���M!q� ����:����`	�u�����o3L9��o�X���o�*����Tvu�.������SL�'�h�����c�&�x������H_c'����������v�wa��Je|b}��3D��8���F3���q��4H\���f�����������g�NI"�S�A���t��-D�����bf��E�&���U|��a�xI
WWa�bA��
Q��Z"��%�
�\}�N�������p�:t}�<�]�������q'FF�C%pC�%����������{}�����l��|VC������6����'�v��BKSh���=�ct[;�c_�uSW��p������Fm���H(���h.?��?��*�����?�:&V�9����x����������\b�=q�\���uM�J�'���IwT�7���SxRX.��Q��njx4�Jp��0����T� B���NR����	k����u����_�����V�����'6fL�#�����Og�����+��A�
��+�!������P���/�S��ps��K���2�?b%�0}�Q�@�E��{s*T��"���7��eC� ���b��L�p|c��U��l\�B��E����cz�g��}�W���R������#]��IQX��j	���OGZ7��r�kw0���� o<�{}������u
�
6qp���{�.{`ZXE��������.;}���2���/�I9�_##���c�iH�>�#o�l,F��g���Z�l�o'��4U�������2�R7B�����i�h\?KU.P�����G��������p�E�X_\�pn]B�m[A���R<����������� Q�����%�0�#i��+b�"��0�N�����`��=t�_�������?�;�q�SS0���)8�!gw
L9�K}���aa�>wI��4V�do,D��k�a�$'^����~�'����;���	L�K(�E.�����L�_��\���BC�M���\zDDAK�ZY.��q��]������M
���La.��=�e�����5��T���&w��G�z��Aq/���^O��r�&��Z8�@���B(q�Z>d����4s/���b���9���f�+�
����FJz)9��7�b�J����3���}���B�;N��P��4��/���r"�w� ��E0+���f8Z�������G7�+Q�R�������G�r����?2�q�t�T���4������%5S� ����9���T����KN�o���V�Tp�^\g�T������I����7��ZK��8�x'��5��������_B@|����_�u_>��������z�h�p,���Dj����7��qw\	���k6����G
M�.�|0g��|�j������p��_q���\:����?����q/}!��ZrR#x����IR[�_��~����{l�������;���^@:_��x��`E�t�52�����0�C�~�����p_�
N���-}���M"�Q0��g>:L�xg�T�v����xM�$��q����[!7I�x�`�<�t."�d�����{+��o;���pn��n��_���~����YzD�OCxO�k��~���)�)��v5h��d�7���s=�IX�3��5��	�c�[�3��������?�S�����I��M��.�+J�K����rgP�G�S&��PB&o
��� �=���IS�\��[��qhS2�����(!?�7�3�����]��� deqi88��j����B1�\)�����wc�>��xA�_d#��?tY��C��+�a���-��~}|-O�S���
��"�Tlr�H�]	��X+��Sl
fX��\�����`��y�u
y�����^�w�_�K o_������n|���R��x:�����Z�����M&�3d��-���{�Lr[�*:��������c��w_����V�A��]|"q���m� NPb3����_-�y{*_�{���>1B�IH�>��8��K��C
�����>��!KE��<���S����]z�=�#���xhp���[u#��������HN�!���5�p4���]����F����t��H�>UBX���A��� <����#�|��`]v
q���7��!�MLi�o�uC;%������F.^�����$����R�dB��L�u\$�i7�[$RD]kN�ao���~��/����W�i|0���Y4 ���Q�Y�!2�FU
M�w�,�3�A.�E���,*@8�{���"�>o��j2nRH�g-VJ�"J}8��U�y��d��o�]��[W�,��CA9�%���S����m���+�q����>�B�}y��I�����Zb�k��2�~�UlQ]f��
A��r-�|�[\O���[�b����W��q����W����P����3�1XE�k�����7�\v����&��/=�(���Cb�K{Cx2�v������+�������������~G�?n�zw���K6����s�R�'���*�A��n�>}���t���w�t���H��� fCXq���������l��x�WW$�����t�#�~�69������m6c��[��*P]�9�������=�_��&�/���K���}�l�`n�G1�7#@|���j:0�����`n�O��/�~8����R�	g�x����tSq�\h2���*��<�X�l�w�C��7�N��v���^C]iz�������E�%�aX�c��2@"�J�\��[�}|l�������������^r1����q���(�=��Y�v�����<\� ����k��7���/�$�^�8 �u������������m��=l��#(�3�l�j��r
�w��j�y����oN������9Y%H�e�������%�m�����[5�
]q�X���w��2��]B������@2��@��ts�xX�{�:�S�����)�(�Y;�.���1������;��,'���uJd<k�i7�QHG$�A<�
p�Y*0��h��L���������/�Gs��O��h��]�s$@D��!�ITq�x���a
����$�&�����m7�A���h�� ��~D��I[n�p�"�;�'���V��*tj>�v����p/d_�����"�����h�-������Z�\.��Gi�[`�������i�@d:]�3����A���'�U�Y�7_�^������?��V��.U�0-��}� 3M���X$�d�	A:���k��?�fITrnjNcF�����n��1i��G�����}<��z����l�n|������cbB��x��m��[|�8��"�������'�J�����x���O�S��c8������>���A�����������n
!Z8n��d�I)�S
�z���J�A�t��R���bV�]�L����gOl���;�=���
�����!=��f#L��[�WF#�K����0�������RU��5)y&<�����I��H�;P�X)��m���U�{�C�v���W,�� R2ud��U���q�,e���au���J�:�6�;�{M���e)7g�|�OM��}�y���z�A
YZHi����|eL)x�����,�
�|/=8���#�5��������?��O�EH3�4�2�_�nR��9��8���5���������7$���su[f�v�w9�f�i)��W�/]�n���R�3���|G3���e[���7[�j���W���P��c��"C!�^��$�AJ{�����,��h��av'�c����i���c���-���\!/$l�De�
.M���R��4�x���`j�*bf5��� 4�� �����+�C����C���?l?���OTdZM����T����[�+���8��4B�x��������������������F��\���0�%�
�|�$�>z�L��4C�3D��PhH���b��RF@�����o�u?�o��i7�|�o����T�?����l���A.��0Y���������%.����I��Bq�r�"�i��T��.�z>�R�=c��Q�r���k��
���RpKX�����ui�B��������Cf*k���0�b�����A�����3=��<���v��-R�!�	�v&o��>������R$��x��fD��(������f;���F�uw���vk�t4%�G�'��mI��}�������@K��r��1R��\��=�9���p��N>��7�������f:0��`����ZP�XkE�J�R���o�;`����j,����R"3k�PC)��@#�j�������M'����������2�i�J����w3~���ih���|����bW&cq�,���z��;E�}!��B���/��F��aS�@��#N]�C��'�@�.�k�fw|�������4��7�����y������\�����6-V���;�i8S��DZLY0���B/�]�+�cn����Oh��~kaS���ti��#��P>�	��
�a�8_�b�����N�Kt�����?����^�����@u��t���)����	�����>�����	@K�f�AJ�����m`2���*!�J�R��u�}��}���h
���lK�H�����}�����{��wsn����{��3<u����~8�.�L{I"Ht��ZKE�i���xA[�k+��Qc�^��h�(��Tp������#�)�2+k����+�����`�/D��>/���@:�[���b8[����2&xoF�����&:{�C�=we;n|F��v5�6(��z{��1&C��<�
��r�"�����o�(&m��w�����0)�l��G����a�%nQ;	$a�z
�h��}��
��f��wG?����@JH��k��Q/��P�A!z%D)����~k�G�����Tw��Q�4b�%����
_�/q��R��X��L|���-����i{�A�_o�?������U�FT;Me����������4�/�*�Q��]���h=���Wr�K���jT@��[�[R��8;��������F\�R�������+���R���P�����=x�9�i(^���s���bRU�'�s�V���������v�z���kob����`��`[FCo�o\�=X
8���x|����#q�~vG���Q >5
�R��.��x}�yr�*����S=����A���j� (	*����.��n�2������py���J_���:���K�(���9=y0�H��X���V"�t�(@<�(^�V�U*�r}����������[g��$�����~���5�����y��Pbmx�g"�E[)
F�X���:��S��O���)�)<�_��:���8)e!����i$X��	J1�������e�C�� �>��oX���n���{[wm������ }v�EW���P.cL0��P��*��`r8{���n�t��?���3mz��[MtD	���R�����*���_AV���sY���#W�����3^L��E���2	�n���@@��BdPUp��M�~�����n��7P
�Y�(�UV�	���W�(+\�������|cC�+��or�C�T��=x��+���:x��@�~�,��3b�B7)@�s��0�:w~�=�3�<)���(�J�t���$x���Tk�)�[RZ
��)�{�r�]��e�PW&.���L�2y{�m����[Kf��(�dSUY�T�32���kX����X_��
�`���F�'�K;9������v�{���0�?���z�a������������>���b=>��#|����'Z)"�����5Iv�A�H��t��*:��{�rD�e���Z��=H�/s2��/�6�)��	f���s5��t�Og_�����S\e��nO!ht.�� 7p�8.i������9���4����	���lx\iI�sd��oNk�~o�m��������$����� D��8��D��0���MR�2A%Cn������J\���*�A
��{p��u������V9��,����Q-=[$nS�.�Ni�{
hz�FjL�/I!&�8���h�`���JQJ���aF������X����c�Whp���I���^�vg��83b������j�w�i1VHu��F�����YZNM�!�D���LXl�1�v��������z}�m�^(�m�4k�J�>��`u����Jw��R��<`�V������R��=	A��H�3	D�b,��`
W��
�xl)�r�.&�b��lt
�$T��,eR����h�1����6�^�K+�{��In�����/�^�Ed��Bc���%a$\�me����TX���0w��3�|�m ��L����e	!��rot%�E&�|�
��~��/O pd�?������E�7TA�3���z��~����=R$�3L��z@���%q�[r��u���/��}��c����Y�.�h0���Pc"���5�2
@�~�1������k� O��
C=�&s;.V��/�g`���2
�Ba[�?��UG�/�����4Y��D-
����x�l&�8��c���t�u)�&�?�,�olW��(��:j��H��iA���!!��<.awx?{�4��6?{���8%��TS��6�h8���L#B���[���"zeO��N������LG���+=���
_�o�Sx�0�5}��D����r�HWH���R��X��5�f&
*-�qBMO�*K��hKA����l��#4H���]N�t(%�3��`�U	�a
\@M
���w���/�X{�4���kV�R�c�.Cn�|����C�i�`�n��EE��iWc��U��$�������y�,E�z�i�A� q����XrZc�!w��Q��N����M������e:�M��F���F&@�Y�r��S�����������F�����+y;/\�i�lR��y���R�-��k�9R_	�$�%+���0������l�M�x7�~��l%�>�<�[�]���(��	8[��#S�����~s�1;�����U%hz�N��#8�q�m��"��0��;���:���\,�{��t2� ���5��S8���FB]
!������@���~}�\�}���m,I�p=p��m�y���8w$�V�����2�A���s������A(��$��l��I�&>T��=7�����On!�;4�T���vh��i�N��`�?~�y:�RK>h��t����
�g����w��Ae(�38�:h�.�gB�o�!�1���A�[���C<c�~��-�d�B�`6���.N�U�Qj$�DKR?;�`���{���o���g�r���)m�-52X�
�R���i{Z��n��W��3/�	u��9Z,;��/q��f*���~����7'��o�N4��[�C����{h9����r^@��w������|����G�N�>$wZfS�"���x)������x�k�"g}Q�&��I���)�������,�����j�w��5�0sL���Y�g�����z	������h����u�o���m�9�����I�QP��a��
&��b(����X����]zi�!+kf�T6���}9��:N���Y����u���$jh�����P������9`��l���<���?����~<n���v��zu@��HWJ��n=���Ja]�9�?���OXD���&�3Z��vU��H���c���M!<�ES�K����h�5F����"��|�m�����kGl1���Lg/��n?2�C�3���������:�L�x������xZQ%�:<��!�
�����ow0N �B>kA�*	+��Yia�k74]8iW��{QQHG�7����@����}����%n��:OU����|�#z��,��yW��r���i��O�D��"��A�G�N�H�����$��8��2��`�D[P��	�������,��������_�]�k`*���rVD���hY�7/����k�~�"���W�%M��	c�}�9�����������"�G�!�����J�<�4i���I-��O* �V���������q��,*��`��j�<:j\���qZ�
��X Ki��yT���,���"w'������ -6���	����p�!Sn4�Z��R0�6?��~�x#^~5^������`���x���Y�C,�������w��4�N��m!��>t��-F2|z��f����[��{%�diN'�*�>�ac�U��b���
���������	��
P;U��EP�wBLV� )�������(��V�*x(nkd��6�oEUX"5f�e)��vp�������m &e��2�>8�+���}0����������	>74=p�LF�C�*�Er,m@p���� #A���kHn�Y+��}^e&R���nK����2ss�����!�;���G���ug���M�ytB��y:	��8�pq�C|�"��92��O�7d��]�%�v�d�6�,g'w�!c�{Y������',?��v u��Wj��QW�F�\�!b�v�$
�lEy9|���8��"#�J�/�	��
IN��o7@oT�o�������m���Ao�f�o��T3��l������	bCQSF���R^wH����np����W!�d-�H��3���� ��
@�f�b?������1����� t�+_T�]2i��^
����
=>�����-�����		Y��c[�b��Kep�X:���(~��X4��$���xZ��Z��e��!$�n�E��� �7T�iK��_*�!c�������u��������O�%EWJ��k�������?�J�M?��m�[�U���F��{������$������g���i_T�.������B	|��Qa�����!��Z���\#K��w�����-������O��}���	V�6:�����SwnMn���+~�K���1q.�U9'��5'o,��nk�������`�[T��$�#<LM���46�^�[��V�W�4��C6���I��kf����d������}��&mn$U
�O��|����F��� +cJyh^���@0��j@3�H4�N���o|���\�KxRJ$1�����(W:���WE^��i���6��X</�g��
��u)/������C;���p�i`v�6���a��9�yJ�.[�����C�����O�����-xs��>A�g����b���Ul����K�<%*��v.��?���3�j��E&�.�RpW�b��c��m������������*�8F��,��.
#��n,
�jo*&�����F����$���]������p���wweX�.8���T4������A>�� �A}����u���a9�N4��Zm������r3�B�I�.]�����s��"��UJ"�mP��_���\u����?<���O�)�4�����L��`�;:��l��7SC.�=�&r��_������C9][�����M{_wm�/t�a���O����f�#P����gBJ�\�\�&G�b����P�.\�Z�b��zl��gF�=�����aw)��f[�������	�R��
wd������K�F�T���e�IxF�e�a3M-�4����1��os��`1-��@f+��+�=-qc6���:,A.s�j�z+���$i����I�L�tG��HkW�����w��a	U�/����oo!%g��2/vJ�$
�����A��Sa�aE?���}����f�@
���M$�� ���e#�d`8��l1B�Q�����������1����s5�h�K�
�OD>-�()AH(���@�mu�
�'������#�s������v�&���6*��������_���YS���QE�~p������a������6�$J���T�B�M.6j�����7p��/�-&L�:�j.��,]��L�6����\���/�\����biV�&����pI�@p���n��R�>�%PL�/;������s��\<7GJ
{��r������]l��];����O=cs*5�.����@;CmNe�~����I�%��B��[�N����.�>�{@�MA��c;E#u�(�pg��8S�P��-��u�#`��9L����J �@������+�JY�k�!O��Uh.��~����|�)�!�����U�m�3�H�	rf���D��H��lE��*<g�����#0����9Y���Lph\r�y�w��B"8����U~s��O>�KQ������1���
Q�"Y$���>l)_��K��.rJ�����C�cpQ9&SN����p���+p|>_��(g���~�+@f<�������H�
�?3a��rH�v�9�!i���|����jw�������������5&��fb���nvF&
H
��C���&��D�	rdW^�IY���bZ�+�\�J�J�5�P��\,���+�������6������i=�0����i$�A��QK*�+�u)>�����'v�[)����<X�c��.h�6/�d��g&�!$�a0[^I[]q�q�L�8�=3P��r�����4S��e�.��pM#�^AA�x�B���]�`e��-�g��q�L�t���.�7�m��8���e^@����~����L89���E�����}�Q+�J�c�p��I���~w�<�t�TG��\"���{ims�c�T�#)��
I�/tN&�G�_���H�1'fB'��]�>�C���������A�r��y�r<U��H}����MUq�k�� ���,*�/A�����Y���yvlMt�:q7O0`����A9S��z}�o�:������+w��Q�Ji�#Z����ES*�;tC�{�k���W�tnh�1��d�������Bg4��"7���5��}~-\�i���whn�\��K8��X)�*�e	�����u��2���<\;������5�q?IMv��o?�M����i����C}

�Y�Q)��*c��������':����n_�������B��=�`M�t��,����6�Qd��r%t)���|SZ������9)aQ�	q�h5��.zFZ�L&u4�i���|���b�{�/�o��J���e[�H��"8\:|�gAl��Xl��R������y�y+�\BTT�������?�f=����y�i����t���#������VV �>��8��
�t@�u�����u���`0��HtP�9;�Q�Jc����b��/�XX�sXqbc��@�����Y��j)�5�<��QA�:�.��������b��3�7�Mi&�P~���	����}�����9��cP�$�7%\�?p��s�|���dB�r��]wz���wW����|J-����Ge["H/�by��6�D'
�z���  ^�����:I����r�+�G��9�H��d��,��_XOL�����?�����SS?/���8�u�uB�"�T������_]�"�oT�����8��A���fAN\�����o�����/��PS*����`������F�X���R�}s8>}��m�S����3\�e��JPk�.V.Lx��`�z�����P}�>l����8����1X��/�rj��Ih$�07!;���r!��k�S��?o��t�
:�4�K�p��/^(�HR��!�
��uy�_��Z����FL����@��^,�5B�)�gw�m�Ez%���r%�G`��u�.z�KY���z��i�y�{���E���^'��
�[	SJ��k;�z����yl��t<lIE�}��?�Q����Eo�N�7�C��D�������	a�WIQ���^,J)����R������x���7�}��M+�(�����@��X �;�����
kO��7��T���T��n�^�8��VP��������~i�$�BC^u�@[���
g P����]wX��\�>�=�g�7���L�]I�	���FP{hV��9�]�t:5S!���u�@yH��1H�SS�K�c���Oc�"iXS�.�M����N.+����9��h��f����Z���	M� +��$�
ho<6�Z��t�>���'���G^E	���-a����O�C-�&r����
���?��u���}����g���#��MOi2A�~�4��C7`P�. VL��*�V�4���)�)?wDV� ��P|���7�{����_����\de��8��s�x-g��z�#�'Xl@����&��R4
�����T�K 
?mg�R�����RJ����W@Q���n�n�Sj���C�+����P���i+	��W���d�A�<Ec����(s'���o���n?E��H�d�W�H��s��bA�7�$�(�6'/E�$����|!���(6�{	T9��q����}L�K��w\��E����V"�]�z~-��E�l6��+?�����p�=3��k#+r	����1�`|Y8q�X�SS)��HK�9��,����]Rm�N��P���w��lM�]���8��E�T���.]�2�w��f��2)��M)Q���"n+��8�,4����+j�O������:���C����=� �'Op�-��~���px>��w^��aY��1D�2DQy�5�a'�����:�>�
�����.5)!��(?X��:�}N	�A�Q�Q��NG�+�
��Wf�����I��Ew9�T�����
@)p{��@�(`~y����RU�,��\��(�KAUHu(�+?7����S�%��	���8�{,:�A���{P�WZ������C}����I���3��(�?DZ�0��m�@���/���A��]>�m���n�u�������}��E"��$p"CwT�����m�����<?5C���d��d!���/�k�Bb�"m�v�C��-��?R]��]��u�����Q}"�E����4��N����*����R&mn7o����7o��f:R�����B
R��N��]Q�JyB��u}i����2cD�������]��0`���,�)�����,cG*5���3��fB�<J~v+��c}����}����y��'�s�.*#�|Z2?�8B�+fK���o��U������7����19�zD��q�<����"�	F�t1	��]�{���_��������xp]�e������U&��X����o���f�vs!|�w�/�@M�4��D�>��f9$�!o"����[v��������2��M������A� ]��;��������?���J�96!�\� u�b�&�sET���NB�u�96�~j������Cp��,R-}�T�����v��	[@�T�����o�Ub�8���y)vO����\��S����0��O���Q{��t �5*
&_*�$w����8Xc���D������n�����to�*9;��#���Z�������;I��y���b�30r�)�>
w�{����Q����9|$��}��������l�.����Jq�
�z��DE�L�k	�br������]Z���C����6a�dKx0f���S�J[d�d����_���^i����"m���FZI�����:��������/�����-*Ip��3����F���X��{p������H��9�n��Z*A#��Yf�J��%�RB��P����d�������Z��G���������g��k^y�kL��e��E]�|I��	�F2�pA�fj(�6>��Oyr.$�����BR��$���4�}z�����,�"z=�0��;%R�r���/_�JZtZ�3S��Mzu�2�m*k�����	�N��v1����>�����R�#��������U�cK�e�bF������
w��{L�W�2L^.��V���B��+�+~v���_���|��*�WfN��q�_h{pv9�Xr��5N��k���7��Y;����=��>��*�/C�JM��J��<���>��OJ ��=JJ���0���o�Q	0U���$q/VZ�}��o�������h�dv�D�LM9���v����0dj:,P����a�|����j�n�)M>#���'�>��3
�j��ge%���@��C���w/�f��di1�U!&p>�X����$Z���0��3��	}b-�LC�/� |K�H���&�I����������{z?t��;G��>���p�Zd��Gt��/�Z�f�����m��fZ��b�����8vsIu���&��Sts��
_����*���#�A=���X���_��
���c�GXT�{��J���B�A�)TFR��S��IZI�L��R�t1���v�v�u��|H�@�@�k�_�+��d��X<%�r&�pC	1���VD>��s�;A��f�dhHd���]��2AU1������u�~��5�����������rx:cuG�R
+v'vE���K\�4������v�UP���{�1=����&i���?6�g�dX|��Fh�j�{���4VJI9�����o�D�_�N,�lpc��B�!����d�^9k`��������z�I��"�������;�])QJ;�p��>[�|H�\�����qQRh\���i�Z������j��gl�o�m�m{���z�T�.��
�}k���#
N�]O�����Rt5	kyy����|6A
�5�+^���?��-�M:���/!�����D���Ef�����U?���a�~���u��c��[�9^��fg���8lw�H�J)�'��$����4�P�t��UD)3�XiRJ���l����8��q��X�S@��!�ut?HL�,�����������#�R�����;��dA��OF�+5��\����I�,��M��L4M��T��<�eQ���Y
�_1dZ��Oj�e�
��������r~M�����o�4���-�$z-�����s9�F�}v��i[	\�M���(�m�9��#
��#�s�9��F�}`���DXMf90o�}hw��cgY�lT���AQ�k�+����
��(�5��_��;��\;I�H������gQ�a*4CS��]����gD*5�nE1GT�����)������C�
;�bn>�������]� �3"������A!Y��'<�=_b}|���oy}��kJ,Mv4��Q�����*P2�*S�����o�z��O:������##��/?���� o!aV���E������k�m��|����T��)��E\jly�r�����K��dKY_��>�F�M��N���8h�6�96���N�����M���vO�M������r���������:w��TeI]zVx��/����M}�5���a�i���>l�C��L#�g�9o�|��?z�8�������VZ����'4����-p�R�<B�=��|/����C�{:�{'��^�d�De�T��-�@���:.�>~]�C��?v;���&�*�0�$��I8:�*l�����{=���s����9������a����%<�8���I�M6���}=��~��'��M%�K	�������]��F���J.}A����L&�pf3v������m���LRv<�~�)����������	F��+�:_�y�������Z����#�Z�}���s��fD�(1?���I���Dm������|z����u�duKT�8^g^eLh����Ay�h���H�K\�@����~I�(cT�M������z\S�k���xhzl�H�&pZ����E�q��
�8-zasi��5�A�{&���^vn$�����i�q?����!t�3���dK�����|��V+C�zx*�����/}
q�m�� ���W��G�\f^k����
������q��Y���'w��9�=�b����n��e��f��>�o~���x�Ea����E��(7.����7��o���h��D��T����d���e��r���It���#D.�|n�"
7���,$Y��`3�B�SS�����
�4&S�EY��!;PF���EP����%"�]��e���������m��g"��A��dD���D���g6����l�C���tn���
~��w�Iga�I�	A5vI���x���r���c#�UXt�����s��e7�����k;i�i-d���,��w'��8��7�|s������N�Q?+�N612!������.���a���CkP.��VFns�1�[�Ff���c|�w��]���5c�]�L������R�r
��vVr���w;�&b����^Y��H��~Aq$���r6X*�"����u._y�)�q�w<���dp���g �{��k��Mi��@u���dqe"_;[V���T�eL]nD6����W(����(��ogmN�!6j�'Cv�XIO�$��
��s�]����.��H:�0�Z����r�2Q��H'�e���$����S�����W�6"�ER��R!���,L�r�QvQ�D3����YE2d��Wv
�:����>�c�=\:(y]'����RQ��D���k���.tI|.�P#����I�����p�|�m-�p>Q&��s�%�ai!��4���J�.W���)���%��J����?�o�����
�r�Oh�n.�+�6]�����J66�������������O�z������]#��<N�m�{Z�^^Q�\�S�b�����q�UW�!T)8��;fD�}��3�
���C�\�M+�Gz���}��n��G|������t��b�s�T�aR�����5��8���}�������u>���w8�}�������4���
U�����Z�r��L�N��@��<-2%��M���5u=��//�IN�E�p�rZ]&F����b�d34�5���r�������c}����8�NG�v�t�=S�y����K>�(�����?R8������]����J��9������K�	N0p��aK����s�����;<PQ8�1xc'�B+�,�)m��;;q��X��7����������oo����o��XkYh��]R@���B��������_H���5�}����,�+-��
>��7w����.��O�!��dG�*�`]�(��!/���+�q&��w7b�����rQ'{�/(�,
�L.���u�������zOC:OG$�����K���a_�K��uAL�b�����}@E$wbb���uo�%���~�q����E���a�>���J������n9:A��}���y������@5+�v�g��m�����4Z�z�#��s�MP��8K�c�[��L�+�gF��[�?	|�2�����&�Z�Q�����
�.qhqSri�� �^�����������������B+p|�(�����b�����q�H�sf�v�)��-�N����u��fz\�`}n�+*
S"�L��	��j5|
�J�:DS�C�8X�U��������pf�������j}q9X2�:���9�g����s9qry7�1[��K���8�����e,���wmG��r��%un�m!���Y����[J��W>�h2�����_�~i.����J���t�
��s��t���3�&���������y�T1C#�� V���b:\�"%�cZ����i�����,�E��+��Z�Q�L�AMN�l�6lVgpI���Ru�����T��SW�9�q�����p"C'S����g��scO6����������yw��~U��x�8lN��<���=�!��\/E��$��|R��9}C���(���v;�c��"P����&��j�����=m�a,q��Q(�r)��F�+H}J��^��fM��2��b�w�)��0�Dw��6q�v��BC[��QI�)��((�6y�F�����q]�B	
���<��.�����9�����%u ��b�t���M����H��xr��(��3�C��~z���eS��A�q*��d=C|��/,NW�h]C���h�&�;L����Q:��2�����#E�gP<G��}�%/�ye���D�g�g�%q��lq2�Q���i��*:={���t3�b��@Ft�@c�K3�%������x:���E0�,����+�=�v�-��MQ�Z�|�g���PTD�8�����%�:I0�b��?��-1��4%QL?~�y.��yH)%K;�jJ���^a-���A�J.��u�����;�7@�X3�����������9����E�M�������z�j4��K$�:V0QH�
����t|q�
�K��7iG3�?zE'�?e��gb"]K*U�4i}-	i����*%B��A1�d�s�����o�q��R�u�������QIlYR���r��E���A�2l�&�,��7�I�[I\[(x���<��7SV���$�����2-�n�Sr4���^���j]������]3�����)�2L�W/I�]��}p�@~�����#����h��,sq� �OBF�����
��B���Fj���@~��jMi��k��X�����C��Y.���T������h��$Tb��t%��E`('������tRU_��������p��_9��r�w�#�������z .]��fA/X�/hg��S��������,���+u9V��,}��:)�m$�F�_�u$t�h�3�YPg�g�ae�B��`��u|�X$CZ4��j�H��:��3���G>��
@�����/V)�z�8J�%��Y���������nO��g�Cj�V8�*\���.'<��nB��+�Q"�������nwJ2/��}{��IY����mt�I�U@eO��C`_d����l��@��s�����������,MtF�x���J�_JVnl6���H2�L��?��X}��T��g�2x��eP'���n���/�j�=}:�6�1f��"#T���1��B[`��"����se�<U�p?����{�O��,<������]BXr���Wf��J���CBc'��e�P�j��
`���T	H4=��k����N��hG�~�K�by�����!��, �9���f�\ngU�O�Cu<A��m�$�X������q<���Sn4����Ei?]�2Q�H
�~fZ�X��J�"kAzy�� 5���^���	��t9����o�]��n$��o�\�~��Z�
1���]�`�����BxG���������
�f������g)F���%@-��w��W�U�������0�2feL��*	�1�6���k����	����:,�
�ob��41�v��TD"�2�6�@����I�
K�'p��	����|�����h&���/���s.�k�@g�8�����I1�N*�gm�8��|nPG�yZQ��E��x�8���QKL�"Y�K<���>���+3��T���������1!M������o��e�y<���Y���im7A�6�F_�yBL���8��/�r�������W���z���c�)�3\Zu�`$e����U����u}�;4U���odE��U����tY���BWB�����������������{e�.��������u+�|s"�FbkJb
N�I����Y<��Y�����k���83�1D�3�������+"�B,N��������6�J[�Op�r����L�i	���������k���G�7(��+L��h7D9�
"RPwP���^���o?U��{���4m�X��H���fFE��li�+���_����PD����8X���?�-,��-��Dd6�J]����q���1(��.D�l�Kb/��(Q"����>R�)�%J�U7i�v
J�@�]����EF��{�&,���H�����?�M0��Q.�bw�8i7�A�#��	U���<�m3�t�.vr��;��Ku�*�W-m�R�����w�y�~��/R���Y	A &�Ii�#n[��$���@:��m��m?�,�d�a����|e}�U�6uD�t��^N�[j���AzyLgyQ�]hc\)�\���1�5�ng�����l?Vp?�����I�*��{�IDv���k�aE�n��@u��6�O����Z1�f�{�&
c�	�P�|��V!��7o*-������R�>T��B}��Hs%��H{O�(y.rcL;g������,*7^�;![��Djj:.3����?��jz���sg�bps�lSz��	����t�\�
*��\�_������@��yD�(�'$��q�\6`s�G�(
yhP)��8~$���0 ���_��M��q-\VS���O&~�6�R�E����m�m�?{Q���'&51+���uDjZ��l�_���p�����3�;,9,�I��,<�������3�j�[l��81���X����@4���rh�o\�(@�!^60�+�o��������$��q�f�6B���r����crI���|��w�m�
���<���GC��8�RmL6,�S[
�w-�0>�$b99�X%��i}`��o�@��9����"�D��j�����(����F�r�}=�>������B|�&
F���6���A�m��.����/���
�_#�X�{�1�p�������r�#��-#�K-f�i��a��@A��y�wp���@��
x�h����"	�<Z��\n�b�����8����14��$n2g6��2�\�o
�N��a�P����w�t����S���\p�NO
:	u��s��+����F:M�w4��:f{:��1�N!}����3�"��(H"��]Y�GW���/R��*�*�3�e�Z������>1�b#M6�Bh0����o���3��������
��;��%/u�"� ���ri��+y��\IU��*n���{��3uf������/��h�D2��!M�g�|�9,�����+�����#jc=c5����!�|��������Y�Q�r�~C��)�MYl+�Bk��
�����������=�c>YKe<����z*�l��s����w~shJ���
�W?��"X�����U.�B���g��������r�3����",:	�:��2E	��8�N�i��.�����d�4m�5q�Zq����������m�<<x+��,�i[����Pb2ND~��s������pv�?�����n�/8K���
%�����P1�l��/<���vA���4�~����i�v����^bO�oM��~:\:/8^�����a;]^�I�������~9�� 3j�'(1�]���v�3������hC�t&�3����F��t���i~5�kv������������������������,��p)�(#
�\�i����u}(��L~$0����l�����W=M���1"���`�������?����k*�78|�:��4�O� 2$l�AjI]R)��i��E�Mu8����k��d��&�����J�;�����;�CE���E��k�vs��8�F�������G�)|�=p>T��v��)��.�w[�����6�4�����R���IP��*Y�H8��3�������K;�<d�r���U��*O4�<���f50��X��-��ISkI�7���eV�����:��o��`�Q��F��R_��9i���yvQ�!m �eB������{��K�*����.5�Q1'i�Lpf�0��s}��P�~��7�C,���>"����o��h���(G�v'������j95�YB�^�{�����+q���^�4���#�:lO�n5��F�c��OL�X�Y��3��WB��}W}�����ZJa���?�#e�v#:+��&��'6G�v5A;���|Y�A�"e�R��u�k�o���}����7���_���k�y�T�j�{��\m���Xo��B��	�7���.��u��u�~�l����xho�������1SF��J&����#:��f<����PU^M2��f�e,aG��yq'U
��4�gGe��_���~����[XL���FG:�g�z#l�Q�����Q��B��.�pu�*�����K=��(����
�Q)l�g���L(}����� �����A���=m!
���o������,���_���|��Fp?H�M���8���>�;���}�5�K�g������v=z����(�����n[����O�����������y)J?�*�'�b�zA
��4�k7V��'���2��K�6�����;:�B�D�S�m��%�D��_.�����k<�������Se�Ua�Se�}�����&�~^Y��@��c-y�����}q���&z-��%m�������s�=����~��ge@�M���M�9��[9��3x��n������O�����$K���d�/������g�zfl.a�R(K]�*"���?y�B����lF^0�����}�}�+V��7o��
Zy�{�<�����Y���q,u�$B�����$N0����`��R�a;��k�C	U@Cy���/��t�-4=�Ng�iw��&*����qd���$X%��r�6I�)�.��$�
���B���`�����^L������n���>�Z��
���q���tvcP[���8���o���~���:O���_�~����e6	�5M1���C}l�z�������G~���%�v��*XmB���b_�7��o�*��k�/f)��Xuh����0G�X��*��-�W|a�$urM�I	�BK�Ef������\���oM�#�2,|�������`����\�!����C�rt�9���V �{ ��2��V���8u�n=~���B�����������X|}wr=�������%c���x}���b��!W.��+=�DF0�����J!q�$&�H�/7�����vU7��
���y�k%�~�)���)
S*��RjwH�]n���:��L%���=�����di0��C���6�	����������*?��!#Fy��_�P&u��o��p\D�B�2Sw��Od/����A�J��:$�\���*]a��Z���p��
�����$�@#��E���D�I6�<�q}�%��f�%�mW�l)�j���@��Z$�-+,)�s�>Ul�i)IW�=���3�D{I���w���uFC��������?mwx��<��U�O�!�12����Y��Y�
1�-)e{~h��U;��/�j�'�$����c����%���jV������j���2B��M9~}�6��Svc�Aa��K����.�L�MI���^���;.�����dn�u�kw����t:��~h��CfN�_'���&��B���U��@�&o����~��_������w@�����$,�d������kO����}
,����	B������n�r�U1�XG�O�'Z�!���+�XNc���J�)�+:w�}�{��i���n���������J�0��*��Bv��z��|���J������������v?�s/7i�YT�5[�C���$��w�*��9m�)X��dR�x��\�N�C9�(	�f�������`n,b�:J����~�s/���s2zt0�|l�_\	�n��+����5G��T���Us0o�B�J�� � �5��^7�>�����y������#a�o*��)W��b���n}�?����g��@$��7�1}���B�r�p��g�'g�C���B��#�!�mA�6,#��
�}mrI��d*��T��	V��X��*������O�N�o�[�����y+UFmdii����m26�ZK�/���t~�m�zs�8�M��a@|���<�!{k*�I��}���DS�@$d��9[����#�FR����>o];mF�%�$���2k���V!����L�2�F��1"��Xa>u��_��"c�����0��M�n�����	v"T���w��R0����M�V�i*A��E��K$.�Q0� ��!��}8Q��dZ���G�����{��
T�8?o�Y����W�Y��o��q:�Al +1�:W�+>�o�����544�Hng�M�5�Q���P�d��f*��!�r��)�V?�yG>���
�iGo��^H�B ���2����:�,p:
h�KnG������P�;�������8`������h}����&����������M���M��������
������h�F�cA	��!ARO�1�N}�i9<�r���v�{}�Z������2�"��1|$[�=��mI��v�+�\�@��=�o�����=�@�J!e�`4AB=�G����r��0Zo���{��g�����lcz	�l�Q7F�V��y�����X�<��x.Nrw�2��\������2�9�C�*���!J�
����T��B�p��� 4bI�v����-���$9J����h�G=��,y�55QM�\���������V������,MyI�iB�%���/�@��M��Mx�m�����1�Nsy,��Y���A�t�6�e��b��`=p���y���/�i1j���� ej�����$��+�_����(��;�hl�M���V��Q�B�����e<����wo:������v�C<K��{���I#����o���~�����77mvf�	S�Ii9g=~�k��������OnQW��[Hs��G�?�����Tj8���l^�����i}�v!+�Wd
�E��\�H��e��,@�c�a/��������c���~7�4��p��p���� 1���Nu)7JT�1�#��w������*���[K��B�R��>���%_�D(E��5K"t��T'��z�i)��c�O��sb����Bkg'���}M��	���{����tk]��V��/!N��;V���n�C���
YEY�t�v���TI�*x�p�
�x�	2Gi���/�w�%������zO��, I�e���ZD��P�
�#_p��B>\Q�Kw=*wK��[ y�Kx��:�a����{��"����
�1��6���`^��Y�3F��2�F�/�	#�t>��"=���v:�b�N���(����������$\����g(0�S�O�c����v:�:'B,l1p��i�`�wzj��0��i	%�V"w�2UAu����8/ -�����f1w�������hj
9��L�2sMMj+�
P�j���L����������{�-�p"T3��09���s�~�aQq�,%��)0��!�:�^�_�����_��n�U��d5u��+AB����&w�����P��)R���������5p�������M�9�[�DL�W(2�����%?���O��N��!J(�<W;��0��aB����F��������o����T����u`��{����@<C���_���%����"���>�������������	2�D�C���=��<���OW���Z�6���M`�^�!oPAhh�0.�w�D����]��OoR�O�{���$�\��2-�A���{���LUl4Z'����]����r�pCLc�\��/�h���^��������=�K*].q��/���HX�(H`��l�k�6x���1%�$b���2o��s�!E%b}>����Aj�����P�C'�LQ|4K*�-�?�nZ�dzI��kT�<���>���[I�3CL���}�|"#�o(���_�����~�<��Nh�K�h~@�7X�D,�)�4�W��vq�'J��]lr�n+
�tWV-�*E��L��&�eI 'L��RJ�m�\x��N(���v����#��8�@���\��r���
.?n;A#�K�����R\&������u�����]i�9��"]�5]��_5�������/�=P� <�jJ�.��w��y��Sn�[s�r`g-��gD��;�����:��t����	2�aO����)����We��+M40���
��hL�,��I�H!��UU�M��k��z�������`!S��91}E����z��D���Sy�b\5��B�R�����M?�^���7i-1��2��o��o����6�_���zz�#[T�x(�&���EM��pN
J-�)��9��E#��a������F���s\Kiu<C��0��"J���a�����J"��90����Ap��	��8�3�3C�t2���_���W���G�)� ��W:��	�����y`��n#��LXbK9�� �/���=��`��3�)�C����P��Z�6--�W"j8	7h���A�720����������nN�ddr �\Q�_T��&�I��z�>Fc���yl��V��v�����tw�� �S�s5.�B����?������:�l��.IyP�Lf���jUQ���wIU�-�Q~��l?;4��s����S���\�����=[�K���B:\��>.�����������������$.��\3
�r�u�@�� ��������	E��^7�m��n���<���S���v�XY�a�JQ�^��9���
�1�6�P\��.�)��<�O{win�n���9zr���z��=��� (�oX������CKy�7��{�l����s����AV��veD1����$�F� L9�v�/;�(���v�v`���rP�g8)����VK����?���2-�#���b�{��o����{��i���=F�����h��s�0Hk��b�q_����9,����p����d���`!y=f�z�+%��\��A�F�w�i�
���J�Diq��b��n��5������i2���^��eW��l@rz��S�Y����\:�����}rF��S,"��Y0	�� B�S���w����]{:5�m�U1��4�S2@�{��
�Rp�K���!�t��`8rT)���k����@��������A7��h�B3s�Z�-� h�c�������0��������	�&�m3�Y�����%51����s����y��}�V�4g�UI'���������	�����7J���*=3S��>U��^���Q)n�;����W����C�C�3�F����?��4���t���H�?��|�i�}�mp����`�PBY�f���E�r�D����dA> ��������>N������G�j{�B�I2*2�e	�.UQ"p� Jm��W�|���~��U��?B\����=`A��v(�V��0�O�r/;M�o���� �nA�,����5����������0���n�*�����}���Sg={$CF�QE��T���B}���f��������~x�����n	V��\>���W
�8��M��p/`��3�#|"XK�^{�'Z�
�i��_�����`�_��C�7����_V^@�D�bF����qO���M�g���^#���yiM�\<��K��t�%4�X
 �6��r�ns�m+�J)��g���K��`W�[[��Nm�CJ���c=<E�r�kN�~]$���F�k	4ZQ$KNH�<
����� ���{����w�����lo�x$�+F�
���s�{���B���|hOK����!h�<�
��uG�\u��LB9�4H=1�c�"��P&H�
�R���m�<��r��5��i��xS)5A������W���s�d�#7��@��T�K�e1._�f��y�n�u����@�F)��1ejH1;6���k�F���I(��c����-���N5�QG>��)�]���+06�H����bb4��y�O������M�������3��E��jR1�k��mm1y�B�a��^����&��N��%���0p���T�dHV��Jd){tRv�����~�2���h��,w�B.6�����s���S����o�c�8�%��&I�X ���M������=G��o��i�j�E�(�kn�A��f���{��!y}m���%���n�I����S���w&&�w��B���Z�5��p�E�B�qW�-E��s��z�9�y�<�(�xD�5�����BN7�CF��9���;��j�XT�5{�%8�������-K(�������"[Y��^��k�M��
��tam)���S�}7����F��R�Xzu}e��	�gp�@;U$�iP���r����'��5���u��v����LH1���}h����������O���9�j��((%�0X>
��r/�XgP�y���;�T�Q+��&V��d~�6��m�\�cw��S_� ;��T�cis����o�_��!I�	�Q:&��O�����z�jT�P�3F�>��]����U)�n�*����j�a-�$�+�RpXJ����@���[�*=
	V�s�)�zEuj�&Y%p�_����&���;�6������r���U��i;$�H$�Ke�Q�����\l��=������$�c!G	Q���Fe>����K��r�6~����v�S8�,/��hX^22
N%�uu��K��1�h��n������l��$(�&�%jQJ)�VP�0�Xj��������J���a��b�&���P�A���p�lW�����C�]���LG�C��5o��B�xB"v
��Rx��]�l?��������@�%��.c��b�[�{�k��c`��~k����u�r���C����G�D��q����7�8\���"����8�:���h
�A�6���a
�x�����Ap;Sb]
����koN.34.`t�" sv�1pa�~]Z,X1]bw���$��5?jw�!�E����1��D�^����c~1:,��t�yW:��Y��k��;��^�&���@'��F�3��������[�Ur��
X\���%7[f��j�nHpW�R�����bHY�!��^�"�;}���Srw���KEr-���t��_����	�*��=!]U��"�����!!�rB�jp(�*������V�oW�q�7�M	�4��B)�gg���q��������5� x�e��`���M�8m;X���	�`�`�������;�����=�C*���n5cM�]$�E�8�u���qPj����_���Wj#[�y�Zu��
���  ���e��7��
�
���Z�.�vS�jeFQ�$�7���Z��.����OQ�>�L���'�J����4^������n��p&���U7�C1�%���,�{����/q�'�|������]s:����od���H����h~/M�Q��f���,�������~h;0A[��S�>_���-*����\9�u�����w����nT�U8��Y��!�&W��*��`����H��5�*#+Uh2gMQ��M9|wZ�K���O����@t����40�C!p�H�rMm`0�M2��da����SR��a�[���U��C�U:.�
�c�o
�9��GM�zee)|�yJ=��3�����P�]�G9�V�8+{:��=�&���b�)���F�*�
w7�-&y n��b�}<x�Xf�>�D)�����g��p�b�����/G��w���"m�g�
l����b$�c��b+\�|��������R�n!�f:�@�P����EK�����;bC6[���}_�~5M�	��(�*Xr��Mw�R�^�k�Q����"���+��6�2�x$���T)�B�����=����d8�*{�>G7/���D1���v���?N;H�r��\��h��"R-`\��K�H��������];0,e|L�S4��*�r�%��Q�i�hE���H��.��L����Vt�����[�`&B�#�
��D�J0aB���H�����(-����)�"Z��/�m{R&yER�7T��������W���8����IJ���]����H�]�na��G2����g�~���Oq��m�3t�r����l��`��Hm��������4g��Cs�?��i��x�`��3.�����'�^iV
C���a<�vCm+!��C���b,��PDFB0L��>Q�G��~�K�	�>����N�I��
p��~s�������}��@���I?B��M%0^��e�@b+���)DA�4x��]����,���y�������ij5%O-x~f����8�����br>�z��!���������[eV�9v�9���dN�_��^���:+Q
�{��������A��J�@3rm(	[,����U�
�<�V��i���{��F;�1N^�J����b��^��W�-*&Ci��d��[��[��]�x�Qo�R2��0$�"�3N�pwW���S�\4j��������w��9>4��y���hf�I�l2�R�[��]H%q��|z
�6���b����FBM?����i1�K,
�I\a�8[�b�?��2��L�)+��H��,(���������By��f@��ar�Nq�W��X�F��G�`�)
�����������)�yh����~ov�6/[�|H��M�����H�na��k�v��E��	�^���(|Lm��!$�n�
�^%l���_q����gD���BD����m�I��O�n��?��^��g����� �;=Ev�f��s�����
waS
�tQ��G�����U�$���oBV`/��Q�%�P�c�t��|~�c[�;���_R�5�j���I��e|��|w>��T��
�z::� !�j�-�f���V�6B��6��Fy���z����H�C�>�h���<l�������F��<�l��� @�����g��ikHJ?��~���f�9���b�L"�u�BJ$���^���&�6��W?����ywAE����_(�#4�1 �n
��q:�l��}=��6G?/��������ji^,=1��y���Ow�8����������W���	4y������J
�]$��JJT��U���U�A�$U�?�������'B�V�Fo�V�C��f�"��<������Q��F�6yHX��[R���8Wr*\e�J������V�����/������~�]�W-M�UK�L~>�\��J���8!������k��k���6��'��42�NB��(���6����qp_���W�����x�+��&��/2���_&Q�J(�����	_d�Z�O��v�G�~���s�z(0���%�aM1�R�E��X��.|�X8w���;�������]�OT�������2FV~~��f�,�����y�����F�;�j%y<T���������'�$"�d����a�����Ju|��g 3�#LN�l�d&W�
����|Je�������;���G�K�IJ�@t��V�}i��P��RjO�6�1��������8ah���I!�;N����`��3�j����=�WE<]���d�t����b���C��E�L��t��i<�z@�$R��h�q|o:�b$��t%��g������$��&�z�NGA�R�������s64�+6�-����JL�v�b:0���Lf�������)����Bf����Tw�u�;~�.�tP�h$U(�!�d�����F�k(�H0��ik�9��'��fH�U����l.�~�6�����$�I�z�GS$�>N5��Z���g��O���j}<���;�VF?{.2�H���G-���+���~u�J�]��cN�G�{������b�!�����A~�!p9#,Ti��O	���BQ���f�C��[b���GO�����;�T1����R��N�X.�A���9%\A\�I` �p��#��]�eSz{j"�A��M��[��k���p�[_��0&V$=z����������M{7h��}%�.
 1��!����Jz�a��Z"��8ZQ(�q�M�V����DX60��;3m����EN
�:�N����#��j+��P	e�1�j�>�5dS���(���Y����p�y�/p��9�<h7�������D�����iH	�@�~�z-W��BGi��"56��59�*&��]��������Yl(�]���Y+A���k�.R��R%DA��f���X����z�~i�X��L����.L^$���8Gy)�����O�M?��������!.�m�[�y�n=]�\z����r��)���py��x6�M���������������f�
��f�O�4��n�
pR�Y��B�XkC�r��p��
{��������y�TN�x�K�����Ww_HQB�9P?�J��]s��<xOd!�Lo��t���{������A�P-�m�����xp���l�}�)R������MK
���u_�b�Q�K�O[X���'�R�lv;���M��7,��������)��Q
I�V��D1N���Q��[���^�3Lg�y�u|�����+eD%�;�"�����V==���S+JT�A(��
�Y\����P@Z?M9\kT>�Z��c��<5Gl���"H�%��������Y������(��h� S�������N��U7-�,���,v�����=���r29<@��E��~���X�\Y��9$W����b��� ;��Iu�>]j���Q$���I/O�mf���T$��@Q����������q]4q�f��n�O���/������A%46��&R��Ey���,��j�����Z���|<����M^��N=g0E��e +��������pc&��\�K��@���7�>Y<?<�Q�]�Q�N8�/��A;v�����k8���Y�rki0��i�oQ C#���}J�n��(���zxs|��_�Ts��hJ��@��w}�������S��-�]����9m�5��]��~�u�d�����;%2z�&�����PbVB�r�Z���������Ri$��:��x��7�����!h�i����h��2#�4�@��
�V���]��b#�k�������|�V��H$��
���8�����0��>{��3����Si�'�19���
��S�.K[~���x��m9������{|�!<T���a�d�L��K�"%R��TO���#����J��3�WIC��\������P�����`Q�LiQYDD��LP���PLp?���8|���i���M=5�H�3�����%Y;�uo��D�J_�����{u��i�����������|#���*T#�w�Y���#~��T����d�h��u����
�}��-�Ta[<U���C��������R�g���1��A��]���%??�7A��\�>������m�-$d/7\�����M�q���a�6o���t�IZ��U0�+�-?����	��?p=$��9!I��vM�F��\�I�����q����9m��m�SM����&��u�K
E�-p2U����K��?Kn{�������M��<i��O�F^/��hM�1��)�"�RI���z$��������A�B�(��E�Q�<0�/4D�fJ�F���2"NX}��
j!�{�E�����F��y����w�|n'+=�
f~��]�
���1����o���C��Jw�����g��RT�����w��8��n^
�=��a}�sv��I�x]�B�5�c�?�L����J����B��2����L�$@$V	T�n��Cn��Y���}��������i:*E(��eYx�T*�T��������`�\��P�h%���Q���0u
�)��:&�g��M'�A/�#���5�^r��������]��*D��=�t5���>�WZ��,���oV-
Y�_@������S����m(��oN�U�R�f-�0 j�K���b�`�G��D��E��K�[��[�Bv��:iEn�dc�]�6��m��U�r��
��HWb�b�j��L�O
��2���c���XIY
���������]��<WI���\(���@��4@��?V���JImO�z�n�*6���������o����2tX��\u���@�=A6����t7r1
�X���h�U��,m�!\��n+R�$Z���O$BpD��������1���������8Gx��M�J9�����n{@��������2
#��~��X�19*�����@7*��r�����!����RZ�	��3�c��V���[SM��(�9aN���J��pw�����;���4���gh/�
eD���"���c� 5��J�D>R���<l��!��G�Np:����EF������uO�V��R������~
s�A���+��Y�9dE�D�QnMAA$9��� @���wv���s**�$6��J�{�ga]YF 	A�:_�E�����<\��-gM�5���	��:md
�X�H��i�k;����5i���]8�����X���!s�����N���O@�Y����Z���I�)�
�X��L������@%.`=o�G �b.3�+zUd��������22�We��#�������n�#nT������#�
��qu��de���z�����Z@����f2mDSV-uE�Ry]2�;�R�U��f3"p�c�;V�%QL
$x`��S���(�ns�v�'jL����������aW�IW,���Z_�����X�^s��T��������������v��W�l-aH@�4�i�W�/�������[��}���
��E���+�
e�vW���l96t��	�B�b���C�o�?<��G���Z��f<���t�����2��d��(+eQ������bE��4��97�n����w�z
�����<�3����a���p��,+������p��,b/����2�)���^�����y��]�8��;�?���E6���(aS��]Q���#$�Z�I}��+CJ�p�]�M7��n��-jB�r�+)��DwjYA�1�$#~!�Uv>��jW�aS��	�x���Wr��0��@��tj}n���Z�$�f����,�!X�,qC_����um9S����lF1��y�a@D%H��� �%�+[�����
��2�L�����c78�0,�*F#�����G�%��L��5>�>O�ke,�>A�#����U�Nz1��������nd$H�d$P:Q��X��m)6��
������>�nk���_����d)��^��
��,x���`�%�$* H��������$exz�eA�h
9��#��O8�� -��!D �w��L�.��:^�P���e��n���=x��+9S2S>u>]���B`�q@t���5o#��=�Jg�P5�ZZK.*���_G���G��t�=w���{~��A�3]�.�A;������o��pq�f�`+��;��><���B��B�	�U�e$�T��5�G�?�����y�����M����(9;���"����q��q�y�����iV����44�RH'#�&�b��7�PZ3��b��e!`e�-�\�8GJ�,���Zbt<�]Tj^[F������k�ry�����*���b�g�fX��kp�=��I	��\+��
*?�)�)��u������Z��3j�d�lS"��-rEi��@d��(�W�����/d\���W3wh1�y�%g��E�,��=>Z�b�g�!e������lpY,IK)������7
dlN�,��/��LW���-_�b��nM?@��B����(S�����<=�0����/y5�����\+>|=�>5�i?�aL���z������i�f�w*h���Q��������0h���������
v���2d�d\Eq�V����Gn���9�AH�[�m�n{�@O��l���b!��X�Zg�r$�����`��T-*f����n�"/�Gh����=A�����{�t�E��������7[.��Mh������G[������Aj$��Q����S�?<�~���e.cRU��CU�v���J�8J	E�d�A���f�q�������b�����9�	��*�D��|fo��o�T�`�{���r~Rc�+i�����{�l�~�����(�u����j?w�r����PH9�%/��:Ux
����d���u����G��y��e3���GR�Q��\������8n��*s�;�G^�`�s����p��q��5�Q�R����Y�^,�����n�=,+�T/7�D]��Z�K��{v���l0.#��J-����yg�*�y�DQ�B��!%��������y���0��XRWf�C8^�=��-&���G�f���=]�}�e������y�xS9%��YJ�-��l;N
���c�#>�h�O������+����:�]�@��5gQ��E��Nk��������c����H���6�x6��w��Q
�^�\�7��]s��6K�L	F��k�((+��(E�	������G����WH�'����KX����$H�d�������k�m��~~����/�R��D}L���1��zS�n�F*K����r�n6������]}����-��*��|	�����������!Yt��>�W�%�tz�IUVZ��u�AZ���q���o<l�����?����k3��[�ZwXE��Q�p�"�Z\��P�s�F�|�7n��~�p��{��T@d�Y:$G�D[���!A�����������U�f��Cu�_2�@��O���(Y��H�I��|���=�Za�MBH���17����(QV��=�v��+<��v�yDm4�H��\�������X��-�>e_�U��vvK4!�Z�r�h�s�����aGJ��F�	���G����������wO�6���Q�"u�������h��3�#����yl��k��l����9���pZ�BW.����*\c���x.�Fd�Ru��Y!��t����p����%aD"H/�&��MX�y����:���j*�:#��G�p��d�s�VK��+<�e���|Q����B��KM��v����fW��3��
0�{w��R�f 0�Au�>�1�����7�#�Kn��K�8Q�XAQ~*��Tn�Jw|��{&
K�$Ilb���l�dK�������Ls�.ML(yp��k��P�	Ef��Rm��e���n�g�o])W��i��y���������0����ML�N�Q��3�,�P�l�������p�����_|3�i�d,�	"��}$t<�)����l���>�j_�&�R�nt�#I @�`���\|n�v�
�����o��	�~)����	^���RRw�M.}�l�
��.A�|�����W
�~S���OU�#8zEb���o�gs��T����[W�X65������3�Q���(�W�	+S
��H��k�h6��k�3702�$Q���S�
0����'M7<1d����>�<Y���o%'��l�����1.��F*r9���q��o����-P
���R
�L��T
�Q�|3����������5�n����7;W�!��4j)���\=q�'��/W�����gN��3�����[�(4G��8��Es�n��cU�m54{��E����q��2=�S������>�Y��I9�����;[�������6L����a3�^��A�������j�U.�/��������C�<#kI�
Ihg�u%��K�m�_���c��O=�%/���R����`p��fWm-�ah�6�1l��_?�H��G(�@�K�	)�0��"���+���jU���W
{B�FE�%H=1n�s~l���MJTI��,�=�=���`�B2UH��@������o�l�
���<=��$��3J`f:��_^����W������^��vk�����z�<x4�O����tE�� C���zwr�55�����q����o����i!T5�>�<�1�j$����G����=�����z����\�>�|b)�Q~���.9:��R��	�
�HXM���Rr�7�
�,�r���{t>�`)�d�s���W��x�<���}�B.Bo ��X�%gFry#�R
�)��m�
��@d�YRM�c	�g���r}��������h�;������;��_��)(��X��,A6G��8������������zk�mn&���+WS�+YE2�
l"�|���3�U����qwzz����R�L�R�`Q����\�.abE^x�?�Nfn�c�l���������S�}Nb�t����r��������F�t����2J�^�{/�,�c��b1n�^=9�C ��]��s8_Q��U����\�q�9>�������5��������{b�[P^�9�bFp�T�s���M������i����=������i��T<zLaM7	���R���F�<� �d������Y��T.���`i�-YNn�=��O�u�?�j����l�8G�#�q���ekM��R�~����\�wD�m���Ez��T��[����'�5������^����������kZlZ���h�y��8�kH����	�a��A��u�P�?��H&�'�����FH�t�[�?�����
��������}���������X@���
`�M$nh��}Zt<5l*���T���Q��s�#����|l����R|#�E��w ��$z�C��"�J�F�9p�#��E3M�U�5���IJ7�LK��a�(�d��m.����C#j�����y
�:(�&v	���K[�,(���lDs��|�����ow�=w�~�������������Z#.h����R�W��'wJ����d���ig�����v>�9�k���B��>
lk��*��LI�V���JW�$v��]�!d.8	x�~�j��/ �L[�R����K��Hc�{O6e6���
���w��&f��X�Y��f���l��1��
�� �b�j�	�<�&�i�,�{8}�s�#���>�%x
�G��JF����L=p���l����r��k�����&q�(��&��Bv{�,��
�Zd�
���w����c�>&����^F�%��Kx^���%�#�����:?�W�����vt��������"����nGK�7���y�n����B\���{���0s�v�rs{�CgL�E��.`?��9	E)��9Dz.d�5����p�8e�B�B�8
BN�����0|;U��V� �Pe�NG%��<}S�1bH���
=�<�7��V��{�\�&M��9�;�~��B�f>v��I*��k��/u���m����XK�C�jB$�c���zJ`����1�����m=����[��m���`������]]�xEwj�o�n�"�|3���U"��dp�_4��\�2$�HS���J,��%er��B��us����J�|�W��WS�*n��yH������a�m{:���j|\�~*	K�r�������@8@�G��6��J/�;L)I,^,E��A���.���MU������?���t��U���7M����:H��S �Sd��}'AZS����)����]��~C�����r��j ��Qb6��2j��\5:��I��*\�����H����X`	Z
���y�M���~Qp�I�
�;����/��0����E�#�	�:�	).��R��|����k CV����E������X'DE=��d���%6���I�����c�d���n��]�����Z��'wE�JX��RE�]|�������4!e(����7!������Hq����yg�{r��4X�/ D�\�5?��k�E�_=�rw�K�c�]�)�1���3�������y����~���:��q��h���S$)����~��t����|h����f�?��z���(T���5���`���`�f)U,��f�m������f�K��{��w���F���6_T�Lo����^�@���g4E�
���iQ{=�@�A �D.m��
�-gkNV\C�1�?�)�W8��������S�]�U��v�y�]�����������[���$��`R6E\��-[O���3��(i��L�>�:NDu�,����s^��Z������3�Z�$I���[yX�i���`�?�Wg��_P�;;��dc��4z��7I�e�7�~*#B�uM���w���s}�����iW|M�\�c�!�v��I�?�f�\�����������_ ,B���hX� ����/�ErGp$PA���t�W~��x�QQ��������D���p�E[�������)��\p4��4���/Sw@i�r�+��W�v���4R�������t_�=u[��o��;�����}���^������W"�0�1rC�������eF���8�����o@�y�B�D�Z+�Q����S��1>3������=B�O��`�I#�������`�9�_d ��]�;{�%<_{�����������)<�	U����u~Wo��������U�U/�"k��K������u0����dri������Zw�0Z�b+J=o�b���lH���A�kO�@3xO���{J�g/�:��k,oT��I�v?Ota\'����Ohy�z��^�'�BwX�7��.��<I�!j,e�#����������0AO+C��d'�:�*7���w-���<�B]�$�����n��K����Hbe.l���hzm���8'/7GTx�!�jZnJ�c��ci����v�Jw���K�N�����x��E/�����	y�~��8U{���,�I�)#�kI�r8Z'$#�����}s�}���n$v���f��HK�w85G��g�o����nLz����]�1:��]�����,]/�Aa�%���2���� �0"�`
���]�UFu�$����C�� oJ
af8��L{��7�����B�P� TF]����O�����k�4MY�����r�k�W�vw�����fA����,)/k;M5LT	�0A��`h.�}�M$�^x���fc�������I�-e�����+�<2zF�P���r���A�4�Q6����]QD�2�R�|��2$�rH��j���F�5�4��g��i�������
��{�e{o�
M�	=%Z���{���?�����t+� iB�Y�&b@�4X�:,E.{����!�t�C���a��.�+�7	�!e.�cU]ST�$t���l97W���t"�Z��
���M�c�sR���	�v��2
���S�����4HH���������C���?��X�\��V�(�����z��wvg�����A�i&��P�.��#��!��L�oTo��W�{�
���b�"�|jW�18)��FnPng3�1�2I��$�f��G������8DJ	}cKq/���*?1H���iE?���+T��+^�U$O�e�s���;{�;[}|�4����M3rb���rj�O�@2wY.��
�;�����z��2:���G���)���&�$3i���c_u�vo�)�y�(�Ip�8�mF.���@���i��h����8��s�<=��}������:���$M����G���"W���z4��=�����o��`A���}<�wmx�m9�DA�#IY�������	��������S��F/�J�l#����xY����W>�`��^����!�6� X���I�.7t�I��+��oyB����&#�v��]
�Xc�ZQ�r��+��w�=� �C�X����5�'�3�WD��8�M��y��h)���8���%*����{X����_������<�������"��5��
x,�\.�#�685%���py�5�?������[p� �Z��1<]��2��:v�;�o���|U����.�
�^t�y?���J���@��m��I`A��^J!JH��(.+�!�$�W��)x��?��<�����R�a0������)��i���
��{
\j,����W���X�]��D�i6��Oh3n����G�������p*������
-��[�G#�������T��X�N���\=���C���{8����N�&��z@�������I.�F���^����H;fn2�;5^(�$�hI����7�Ie������u}�0�M��9A�Q6�m`r6����Z��H_~J"~qi&Ar��2428���P����O����[��L_R��'�8	�$�Q�"�-���~x���nh��rx=�Ua��������.eHvxq�l��7��?�
�$>}����
Z�r\���cm�m$��|A�QW������4bVs��"��w�'�j7��l�M^MF���
a�_4��S�:(_�F������R)��U6L?�mg����t���ff|5��Ip8x�1���`�G��[w4��v�HD	�Hx-K�����T���(�p{��Q^\E��-A1�����V_��U����M�������J���*�]�ZB
*"�����Q��B��@D6/s)J
�6+�!��C~q��G�<bSF���i�'��"?�O����T:t��][��D��+<�������``k/�&vf�E��R�k]7$�����QU��r�2C��!�pc93���Q�|�z�cJM�)���N<�#�����~��t�\���<�kD!���k��g0/~����\?��f���0mF��P�N�qu���G��E�f�����
u	,JEA�8����N����'y�?'&�_F�~�Z�=W��>��uA��$���5I*LAB@���k=8.����:�?����q��T�g��������$�����7����m*�q�lz�Q2I��7���J&HRe���e0�oNW�������e�C�����ygaaVX���8��&�@���xY(��:���LsB�^��M�\���G2���G���������\-�=?.�U��!jzJ#�TJ$r��]�l��5e��K{KY�$n��\)8�^N�&W��&)m<$��\0c�{��{����z9����a@�%$��V��R��T��D��u�L��s�R�AZ�P���%�S���f�;na3�"l�i&C��$����v���J�c�O{0s���?U�s��r���>^`�Q��~���40R����E��;�!��<��a�k�j����[�~�i������,@�o�2�D�O�W�������p�t	��b�wN��{;MY�DC�!x�0��zFL�gh%/�|���9w�]�)9V�"���\ #��p,)��7j�;>;��>��@�����1M��b����u�[��pet�B���)$
��#�&��s8���b !/9�B�<�����r��4�u�N��l��J�(�~�+D����~�v����>C�*W,�9�2^��
���H���;�f?\��(a	�\Nm���=h<�3�\t��N,A��	J4H���O��sw���~.�_8�1!�^+$�a]�p�(ep�a����u	�%�Stu�\��?�q/����,���)��r�l�VtF�VN�A��9�zb�7K�����u�\�!��F��.�@����Zv���j@���������>�������l2�#/�M6��}v��C��P77�������5��gi�����"CT����B���(��P��?k��0e*��lQAX�H��r���Y�K�� �W"����m`�����1��:��q�[��l���t��)(���� ����
�Zy����wN���y��L��N]�o��u�^�q����o6�i}BP<�'�?��������a5��m�_oO�7��n�_����o��x/
i$����.x>hE�Bb��`���O9�$8R�f"�	Y�L��4����o��S�	v�t�i����d����|�!"�i���(ua�����m���r��09K��_
!�������>��B~��6r_���jK��A�Q�(�eUY
���:o���>mv�myxt����G 
S;Y���X�{>u�Y��K����~���Au��K������./g��,:���8KuG����R�;t�>��q��l.���K���
eS�!��=����NC��[F��/;������.�����=���h49en��v@GD��h�y0��z���tp��48��%� $�k�WXd``x��	�k%>|N�!�*���1����x�<��8�m�@$%��{B}��8��
)W��+�
��%��v��d9��:?�=�`�������G��(B|w��c�;vb}8�����A�v��IeS]X�U��o8��_g�m�Vm�x��7n���������)�.pb�)�q����8�Dj�	�F5UJ.}B��,��vMul�N���;
�����*}!PVZ��vJ��������|���{��%�}�{�;:���c��	����D�������G�Y�D��8�
������2���yE:��)w�	�W� � ��
���iw��z~��	F�<��n���4�J����~�Q�_������jE�(%����|��t-�k?�nu6�~}:�@t��[K�M�����!��0C2��O������
G����'���"����p[0������g~<�@��-�sXDp�!�0��h��������)d+]����fW=�B���N�4l��F@�����
)8rG�J�\H0e���m������$��!F�&���=Uw`��CS�g��mW�1=����@����1R3)u��$�/��(���#�����#��D	�p���9�
��;yDY��sw����X��4)��P�uN������e���������~�o������uH��EM��0:
1B�Q��QO��`3�k���*��t�����T���=���4�t�&�u!��ab���jzs7X�;U0��TNlyf���i$�H���M��KK�i�Q$�t4+
�����`�T�|�x"!v�U\o,��1�*N!������e��������hI�����������t^v\~��g%���r@���i6�r-��������sE~w�-�J�������K`�	+O4�3���Iw�����%����Id"F�!/���'sa�L�}�`	�9yk;����)��pX,9��e9��}N��p�����_����L�tpm<{��5��
KN2��[�s�����s�s�|u���ch|������g�c�������Ux�M����
p��_��������Q��s+>>�I��dge������ks�9o�����$���w
d��#]���l��g�c���������<wE����y��X��|!�Vh�(��>_+<��t?�9&p�D������9��mre����h�8	�(�r"����5����
����(?�A7�?�S�����F!`o�� �E��s��5����8�B�o�\�D\�!��L3�#I|l��$�J_Da
\��%��0��$��B�A$���'����DQd�$�x���&��!R��XF��,u��N��"���g�e0�w���CY]�iAL���~���5f	J`��,B��a�\�\�T��@����>��Q���h�B��4-���5���q�L"�7��J�\��������G\�d!�f�=y���h������0����y��h0��1d^m]'���A~�	��:��MP�I�;�x�A���7�Z�]��~�*/x�O������m�/�����A�����x�Lj�\m����6���14�~�]|�T���������L��
����t����(�D*��N�7��?��R&,D���S��l���#�[���$I.����)�qJC���m(h*N:����1��Jh���������F2��[02�vjd���%�H�f�M�
�%+J���F�=��^�������6o*������0)�QVx�f�,���zs�����NK���OWg���d2��!�+}�I�\�T�������K�����������PD�@�������|�=����v����R&�b��C	�$Sp�{6�"�\(�#	�N0�e>�7}�Y�G��b�{M6nh��\�[��n���]S���\GTo��7D�h���	�^n���K�G��O>R�P;
���I����R� �����}�%-�t��CJ{#���`�s��I��L�M���<WT��s���W}XKB<�&~.G���� w��(����	��z�����r<m���x��5<���{7��=�:���Or���p����e�|:���9��~�#���&.�J!)?�!����+F�bTK�;9I&)r��3r'wr�^��X���t:��|V��<\��;5�|kM<��h�{������W����dd�e�<������;j�i��Y���<W��\�*��N�I�����1ZL�!��$bjGR�c��
I�Je�"�;n�?�m��Oro'8.���OVG&h�������0����w���K2h;;�1����&H���|�P�����BUt���d�T���f�$g	1B�S��<!�vi�C�@8\�8	 �����j:���j�{w���3�P��j������\������@8���`@����:�SF�w:[t?#�=��T������P���~����G�����y|]�����	r_�1�w���#�8M �������T�]������b���7�����&��v:N�.�`���L��>_����������ss:����)���D#�
��t.��t7�e3Fz���z_=[���Wp����b8��U�&+�r9���?��e
T�K%�0��v��i��
�I�\�[EV6�����SxA�b~���>!}L���d8S(��B��(:\�������D�
1u�`�jJI��y�#j(�
G��A�2��tl2!:�6H@�>�R�>�5V���"��8^��!��!��B�Y��42ML
��<�����n����H���p�*�&��~H
dZ�t��7�_~���-��Q|��7��$�$���(�y.��q��B�GsCR��!m�'>��i�B �Qd�-�����D��7��!G�q>t\�(G�eP����0�?5]��f]A���qC`p�f�
a�c�,��`c���6�f��|C�����K�|y]����qW���[`�\�q%GH��Z9�'���%�pw(�P�����@T����j��o������aO7������F��F�l:�,R�?N�\>�����������u��O
�"���KROA�!���6C?�|m4��o�d�	K#��a�R>K3�2���B�R��w(?U���B�����6�������R��1��)�C}X���m���|��	�k`
���3��Y@~����9Eh����O����:�rq]f���=:��s6�����(�^/8�����\s��������sP�]�q7�n�g��������>8���5�e����M�������������$�s����X4��+xc����D��pP�&I��5���Tw�o��V����O��~�YPGCh2���D
����;U�4�� �Tp_Aa���Tve������7��n���%��n�]���s�t�y��`8[=%�'�`&�}�W���u��������I-ga�v����O*�8_���'2��?5��?�� '�4��PL�B�L{#�p����R�!7��~����S���j������w�wu�^��D���Rmn�%b"`����$�����w��w����>���TH�5���u��g�%�W�/�x���~�����k���fZ�9�v����HI�O���W����WGRHi���@p#����
���nN�W{���w����?���n�����P�i��,���b�#�����.��j��|�����:n�]�^�r}��O����4�G
P�n�#Fl"l�IAI!p�&Jz��-������0��&I
�R��I������R���l@�����N�L%�$���I ]u>+�m�X9��C�������>|��;��7����N��`45u���`�s$%z��2pu�����:�����l`��f��j���=����\npG&����d�:�	'��B��V���?J����L��El�a���N���	����o��=1�r.��i���~���
�d��>��I!�Lg^�3����,(7A��Y�Q��Pl�p�3�E�?�x��9A��8,��{}�d�fe���_w��9������[h2���N�����O��L�
�r������q5�q��7�JK
��yJ4��9� �A*��
�x��J�x�i��@�1�`�#O�u���F,�G�����z��T�|���|u�KH�7�s?����F������_�|u��c^��3�S�U�����Oj\&s��_�
LJ��s�fwp@Ih���r�tX���1H���I�����e<��[�fH���07�q��z����8-%��L'g(�8t�B<A6�K�h���d	�l%��x:M�v	�%�����������������A����4��S�%5�%6��`�����$��r�SoWh������3���QP�k�n�R��mL?��|��fjq�C����lx���Q����u�9�e7���]�V$z��0A��l�������1�F(��	i�Ty��t7w�*�uc%����p.��J��:n���������L_�����u���d6������Ck�v�
�����s�@2}fJ�4�%`>�����}������q6��z� i��z�\)�LdDE��&���^��R��\o4v3G��V]~�g���*N��vX'��B��1Hv�!C�_L�������\c�(���D=Sr�4���c�c%E=��A��G���NE�O\J�j�5l���Ut6���^��i�c�/��n_��S>y�{��	W�2Q���>�e��� ��n��BKXm��$e��^�b��#
Cr ��;d/_���J�����j��!��x��R��G*��4�w����������*����b�5E��I	3^/8Zv�y_.^0�O
%��X�v���������o��a���>���Z���1��QdP��v�}�Edu:��@A����l��%$�F2t�2d5,�����l��|jN���&40������/�_OKd���T��^O0�U<��S����j*�\�;�$��o����m#K�u@0r_n���j��[nI��\,��7����_?�%�(�J!��
����T�[��h�y	�e/<���#g�F�����U�B�.���#W���`������Jv��.�����N}�uIr��#�X�������n5���e�����X�����b<Y���e
�7a��m����xW|Bqfj���3�E����������6?-������o��KIsAFki[��
�OH7u�[��\�P�Wz&y���&�XF�����z�=�n"�@"K[s�'�2|�0Me�~,��([@�D��v�v�M�/��f�41�??1�%�<�k�T�b����5�����^��2�<����R��`Fn���I������	�3��
��}�{�>(�*����� >�2��ayWJuy������yF��C����+Wh�GCxXx�\�r/SpM��x��Ib��6�����F|�A����zS���L$/��$��K4��b��4Q|�����!�4yh;���m�ArO�������}�}�3�&�Qr?wD�Q���8��^'��s�3��dyil-�7��I��������d n����"�k~��]����+$V,0�,v<�=���C��Y1�n+���X�m>�7�z�������\>����."�BA����}d�@�/��P���j�wi��|J���8�O������]<kM���@��������n*=�l�4� I�o�N_U��W���1f�])�,��f�r7h��
�����T�Cw��*����
r{��,+��f���x-GC�I��,��v�F�2�N��e�A��,+�V$����B������V����W8���OR���9,(��G�bG��(�O�������D*�/�~��`z.g��J&�C������"@����tq����}s@�	!���E����2w�N����)e�6^����{��;�I��&��HC���j{��]���n����	�����?�#�m�.��B��C���4���q�?�'	�{w�l��H@�y�[cc�l~�.X�l���D�� )TQ��x�9�bY$�%�67�1�hsCGo1��~��C�^���2������q0b�O�����8�P��b����I�����!�r��>/u����pP��Y���?��u��"r��l�Lk`\�^�9���U�}���R��h�y�g��3��@uWM\E
�8-���Ln�J���z�����_?<}@��/�8�]b�P	'��'c����������q#���`�����-F�uE�N�!�I���;v����W���L^�o� ��k���f{��uy�]������|F�W������^�8�=6��]��-*9���:4�Gd��WkZF��q�aWp������x]�<���]�_�����
�����������$-��.vU����'���^��Z��-�O8D�A�o��A�!*�	_���
;��W��i�]���`]W��[�wX���ke%��">�����X([�n�7KPQ91�%a��Ea�K�GF(��#���?qT���
�0�����������7��qX�+�����G�-UT+���b���������n>�/�@!��gL2����K�iTvLFp"S��2bB5����@0��{��N�Z�O�&\kpb��������|��"����s��	U���<Z�jB�u}uY��i��dL
�uS�_�bH3�5-�y���w��KWA�^���Z�o�Vm��*��?7����!����d<6�N�V� ������HD�-����"�����*|�dR�O�\x����}(*���x��w"�W�Y���W:aF?]�b�-����[xQJ�46��D�s*l~Q�|{�mW3*�Wr��A;������ ��5"��s�}0p���^������_���':��
����6�y��E2e��,���*O]P&:��=�����m�Y�J�9���&��i$��J�<�����U�?������tJ|�1��=)@1��������Q����L��Jp�,w��6��sj�i���(H��~Dj�����E ������]��D�3�bBPa�$�s�|�7��Mc\��.:�6g�{���y~�!�I�c'=��x7�R��RB�^W���.�����4�p�$L��Cq_��3A]�j�&V���������N�3������
<�(P�b)/�_VJ�L�Kaw��8#YB�'3zg�PL�}���������+�dR�k	�\"�����X@�
])3V�M�Jq=�.1u��R�T�H
mA���@uv�����������H>l�{L���H^��7�m�~��;"������ux�&�J����\��Q��8��+���*�M�3yZ�V*��P-y(�I��&������2��f{��_�t%��R��R�g#����v���^)tA���OP����N��_���,��/x��eg `Z��Y 
�R�� ?^?�F��`TMq�q�;B)mC�}B��TFGM$���v!���1f��u�K��������|�����J����@7��T��A=p>�D)�����c�����>a�T1��O����X�o2�
�^���r2^�$������A����0�Qd�Dy]?$�����q�z3�C���c�kn~��2�
6WBN#��R4�M�����������/���z����[�p�E"eF���i�>H=��WG�?�>����{t9Bd���+��7��s�P ��W~�l����j:�3�r_n.L�^��9�������
��+�
���������k7+�D���Y�
�R�A	��S����ef������fy84�������� ��I��);\]�J�n�s� ���H��"
�5���{�� �gh����vF�����)�]XV��#���qh��
v�y��,����b=�kH�Y�%,}�4��� 4�<��LG���H�J������@}���>6��~��#"b��3m�����x�6	�ra�AT�E�]����_���z��[�����8Nf}�b�����i��l�T@UQJ]5h�I>��kFF�G�T��Y�k(��(��g����"*�m�b�bn����[�����&kW\�t�.|T�	�O���7����b!T��������%��y��H��'�!�z���$��r�����u�	y��gm�bZ(����2B�i*���%�z(��(%@���.�AGDpx1�d&{G?�_�D��a��4��<n
����m��z�U'��?�"_��g��x�9'��<�"�
����#�mh�����t_J%�AI4p`���%\�n���`{�+���K� �4e=0	f����
7�d��?Q!R�S���|h>������W�	��Q���_5]��`�zu)S1��($7lLM0C�8�*�����k�x�2�6��(�	��j,�g�>�=4y�t0��s���2{-��fL�����������"�V�2�L����n�����3�F��yizs�����;-7+Ba T���6�M_�RxBa09r�
�eD���������%��7��2�tL'���Tv�C�]�,sQLS��9�O��y�Z"���,QIn��������f�����>�I�`���R�q�L�EA?���I1����|�����|~�P�p_o����<r8�^\�������8���a�
0B��~[^�v7�?Q��(�`d��kH�&�r4�-B�M��]K�o������������z��P�l��F�,A���_�3��4n�������3�+��5{��Vyl��<1#���8"E�p�����[n�������r�;���6;7�F'����.�lRs�q����4�~���W��y���	�NSWf������K��&kM�q ��:-B9����.{��
��)������M�x�<�����vKu���Q)K9������c�a�+�g���'���rS�*p�H��V��������]�[m���(q���*6z	�����*MjB$�����u�B}�i���&|@���5��Y�h�9�7,yE3���R�F�C`)����������OG���@���� ���K|�h;!TE����d ��?�nj��A�y��	M�i��
�`��l���K�#�'���i��������Z�hl3H��p�:rDOoi|�m*`�va�S^w���eDe���I�R��r_�wL��"�
����q�b�;������3�X�<���������&$��P)h��������|���`�s\�7M��`!���\5qB.�1i��X9�&��_����.y�<�*gIi�%e�4�BV?��C���@�P.�������
*w�j�c�W�H�������j��TT���,���M�B�&o)�Y����G4uM
�/f��b�=�M�z����E�����=�R��c��(���Z(UJ����.���w����}7O��Ev��>y=����a�������XI@S=�1S�"��H��L/6p��9�~��hB_�#�g�ns�M\���u�H���lM��i�6b�����o_S����7(�^���],\�"7�2�H9����&F�<)�����|�����W��a����2c`�Le a�x��������Xd�
"a���������4�,����'%��J���%��VQ�>��~w\�7m
?����>O��K��w��ed�_�O��&�p���*D4+�b��@���[v'�|@��#��)���X��'W�J@>�D�g�L�*2�U�;�H�#l{���j&��h���`��_��"�[n��/���+9C��2=GG\	���(GSiC����r�����L�9�J����R��RU���"y)�h������0eL��I���JN����A~���cJ�]��R�R���r�������a%���Rrx�Nm��^58�*5Q�G$����JMMuj�<�(�}X��
1:/����b�>G��X��JAB/�1�s��bW����_O����O�_#�#���>L��3k ��T�R���C
[���y���<H[����Kpi�D�D6<�@�80�����.
�U�r$��[5�����d���Q0A�|{1��H�V,���LG��x�S$��*����r��}��f�?��z��nq���%:�</����!��"-h��~�y���e�����,v&��9O-,����HeR?���?4DZ��>q�~�x!����
!�����bH-���
�f}��?���uW��q��W����t��e�; ��X9��J�c�v��\>w
��6��K�5#k��]E�N3H@�������i�f4��'i��!�� �p����R&��&��e�������X�D�H������� y�s����y2�bq1�aO��|�e��K��`	t�H�o&�9c"%]�.jS��E�g}��O�}��o�c��=uAj����71k����&�{�j>h�� ��Nj����`�gC���}��`��>����9��R���"������7�Q�����P��������'s,A�����DMl�d4�.`���uu���~[��"�������F�y@�-��%���
�����������l�H�\ ��x@��<����C�m��2=�}�R����������.�����E����}�����~��4��%}R��X���g&H�6M,D�/�l)�e������=���s=m!eA
%�UXm"RF���U���hf��rF�������P/�U�s�u�����]���lT�\�*�@+Ol���td0O���cx�������:-7q��/u������8�������D��bau),���B����p��?C���3��W����2UFi�������R45}&�B�R�&X�w�������.���X�����,1_YM��B^�9�`fy��0r�+(.85X#0�B�"�OZ-�c���W�8F������u	��~���g�@Ecp�������U������}��Js��H��I��B�l�
���D��_4j��������4p��������G"�X>er�������>�
�O����� �����X���hS5��j���I�8�����\����m������l�L>�;�H��"gd��A���Tt��B��Xm�����8�H��k:9
JTg�$���d]@�y�-���#�d�L�H$R#fX�+^����hM}��5���?7���������T*v:�dk5��*��fy�/L1�{:eY��J6���n����\�� ���}�,��tV�^�������Qur7/�w?��g�����2J������O���r��it����&�t�h�/�������waN��=�	���L0��8F+�L��e��i�z
o[c��]��}�	2�� h�K���jSy.��.9���*tD���q<
�����$��N���E��������;6,:
x�k@G,�s��-����DC$�g���.�)��8M�?CP</f���N�E���_�8%
��@&s�u�y
�����gF�y~s�^\r?����7F�t�d��_n�$��n��������
*xY�g���!��%����C�nKa�GA��-�����Sy��������z�D������=G��3|^���.:���	rhDp��J���[t6�]�t/:5��;���_��]\C��I����o�u��m��\�
�WZ}*5��P���%=]yb�N�X���J)����S��L%�P��!�3x��0���<��7����%R_[T��J�<U�
�Bs|�Nr��J@P�zV
�������uW��������> �/���v�#��%uO������X��_?�&�`�����A�Ub1#_��[��;N�'�&�@Z�f
\�����@u�qOK�`��=_J��we�>�e�nD������,m�i�!o�O�L��y��`(~�d��"f�������g�x�t%
�"`�����WAD��2�ci4���n�oU�H���i�"�U:����{��J',��k�Q��r%�]�q���B��8n��l ��n�'��}4�M�y���7��g���<�����s�����Hc'.����n�4��|�a%���I�YY?�y�+PI�9:��&����*�8��0t`�� BE��;�������.�{����g����q��������W�++S�H���.�u.7���Xw���J14*����� ���J���-�,m.Q�U_����g&�/B�nR�f���B6�TX��38J�����}s������I#����m��V����[��xx!�_e�d�;��p(�����>�8`)z�3
|��!����)4/psc��M6��_�'��H&�ZJ����r�)/F�����������Wn=�4Sl)!�M�������#L	�\v�3�K_i������&#�&��,����(�K�4j�9+�w\m�?,F�����!�x�������m=�\~���l�6���a���h|Y'�Zi���UOPcDV��b��{�R��jp�)���������^*?/^�����9�J�H5�33���0J�w?_`O�bf��d�,D�,*h��t2PE����#������|��x�����o;]��0����>Ix��2�W��:E��X�d�u�FIT$q��vO\�}���]_>=�_�f�'C<Q�+�� �����$[GpSh�p<����2��v�]�+8|�097A���V��U�p'hsYh����B-ky���[�A�����eHL�t�f���=8��F�������
��AJ�����qUD���QK�,X����]��
���a���v�Y�,�#<+����`!,�(:�{k�����������'�/N�
U�����8��KA9�L���b��8��!����o�"ta^(��
�b�lZ�c?t��u��������
��K�;A��-Q���%���]�o����@��Jl�s�*1�h.� �Z%	�&bH`�bCT.]�~���r)R�S1��.�>�j>��k	#�G~d������r�?�.��+�� ����r9@�Q�-������H���Cw�`���5��W���g�j6q\E-��7�'�j���r�>Q1��WS1��F�\`�u���8�5�rYa�{8���j���j��_��@E{!!����Z;P)F�
|AH����=�������W��P���S�E���b��L�:Fe�_dd��9`	�{���y�VBw�9'V���0y���Y����s��%�C�m�rx9�67�7@�x�G��L��Fp(��
����>��xm�|O"10�E�1fk�R��kw��\�GR���'��uQ��
"j�������
W�y.��_Hi�q,��|��Q�:u�A'�n'f���_Z~]�<��M*�9]}?�����K�6L��@�t���T{Y�d1��8���+�C�7p<?���U�=}wMaL\���yV�����
��g��]f�^
��B=�9DhPY."��r��{5�1}���j���c�}=�n����I�((�����TaJ��OT�[�.*�
!���d'Q�l�x��_P�����B?�����i6��w�s�C���e�q3���[Z����X�c�M����E��h��$Z��s�����}�����I\�SN6������p|��V��a�����}R�\���PR���g�Vc�w��������~j��}����}?
���Q�N4Y?�p������SX�Q:��C�����=�r|	�[��M=HHC
�I����V��:L��	�cA�3�[h ��.����tj�A�tE�m�<��������U2:��P�3��G�J-paN�2T�)�����^x�
]]_�I�
Kw*K��u�L2��j�(D���e����m��~�#J~6A�����u%��������@�����;��az1Y�A��Yf�L�W�K��*��jC�>w�P��2g��5�=�����^L
{L�1*��<>�����\:���fx(��4[
J�.5�>-B~m;Cy�1��"M���n?���h�-F�2�i]E+�Qg4,�����Z/t8�V��KB�qm���Z#��`LB�i�]�3���I���}76~#���!&u����c�����%S7�^u��vW?���2a��x@����,�<g��@9���S|I��A���M0A���-!����k�8��hSi?�<��^�8c0��z��<�&XHzMl6
���c�J�=V�W�0��7�����u�x��%���v��:Ne�	��J^���.��l����������^����U��,��^N6k$e������l��C+}��,��X�HPC�JZ�K��������aF�}�E�C�a���
���S�K��5i}���8:�0oI�@��J���������
�f�(������p�qE�I���Z]�"~�`��si�Zo8�%�[������S}�w��bs��E)��$)�I�u$;�}�X�K{4M��;�o��6����-��&<\UT��Zu�
�-`Gd�/h����g_^vE4����"���W�l��qL\�@�2�:�� �r��zda���<#��/�����v���<�i��o&?�Fr�A�.!�f�t��(>����/PHz�B�J4=/+y��6r1���Em��'	OlS�?K�������&hE�*>�,*%V�\��.sA�eS�����D�e����Xz
����jy�X.}]�������cYm<l;�	�Y�����<���d#jIKC� U}�|7������Kg�v�?�������5&%�������m�Y%vCY.X�����r���&�t�b^�(g����+�q.��'�����v%8Y���	�C���D�H�3R����~�2Gt�w����q ��n �G���R��������>p:�@$z{��=`9AZ�5������AcD�K�d� *\s��C�����:�w�X�QT�o.�*\1Yyh�I��-Ub�wB����e6�
����;�c���G5$�o��.DzY���B"��a����i�Q����u�t����D����l=��42?	��\�o��p���o_�������=�-��L���}���AY���`dHs�@���'H-�f+���~�28N���'u��d�#B�z�s���o��J^���1�e:H��,�(�J�ZNfY���BK����6@d�I�*8(. '��_Z�a���A�t8����G�����m����K=V O�B�fw��N�����u�O6���1��T�B��"!����&���S��e.����h����O���-��q�� vZ�be���4T�a���q"J\��#G4? !c_���X�����u����~,�u�5s�.�����Z���mb��
?���j;���x��rI�_��T���R�T��/�n?���;���c�N��>
������U
0xI�w�f��A }��o��}5T0@;H��
J�	:g�N���p+LJA��B��s��X)�,�=da)��������:
Er� ]�Xa�.��3�9����pQ��o�]�i���J/w�A�~EZ�P�����c��!6��#��J���`�^��i��pv���Opc}y��c��ttT�[W]�\������O����P�[�$���K�Wd�i
I�vc���O�w�'�Tue�����-)a�����T|�5edj��Y���w	��r�,_���,����a��\6���	���h�6����\���]p�^��*���`FIU�*t��7��.�E.U��I�yh�=8�����\%��I��Q�R�
�z2��]'4J���F����0�_}J��?D���pG��SJ�a��]S�p���j�@�!�����$Q���/e�>eb�MT��!gP^����*N	�t	��Zedv��i���{��?���� #���p�����������r]s����
2�$;�L�0w�hK'��2�%xE�i�^��R[��r��g��K��X7�(���1��?���t��K�s���������2���X�������p�Nq�4�*��������0�W6U;����3�*���!���]�����������/KU^�'���.���%e��*�D�y�|�R�~N����k/��:>��$�V0��zY�g�l�������5���5'����O�Ar):�1v���?��$[4E4H�r�(dl=U
9��|�l.��S����B4����s:�
��������1��x��a87^G!����Y9�"��:np�X�9�I���d��m��q�>��,�xPrn�z������]�u.�4��7T�{W �����O�~��\
��FD.����F�;T�������dg:t��v/����v�.g��q��8F&Ul���>�sw(����Y����u��{%���]����Q\�
I�Fb�����4���s?��$O(Hpw���d��p��i18�Ko9�<��bp
/��z]���A��#�l���������
h������%�@����d"�J��@��-Sp9E8�7%�������Q����6sI�\�=c�Q=VA�x�a����������}��A!����u��Y�Ja�k��q�������^��@]���7�Sil	���&����qP�%��2X�^����<����F��x��Y�(n���u���
�f�������~U1^@�WK �*s����T1�}��;��q���o�����K	t�
iM��~�����%X��y��:�0`�Q%�w��=�d�u_�zlZ@p��9m:z3i�^�p��F!a_]�lW��A��VA0�j��F�g�{���g`Qh�%��B��C�fjB{�Ei_�I��{��������
��G�E��X� �Q�]�/���`�&T�9M	� �t@�<�������|\��k�gQh���]Pk��^0zQZ�I�X�P�������8��`���fN�L��!:{�^�c|������2�s���U�+GDL3���`S������<kK��S*���
�)Y.a~�x��M{w(���|�ik3��H�u,��t.�q�a�����`����Ly��^g��r�����$��������qT���l�mo��A�����]Cnr)5��+����H�b	�*a<!u!��Ze�Bcu�\
�Q4���I�X����sy�,��3�Lq��0c�`9,��l,9�$71�����p��q#���j��
�����f����(r!~tgyW�@@&������I�P��G`�c�N��:\�v�g9z��b�<@�r�&/�Zv�A�y��:.5�:m�����p=�ud��0{`�����2�NY����0�$Th�q��H/��nh���N��\�
����i��\=Yz��Bc�ZY���D����y��l!���!�OX��	�>Ty��_B���\.)���5�����$2@
K(�2���R���2��';��q�&u���<��S9���mW?�����������&mRsU�I�T(>��Z�B�& v"s��\o#U|�1�l#uO�R��m����s�P������hTBb?H���Br�j�\�dQ��K���1b��nT���v/?��"��o����z��b��0��!��Q~������1��K3<����S�?�<^�����iGHK����_B��s�G�\�� �|��2�`�n��w9�[�s��]���g�r{���<������\�j����i���l��Ef�Mk�O�������_���?V[X�&�'�������y�~��}w8MA��mA�D�g���\�N�~�g�^R$�1��Y#����>��$��������\���\.n*�}�
�k�U�$X����[RZ�|)�A������$���"+���3H��70�u�A'&\����e���'��N��b�G�L���@��#�w>�_dP%���~*KO/�4�Pr��[�J��h[��S��i.����AXs���D��Z�	���H��&zCL.'xj��r:���P����^����+(u�@�YR��+�ml�B!%���2�R����o��u��^���������~�R����	��@����m��G�U�Rw^*I���,��M�|�8j�� �"�����,�������'M�.��������l����Pn���������3�L�M��qS��7t��[�fP�����:6�rZ7�Pw��&A*.	����/����8��LU�L(���g��-���q{|)���y����	�*��)y*'FA($_������]�PEa��(i����C^/��,J������u�����oP�+�\a�%h�}���)�sT[m���j��;]�\��#U��zHr�G�C�"��g������O������;����XL�^r�������zG��_�c�����'{v�.���Z�=<�(-E���>]���v6X�_�����f�WU�e������p�Xi/���hqO���r�����q��K���)(:V�}���	19���yaI��/:;���	��������������x6o����� �<=�v�!�����;{�z6n*$��EG�(�R�0�}�=A::�>!t4���o��BJ�zJ�������������l7~s�a�TG����Y
\�w+��x0�D�EFi�#<�����$�����K��$��0(�����,EQ���@�y��K2_����������@.T��l���.u�k��T�4N�bp�a_LH�����f��&T���p���=ya92q�v���}�>9�0�������>%��}TH����M�`N������w6�����M�G�����u����(��@���G���=��%�3�-nKA&��p�U\/��qBV�B��2�k@�����`���n�:��������In�|%�?O�B�d����I��,��	e���E�C�A��m��d�Ds�SIp6����]��.4�f��s<�������������d���zb[r�-�8ZC�km���g�gW�e���|;B��+:
���<5��d`Q��r�H@��c�����&��S}���u��K[w�Cd��
��	�Q�|����8���TX5- o���U����{2�PK<-��i���1B��F�|�D�K�8w/�����h�:(����K�r�����B��fD9%�Ou?�t	IIX���P�0�I:d�����v���|�B�x�c|��"��|1���"9���s�b�����M���*�53_���?��6����
|�8.�"l2'����u�r��vgZ�=H�L<[��kI��K�����%������D|}�>�����#�
�
��}�i9���k3��n�p�M�yA�<�/�<C}������Y������t
C���$LZul�2�y/5�0�&��Zs�Uf�y�%-��|((���Lf~yK�J�p��B�u�����8������u`�A�Y|����-bJS(�T�r!3�z�u�����������2�zS�����<���X������K���s>���������lR>j�k,������O?�J����C�)s����A�[��yN��{��F�U�S55<��U�o�u�u?���>��BE�G�@c�v\��k�����v�z�������S*�a^�%wW0�B�8W�9�������'�oi��_�O8R ������>��P�wu]�T�g0��"��k��M��j)$r$!M�������z�0���X�	����DO�@�-X�F����g��\%L��I���]�n
�,��fs�2J>S	,�i�\�bz�
{ �������}����v�&��~Sqo[6)-�'��@Iq��4�������_�c�}�<�v��v�O�L��q�����#J D�JwxkR���+z�+�8y��X@U���3�v���!~�fc;�w'!/�Pu����2���!�x����(l�1��] ��������t���aH.�~�a%�8��)1-���ER
),��[�;��%��&O*�������,����7�����C�q���o�s��^����}���Q��(����������+0l3	�+�dI4��bE��D�L6��q�|���*w3�'@����3(�"�t�RX�3�!C�\@p6,�`�-��QAq�}�K(^������[������6�)VF�2#I��V��X6�k�n���bZ�!l�l�5r�,�\������|��)R�\o*��7���.?l�H���$�i�4~���������?$&�-���'G�R �q@Ab*����e�d�y� I���Z	���i��C���N����6tv�a�0�f*�?�`�T0��]���f�.�����t������ ��0W6����>�k����-%l�����P��@����^�U;����#��`�1�4�rQ�3#��s��
����(��������n��D���5h8���M���s�o�-�6�M�(y�����m��+I�n.z� v�MM������"H�1�Dq@�$�����[[u���(���10�X��J���8�d.O������@
;8��E��Qr*$��O����N�]�s��t��e-:�+t;�^IpY�g�fF��q�Nj�������w�3�_�-�}�����,d��Y��P���Z��Xq$OM�gy�pK�'�2���w���L�76c6X�w��<���Gr�����@O}j���;�����x�4w
�F�'@�.	�(q���l�/`��W��\�h��7:������%��=�*��ye��t\����WR��P3��hs�L7�����6�23���q�#�_
�>.Uw�ZNB��N	4N������i����������q$�R;��,?�������9���:�	���J��,VP�
��T�\&J/�\;��x�@1N����I*�X�.�kI!j=q�����{=����z��4
i�
{4������8�P�*F�������U.wq����vc�t�i����@S�C�O%C�K`��l�e��������w3a�2����B�B�m�b��A@���
X�o��M�>�-.)��p���O���]{�G.��8����q&^�A�>*q�$�.@��z5��S�s�97�����)L6Y�.I����I�-X��"Mo�m�����P=go�
�p&G�c����(�����8�p�����E�����c	qZh^PLw�Rgs���o��k��R��m�j�]��:5����
�����t��3�����\��W�wt�{�d8cl.n~�k���_PJ����g�g~�����aQ�LBS�'G]��TB��E�N�]�u^���o*���t�+�c{��[�����_����/�1�_��5~�������{�����_pmQ��q��.�N;&��.����#�����/UWO�J���@���,����r$�K�>�(����H�P=���8����?r��9���$�t������&s��r���W��
��L���b��F�z�?����,/�����i���"f�@���.�5�D�#qx5y.y���������G	���v����,�e�DW,T�Al�����/!O#���-$3K�}(������S;�z�x"����YLl��$�V&K
���������0�u�HT�-��.��(/^�6����!Ji�F�Z��W ��2��/0�/��?�|Q�5���`>�p�C\m�K�Y��i���}��e�{��#ngLY�E��L�O];���zPf&S��o���e7NB�!K��.�a+DJH@��Oz���@�_�'����Cm�@�]	 �
�����J�>y������2�,&��y��EVO���@N���qI������O���k:�j���K��$Xy��*�jT.�7����q����W���St��4���E��j���
<B����?��a]���!3y����IS��&{����A��P��{3�����Aw^��4.�Xa��A�}o��
����~���d��L���0n���j��z��pJ�����$���������z���<����?������xn2�4h���;�4p7�[�'w|���j~_�@��2�F�U��/�9	�u����Y)�K
�������_t;�E���Y�Jo�9��XXgJl?%��~��~�S����������Zm�����8��E�=������K�m�zk%�O�@6tx�j��IP�X��K��h��xab"���t��0X��!E9>�7��%lo��y��B\����������n�;�������=����_�fIJ<#�oV��2K�}�������7�w8�\�S�-���$rS��FJ":��:�T��h�t���m|�.$:�-�����I�YX���������m���I$)-��2�dr3�N�S�3�������H���b��6��C���\g�����G�X���m�~��������� �'�e@�|��,�����$��.�E�5H;��7��`2����zW�v�O����<I\:��n1�z��}�� �H�&�v����c�mN��������RlZ�����p��BY�(�q���Z_�#\��Zo6�,�{���f�r�UF>���y���7eT���%}c�0�e����M�3�3�oL�^q5����t�4�0�@�@Z�(3�a�+�3��R�����u�;��Tr~�!����>Ju	|�v��*���1!����}��g���������M�0�4K�;���O��xZ�P��]Y$��P��,,T��f����%���Q(X���4~K�\��[�[SPE'e��� O�:na|	;��}��N�0s1
�#�C��<�d`{�t��j�M.�h1�\�H4������B4V�����[�b#�� \h���>�N����}h@��<����s���xral��������>� �UJ�/M��n)���"����s��X��\b&��`A���S��iJ>��eP����W5�SIi����O���-
����!SU��^�������]up?���J��T����E
>������/���WuG�x��p�;]�����-z��\�������++CXo���'V����5?�H�������r&~G�8c�y��bC�� j����rd��v_��O����q�5�]�9F�16��4�99���KZ oi�C�y%M.)��h\������AZ
SO�0b��A.��c�uT6�m]�����v�w�Z�k���i��Wj��lT�!��w��������S�z"��$lMt09=B*FK�z����$��q��|:�]]m���r���*^�$���\��p_�BF�I�<(���r��X��n
q. b{��U�e7)-|v��r���������k��g@9l�^��A�g�0��g�s�+�\#]������>�O�����������{��f�S�x����j�"��pH�H�����t���;�U���R��P�h��E�(��
�����@e����<�{�_!�!���5A����>XR�����H.P�c�
�����X�����%/8A��Ce&���}>����_��~�8D	o95CH�SO��r��!��������KY�]����L�	��_N���s@}��+g6�r������]z9 c�+76�����aI*G������8��tK:�[
�-qE����3��Q��<W�=]��w�y��;p�)��S
g	�;��(�z�$E.�)1>r9&X���A���Zu��s���I	A	�bk�N�Y����Mz25��-���)D��Z�h��N�=�$��FO���B�p���dB��!�^�5��\�����b�[Y�In���E�Yh�c�)�W��r+�����<l+�<�����U�&,�hk��hX�����������}�n�nS�����7�������,T�l��o
���S7��*
��v��x��?�>���Y��Cyj�M�<���3c��*����e;�(�^rv�L�`����S��x=�AN���+dc)`��r�����+EO���x������J��\z���*+r)�B���t����:���c���C������vB�.���&������+��zz����E��T�s>f��b��)�)$��]������aGX�����4h��=��rY��q��D��@Y`�1Hq��uW6-���R�n��w��`�:O�U�"-�Y�'W�
�� �����M�4`�� w�����6���A���8����4���������B�CE�}}S��	������K��5d��gYi������s�*(;��'�^ �Z#I��S�����o��Z�}�#4K��1�;_P�->2��@�	9�`T~m�oc����5�������BZ����`&���CJpIAk�Iv*���R� �qh�Uw�����?���w)D$N�G��|NnF�l�����&x$b���7�&�p���,�M�4���2�WV*��SY��2_��� $�}& 5J�BT�6�c2�$RZ��f�h�
�T�x[�P�fT�}u|.��t�z�R��/��8x����Pa|k������,w�[�GK6���Dc�nF�T6�M��`���C�
y��1���-�{��
��q�����_N`zX���k�����u7��VF;(/��^�	N��/:piP+��&����[���y��j�����k�0��ykk�b~��	@�������Bv�bTAX��\D��u\e�D��Aw9F`���u�-���
~<�o<h���l"��d��Pg?g.����+�WJl��>g��G�-v�d���X.��9��X� &�����XR�Ml��$�Dbs�������	��,w��4���bASp$�����\����i[we�@!	>�Z�@v`�u�Uv�Z�ieW$��7�qO�K<���($A���:���b�d���$��*�p<'n/�5����E�������z�6��_���B���Gkyx���X,����q]&Y:��f$��bu*��]*�uK�dF����x�0�C--�n��3��:L!-d�n�2�bH��������������iwh7��k�]�x�&���}5j�!�+Vs<��)���r�E��#6X&�%�8=���2Y�\���}���`K�_YR!�I
�cU���� N�K�' NZ���3Y�nhL��"n�rn	�����T����w�m�`�������tt��Cu_ZTe��P��q������$Ja��u�u�3�F{����UdMm���.�����Y�y�_$�����H�"iI���j���(1��!-�)�'�}m�/��s��=��j������,8y�B�������i46�����6���\	m��D#��uWr��c�r�2y����@6������?�4�'�[n��B�U/�(�,�J:���}����@)��������6���HR1���"s�����pQ
Q�z����}���f�uJ���*etv���m�U7��h����GX vc�����6�Gb�[�sWlId&JH���q���=(��CS����m����{�e�G�������.
�[�%�s���/����_����+DV���-���J��4�������Q�X��r��a��P�Q��|�}�����s�_�'G�^M��"������dd��������"����������O���M��~>}j��d���w�"��u�IVuYAxn�Nf����J��I9�&�L�j�"��$�STV �l�U�)/���z�C���-o�n���������Hr�����)��va���}����C�V�j�t�����%���m���x�������_��}��C���mw>z[^��

F�Bc}��7�s��iu��$r+��&4K�i�~4kl���=�gk���MET�����sz� �y]��l3��#�B�(�3Y�8&!-T��I��W�������@�z	G*d�PR�����Qf���p}R9�4-����t����8"F'ln��:UH��.����?yx�����a}i�#����.id����j�'��l1f���j���_���'xV�;'�(�IDh�o���SxN�I$�F'�
~���n��i��N�cf,�F^C�.���#K9��������7�����;Ek�e�5�,�+��Pw;�@�|CO�������aBA��,2�4��~^�>���JpU)�cb#C���?�]{t������!���U��+b���F��,��&y�$�$O7V�����������N4��n��ff1���&�#���VT�Ce?��<�K��m_4��m4��#�����]=����D�M�6`v6/f�ltI���<$���i!{�$A������"�����9=��;mt�O��9�\z��"]�{ZR��*���L%C��-���������X�S�R��vx��u������El/��LT��t9��2Hkn�d���u�+Kn�����)i�?Nk��t�O'd8b�
N�����B��f���K�����m1w�f��<��#�����r`�����N��o��o��M'����I<'���]�|:������8�A�@��j$4M��-��W�<*���f-
���}��z��-��'�������|C�9�E�����p��&��2h����������mp��k����+��V_Kw�0��#�B�-}����_��o�A�9�{/�ty�1�
*�Q.���IHB�j�I�
���HPnj��!I����4������
�bC@�m1������[��_a���@k[��Ar;���H���b����O'q����(v/!g��j�h��	��&H�Z�R�h��Rn�TV��\���7�s���,+����_"�^|IJ�e=�a��t��_�:tX���B�����6P���h��(Y�#M�w��rM���
�>����|�d�)*��_�g���'����`yY���qf�����n�8���||c��i�~����_�������Y�g_G���\jTm�����=/
 �kr��(t�D���!�������=-���SKx(F��#7����h�u%���C	]��}����>�O����-���(�����]�y��b`,���}���l�c���l���AZ�dnR����F���}
��7�Zch*���K�+��l$-e��z�/c��w�+��sHI�g�\�R0.FF	J�����Ll'��k_����A�����}6�+�%j������������n��b���(C<�z�"{4��0a4w�I�$�}���UZ���kN����7�K��;��>I)	��L,]I�D��r�9+����m.^������~�}r����fh���'s�����=6o���
���I���������ld)�����D�3��F8��Q���m5n*���S��bo���Z���3��W7�n���#I�a�P��g�m��n�[
�����X��2���uR�!��Dq�w��(e�14��]�K���`�L}[�-v=c���Y�!������};��m/WV��Rp�K����,�^#��Z���������z��t�|��P�O��~SF��d�v#&A�P�H)����?_�m�u�?�M�������S���<Af����Z��/K�L��D����?��v�[�����?�vA���� ��h��	�>+o/-��w�����K�r#�xe����\�6�r�7��� Fv&"�l�s�:0$�d�����\1��[

�����������D��yT�����)�N�9R��>�f}[��kp���%�,��������w�4M2�����_BY)���)}�K8�}���s�sw~�Q�����&p�.��S�������8D�,y��g��O*+��V���&5F�l����;o���A���K>7J�zb+�bph��Y������k�S��FSv�!\S���R�o%��p��i���W�G����	��|�N�������"��D�s��|���Z`�d�R�b��=07	,��~��[6\���T�[Xc <��0,(Y�6S�,
�:�0
swQ�b3)X�%�p���/���,�;���N�\y��J2���Im������_�s��	VjB����u����1w�����|�]{��=�
HW�.
f��_I#�2<=�k��E�����rR3��?��\�����d�%\T��|��{��%��\�?������W��)�m��M��U#�e����}�xv�)������h2����T�u��b5�����Gp�O�P���h��/�����wd�8�ngV�Zk.��`q�}�5a=Y@Kt�H�T�}� !^E����-+c*�����b|���������bN�Q���<x��IJ4R��� �VA�a��/1��.��+)	�!wF�;P���_��J/}��N'�>��,�J�V'�Wq�/R>14����S?y������l�ET��Hh+o�$dZqN|�<�����W>���k��������i�D��t+�/�AZ���j	Y�,�����,h��=p�u��R�R{F���t9\��a1�����K<�o�4}�P���YrJ���;i?��������� 'w�z��.1j;IN?�A�7%1�G_��\��������,cu���$f��
�
~�K� n:-�Gz�w����u���R^�%F�>u�D�d�_a��A���FV�j����bE��
6���frs��g�"����b:�{�2��V|��V[=
[)�t�S�%z|���{��d�v'��2�)����#2�Xb�E��t��e+�M��H�����8X�#�������Y�D<V�Y�K�����TE��26<��_�m�ZK�_?r��Q_���K����1�\���R��A7����i��=������Y�����$9�j<��N�dn��T :U��N!��;o�f{�~l�h�W$'3*��Q
���dE�zSw�g~���G@c}7q~ �����rD74��Nd���9N�����+��@^�(�8���h��uM�B����h��$���h$rD9)�]hZ�~�����"K_=�p�9�W�n.��4��c�If7���*�����
���U��D�P�)�i���Dp~���0y?F>'x��BF"��R��0������h��7hrI1
8n��e�,����&�EJ����������R7*�ARGl����������p��{��kv�/�y8���^�i��
����]�0
+I�pQL� ���/[������_�������W�O���bj
Oz�B�K�h��(6kW[	�Zc�����`h_���v�c~���Hx���f���8��d"�+�C2-��m�w<��a[�m���0$�g�Yoi����o�(��$l	iu�������F���IP�{Uw����R��������*%�������AR�c*�q{4�J�	�Wl���kNRSend�~O)nCNg�������\�������?�}���;���4J#���<�����G����I�yN#ii�n��\ExD���-*�4�p�x)����!8`\Y��������-��O�j���;��Yn�f�FC2:��]��9�t�M�:�@��F6��I|�,�|��y?Y�~�����7"��Q9'k��!8
��q���!��������t���^��{�s�L�G�aY!W���$6�W��q/��aD�g*wO�:xe�C0�%��=��fh�g�Yys>^�C������1���<.�\E�$���
�Xv�Rl��	��WZqA�0��\�x1�hi9
n;D�4����[_��Lv������P��Mo�������{og���[���|w�+2��k����i��d�:]�t���zw�L����+�vNB]UEl��#�-�������%`s���bX����-��+�P�5ws�����h�������QJ��6�����<mzsQ����j#�I����1��������(FY��)� HcX>��v�M)� ��`R���?q��Y��(1&�jG���(0[a>>��O?�v�}����k!���w���FG��5�W�����������#�����u�|�]���Nb��H�$�;U��QDn-�K�����u�H�<z���j��\9,6��j���~�`"?�#����)��5�v�R���Tk�vN|�Nz*�l^�\��}��x���5�B$����<	/{�T��u����|6.�E
�����Y�n&���]q�nLV��=�I��h����-�c�f����
&��D�7�VRL����#OEq�M��V��z�v����$a(�����
I0F�K��7/��4����������+2�g�h�{f�#������$�Y]$	I���%�q+�0�5����~0I6z��
�����������K>�r��p��
����#IaJI4*{=�U�����V�NG���J�����;��|�n�������04m��\�Q#h\m�����F���s��
P�����q��p�WS��x��bJ���L�������f��^x<�Y���{����� ����%�tE�D��S3���-�Q��D5�C~���7��d�}��T'�����X��|Eg��Ni���;A�w.l�l)�2C����sw|���o|���C�uV)���&V� ]�"����{M�5?Rb#X)|�W���,x���s�����HVT3$���&�Q��7��0���\U�,�x4�KM�-���$6/�O�D&i�{$��;�S;�
�/^��&�;>g�����l6S9n��Jg����� ,q�z��	��<2*�1I��_�m�%�b�p���	 �8|���a���2jT�`��tu7n�D�h:=�7���f�oj=G��y�85�B.��������!eMa>7"�5R��~I0�#��a���R���!Qd^*�D�?k���!T/�tAv���wwi�����x�F�]�mg)�1�^��k��"�������f$e�w�h,�����u4g�A�{E)'���LUdv��q@�G�%�#��KM�����|��2w����d`_���XwHT�J9���
}p�:��mg�v������.
���@oO#��[���zL���������N����K������g�P>~�6��r��������:4�N�}��D5W�vB7o����S-�"tS���_?�O����jH�p=��,[|�*�#��S��)��4���6�55�"��z������PE�^�<��t�%�J\x�2�
�i���������dc�u���"q�K�N2�J\AR�$����`��y�lIY@���2%H Iq��U����c�=��:�!������F�,_��B.&��`nar��2��5��s����P� �������������rE:��~��`u�Qp]q�<c|Cu)]��7�����J���+�������CN�`4c��p:j�<
HC��kN�c�����S�UD@��=����7��JP�*��������#�h�K,e(���1��B�&��"��~_�������q�7y��`���ZD��RZp$.����l���������x_���`��l���h��L���R��4!�!�k�� �����]���r=�OMWo=���S����4�md�����������e���5���7XR��J2����`����w(*'���b��\�����4"��5��%j�%���&-�>?1������m�����1���0�$/`��>]��C�Q�N��q��\��sy���k��)����t�IpsqtJ�
��5�\x��X���R�R�B�}�Thg*3ev_ �:��B$MS�)�;<s�|��j:�Y���C|�J��R��p������[J�0��K���}Dy�o�h|D��.,����:��!�Lb�Rg�����j�p:���Z��4-���f���1	�I2�����������j�rSu�]oM.����J�����
�`PE��~��A�U�:��qBs������j�S�j�^��L���K3�����VK#�\�k4�!�`�t����H��U�bB���n���~��r)�?���tU��A����]��kJ��-yl-��Q9$Y�- ���7���S
1�u�^��{����R_z)�M�x�(2��}��,Sn���^-���l7w�j��S�t�"W�����qdV�5��fp�����/�F�\]��O}�C!�I�A;Sa=o�r���L�*;v��f�=������W��N��\����j/H!q�$A�]�s��_��U��*��	���FBS��F	���d�����.L:�MQ�(�v�k�!�g���]�l�,�yrE�N'��r>^����+�(�����\����\��7��yY�N��A?�(4�*&�{����iP�����aS��)����
���T�c��� �A|�B�}o��k��S��N�'K6�\IsR������2^0�;ap���|����������(R���{u��&��\@�M]������=���s�@,4Q��PG��kC�l�A�&L.P�4��v���|�o2����/D[Q��t���������v=��-��Y����9���YF{�^���[+�?�����G�O�0�T5���f�r��R�-�MC.(��*\'�KS9]\f�#����RH�]	
�l"�6���������+#e0�D���2-,��U��NI����/x3��XJ��^l	���j$�K�����p�����������4����C���5������G�uW�Wm������n]���<<FP�\@�lE���@��z��������L$,�$��-R�CQ6� d����O~��SH	�����)����=~c��A-�>�������g�?�����n��@��m�����CW@�sf��).������c����p��Y q(Q���� }�;�>x��g�e��8�����r;S�_���]��e�$�9?|�o��gfc���9%c�@�JJ�.B.P�1�R",i�%��-��7P�K�����mRf%G��[�������W�����.HA�	Q�{e�3!(jU6�W�E�e�P�}	�S�!d��lFC`�S>��>39�<,;�MM�m:8.q�;���n����o9cd����X�6]s�X��%��l�D(�����^���\?
fH�g��l�$��Gk��[�}�����!4]\�k��d=�#��L"����%�/����E`�����IF�Nly.��������'m�=������-�?�ue2=$V�?X�GI�Mo(R�,[��C�4�����;A����
cT�M����U��S�=/�%.�V�;1q�~��z�����u����p>�p�����t<��{(0��HPYi�b:�j`�7��su���5NA�����������G��|���}��������IQ�:��!Q&J���\��3�$E��$�D�(-,�*1!�0�B��}Y��!Z�+�p���������eQ(^p�n����g���{6���6L���VK�5R���VU��[��Q6�V���n��AwDR_	-�����h�{�����<�fS�]��U�����~��l��?��	fTl�
"���4������0���l�ez<��+M!I�BA8d�d3|s��+���_��4R��!�v�r-:��8>"u��f��'@PmMr�]���/</�JY]��{����~��8U����m��e�:�E 64�: ��u�-�����@�R�\����y�	��!�I�R2#��Aw��J'WC��a�zz��F&�B��I.{��	0cpS�p8u���\�� ,J��$
-�K�'�2o�8������o@l���*��`���v�����������7��I9��s�����w���QW|��}���I�i�S�y���� �p����\hn�����(K���2���0a��h��w�p�$����Z�3�����B�n�����0�(��q�t�xR��� c!�1&�O��5��_e3W����8���<�T����������3$p�Mw��w��mk�z��6�]�������Y�q
7%���g�����H�`k���fP��o�X�Mi���O/��NS����0U��=��e��_w��2=�N�=���Guv���?��f[&4L�w#Pdrx�x!5������4���E;���xKQ���HQ����3*#G$�����m\�����&�H��;���7����yve��2�L�������R��2�<;��KJ�*�Zm�������LO%�F7dT�!i
����+Es�!�TWQ(2��4$RWLt9I��b^��K\Q	0Q6r����&m����.�*!��Zr0������B�iy�7_�uy�%�$�b�)����Y��)ks�#2:��e�.�p�~��h
��h���[�3�5S�Q�Z�3������P��\�d��]��81��C
Fr�����C hC@U6����s����#I����	������w�` O����\��b����x�(�]������~�*U����*���7�9���S�b�,��E�
f�"�`H�,P�h��i�_��&�r�<�����`AR�0�.�������2%=�����$�������JX����u,7��nm���t�7��t��f��m��g[������L��'������2����tW?Vm���$R6���<D�H�VKB�/
����#A
`����c��o@�������������+��X����3�(
F��K��Yo���d����cuq$H�
����J�@M�i��k��!�I)�\L����O?�arx*��D���#p9`��V�
��%D����8���e��>z���>����;�i�]�[ui�V�r:P�
�h/��`�������G����?
jg��3��pWw�H����r��Kl����X���]�q�k�6F��,#)�p��*�
� ��	kV����]�����w������Iz�<���"�
	`,jD�Q���-99��������]�^��2�.%��g�K������2�^I�}K3 &@j��]����j��v>`�A(���Y��Lc��`i�����rl�n?���X���e?��r�����;qA�w��#y�m�X��I�����a4�n����FZ;�����O[*"�09�|���WK����M����t�rW���m4���O�W��M&P
!��|UnV��2�Nh�f�HHT
���
��K��<��
��~.�u�����m;(D���DQ4(��z��D�Hc�	4���!��,�!�	yi'6�����SH��6H���W[0�8�����������p���d��'X3�Y%p����m�����8��A��d���[��X�$�x����}�T.�p���m���Q(��TC�&�������n�Pi����f���vx|S?6����>u�8�q2	�r�=8��D6��=d�L5v��	J�]������Cw�x�0�E�,%orR��;'�'�pz�'�G����2)t3������{\V�4� C
l`:��F����+��k��P��A��D�r\R�f��O�����v�\���]�fDy!�A�����^����x%�0��r�}8��:�)2���d�ntW$��v�����U�������������=ilz*��,:"r����u�2���6n�����h�shOS>z�aHO�=[��8��0�'h@�^�6&���&���8&0��Ev?��x���L y�l�>5l�w�^Nn=�kn��(���������������O����@���#�xp)������S�y��R^�������)a"�I����=\��T�^Q�K�~2���N��Fx��bgl	��
�^�l�96�6^w�if7P�l���hOGX>9zH2�1t�X����"-�Tw���-�-�t��N*6I��@�D�e/��r���YWFM���kY�����T�m[��~��e�$/��u�����X�5�d(�[��x@���/�~Ya��4,�}6!�R�y��($rD��<M6�R�=�������V�Jiq/�l!�YV��8���������������ECpy�����q��+\N���e28���m�f����A7i��M��
?#�K'��w0,ySr�Y��+����/�s���z��N�����U7����F5�H��O�O�s�
����:
�_�-��k�|��q��u�����7AA?RA=?�].c����d&�J�D~�k�v{h���I�����f�0��=J�9F��A6A�L��J����c�.�fW���������W3��M���/8[�z#����a���g�������l_�,�I
�3���q�{�����R�dsu�u}�y���/�
�^Qn'p�.e�	!����M.����@�K/����(*f���@!����^����������tv&�Ls����1���F�����%�N��PB���NL��D �re�o�J=,���gn_X�"��a+T %���a'����H/���[9�y�g�AY�^\�c���fP��muz���S�|��})���I ����$
M����t��W�����.5k�!i�T�!z���Ba�:\�>�����D�;d�5�N`�"��\������Q	�����=�	x0��:2����%��� �C�@����,U(����x.����>]�t���WO"��)�'q�i���r�>4����;M����	�y���n���d�������>�Nt��������6�������4Pl�� �ls&����Bj{���@��;[w[n�=���5�$��t��aH$N*�EU����+S��@
Z�PW���7��%��W}�[��GKa�1��
it�$_�l���O��+�����:�����d!��E�Q>��-��I�;��"�,��[�]��U�b��]�ir�u�.%��y.��s�MZ�g���/�9�
*�dPgf3*�%�Aeos�xr��{^��m�,z]��G>3�3��qA��IZ��`@|��Cp�@g�K��7�e�����[��+�(xOrojti��0m�����V@d�5��R�_;����������B��z�^�lf��}8�V���.�{Alj���A�t��b������������i����R/�n�%h�	��q�TCBH���[��Zy�,��XF�w<���=��D��&�<��q�2=��J�^|qV����%���@��n�O��=A�j�l��o����h�}���	��b��L�)����oo�]��|�_��:�R�d|�������#�O�W|�>��34�z}>��������&E��*���_�d�a�
����'�~z��<
�hF��y��Q��cD�M%xLh��GY�Q�/����i��K�v�R��2���)��0�ZB&�w�������n�>�������F�1s�`�'�����5i�G'g�?�a���i���ed�/�jZ�S�{��4�hHeH�E���
YzB��%��(��VyXwU?���Z;�	���=���}	�o5W3B7��n<#���`=q��+#syz.g�L��?=��R S���D���Bd4��}�g��W�M!�^�33y�]������G;�Ho~����u{8���?������E�j�s���	����^�/�B��``�����f���4�j�w�4������oS8;EJU���d6�q[����$rp��@���j������4��@	#�@��������z����	e�dDD��}�h!��%�>3 �8oM;Q���#�-�Z�9{#���+��f�,l(.����,�1	�]:��`W�+"W&]���g�<��������#=���3�����b��"yq(����8��7�A��{*���7/j�!�W���T>��-~{$/4��EF{>foOY�o��=��=x��Hk��n�}?d.� �0���Ad
����7ht(d&����[�������- n���������(5��������b.�P&��g��	�81��������u�J%!_n=<4�G!_Q�J��[B�	o2�KG�5������|�{����� ��Fg8�����~����f�)����/��?s��N7=��g�v�FE��$�S#W��cx�Vt�1���x���<����5*�y�}��G� �' M�9]����dj�F5�pqJ���CGB�f�e U�Ku��.�O	w��1���_@�����T.����J8r�p�����4 s,�P���_����m�J�?������Q��U��ke��}AaH��IP (k����dcN����b�$vI���s�.�1@~���Po�]��nw���7XB�F'���F1��IkI4����}O��[��i��%�rv�..���������@�[�7f�"��Wj<���D�B�h8�Qm��I�?���$���*����)��q))�d���L������-��wS�b�VQ����T���N�5M�K���D��#��2EcL�*I<�6
Y���$��rs� ������i�:�����pG�-�{�����}���(��$|$���3�E5S�F�: �K�n���E�;4���AoY7e��O�16�y��-��n*Rf2��l1��	�*��w�w?��/��VfP/��I����	zKUI�v�Y���4���=��n|�.����������Q����u�7���Bf�u��Gl��ri�>����?K���A�U�����cT��2a�G�#F�2G,���|��) �N�*�o�u��lP�#d��^z#����g>C���mj��=�c9W)��#�� ���y��Cs%�	L��&��V�3=�_��<���~m�C�f��4_��^E[��Nc����g�����Bfl4>5�O�)����G�� �C	[)�?4��W��2V�1��)�_�"q'���YmBW9���m,Mw/��y�m����e�0g.���[O���=��
�$W^�k���=
��`���b��l�
��8��9��bH�SqQiLdL`t�
��4z��+9�5�-Sm�d�9�&!���1\P���|n����9tBh���d���b�F�^��
��c��F���z��s��
[��7��r���{��R�G��C���n1�:����Jz)-H��*�C�v�r<~Y�����B>�\
��y��Ta*c+�L��I�>����O��r��-*^/;/@is=Gao�q.� ��epx����������r�z���R���|�JP�1���0"r�
�Y�W�|�eyYDf
,�J������iXv��J$���^?u�����~���'��xS3���'����8��S�dj�=>F�;������<)�8'g$1�s"�7��t^��-x�"�����#z,5���k��LJ�����1��s�5��
�A��b�\{���6AdBME��"�rW��9���P
���-�>���M����������c�C������b�P2��[">*v_������~�7��@���w���Om�)O�P��>��\�,��b`1�_�����O��0�~hv;�<t�=�z��`w�[5?�u��t�&S��]��Li� ��N <����zt���s�M!������hA���a���_��������+�*�h��Z-���B�&@�N
���u�?6=J%�>�������x:�� d�[��(�����E1���8��]�M��rR�~`�BE�2nF���_%�%;]l��?��L��~�[���	�b�i����i�lh�XKZ
��{<���2�����@�]��;�q����(:`7����G���WVnZ��������M�7��S��
����P��RR}�H��<w)����HH�D��Q�i�����ES��|uW��e�H���<��Ud�3� �R���]@v5�W��RL'�)�|�F`�Dv�/(�����F�X���NC����?p�m�5�����>�!f�f�$����|�����P.��)N&�sVy_����Q�������]��6Z�[n��I��D�VVV�Vcr(������\
�.9jf�MQ �*M-�.-�r���u�����cPu�fY�Bb�B�f"�F'�ZU�xR��y������,.��_q��4j1����[I��h�Y��M��^B��������3�G�hvg8Y����8c#�A��8G_dZ���P����_U
�_f�x����r�t����� v�;(���������$�>S�G%$��];_�
��xn���^j��`���{R�t� %�x�s d�tm������
x�cw+2XW�G�e6j[yh+Y%I	�pM1r]�t=��_���������,YV>SQ���\z��pB��T�E�H�>�F���7��nq�^?A������e(������,@o��!N59��D1N����D��~�:��QO�g��q�/����1�X��"6�����iDl:�K������������=-�C�d�5�<|r:7�vx����X8���r
9x4�2�RS(���V����]R2�U����+3��!lzr���]���4������������lO�]����t�����]�����U��T%�5(�!/��7��Q�p���SN��/m���:I��'x��:;��,H���]$�0����k����>O��*7=(H�}��3�4���-�n��pG�����A�B���5���f��K�A�����S��L��$��#��j�o�j6;��pc�h�����T�7")���M������H���T��z��g��X��V��AN[�YYI|��h���])-����$.��;��t���Cu-������$�KrR�>m�?������ �����b�'�4f'���TgP����~���"����v�����@De�G�}.Q@�!��E(�tIC����vXG��������
_���&�%�LmN��4�4��S�R��E�|���52����G�2%;���uJg��L�"S#����J	��$�v�4m=!U��<"mL�U�i���3�Ngz�1IY�NB�A!/�
��fQ�����G���,��8O��1S��/"}����@�����\���<��k����������&uD�Lb�ex�����U��W�NG���I���7�����h4�}<�\�+~��=�L�_m������I�e��=R��� ��6E����\��Ned������$�
�1c�|��������)��\u�����gZ(�>E�X0"��i������hS�on�u��gak$���zk��h6���JD��FI)�Q�^������en����L���Ew�#wTX�s:�Sl�'j�|lw��X���R3�T��]��	�A�R���������5H.��\��9"}CC ���������$�S$K�V��"�uS\~o�F�h�J�v�-s)E���'�����������4�����>�ZQ�q�g���2��BB��.a!�����Y����
���H����(_@����6R���W@��'TV��g��=����-���������H0*�BH
*R5����d�NR�������P�C�I��������mn�k?)�M�O���������t��"� �5��?�����u��t.�Ssa)#|����id���$O	c��F��6�����pFM��+#���(�Z�]�t��������/��/#W7���b����������3�I"n}!���5s?����C���?��BGn*�&Y\$�N����j��������,n�[)V�����_
�P�GL��I1�tB�w^���n�����[9QJ��l��>�����1thn��XS�[!iH-i����z3��t��L�i��~�;~�	�����f;��z"�*��3N��&h�D]����	N��Q���������n'���
���^l*�����l'$+n��DND |�����n6�z���P����;RGi9'{
<�&�X�&���	^;:�]6"W,����rA�ZU��p���e)�(L�R�8���3��*�|v�)C
�����G���w8��f�r��-���%�CXHvq���9hf��t���9h*S�;����)5N�J�h��������-|�?����g8�?��54J��!4�����Jtzcg��P6YOu2bb�d)W��K�������L29��L�3M��n�/j�_�A�6`�O��I�B~9� ������'��<�4�l���� �pDS�A���������#���K�8�!�XMI
��An�#�b)p8B�������h��cs!����}�'82�m���������0�};��}��=v��^��������@h�C����%�mT��������V���Pr�#�0��\1K�7�K
����	������v���Q��Gs_�O{pH�j9mCM/b>�Z�)�H)F���G�,���G-������]���>#��
�W�^z���d����^
���m}r=����6����h!6���Q@�|�����~�8jrr=����f���L�����'���w�h��8����U���bd�<k�Pg%�����s�Jb>��6<�	��C8���f���L1�{���tje��C�V��$�sv`�+
������Jc��G%�r��l�g[���
E���{7kX>�<�������H��*{��*�qdT�[9':\?>6�}��bS.+�r�b6���`�<+�izWl�Y�����n���`��WhkO3$16�t@��a�n������n�	��������G����nF���x��-�F 0o����M������S�lwpT����A0��=�:R�(�D�;R��[Nd�#DAE������+���&��Z�d���GJ�{I����F��9�b�q�D�u����k{8���.
�i�����!�#� �������q8�bH�}�������7���p���t�H���\o*m��c���R.��s���V��T1�����5(��0���t�R��
]��]Gv������S�s�;
t�V�(w�%��]��������8��A�9��:D���>	w��_J�t:�~{_7�.8��I"g�&0���`������������s�����D��i��}���D�u����p�����z��a{���P�|�������e�?��L��N�syTK.�^^��C���&O��L�������4�%�����a��\#S����
����;����{'����&.�+�S�j�i?JS�cG]��{d��-�J8f�����=f�����Y�Md0g��:�������,E���%G�L����`'�K�0E�S7De�K���X.'�$.V� ��Y�<��VA�"�-�OQ��]��h�@x��=]- ��&.�9t@���C���vh�!x	gdwHB��AP�#J��AW��P5o���Kx,S����(\eL%h�7�X�?+�a�YM���&��S&D^��0�<�'HT@��%�a����$*?�'��8��E�����CiS�*���F�R$��i�����V����wJI%I�~������;0O�8�����#�K�}>�v��������c�
�&�)��0�&�������"j,%x)�%�v8���Xr�*���v���l#_IEee�qW@D��C��@�o����x���������C���l*(�N^AZ>w�AU9�*^	xW8������>h�������b���*~�����������!F�J�R^~$������0+A`X,�s�<��J��=�s�JQ�l��R�����"�Ogi����M������o�d�$S�"v�V	U�gz����i����_J��P��+�>e��3����d7:�G����:z��J�R��_��?\�u*��j3iT�"jQ���>[�#jI�K�����	�����������ys'��,IV��������9��Qa���;�x%�i�H��4��f;� �J�����e�*������N�e��\L��%&g��� �Bm1-���t?w��T���H>�.�����*Q����u�/��r���4���L����)���BH
q),P,�������0�_F�����=���Wj�,��5�d�4'H��vD��t&F<������n�����g����M{�@�8����U;�~�BIw����1�c�����T_k.�
w	}O���qf|3
�������(�!f�1�V'�Z/��7R+�p�W2%�?
�jk3�$����-�P��z���w<���u/�$�������ss���fw�r%�G��|��Ej����G����S;���o�-o�@*R���{����m~O���&��~���(���
M�r����T���kw\[@�I�&%���w{�������������G��Od�9�%'$'A��r)EW����������+�*n��;�p�j�,c�:^�&(%+�hi�}�4x�*{�N������9�7H��:saH���&R���D���vb�|)X�?�������s�I��u-�u��wO������D�����1�i�T4u�%� s�S	�b�L
�[�����������C������S��!�n�����T�N[��H.��1j#��M�erH�L|L�����Z�����~5�. \������LJ�q&�������NC>&��i��veE)��e�P":��6���
�*O�Z�Pm�b6a��&���u�;A����E��������xE����V��f�5`��7��lv5B=������}
6Le�_F��v�s�'n�!����U,)���V��S&��i��S���4-����n��\d���s��i-`L���z!�YA>��$�����)T������]�yP�#����x���?>�\G�_���/�[�g�����J�*#�Cd�%�
����D�\.^H�b��7k�hJD�<v^�n���Z�����������_��}`�p'��I�����L����-���%J!�$�_�8�@�2B�lo�
���T,�����������}W�4����.&��Yk:�A�������������N |:nj��%��.��;�1#����XiSTO��+sNu��=��_��T��]S���C}��+!���'���;�Eq������������Hw����m�Gf3M����Me%	�� �3�F�ln����+�9�����TA�bm$��WHt��61�<�Y��g�tl�P�a��`1��ff*#A+�Bw���lw��om;i�u�d2{��N\�@�_��m�K�
�������'Pj�(�����Ws���	:��:�m���������r3�'
�N����I�2��br�*L��q��f������,`P��,!�����A��?V�q���u���p[v�#�h\�j�To&���a;!cwF���+�U�z������S�����	��L�"�@H}�e�?�1�4����B�0SA��7�j�1�R������]�=6�/�~�a�����4����������@%�W"���.����`{~�������M;�'���������_/��'3�!�������f��|�������4�]x�^�j�~�|�%r�����nT�	o�aR���DN�n<���AA��ub.�}���������)R��\9�#$�6����7jU�D���L���?[���EC�����g������:�/���+���Brg��CIO�$h�J<��t���6)j9�f��S�
cf�T�BFHr��<�(�yF����L�������Ss`k�"s����N�d�q�-�p3�~�qS$P�fC��~�m�4�3�5X��Wo�r�D�� c��Y�L;�`����?��a�]I����r����>��)W��#2��P����al���N���o��;�����r�x$�n��_b31mt���@�����vEL����
bf����2�hbGK��O_���<>V����������o6��WcPy������ks�Q����3o��g�������sY[}��J=��RO<���^��Y.eR0�����j	W��(�N��Uu
nV���9z��C������\�~@od-�2�r����X���RP]�e�OS�[J@����O�]���*��jRB�=��,lO<�1c),�)�^��1�X�so����h�1`����-j�:���BM���&I����:�����i5�`�R%�*I����M�e,�p�^S�m����O���GT���1��C�Q�qR����l?��L6��l����5C3�]��fH�������j�v�V�#'���Z�l��~��M�m���
,@F�$Z��G����f	/�CX������]'����\3gw��>����@���*1E�d0�s2��hFUm����M%��[g��"���
P������x�uu������i>5�[�xn����]^����9���m~�.��i!��C�p6z����]�/�XHvQ��%��&���#9��7�*J����O�+�r��t����T'������c�������6�\e!�6M�����K�04�����:�0I��u�ni���G=LR�)�f��w2��
������P���,<����g!�����?5���?t�f�v�B!��S����"��^��FI�r������<�`k:0����*��)D��|z5�HIO��&�`1�?����(�<���`��:�1�17�����}_Ir�U��~�F���U'��xl��}�y��t�;_���+sI�;�;�k�������0��E�%U>���_f�PUh��@�sN���������6�}�5��x��4rr>NN)�C��Oc�LEs�bW�Cuo+w�����J]�X�y��+������<��i���V2D��,n���7��G�<�Q5V�tU����'L��2��\S��L��@.���;d��P��''�����\]N;�+�z�T��v*����� �������_B�8�[ �r�!J��[_����W��Wl������{m�	`0��o�g��������^�7�4�/|c��Q�����?�W%]�*KG�F@h�"$������?:|,�Q���,��0�e6��~����O�����<9md@�������
���W2/����5U������'�)����FI���i8{
����d����W����G��`�������a�sD`
�,���nQx�p�`E<�;�������j�V�z8>�}S��v�%����w:%=���@���+�����f���d�u�����)��|b$B�xIx�@.oRZ(Y���`g����h�P�g��T��Z�#D<�2�qKI��_����&���)���U~&�B���c�t;��tG����W��R~~<�d��>���Y���i%�e
za&#��Tk1��V�%E^���mh.���[��v�"�p�D���9�jx��/�s�pcN&f���o}e?��zSmXP��d�&S#���C���eQ0U9�R������o����aM����l��dE��T��DuNY�	��c�:g��Sd��*����({��nO{������r?7��I�?����6����i�xs��H��d%za�^�T/n�Jb�x��p�{H�J��}��r��Mj%������J�4A�4)��x�*RG_%]]l��+�H9�j[
i�z6,����V�q�Vg!!q��d���1]��������\���wh7��x	5��3v�2�k
(�p���f��\��N�������4��$��������(�'2�������_�������B��6'�2}��[!p;���������NXG2���O6�]G��M�V��?4�a���'O����Q��r���	1C��LMa����v�`���;�!��mS��Xm&B\rS.D!5��1f2A��=����5_������������J~y ���������>������7o����ud2AeI���|�������(5,Rt���`�y��t�{���Y|h$��E8���+�ra�'+G�xf�4�"-���F`��,���H�������.��~�T�������x�D)�W��K���
����V����<:������w*���1���si�"/itJB�����&�C?���C+�@��>����G�s`����������>y�y$��U���-`���4�u�Z��	3x�-K$���q����|�Nw�|�ug�o��T��P@�
[���B��G�6.B��Uj���,}�K���h��S>��V�)I4�����[���p!H�>f2lV��z;T���n-��-!$]�[�5A��M*mg����z�M.0��a���yq�g�y�44'Lz"��^ZTi�iEc?H����\��z�j�;@N	7q;���`��8��E�qS��.���+�n�x�����7_�}�0�S���,�:$��Y��?�@�7���
xKnG�m����9�^�+�/'t2�QA�H<��N�������������	�/U�{(�����d<����G�*��i�x}�>u������M����k�L����+q��('
��7�[���f��L9��v3���B�.J�����9����-��I�2�c���R(�*P'�\���f���\���i��Tdn��BK�;m�?/��ENYP�n2!�|~o|I��N��5�C7T`����B�s��������2,��D��'i��g��ZHuR<n-�&O�Y���a
E�n������������S�V���u\P}�E�l� ��w���T�tA�2?���1�����`�u������WI!�HFB�/���Mq�=�N��L�>��S�ya^�����0H�&4������l�J30�suS��C���S�pC�8�ad�,vD��NQ�Q$#�5��=/�hh�	lM�h�m�&�@�G���u�l���FZ�� ^��l��q8yi5�t����?��TR���x�_��]S�����n�e�xj8W~��\�Z ��H�C�������	�r�fZ�{���d���zAF2��C���I%>�)��c�6���/�}��(���<�x�/v���k	�&V���n�9��Wc�qJ
g��@f�Ev��9�������������
���A�o�
"�y�D�Q
�#J^HT�����uz�����|��i�d[��!{���	��
	ZJ{���O]�R�T0G����*p�=? �!���n���.���f����1��u���" +�8 ,��9���6����#�����r8o�6d]��xIH�������F�)V%���a���WM�l��;������ ����<G��fJ�e�I���^��o=�Fh
���a]�������/��V���@Y�4b���D��zF�')fP����>�I�S���������J����l���>��m�V��_����6B��"�i
�d��n�����6|\J�+���sl,��� �!�
��l��~]����.g
QN��p�L���h���8
F}K�n�
���7�����F��+w[��T��#�nj�^X���sW
�����i��D�^�r�#G����e.��~������>Vu���&~�
����u�(Q��Qf%�q?�v�+ ��=���-�AK!G~E�G.2��!n�eV���	�eM�?���>3�_z�9u�#8�3��f`�.�U���/3
��[%�,����������^Yzp�ba*����Qs�F�$'[�)���G���p�t'0����}
f{��S����������X~���Ffe��\9�'37��T�
r�0 �{B�/���c�(��`?GM����2��\�*&,���K��H7�4C��'47���O�}���f��������v��=�Zgb�P����=����'��x��0� ?������V������|h���}��B6��2k3�8/��p5D��"��
j7�D��J��2����S�W������2��#���!^\d�N9RHM�-<�����U� ,����� ��^�p=���Hj��q���&�U�"�h���a����
����D��L��=�f�M�m)5r�i7��$�lS4����_X7����&S����-��L@���j~�������z��Y7^)q2]�:�sD������b�������/���������+T�i����@���Tm�������9v��J�v���f��M�J]�.��6��&� �%��G��?*�49 ���$9X��
�����7_�_���8�v�2;����)@��������<���xV���nP7����G^������	�8���+�M���C�8�����-�]��)��Xq��v_.�����z`�A�<- u���b���'q�:|�����g��+���A��m]����*�Bu{[R������3b��P����f;T^]="��p8����r�X\�����e����TLy7��[�3%S�L��+$�w�f6�-'M�����C���4���@#}�*�D�p��>	�u�w���&&J��C��VjJ|�z�����sF	��~��y��P�PH��f�������Qg��k��16=��7Y��2����Xc�����Ac�i���������Y0���5����b1�!'L�i`�.�A�`���X���8�R��+�@-d���f �b4��qrJ�e6����r�E�q��E��<�fy��KsQ�/)��D����F3����c,����[\) ��S��u�l� $�KP6a���z�j��������t��.h)�)jV,��5�0a��'�+BdG!q�>hzt6�$���c��m%���X&��NYpV`����lr�>l[?K�<Ip:�1����������J�K�5�]�n���G0���z���2�|��umo0 �,F���I�Al�^�l������a�v�A���G�&jv2jn��5^�JM�<���������2�R�~H��	�b?N
J�y��yt7����|����,�.��`F����,X����dDLA��_����-@��[I�E������T���gm�5_	������������o�����ly�a� C������xm���1`�X�2������ >'� =S�@+N#Y�@ptBeV��r�/|H|�n�Th&O�(���+�l���A�����������/�Kss�p2���{��_N���Lk��Q��������\i-Bda��<`.�\<���="��[1����.a����v]
������;�$(^�C�*Z�F����'z��o�1_�o�����1w��4(�<�Ie��q�,�@�EF�)4�{�~l�4�j�����J�C���q����7��4N8�Ny��q�C���*UB�>�������]%�a���.ri�wkw�i�L��!��x���@`���n��oO]g��;���]D��x(JQ/yu3C��E��3q8���+�r����S�9�G���7�����[�N����;���R�=�F!�p�3?p�@�
���t�|��b��br�bri�A�Nc��y����e7r�
��2�,G�S�N��@h��	fE��Tn�T�j$��;OJ����d!nl0P6���������ra������H��6�G����Bb�&���G���_���_��/e[>]^�S}���:'�g��Bl:!�V{�{Tr��4��v�whs��4���Y7�F�|�`Z��`�.��0]���Hw��I��d�U�a(�(g� �L�i����I��
��Z�T���E�<�I���J�����l��l?4��#z|n�Wy���R+�7�k��\8�&n�R��'���4Ki|	�vG�����S�O���22�C�u��MZ$SF���Q��KI|���<+(��N�$"��k��.v'������9�5d�R�Y����%�'Pux)��+�B�1?cN�no�����_���Hm'������J6a�2���Gq@uq(�-��*��K'��"���M2m*���\��$h���P�)(���t��A���S3�����3��_����|������_��������^0����*r~��r��������n���A�j��5��aJ�hP"�xq����B8������;h�~*[{w�c(**>0�5����*�c�ZO~u	,���~��2���^�"n�'�D������H��	����p�y*����8��Ss���
��~.O�j�]
\�������r�lH`a�����;���d�
yg2���ZX,�9�������me~��������+��3�G�;�E-7,w��F���4������q`)��Kg��YTf�$��?h�.uStt����Hg|�wg�"����b�B
�'rd�Q���a����3_����>����,�:Z5fv�Bo�R�BO7�j�OM�-
��9z|���%r)�i��S�P$��;����
6�����s��J���,�v)��3a���;x�:�W����yh�igz^�+B����~�:X��
M���6������i�wv���Bd�s��&�i(rKH�PA�x~�V��1�o�����r��X�7����E50a���J��8�9�l��H"�����1�T��4[IfL�R���O��U�p�M*��'�
�l�(F�d���m ��a"�����d�����n��,�'_0���<>�K)A>K���9��w���+��`@�y���0�{t�&G�P�,G@�=�{�M��M(4��B�_��!�}B?o�/���?:2e������7����FyeNMyH8���8DE(�AZ�~w#0��"�>0���^m~t�9�<������������H>.,��F�^K�@���_Mw�/�5<����|����S�l�������v9�
A�r8_\/������v��i��+���:�U���x�>I�0��a'���xQ��Omy���>�es�F2/���r����4�1,��B�O��I�ig������������V-t��_l%�������dj%m�E����`�FR"%��g��t�d����<?Aq	���0A�?�d@m}��9W1������	!9����U�o�8�j�n}9���s��g�����{���_����ZFsu�C����R��d&���ZG(|�wn������>�������LE|_R��*T�N��+z��E�OV6I���~�1����*����{0��	%<�7�Y���7��mpv����������;�B���yy-/��b`���!������b]�������7�]��G�)�C�ea�h�5e�$�h�Ga����9�*���N����_�)m�.��@�i	�k��F� c�����&��:+r���H�v=��V�
�K�F�\��"�t����;R��	��������$��L}9T�~��'�o.�z$$A�fvQ���Nu�|����r�p���P~�5���	�P�9�#bM�W���9�ci��<W�u�4�������m~�\���k�Q������ 9�3���^W�A��t�_���6������NX1Jd�i4�B:���A�T�n���`*��s<���3�����i_����|9j��kQ�+�o��]����n�����n��0�:Q+kC[|&�
��vy��w_J9�T\c�i��d����~���S�	����)���U>X�������/���/mf���H��qXP��?v]�r���;NE0eR�q�w����~�"��Lj�FJ���n����r����`���C��2yh�@�,���
1\�8[�:��(���P`?�g`�>��p�wp�0���Q~������s����������6g����:����C��9��o	��dF�_�t(*l��R��}85e�?V����v��)��!9aA2U>�����H�UJ�X�a�G?�}I���B��z��������4��^j��Da����6_ ��*/���WOa�DV %tD������?�������"H��Va��/�gJ7���RI�-�Xa������
���y ���q�mW�&�i������y���C��k_�����!`I$���z"$�>.���N�g;�	����J�VA3�C�o���K�#$��x��P�=�i\����P�[x4m���4��{���"'�
#39Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Michael Paquier (#37)
Re: Speedup twophase transactions

Michael Paquier wrote:

On Tue, Jan 12, 2016 at 5:21 PM, Simon Riggs <simon@2ndquadrant.com> wrote:

Should we just move the code somewhere just to imply it is generic? Seems
pointless refactoring to me.

Er, why not xlogutils.c? Having the 2PC code depending directly on
something that is within logicalfuncs.c is weird.

Yes, I agree with Michael -- it's better to place code in its logical
location than keep it somewhere else just because historically it was
there. That way, future coders can find the function more easily.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#40Andres Freund
andres@anarazel.de
In reply to: Simon Riggs (#24)
Re: Speedup twophase transactions

Hi,

On 2016-01-11 19:39:14 +0000, Simon Riggs wrote:

Currently, the patch reuses all of the code related to reading/write state
files, so it is the minimal patch that can implement the important things
for performance. The current patch succeeds in its goal to improve
performance, so I personally see no further need for code churn.

Sorry, I don't buy that argument. This approach leaves us with a bunch
of code related to statefiles that's barely ever going to be exercised,
and leaves the performance bottleneck on WAL replay in place.

As you suggest, we could also completely redesign the state file mechanism
and/or put it in WAL at checkpoint. That's all very nice but is much more
code and doesn't anything more for performance, since the current mainline
path writes ZERO files at checkpoint.

Well, on the primary, yes.

If you want that for some other reason or refactoring, I won't stop
you, but its a separate patch for a separate purpose.

Maintainability/complexity very much has to be considered during review
and can't just be argued away with "but this is what we implemented".

- *		In order to survive crashes and shutdowns, all prepared
- *		transactions must be stored in permanent storage. This includes
- *		locking information, pending notifications etc. All that state
- *		information is written to the per-transaction state file in
- *		the pg_twophase directory.
+ * 		Information to recover prepared transactions in case of crash is
+ * 		now stored in WAL for the common case. In some cases there will be
+ * 		an extended period between preparing a GXACT and commit/abort, in

Absolutely minor: The previous lines were differently indented (two tabs
before, one space + two tabs now), which will probably mean pgindent
will yank all of it around, besides looking confusing with different tab
settings.

* * In case of crash replay will move data from xlog to files, if that
* hasn't happened before. XXX TODO - move to shmem in replay also

This is a bit confusing - causing my earlier confusion about using
XlogReadTwoPhaseData in recovery - because what this actually means is
that we get the data from normal WAL replay, not our new way of getting
things from the WAL.

@@ -772,7 +769,7 @@ TwoPhaseGetGXact(TransactionId xid)
* During a recovery, COMMIT PREPARED, or ABORT PREPARED, we'll be called
* repeatedly for the same XID. We can save work with a simple cache.
*/
- if (xid == cached_xid)
+ if (xid == cached_xid && cached_gxact)
return cached_gxact;

What's that about? When can cached_xid be be equal xid and cached_gxact
not set? And why did that change with this patch?

/*
* Finish preparing state file.
*
* Calculates CRC and writes state file to WAL and in pg_twophase directory.
*/
void
EndPrepare(GlobalTransaction gxact)

In contrast to that comment we're not writing to pg_twophase anymore.

/*
* If the file size exceeds MaxAllocSize, we won't be able to read it in
* ReadTwoPhaseFile. Check for that now, rather than fail at commit time.
*/
if (hdr->total_len > MaxAllocSize)
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("two-phase state file maximum length exceeded")));

Outdated comment.

+/*
+ * Reads 2PC data from xlog. During checkpoint this data will be moved to
+ * twophase files and ReadTwoPhaseFile should be used instead.
+ */
+static void
+XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
+{
+	XLogRecord *record;
+	XLogReaderState *xlogreader;
+	char	   *errormsg;
+
+	xlogreader = XLogReaderAllocate(&logical_read_local_xlog_page, NULL);
+	if (!xlogreader)
+		ereport(ERROR,
+				(errcode(ERRCODE_OUT_OF_MEMORY),
+				 errmsg("out of memory"),
+				 errdetail("Failed while allocating an XLog reading processor.")));

Creating and deleting an xlogreader for every 2pc transaction isn't
particularly efficient. Reading the 2pc state from WAL will often also
mean hitting disk if there's significant WAL volume (we even hint that
we want the cache to be throw away for low wal_level!).

If we really go this way, we really need a) a comment here explaining
why timelines are never an issue b) an assert, preventing to be called
during recovery.

+	record = XLogReadRecord(xlogreader, lsn, &errormsg);
+	if (record == NULL ||
+		XLogRecGetRmid(xlogreader) != RM_XACT_ID ||
+		(XLogRecGetInfo(xlogreader) & XLOG_XACT_OPMASK) != XLOG_XACT_PREPARE)
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not read two-phase state from xlog at %X/%X",
+							(uint32) (lsn >> 32),
+							(uint32) lsn)));

I think the record == NULL case should be handled separately (printing
->errormsg), and XLogRecGetRmid(xlogreader) != RM_XACT_ID &
(XLogRecGetInfo(xlogreader) & XLOG_XACT_OPMASK) != XLOG_XACT_PREPARE)
should get a more descriptive error message.

/*
* Scan a 2PC state file (already read into memory by ReadTwoPhaseFile)
* and call the indicated callbacks for each 2PC record.
*/
static void
ProcessRecords(char *bufptr, TransactionId xid,
const TwoPhaseCallback callbacks[])

The data isn't neccesarily coming from a statefile anymore.

void
CheckPointTwoPhase(XLogRecPtr redo_horizon)
{
-	TransactionId *xids;
-	int			nxids;
-	char		path[MAXPGPATH];
int			i;
+	int			n = 0;

s/n/serialized_xacts/?

Maybe also add a quick exit for when this is called during recovery?

+	/*
+	 * We are expecting there to be zero GXACTs that need to be
+	 * copied to disk, so we perform all I/O while holding
+	 * TwoPhaseStateLock for simplicity. This prevents any new xacts
+	 * from preparing while this occurs, which shouldn't be a problem
+	 * since the presence of long-lived prepared xacts indicates the
+	 * transaction manager isn't active.

It's not *that* unlikely. Depending on settings the time between the
computation of the redo pointer and CheckPointTwoPhase() isn't
necessarily that large.

I wonder if we can address the replay performance issue significantly
enough by simply not fsyncing in RecreateTwoPhaseFile() during WAL
replay. If we make CheckPointTwoPhase() do that for the relevant 2pc
state files, we ought to be good, no? Now that'd still not get close to
the performance on the primary (we do many more file creations!), but
it'd remove the most expensive part, the fsync.

Greetings,

Andres Freund

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#41Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Alvaro Herrera (#39)
Re: Speedup twophase transactions

My +1 for moving function to xlogutils.c too.

Now call to this function goes through series of callbacks so it is hard to find it.
Personally I found it only after I have implemented same function by myself (based on code in pg_xlogdump).

Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On 12 Jan 2016, at 18:56, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:

Michael Paquier wrote:

On Tue, Jan 12, 2016 at 5:21 PM, Simon Riggs <simon@2ndquadrant.com> wrote:

Should we just move the code somewhere just to imply it is generic? Seems
pointless refactoring to me.

Er, why not xlogutils.c? Having the 2PC code depending directly on
something that is within logicalfuncs.c is weird.

Yes, I agree with Michael -- it's better to place code in its logical
location than keep it somewhere else just because historically it was
there. That way, future coders can find the function more easily.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#42Simon Riggs
simon@2ndQuadrant.com
In reply to: Andres Freund (#40)
Re: Speedup twophase transactions

On 12 January 2016 at 18:14, Andres Freund <andres@anarazel.de> wrote:

Hi,

Thank you for the additional review.

On 2016-01-11 19:39:14 +0000, Simon Riggs wrote:

Currently, the patch reuses all of the code related to reading/write

state

files, so it is the minimal patch that can implement the important things
for performance. The current patch succeeds in its goal to improve
performance, so I personally see no further need for code churn.

Sorry, I don't buy that argument. This approach leaves us with a bunch
of code related to statefiles that's barely ever going to be exercised,
and leaves the performance bottleneck on WAL replay in place.

I raised the issue of WAL replay performance before you were involved, as
has been mentioned already. I don't see it as a blocker for this patch. I
have already requested it from Stas and he has already agreed to write that.

Anyway, we know the statefile code works, so I'd prefer to keep it, rather
than write a whole load of new code that would almost certainly fail.
Whatever the code looks like, the frequency of usage is the same. As I
already said, you can submit a patch for the new way if you wish; the
reality is that this code works and there's no additional performance gain
from doing it a different way.

As you suggest, we could also completely redesign the state file

mechanism

and/or put it in WAL at checkpoint. That's all very nice but is much more
code and doesn't anything more for performance, since the current

mainline

path writes ZERO files at checkpoint.

Well, on the primary, yes.

Your changes proposed earlier wouldn't change performance on the standby.

If you want that for some other reason or refactoring, I won't stop
you, but its a separate patch for a separate purpose.

Maintainability/complexity very much has to be considered during review
and can't just be argued away with "but this is what we implemented".

;-) ehem, please don't make the mistake of thinking that because your
judgement differs to mine that you can claim that you are the only one that
has thought about maintainability and complexity.

I'm happy to do some refactoring if you and Michael think it necessary.

- * In order to survive crashes and shutdowns, all prepared
- * transactions must be stored in permanent storage. This

includes

- * locking information, pending notifications etc. All that

state

- *           information is written to the per-transaction state file in
- *           the pg_twophase directory.
+ *           Information to recover prepared transactions in case of

crash is

+ * now stored in WAL for the common case. In some cases there

will be

+ * an extended period between preparing a GXACT and

commit/abort, in

Absolutely minor: The previous lines were differently indented (two tabs
before, one space + two tabs now), which will probably mean pgindent
will yank all of it around, besides looking confusing with different tab
settings.

* * In case of crash replay will move data from xlog to

files, if that

* hasn't happened before. XXX TODO - move to shmem in

replay also

This is a bit confusing - causing my earlier confusion about using
XlogReadTwoPhaseData in recovery - because what this actually means is
that we get the data from normal WAL replay, not our new way of getting
things from the WAL.

@@ -772,7 +769,7 @@ TwoPhaseGetGXact(TransactionId xid)
* During a recovery, COMMIT PREPARED, or ABORT PREPARED, we'll be

called

* repeatedly for the same XID. We can save work with a simple

cache.

*/
-     if (xid == cached_xid)
+     if (xid == cached_xid && cached_gxact)
return cached_gxact;

What's that about? When can cached_xid be be equal xid and cached_gxact
not set? And why did that change with this patch?

/*
* Finish preparing state file.
*
* Calculates CRC and writes state file to WAL and in pg_twophase

directory.

*/
void
EndPrepare(GlobalTransaction gxact)

In contrast to that comment we're not writing to pg_twophase anymore.

/*
* If the file size exceeds MaxAllocSize, we won't be able to read

it in

* ReadTwoPhaseFile. Check for that now, rather than fail at

commit time.

*/
if (hdr->total_len > MaxAllocSize)
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("two-phase state file maximum

length exceeded")));

Outdated comment.

Ack all above.

+/*
+ * Reads 2PC data from xlog. During checkpoint this data will be moved

to

+ * twophase files and ReadTwoPhaseFile should be used instead.
+ */
+static void
+XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
+{
+     XLogRecord *record;
+     XLogReaderState *xlogreader;
+     char       *errormsg;
+
+     xlogreader = XLogReaderAllocate(&logical_read_local_xlog_page,

NULL);

+     if (!xlogreader)
+             ereport(ERROR,
+                             (errcode(ERRCODE_OUT_OF_MEMORY),
+                              errmsg("out of memory"),
+                              errdetail("Failed while allocating an

XLog reading processor.")));

Creating and deleting an xlogreader for every 2pc transaction isn't
particularly efficient.

Is keeping an xlogreader around in a backend for potentially long periods a
better solution? I'd be happy to hear that a statically allocated one would
be better.

Reading the 2pc state from WAL will often also
mean hitting disk if there's significant WAL volume (we even hint that
we want the cache to be throw away for low wal_level!).

Nobody has yet proposed an alternative to this design (reading the WAL at
commit prepared).

It's better than the last one and I haven't thought of anything better.

If we really go this way, we really need a) a comment here explaining
why timelines are never an issue b) an assert, preventing to be called
during recovery.

Sure

+     record = XLogReadRecord(xlogreader, lsn, &errormsg);
+     if (record == NULL ||
+             XLogRecGetRmid(xlogreader) != RM_XACT_ID ||
+             (XLogRecGetInfo(xlogreader) & XLOG_XACT_OPMASK) !=

XLOG_XACT_PREPARE)

+             ereport(ERROR,
+                             (errcode_for_file_access(),
+                              errmsg("could not read two-phase state

from xlog at %X/%X",

+ (uint32) (lsn >>

32),

+ (uint32) lsn)));

I think the record == NULL case should be handled separately (printing
->errormsg), and XLogRecGetRmid(xlogreader) != RM_XACT_ID &
(XLogRecGetInfo(xlogreader) & XLOG_XACT_OPMASK) != XLOG_XACT_PREPARE)
should get a more descriptive error message.

OK

/*
* Scan a 2PC state file (already read into memory by ReadTwoPhaseFile)
* and call the indicated callbacks for each 2PC record.
*/
static void
ProcessRecords(char *bufptr, TransactionId xid,
const TwoPhaseCallback callbacks[])

The data isn't neccesarily coming from a statefile anymore.

void
CheckPointTwoPhase(XLogRecPtr redo_horizon)
{
-     TransactionId *xids;
-     int                     nxids;
-     char            path[MAXPGPATH];
int                     i;
+     int                     n = 0;

s/n/serialized_xacts/?

Maybe also add a quick exit for when this is called during recovery?

OK

+     /*
+      * We are expecting there to be zero GXACTs that need to be
+      * copied to disk, so we perform all I/O while holding
+      * TwoPhaseStateLock for simplicity. This prevents any new xacts
+      * from preparing while this occurs, which shouldn't be a problem
+      * since the presence of long-lived prepared xacts indicates the
+      * transaction manager isn't active.

It's not *that* unlikely. Depending on settings the time between the
computation of the redo pointer and CheckPointTwoPhase() isn't
necessarily that large.

CheckPointTwoPhase() deliberately happens after CheckPointBuffers()

Default settings would make that gap 2.5 minutes. Common tuning parameters
would take that to >9 minutes.

That is much, much longer than acceptable transaction response times. So in
normal circumstances there will be zero transactions and I concur with the
decision not to bother with complex locking to avoid longer lock times,
robustness being a consideration for seldom executed code.

I wonder if we can address the replay performance issue significantly
enough by simply not fsyncing in RecreateTwoPhaseFile() during WAL
replay. If we make CheckPointTwoPhase() do that for the relevant 2pc
state files, we ought to be good, no?

That was the design I was thinking for simplicity, but we could do better.

Now that'd still not get close to
the performance on the primary (we do many more file creations!), but
it'd remove the most expensive part, the fsync.

Which is why I asked Stas to consider it. As soon as I realised the
potential timeline issues was the point where I say "separate patch".

This is a good performance patch with some subtle code that after much
thought I agree with. I'd like to see more from Stas and I trust that he
will progress to the next performance patch after this.

So, I will make some refactoring changes, fix your code suggestions above
and commit.

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#43Simon Riggs
simon@2ndQuadrant.com
In reply to: Michael Paquier (#37)
Re: Speedup twophase transactions

On 12 January 2016 at 12:53, Michael Paquier <michael.paquier@gmail.com>
wrote:

On Tue, Jan 12, 2016 at 5:21 PM, Simon Riggs <simon@2ndquadrant.com>
wrote:

Should we just move the code somewhere just to imply it is generic? Seems
pointless refactoring to me.

Er, why not xlogutils.c? Having the 2PC code depending directly on
something that is within logicalfuncs.c is weird.

If that sounds better, I'm happy to move the code there.

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#44Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Simon Riggs (#42)
3 attachment(s)
Re: Speedup twophase transactions

Hi,

Thanks for reviews and commit!

As Simon and Andres already mentioned in this thread replay of twophase transaction is significantly slower then the same operations in normal mode. Major reason is that each state file is fsynced during replay and while it is not a problem for recovery, it is a problem for replication. Under high 2pc update load lag between master and async replica is constantly increasing (see graph below).

One way to improve things is to move fsyncs to restartpoints, but as we saw previously it is a half-measure and just frequent calls to fopen can cause bottleneck.

Other option is to use the same scenario for replay that was used already for non-recovery mode: read state files to memory during replay of prepare, and if checkpoint/restartpoint occurs between prepare and commit move data to files. On commit we can read xlog or files. So here is the patch that implements this scenario for replay.

Patch is quite straightforward. During replay of prepare records RecoverPreparedFromXLOG() is called to create memory state in GXACT, PROC, PGPROC; on commit XlogRedoFinishPrepared() is called to clean up that state. Also there are several functions (PrescanPreparedTransactions, StandbyTransactionIdIsPrepared) that were assuming that during replay all prepared xacts have files in pg_twophase, so I have extended them to check GXACT too.
Side effect of that behaviour is that we can see prepared xacts in pg_prepared_xacts view on slave.

While this patch touches quite sensible part of postgres replay and there is some rarely used code paths, I wrote shell script to setup master/slave replication and test different failure scenarios that can happened with instances. Attaching this file to show test scenarios that I have tested and more importantly to show what I didn’t tested. Particularly I failed to reproduce situation where StandbyTransactionIdIsPrepared() is called, may be somebody can suggest way how to force it’s usage. Also I’m not too sure about necessity of calling cache invalidation callbacks during XlogRedoFinishPrepared(), I’ve marked this place in patch with 2REVIEWER comment.

Tests shows that this patch increases speed of 2pc replay to the level when replica can keep pace with master.

Graph: replica lag under a pgbench run for a 200 seconds with 2pc update transactions (80 connections, one update per 2pc tx, two servers with 12 cores each, 10GbE interconnect) on current master and with suggested patch. Replica lag measured with "select sent_location-replay_location as delay from pg_stat_replication;" each second.

Attachments:

replica_lag.pngimage/png; name=replica_lag.png; x-unix-mode=0644Download
�PNG


IHDR������sRGB���@IDATx��\T���}�(((�����HM>Hm1�J��M5���6����zh�j����TK�&jm"$�iDc�&�� �&Fk1R5*
PAA����3{��4�qf�{�f������3���Y
"��H�H�H�H�H�%x��(9H     �
x>$@$@$@$@$�Bh�����PI�H�H�H�H�<�    p!4�]�fq�$@$@$@$@$@��	�	�	�	�	��.t�8T    ��g�H�H�H�H�\�������C]�zu��UYY	??�N�����#���]1���:���B��vEsl�M	��zx{{����M	p�]A�]���������]�Z.�����e/��[��;��������$`>Gvab�6TWW���#G�l��H�mW�\�����o��2�� ����M)�=v%
�6{�~��_@�:��}�]<��#�i�uI|��t�@yy9�=�{����M��8x� ���A��1N����oZ}}�]�����	�	�	�	��s�������+>��S�V��}�s��	�	�	�	�@W��I�111�={v'[au    ��
x�8�Z��[nA\\\���     ��$@
|W�d[$@$@$@$@$��h�w3`6O$@$@$@$@]I�|W�d[$@$@$@$@$��h�w3`6O$@$@$@$@]I�|W�d[$@$@$@$@$��h�w3`6O$@$@$@$@]I�|W�d[$@$@$@$@$��h�w3`6O$@$@$@$@]I�|W�d[$@$@$@$@$��h�w3`6O$@$@$@$@]I�|W�d[$@$@$@$@$��h�w3`6O$@$@$@$@]I@����-    ��@}C=�_�
��<�c�%(�)���D��X{��u���������a��V�����yB$@$@$@$�H���s���n�j�(��_�����Oh�������<�/^lwy$   w&���s�U�gk�+F�	D_�>��_F�0��l�o�Q�%0t��6�0�H�H�H���@CC�^����O�4��������!��{��}a�����jwGes�4�m"b    ��(5���s������� /?�m�l<�?F9�j*��^e��;���F��I���Z�y)������	�	�	�����yb��x�v�Q���l���y���TVWe>�A�h����WI���^�F�{��e-�!  p
_�K{�U\0T�KHfR?��G?�P��	Xy����b��	
����Z�'`(����H��)3n������j;��/���s�>c�U�����_���u�7u^
v���9���c�<c=�VJ�~��.��O$eyI���EY���]�1�vc[3���b��-�q�N"^��)S=0/����	�	��+P��KB>�-�������S���������{K
|{hus�rC�X�}��{������@��Z�m�������+>y�F�������R��O�����x�JD=�����@�t�Z��%��`U�o�JB|�Q�T��DmRGN��������}���(��n�pD�p�y��5�a�ym&%e"ja�,������q�v"��9�X)^�b����M��I�H�H�i	�#�����,���T���������:^?�����bq�����Z��
x'y��>���q����u�q��0xjq:����x��%ac6'NN����6��Q�3�b��.�E\�U���s@��<�8��$v�ud��Am�8�����-a���?]�O��!�|�@o�O��_�����q���Dh���!�������,Zik���p{H���8�S.!69������?I�H�H�%\�����(4�K��4�g�����K�E��BO�m\4����Q^*4�@���^zr�7�g�	���S~!����./�E�{��\���	"��������������u��C#��,�B���H�;q�^��H�1@������$��d����7E[�(�L�����e<\o��3f��x�H�H�\���Y��q��\�m>��i��q��0�&!�J��Xm���jb�/���c��J^�O��7�M�1#J�>����m[%����mT�*�t�T1���K]MF�(�����RX�Q�1s���4)��F&M8V7�!n�fdf�����K�M������e�����.T���@�b��"-5��$J7��:y&  p%���f�}��{���q��i����$��2�J����{|��S����d�|��0�������y!���AB-���"i�����I2��|����/`��PI�~��A�t�C_��W�aY�2��\�t�cW����	�,x�����_��9���C�������\���J1�   �! �/j�xG��wYO
%i�O^e��{�e�-�����tYLr�`!������3��'�
QE�������{�#h�t�W���Rh�� #c9�N���r��+:��G��~������Z��������NHu��=���
�d�8���5kP�_>}L$@$@$����5���k���
���m�]���{�L�]y|�m���c�re-�E����@s�Z���/Gy�x0|��>\Z%	b��C�E�oBlz�m��%���������w�_�eSW��(�aUj<�E�������u��^�s�6!�a��[�)��b�u��������p��I��J�&+r���\��$���R#5��G
^��'zE�q�K��H�H����e ��(
�X�������q����9j|��h�����
	���#���>]p�}p(})����� _,�z��{9�6,aD�p��1���*�.�;��d�#�E� ������$NP���H����2"�)MH���������;QQr��PQ���V`��WU��b���E��*���pJ)�s&?t�N[�Z�l�ew��zo-J.��B	*��bz��W�D���:��	�	��s�����3x�v���nM����:�+�J�������\4k���J�.��"�����������"&����������s��e�	��A����G�����V����m)����1k�$^�ew���@Z�"DG
�|d26��o��PxE'�2J����,�;'��m��_~���uz����G�<P��j��60��|Q�e���3nn�$@$@$����
x?o��c��:oe�8�^%����M�z�N�����($	Y��d�����b���z��/2|v^����!����7v96�U!q�I��B���]4��>t�J��T�NO�\��z������h�D�`���s��-���c%p������y.\@}}��i������9q���o�"3H���#<��*�MQii)���!�:�+�=��������������������[D\��]s��
 �U5��S_�F�
�	���"D���x!O��������@����J!w�!u�]�N.�{*����a_�LIX���d�nD�"����>�P�1�dlF���h�����G��nBHL2����l��Zq������/&M����8����y��V3�C�V~��A�Y��a�H��������{;T��H@8x� ���A�dh\&w"���n�=�!};d���N��gN(���2
���n;�F�_}��]Na9�2����Z$�~
&�Y��P��+���L��+g#B�8b�,Y9F������D�AV�q:�����n�Q������� ����	]"���&���(-�������6Zm�����h$%������*	�	�	�	�zj�@�!SG���U���i�8�#
�A�_�j?S������a��&	�v#���<�O"Wi(GO��U��0�Q-�N��V��]r�����saI   ���8���z�� ���t��.Q�������������$d��b��2l^4�$�)��B�����}�����aG[��q��9s��U���3Bb!����		�	�	�	�����6r�����n�|������}�f��0�O�a�
L�h�l���% 'y:&�OCZT���~W���`�&�2y�w��\�;V�	�_n��D$@$@$@$�	���x���n�����A
|�������s�4��E��;��]�}�-�[_���j5�����.�w�m�bYn�����k���T s%�^u/^y��_�2���g�����3H�H�H�H�� �(�i���Q�Y����~�J]%��o���B���>Rp�d���_�� G���q<K�1�����)v�gF�4�[�#xj��-���/?cL��.���E���������X���}\������$u�Ci>�����G[,��L��K$@$@�'�Jh:��U� ��]7��%4��7�NLT��aj'w�\4Q8"�����.w�l����.���>d����6��B!N�:%v����s����:@�`�*�K����h-U!��xL�e��B�v_�:�����"zR���	�	�	t��uB-!RG5������MJh$�VS������2�������C}���C��b62��h��	�XU��0��|�8�����}�>	�	�	�C@'�w���7�����m�ng������B@_�KWa����w���n��t�#	���t��z���q�<��d���?�$6�K����(R"]�c��DcY���~�����9��U�L�S������[W%�vQ����HY�������j�T��������"Q7��>�H�H�H��T��F�$4F��������������!8������gO9�p\r #��G�zmE�f�/`������g�%6����F��ji�_CT|�������{	���@Y��~�%��y��5���~�x�*:!m�C�����u�{��'L�]�������e#>M�S�0�$S������"���n��wc��x�����J�	�>B��bQw�����$������9G   ��"�.`��
��FN������(4
�V��oMgH�=.z�:A���0r��e��^6������h&�|� J�o�/��3��-M�y�2����2����I,J��_�����q���D�h;��=~�A�_A�l��]��Y�pT7�~aQ�(�]�p����jdNZ���&d�9�}��*���_�����D�Y����{`�0��My��MB���8��Rv&��H�H�H�)�!$�`:%���<����y_i���������G��S������D���>}5.�����B�O`oF�))��>��J�*���U�8.�V�]����gTT��_�/E���f(�������p�C��/��Gx5��8T���l�p*��������� JD_�������TUHx�H�H�H�IX����=����}ci�����r��Cz�)��s;%4�N�d�+�?M�I��II2�i,�$O��M�X����M;��X�����l��TZ}���f�Z���2\Y��ol*�b}p�p=*.�1�S�MY��F"<" p4UB�!"7d�h
2��Z�YCC4^m�W���z�D�kr'N���={���6m����'���C�mw{,h�@>�-��hc|����A�����������	;�C�}�`�4���t%�j�����B%S�������
�!�y2���u2���h���I,�MY[���A�;Uh���I}"&�����h�+���H�H�)��X�{j����)�^NJ~(��s��9� h�w��8p�/_�j���U+<�	�c��S/ ������aOFl�dK�8+_cDm����_t��	BB#
]�������Z��)�����b��5v#n���mLd��?��3�y���4^���Q�B�����cc�D�$��5�~�(*������c����Y$@$@=L�Z�^��3�w������@���H�52v^�?>RRR�,�b�F 6>��	1
+Y���V��l�0��0E�N)>uR�c��xl�/�S�&��+�'���XY�1���_��<�"���W���J�?��o
4��Q�o#�Oo���
%���������Qs�0>#I�OH�Ef��T�0�	�	��SP%4��@#'�n�$�����e4�[���nA ������b\���BU��NNDC��(�\����9���Z��M�
�V�\lZ��Vf����
Ux��Lh�}h��'�m��U����S�4��p�W��/o��X��e&�Y���gq��Aa������|'  �P%4�����RBSV'�83�H�|�Xx�=T�R�����1V��"<<�����!o�����/������q��/���@1F��Xg��H�H�J�����:6�@.��n.z�[ghm�^�9$��x{��a�"�^51N�H�H�@@
#�Y��X+��\�+�=���<�����U�{�F`�� ��9�������a���hh�����;  ��T���X%JR1�M��W�|���Q�_+xCD��������#����I������H�H�H�H�&�F	�q#&��(�1�!5�(��o|��l�K�����t�p;�[���������9C   �V]��UR��)��K��)���!���A�������W���"n��t����q��zg8.   pu
�������=�������?F=���N�b[���B���<!"	�	�	�	��SP��rP��/�P=����~c���	���G�����q3��zu?ts�[P����$   ��!��gd��^�#)���?��p;��d3�]�qw��0�	�	�	�	t����UV�
������\���V��z�+��o3H�H�H�H�H�%���{�e�X3-�(<����=�RB��s��$@$@$@$�c�]��-������X�]������a$���IO
�=�X�H�H�H�H�K	�������8QsW]��W%4���t���]
��1j�-`���>��xx�[
�'''��!�   p+


Xy�#<W�m������cW9P%4�^=�}�\T�&�H��c�\�����N�s�Aq4$@$@$@NK����-���K?���e�}<�P�P�2C����������"|k}4����*�a��OD;�z�!�������	�	�	�#����^��
����}G����������z=��H^��L�����+"���L�y�:U�Bc/2�#   h/�s5�����>��H�O����Q}����OUS���U�Z��[GHh$�*7���xG=���H�H�H�m���V�����.{��E�4�?�B?~�P�A���<g?h��wMI{��JhdY)��5��jO��P�u���
�9   �# ���"�L���2������wyq��������t����[��n�$�U��BVz���.<    ��# #�l��9�P�W�*��<?xw�N��
�����
9�GF��H�q!+
x'�e��H�H�H�\�����qr=��V(��������'��U����22�eD�R�
eN��B#;�t���4��G�?H�H�H�H���<g6�g���|?�|������:#
��:=���
i��3x���q�$�����a�Qhd���|�=j��H�H�H�]XF�y+��q�m��BF#
xg��_����>���G �1�iv�(<>`���~6]��]:U4���vp0$@$@$@���5�W��g��w9g���i=�%B*����(6��j0(�i�w���������|Mww�t���w�[��	�	�	�:��]����4����������U�G������`��o�W!���������������!
xG��O$@$@$��t$2��Y��\��!���8V]j�?�b�m��;���1�����>�IR��i�c�����H�H�H�H�7h���]j�er
�4���~�l�?���b�G��SC�{n�j[�������g��01�H�H�H�H��x�w'
2m���(4��~��D���B�����g�=ZF�h�&R���G��3   ���x��x�FRF��_x������g5���T��zj�{�oQ7��o�rss�z��m��9OH�H�H�����&Pc��j�ub�Z���'�dWR-�)���8-���S�������OD���x���&p�m��{�����Y�H�H�H��h���/�Q��H:eb3��=�5?p�����s�/���������7J��W�����)7a�<���N96�H�H�H�15�������H4=a���)�/.�o]�W@y	����gan����^�"V�h����	�	�	�	4'P�P���j%c�����FV��H4W7���l�,b�W��L���Qx��f\!�=��B�
��c$   �% �wu���v�Az���DS]o@Z�������Z{��
O���o��'�WDvq���xW�U'	�	�	�	87U�.G���X�{j!%,ub������W]��s������j������:Ml���Y����`cY    ��T��������R����w�n�za�����Xq�C�\ff����!�A�����w���(4#�27�%   g$�Q����D#
x�����p�rP�����T����e�sb@Tg�wx}z�~8   ���r6�����j,��z��_��S�w�������!#�{���<u��F��i������1M;��AQ=�Ku�6��D�7�S$  �.'�z�5Bc����]���h:��7��2�.���W>S��:��L����;}C�5g/l���aOg�9=5>}����EPH��1��=��T���H�H�H��P5��Y��N>���#Qh�����~�l�G���q?A��'z��.y5j���tT���N^��|�|�������lB��8|Plp�g�s$  �B��#��k����'�����������J���O��1��V���4�Re�x7�BC^yT
�$#*
�<|�K=�,	�	�	���P=�����tT���]����a��ob[��-�q���D5�e���������:}{o��rZ�e��������-���+�@�p������P������s%  �� ����V�_��P�^�������X�o���������
�9��T����
������ ����t�^@@�}���d������
BHXB�|�t�S�����HH��c��oK*����j���Y�H�H�H�}	�Qh:��7j��A*���JkJ>��D|w��������������i���F���+�`eW�.R�I]�:����71w� ���M��#��X?k�R�r3��#/!&i-�n����U���5w�/��;N��L -g�#�@d26��o���#'4���<77[�n������9sZ��k$@$@$@��@�^��LjYMF�	���f������Z�W�x�v(�F�������]�L]�*g�n�$������Q1�V��
@���Tx���jua���Qn��s`�cQH�S+Y����*<�Q����D)r�6����b���+����~b4P���E����l�����7��g��������^hz�|~��	�����s�@G�9�5��$PUU���R���[^�1	������q��Y��=����K
�zv���b��e������5���w7F��mF�������~%J���~X~~8�0��*��{�����S�X���c|�+W��1�>|�]CV�b�
�X!�h|P����3��$x5�����)��1�
��Obf@*��`���H��E�0����a�[�j�F�x�/�ct�89�C�&���� 3�X j�&�0I|*h#�����q��(e�z��GZ-#����[��� ���;D@�G���������D����1f�4�m�<}Tx�B���}��|�]�j�����/��{�����6����J��KWs���o����o�G��a�7%{�������U�}�}����������B}}�]��9
x1t�22
�n<y��n����(��@V�F�	������kZI��G=��[��^MZ�rO��'t����������D�Zo�=�j����x�	�������$@$@$@nF�sQh������\�5|��+8-|��xx��Aw!m��^m���G�B#�e|wJNk��u&���%+�`e^,��$�m?�h���|`l�Z.R��!����Gp�z���G$@$@$@�	����c�^��B�/vn�����e�H4���'<��0��}4�]���c�-���Q8�A�I�0����vh5��V1�3��V�"]+9#t�b�U�k<$   ��P���~G���zA�^z���]��K���a�!c����x���Z,�u�(4.g����!;9�6% )f�-C��;�,
�mB-��|
�yE��Rh����:&   �45�l�#;��zj$���r1��:E6#����>�M�E��>�x	.2��W08���������$O���iH��DT�.���"�;���U�QP����.g��g����h��$@$@$@$��.����7y�K�!��Z
^>?R���17�Ru���;�����13

� C�,���S������[o�<5�������0f����-yX��6OI�H�H�H�c��?T�P���H�8�����Web�.c����w��;�"V������w�����������mF���PRe��7�\�jM!  �����,��E��^�=�W�O��4���ewM�n�����?���h�\��'���	�	�	�(�����"�cG����$$3L�	�%4
M"
����%\p�K���I�H�H�z1U�i�^����f6�E�NwJ4���ns�$@$@$@�J@�������M�K
�L�������	�	�	�	t�=��%���j,xw[�J�}�	K�	�	�	�@�=��V�0��4z�)��:�l�H�H�H�� P%4���8���z;J;o5
MGwau��9���(4�\���7�YGUSS��7oZ��u�	�	�	8#���q���{�U����3��1�=�^���
��E��F��N<4�����1x�`�W'�du  p_V#�������t�����n���vSu�����U�x�Y�4����7�q���������	�	����y���\���
���*���x���g���������i���inB$@$@�A��P�����m���x��^�pe����$PQ_���������g�B$@$@$���R�)������
���gc�?N9/3T�,���z��A�j�{�>����FN���NH�H�H�����������W��d�O���W�=/��Y��qe�UC�����y���9�Y�=�|�  p[�N"���f�t�X�����?c�LT�W���7����L)�������	�0����zz+��Y�A����p��������I�H�H������Ki��;��g��~��n�������?���|�
���1vVU���A�!�g�+������*'"#��os�!���w���#	�	�	8)us�[�!�E����r��^�Z$�w��U%{����~c�6t�r��O@�B#{���%��w�;�y�	�	�������Z�)e5j*xWI�*����o+���bA�����F���H�B#�r�����w���c   �^L@]�:��Ho:]K�Z�ig;�Z_�G��R�?�S�]#���9:��{�x�(4r.�
����{�9�	�	�	8{<��;�a�+���X[�ig6���xJ��Q��H!b�Y�xw�����}��	�	�	�Uo�eo
�SDQ%6���?[s
��
k��S|��:�	L�7���x��5��=��stA�9�GA$@$@����/d&m%��w�E�����S/�hu�2�?�M�����i1�	���p�4	�	�	���P=����j(Ig���y-�.x�7�-|��}$
�v��3W7r�Cq�(4���@$@$@���Al�s��K�*�im��x��BS&vX}�x�v-Oz����>��"Z�
����@��h/���]
�N����B|���V��w�}V�<!  w%`�M���7�����hn�W|�����x��Xn����sp�6��Cc�����|�a�x�J��<��\�xx�N���k6�h�w��|�c��Y���~�s��	�	���P��r����5j��x;:�5����?���O������<x
�o����I�������������K�+/���8�_����C��M��&�@�Q�\p���HIIi9�WI�H�H��	X��$4A?���7r�\���so���x"�a��4�7����sN�����x�
N]��c��
���	�	�	�@�P���l����)�y�����C���H�u�,n��5�C�����	(�=���J�������B~P�k��{�&�����I�H�H�W�4�mi�U	��� �$�o��TXS���;��'�.e����<��fzr=5���������	��)���e��h�;��  ��T	�\��+��m� ��NxRu��0Lm��l���������G�(&w���5"r����6�������e�ky�
���P����V�������?������MN�H�H����*�`c'������R�����7��������K��h&�C��A�Bzu�H�����VF&������m@Q�>d�m^[�c�S�!G�&`��0�����k�5yF$@$@$��	��@�1Dd[�U7r�e��f$�*������X~q�������������1�O��i��.K�S��;�������{�t�7FF���V�{�������3�2)�C|1o�V��Y<   ps��n|7�����?�P���Xk6�Gx��������4���Y�5}�^�utjn����5e���am6���-����TU��E%��������(C��dmJC���3W���<���B�8 �'  h���z�������,��j��[n_�U�����7����Q#t���}�����[~��4:&��*K�U/�ptj&������k��0u�>�("C�[�Z�@�G��DL�������YX�6�nX�M_|�2^&  p:����-��F��X�Z-l��5��T�0��\�E�����Bijj����8�zt�i�u3�b3/��;\�=X����'����o��]����xqo~^���n}�!  �^N�Z�^��=xYP.d�d�!�vn7V)�y�z��M�c�.U� 
����bEH,<�HUa��P6;�"Vm0��~H����~������P�
���~2R���'�,y>"��z-BC�6`?�$  �=T
�e���f'�J���h��U����t���K{�SUl.#vQ���8��w��H�=T�SJh�M���� Sf���!�
���3��K����Xs��+�����5^   p#�0�v�tW
��x�/�����v�1m�$1���UbC�����L$�^N$�i��������`4�c�`���(?���<y�ac�\��Y9�Mj��H�H�H��	��v��fN�v�^~2������{D� �m���,�w{!�\��'��������������_�4����������=�	��x��K������9if�	�	�	���z�	d�A�t{4��B{���I��[�*.t���v��|�nY��n��)v�{��l���*�q�E�-N8�>�Ud���|��-�(<0)T)�8B��u�-���?��sl���j��>���9OH�H�H�	XF���o
%iY�-v�J?����)E�h����?��~����<��F���l:#y!�%�������,���F���K�p��C�����n����+�����l��.	�	�	�&���eIU�n��Z�
���/��d�K��{�~����3��T|�$��dc��?JJ��`���-�c�z�(Fzb�2�G^l�tw������}�Q���8�ovJ$@$@�L@���1�+��Qhd�%��]���`�A�#��{F/@���3�@wP7r�w~
<:m%>^���El*^yf"�?��L���-y��4��xD$@$@$5�Da��F����F}
j��N-���������VxB�j�����h��������X_!�s��o6U-�.JG��?@We�`�X "��Vl;��q�"<�q��a�  ps�����
��F��:��^������.�R���������'��Is�=A��� �5�����_8�}��/�U�o��+Da�i�'��2�	�	�	��5u!���}<m�vMU�4F	�<��$-
��\��W�W)�={F�G�6�T�o$���)��M�~&�,���R��g#�q�,��^@�����H�H�H�T��^���g��Wu����ta'^-�����`a�/@�w���I��	4F��+/�l��J��`6�������}G#aI�r��Gt��{'  W%P/���r����X�{3���Qn��9�r��_)��������������D��@2��	�Qh�� 
�M��'{��� }�|L�_Z��Z�c��{�qj�8���'i�w���I�H�z=��7��������(F�+e�{��U
��X��-�~QU��xOg��x�	�����������Sp!|9�f�0��"L�1�{m�����Y3��:\�H�H�\��q}	+�������E��/��0������4�m�I���Oe����8"�x���o��
;����a>	�����w�8�6<������@���.B7S�����������`�XY,�[�|M]�'�!�	�	83�2C%~|nn;��8�u�wD�X�2����R�~���h���,%7>��u���B����h�6��K	�Px-��d�l���*���D����{�H��e�o�b�+�.�C;�N��)�*��S��)Sx��q.7�H�H��	���n�V�%���f�C8~�O�7\��p��ga��:"��-��3@�`��`#��O�m_�,E�H����G<��s3�&{%��4c��a>F�����x�rG��P�����#v]'2����J�H������re�S�����y�7.Ro�F��G����1,Y�������U�A�2=@�,����lHh��4X]U�M�r�j�������s����Kt�������(-���t�:4~_�X��/Gim�I����$w�e"  �.&�����
5��� /c��2�����Jhna"�C�M���<�d�"�z�@I�"�L������8�����:������B���h,w�~J�5e|�B�� _L]�:�[��[�Dw���+�������7!�6�n=��v6������iG��&�yJ$@$@6	����^����7~�wV��X�H���w���z<s6�/�jwbB`("��2�����u9X4�k��E����5YX�q7�=0_��[�Z��w�d��������� �_��YR��e��G^BL�ZD-��m)����1k�$^�e��v��{�F��{(����N�]��s���UD�>��m�Y�H�H�z5����h�����r�"����0d��ZS���D�q��r����*�������.�n��/�����Q���ow`��B:��T[������LW�t�[�o��ja��I���A~\�X�K�/����+���o�4~X0���V�q�;IH@IDATx,
Iy-5�CV�b *�����H��mH{/�3>���sD�!3��`�<�#��z��/���;m�~��7Z-s��y<x��|f��=��C�e�"PYY�.�����b�������I�2����8X����e�?F�8���~�1�b'�sH�����q���92��Q|��o:d<��5	8����^���Z~v�sh`[���g����A������;�����d,OB�z����!]�}�M���6k������F���oC~�X8�?t_mB�����W���'13 
b���E��[�D�������e������1H������&6��_�i����;����WD�%ooo��������7����o�hX��	�9r�'������@MM
�?�<�V[�V�G���e;d8"���k���!%o@�PDh�g��N���#p�_p�8�[��B��`�
��(��7-�J8����=
>�Z��Q

��-�I-������������[��2S��7����>M�������;'cbh�M5)o�i����Z����Qh����"�����-���%1����2a��q~M�����-D5O��x��e���� c7N��������a�^��K�"%%��J-����Cp�Q��B6/��]������ ��h�/�/��Gm@�d���2�d����~���GTK�����Ms���Nv����3$
�R_�!���5p(�}��8��kp��i�]2������>c��;��_?�����e��?�LZ�;�V���}�8&��A���'�!�0�� 1.�w�c��LJ����a��X�rV��b_I"l�(�]���O���*;	]��A�f��$@$@��@�E������X%��^�@#��t��	��u)r��#�M���s���?�cuJ:�����=9C��F����?/���&b��?�m������ ,>�\��l=]���� n�\���]+9#t�!��6[a&	�	�	t������iy�������k��������W����Y< �!�.b��sp$���O�6cM�D����T����#Ev��%b�i�7�~Zt
�F���Z����!;9�6% )f��?l����M�e���@��Dv�~�(�h5�l�	�	�@P#��f�Ff���O��{U�-t�T�������Ky�2��|�����hW��w#K|��B��d�����&.;�MiK��4k����A��:���EQ�7Zjc����W��qH����E8��y,]#��%S���+,�K@N�tL�����LD%�j�A����R��Z<�j;
�D�������-t��,	�	�	t5��^����y3�:��r�t�N�������B���E���������D8Zh����;2��w���e
�������|�Z���s��Yxk�{����b��c��W�&`���3I���CZe���"b��R|N��&V�O�7���X���e�����GF����	�S��o�	LY<cV.���E���H���yJ$@$����+SS��:� �/���
��x�k�\f����zy/��qP?�>xiX<���N��$��<=T���n6F2/;x�q��9#Gcx�a�����������%�xS*��w"���,����_(BY��{jx�����P93��OML^�3�C���
mvb�M��9	�	�	� �z�����j�������/��'k�*S��i�a�]�4���(uZ|'�^A��"�����m��Jp�_a[�K��j4���\D�Y��GW��UI��^���EI�2t�����6����
��H�H�H��	��V=�b!�LeN��?]]�����On�`��P�1���8T�a����\��{Xz��� ��M����X���0�-�vS}a*~��Y��	S+�g>�	S����pZ(X�������NI�H����Z��=�W���]|7�k�����;�����(rN�	t�����%4��a6����`i�l<t�$��V�-�����kc�[��  pE�x�����^&g������obW�qe�}=��6t�g`4<E&��,��8�"���f u�2�y�^D�t��cNz�x��[���	�	�@�	�����&�{�u�Rvc�7�F���EP������5\��Uxg��DL[�0�`�N��Ur.������}����G&#T����'  �#`3
�I/�6�b�Gg�l��)�wNo2/T���{����[z#{ ;"�|��^B#7NJ������23�"I,b����O�������  pi�z��qp���&I��&���,.��@O.R�Y�dG
����C	Xj����h�`�������-8�w��N!����/wG���IO`���P�9	�	�	8;�F
|�_]�x9G��/�V��3����V��>b'����������,%4N���a�[
��K�h��QxD&N��)�f"ne�ag&���[	�`�6l�����z<r���9OH�H�H@__�j���L�E���gU&vc�0������+����8/�x���
�����m�!=7�DNH�RB�h��0��
�X�qkK
�5��c��h�3�{�1<��SN��qH$@$@�D@���1
0i������w�����x��k�d����&u�X<�z��7��nI��_��;��b����U�+3 ����E_�*7���any3CCCq��w���9i  �	�1�e
KC��� ���Pe������<���o�J]%����f�l<���n��������+A����QhZ����:��K/d���^'F��Y�`��"�DT}y1m_�1�������~�w�=�	�	�	�$`��oY��S�~�}��z����<��-���Z
�>2����;��F�K|�U������l�I�RG���ebVL�h�����H�H�H�Z!`�^V�^x�����wwz��0~r��j0����-b����c��k�O.G���_�`|xo�H��D�H����\�W��1�t-//
���b�Y��H�H�������W�0��B����qUHY�3�z�0^�����^����_��|��[�M.K��_�z�������t�����:���s  �^F@�/%2�",ckI�������T�c6����F��}(�m���:	H�xG{��<�{�/���U[QP����EG�ci�v�k�
'  �T��6���\��J�;$4�"��o/g���w(���>A��c�<	� `���q����-���s1&���^���r���r������9i��a�K�	�	��P5��E�QQ�=�],���f!�<�'���������D��#����5�I�� ��q����QvjR��kW.@�J1��x,��M��C��]��7o�R�q�~zk3��.����'��
�"  w#����ms�R/SWz���}�0�L=����a��`��7 [,=�N>0b*^�[�g����=�YX��h��4���MX�t6&��0��z�F$@$@�@�����I�����7�cq���p���x�0���(f���; �I�K	Xj�-�i���<U
"&�A��9�����{_/���e��/|�`��a�-j"��5�m�dn�$@$@$���xOx�1_-�QV7E(��{o]�W���/��$M�w�]�#w%`��N|8vdj��oV`x$�����k<"  �������JhB�^Q�G��E�MG 
eFJf.�k�����������4��s�������w,G$@$@$`/��>�����U	�lW���2�
"�L��2U����+8SS�<�E��2���������GE��q+�B�A$`K�����k
,D$@$@$��x�"V���?�">���|v��ED�:\y�
r'��ir�������n�a�3y�H��,5�N����3b   �V	�����J�/��F]�Ds���z�������q>�0J������8<iY��$@�$`��^lz��d���t����_B�,S��o��XxL$@$�x��V�V��o#)G�\�1�����]�/�{��}}��k$b��c�o(���K4L$�m,ehu�7���|'�����EJJJ'[au  ��L@���9���U.4��������<c����?��h�'�1_�	�@�����9�Yk��7��q�B�
2����E��P���A��$@$@�G@��������2	�w��k��3�jCp��0q�a��-��u�H������������Vm�y/�a����[��MH��K��s'V&<$ h�O�M]]5�ni_E'-���B���)/'��EnK@z�����8�6cJ�z�)������HKK���(��	��y��x�[�"��$@$@$`����(��w��3&d������gmWt�jx9T{<�.0%����*�q���_Qr]�AQK���6��W`��H����cY���Q#e�^�~��g-���K�����& h'����������x�������N�\5�������h1��|\y*;	�55������4�g����9n"�	���?���[���:h��b�Fi�_Fe�ai���s�$@$@��R�?������7�C;Z��S&�7$Uokko�+�@����������0�u8�#
�����C�C�O%�
V����VexB$@$��V\�������������~�/��C5�
5��SR=�4�]�Nr��N��C�o8������,�}��Rk��#X�p���B��@@;�e�y1�	�	��MW����>�����1J� �fF��k�!'-�EU1����DH&����FqX$`'/��8Z�D�|��b����v.B��!a�T����9X�a2�����!�1�{<2eK~�(���y��B$@$@�@e}
j����7�)����Q�����r�C)�����k_Z���� ��xB$�T����G�d�_���`mv23�CJF%�����@k8����������n�H$@$�}T]���i|� S�sW��l�t���](����(�����H�q>��([&�v�����K���xqo-R
�!����k5��Q�'`Bx0t����N������x�����	������4���=�.���w%� �����_��L8�~3�~��U�H�\����������(����
�o��9i4(>�a1��v������H�H�� �VltU�J�W���l��'�������A�EH�\��������/����c����H�H���@y]��!�%4�����W���ua�
�}_�Bi�s�Zl�H��L��]�-�V�pR��$z�����H�H�U�%�Q��H�|CC�SO����1��6��������C���oG&�j��>�'o�U&�)�
���,��7�*�%9[DI��#�_�,��������k��ms�|H�H�;��>Pu�j?�"V�G�����I� � ��/�]����W�?B�wg.�D$�T	��/b���2Qixx���k0N��?S�D�	�8����������X4��bGn������z$���:l�U&OH�H�j��n�E5���2CU�(5�eq�_u	��n���W����
���`T� G�}�	�UB��E�6vb|$��k�U��c�Ta����9�C$���9s����9W7?��O 
v��s���! �$�Jh����h-%(������>���x_4�|�H��93������m��Kh�%
v��3���X��
�Y���q��sU���2�nz�	�	�	��
�������w�Fs��-7��w�-�X��Z�`�O-v�����G���Mu��N���s0Kh�6���8cz\��B��Y8��������5k����S�d�������! pBj��1��P-�zg�����{���RQ!y����E��>U	t/��E�������mJh�#�����HM���H+x:�m\hl�d�'l���#�6�b6	�	�	4h���y7?�xG��u�p��
f�����g��8���opO�oGk�mz��(�#�b���fX��Q���q��
�a
��n��  �4j�[�"7s����#=�Y��bV�V��>^x9lM�k~,D$����er�(4z���j�&���������1������&Y�H�H�
���\�r�A^~8_[!x�x�w]?f6��h�a��'q7wV��E<&�#���w���Mn].���Q��v��(��E��E�����	�	4%�VIYV
%��?*��.�DJ��4��G�����S�9	���P5�������		����}�@W�r�8���nv9]  ���o����Z��RI�!=�2��z��~��Q�x�k�h��ovA�D�Y��7��k#�l�fWb���	�	��}W��nUB#4�2��"��T�G|�fT�=X�����w{�8\�^�0�q�8���iZ�������k��CN3D�H�zU>#��RIy]�������qg^�
�=��{F��x�f��`"0P=����-��v,��:�?��J����3e1��o�[L��$ ��&`)�i��e���u�F��"LeZ�����e�&��Y#�a�_XOca$@NN@�����1�L�%4�����Gx��W��/F��$Nt���1	�	�f�|kx���.�A�d�_�����>��l�O��$@";�DP�W\�*�`T�-�~6�W���b��VYZ���R��DA�7PA	H�$�$$$!�$��sg����2��d�����{�~��2�����=���l��j\)���q?�o����sl$@�$`�BC�a��d���� ������
J
�����u�*
QX \��E#�c]����fcrZ2���p��B$@$p^�&N����v
4-b��IK�YK��y��/���8�f����-Z�v~��<��t����s �p?�&K|K��wRRX�
C��6T*s�b�E!>>^|bt��*t���/"*�Z����%�c�	�=V���wYRM'��������t�i��^�	�	8"�4�AbAXC������
-cFS,�>z2
W^b�o��{/�����M����1�|�@��y����W���:(H��u!0;_F��i���3����I�&��a0vv.zA��"a�4���(b0J*��&�C�������{���o�x�,���6}-��2�O]��)K�f���������Pt�����1��*�l�s�\��=u�l����.�
'��|��W
VTVV�3g�4��p��#g(1Oc���p���6�>�*)���^,m��������#�����usN6�g`����S[����g;���0��ID�)��ps�����P�P�I�$�\���f�2�tZ^U��wWs�\ZZ��`��4:�g.��9wc�s[���&����y����_���=S��<���B����1r���#eP�G%�X��oI����J�����+A�<�@�0�� Nd�����Y�>Aj�D�H����2]x����q�b�o�x��ulu�����c


�Xz��@:��t0x�,�����m�}t$(�Sa��+�Zw��${2!��l��U>	�����p �H+�'�W�e���U!���Z�@NN�������r��&�#���i�!�{IH�EB�}0�e�;�����R���;��?�W��� ����p�����?mcoE���x���Q�7NxK�6n��y�02yr�/CZ�����SvW#EP��x�K�PY,.�B�%6}�����M�C�^��u[1��W���[������b���x����R�^����V_j�F�Q�cH�	x5��K@jM��Z[|�8�+�b�[DG������s5b���?���m`o�hI��n(r��t��|E�
�d�KB�`�����
c�y���~���s���Y��2w�M�|�0P|��wW�����;o_|��SJaY�����P������P�9A�����(_7	%�V��� =�;����`"=w������|�0iI[���N��wa���H|�=L��s���-7�GS'�C�Z���F�Lp�M�nJ
������8X��a��H�"�2M;vDc�y�cv �	gk�������]@�0��G�p��T���w��gh�,*1Bh�������`��I�H�)�E���/�����.bo�=�K�����p��>Z,��@���� m��'c��2�R�*��a��u0�c���/����+����ZS[��H��FM�J"���TFkh(�*��kb  �  }����/����h��J���O��Yx�������N����Qx�PH��M�]�:�5ev�Y���A����'L.seV=*^��dH�������xh���c+N�������uv�-O�����c�K��Gv�E"�h�i���w6��x8�����I�H�Z������x�v�e3'����:�!{���EK�4�+����K��5���$@$�$����5F���G"A
��7����E��;w�"���b���<a�R�/h_�N��a�	�|� ��4�������Y���9{'�����H������?]p����g�a�d�xo��3!��[�J��H�H��(7�5��Z'
���uz�z�����_��|!��]�����f�5���0{W������C
|����z�6wF/���7��]/�0~��!�������-4�H���y���xt�j������`���%��{����+0D�N����F�q�v(n�,�\������{�����
X��M:�x�0�H�H�E(�5�f���O��������"�`S�Ih��K��*e<	�	��rd�q��p�i���(>��^,|/
MH�l[2#�.�x�I^�
��m�/��������'?���L��,���M��H���&?f�ll[t#�M@���
��JG�0���8h��$@$@M `5�	k�Tt�)�!
|Em5&�xo����(!���!������v��$@>E�_i�=a#�������Q�	JLY�������r�E�#.���wS#�6i����,a�-�p�5��I+`�d���$��+P<�/�-7 0,=�����6^�	�@k��K���n����&�Nl�>�d�e�U�2�hU:kB�Ln��8U]�q�V����Z����a]��o�����I�H��X5�nmoc��o`�0������>�a�������\xno���hl����'�q=��o_
��@1�H�%2*�1��2<p���;s5�W5}�"�t�	����zp,�+
�u����rf;��x�"�K�z?L��	���$@M'�4�5���/���FN5�0�:�j��G,�6�_��A}���H�H��^����O}�r��R4��=�;X���D�Mwlo^�j(C����B�W�G����1y�>�_������4	����?��p�Ix;#
>N���S�!7=
�rG����V����g�H�H���J���4�=��j���n7�:t���Mw��W���jL��f�l��7����w���hF�	�8w�����68�jh9!�����vK^c�����J�Y����	$@$@�8U]b�i�^��(_�zmu�=0�PgB������a�E�2�����lRp������>�����
e<	�	�8w���������a�[�yB$@$�<z�tOa2�Q�@<#4����w��FQ}��Xz�8��U<�$@$�,~�]l�@��i62�<�
d�X�2%���]�7�db���@[�e$@$�Ql���aB�W�7[n�<w��*D
W����A��lN��M����������� �>�����������
��p2z�O��)���p����ei[M3�0���%��������m��E��\��H�8[k2C��t�A��i
l`��^wLW^h�CJc}�;�~�7�j+qSdKm�X�L# ��	(�C��5����1��v�3���
X(���i���:��)���o��W�C�����RZ���H��N@�L�?!�{�^�9�WGD�P�<�	��K	x�
���i�#���=Gq��_z
�{vt)HW7~��W#55���`�$@N@�L�����&�������g�}5&G��6�xA$@n@@�������h�+�}���~}���ld��v5�f����bk
8|��D|����<# p��U�5i��x����g�1	�	�e_k��V-��SW�/�����#	�Y�A+��0�l�n[Vw�����1�7��uxJ$@�#��J�^j�ok��y/�c���	�	�=e�~;�����&���^�����������!_�����0�H�H�aV
|�����^���dU��Y�We��c#� pCJ_w3�	���m�S
����e����H�H�5����
��j+P+l-��VO�
���u��L�6�KB�`j�0O�N$���[[�3�A2�
|yC;7�;YA����u
r���H  [
i��"����V�R�q�8�L�^�Y���B�s"v$��<c�%	�	�	��	������7e�h�Z�K$@��h��n$e>���
�>���w����Hg��H�H��(���u��b�jx��v�e8]v!�5/5�6
�/	�	�����F�������{t��^l.9��������V���n��g2b�9Y��	����j����/f�X�~��# �Rc��6\	�������{B��e��M���cwE�������z�yo`Z�kI�		�	x��w�E�
�<�c-V�����K�9"q���x��q�����
`c4	�	8$��gdF����e\��}]IJ����x�h/6�d�}A�Z�~��x<&�;E�?'�\2�	x6��S�Q�Pr]p���S������2-m%��,����D��	�	�@�	�x%�K������wG
��s����[�E�Q��64����v}����pW��=�{N�'�H���@L���{��C���y�}��wp_�%��3���&��4`��d\���1��$ �&����(�^�K3�vQ�.�]~*����6c��K7����a`B����C��~"$@$�����7P#���28�K�����	X��9&
�h�k��}1$�#�����m��{��8�Zx ��	����.�2�������E��=Lh^���Nm�h�{u�b�����=.�������l$@$��������eo$��
�������!���+,Q����	�	8I@�a��s�.����V�7'�������D���$if2�mwr���H�+X����
|G\44	H+Fn������:vP��Q�����i� �-[�M�6���m��5/H�H�1�>Dll�o���<��R�%��9����������D��llB�F$�����o���	��>��������������BuI�}��OxN����%%��A���
wa7�|3|�Am��C$�2��w��-��:i�J�����k�ew��M�Z���,K�o!�`�>���aXwPu�{�]#h]J������$�0��b������:���E(�����n�-����z�Bbb���]'�"P&m�y�-lIM���� \xc���4�z�w9^��W~[2xW��|��;(3V#P�����DJ�am��E$@nG@i�e��BV%��uG����� 8\�I��rV� �K����#�w���g  ��(��G����5�R��'����n^	�z4r�J�oK
|�_�:����]@�P�#Lfn���]�	�	�2�^2��Q.��P�8q)���@$@�I�Z�Fz��7����M3R�������{
��n����n��(����U�&�hB:������)�fNxA$@�J@������ �v����:�wx���S�:pg����������@���0�;�*L<�	4�����}b�K-O�R�����w�5���7�rw�s5�hr�m�W���\<x��P�a��� ���^��LDwP��H�|���/�������
����9Ko�^����b� c�l;��x��9lhgV_����j��p�"��i%�l�e���Lk��>�=R�?D�Qx!��V������H�H@G@���Q�F���4�����i��/Z��)��c~�9GX$O.&E�:|v���y  ��PB�\ )|Y�����|�i=m���!����^����d�k��cJ��������G���+(��$d�E$�U��Vk��QW����M>�m����Is����������yc��J|�A	�U��vI�(U
��~��IU�7���1��e����E����}�-�<,�f���p'��p���D5�#	�	�@=�^h�X_�}[����E��1�0�+m��'�������v� h����zaMEs���2j����>���\p������*����B��Q�I��A�4�I�H��!�!�0D���#�Vn7o�����b�=���2�H�%�4���b��0����F;��D�:S.R��c`�M��{���AK������������;�����~!�l��	�	�@��6�5�k�a���l�
�>�%�ENrHR\/9�	���]�k�j��v�?$@$�T�f$�f�o%�7�>O�_h(�t�^/��|5�E�M�#'�������)�6gZ��\���>OH�H�'`cB�!6���Rz��{�L�~`��:���Z�14J�S!y&�TW<�	�@�	(A]-�l)���i�gk�����������F�����,��������a`HVt��+�/p��# lLh�xW��X������r�j��<?
+&��{]6�!p���Z���b�N%�;_�g����a�>4�MD��s�=�?�
^
�R���iv����=h^�	���x7�/F���]��'�q�d������?�E:T�;��YH�|��d�6�&a��l����zp��S/���g9dW��g��������R�*\l��p�!�s
.��2�	�	4�@�����^h��
���DNq��;�W�8�v�����`��jTW��0/i�A���0�j���
,E$eJ��Xu�z
�XL*��Ih_�N��Cc6��*N��S��]�f+��U���S;_�������4I�H��N ��G�,�~~�
���� ���������Kg`G�a��1
AAa�BTT�ca�s;-^i����K����8H���#b�I�5�5�J�,Z�Z��6
�xW���o/����!o<;���xx>,9`#��������q�A�����$@$@M#���x6��s��w���t�Y���Mqu��]}3<���x���}s�5	��j�������Y�H��Yn4$}�w
��J�B��E5�
�Z�k�)e_�������J�]�_ �I��ba���q�M�$�I�H�����,|�A�"�K����*2���c0u�)~�;;��-��2�N2(�����V������ 2��C>6�nP�j� �l�e�r�U�WcW,r���8H���'������U��7�Zl����k?������9	�	�@�k�]���_r;�������.�X����G�?.#���+�d�% ��!��HdJ�UGg2��������U�^��x��{�M�
���>�qYX�M</H�H�Z��~#�Z��/la4}r���C�^�$u�`
��hxE$p��+�Q^h��Y�����7���`���KP;�%�	���v}�Ylt�Q���k��@�	�	�w�_W���## e��J��jC�O$@M%��XYN	��(�
t���ko
��D=���-Q��:3f$����`��1 4�>�I�H����G��7�aPn���9	�@PB��R	����������>��u�������E�� ���v�>��$@$@."���������X��=�L~f�B|�����6K>@@-��TB���1�P�4��D-���1q$@$���6�5.tq��	����m�F�l�4	�H�'S�1n��������2m��[�W��Z��U��]�)��'���*�U;�����KCxu���	�	�C@������|u5�������v�OGZ����5!e���	�@�4dF�<�x����m(���XQ�o�-��$@$�6~��Jym���M��m�m�T���+va\�P�]o�m���a���	�%�+�w��j���H���cym5~#\B�>�e��7
o2$@$@�M���Fbpb�{�k������C����W��_�~#���k/l��X	�@=���4J����GDI;�/K���s��m�q|W~���w�efm���
�u�c'I�H����YF\k�{��&�5�qM?\�j$z�U�xc�4$-���!]�/6O�A@	��9-xu#y�2=��K�����L����aQ��Ed@�4F�	�	�������!�W��Dl/��v�[��(),GX�H��PQ���C]o}��fcrZ2���p�L!hQ
-�T�9)�Em��NT��u��!�p����5�����]�����,i<! p6&4.��7�F�]�}���cq��&[Qg�U��EDE_��%���X:ABp���Gltf���� �3���1%�/h/����%	�@�Pzxu#Y*�c�<��&��_�����S�s���+3c�����X	�	��"V��B�V�j'����"��P�oh�6>Th�M&/��Q7j�y��`���`��Y�����V��^����0e	���{���n�s13���a��MxN�K���S�������b�����X��af _%��
|����h@e�!bs'w���}�����"G����c��NW�{��?  '�m�����h�,���=MJ��P�w9bG����r�����{�-����]	�j��R+[��y���y��4q"n��k0C,f-���!�s�m]L��ANZ����"??���������� b<���,��������pD+�e��J�,��%T���5]����k�-i�tR#~)��a��S���
�{�������X/�yB(//G^^
=�����8y�$�?��p���v��rs���V�Wt7�����P~��HJ�p����������=o`��4�����%`�y 1�3�m�*�L���������&{����HV�;��p!��Ce��>���H�..2J ���
+���m��G_�c7o)bw!�Y��c���K�{y�w��4�H����2�t<�>:WS�����Fr��a���%�Qu);�?��]_1�\������*�F��s���h/������"�1Q���d������b)�}����k�QQ<�@�	l�����C����\�H@p��4�{wh��|���a���y�����Z����_��'L�o�-�������������07=	�rS`�cNx���l�����4���P�,���%)��-F���//��2$'�zH��E@�����^h�"V���'��*d��`�*=&6\������EK�	w����H��C�
�����#r ~��b�����9	�	�@�l�����
n(��`��,�t����KL?�:������n��#�H������K�`91]V�U�C�Uy�4���H��(�w��r�Z�_���Q�k�cqM�>�>,��CU�V�+��o
���0�]�O�#���L$ �8zx���O_�>��LF,�oo��3g���..�&�o��,�>=[g�a���`��T\]���[����D�Je��p6�����3��$��H���5��^h���2C�8��Z�kr������j[;�@�M�`��;0$w	
�]Qchx|kv�u�	�	�����OLXW<�d	
	Aee%BB*���iX�{6���s��������1y��@K�@IDAT/�G��	0.h��.V.��z�g&��ay&��+��#m�����s ��'���w���o����(h���
eX��5����&����:^���9�H�[j� ��:�u@��G�b$@$@�B@��U�T�*��	M`O�OI��0oMC�����"
���~�������.<�������s��g���
���	?����tv�
V�hS�U��r�}����1��i�m��2��v�Q��XLf5R���-�G�����l�(�I|~D��df�U����}n������=^�	��o�W����7:��&�J;��F��HM��.N�5���i�o����U�Hf��HuL"�6'���l��X(�Cy���J0��{}[��fN��������A�h���(�����K� ������R��~0���!�3�	�	��=�p�<������6y����m���8i�����������r�b�3��W�)��=�L�5�p��&g������L(�%��xg�EbA����$C[��c8W�ha��H�x���
v�(�NiF�	�	��&�Z(���o�Y���	��m�M5��2+��ug��9����J0W��=���f0*�}zC�Y�E�uj��mW�OU���H�k�����	�����H��#	�	�	4D@�����X"�x�QJ+-�F,��f^=����WS�{g�H�����gs?G���]��?����~���1Q,J�/DRm�H$@$@��Kw�B��E��H1�|��^���i���Z�r]�j�j3'=���y�,�����V�[�m{�Qlu-B�_ �s-~��
�	������H$@$@M%�\I����������^X�&4�
����l�3��bz�z\�[���J������l:6�v{3�_E]��nC�`
��|�pl$@$���fN\�����	�9eV"����2R�2�S������h�"a#?�i�xuDO������?�s}�G$@$@�E�_l�'��D���~~�]E�������
�7���!
���/����b������QdV���qihW<(6\���Rt��I�	�	�	�4�����](��'��{�b���6��?���$�TJ+-��{�	�zXiH/����R�#y�T�@���� �?Q�2��H�H��������Mp�N#�������d�|����Z�������
�������������w
�#X{&�& h;��\��&4mG��[�8q"�������:_&Ph0yeQ��>��a�kCx��G  p���d��4� ��r��px��������/�a�s�i�&KOH�H�H�
	(
<������K$�*�&\��m6����0>s52���!b�X�:D���@$@$@�N@�������GmH@yeQMz����]�zr��KHni�����������H$@$@nM ���F�^h�z��9hS�����(�Ho,��b����x�`�6���x���i?�S��~�	�	�(����B��7�M�����/���
����W�+�Jj*1A��l>��u���nX�s"��tre��6	�	�	4���7��w���W#��$�u�����1��w{{*Nk���}^�~������>�	�	�@S	P�Tb�O>@@i�C�Qa���$��s4�+����/!m�+����c��_�����@>p�q�$@$@(x.bu��$�K�
|��h������)Y��������v���L\�S_��G����i�XI�H�����<�z�sX$�V�#�^�����������kq��Pu�R:]����A\PT3H�	�	�	����x����\F@��t�v��(���;k�����9�b~�6M�.��?�i��z���3�	�	��7P6��M�+���F�&	�@��"��a��	��5�
pa���"�w�m|]�����i_~�x�����b�$@$@$�z,x��rU��*�l� �4�R�.�C(pK
��%�(��35��H����u��#��40�&  �:���6�u�0�|�����4���w��_zf���}��T���6<�y���N$@$�;�"V��x�����(-�5o��_���#b�]I@>���Vj]�~�;Dh���;���j<��	^��J�N|`�z=������=��H�H�Z����F����@��$����u�U�8	�H,XU_����n!��M�v�gaS�A|v�0��������{?�����9  ����x�����s�������D@���>)xy��^h�����9k~�����
c�ae�{�. DE�H$@$@>A �O�[h���A��c��]�t��x�Y�>�|P|���A������=j���KB�����H�H�|�����X�����iB����"	4H@�������B#C�0[)��B��v�>(��N��9����bV�HHS��!1���Ekpg�$@$@�E@������&��%��"`kB#��������M�Xs�aQ����Si1����.I�CL"���   +��VZ����B�	�2eB#MV��]��WA������E�eB��x�F�R�C��gPl��0.��"��  �6�����)�xH��(
�t!)����j�-�^�^������EZ�Ua���KP�mF^�	�	�	X��OH�$�������]�>��v�-C3����������!K��;^�W��!�?��   ���
|
m���a	�"�BC�6l��u��I/5V��+I�@���[���oQm��,Le�7#���E�3	�	�	4���BC?�MF�$�����x�BRiFS*�\�
���(�Y��8\u;�N`sI�*�D��F���(��r���pr���X��H�H�@?��	H�l�E����
��F��;������x6�sl~�����y�Z�G���_�����v�$��	�	�	8A@i���	X�B�@�~��Z����>�������������"z����M���t���$@$@$@�P6�4�q�s���Px�	M��m��i�������1;g3*�L�11"�c�4F���7�zw�����9 	�	�@[���7��U�u����u�0�\G�������7s�W��Ug�]�2dVb���-�d����n��NW"�wlu�L$@$@�'`1�����`��@qM����E3�h]���=n~������B�iq'������n�����x�`���LRD<|�#��-/��	�	�	����&4���Mj}��7���_���a��k^�i���x���x�������K��V�Ry��-b�����7�O���B�0��uWA���Ix�y"&��HE�H$@$@$�J,x[������1�Fs>��{o�y�x~�	S�T�_En^Zi�e7�xy���WkQ�?���F���]�ax\��9Ub   ��!`Y�����xk�2`��v�m�Q5�4P;�}�����Om��kBX7���?�r��
��M�[�yxN$@$@$���Kfj��"�nk��N
|�\�F�_`�w���hW�8e��x���������G   �6&�4�5.�������	4��2-�k��\��?��7r�����$@$@$�3�
�+�R�����s�4�F*�BOoU����>��64�j����2�C$@$�-�x����9�8Z�@���R&<�� ��q���[���\�y�-6/�����������L0�A$@$�u��6�^7�PKP�3�>iF�.=����n?���m�o����n���]<z\�<	�	�	x3�x��s�7�a^06e>��R`(S�}�\rw[�R�v!T�r_������;O$@$@�@����x_�o���x���'�F��9c�����)#�����^ad���<,��H�H�|�����u���X��V�����g��x�����:��Lh��������'\x�EmWY)��F���S(�;�t  p#��.�����?�a����7r���Bw�����^_P��&4����G�>��Q���*vQ�K����q�1�	�	�	x��w�x��k�K$z��4�8xc�4$-���!=�f���z����]���>($F��������u�8.  �j���;��?�y�8]V� �u$.�)��%���	�'CE!
���h�t��p��������k��IcD����{�fN�)��GN�+f�~�����7����'[$  8?���wSx�\����x����������"^��EDE_��%���X:ABp���Gltf���� �3���1%�/h/����e�8[k�hU�[i���y��:������kP#��A������d)  p#JO?�v�rx�9m�f���)����j���\$�B���u���x��=���7X<a���v?��^����0e	���{���n�s131F��b�&<'����SU����w�a��e�M�2�af�K��Lhr
��8�,����U��N���$��Pw��!  �(�_�t���B�2k��)Z��6~?�~��_�Nx��/&3C�0�1�k���05��JK�6o�0��@Z|v
�o���e� 5q"�1���k�)�1��eM}����?������M�G��5l�|��A�_�^���f?w:DXq�������UN`����g�rQ�g�R������[|���h�3�G����*-//G^^
}j�l�8y�$�?���������w�M��]
T�ZT~��?�w���<���H���1o��Z
� '�4����#6i���#eP�G%�X,8��:)���"��;��p!��Ce��>H�6�H�.N3J ,��_���m��G_mP��J�cDD�������_��<�q�
�I�����@�+G�J��/��;�l�e[#9]]�y�[����Pm^���!�/3=���v��z������44�}��}���k���M`������:w���:X���7-���@��������w�^���:5�n(�b�����o�����b�J i�}���a�6oF&OC��eHKZ�!���i8�!���X���[�Q��4M�)6F$�6�`I
�EF"e�IsjW���q��a��9��ab�	�]�jk������Ql��b�W�S���n���H�`L��
�;_����_,M$@$@$����X��[cn(��a�g�k�6z*����E�r�0-1���0}n?�MO���8�qV�:-'��J��u(�ZU�*#�nB@	���_�\��^h�
�x��;����F)�/S���]���M��$@$@$@�A@��K'�
n*�buJ4X&����m/<���:1=;�5b[��H!�|c!�.��Jmr��=*��G��jl�y��"�>�cAy�L������YZ����Z�[�3fdo@�A�d�"����w�W����H�	�	�	�@C��.�����5���������7R5�}��C0�{�VxG�=[g�a��dL��=�*��a-�>�=�%�������>��[�J��=(
|o����Z�jc
�*���.�=���	�r7�?w����5�w��U�  hb��
�r%���|�n���������/Q�Q��]���:f�'c�Wc0$�
�_���`\0F�����&�V��o���>4_.���f�c7����)�(���s��d��[h�U(�)C��KF�@ui�wXr�;�2X7�,������z�.�H$@$@$�#�u���l{��W�nJY8�&�K��&����[���*:3w1S]��A!���*p���Q��m�A�_~��5e��*��L>�M1��N����S�6��gV�o�_�����*���?�&����*�G   # ���P�";x����!�/Pd�q8i���"1u��������gW��Sr�ke>#�d����?[>��.�������]6.!�n	R;_�=Z�Q�H$@$@$�Q����>����V��S�_��c��E@/�_%�*�k�u��exi��P��*����5l���w���c�E�h��H�H�H�$�"Vj�yO���	MG�FR~rkJ[������1��Z�Ri�,*D���&��?.w����^�	�	�	�Vo��nS2>��oS�l���5��h��?��
R�����i�����a�����b]0{���<	�	�	��w�����w�-G��^n���i�e���F2Ch��?�6v�gk���W/������
  �vz/44�������L@i�;��je�n�2�����'L�Zg�u/�.��N��Z��C$@$@$@����E���b�/8[k���C�I��6�2H?�M
�bq�cY�
�!�b��K�c�p���V��$@$@$@>L@���~�]h����;���O����K�k^4��2����2D���M���,��Da2s���V���]�����=4 �C$@$@$�4�
<Mh���^/^��_��S9996��h�	���	n$�}�'s6[6dz��p���m���k�f��H�H�H��	�x���{��0g��C����+g4�rc�G�����{�c������w���A-�Q�H$@$@$�3l�����g��u���>Z����H���
��������T�����G%��k�{�����7d��#I�H�H��'`���
�����'Xx���4����L���S&�./�������B���efE�{h2�wI$@$@�O@���o�]h�
�l�)V[/4��4����������Dx��n��x�k�|�����s�=f"   G���!��4��������Ch�."U��]?<���"��(I�H�H�H�E���P�"LY���6���X������	��T�������'���]��{���a]�k�!   ��&`cB���4_���������I���qR�K�w����).0{OB��2��H�H�H�Z�@��4��E^h�w��*�e�$�<e>#K������0?���m�Gq���8'�F���#
���eN   �f�U|���fsdA/$`+��l��0�'�rC	^>�Cu�_���.
���8$   w#�O
��M	��.�3��}���H������!���t��   �r������r��g�������������Vh�GG
�����$@$@$@M@���q�
<��{�-���Wx)���Y�?��kv��r#��h��		�	�	�	���o�l�#	(
|����=��G��&  �x���&4y�NViS.$�s�	�	�	����xO��z�{��Al���&��[n�������[�?d�%�i��E9_�9I�H�H�H��	����FN�L������o���cS;xN_�T��{2����3Z�����(���3#	�	�	�	�6/4���n������^,�GN�4	s��q�s�K��������Pf���S���q�!�?����$   W�{����S�6]M�����N���}�uE.Z}���5��]c�$@$@$@$P�5�u�0�W�5x�`��n�	���A!1H�� ��t�'	�	�	�������P�a���6������`'�u�[�*��1��%x���b��		�	�	�	���Zn��n����$�\�9<���,�e�s��_��z�������.   h6~��E������@�0�Y��
������J�3�����1��5�!��k;��I�H�H�H�	��j��@�Y��@Em5>9w�����2k�
���:
Cj���'$������	�	�	�@}�xz����<�@z�)�9���gm�~{��w;.�U.�   �<R/�������c�m	l������sb7Uzu��Q1A,P��]���"   %�xj�=t�m������!�)E���V��w���H�H�H����t%i0�xo�X_������b3&z���^cpXW_B���	�	�	��fN5����tE�l�{�����n�tYh7l�=	]�"�g�		�	�	���^'�����H���
����~�}l�=}�E5x��F�Z��6�+�������10������	�@�&3�}������1�H��	�E�2��������-���S�pZl�$�5�=�Q���5c�@H�H���B�f�������b	�M/�K?�:�_|����mz���O�\{��������6`{�	mHr���+����!�2�	��W8��:���sk,�H�|���^��H;���'���
�g-�]<��3�7��E{,v^������8��	��W���s�4G$P�����G��n����3�z��.�:��/�1��L��Q�k���P���'p0�	���8��:R��5�g$@>B@Z!�@
��L�'
����?��&�����$���p��x�4��$@-A��8y�Z�)aJX*�E�����H�gH?�2�Jok��3�9PG�T��#��hu!������x�����c:	��w8��u�6Ba�J-�w�5GENP6�rKW
�����m�
���A��E8Re�_��.a23��{���	�@+����Y[:�'�%@�������L9�>�5{�	�����Aa��@$@>M�F�������&���p�$�s\mBC/4>w��pIM%�>�����yC�Q}�����	��[�1$@$�k��LL7 8�:F�����t��U&4�u��k���{��v��3�q�P�
?: �w����$@$@fJ���" H�{!x%�	���Pn$k���������'%�c���6��������������;����s�!�x�E�._?}���Vt
?�6X	�	x
��j��!�p���mM^i*���0�*B$��"V����k�\�[e�*z�����+�����7��[�U"�N�Z�-"��:-{=��;E������3��`K�K� �cf\���xL"�
�n���gHNN�/�����c��i��y��m:!!!�������M�\�)��/C�A&y.,�;����� p��<�����fff�		4�����|�b�
�Y��Yu�v!�Ij�[����+P�W�r".���������}�}���R,l��-��J������"��&7����P�f)�-0��G`r���i��		�	��e>#��	���WA�Q�W4x$�!`�����v��N7��x�������X��F�����4��TX�{!�K�"!�+^�>�z����-��4kn�g+����{���X"Lq�|O�^O��1>� �uz[w���������H��	(
|��l�)����	HC2��!}�"`�4�z�a������7���V�@��Lh�eX[���?���]�����6}��������id�$@$@NPBz�N@�X �=k^������YH��������FN������b�hz��W1fp_�5��'b��1K�vLi�����}�,m?�i8��!�--5*j����'\wx�g���\�-��G7chcE�;��� h
%���*�sn����H>E�~�]<�gN=���V�������bGF!�
i�>77���w8�A�t�\u���+qsl9:������O���p	s[�@���%������,|]��`�]���bU���Z��gk+p�P�l�9����;uXM-�����%9�G Ph�^������K�����S��1�0C}��!�1/�@�x7Q^��'��* ��N���rQW���22��|�c���u�#**
7�p�)M��oW���C�O/��*��l[������>�q}��7(?�n�,���'S����e����M���TY�/��t%#Lgr|rA�l� XH�r�&VE8p�|�y3R{�T�b���ck���W�)��y�C��lQ)>�b�j|Ab|b�$S����y�d�ENe>.�q��`��d?/[�i�������_�i����q�Z�%�-?�_b�O���?����0��"U����'�k��4RS�2]�P�T��o-������}��G�.�9�Lt�E��Dd��\�O��_�,ed��\L�L�c~{�
��/�B�W��a����,��������M0_�x9f�-�X�v����}��"�����@�2I�����h|����9M\�����@t��-��8�x�X�8�=��e�v�
.����'�s���s!�@���������c��q#R��ie�>���~��C���G��{��=&X)���Zk�J�����+��������W�#L�z����~
d�>g��oi[�)���o��9sfc���h/��9�s!�"�r��Y�QA��H�G����:o���6��,>#���n����p��
�3#U�<����Ru��s���#b����}������n2UdYD�����0
Q ��7�������O�m��,�X�32�d$�#]=�?����7f��Fm<�TuyM�����rv����~�?��S~�[}����W��y���A��c������YpP���M�����'�R|=��L3+/��|����UY��\�(,bu�T�2I��C������'�`�e:��*V-���wDe����;:X�F�qk}��U���n��_������'�yk�j������?1j�(\|����'�k\Y]�C�bg��������f2yvX�9CJJ
�3f8[D��g�I%�,�����0�7U/@_��I��=K�0�v �������/����S�F��|���+���/����8  7"p����u�x�G����z��z���	����w���;���n�^�]�v�V�Z����1G�,�:����B���������`�4�/�����A�������ns�\ozKD�o��Ka��*�	�^?�����Y����$�I���Z�e0��5	�4�B@���~R����(50��V=�w�]#/'`��D�{&#�g?m�~�:�v�*��9��#7FD��G_�E���Q������y��98h�����#��y��:���];H9q���N���.��gC-zI|5����-8Z�����F�����i�|F
��������f�0�K�
�{��[�����/^}*�D���RPV_��(����W��+(���2�J�Q�=�����P����o����q�m��^��������U)X�WD~�lB�-79��Z� /���M�h�����?��0{��4-QA��2N������U-^�[�*��*���<��J��Z_�'�/�~����������l��
e�L���W��O�q�#��9V��|M(_�v?��s,xq�>���_�U���U��(�?��X�W�!������|
)_���G�g�#������g�xq�����Oy�m�������2w�^���2M�Ue[�o�]�*X��{\W�IOG��L�k�o�;���}�Ms�����G�J���}���I���E�1t�5�9P�����>�ku�����5�����}}Uy9f1��zc�f�~�W+__^}�:WG5���&��s`nK���_��� 7K���3&�?UC��=�O�+�]����o���aW!D���eK1h�0���/�iA��u�{@|�h���gy_J���E1�ed~U�<��gey9������3����H��H�m�>R�i��1?2�2)S��G�8ey���w&W]y�9����?}?M)��~Q�N�aS�x���)�9���H�O�-�h<��(��@�nq�1�h���}Te,se�(o�O�_m>e�2^�U|d?���u7��g��4Fa�#���#>2��,��Ws-�����~h��y����A�U}0��8��_O������{�,r�����xGy��r��������������b<����1���M��<Vk����K�c���K_�����>�|�M}.���/� �y����~�TN�O�G�O~��{"m��{�2u*��v�H7�W��G5��n�Q��6�Q^���'��	e�����?�����*�:ZSLgr���V���y���������r�l�[o����~=z���%Kl[�j��	�&-7��7��1�jk@X;m�L�:y�"X�����)�s�8�o?�]{�EI�!=�C������:�	J�o���>�H���~�s��SV����M�Z(�/�2��������?d��P5[�7�	��.o(K�����������SS:������O��e�5��y����A�������#�(��&dW����V#,L|I�Q���~w6���Z�dF���q�t^�����{�����W_o�6�='��r��v�x�+�������<<���Z�1s���CV���![�tfY�|��/6m�:���K�W?�i������L�x�|��v��[��Q{hn�6������'��ko�	[���=;(�>���6k�J<pd������;�~�N���g_k;�h�����3|��v���b-���'�5������Bi�D�yx���������tl;4?����"a�g���(��($��
���6"�i���M��>���oN�zD5Bc\-��R�@�% wa
o����I�W�(�7��)���'<�tU&y	�V�@��d     �"@
����<��v}a����I�H�H�H���xw�
��H�H�H�H�h��n:�d     �(�;f�$@$@$@$@$�6(���T�#$@$@$@$@$��x����H�H�H�H���xO���ybc��B�!	����<m���a*	�	T������b���B}v���t]V�z9J
K�����y�����*�����)~����qt��7NlP�$kfkq�X:Aa����Gltf������B�"P�g������3
;J���8���Q����UT�8|pX%z��p_T4^�i��Y�g-���!:V�7���{p12u����f��E<�@���w�����j���:�M~x~�������S��]7���)��G�^���������/�	��sadp	�����b�	S��;d|gv�v=[�K��F=������>I6.Y���J�,7._���U-�_m�8;A��f��n<��3�q���;�=c|�e+(�2���f��l�?�,�6T��r���2�o4}7az���Dp�n��g^N �P���w��j�+cm�W�wj�������V������Y���y������3]��J��J�}>[5O�F�;��M��	������y[(6.+��y�,K����	".y���ugI�		X�/zI�,�6'�����,����L�F��fI���(6+��@~���4�����o����B!��]<�9J��>?�b��e��[�8}�Hk�_/'������"�>]�C��zU�)n�q�����q6wMh���+B���u� D[�D���"���8����@~�j+p�<��<�`
�,^e!S���4�}���Y
v�
�-���9�8����!���6�X�Iv�?s���������/}�����p�nW/��@$RvWK��/��*��P])"����g0���2�yl9��g��������+�c��kiP��|�E�84v��7�BM�P����F�(���C����#��o�F���Q���'(Gv��g�2����pl�s�&�_T
v�{��TeK(N��[��_'(�!������K���%�/��_���I@�D@xH���j��p�Z������E��/������1�2q&�Je�}�X;�AL^)����}
��FJ���m(��i��y�j�rb�����Pt�>���<������4K����e������K�ex|y���D�#���6�����6�f�I�����Fq�w�!�������p�'B�����Q�����'��
��a��*6J%bT]�Q!�Q�&��T��*���z2aN�6�[�����u[�b���4�/pg+�rku�������'e�����y�y�����g���v��f��{k�S`��[U�pJ��Kjiy���g@u�b�[?���G]f�E�3o~�,�"�����J��H��������*T��Y�9OV,N������q$�����-,�ZS��_~�S�/������I�rTX�D������}eMfS����!2�U��{�Sy�L,��2�Y@`�@��&��u�L��p����j��L�����2�b�:v�[*I����7��-�W���3o>�w��-���)�u�z+�7tS������%7*�3.��>��a����f�L|�����rD����@���3;X@��<�L�2�c�L���j?V�u�7w��WF,1�������4r�.�/'t*~5{^}���m��1~���	�������U���?���f�h4�U|�y`�����=�qBj��u��6�.�x��<)�3�cesnie��	�u�S'U4tXC�q����F��������*zn�Y�z�������<l���������!{�)���]o��l�^�JMXc��F�qx��o����>����X��R��74l
t��M��nw�����jj�c��J�����f�:Um��D����5:6lu5�8�M��N����c/�#i�������ZM�:�Sf�Q{����f����3���>��/	�W��.'���j�
�N��g���Cm�=~�}�V4u����y#��vdj�j�p�'w��9��0[��y)���-'�77��r���f�����������[��.oo�DV}�{����N+��������u�?%�}9yO:�QES�\��<�3.s-�� �G$0ycDcS��,\���^}D/��U�tB�c�����QdE���S�����k��t����U<c���]Z��&�um���P��K<3d���m�Cj&��m�����ERy�+T2��I|����o�����	�q��D?W�V�� � ��nb��D@@��	�\y%h � �>H�} Q@@��"@?W^	�� � ��xHA@@`�����W�v � � �C��E@\�i3���[p�L�����V"g#+ �@h$����S`R'w.��>�U-h�u��\�+	�
����U��]����'h}r��q��2��W#jX^������'�@��H����@�/'p{�g������U��/�>�	�����}q��o��C�k.����~���U/�����O�W������H�9@��&P�?��)MM�������J����>���}S��w}OC���k]�Ck�=+���?�~B�6j��';}��'��w�l��I�up�a]��w9� �A���H@��@$���|�QD����ya�b���������S�w���O���F�I�����+����_U��b����i�O'4x�E�UUf�:�����k�w�*]�=�jg�����c�(v����N����:���������J���:���������&n������b�b;M�.(���:{|���X����C���v:yH;S&�����c�Kb�����������O�������L�u��w��m~�����
 ��?� ��#�7W[����l�����Jg_��Mg_z�.���:��RqoZ�5����l�h�5��Q�g��eRSS�����n���VTX�v5��lY>���[5���VEE�}��.��v*�j����J������R]��S�������4M�&��i�������9��cZ�+��� 0���:�]���{H�Ac��3M���jx*)+9���J*�+������n�.3F�:�
O��tI����s��_�]���saz�?�co�\�����I�K�Ag��oL�zz�c�
�o	�c��������w�e7O�z]�����K8%�yG�j�t���7��m��l����3g��>�}��v���WY'��3.�������|��zb
�Y	�IDATO�d�U�k`l�x&5��d�<���9�x�v8
����r���',"���#@�������@�����V;�-.A��R��mxUef��"%���v&)�������>lsR_5���2��A�x���^�S������*�V���g�c�
�GA�����5��!w��?��3�bD�_����u~�W�>�M��A������&���x�����#-����2��L��[���T�H�zI����G9}��w��+�)�����'�p�~t����i���������"Z�#s�>"����t�-Ye/�������*�F��H��7%" ����w�w����L)�Z��5�����������?�����q�/c����zs��������T2�Q��{��m&_�W��Kj��*�=�}}^��6�17���7�J�7�ti�����g�_�S��3e���S��������;���Nl��\���{���g���:+^����f{im�V����.��r�2f �@$�a�Q��t���nr��AU�<��U����T��e&]�r�5����P���k��.�w������������D�������������)S�5���N�����B�y:�|qn�H�S���K�{��M'��S�l��������K
=k,"��x������ ��
�Nz.3�	�p���r�&�������������J��Z���r*?���3����W�'5���b��9�b�g���~���K���w�	]|�/�����������{��fq��V��?�[=�t����2����xU�=�yb8�6����Z�j���>S�~�0u^7uy�}��K�(>��gS�=��IO�O�,"�Ap>hQ�!�sM R�����q��3TfZ=G�P��7����{�z��_s����}Q�}$�u`�K�\�O����V����� _d������7��3��S-�4��A�#��7N������J�^p�����?^���n��c������Q�wO��O���g���g��g�{��s��~z����e������ �@$� xd&�N��$�3��ov8[;�0���Cjo?��U�zi�{#g���Q�
���i���i���7�?�j�KA���t�����W����|�����17����X�}���|bWT��G�b�z/_���j�;���D��mt���.7e�����:T�T;N�7�V���
�~?J�;��.;�Qcr\U/��3�����g�������_�
e�K��X�F�z^J�!����Zc��s��9>�����}�9�����f.sw��+�����M��n�������q�>�N�Yiu��y��Ag.u�������u�)-35���ze�50��0�o���M�<�������OY��~
u5��/Zcug�����y �V�G�f�X�����\��G�U_����.���MN|3��L@� �
,�c�7- ��B`R#WF5a�ZX�L+J��ig�{�Z�����C��m�^�5>2�O&�P����h�*�)��@��&nid�\����R��T�w�U+1�+W?1�y�������������8=yCC���X��b������cf�gB�;j���1������;�%@��H����@��"������~���>:�-%$��W~z���V��o�u*y��o)�@�`�#Q@�'\ _������������F�'��������[w�,?�I�}�QH����8�GE��������2s���
L�W�^����3��� �Ch�&> � �
p>@LB!� � �	|���G@@ @�1	� � �@�$�a@@�H��$ � �a���-L|@@ ��P � ��-@�0�@@P�>@LB!� � �	|���G@@ @�1	� � �@�$�a@@�H��$ � �a���-L|@@ ��P � ��-@�0�@@P�>@LB!� � �	|���G@@ @�1	� � �@�$�a@@�H��$ � �a���-L|@@ ��P � ��-@�0�@@P�>@LB!� � �	|���G@@ @�1	� � �@�$�a@@�H��$ � �a���-L|@@ ��P � ��-@�0�@@P�>@LB!� � �	|���G@@ @���3�����IEND�B`�
twophase_replay.diffapplication/octet-stream; name=twophase_replay.diffDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 8a22836..7119f22 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,8 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *
+ *		The same procedure happens during replication and crash recovery.
  *
  *-------------------------------------------------------------------------
  */
@@ -1248,8 +1248,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1292,12 +1290,30 @@ StandbyTransactionIdIsPrepared(TransactionId xid)
 	char	   *buf;
 	TwoPhaseFileHeader *hdr;
 	bool		result;
+	int			i;
 
 	Assert(TransactionIdIsValid(xid));
 
 	if (max_prepared_xacts <= 0)
 		return false;			/* nothing to do */
 
+	/*
+	 * At first check prepared tx that have no saved state files.
+	 */
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		PGXACT *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (TransactionIdEquals(pgxact->xid, xid))
+		{
+			LWLockRelease(TwoPhaseStateLock);
+			return true;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
 	/* Read and validate file */
 	buf = ReadTwoPhaseFile(xid, false);
 	if (buf == NULL)
@@ -1571,6 +1587,109 @@ RecreateTwoPhaseFile(TransactionId xid, void *content, int len)
 				 errmsg("could not close two-phase state file: %m")));
 }
 
+
+/*
+ * XlogRedoFinishPrepared()
+ *
+ * This function is called during replay when xlog reader faces 2pc commit or
+ * abort record. That function should clean up memory state that was created
+ * while replaying prepare xlog record.
+ * Actions are the same as in FinishPreparedTransaction() but without any
+ * writes to xlog and files (as it was already done).
+ */
+void
+XlogRedoFinishPrepared(TransactionId xid)
+{
+	int			i;
+	char 	   *buf;
+	char	   *bufptr;
+	TwoPhaseFileHeader *hdr;
+	TransactionId latestXid;
+	TransactionId *children;
+
+	GlobalTransaction gxact;
+	PGPROC	   *proc;
+	PGXACT	   *pgxact;
+
+	/* During replay that lock isn't really necessary, but let's take it anyway */
+	LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		gxact = TwoPhaseState->prepXacts[i];
+		proc = &ProcGlobal->allProcs[gxact->pgprocno];
+		pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (TransactionIdEquals(xid, pgxact->xid))
+		{
+			gxact->locking_backend = MyBackendId;
+			MyLockedGxact = gxact;
+			break;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+	/*
+	 * If requested xid isn't in numPrepXacts array that means that prepare
+	 * record was moved to files before our replay started. That's okay and we
+	 * have nothing to clean.
+	 */
+	if (i == TwoPhaseState->numPrepXacts)
+		return;
+
+	if (gxact->ondisk)
+		buf = ReadTwoPhaseFile(xid, true);
+	else
+		XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, NULL);
+
+	/*
+	 * Disassemble the header area
+	 */
+	hdr = (TwoPhaseFileHeader *) buf;
+
+	Assert(TransactionIdEquals(hdr->xid, xid));
+
+	bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+	children = (TransactionId *) bufptr;
+	bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
+	bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+	/*
+	 * Here we don't need to care about putting records to xlog or
+	 * deleting files, as it already done by process that have written
+	 * that xlog record. We need just to clen up memmory state.
+	 */
+
+	latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+	ProcArrayRemove(proc, latestXid);
+	gxact->valid = false;
+
+	/*
+	 * 2REVIEWER: I assume that we can skip invalidation callbacks here,
+	 * aren't we?
+	 */
+
+	/* And release locks */
+	if (true)
+		ProcessRecords(bufptr, xid, twophase_postcommit_callbacks);
+	else
+		ProcessRecords(bufptr, xid, twophase_postabort_callbacks);
+	PredicateLockTwoPhaseFinish(xid, true);
+	RemoveGXact(gxact);
+	MyLockedGxact = NULL;
+
+	/*
+	 * And now we can clean up any files we may have left.
+	 */
+	if (gxact->ondisk)
+		RemoveTwoPhaseFile(xid, true);
+
+	pfree(buf);
+}
+
+
+
 /*
  * CheckPointTwoPhase -- handle 2PC component of checkpointing.
  *
@@ -1685,7 +1804,51 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 	TransactionId *xids = NULL;
 	int			nxids = 0;
 	int			allocsize = 0;
+	int			i;
+
+	/*
+	 * This is usually called after end-of-recovery checkpoint, so all 2pc
+	 * files moved xlog to files. But if we restart slave when master is
+	 * switched off this function will be called before checkpoint ans we need
+	 * to check PGXACT array as it can contain prepared transactions that
+	 * didn't created any state files yet.
+	 */
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (!gxact->valid)
+			continue;
 
+		if (TransactionIdPrecedes(pgxact->xid, result))
+			result = pgxact->xid;
+
+		if (xids_p)
+		{
+			if (nxids == allocsize)
+			{
+				if (nxids == 0)
+				{
+					allocsize = 10;
+					xids = palloc(allocsize * sizeof(TransactionId));
+				}
+				else
+				{
+					allocsize = allocsize * 2;
+					xids = repalloc(xids, allocsize * sizeof(TransactionId));
+				}
+			}
+			xids[nxids++] = pgxact->xid;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+
+	/*
+	 * And now scan files in pg_twophase directory
+	 */
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
 	{
@@ -1696,7 +1859,6 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 			char	   *buf;
 			TwoPhaseFileHeader *hdr;
 			TransactionId *subxids;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
@@ -1804,97 +1966,14 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 }
 
 /*
- * StandbyRecoverPreparedTransactions
- *
- * Scan the pg_twophase directory and setup all the required information to
- * allow standby queries to treat prepared transactions as still active.
- * This is never called at the end of recovery - we use
- * RecoverPreparedTransactions() at that point.
- *
- * Currently we simply call SubTransSetParent() for any subxids of prepared
- * transactions. If overwriteOK is true, it's OK if some XIDs have already
- * been marked in pg_subtrans.
- */
-void
-StandbyRecoverPreparedTransactions(bool overwriteOK)
-{
-	DIR		   *cldir;
-	struct dirent *clde;
-
-	cldir = AllocateDir(TWOPHASE_DIR);
-	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
-	{
-		if (strlen(clde->d_name) == 8 &&
-			strspn(clde->d_name, "0123456789ABCDEF") == 8)
-		{
-			TransactionId xid;
-			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
-
-			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
-
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
-
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
-
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				pfree(buf);
-				continue;
-			}
-
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID.
-			 */
-			subxids = (TransactionId *)
-				(buf + MAXALIGN(sizeof(TwoPhaseFileHeader)));
-			for (i = 0; i < hdr->nsubxacts; i++)
-			{
-				TransactionId subxid = subxids[i];
-
-				Assert(TransactionIdFollows(subxid, xid));
-				SubTransSetParent(xid, subxid, overwriteOK);
-			}
-		}
-	}
-	FreeDir(cldir);
-}
-
-/*
- * RecoverPreparedTransactions
+ * RecoverPreparedFromFiles
  *
  * Scan the pg_twophase directory and reload shared-memory state for each
  * prepared transaction (reacquire locks, etc).  This is run during database
  * startup.
  */
 void
-RecoverPreparedTransactions(void)
+RecoverPreparedFromFiles(bool forceOverwriteOK)
 {
 	char		dir[MAXPGPATH];
 	DIR		   *cldir;
@@ -1916,9 +1995,25 @@ RecoverPreparedTransactions(void)
 			TransactionId *subxids;
 			GlobalTransaction gxact;
 			int			i;
+			PGXACT	   *pgxact;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
+			/* Already recovered from WAL? */
+			LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+			for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+			{
+				gxact = TwoPhaseState->prepXacts[i];
+				pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+				if (TransactionIdEquals(xid, pgxact->xid))
+				{
+					LWLockRelease(TwoPhaseStateLock);
+					goto next_file;
+				}
+			}
+			LWLockRelease(TwoPhaseStateLock);
+
 			/* Already processed? */
 			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
 			{
@@ -1963,6 +2058,12 @@ RecoverPreparedTransactions(void)
 				overwriteOK = true;
 
 			/*
+			 * Caller can also force overwriteOK.
+			 */
+			if (forceOverwriteOK)
+				overwriteOK = true;
+
+			/*
 			 * Reconstruct subtrans state for the transaction --- needed
 			 * because pg_subtrans is not preserved over a restart.  Note that
 			 * we are linking all the subtransactions directly to the
@@ -2004,10 +2105,94 @@ RecoverPreparedTransactions(void)
 
 			pfree(buf);
 		}
+
+next_file:
+		continue;
+
 	}
 	FreeDir(cldir);
 }
 
+
+void
+RecoverPreparedFromXLOG(XLogReaderState *record)
+{
+	bool		overwriteOK = false;
+	TransactionId xid = XLogRecGetXid(record);
+	char *buf = (char *) XLogRecGetData(record);
+	char	   *bufptr;
+	TwoPhaseFileHeader *hdr;
+	TransactionId *subxids;
+	GlobalTransaction gxact;
+	int			i;
+
+	/* Deconstruct header */
+	hdr = (TwoPhaseFileHeader *) buf;
+	Assert(TransactionIdEquals(hdr->xid, xid));
+	bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+	subxids = (TransactionId *) bufptr;
+	bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
+	bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+	/*
+	 * It's possible that SubTransSetParent has been set before, if
+	 * the prepared transaction generated xid assignment records. Test
+	 * here must match one used in AssignTransactionId().
+	 */
+	if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
+						 XLogLogicalInfoActive()))
+		overwriteOK = true;
+
+	/*
+	 * Reconstruct subtrans state for the transaction --- needed
+	 * because pg_subtrans is not preserved over a restart.  Note that
+	 * we are linking all the subtransactions directly to the
+	 * top-level XID; there may originally have been a more complex
+	 * hierarchy, but there's no need to restore that exactly.
+	 */
+	for (i = 0; i < hdr->nsubxacts; i++)
+		SubTransSetParent(subxids[i], xid, overwriteOK);
+
+	/*
+	 * Recreate its GXACT and dummy PGPROC
+	 *
+	 * MarkAsPreparing sets prepare_start_lsn to InvalidXLogRecPtr
+	 * so next checkpoint will skip that transaction.
+	 */
+	gxact = MarkAsPreparing(xid, hdr->gid,
+							hdr->prepared_at,
+							hdr->owner, hdr->database);
+	GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
+	MarkAsPrepared(gxact);
+
+	gxact->prepare_start_lsn = record->ReadRecPtr;
+	gxact->prepare_end_lsn = record->EndRecPtr;
+
+	/*
+	 * Recover other state (notably locks) using resource managers
+	 */
+	ProcessRecords(bufptr, xid, twophase_recover_callbacks);
+
+	/*
+	 * Release locks held by the standby process after we process each
+	 * prepared transaction. As a result, we don't need too many
+	 * additional locks at any one time.
+	 */
+	if (InHotStandby)
+		StandbyReleaseLockTree(xid, hdr->nsubxacts, subxids);
+
+	/*
+	 * We're done with recovering this transaction. Clear
+	 * MyLockedGxact, like we do in PrepareTransaction() during normal
+	 * operation.
+	 */
+	PostPrepare_Twophase();
+}
+
+
+
 /*
  *	RecordTransactionCommitPrepared
  *
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index b0d5440..5e9ad8d 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -5567,7 +5567,7 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+			XlogRedoFinishPrepared(parsed.twophase_xid);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5587,14 +5587,21 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+			XlogRedoFinishPrepared(parsed.twophase_xid);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		/*
+		 * To avoid creation of state files during replay we registering
+		 * prepare xlog resords in shared memory in the same way as it happens
+		 * while not in recovery. If replay faces commit xlog record before
+		 * checkpoint/restartpoint happens then we avoid using files at all.
+		 *
+		 * We need such behaviour because speed of 2PC replay on replica should
+		 * be at least not slower than 2PC tx speed on master.
+		 */
+		RecoverPreparedFromXLOG(record);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index a2846c4..4d5fee5 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6605,7 +6605,7 @@ StartupXLOG(void)
 
 				ProcArrayApplyRecoveryInfo(&running);
 
-				StandbyRecoverPreparedTransactions(false);
+				RecoverPreparedFromFiles(false);
 			}
 		}
 
@@ -7351,7 +7351,7 @@ StartupXLOG(void)
 	TrimMultiXact();
 
 	/* Reload shared-memory state for prepared transactions */
-	RecoverPreparedTransactions();
+	RecoverPreparedFromFiles(false);
 
 	/*
 	 * Shutdown the recovery environment. This must occur after
@@ -9269,7 +9269,7 @@ xlog_redo(XLogReaderState *record)
 
 			ProcArrayApplyRecoveryInfo(&running);
 
-			StandbyRecoverPreparedTransactions(true);
+			RecoverPreparedFromFiles(true);
 		}
 
 		/* ControlFile->checkPointCopy always tracks the latest ckpt XID */
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b7ce0c6..6b7bc7d 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -17,6 +17,7 @@
 #include "access/xlogdefs.h"
 #include "datatype/timestamp.h"
 #include "storage/lock.h"
+#include "access/xlogreader.h"
 
 /*
  * GlobalTransactionData is defined in twophase.c; other places have no
@@ -46,8 +47,8 @@ extern bool StandbyTransactionIdIsPrepared(TransactionId xid);
 
 extern TransactionId PrescanPreparedTransactions(TransactionId **xids_p,
 							int *nxids_p);
-extern void StandbyRecoverPreparedTransactions(bool overwriteOK);
-extern void RecoverPreparedTransactions(void);
+extern void RecoverPreparedFromFiles(bool overwriteOK);
+extern void RecoverPreparedFromXLOG(XLogReaderState *record);
 
 extern void RecreateTwoPhaseFile(TransactionId xid, void *content, int len);
 extern void RemoveTwoPhaseFile(TransactionId xid, bool giveWarning);
@@ -56,4 +57,5 @@ extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void XlogRedoFinishPrepared(TransactionId xid);
 #endif   /* TWOPHASE_H */
check.shapplication/octet-stream; name=check.shDownload
#45Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Stas Kelvich (#44)
Re: Speedup twophase transactions

Stas Kelvich wrote:

While this patch touches quite sensible part of postgres replay and there is some rarely used code paths, I wrote shell script to setup master/slave replication and test different failure scenarios that can happened with instances. Attaching this file to show test scenarios that I have tested and more importantly to show what I didn’t tested. Particularly I failed to reproduce situation where StandbyTransactionIdIsPrepared() is called, may be somebody can suggest way how to force it’s usage. Also I’m not too sure about necessity of calling cache invalidation callbacks during XlogRedoFinishPrepared(), I’ve marked this place in patch with 2REVIEWER comment.

I think this is the third thread in which I say this: We need to push
Michael Paquier's recovery test framework, then convert your test script
to that. That way we can put your tests in core.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#46Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Alvaro Herrera (#45)
Re: Speedup twophase transactions

Agree, I had the same idea in my mind when was writing that script.

I will migrate it to TAP suite and write a review for Michael Paquier's patch.

Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On 26 Jan 2016, at 20:20, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:

Stas Kelvich wrote:

While this patch touches quite sensible part of postgres replay and there is some rarely used code paths, I wrote shell script to setup master/slave replication and test different failure scenarios that can happened with instances. Attaching this file to show test scenarios that I have tested and more importantly to show what I didn’t tested. Particularly I failed to reproduce situation where StandbyTransactionIdIsPrepared() is called, may be somebody can suggest way how to force it’s usage. Also I’m not too sure about necessity of calling cache invalidation callbacks during XlogRedoFinishPrepared(), I’ve marked this place in patch with 2REVIEWER comment.

I think this is the third thread in which I say this: We need to push
Michael Paquier's recovery test framework, then convert your test script
to that. That way we can put your tests in core.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#47Michael Paquier
michael.paquier@gmail.com
In reply to: Stas Kelvich (#46)
Re: Speedup twophase transactions

On Wed, Jan 27, 2016 at 5:39 AM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Agree, I had the same idea in my mind when was writing that script.
I will migrate it to TAP suite and write a review for Michael Paquier's patch.

Yeah, please! And you have won a free-hug coupon that I can give in
person next week.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#48Jesper Pedersen
jesper.pedersen@redhat.com
In reply to: Stas Kelvich (#44)
Re: Speedup twophase transactions

On 01/26/2016 07:43 AM, Stas Kelvich wrote:

Thanks for reviews and commit!

As Simon and Andres already mentioned in this thread replay of twophase transaction is significantly slower then the same operations in normal mode. Major reason is that each state file is fsynced during replay and while it is not a problem for recovery, it is a problem for replication. Under high 2pc update load lag between master and async replica is constantly increasing (see graph below).

One way to improve things is to move fsyncs to restartpoints, but as we saw previously it is a half-measure and just frequent calls to fopen can cause bottleneck.

Other option is to use the same scenario for replay that was used already for non-recovery mode: read state files to memory during replay of prepare, and if checkpoint/restartpoint occurs between prepare and commit move data to files. On commit we can read xlog or files. So here is the patch that implements this scenario for replay.

Patch is quite straightforward. During replay of prepare records RecoverPreparedFromXLOG() is called to create memory state in GXACT, PROC, PGPROC; on commit XlogRedoFinishPrepared() is called to clean up that state. Also there are several functions (PrescanPreparedTransactions, StandbyTransactionIdIsPrepared) that were assuming that during replay all prepared xacts have files in pg_twophase, so I have extended them to check GXACT too.
Side effect of that behaviour is that we can see prepared xacts in pg_prepared_xacts view on slave.

While this patch touches quite sensible part of postgres replay and there is some rarely used code paths, I wrote shell script to setup master/slave replication and test different failure scenarios that can happened with instances. Attaching this file to show test scenarios that I have tested and more importantly to show what I didn�t tested. Particularly I failed to reproduce situation where StandbyTransactionIdIsPrepared() is called, may be somebody can suggest way how to force it�s usage. Also I�m not too sure about necessity of calling cache invalidation callbacks during XlogRedoFinishPrepared(), I�ve marked this place in patch with 2REVIEWER comment.

Tests shows that this patch increases speed of 2pc replay to the level when replica can keep pace with master.

Graph: replica lag under a pgbench run for a 200 seconds with 2pc update transactions (80 connections, one update per 2pc tx, two servers with 12 cores each, 10GbE interconnect) on current master and with suggested patch. Replica lag measured with "select sent_location-replay_location as delay from pg_stat_replication;" each second.

Some comments:

* The patch needs a rebase against the latest TwoPhaseFileHeader change
* Rework the check.sh script into a TAP test case (src/test/recovery),
as suggested by Alvaro and Michael down thread
* Add documentation for RecoverPreparedFromXLOG

+ * that xlog record. We need just to clen up memmory state.

'clean' + 'memory'

+	 * This is usually called after end-of-recovery checkpoint, so all 2pc
+	 * files moved xlog to files. But if we restart slave when master is
+	 * switched off this function will be called before checkpoint ans we need
+	 * to check PGXACT array as it can contain prepared transactions that
+	 * didn't created any state files yet.

=>

"We need to check the PGXACT array for prepared transactions that
doesn't have any state file in case of a slave restart with the master
being off."

+ * prepare xlog resords in shared memory in the same way as it happens

'records'

+		 * We need such behaviour because speed of 2PC replay on replica should
+		 * be at least not slower than 2PC tx speed on master.

=>

"We need this behaviour because the speed of the 2PC replay on the
replica should be at least the same as the 2PC transaction speed of the
master."

I'll leave the 2REVIEWER section to Simon.

Best regards,
Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#49David Steele
david@pgmasters.net
In reply to: Stas Kelvich (#46)
Re: Speedup twophase transactions

On 1/26/16 3:39 PM, Stas Kelvich wrote:

Agree, I had the same idea in my mind when was writing that script.

I will migrate it to TAP suite and write a review for Michael Paquier's patch.

Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On 26 Jan 2016, at 20:20, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:

Stas Kelvich wrote:

While this patch touches quite sensible part of postgres replay and there is some rarely used code paths, I wrote shell script to setup master/slave replication and test different failure scenarios that can happened with instances. Attaching this file to show test scenarios that I have tested and more importantly to show what I didn�t tested. Particularly I failed to reproduce situation where StandbyTransactionIdIsPrepared() is called, may be somebody can suggest way how to force it�s usage. Also I�m not too sure about necessity of calling cache invalidation callbacks during XlogRedoFinishPrepared(), I�ve marked this place in patch with 2REVIEWER comment.

I think this is the third thread in which I say this: We need to push
Michael Paquier's recovery test framework, then convert your test script
to that. That way we can put your tests in core.

It seems this thread has been waiting quite a while on a new patch. If
one doesn't appear by Monday I will mark this "returned with feedback".

--
-David
david@pgmasters.net

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#50Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Jesper Pedersen (#48)
1 attachment(s)
Re: Speedup twophase transactions

On 11 Mar 2016, at 19:41, Jesper Pedersen <jesper.pedersen@redhat.com> wrote:

Thanks for review, Jesper.

Some comments:

* The patch needs a rebase against the latest TwoPhaseFileHeader change

Done.

* Rework the check.sh script into a TAP test case (src/test/recovery), as suggested by Alvaro and Michael down thread

Done. Originally I thought about reducing number of tests (11 right now), but now, after some debugging, I’m more convinced that it is better to include them all, as they are really testing different code paths.

* Add documentation for RecoverPreparedFromXLOG

Done.

---
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company

Attachments:

twophase_replay.v2.diffapplication/octet-stream; name=twophase_replay.v2.diffDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index c4fd9ef..55352a6 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,8 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *
+ *		The same procedure happens during replication and crash recovery.
  *
  *-------------------------------------------------------------------------
  */
@@ -1252,8 +1252,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1296,12 +1294,30 @@ StandbyTransactionIdIsPrepared(TransactionId xid)
 	char	   *buf;
 	TwoPhaseFileHeader *hdr;
 	bool		result;
+	int			i;
 
 	Assert(TransactionIdIsValid(xid));
 
 	if (max_prepared_xacts <= 0)
 		return false;			/* nothing to do */
 
+	/*
+	 * At first check prepared tx that wasn't yet moved to disk.
+	 */
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		PGXACT *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (TransactionIdEquals(pgxact->xid, xid))
+		{
+			LWLockRelease(TwoPhaseStateLock);
+			return true;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
 	/* Read and validate file */
 	buf = ReadTwoPhaseFile(xid, false);
 	if (buf == NULL)
@@ -1576,6 +1592,110 @@ RecreateTwoPhaseFile(TransactionId xid, void *content, int len)
 				 errmsg("could not close two-phase state file: %m")));
 }
 
+
+/*
+ * XlogRedoFinishPrepared()
+ *
+ * This function is called during replay when xlog reader faces 2pc commit or
+ * abort record. That function should clean up memory state that was created
+ * while replaying prepare xlog record.
+ * Actions are the same as in FinishPreparedTransaction() but without any
+ * writes to xlog and files (as it was already done).
+ */
+void
+XlogRedoFinishPrepared(TransactionId xid, bool isCommit)
+{
+	int			i;
+	char 	   *buf;
+	char	   *bufptr;
+	TwoPhaseFileHeader *hdr;
+	TransactionId latestXid;
+	TransactionId *children;
+
+	GlobalTransaction gxact;
+	PGPROC	   *proc;
+	PGXACT	   *pgxact;
+
+	/* During replay that lock isn't really necessary, but let's take it anyway */
+	LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		gxact = TwoPhaseState->prepXacts[i];
+		proc = &ProcGlobal->allProcs[gxact->pgprocno];
+		pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (TransactionIdEquals(xid, pgxact->xid))
+		{
+			gxact->locking_backend = MyBackendId;
+			MyLockedGxact = gxact;
+			break;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+	/*
+	 * If requested xid isn't in numPrepXacts array that means that prepare
+	 * record was moved to files before our replay started. That's okay and we
+	 * have nothing to clean.
+	 */
+	if (i == TwoPhaseState->numPrepXacts)
+		return;
+
+	if (gxact->ondisk)
+		buf = ReadTwoPhaseFile(xid, true);
+	else
+		XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, NULL);
+
+	/*
+	 * Disassemble the header area
+	 */
+	hdr = (TwoPhaseFileHeader *) buf;
+
+	Assert(TransactionIdEquals(hdr->xid, xid));
+
+	bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+	children = (TransactionId *) bufptr;
+	bufptr += MAXALIGN(hdr->gidlen);
+	bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
+	bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+	/*
+	 * Here we don't need to care about putting records to xlog or
+	 * deleting files, as it already done by process that have written
+	 * that xlog record. We need just to clean up memory state.
+	 */
+	latestXid = TransactionIdLatest(xid, hdr->nsubxacts, children);
+	ProcArrayRemove(proc, latestXid);
+	gxact->valid = false;
+
+	/*
+	 * 2REVIEWER: I assume that we can skip invalidation callbacks here,
+	 * as they were executed in xact_redo_commit().
+	 */
+
+	/* And release locks */
+	if (isCommit)
+		ProcessRecords(bufptr, xid, twophase_postcommit_callbacks);
+	else
+		ProcessRecords(bufptr, xid, twophase_postabort_callbacks);
+
+	PredicateLockTwoPhaseFinish(xid, true);
+	RemoveGXact(gxact);
+	MyLockedGxact = NULL;
+
+	/*
+	 * And now we can clean up any files we may have left.
+	 */
+	if (gxact->ondisk)
+		RemoveTwoPhaseFile(xid, true);
+
+	pfree(buf);
+}
+
+
+
 /*
  * CheckPointTwoPhase -- handle 2PC component of checkpointing.
  *
@@ -1690,7 +1810,48 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 	TransactionId *xids = NULL;
 	int			nxids = 0;
 	int			allocsize = 0;
+	int			i;
 
+	/*
+	 * We need to check the PGXACT array for prepared transactions that doesn't
+	 * have any state file in case of a slave restart with the master being off.
+	 */
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (!gxact->valid)
+			continue;
+
+		if (TransactionIdPrecedes(pgxact->xid, result))
+			result = pgxact->xid;
+
+		if (xids_p)
+		{
+			if (nxids == allocsize)
+			{
+				if (nxids == 0)
+				{
+					allocsize = 10;
+					xids = palloc(allocsize * sizeof(TransactionId));
+				}
+				else
+				{
+					allocsize = allocsize * 2;
+					xids = repalloc(xids, allocsize * sizeof(TransactionId));
+				}
+			}
+			xids[nxids++] = pgxact->xid;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+
+	/*
+	 * And now scan files in pg_twophase directory
+	 */
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
 	{
@@ -1701,7 +1862,6 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 			char	   *buf;
 			TwoPhaseFileHeader *hdr;
 			TransactionId *subxids;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
@@ -1809,97 +1969,14 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 }
 
 /*
- * StandbyRecoverPreparedTransactions
- *
- * Scan the pg_twophase directory and setup all the required information to
- * allow standby queries to treat prepared transactions as still active.
- * This is never called at the end of recovery - we use
- * RecoverPreparedTransactions() at that point.
- *
- * Currently we simply call SubTransSetParent() for any subxids of prepared
- * transactions. If overwriteOK is true, it's OK if some XIDs have already
- * been marked in pg_subtrans.
- */
-void
-StandbyRecoverPreparedTransactions(bool overwriteOK)
-{
-	DIR		   *cldir;
-	struct dirent *clde;
-
-	cldir = AllocateDir(TWOPHASE_DIR);
-	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
-	{
-		if (strlen(clde->d_name) == 8 &&
-			strspn(clde->d_name, "0123456789ABCDEF") == 8)
-		{
-			TransactionId xid;
-			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
-
-			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
-
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
-
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
-
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				pfree(buf);
-				continue;
-			}
-
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID.
-			 */
-			subxids = (TransactionId *)
-				(buf + MAXALIGN(sizeof(TwoPhaseFileHeader)));
-			for (i = 0; i < hdr->nsubxacts; i++)
-			{
-				TransactionId subxid = subxids[i];
-
-				Assert(TransactionIdFollows(subxid, xid));
-				SubTransSetParent(xid, subxid, overwriteOK);
-			}
-		}
-	}
-	FreeDir(cldir);
-}
-
-/*
- * RecoverPreparedTransactions
+ * RecoverPreparedFromFiles
  *
  * Scan the pg_twophase directory and reload shared-memory state for each
  * prepared transaction (reacquire locks, etc).  This is run during database
  * startup.
  */
 void
-RecoverPreparedTransactions(void)
+RecoverPreparedFromFiles(bool forceOverwriteOK)
 {
 	char		dir[MAXPGPATH];
 	DIR		   *cldir;
@@ -1922,9 +1999,25 @@ RecoverPreparedTransactions(void)
 			GlobalTransaction gxact;
 			const char	*gid;
 			int			i;
+			PGXACT	   *pgxact;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
+			/* Already recovered from WAL? */
+			LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+			for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+			{
+				gxact = TwoPhaseState->prepXacts[i];
+				pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+				if (TransactionIdEquals(xid, pgxact->xid))
+				{
+					LWLockRelease(TwoPhaseStateLock);
+					goto next_file;
+				}
+			}
+			LWLockRelease(TwoPhaseStateLock);
+
 			/* Already processed? */
 			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
 			{
@@ -1971,6 +2064,12 @@ RecoverPreparedTransactions(void)
 				overwriteOK = true;
 
 			/*
+			 * Caller can also force overwriteOK.
+			 */
+			if (forceOverwriteOK)
+				overwriteOK = true;
+
+			/*
 			 * Reconstruct subtrans state for the transaction --- needed
 			 * because pg_subtrans is not preserved over a restart.  Note that
 			 * we are linking all the subtransactions directly to the
@@ -2012,10 +2111,108 @@ RecoverPreparedTransactions(void)
 
 			pfree(buf);
 		}
+
+next_file:
+		continue;
+
 	}
 	FreeDir(cldir);
 }
 
+
+/*
+ * RecoverPreparedFromXLOG
+ *
+ * To avoid creation of state files during replay we registering
+ * prepare xlog records in shared memory in the same way as it happens
+ * while not in recovery. If replay faces commit xlog record before
+ * checkpoint/restartpoint happens then we avoid using files at all.
+ *
+ * We need this behaviour because the speed of the 2PC replay on the replica
+ * should be at least the same as the 2PC transaction speed of the master.
+ */
+void
+RecoverPreparedFromXLOG(XLogReaderState *record)
+{
+	bool		overwriteOK = false;
+	TransactionId xid = XLogRecGetXid(record);
+	char *buf = (char *) XLogRecGetData(record);
+	char	   *bufptr;
+	const char *gid;
+	TwoPhaseFileHeader *hdr;
+	TransactionId *subxids;
+	GlobalTransaction gxact;
+	int			i;
+
+	/* Deconstruct header */
+	hdr = (TwoPhaseFileHeader *) buf;
+	Assert(TransactionIdEquals(hdr->xid, xid));
+	bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+	gid = (const char *) bufptr;
+	bufptr += MAXALIGN(hdr->gidlen);
+	subxids = (TransactionId *) bufptr;
+	bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
+	bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+	/*
+	 * It's possible that SubTransSetParent has been set before, if
+	 * the prepared transaction generated xid assignment records. Test
+	 * here must match one used in AssignTransactionId().
+	 */
+	if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
+						 XLogLogicalInfoActive()))
+		overwriteOK = true;
+
+	/*
+	 * Reconstruct subtrans state for the transaction --- needed
+	 * because pg_subtrans is not preserved over a restart.  Note that
+	 * we are linking all the subtransactions directly to the
+	 * top-level XID; there may originally have been a more complex
+	 * hierarchy, but there's no need to restore that exactly.
+	 */
+	for (i = 0; i < hdr->nsubxacts; i++)
+		SubTransSetParent(subxids[i], xid, overwriteOK);
+
+	/*
+	 * Recreate its GXACT and dummy PGPROC
+	 *
+	 * MarkAsPreparing sets prepare_start_lsn to InvalidXLogRecPtr
+	 * so next checkpoint will skip that transaction.
+	 */
+	gxact = MarkAsPreparing(xid, gid,
+							hdr->prepared_at,
+							hdr->owner, hdr->database);
+	GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
+	MarkAsPrepared(gxact);
+
+	gxact->prepare_start_lsn = record->ReadRecPtr;
+	gxact->prepare_end_lsn = record->EndRecPtr;
+
+	/*
+	 * Recover other state (notably locks) using resource managers
+	 */
+	ProcessRecords(bufptr, xid, twophase_recover_callbacks);
+
+	/*
+	 * Release locks held by the standby process after we process each
+	 * prepared transaction. As a result, we don't need too many
+	 * additional locks at any one time.
+	 */
+	if (InHotStandby)
+		StandbyReleaseLockTree(xid, hdr->nsubxacts, subxids);
+
+	/*
+	 * We're done with recovering this transaction. Clear
+	 * MyLockedGxact, like we do in PrepareTransaction() during normal
+	 * operation.
+	 */
+	PostPrepare_Twophase();
+}
+
+
+
 /*
  *	RecordTransactionCommitPrepared
  *
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 8a2cd45..a2a2f58 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -5573,7 +5573,7 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+			XlogRedoFinishPrepared(parsed.twophase_xid, true);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5593,14 +5593,12 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+			XlogRedoFinishPrepared(parsed.twophase_xid, false);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		RecoverPreparedFromXLOG(record);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 5b1c361..53e2ac3 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6627,7 +6627,7 @@ StartupXLOG(void)
 
 				ProcArrayApplyRecoveryInfo(&running);
 
-				StandbyRecoverPreparedTransactions(false);
+				RecoverPreparedFromFiles(false);
 			}
 		}
 
@@ -7369,7 +7369,7 @@ StartupXLOG(void)
 	TrimMultiXact();
 
 	/* Reload shared-memory state for prepared transactions */
-	RecoverPreparedTransactions();
+	RecoverPreparedFromFiles(false);
 
 	/*
 	 * Shutdown the recovery environment. This must occur after
@@ -9283,7 +9283,7 @@ xlog_redo(XLogReaderState *record)
 
 			ProcArrayApplyRecoveryInfo(&running);
 
-			StandbyRecoverPreparedTransactions(true);
+			RecoverPreparedFromFiles(true);
 		}
 
 		/* ControlFile->checkPointCopy always tracks the latest ckpt XID */
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b7ce0c6..416ef5e 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -17,6 +17,7 @@
 #include "access/xlogdefs.h"
 #include "datatype/timestamp.h"
 #include "storage/lock.h"
+#include "access/xlogreader.h"
 
 /*
  * GlobalTransactionData is defined in twophase.c; other places have no
@@ -46,8 +47,8 @@ extern bool StandbyTransactionIdIsPrepared(TransactionId xid);
 
 extern TransactionId PrescanPreparedTransactions(TransactionId **xids_p,
 							int *nxids_p);
-extern void StandbyRecoverPreparedTransactions(bool overwriteOK);
-extern void RecoverPreparedTransactions(void);
+extern void RecoverPreparedFromFiles(bool overwriteOK);
+extern void RecoverPreparedFromXLOG(XLogReaderState *record);
 
 extern void RecreateTwoPhaseFile(TransactionId xid, void *content, int len);
 extern void RemoveTwoPhaseFile(TransactionId xid, bool giveWarning);
@@ -56,4 +57,5 @@ extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void XlogRedoFinishPrepared(TransactionId xid, bool isCommit);
 #endif   /* TWOPHASE_H */
diff --git a/src/test/recovery/t/006_twophase.pl b/src/test/recovery/t/006_twophase.pl
new file mode 100644
index 0000000..82191d4
--- /dev/null
+++ b/src/test/recovery/t/006_twophase.pl
@@ -0,0 +1,226 @@
+# Checks for recovery_min_apply_delay
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 11;
+
+# Setup master node
+my $node_master = get_new_node("Candie");
+$node_master->init(allows_streaming => 1);
+$node_master->append_conf('postgresql.conf', qq(
+max_prepared_transactions = 10
+));
+$node_master->start;
+$node_master->backup('master_backup');
+$node_master->psql('postgres', "create table t(id int)");
+
+# Setup master node
+my $node_slave = get_new_node('Django');
+$node_slave->init_from_backup($node_master, 'master_backup', has_streaming => 1);
+$node_slave->start;
+
+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+synchronous_standby_names = '*'
+));
+$node_master->restart;
+
+my $psql_out = '';
+my $psql_rc = '';
+
+###############################################################################
+# Check that we can commit and abort tx after soft restart.
+# Here checkpoint happens before shutdown and no WAL replay will not occur
+# during start. So postgres should re-create memory state from files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	prepare transaction 'y';");
+$node_master->stop;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared tx after restart.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared tx after restart.');
+
+###############################################################################
+# Check that we can commit and abort after hard restart.
+# On startup WAL replay will re-create memory for global transactions that
+# happend after the last checkpoint.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	prepare transaction 'y';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared tx after teardown.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared tx after teardown.');
+
+###############################################################################
+# Check that we can replay several tx with same name.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Replay several tx with same name.');
+
+###############################################################################
+# Check that WAL replay will cleanup it's memory state and release locks while
+# replaying commit.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_master->teardown_node;
+$node_master->start;
+$psql_rc = $node_master->psql('postgres',"
+	begin;
+	insert into t values (42);
+	-- This prepare can fail due to 2pc identifier or locks conflicts if replay
+	-- didn't fully cleanup it's state on commit.
+	prepare transaction 'x';");
+is($psql_rc, '0', "Check that replay will cleanup it's memory state");
+
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that WAL replay will cleanup it's memory state on running slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';
+	");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;", stdout => \$psql_out);
+is($psql_out, '0', "Check that replay will cleanup it's memory state on running slave");
+
+###############################################################################
+# The same as in previous case, but let's force checkpoint on slave between
+# prepare and commit.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	");
+$node_slave->psql('postgres',"checkpoint;");
+$node_master->psql('postgres', "commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;", stdout => \$psql_out);
+is($psql_out, '0', "Check that replay will cleanup it's memory state on slave after checkpoint");
+
+###############################################################################
+# Check that we can commit transaction on promoted slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	");
+$node_master->teardown_node;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$psql_rc = $node_slave->psql('postgres', "commit prepared 'x';");
+is($psql_rc, '0', "Restore prepared transaction on promoted slave.");
+
+# change roles
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+
+###############################################################################
+# Check that we restore prepared xacts after slave soft restart while master is
+# down. Since slave knows that master is down it uses different code path on
+# start.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->restart;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
+is($psql_out, '1', "Restore prepared xacts after slave soft restart while master is down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres',"commit prepared 'x'");
+
+###############################################################################
+# Check that we restore prepared xacts after slave hard restart while master is
+# down.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (242);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->teardown_node;
+$node_slave->start;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
+is($psql_out, '1', "Restore prepared xacts after slave hard restart while master is down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres',"commit prepared 'x'");
+
+
#51Jesper Pedersen
jesper.pedersen@redhat.com
In reply to: Stas Kelvich (#50)
Re: Speedup twophase transactions

On 03/18/2016 12:50 PM, Stas Kelvich wrote:

On 11 Mar 2016, at 19:41, Jesper Pedersen <jesper.pedersen@redhat.com> wrote:

Thanks for review, Jesper.

Some comments:

* The patch needs a rebase against the latest TwoPhaseFileHeader change

Done.

* Rework the check.sh script into a TAP test case (src/test/recovery), as suggested by Alvaro and Michael down thread

Done. Originally I thought about reducing number of tests (11 right now), but now, after some debugging, I�m more convinced that it is better to include them all, as they are really testing different code paths.

* Add documentation for RecoverPreparedFromXLOG

Done.

Thanks, Stas.

I have gone over this version, and tested with --enable-tap-tests + make
check in src/test/recovery, which passes.

Simon, do you want to move this entry to "Ready for Committer" and take
the 2REVIEWER section as part of that, or leave it in "Needs Review"
and update the thread ?

Best regards,
Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#52Michael Paquier
michael.paquier@gmail.com
In reply to: Jesper Pedersen (#51)
Re: Speedup twophase transactions

On Tue, Mar 22, 2016 at 1:53 AM, Jesper Pedersen
<jesper.pedersen@redhat.com> wrote:

On 03/18/2016 12:50 PM, Stas Kelvich wrote:

On 11 Mar 2016, at 19:41, Jesper Pedersen <jesper.pedersen@redhat.com>
wrote:

Thanks for review, Jesper.

Some comments:

* The patch needs a rebase against the latest TwoPhaseFileHeader change

Done.

* Rework the check.sh script into a TAP test case (src/test/recovery), as
suggested by Alvaro and Michael down thread

Done. Originally I thought about reducing number of tests (11 right now),
but now, after some debugging, I’m more convinced that it is better to
include them all, as they are really testing different code paths.

* Add documentation for RecoverPreparedFromXLOG

Done.

Thanks, Stas.

I have gone over this version, and tested with --enable-tap-tests + make
check in src/test/recovery, which passes.

Simon, do you want to move this entry to "Ready for Committer" and take the
2REVIEWER section as part of that, or leave it in "Needs Review" and update
the thread ?

Looking at this patch....

+++ b/src/test/recovery/t/006_twophase.pl
@@ -0,0 +1,226 @@
+# Checks for recovery_min_apply_delay
+use strict;
This description is wrong, this file has been copied from 005.
+my $node_master = get_new_node("Candie");
+my $node_slave = get_new_node('Django');
Please let's use a node names that are more descriptive.
+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+synchronous_standby_names = '*'
+));
+$node_master->restart;
Reloading would be fine.
+   /* During replay that lock isn't really necessary, but let's take
it anyway */
+   LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+   for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+   {
+       gxact = TwoPhaseState->prepXacts[i];
+       proc = &ProcGlobal->allProcs[gxact->pgprocno];
+       pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+       if (TransactionIdEquals(xid, pgxact->xid))
+       {
+           gxact->locking_backend = MyBackendId;
+           MyLockedGxact = gxact;
+           break;
+       }
+   }
+   LWLockRelease(TwoPhaseStateLock);
Not taking ProcArrayLock here?

The comment at the top of XlogReadTwoPhaseData is incorrect.

RecoverPreparedFromXLOG and RecoverPreparedFromFiles have a lot of
code in common, having this duplication is not good, and you could
simplify your patch.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#53David Steele
david@pgmasters.net
In reply to: Michael Paquier (#52)
Re: Speedup twophase transactions

Hi Stas,

On 3/22/16 9:20 AM, Michael Paquier wrote:

Not taking ProcArrayLock here?

The comment at the top of XlogReadTwoPhaseData is incorrect.

RecoverPreparedFromXLOG and RecoverPreparedFromFiles have a lot of
code in common, having this duplication is not good, and you could
simplify your patch.

It looks like you should post a new patch or respond to Michael's
comments. Marked as "waiting on author".

Thanks,
--
-David
david@pgmasters.net

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#54Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#52)
1 attachment(s)
Re: Speedup twophase transactions

On Mar 29, 2016, at 6:04 PM, David Steele <david@pgmasters.net> wrote:

It looks like you should post a new patch or respond to Michael's comments. Marked as "waiting on author".

Yep, here it is.

On Mar 22, 2016, at 4:20 PM, Michael Paquier <michael.paquier@gmail.com> wrote:

Looking at this patch….

Thanks.

+++ b/src/test/recovery/t/006_twophase.pl
@@ -0,0 +1,226 @@
+# Checks for recovery_min_apply_delay
+use strict;
This description is wrong, this file has been copied from 005.

Yep, done.

+my $node_master = get_new_node("Candie");
+my $node_slave = get_new_node('Django');
Please let's use a node names that are more descriptive.

Hm, it’s hard to create descriptive names because test changes master/slave roles for that nodes several times during test. It’s possible to call them “node1” and “node2” but I’m not sure that improves something. But anyway I’m not insisting on that particular names and will agree with any reasonable suggestion.

+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+synchronous_standby_names = '*'
+));
+$node_master->restart;
Reloading would be fine.

Good catch, done.

+   /* During replay that lock isn't really necessary, but let's take
it anyway */
+   LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+   for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+   {
+       gxact = TwoPhaseState->prepXacts[i];
+       proc = &ProcGlobal->allProcs[gxact->pgprocno];
+       pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+       if (TransactionIdEquals(xid, pgxact->xid))
+       {
+           gxact->locking_backend = MyBackendId;
+           MyLockedGxact = gxact;
+           break;
+       }
+   }
+   LWLockRelease(TwoPhaseStateLock);
Not taking ProcArrayLock here?

All accesses to 2pc dummies in ProcArray are covered with TwoPhaseStateLock, so I thick that’s safe. Also I’ve deleted comment above that block, probably it’s more confusing than descriptive.

The comment at the top of XlogReadTwoPhaseData is incorrect.

Yep, fixed.

RecoverPreparedFromXLOG and RecoverPreparedFromFiles have a lot of
code in common, having this duplication is not good, and you could
simplify your patch.

I reworked patch to avoid duplicated code between RecoverPreparedFromXLOG/RecoverPreparedFromFiles and also between FinishPreparedTransaction/XlogRedoFinishPrepared.

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com <http://www.postgrespro.com/&gt;
Russian Postgres Company

Attachments:

twophase_replay.v3.diffapplication/octet-stream; name=twophase_replay.v3.diffDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index e7234c8..4e5ba38 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,8 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *
+ *		The same procedure happens during replication and crash recovery.
  *
  *-------------------------------------------------------------------------
  */
@@ -578,6 +578,37 @@ LockGXact(const char *gid, Oid user)
 }
 
 /*
+ * LockGXactByXid
+ *
+ * Find prepared transaction by xid and lock corresponding gxact.
+ * This is used during recovery as an alternative to LockGXact().
+ */
+static GlobalTransaction
+LockGXactByXid(TransactionId xid)
+{
+	int i;
+	GlobalTransaction gxact = NULL;
+	PGXACT	   *pgxact;
+
+	LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		gxact = TwoPhaseState->prepXacts[i];
+		pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (TransactionIdEquals(xid, pgxact->xid))
+		{
+			gxact->locking_backend = MyBackendId;
+			MyLockedGxact = gxact;
+			break;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+	return gxact;
+}
+
+/*
  * RemoveGXact
  *		Remove the prepared transaction from the shared memory array.
  *
@@ -1241,9 +1272,9 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
  * Reads 2PC data from xlog. During checkpoint this data will be moved to
  * twophase files and ReadTwoPhaseFile should be used instead.
  *
- * Note clearly that this function accesses WAL during normal operation, similarly
- * to the way WALSender or Logical Decoding would do. It does not run during
- * crash recovery or standby processing.
+ * Note clearly that this function access WAL not only during recovery/replay
+ * but also during normal operation, similarly to the way WALSender or
+ * Logical Decoding would do.
  */
 static void
 XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
@@ -1252,8 +1283,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1296,12 +1325,30 @@ StandbyTransactionIdIsPrepared(TransactionId xid)
 	char	   *buf;
 	TwoPhaseFileHeader *hdr;
 	bool		result;
+	int			i;
 
 	Assert(TransactionIdIsValid(xid));
 
 	if (max_prepared_xacts <= 0)
 		return false;			/* nothing to do */
 
+	/*
+	 * At first check prepared tx that wasn't yet moved to disk.
+	 */
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		PGXACT *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (TransactionIdEquals(pgxact->xid, xid))
+		{
+			LWLockRelease(TwoPhaseStateLock);
+			return true;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
 	/* Read and validate file */
 	buf = ReadTwoPhaseFile(xid, false);
 	if (buf == NULL)
@@ -1316,12 +1363,17 @@ StandbyTransactionIdIsPrepared(TransactionId xid)
 }
 
 /*
- * FinishPreparedTransaction: execute COMMIT PREPARED or ROLLBACK PREPARED
+ * FinishGXact
+ *
+ * Do the actual finish of COMMIT/ABORT PREPARED. It is a caller
+ * responsibility to properly lock corresponding gxact.
+ *
+ * This function can be called during replay to clean memory state
+ * for previously prepared xact. In that case actions are the same
+ * as in normal mode but without any writes to WAL or files.
  */
-void
-FinishPreparedTransaction(const char *gid, bool isCommit)
+static void FinishGXact(GlobalTransaction gxact, bool isCommit)
 {
-	GlobalTransaction gxact;
 	PGPROC	   *proc;
 	PGXACT	   *pgxact;
 	TransactionId xid;
@@ -1337,11 +1389,6 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	SharedInvalidationMessage *invalmsgs;
 	int			i;
 
-	/*
-	 * Validate the GID, and lock the GXACT to ensure that two backends do not
-	 * try to commit the same GID at once.
-	 */
-	gxact = LockGXact(gid, GetUserId());
 	proc = &ProcGlobal->allProcs[gxact->pgprocno];
 	pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 	xid = pgxact->xid;
@@ -1385,16 +1432,19 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	 * progress), then run the post-commit or post-abort callbacks. The
 	 * callbacks will release the locks the transaction held.
 	 */
-	if (isCommit)
-		RecordTransactionCommitPrepared(xid,
+	if (!RecoveryInProgress())
+	{
+		if (isCommit)
+			RecordTransactionCommitPrepared(xid,
 										hdr->nsubxacts, children,
 										hdr->ncommitrels, commitrels,
 										hdr->ninvalmsgs, invalmsgs,
 										hdr->initfileinval);
-	else
-		RecordTransactionAbortPrepared(xid,
+		else
+			RecordTransactionAbortPrepared(xid,
 									   hdr->nsubxacts, children,
 									   hdr->nabortrels, abortrels);
+	}
 
 	ProcArrayRemove(proc, latestXid);
 
@@ -1425,12 +1475,15 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 		delrels = abortrels;
 		ndelrels = hdr->nabortrels;
 	}
-	for (i = 0; i < ndelrels; i++)
+	if (!RecoveryInProgress())
 	{
-		SMgrRelation srel = smgropen(delrels[i], InvalidBackendId);
+		for (i = 0; i < ndelrels; i++)
+		{
+			SMgrRelation srel = smgropen(delrels[i], InvalidBackendId);
 
-		smgrdounlink(srel, false);
-		smgrclose(srel);
+			smgrdounlink(srel, false);
+			smgrclose(srel);
+		}
 	}
 
 	/*
@@ -1439,11 +1492,14 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	 * Relcache init file invalidation requires processing both before and
 	 * after we send the SI messages. See AtEOXact_Inval()
 	 */
-	if (hdr->initfileinval)
-		RelationCacheInitFilePreInvalidate();
-	SendSharedInvalidMessages(invalmsgs, hdr->ninvalmsgs);
-	if (hdr->initfileinval)
-		RelationCacheInitFilePostInvalidate();
+	if (!RecoveryInProgress())
+	{
+		if (hdr->initfileinval)
+			RelationCacheInitFilePreInvalidate();
+		SendSharedInvalidMessages(invalmsgs, hdr->ninvalmsgs);
+		if (hdr->initfileinval)
+			RelationCacheInitFilePostInvalidate();
+	}
 
 	/* And now do the callbacks */
 	if (isCommit)
@@ -1469,6 +1525,49 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 }
 
 /*
+ * FinishPreparedTransaction: execute COMMIT PREPARED or ROLLBACK PREPARED
+ */
+void
+FinishPreparedTransaction(const char *gid, bool isCommit)
+{
+	GlobalTransaction gxact;
+
+	/*
+	 * Validate the GID, and lock the GXACT to ensure that two backends do not
+	 * try to commit the same GID at once.
+	 */
+	gxact = LockGXact(gid, GetUserId());
+	FinishGXact(gxact, isCommit);
+}
+
+/*
+ * XlogRedoFinishPrepared()
+ *
+ * This function is called during replay when xlog reader faces 2pc commit or
+ * abort record. That function should clean up memory state that was created
+ * while replaying prepare xlog record.
+ */
+void
+XlogRedoFinishPrepared(TransactionId xid, bool isCommit)
+{
+	GlobalTransaction gxact;
+
+	Assert(RecoveryInProgress());
+
+	gxact = LockGXactByXid(xid);
+
+	/*
+	 * If requested xid wasn't found that means that prepare record was moved
+	 * to files before our replay started. That's okay and we have nothing to
+	 * clean/finish.
+	 */
+	if (!gxact)
+		return;
+
+	FinishGXact(gxact, isCommit);
+}
+
+/*
  * Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
  */
 static void
@@ -1690,7 +1789,48 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 	TransactionId *xids = NULL;
 	int			nxids = 0;
 	int			allocsize = 0;
+	int			i;
 
+	/*
+	 * We need to check the PGXACT array for prepared transactions that doesn't
+	 * have any state file in case of a slave restart with the master being off.
+	 */
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (!gxact->valid)
+			continue;
+
+		if (TransactionIdPrecedes(pgxact->xid, result))
+			result = pgxact->xid;
+
+		if (xids_p)
+		{
+			if (nxids == allocsize)
+			{
+				if (nxids == 0)
+				{
+					allocsize = 10;
+					xids = palloc(allocsize * sizeof(TransactionId));
+				}
+				else
+				{
+					allocsize = allocsize * 2;
+					xids = repalloc(xids, allocsize * sizeof(TransactionId));
+				}
+			}
+			xids[nxids++] = pgxact->xid;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+
+	/*
+	 * And now scan files in pg_twophase directory
+	 */
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
 	{
@@ -1701,7 +1841,6 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 			char	   *buf;
 			TwoPhaseFileHeader *hdr;
 			TransactionId *subxids;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
@@ -1809,102 +1948,105 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 }
 
 /*
- * StandbyRecoverPreparedTransactions
+ * RecoverPreparedFromBuffer
+ *
+ * Parse data in given buffer (that can be a pointer to WAL record or file)
+ * and load shared-memory state for that prepared transaction.
  *
- * Scan the pg_twophase directory and setup all the required information to
- * allow standby queries to treat prepared transactions as still active.
- * This is never called at the end of recovery - we use
- * RecoverPreparedTransactions() at that point.
+ * It's a caller responsibility to call MarkAsPrepared() on returned gxact.
  *
- * Currently we simply call SubTransSetParent() for any subxids of prepared
- * transactions. If overwriteOK is true, it's OK if some XIDs have already
- * been marked in pg_subtrans.
  */
-void
-StandbyRecoverPreparedTransactions(bool overwriteOK)
+static GlobalTransaction
+RecoverPreparedFromBuffer(char *buf, bool forceOverwriteOK)
 {
-	DIR		   *cldir;
-	struct dirent *clde;
+	char			*bufptr;
+	const char		*gid;
+	TransactionId	*subxids;
+	bool			overwriteOK = false;
+	int				i;
+	GlobalTransaction gxact;
+	TwoPhaseFileHeader	*hdr;
 
-	cldir = AllocateDir(TWOPHASE_DIR);
-	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
-	{
-		if (strlen(clde->d_name) == 8 &&
-			strspn(clde->d_name, "0123456789ABCDEF") == 8)
-		{
-			TransactionId xid;
-			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
+	/* Deconstruct header */
+	hdr = (TwoPhaseFileHeader *) buf;
+	bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+	gid = (const char *) bufptr;
+	bufptr += MAXALIGN(hdr->gidlen);
+	subxids = (TransactionId *) bufptr;
+	bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
+	bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
 
-			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
+	/*
+	 * It's possible that SubTransSetParent has been set before, if
+	 * the prepared transaction generated xid assignment records. Test
+	 * here must match one used in AssignTransactionId().
+	 */
+	if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
+						 XLogLogicalInfoActive()))
+		overwriteOK = true;
 
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+	/*
+	 * Caller can also force overwriteOK.
+	 */
+	if (forceOverwriteOK)
+		overwriteOK = true;
 
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+	/*
+	 * Reconstruct subtrans state for the transaction --- needed
+	 * because pg_subtrans is not preserved over a restart.  Note that
+	 * we are linking all the subtransactions directly to the
+	 * top-level XID; there may originally have been a more complex
+	 * hierarchy, but there's no need to restore that exactly.
+	 */
+	for (i = 0; i < hdr->nsubxacts; i++)
+		SubTransSetParent(subxids[i], hdr->xid, overwriteOK);
 
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				pfree(buf);
-				continue;
-			}
+	/*
+	 * Recreate its GXACT and dummy PGPROC
+	 */
+	gxact = MarkAsPreparing(hdr->xid, gid,
+							hdr->prepared_at,
+							hdr->owner, hdr->database);
+	GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
 
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID.
-			 */
-			subxids = (TransactionId *)
-				(buf + MAXALIGN(sizeof(TwoPhaseFileHeader)));
-			for (i = 0; i < hdr->nsubxacts; i++)
-			{
-				TransactionId subxid = subxids[i];
+	/*
+	 * Recover other state (notably locks) using resource managers
+	 */
+	ProcessRecords(bufptr, hdr->xid, twophase_recover_callbacks);
 
-				Assert(TransactionIdFollows(subxid, xid));
-				SubTransSetParent(xid, subxid, overwriteOK);
-			}
-		}
-	}
-	FreeDir(cldir);
+	/*
+	 * Release locks held by the standby process after we process each
+	 * prepared transaction. As a result, we don't need too many
+	 * additional locks at any one time.
+	 */
+	if (InHotStandby)
+		StandbyReleaseLockTree(hdr->xid, hdr->nsubxacts, subxids);
+
+	/*
+	 * We're done with recovering this transaction. Clear
+	 * MyLockedGxact, like we do in PrepareTransaction() during normal
+	 * operation.
+	 */
+	PostPrepare_Twophase();
+
+	return gxact;
 }
 
 /*
- * RecoverPreparedTransactions
+ * RecoverPreparedFromFiles
  *
  * Scan the pg_twophase directory and reload shared-memory state for each
  * prepared transaction (reacquire locks, etc).  This is run during database
  * startup.
  */
 void
-RecoverPreparedTransactions(void)
+RecoverPreparedFromFiles(bool forceOverwriteOK)
 {
 	char		dir[MAXPGPATH];
 	DIR		   *cldir;
 	struct dirent *clde;
-	bool		overwriteOK = false;
 
 	snprintf(dir, MAXPGPATH, "%s", TWOPHASE_DIR);
 
@@ -1916,15 +2058,27 @@ RecoverPreparedTransactions(void)
 		{
 			TransactionId xid;
 			char	   *buf;
-			char	   *bufptr;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
 			GlobalTransaction gxact;
-			const char	*gid;
 			int			i;
+			PGXACT	   *pgxact;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
+			/* Already recovered from WAL? */
+			LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+			for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+			{
+				gxact = TwoPhaseState->prepXacts[i];
+				pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+				if (TransactionIdEquals(xid, pgxact->xid))
+				{
+					LWLockRelease(TwoPhaseStateLock);
+					goto next_file;
+				}
+			}
+			LWLockRelease(TwoPhaseStateLock);
+
 			/* Already processed? */
 			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
 			{
@@ -1949,73 +2103,44 @@ RecoverPreparedTransactions(void)
 			ereport(LOG,
 					(errmsg("recovering prepared transaction %u", xid)));
 
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			Assert(TransactionIdEquals(hdr->xid, xid));
-			bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
-			gid = (const char *) bufptr;
-			bufptr += MAXALIGN(hdr->gidlen);
-			subxids = (TransactionId *) bufptr;
-			bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
-			bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
-
-			/*
-			 * It's possible that SubTransSetParent has been set before, if
-			 * the prepared transaction generated xid assignment records. Test
-			 * here must match one used in AssignTransactionId().
-			 */
-			if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
-								 XLogLogicalInfoActive()))
-				overwriteOK = true;
-
-			/*
-			 * Reconstruct subtrans state for the transaction --- needed
-			 * because pg_subtrans is not preserved over a restart.  Note that
-			 * we are linking all the subtransactions directly to the
-			 * top-level XID; there may originally have been a more complex
-			 * hierarchy, but there's no need to restore that exactly.
-			 */
-			for (i = 0; i < hdr->nsubxacts; i++)
-				SubTransSetParent(subxids[i], xid, overwriteOK);
-
-			/*
-			 * Recreate its GXACT and dummy PGPROC
-			 */
-			gxact = MarkAsPreparing(xid, gid,
-									hdr->prepared_at,
-									hdr->owner, hdr->database);
+			gxact = RecoverPreparedFromBuffer(buf, forceOverwriteOK);
 			gxact->ondisk = true;
-			GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
 			MarkAsPrepared(gxact);
 
-			/*
-			 * Recover other state (notably locks) using resource managers
-			 */
-			ProcessRecords(bufptr, xid, twophase_recover_callbacks);
-
-			/*
-			 * Release locks held by the standby process after we process each
-			 * prepared transaction. As a result, we don't need too many
-			 * additional locks at any one time.
-			 */
-			if (InHotStandby)
-				StandbyReleaseLockTree(xid, hdr->nsubxacts, subxids);
-
-			/*
-			 * We're done with recovering this transaction. Clear
-			 * MyLockedGxact, like we do in PrepareTransaction() during normal
-			 * operation.
-			 */
-			PostPrepare_Twophase();
-
 			pfree(buf);
 		}
+
+next_file:
+		continue;
+
 	}
 	FreeDir(cldir);
 }
 
+
+/*
+ * RecoverPreparedFromXLOG
+ *
+ * To avoid creation of state files during replay we registering
+ * prepare xlog records in shared memory in the same way as it happens
+ * while not in recovery. If replay faces commit xlog record before
+ * checkpoint/restartpoint happens then we avoid using files at all.
+ *
+ * We need this behaviour because the speed of the 2PC replay on the replica
+ * should be at least the same as the 2PC transaction speed of the master.
+ */
+void
+RecoverPreparedFromXLOG(XLogReaderState *record)
+{
+	GlobalTransaction gxact;
+
+	gxact = RecoverPreparedFromBuffer((char *) XLogRecGetData(record), false);
+	gxact->prepare_start_lsn = record->ReadRecPtr;
+	gxact->prepare_end_lsn = record->EndRecPtr;
+	MarkAsPrepared(gxact);
+}
+
+
 /*
  *	RecordTransactionCommitPrepared
  *
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index e315405..caed84c 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -5574,7 +5574,7 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+			XlogRedoFinishPrepared(parsed.twophase_xid, true);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5594,14 +5594,12 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+			XlogRedoFinishPrepared(parsed.twophase_xid, false);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		RecoverPreparedFromXLOG(record);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index b119a47..754280f 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6627,7 +6627,7 @@ StartupXLOG(void)
 
 				ProcArrayApplyRecoveryInfo(&running);
 
-				StandbyRecoverPreparedTransactions(false);
+				RecoverPreparedFromFiles(false);
 			}
 		}
 
@@ -7369,7 +7369,7 @@ StartupXLOG(void)
 	TrimMultiXact();
 
 	/* Reload shared-memory state for prepared transactions */
-	RecoverPreparedTransactions();
+	RecoverPreparedFromFiles(false);
 
 	/*
 	 * Shutdown the recovery environment. This must occur after
@@ -9283,7 +9283,7 @@ xlog_redo(XLogReaderState *record)
 
 			ProcArrayApplyRecoveryInfo(&running);
 
-			StandbyRecoverPreparedTransactions(true);
+			RecoverPreparedFromFiles(true);
 		}
 
 		/* ControlFile->checkPointCopy always tracks the latest ckpt XID */
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b7ce0c6..416ef5e 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -17,6 +17,7 @@
 #include "access/xlogdefs.h"
 #include "datatype/timestamp.h"
 #include "storage/lock.h"
+#include "access/xlogreader.h"
 
 /*
  * GlobalTransactionData is defined in twophase.c; other places have no
@@ -46,8 +47,8 @@ extern bool StandbyTransactionIdIsPrepared(TransactionId xid);
 
 extern TransactionId PrescanPreparedTransactions(TransactionId **xids_p,
 							int *nxids_p);
-extern void StandbyRecoverPreparedTransactions(bool overwriteOK);
-extern void RecoverPreparedTransactions(void);
+extern void RecoverPreparedFromFiles(bool overwriteOK);
+extern void RecoverPreparedFromXLOG(XLogReaderState *record);
 
 extern void RecreateTwoPhaseFile(TransactionId xid, void *content, int len);
 extern void RemoveTwoPhaseFile(TransactionId xid, bool giveWarning);
@@ -56,4 +57,5 @@ extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void XlogRedoFinishPrepared(TransactionId xid, bool isCommit);
 #endif   /* TWOPHASE_H */
diff --git a/src/test/recovery/t/006_twophase.pl b/src/test/recovery/t/006_twophase.pl
new file mode 100644
index 0000000..e73fbb4
--- /dev/null
+++ b/src/test/recovery/t/006_twophase.pl
@@ -0,0 +1,225 @@
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 11;
+
+# Setup master node
+my $node_master = get_new_node("Candie");
+$node_master->init(allows_streaming => 1);
+$node_master->append_conf('postgresql.conf', qq(
+max_prepared_transactions = 10
+));
+$node_master->start;
+$node_master->backup('master_backup');
+$node_master->psql('postgres', "create table t(id int)");
+
+# Setup master node
+my $node_slave = get_new_node('Django');
+$node_slave->init_from_backup($node_master, 'master_backup', has_streaming => 1);
+$node_slave->start;
+
+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+synchronous_standby_names = '*'
+));
+$node_master->psql('postgres', "select pg_reload_conf()");
+
+my $psql_out = '';
+my $psql_rc = '';
+
+###############################################################################
+# Check that we can commit and abort tx after soft restart.
+# Here checkpoint happens before shutdown and no WAL replay will not occur
+# during start. So postgres should re-create memory state from files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	prepare transaction 'y';");
+$node_master->stop;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared tx after restart.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared tx after restart.');
+
+###############################################################################
+# Check that we can commit and abort after hard restart.
+# On startup WAL replay will re-create memory for global transactions that
+# happend after the last checkpoint.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	prepare transaction 'y';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared tx after teardown.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared tx after teardown.');
+
+###############################################################################
+# Check that we can replay several tx with same name.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Replay several tx with same name.');
+
+###############################################################################
+# Check that WAL replay will cleanup it's memory state and release locks while
+# replaying commit.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_master->teardown_node;
+$node_master->start;
+$psql_rc = $node_master->psql('postgres',"
+	begin;
+	insert into t values (42);
+	-- This prepare can fail due to 2pc identifier or locks conflicts if replay
+	-- didn't fully cleanup it's state on commit.
+	prepare transaction 'x';");
+is($psql_rc, '0', "Check that replay will cleanup it's memory state");
+
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that WAL replay will cleanup it's memory state on running slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';
+	");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;", stdout => \$psql_out);
+is($psql_out, '0', "Check that replay will cleanup it's memory state on running slave");
+
+###############################################################################
+# The same as in previous case, but let's force checkpoint on slave between
+# prepare and commit.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	");
+$node_slave->psql('postgres',"checkpoint;");
+$node_master->psql('postgres', "commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;", stdout => \$psql_out);
+is($psql_out, '0', "Check that replay will cleanup it's memory state on slave after checkpoint");
+
+###############################################################################
+# Check that we can commit transaction on promoted slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	");
+$node_master->teardown_node;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$psql_rc = $node_slave->psql('postgres', "commit prepared 'x';");
+is($psql_rc, '0', "Restore prepared transaction on promoted slave.");
+
+# change roles
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+
+###############################################################################
+# Check that we restore prepared xacts after slave soft restart while master is
+# down. Since slave knows that master is down it uses different code path on
+# start.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->restart;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
+is($psql_out, '1', "Restore prepared xacts after slave soft restart while master is down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres',"commit prepared 'x'");
+
+###############################################################################
+# Check that we restore prepared xacts after slave hard restart while master is
+# down.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (242);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->teardown_node;
+$node_slave->start;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
+is($psql_out, '1', "Restore prepared xacts after slave hard restart while master is down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres',"commit prepared 'x'");
+
+
#55Jesper Pedersen
jesper.pedersen@redhat.com
In reply to: Stas Kelvich (#54)
Re: Speedup twophase transactions

On 03/30/2016 09:19 AM, Stas Kelvich wrote:

+++ b/src/test/recovery/t/006_twophase.pl
@@ -0,0 +1,226 @@
+# Checks for recovery_min_apply_delay
+use strict;
This description is wrong, this file has been copied from 005.

Yep, done.

+my $node_master = get_new_node("Candie");
+my $node_slave = get_new_node('Django');
Please let's use a node names that are more descriptive.

Hm, it�s hard to create descriptive names because test changes master/slave
roles for that nodes several times during test. It�s possible to call them
�node1� and �node2� but I�m not sure that improves something. But anyway I�m not
insisting on that particular names and will agree with any reasonable suggestion.

+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+synchronous_standby_names = '*'
+));
+$node_master->restart;
Reloading would be fine.

Good catch, done.

+   /* During replay that lock isn't really necessary, but let's take
it anyway */
+   LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+   for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+   {
+       gxact = TwoPhaseState->prepXacts[i];
+       proc = &ProcGlobal->allProcs[gxact->pgprocno];
+       pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+       if (TransactionIdEquals(xid, pgxact->xid))
+       {
+           gxact->locking_backend = MyBackendId;
+           MyLockedGxact = gxact;
+           break;
+       }
+   }
+   LWLockRelease(TwoPhaseStateLock);
Not taking ProcArrayLock here?

All accesses to 2pc dummies in ProcArray are covered with TwoPhaseStateLock, so
I thick that�s safe. Also I�ve deleted comment above that block, probably it�s
more confusing than descriptive.

The comment at the top of XlogReadTwoPhaseData is incorrect.

Yep, fixed.

RecoverPreparedFromXLOG and RecoverPreparedFromFiles have a lot of
code in common, having this duplication is not good, and you could
simplify your patch.

I reworked patch to avoid duplicated code between
RecoverPreparedFromXLOG/RecoverPreparedFromFiles and also
between FinishPreparedTransaction/XlogRedoFinishPrepared.

Patch applies with hunks, and test cases are passing.

Best regards,
Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#56Michael Paquier
michael.paquier@gmail.com
In reply to: Stas Kelvich (#54)
Re: Speedup twophase transactions

On Wed, Mar 30, 2016 at 10:19 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Hm, it’s hard to create descriptive names because test changes master/slave
roles for that nodes several times during test.

Really? the names used in the patch help less then.

It’s possible to call them
“node1” and “node2” but I’m not sure that improves something. But anyway I’m
not insisting on that particular names and will agree with any reasonable
suggestion.

I would suggest the following name modifications, node names have been
introduced to help tracking of each node's log:
- Candie => master
- Django => slave or just standby
There is no need for complication! And each node's log file is
prefixed by the test number. Note that in other tests there are no
promotions, or fallbacks done, but we stick with a node name that
represents the initial state of the cluster.

+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+synchronous_standby_names = '*'
+));
+$node_master->restart;
Reloading would be fine.

Good catch, done.

+$node_master->psql('postgres', "select pg_reload_conf()");

It would be cleaner to introduce a new routine in PostgresNode that
calls pg_ctl reload (mentioned in the N-sync patch as well, that would
be useful for many purposes).

All accesses to 2pc dummies in ProcArray are covered with TwoPhaseStateLock,
so I thick that’s safe. Also I’ve deleted comment above that block, probably
it’s more confusing than descriptive.

OK, you removed the use to allProcs. Though by reading again the code
just holding TwoPhaseStateLock that's actually fine.

The patch needs a small cleanup:
$ git diff master --check
src/test/recovery/t/006_twophase.pl:224: new blank line at EOF.

006_twophase.pl should be renamed to 007. It keeps its license to
kill, and gains in being famous.

- * Scan the pg_twophase directory and setup all the required information to
- * allow standby queries to treat prepared transactions as still active.
- * This is never called at the end of recovery - we use
- * RecoverPreparedTransactions() at that point.
+ * It's a caller responsibility to call MarkAsPrepared() on returned gxact.
  *
Wouldn't it be more simple to just call MarkAsPrepared at the end of
RecoverPreparedFromBuffer?

While testing the patch, I found a bug in the recovery conflict code
path. You can do the following to reproduce it:
1) Start a master with a standby
2) prepare a transaction on master
3) Stop immediate on standby to force replay
4) commit prepare transaction on master
5) When starting the standby, it remains stuck here:
* thread #1: tid = 0x229b4, 0x00007fff8e2d4f96
libsystem_kernel.dylib`poll + 10, queue = 'com.apple.main-thread',
stop reason = signal SIGSTOP
* frame #0: 0x00007fff8e2d4f96 libsystem_kernel.dylib`poll + 10
frame #1: 0x0000000107e5e043
postgres`WaitEventSetWaitBlock(set=0x00007f90c20596a8, cur_timeout=-1,
occurred_events=0x00007fff581efd28, nevents=1) + 51 at latch.c:1102
frame #2: 0x0000000107e5da26
postgres`WaitEventSetWait(set=0x00007f90c20596a8, timeout=-1,
occurred_events=0x00007fff581efd28, nevents=1) + 390 at latch.c:935
frame #3: 0x0000000107e5d4c7
postgres`WaitLatchOrSocket(latch=0x0000000111432464, wakeEvents=1,
sock=-1, timeout=-1) + 343 at latch.c:347
frame #4: 0x0000000107e5d36a
postgres`WaitLatch(latch=0x0000000111432464, wakeEvents=1, timeout=0)
+ 42 at latch.c:302
frame #5: 0x0000000107e7b5a6 postgres`ProcWaitForSignal + 38 at proc.c:1731
frame #6: 0x0000000107e6a4eb
postgres`ResolveRecoveryConflictWithLock(locktag=LOCKTAG at
0x00007fff581efde8) + 187 at standby.c:391
frame #7: 0x0000000107e7a6a8
postgres`ProcSleep(locallock=0x00007f90c203dac8,
lockMethodTable=0x00000001082f6218) + 1128 at proc.c:1215
frame #8: 0x0000000107e72886
postgres`WaitOnLock(locallock=0x00007f90c203dac8,
owner=0x0000000000000000) + 358 at lock.c:1703
frame #9: 0x0000000107e70f93
postgres`LockAcquireExtended(locktag=0x00007fff581f0238, lockmode=8,
sessionLock='\x01', dontWait='\0', reportMemoryError='\0') + 2819 at
lock.c:998
frame #10: 0x0000000107e6a9a6
postgres`StandbyAcquireAccessExclusiveLock(xid=863, dbOid=16384,
relOid=16385) + 358 at standby.c:627
frame #11: 0x0000000107e6af0b
postgres`standby_redo(record=0x00007f90c2041e38) + 251 at
standby.c:809
frame #12: 0x0000000107b0e227 postgres`StartupXLOG + 9351 at xlog.c:6871
It seems that the replay on on-memory state of the PREPARE transaction
is conflicting directly with replay.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#57Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#56)
Re: Speedup twophase transactions

On Apr 1, 2016, at 10:04 AM, Michael Paquier <michael.paquier@gmail.com> wrote:

I would suggest the following name modifications, node names have been
introduced to help tracking of each node's log:
- Candie => master
- Django => slave or just standby
There is no need for complication! And each node's log file is
prefixed by the test number. Note that in other tests there are no
promotions, or fallbacks done, but we stick with a node name that
represents the initial state of the cluster.

Ok, let’s reflect initial state in node names. So master and standby then.

+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+synchronous_standby_names = '*'
+));
+$node_master->restart;
Reloading would be fine.

Good catch, done.

+$node_master->psql('postgres', "select pg_reload_conf()");

It would be cleaner to introduce a new routine in PostgresNode that
calls pg_ctl reload (mentioned in the N-sync patch as well, that would
be useful for many purposes).

Okay.

All accesses to 2pc dummies in ProcArray are covered with TwoPhaseStateLock,
so I thick that’s safe. Also I’ve deleted comment above that block, probably
it’s more confusing than descriptive.

OK, you removed the use to allProcs. Though by reading again the code
just holding TwoPhaseStateLock that's actually fine.

The patch needs a small cleanup:
$ git diff master --check
src/test/recovery/t/006_twophase.pl:224: new blank line at EOF.

006_twophase.pl should be renamed to 007. It keeps its license to
kill, and gains in being famous.

Huh, eventually there will be Fleming reference, instead of Tarantino one in node names.

- * Scan the pg_twophase directory and setup all the required information to
- * allow standby queries to treat prepared transactions as still active.
- * This is never called at the end of recovery - we use
- * RecoverPreparedTransactions() at that point.
+ * It's a caller responsibility to call MarkAsPrepared() on returned gxact.
*
Wouldn't it be more simple to just call MarkAsPrepared at the end of
RecoverPreparedFromBuffer?

I did that intentionally to allow modification of gxact before unlock.

RecoverPreparedFromXLOG:
gxact = RecoverPreparedFromBuffer((char *) XLogRecGetData(record), false);
gxact->prepare_start_lsn = record->ReadRecPtr;
gxact->prepare_end_lsn = record->EndRecPtr;
MarkAsPrepared(gxact);

RecoverPreparedFromFiles:
gxact = RecoverPreparedFromBuffer(buf, forceOverwriteOK);
gxact->ondisk = true;
MarkAsPrepared(gxact);

While both function are only called during recovery, I think that it is better to write that
in a way when it possible to use it in multiprocess environment.

While testing the patch, I found a bug in the recovery conflict code
path. You can do the following to reproduce it:
1) Start a master with a standby
2) prepare a transaction on master
3) Stop immediate on standby to force replay
4) commit prepare transaction on master
5) When starting the standby, it remains stuck here:

Hm, I wasn’t able to reproduce that. Do you mean following scenario or am I missing something?

(async replication)

$node_master->psql('postgres', "
begin;
insert into t values (1);
prepare transaction 'x';
");
$node_slave->teardown_node;
$node_master->psql('postgres',"commit prepared 'x'");
$node_slave->start;
$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
is($psql_out, '0', "Commit prepared on master while slave is down.");

* thread #1: tid = 0x229b4, 0x00007fff8e2d4f96
libsystem_kernel.dylib`poll + 10, queue = 'com.apple.main-thread',
stop reason = signal SIGSTOP
* frame #0: 0x00007fff8e2d4f96 libsystem_kernel.dylib`poll + 10
frame #1: 0x0000000107e5e043
postgres`WaitEventSetWaitBlock(set=0x00007f90c20596a8, cur_timeout=-1,
occurred_events=0x00007fff581efd28, nevents=1) + 51 at latch.c:1102
frame #2: 0x0000000107e5da26
postgres`WaitEventSetWait(set=0x00007f90c20596a8, timeout=-1,
occurred_events=0x00007fff581efd28, nevents=1) + 390 at latch.c:935
frame #3: 0x0000000107e5d4c7
postgres`WaitLatchOrSocket(latch=0x0000000111432464, wakeEvents=1,
sock=-1, timeout=-1) + 343 at latch.c:347
frame #4: 0x0000000107e5d36a
postgres`WaitLatch(latch=0x0000000111432464, wakeEvents=1, timeout=0)
+ 42 at latch.c:302
frame #5: 0x0000000107e7b5a6 postgres`ProcWaitForSignal + 38 at proc.c:1731
frame #6: 0x0000000107e6a4eb
postgres`ResolveRecoveryConflictWithLock(locktag=LOCKTAG at
0x00007fff581efde8) + 187 at standby.c:391
frame #7: 0x0000000107e7a6a8
postgres`ProcSleep(locallock=0x00007f90c203dac8,
lockMethodTable=0x00000001082f6218) + 1128 at proc.c:1215
frame #8: 0x0000000107e72886
postgres`WaitOnLock(locallock=0x00007f90c203dac8,
owner=0x0000000000000000) + 358 at lock.c:1703
frame #9: 0x0000000107e70f93
postgres`LockAcquireExtended(locktag=0x00007fff581f0238, lockmode=8,
sessionLock='\x01', dontWait='\0', reportMemoryError='\0') + 2819 at
lock.c:998
frame #10: 0x0000000107e6a9a6
postgres`StandbyAcquireAccessExclusiveLock(xid=863, dbOid=16384,
relOid=16385) + 358 at standby.c:627
frame #11: 0x0000000107e6af0b
postgres`standby_redo(record=0x00007f90c2041e38) + 251 at
standby.c:809
frame #12: 0x0000000107b0e227 postgres`StartupXLOG + 9351 at xlog.c:6871
It seems that the replay on on-memory state of the PREPARE transaction
is conflicting directly with replay.

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#58Michael Paquier
michael.paquier@gmail.com
In reply to: Stas Kelvich (#57)
Re: Speedup twophase transactions

On Fri, Apr 1, 2016 at 10:53 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

I wrote:

While testing the patch, I found a bug in the recovery conflict code
path. You can do the following to reproduce it:
1) Start a master with a standby
2) prepare a transaction on master
3) Stop immediate on standby to force replay
4) commit prepare transaction on master
5) When starting the standby, it remains stuck here:

Hm, I wasn’t able to reproduce that. Do you mean following scenario or am I missing something?

(async replication)

$node_master->psql('postgres', "
begin;
insert into t values (1);
prepare transaction 'x';
");
$node_slave->teardown_node;
$node_master->psql('postgres',"commit prepared 'x'");
$node_slave->start;
$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
is($psql_out, '0', "Commit prepared on master while slave is down.");

Actually, not exactly, the transaction prepared on master created a
table. Sorry for the lack of precisions in my review.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#59Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#58)
1 attachment(s)
Re: Speedup twophase transactions

On Apr 2, 2016, at 3:14 AM, Michael Paquier <michael.paquier@gmail.com> wrote:

On Fri, Apr 1, 2016 at 10:53 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

I wrote:

While testing the patch, I found a bug in the recovery conflict code
path. You can do the following to reproduce it:
1) Start a master with a standby
2) prepare a transaction on master
3) Stop immediate on standby to force replay
4) commit prepare transaction on master
5) When starting the standby, it remains stuck here:

Hm, I wasn’t able to reproduce that. Do you mean following scenario or am I missing something?

(async replication)

$node_master->psql('postgres', "
begin;
insert into t values (1);
prepare transaction 'x';
");
$node_slave->teardown_node;
$node_master->psql('postgres',"commit prepared 'x'");
$node_slave->start;
$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
is($psql_out, '0', "Commit prepared on master while slave is down.");

Actually, not exactly, the transaction prepared on master created a
table. Sorry for the lack of precisions in my review.

Sorry for delay.

Actually I can’t reproduce that again, tried with following tx:

begin;
insert into t values(0);
create table t1(id int);
insert into t1 values(1);
create table t2(id int);
insert into t2 values(2);
savepoint s1;
drop table t1;
select * from t for update;
select * from t2 for share;
prepare transaction 'x’;

Attachments:

twophase_recovery_bug.pltext/x-perl-script; name=twophase_recovery_bug.plDownload
#60Michael Paquier
michael.paquier@gmail.com
In reply to: Stas Kelvich (#59)
1 attachment(s)
Re: Speedup twophase transactions

On Wed, Apr 6, 2016 at 6:47 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

On Apr 2, 2016, at 3:14 AM, Michael Paquier <michael.paquier@gmail.com> wrote:

On Fri, Apr 1, 2016 at 10:53 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

I wrote:

While testing the patch, I found a bug in the recovery conflict code
path. You can do the following to reproduce it:
1) Start a master with a standby
2) prepare a transaction on master
3) Stop immediate on standby to force replay
4) commit prepare transaction on master
5) When starting the standby, it remains stuck here:

Hm, I wasn’t able to reproduce that. Do you mean following scenario or am I missing something?

(async replication)

$node_master->psql('postgres', "
begin;
insert into t values (1);
prepare transaction 'x';
");
$node_slave->teardown_node;
$node_master->psql('postgres',"commit prepared 'x'");
$node_slave->start;
$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
is($psql_out, '0', "Commit prepared on master while slave is down.");

Actually, not exactly, the transaction prepared on master created a
table. Sorry for the lack of precisions in my review.

Sorry for delay.

Actually I can’t reproduce that again, tried with following tx:

Well not for me, here are more details, with a test case attached:
* thread #1: tid = 0x50c5b, 0x00007fff93822f96
libsystem_kernel.dylib`poll + 10, queue = 'com.apple.main-thread',
stop reason = signal SIGSTOP
* frame #0: 0x00007fff93822f96 libsystem_kernel.dylib`poll + 10
frame #1: 0x00000001023cdda3
postgres`WaitEventSetWaitBlock(set=0x00007fde50858f28, cur_timeout=-1,
occurred_events=0x00007fff5dc87cf8, nevents=1) + 51 at latch.c:1102
frame #2: 0x00000001023cd786
postgres`WaitEventSetWait(set=0x00007fde50858f28, timeout=-1,
occurred_events=0x00007fff5dc87cf8, nevents=1) + 390 at latch.c:935
frame #3: 0x00000001023cd227
postgres`WaitLatchOrSocket(latch=0x000000010b9b12e4, wakeEvents=1,
sock=-1, timeout=-1) + 343 at latch.c:347
frame #4: 0x00000001023cd0ca
postgres`WaitLatch(latch=0x000000010b9b12e4, wakeEvents=1, timeout=0)
+ 42 at latch.c:302
frame #5: 0x00000001023eb306 postgres`ProcWaitForSignal + 38 at proc.c:1731
frame #6: 0x00000001023da24b
postgres`ResolveRecoveryConflictWithLock(locktag=LOCKTAG at
0x00007fff5dc87db8) + 187 at standby.c:391
frame #7: 0x00000001023ea408
postgres`ProcSleep(locallock=0x00007fde5083dac8,
lockMethodTable=0x000000010286e278) + 1128 at proc.c:1215
frame #8: 0x00000001023e25e6
postgres`WaitOnLock(locallock=0x00007fde5083dac8,
owner=0x0000000000000000) + 358 at lock.c:1703
frame #9: 0x00000001023e0cf3
postgres`LockAcquireExtended(locktag=0x00007fff5dc88208, lockmode=8,
sessionLock='\x01', dontWait='\0', reportMemoryError='\0') + 2819 at
lock.c:998
frame #10: 0x00000001023da706
postgres`StandbyAcquireAccessExclusiveLock(xid=867, dbOid=16384,
relOid=16385) + 358 at standby.c:627
frame #11: 0x00000001023dac6b
postgres`standby_redo(record=0x00007fde50841e38) + 251 at
standby.c:809

(LOCALLOCK) $1 = {
tag = {
lock = {
locktag_field1 = 16384
locktag_field2 = 16385
locktag_field3 = 0
locktag_field4 = 0
locktag_type = '\0'
locktag_lockmethodid = '\x01'
}
mode = 8
}

=# select relname, oid from pg_class where oid > 16000;
relname | oid
---------+-------
aa | 16385
(1 row)
So recovery is conflicting here. My guess is that this patch is
missing some lock cleanup.

With the test case attached in my case the COMMIT PREPARED record does
not even get replayed.
--
Michael

Attachments:

twophase_recovery_bug_2.pltext/x-perl; charset=US-ASCII; name=twophase_recovery_bug_2.plDownload
#61Jesper Pedersen
jesper.pedersen@redhat.com
In reply to: Michael Paquier (#60)
Re: Speedup twophase transactions

On 04/07/2016 02:29 AM, Michael Paquier wrote:

So recovery is conflicting here. My guess is that this patch is
missing some lock cleanup.

With the test case attached in my case the COMMIT PREPARED record does
not even get replayed.

Should we create an entry for the open item list [0]https://wiki.postgresql.org/wiki/PostgreSQL_9.6_Open_Items for this, due to
the replication lag [1]/messages/by-id/E7497864-DE11-4099-83F5-89FB97AF5073@postgrespro.ru ?

CommitFest entry [2]https://commitfest.postgresql.org/9/523/
Original commit [3]http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=978b2f65aa1262eb4ecbf8b3785cb1b9cf4db78e

Cc'ed RMT.

[0]: https://wiki.postgresql.org/wiki/PostgreSQL_9.6_Open_Items
[1]: /messages/by-id/E7497864-DE11-4099-83F5-89FB97AF5073@postgrespro.ru
/messages/by-id/E7497864-DE11-4099-83F5-89FB97AF5073@postgrespro.ru
[2]: https://commitfest.postgresql.org/9/523/
[3]: http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=978b2f65aa1262eb4ecbf8b3785cb1b9cf4db78e
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=978b2f65aa1262eb4ecbf8b3785cb1b9cf4db78e

Best regards,
Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#62Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#60)
1 attachment(s)
Re: Speedup twophase transactions

On 07 Apr 2016, at 09:29, Michael Paquier <michael.paquier@gmail.com> wrote:

relOid=16385) + 358 at standby.c:627
frame #11: 0x00000001023dac6b
postgres`standby_redo(record=0x00007fde50841e38) + 251 at
standby.c:809

(LOCALLOCK) $1 = {
tag = {
lock = {
locktag_field1 = 16384
locktag_field2 = 16385
locktag_field3 = 0
locktag_field4 = 0
locktag_type = '\0'
locktag_lockmethodid = '\x01'
}
mode = 8
}

=# select relname, oid from pg_class where oid > 16000;
relname | oid

Tried on linux and os x 10.11 and 10.4.

Still can’t reproduce, but have played around with your backtrace.

I see in xlodump on slave following sequence of records:

rmgr: Storage len (rec/tot): 16/ 42, tx: 0, lsn: 0/03015AF0, prev 0/03015958, desc: CREATE base/12669/16387
rmgr: Heap len (rec/tot): 3/ 1518, tx: 867, lsn: 0/03015B20, prev 0/03015AF0, desc: INSERT off 8, blkref #0: rel 1663/12669/1247 blk 8 FPW
<...>
rmgr: Btree len (rec/tot): 2/ 72, tx: 867, lsn: 0/03019CD0, prev 0/03019C88, desc: INSERT_LEAF off 114, blkref #0: rel 1663/12669/2674 blk 22
rmgr: Standby len (rec/tot): 16/ 42, tx: 867, lsn: 0/03019D18, prev 0/03019CD0, desc: LOCK xid 867 db 12669 rel 16387
rmgr: Transaction len (rec/tot): 784/ 813, tx: 867, lsn: 0/03019D48, prev 0/03019D18, desc: PREPARE
rmgr: Transaction len (rec/tot): 380/ 409, tx: 0, lsn: 0/0301A090, prev 0/03019D48, desc: COMMIT_PREPARED 867: 2016-04-08 14:38:33.347851 MSK;

It looks like that you had stuck in LOCK xid 867 even before replaying PREPARE record, so I have not that much ideas on why that can be caused by changing procedures of PREPARE replay.

Just to keep things sane, here is my current diff:

Attachments:

twophase_replay.v4.patchapplication/octet-stream; name=twophase_replay.v4.patchDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index a65048b..bc9bffa 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,8 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *
+ *		The same procedure happens during replication and crash recovery.
  *
  *-------------------------------------------------------------------------
  */
@@ -578,6 +578,37 @@ LockGXact(const char *gid, Oid user)
 }
 
 /*
+ * LockGXactByXid
+ *
+ * Find prepared transaction by xid and lock corresponding gxact.
+ * This is used during recovery as an alternative to LockGXact().
+ */
+static GlobalTransaction
+LockGXactByXid(TransactionId xid)
+{
+	int i;
+	GlobalTransaction gxact = NULL;
+	PGXACT	   *pgxact;
+
+	LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		gxact = TwoPhaseState->prepXacts[i];
+		pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (TransactionIdEquals(xid, pgxact->xid))
+		{
+			gxact->locking_backend = MyBackendId;
+			MyLockedGxact = gxact;
+			break;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+	return gxact;
+}
+
+/*
  * RemoveGXact
  *		Remove the prepared transaction from the shared memory array.
  *
@@ -1241,9 +1272,9 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
  * Reads 2PC data from xlog. During checkpoint this data will be moved to
  * twophase files and ReadTwoPhaseFile should be used instead.
  *
- * Note clearly that this function accesses WAL during normal operation, similarly
- * to the way WALSender or Logical Decoding would do. It does not run during
- * crash recovery or standby processing.
+ * Note clearly that this function access WAL not only during recovery/replay
+ * but also during normal operation, similarly to the way WALSender or
+ * Logical Decoding would do.
  */
 static void
 XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
@@ -1252,8 +1283,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1296,12 +1325,30 @@ StandbyTransactionIdIsPrepared(TransactionId xid)
 	char	   *buf;
 	TwoPhaseFileHeader *hdr;
 	bool		result;
+	int			i;
 
 	Assert(TransactionIdIsValid(xid));
 
 	if (max_prepared_xacts <= 0)
 		return false;			/* nothing to do */
 
+	/*
+	 * At first check prepared tx that wasn't yet moved to disk.
+	 */
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		PGXACT *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (TransactionIdEquals(pgxact->xid, xid))
+		{
+			LWLockRelease(TwoPhaseStateLock);
+			return true;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
 	/* Read and validate file */
 	buf = ReadTwoPhaseFile(xid, false);
 	if (buf == NULL)
@@ -1316,12 +1363,17 @@ StandbyTransactionIdIsPrepared(TransactionId xid)
 }
 
 /*
- * FinishPreparedTransaction: execute COMMIT PREPARED or ROLLBACK PREPARED
+ * FinishGXact
+ *
+ * Do the actual finish of COMMIT/ABORT PREPARED. It is a caller
+ * responsibility to properly lock corresponding gxact.
+ *
+ * This function can be called during replay to clean memory state
+ * for previously prepared xact. In that case actions are the same
+ * as in normal mode but without any writes to WAL or files.
  */
-void
-FinishPreparedTransaction(const char *gid, bool isCommit)
+static void FinishGXact(GlobalTransaction gxact, bool isCommit)
 {
-	GlobalTransaction gxact;
 	PGPROC	   *proc;
 	PGXACT	   *pgxact;
 	TransactionId xid;
@@ -1337,11 +1389,6 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	SharedInvalidationMessage *invalmsgs;
 	int			i;
 
-	/*
-	 * Validate the GID, and lock the GXACT to ensure that two backends do not
-	 * try to commit the same GID at once.
-	 */
-	gxact = LockGXact(gid, GetUserId());
 	proc = &ProcGlobal->allProcs[gxact->pgprocno];
 	pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 	xid = pgxact->xid;
@@ -1385,16 +1432,19 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	 * progress), then run the post-commit or post-abort callbacks. The
 	 * callbacks will release the locks the transaction held.
 	 */
-	if (isCommit)
-		RecordTransactionCommitPrepared(xid,
+	if (!RecoveryInProgress())
+	{
+		if (isCommit)
+			RecordTransactionCommitPrepared(xid,
 										hdr->nsubxacts, children,
 										hdr->ncommitrels, commitrels,
 										hdr->ninvalmsgs, invalmsgs,
 										hdr->initfileinval);
-	else
-		RecordTransactionAbortPrepared(xid,
+		else
+			RecordTransactionAbortPrepared(xid,
 									   hdr->nsubxacts, children,
 									   hdr->nabortrels, abortrels);
+	}
 
 	ProcArrayRemove(proc, latestXid);
 
@@ -1425,12 +1475,15 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 		delrels = abortrels;
 		ndelrels = hdr->nabortrels;
 	}
-	for (i = 0; i < ndelrels; i++)
+	if (!RecoveryInProgress())
 	{
-		SMgrRelation srel = smgropen(delrels[i], InvalidBackendId);
+		for (i = 0; i < ndelrels; i++)
+		{
+			SMgrRelation srel = smgropen(delrels[i], InvalidBackendId);
 
-		smgrdounlink(srel, false);
-		smgrclose(srel);
+			smgrdounlink(srel, false);
+			smgrclose(srel);
+		}
 	}
 
 	/*
@@ -1439,11 +1492,14 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	 * Relcache init file invalidation requires processing both before and
 	 * after we send the SI messages. See AtEOXact_Inval()
 	 */
-	if (hdr->initfileinval)
-		RelationCacheInitFilePreInvalidate();
-	SendSharedInvalidMessages(invalmsgs, hdr->ninvalmsgs);
-	if (hdr->initfileinval)
-		RelationCacheInitFilePostInvalidate();
+	if (!RecoveryInProgress())
+	{
+		if (hdr->initfileinval)
+			RelationCacheInitFilePreInvalidate();
+		SendSharedInvalidMessages(invalmsgs, hdr->ninvalmsgs);
+		if (hdr->initfileinval)
+			RelationCacheInitFilePostInvalidate();
+	}
 
 	/* And now do the callbacks */
 	if (isCommit)
@@ -1469,6 +1525,49 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 }
 
 /*
+ * FinishPreparedTransaction: execute COMMIT PREPARED or ROLLBACK PREPARED
+ */
+void
+FinishPreparedTransaction(const char *gid, bool isCommit)
+{
+	GlobalTransaction gxact;
+
+	/*
+	 * Validate the GID, and lock the GXACT to ensure that two backends do not
+	 * try to commit the same GID at once.
+	 */
+	gxact = LockGXact(gid, GetUserId());
+	FinishGXact(gxact, isCommit);
+}
+
+/*
+ * XlogRedoFinishPrepared()
+ *
+ * This function is called during replay when xlog reader faces 2pc commit or
+ * abort record. That function should clean up memory state that was created
+ * while replaying prepare xlog record.
+ */
+void
+XlogRedoFinishPrepared(TransactionId xid, bool isCommit)
+{
+	GlobalTransaction gxact;
+
+	Assert(RecoveryInProgress());
+
+	gxact = LockGXactByXid(xid);
+
+	/*
+	 * If requested xid wasn't found that means that prepare record was moved
+	 * to files before our replay started. That's okay and we have nothing to
+	 * clean/finish.
+	 */
+	if (!gxact)
+		return;
+
+	FinishGXact(gxact, isCommit);
+}
+
+/*
  * Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
  */
 static void
@@ -1690,7 +1789,48 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 	TransactionId *xids = NULL;
 	int			nxids = 0;
 	int			allocsize = 0;
+	int			i;
 
+	/*
+	 * We need to check the PGXACT array for prepared transactions that doesn't
+	 * have any state file in case of a slave restart with the master being off.
+	 */
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (!gxact->valid)
+			continue;
+
+		if (TransactionIdPrecedes(pgxact->xid, result))
+			result = pgxact->xid;
+
+		if (xids_p)
+		{
+			if (nxids == allocsize)
+			{
+				if (nxids == 0)
+				{
+					allocsize = 10;
+					xids = palloc(allocsize * sizeof(TransactionId));
+				}
+				else
+				{
+					allocsize = allocsize * 2;
+					xids = repalloc(xids, allocsize * sizeof(TransactionId));
+				}
+			}
+			xids[nxids++] = pgxact->xid;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+
+	/*
+	 * And now scan files in pg_twophase directory
+	 */
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
 	{
@@ -1701,7 +1841,6 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 			char	   *buf;
 			TwoPhaseFileHeader *hdr;
 			TransactionId *subxids;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
@@ -1809,102 +1948,105 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 }
 
 /*
- * StandbyRecoverPreparedTransactions
+ * RecoverPreparedFromBuffer
+ *
+ * Parse data in given buffer (that can be a pointer to WAL record or file)
+ * and load shared-memory state for that prepared transaction.
  *
- * Scan the pg_twophase directory and setup all the required information to
- * allow standby queries to treat prepared transactions as still active.
- * This is never called at the end of recovery - we use
- * RecoverPreparedTransactions() at that point.
+ * It's a caller responsibility to call MarkAsPrepared() on returned gxact.
  *
- * Currently we simply call SubTransSetParent() for any subxids of prepared
- * transactions. If overwriteOK is true, it's OK if some XIDs have already
- * been marked in pg_subtrans.
  */
-void
-StandbyRecoverPreparedTransactions(bool overwriteOK)
+static GlobalTransaction
+RecoverPreparedFromBuffer(char *buf, bool forceOverwriteOK)
 {
-	DIR		   *cldir;
-	struct dirent *clde;
+	char			*bufptr;
+	const char		*gid;
+	TransactionId	*subxids;
+	bool			overwriteOK = false;
+	int				i;
+	GlobalTransaction gxact;
+	TwoPhaseFileHeader	*hdr;
 
-	cldir = AllocateDir(TWOPHASE_DIR);
-	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
-	{
-		if (strlen(clde->d_name) == 8 &&
-			strspn(clde->d_name, "0123456789ABCDEF") == 8)
-		{
-			TransactionId xid;
-			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
+	/* Deconstruct header */
+	hdr = (TwoPhaseFileHeader *) buf;
+	bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+	gid = (const char *) bufptr;
+	bufptr += MAXALIGN(hdr->gidlen);
+	subxids = (TransactionId *) bufptr;
+	bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
+	bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
 
-			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
+	/*
+	 * It's possible that SubTransSetParent has been set before, if
+	 * the prepared transaction generated xid assignment records. Test
+	 * here must match one used in AssignTransactionId().
+	 */
+	if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
+						 XLogLogicalInfoActive()))
+		overwriteOK = true;
 
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+	/*
+	 * Caller can also force overwriteOK.
+	 */
+	if (forceOverwriteOK)
+		overwriteOK = true;
 
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+	/*
+	 * Reconstruct subtrans state for the transaction --- needed
+	 * because pg_subtrans is not preserved over a restart.  Note that
+	 * we are linking all the subtransactions directly to the
+	 * top-level XID; there may originally have been a more complex
+	 * hierarchy, but there's no need to restore that exactly.
+	 */
+	for (i = 0; i < hdr->nsubxacts; i++)
+		SubTransSetParent(subxids[i], hdr->xid, overwriteOK);
 
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				pfree(buf);
-				continue;
-			}
+	/*
+	 * Recreate its GXACT and dummy PGPROC
+	 */
+	gxact = MarkAsPreparing(hdr->xid, gid,
+							hdr->prepared_at,
+							hdr->owner, hdr->database);
+	GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
 
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID.
-			 */
-			subxids = (TransactionId *)
-				(buf + MAXALIGN(sizeof(TwoPhaseFileHeader)));
-			for (i = 0; i < hdr->nsubxacts; i++)
-			{
-				TransactionId subxid = subxids[i];
+	/*
+	 * Recover other state (notably locks) using resource managers
+	 */
+	ProcessRecords(bufptr, hdr->xid, twophase_recover_callbacks);
 
-				Assert(TransactionIdFollows(subxid, xid));
-				SubTransSetParent(xid, subxid, overwriteOK);
-			}
-		}
-	}
-	FreeDir(cldir);
+	/*
+	 * Release locks held by the standby process after we process each
+	 * prepared transaction. As a result, we don't need too many
+	 * additional locks at any one time.
+	 */
+	if (InHotStandby)
+		StandbyReleaseLockTree(hdr->xid, hdr->nsubxacts, subxids);
+
+	/*
+	 * We're done with recovering this transaction. Clear
+	 * MyLockedGxact, like we do in PrepareTransaction() during normal
+	 * operation.
+	 */
+	PostPrepare_Twophase();
+
+	return gxact;
 }
 
 /*
- * RecoverPreparedTransactions
+ * RecoverPreparedFromFiles
  *
  * Scan the pg_twophase directory and reload shared-memory state for each
  * prepared transaction (reacquire locks, etc).  This is run during database
  * startup.
  */
 void
-RecoverPreparedTransactions(void)
+RecoverPreparedFromFiles(bool forceOverwriteOK)
 {
 	char		dir[MAXPGPATH];
 	DIR		   *cldir;
 	struct dirent *clde;
-	bool		overwriteOK = false;
 
 	snprintf(dir, MAXPGPATH, "%s", TWOPHASE_DIR);
 
@@ -1916,15 +2058,27 @@ RecoverPreparedTransactions(void)
 		{
 			TransactionId xid;
 			char	   *buf;
-			char	   *bufptr;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
 			GlobalTransaction gxact;
-			const char	*gid;
 			int			i;
+			PGXACT	   *pgxact;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
+			/* Already recovered from WAL? */
+			LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+			for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+			{
+				gxact = TwoPhaseState->prepXacts[i];
+				pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+				if (TransactionIdEquals(xid, pgxact->xid))
+				{
+					LWLockRelease(TwoPhaseStateLock);
+					goto next_file;
+				}
+			}
+			LWLockRelease(TwoPhaseStateLock);
+
 			/* Already processed? */
 			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
 			{
@@ -1949,73 +2103,44 @@ RecoverPreparedTransactions(void)
 			ereport(LOG,
 					(errmsg("recovering prepared transaction %u", xid)));
 
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			Assert(TransactionIdEquals(hdr->xid, xid));
-			bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
-			gid = (const char *) bufptr;
-			bufptr += MAXALIGN(hdr->gidlen);
-			subxids = (TransactionId *) bufptr;
-			bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
-			bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
-
-			/*
-			 * It's possible that SubTransSetParent has been set before, if
-			 * the prepared transaction generated xid assignment records. Test
-			 * here must match one used in AssignTransactionId().
-			 */
-			if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
-								 XLogLogicalInfoActive()))
-				overwriteOK = true;
-
-			/*
-			 * Reconstruct subtrans state for the transaction --- needed
-			 * because pg_subtrans is not preserved over a restart.  Note that
-			 * we are linking all the subtransactions directly to the
-			 * top-level XID; there may originally have been a more complex
-			 * hierarchy, but there's no need to restore that exactly.
-			 */
-			for (i = 0; i < hdr->nsubxacts; i++)
-				SubTransSetParent(subxids[i], xid, overwriteOK);
-
-			/*
-			 * Recreate its GXACT and dummy PGPROC
-			 */
-			gxact = MarkAsPreparing(xid, gid,
-									hdr->prepared_at,
-									hdr->owner, hdr->database);
+			gxact = RecoverPreparedFromBuffer(buf, forceOverwriteOK);
 			gxact->ondisk = true;
-			GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
 			MarkAsPrepared(gxact);
 
-			/*
-			 * Recover other state (notably locks) using resource managers
-			 */
-			ProcessRecords(bufptr, xid, twophase_recover_callbacks);
-
-			/*
-			 * Release locks held by the standby process after we process each
-			 * prepared transaction. As a result, we don't need too many
-			 * additional locks at any one time.
-			 */
-			if (InHotStandby)
-				StandbyReleaseLockTree(xid, hdr->nsubxacts, subxids);
-
-			/*
-			 * We're done with recovering this transaction. Clear
-			 * MyLockedGxact, like we do in PrepareTransaction() during normal
-			 * operation.
-			 */
-			PostPrepare_Twophase();
-
 			pfree(buf);
 		}
+
+next_file:
+		continue;
+
 	}
 	FreeDir(cldir);
 }
 
+
+/*
+ * RecoverPreparedFromXLOG
+ *
+ * To avoid creation of state files during replay we registering
+ * prepare xlog records in shared memory in the same way as it happens
+ * while not in recovery. If replay faces commit xlog record before
+ * checkpoint/restartpoint happens then we avoid using files at all.
+ *
+ * We need this behaviour because the speed of the 2PC replay on the replica
+ * should be at least the same as the 2PC transaction speed of the master.
+ */
+void
+RecoverPreparedFromXLOG(XLogReaderState *record)
+{
+	GlobalTransaction gxact;
+
+	gxact = RecoverPreparedFromBuffer((char *) XLogRecGetData(record), false);
+	gxact->prepare_start_lsn = record->ReadRecPtr;
+	gxact->prepare_end_lsn = record->EndRecPtr;
+	MarkAsPrepared(gxact);
+}
+
+
 /*
  *	RecordTransactionCommitPrepared
  *
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 7e37331..9323ba2 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -5588,7 +5588,7 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+			XlogRedoFinishPrepared(parsed.twophase_xid, true);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5608,14 +5608,12 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+			XlogRedoFinishPrepared(parsed.twophase_xid, false);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		RecoverPreparedFromXLOG(record);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index f9644db..11c06b4 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6630,7 +6630,7 @@ StartupXLOG(void)
 
 				ProcArrayApplyRecoveryInfo(&running);
 
-				StandbyRecoverPreparedTransactions(false);
+				RecoverPreparedFromFiles(false);
 			}
 		}
 
@@ -7383,7 +7383,7 @@ StartupXLOG(void)
 	TrimMultiXact();
 
 	/* Reload shared-memory state for prepared transactions */
-	RecoverPreparedTransactions();
+	RecoverPreparedFromFiles(false);
 
 	/*
 	 * Shutdown the recovery environment. This must occur after
@@ -9297,7 +9297,7 @@ xlog_redo(XLogReaderState *record)
 
 			ProcArrayApplyRecoveryInfo(&running);
 
-			StandbyRecoverPreparedTransactions(true);
+			RecoverPreparedFromFiles(true);
 		}
 
 		/* ControlFile->checkPointCopy always tracks the latest ckpt XID */
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b7ce0c6..416ef5e 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -17,6 +17,7 @@
 #include "access/xlogdefs.h"
 #include "datatype/timestamp.h"
 #include "storage/lock.h"
+#include "access/xlogreader.h"
 
 /*
  * GlobalTransactionData is defined in twophase.c; other places have no
@@ -46,8 +47,8 @@ extern bool StandbyTransactionIdIsPrepared(TransactionId xid);
 
 extern TransactionId PrescanPreparedTransactions(TransactionId **xids_p,
 							int *nxids_p);
-extern void StandbyRecoverPreparedTransactions(bool overwriteOK);
-extern void RecoverPreparedTransactions(void);
+extern void RecoverPreparedFromFiles(bool overwriteOK);
+extern void RecoverPreparedFromXLOG(XLogReaderState *record);
 
 extern void RecreateTwoPhaseFile(TransactionId xid, void *content, int len);
 extern void RemoveTwoPhaseFile(TransactionId xid, bool giveWarning);
@@ -56,4 +57,5 @@ extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void XlogRedoFinishPrepared(TransactionId xid, bool isCommit);
 #endif   /* TWOPHASE_H */
diff --git a/src/test/recovery/t/007_twophase.pl b/src/test/recovery/t/007_twophase.pl
new file mode 100644
index 0000000..cfc7316
--- /dev/null
+++ b/src/test/recovery/t/007_twophase.pl
@@ -0,0 +1,236 @@
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 11;
+
+# Setup master node
+my $node_master = get_new_node("master");
+$node_master->init(allows_streaming => 1);
+$node_master->append_conf('postgresql.conf', qq(
+max_prepared_transactions = 10
+));
+$node_master->start;
+$node_master->backup('master_backup');
+$node_master->psql('postgres', "create table t(id int)");
+
+# Setup master node
+my $node_slave = get_new_node('slave');
+$node_slave->init_from_backup($node_master, 'master_backup', has_streaming => 1);
+$node_slave->start;
+
+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+	synchronous_standby_names = '*'
+));
+$node_master->psql('postgres', "select pg_reload_conf()");
+
+my $psql_out = '';
+my $psql_rc = '';
+
+###############################################################################
+# Check that we can commit and abort tx after soft restart.
+# Here checkpoint happens before shutdown and no WAL replay will not occur
+# during start. So postgres should re-create memory state from files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	prepare transaction 'y';");
+$node_master->stop;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared tx after restart.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared tx after restart.');
+
+###############################################################################
+# Check that we can commit and abort after hard restart.
+# On startup WAL replay will re-create memory for global transactions that
+# happend after the last checkpoint.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	prepare transaction 'y';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared tx after teardown.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared tx after teardown.');
+
+###############################################################################
+# Check that we can replay several tx with same name.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Replay several tx with same name.');
+
+###############################################################################
+# Check that WAL replay will cleanup it's memory state and release locks while
+# replaying commit.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_master->teardown_node;
+$node_master->start;
+$psql_rc = $node_master->psql('postgres',"
+	begin;
+	insert into t values (42);
+	-- This prepare can fail due to 2pc identifier or locks conflicts if replay
+	-- didn't fully cleanup it's state on commit.
+	prepare transaction 'x';");
+is($psql_rc, '0', "Check that replay will cleanup it's memory state");
+
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that WAL replay will cleanup it's memory state on running slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';
+	");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;", stdout => \$psql_out);
+is($psql_out, '0', "Check that replay will cleanup it's memory state on running slave");
+
+###############################################################################
+# The same as in previous case, but let's force checkpoint on slave between
+# prepare and commit.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	");
+$node_slave->psql('postgres',"checkpoint;");
+$node_master->psql('postgres', "commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;", stdout => \$psql_out);
+is($psql_out, '0', "Check that replay will cleanup it's memory state on slave after checkpoint");
+
+###############################################################################
+# Check that we can commit transaction on promoted slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	");
+$node_master->teardown_node;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$psql_rc = $node_slave->psql('postgres', "commit prepared 'x';");
+is($psql_rc, '0', "Restore prepared transaction on promoted slave.");
+
+# change roles
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+
+###############################################################################
+# Check that we restore prepared xacts after slave soft restart while master is
+# down. Since slave knows that master is down it uses different code path on
+# start.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->restart;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
+is($psql_out, '1', "Restore prepared xacts after slave soft restart while master is down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres',"commit prepared 'x'");
+
+###############################################################################
+# Check that we restore prepared xacts after slave hard restart while master is
+# down.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (242);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->teardown_node;
+$node_slave->start;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
+is($psql_out, '1', "Restore prepared xacts after slave hard restart while master is down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres',"commit prepared 'x'");
+
+
+###############################################################################
+# Commit prepared on master while slave is down.
+###############################################################################
+
+# Switch to asynchronous replication
+#$node_master->append_conf('postgresql.conf', qq(
+#	synchronous_standby_names = ''
+#));
+#$node_master->psql('postgres', "select pg_reload_conf()");
+
+
diff --git a/src/test/recovery/t/twophase_recovery_bug.pl b/src/test/recovery/t/twophase_recovery_bug.pl
new file mode 100644
index 0000000..9b03878
--- /dev/null
+++ b/src/test/recovery/t/twophase_recovery_bug.pl
@@ -0,0 +1,45 @@
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 2;
+
+# Setup master node
+my $node_master = get_new_node("master");
+$node_master->init(allows_streaming => 1);
+$node_master->append_conf('postgresql.conf', qq(
+max_prepared_transactions = 10
+));
+$node_master->start;
+$node_master->backup('master_backup');
+$node_master->psql('postgres', "create table t(id int)");
+
+# Setup slave node
+my $node_slave = get_new_node('slave');
+$node_slave->init_from_backup($node_master, 'master_backup', has_streaming => 1);
+$node_slave->start;
+
+my $psql_out = '';
+my $psql_rc = '';
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values(0);
+	create table t1(id int);
+	insert into t1 values(1);
+	create table t2(id int);
+	insert into t2 values(2);
+	savepoint s1;
+	drop table t1;
+	select * from t for update;
+	select * from t2 for share;
+	prepare transaction 'x';
+");
+sleep 2; # wait for changes to arrive on slave
+$node_slave->teardown_node;
+$node_master->psql('postgres',"commit prepared 'x'");
+$node_slave->start;
+$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
+is($psql_out, '0', "Commit prepared on master while slave is down.");
+$node_slave->psql('postgres',"select sum(id) from t2", stdout => \$psql_out);
+is($psql_out, '2', "Check that tx changes are visible.");
#63Simon Riggs
simon@2ndQuadrant.com
In reply to: Michael Paquier (#60)
Re: Speedup twophase transactions

On 7 April 2016 at 07:29, Michael Paquier <michael.paquier@gmail.com> wrote:

With the test case attached in my case the COMMIT PREPARED record does

not even get replayed.

I was surprised to see this in the test...

sleep 2; # wait for changes to arrive on slave

I think the test framework needs a WaitForLSN function to allow us to know
for certain that something has been delivered.

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#64Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Simon Riggs (#63)
Re: Speedup twophase transactions

On 08 Apr 2016, at 16:36, Simon Riggs <simon@2ndQuadrant.com> wrote:

On 7 April 2016 at 07:29, Michael Paquier <michael.paquier@gmail.com> wrote:

With the test case attached in my case the COMMIT PREPARED record does
not even get replayed.

I was surprised to see this in the test...

sleep 2; # wait for changes to arrive on slave

I think the test framework needs a WaitForLSN function to allow us to know for certain that something has been delivered.

Yes, test framework already has that function. That was just quick script to reproduce bug, that Michael faced.
If there will be deterministic way to reproduce that bug, i'll rework it and move to 00X_twophase.pl

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#65Robert Haas
robertmhaas@gmail.com
In reply to: Jesper Pedersen (#61)
Re: Speedup twophase transactions

On Fri, Apr 8, 2016 at 8:49 AM, Jesper Pedersen
<jesper.pedersen@redhat.com> wrote:

On 04/07/2016 02:29 AM, Michael Paquier wrote:

So recovery is conflicting here. My guess is that this patch is
missing some lock cleanup.

With the test case attached in my case the COMMIT PREPARED record does
not even get replayed.

Should we create an entry for the open item list [0] for this, due to the
replication lag [1] ?

CommitFest entry [2]
Original commit [3]

Cc'ed RMT.

If there is something you think needs to be fixed that is a new issue
in 9.6, then yes you should. I don't quite understand what thing is
from reading this, so please make sure to describe it clearly.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#66Robert Haas
robertmhaas@gmail.com
In reply to: Stas Kelvich (#44)
Re: Speedup twophase transactions

On Tue, Jan 26, 2016 at 7:43 AM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Hi,

Thanks for reviews and commit!

I apologize for being clueless here, but was this patch committed?
It's still marked as "Needs Review" in the CommitFest application.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#67Jesper Pedersen
jesper.pedersen@redhat.com
In reply to: Robert Haas (#66)
Re: Speedup twophase transactions

On 04/08/2016 02:42 PM, Robert Haas wrote:

On Tue, Jan 26, 2016 at 7:43 AM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Hi,

Thanks for reviews and commit!

I apologize for being clueless here, but was this patch committed?
It's still marked as "Needs Review" in the CommitFest application.

There are 2 parts to this - both in the same email thread.

Part 1 [0]http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=978b2f65aa1262eb4ecbf8b3785cb1b9cf4db78e dealt with 2-phase commits on the master node. Part 2 [1]https://commitfest.postgresql.org/9/523/
deals with replaying on slaves, which currently shows lag.

There is still an open item found by Michael, so part 2 isn't ready to
be moved to "Ready for Committer" yet.

[0]: http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=978b2f65aa1262eb4ecbf8b3785cb1b9cf4db78e
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=978b2f65aa1262eb4ecbf8b3785cb1b9cf4db78e
[1]: https://commitfest.postgresql.org/9/523/

Best regards,
Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#68Jesper Pedersen
jesper.pedersen@redhat.com
In reply to: Robert Haas (#65)
Re: Speedup twophase transactions

On 04/08/2016 02:37 PM, Robert Haas wrote:

On Fri, Apr 8, 2016 at 8:49 AM, Jesper Pedersen
<jesper.pedersen@redhat.com> wrote:

On 04/07/2016 02:29 AM, Michael Paquier wrote:

So recovery is conflicting here. My guess is that this patch is
missing some lock cleanup.

With the test case attached in my case the COMMIT PREPARED record does
not even get replayed.

Should we create an entry for the open item list [0] for this, due to the
replication lag [1] ?

CommitFest entry [2]
Original commit [3]

Cc'ed RMT.

If there is something you think needs to be fixed that is a new issue
in 9.6, then yes you should. I don't quite understand what thing is
from reading this, so please make sure to describe it clearly.

Michael, you seem to have the necessary permission for this. Could you
add an entry ?

Best regards,
Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#69Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Robert Haas (#66)
Re: Speedup twophase transactions

On 08 Apr 2016, at 21:42, Robert Haas <robertmhaas@gmail.com> wrote:

On Tue, Jan 26, 2016 at 7:43 AM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Hi,

Thanks for reviews and commit!

I apologize for being clueless here, but was this patch committed?
It's still marked as "Needs Review" in the CommitFest application.

There was a patch to skip two phase file creation when there were no checkpoint
between PREPARE and COMMIT, and that patch was commited.
But that patch didn’t touch anything in replay, so replay speed of 2pc is significantly slower
than 2pc in normal mode. And that can cause constantly increasing replication lag for async
replication.
After that i’ve wrote new patch introducing same behaviour in replay and used the same
mail thread. Now Michael found a (heisen)bug in second patch, that i can’t reproduce.

Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#70Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Stas Kelvich (#69)
Fwd: Speedup twophase transactions

On 08 Apr 2016, at 21:55, Jesper Pedersen <jesper.pedersen@redhat.com> wrote:

On 04/08/2016 02:42 PM, Robert Haas wrote:

On Tue, Jan 26, 2016 at 7:43 AM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Hi,

Thanks for reviews and commit!

I apologize for being clueless here, but was this patch committed?
It's still marked as "Needs Review" in the CommitFest application.

There are 2 parts to this - both in the same email thread.

Part 1 [0] dealt with 2-phase commits on the master node. Part 2 [1] deals with replaying on slaves, which currently shows lag.

There is still an open item found by Michael, so part 2 isn't ready to be moved to "Ready for Committer" yet.

[0] http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=978b2f65aa1262eb4ecbf8b3785cb1b9cf4db78e
[1] https://commitfest.postgresql.org/9/523/

Best regards,
Jesper

By the way, Jesper, can you, please, try to run tests in diff, that i’ve sent today? It includes test scenario that was
causing problems for Michael, but works fine on all systems that I have access to.

Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#71Noah Misch
noah@leadboat.com
In reply to: Jesper Pedersen (#68)
Re: Speedup twophase transactions

On Fri, Apr 08, 2016 at 02:57:00PM -0400, Jesper Pedersen wrote:

On 04/08/2016 02:37 PM, Robert Haas wrote:

On Fri, Apr 8, 2016 at 8:49 AM, Jesper Pedersen <jesper.pedersen@redhat.com> wrote:

Should we create an entry for the open item list [0] for this, due to the
replication lag [1] ?

CommitFest entry [2]
Original commit [3]

Cc'ed RMT.

If there is something you think needs to be fixed that is a new issue
in 9.6, then yes you should. I don't quite understand what thing is
from reading this, so please make sure to describe it clearly.

Michael, you seem to have the necessary permission for this. Could you add
an entry ?

Everyone may edit the list; follow
https://wiki.postgresql.org/wiki/WikiEditing to setup access.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#72Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Stas Kelvich (#62)
1 attachment(s)
Re: Speedup twophase transactions

On 08 Apr 2016, at 16:09, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Tried on linux and os x 10.11 and 10.4.

Still can’t reproduce, but have played around with your backtrace.

I see in xlodump on slave following sequence of records:

rmgr: Storage len (rec/tot): 16/ 42, tx: 0, lsn: 0/03015AF0, prev 0/03015958, desc: CREATE base/12669/16387
rmgr: Heap len (rec/tot): 3/ 1518, tx: 867, lsn: 0/03015B20, prev 0/03015AF0, desc: INSERT off 8, blkref #0: rel 1663/12669/1247 blk 8 FPW
<...>
rmgr: Btree len (rec/tot): 2/ 72, tx: 867, lsn: 0/03019CD0, prev 0/03019C88, desc: INSERT_LEAF off 114, blkref #0: rel 1663/12669/2674 blk 22
rmgr: Standby len (rec/tot): 16/ 42, tx: 867, lsn: 0/03019D18, prev 0/03019CD0, desc: LOCK xid 867 db 12669 rel 16387
rmgr: Transaction len (rec/tot): 784/ 813, tx: 867, lsn: 0/03019D48, prev 0/03019D18, desc: PREPARE
rmgr: Transaction len (rec/tot): 380/ 409, tx: 0, lsn: 0/0301A090, prev 0/03019D48, desc: COMMIT_PREPARED 867: 2016-04-08 14:38:33.347851 MSK;

It looks like that you had stuck in LOCK xid 867 even before replaying PREPARE record, so I have not that much ideas on why that can be caused by changing procedures of PREPARE replay.

Just to keep things sane, here is my current diff:

<twophase_replay.v4.patch>

Michael, it looks like that you are the only one person who can reproduce that bug. I’ve tried on bunch of OS’s and didn’t observe that behaviour, also looking at your backtraces I can’t get who is holding this lock (and all of that happens before first prepare record is replayed).

Can you investigate it more? Particularly find out who holds the lock?

There is last version of the patch:

Attachments:

twophase_replay.v4.patchapplication/octet-stream; name=twophase_replay.v4.patchDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index a65048b..bc9bffa 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,8 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *
+ *		The same procedure happens during replication and crash recovery.
  *
  *-------------------------------------------------------------------------
  */
@@ -578,6 +578,37 @@ LockGXact(const char *gid, Oid user)
 }
 
 /*
+ * LockGXactByXid
+ *
+ * Find prepared transaction by xid and lock corresponding gxact.
+ * This is used during recovery as an alternative to LockGXact().
+ */
+static GlobalTransaction
+LockGXactByXid(TransactionId xid)
+{
+	int i;
+	GlobalTransaction gxact = NULL;
+	PGXACT	   *pgxact;
+
+	LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		gxact = TwoPhaseState->prepXacts[i];
+		pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (TransactionIdEquals(xid, pgxact->xid))
+		{
+			gxact->locking_backend = MyBackendId;
+			MyLockedGxact = gxact;
+			break;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+	return gxact;
+}
+
+/*
  * RemoveGXact
  *		Remove the prepared transaction from the shared memory array.
  *
@@ -1241,9 +1272,9 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
  * Reads 2PC data from xlog. During checkpoint this data will be moved to
  * twophase files and ReadTwoPhaseFile should be used instead.
  *
- * Note clearly that this function accesses WAL during normal operation, similarly
- * to the way WALSender or Logical Decoding would do. It does not run during
- * crash recovery or standby processing.
+ * Note clearly that this function access WAL not only during recovery/replay
+ * but also during normal operation, similarly to the way WALSender or
+ * Logical Decoding would do.
  */
 static void
 XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
@@ -1252,8 +1283,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1296,12 +1325,30 @@ StandbyTransactionIdIsPrepared(TransactionId xid)
 	char	   *buf;
 	TwoPhaseFileHeader *hdr;
 	bool		result;
+	int			i;
 
 	Assert(TransactionIdIsValid(xid));
 
 	if (max_prepared_xacts <= 0)
 		return false;			/* nothing to do */
 
+	/*
+	 * At first check prepared tx that wasn't yet moved to disk.
+	 */
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		PGXACT *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (TransactionIdEquals(pgxact->xid, xid))
+		{
+			LWLockRelease(TwoPhaseStateLock);
+			return true;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
 	/* Read and validate file */
 	buf = ReadTwoPhaseFile(xid, false);
 	if (buf == NULL)
@@ -1316,12 +1363,17 @@ StandbyTransactionIdIsPrepared(TransactionId xid)
 }
 
 /*
- * FinishPreparedTransaction: execute COMMIT PREPARED or ROLLBACK PREPARED
+ * FinishGXact
+ *
+ * Do the actual finish of COMMIT/ABORT PREPARED. It is a caller
+ * responsibility to properly lock corresponding gxact.
+ *
+ * This function can be called during replay to clean memory state
+ * for previously prepared xact. In that case actions are the same
+ * as in normal mode but without any writes to WAL or files.
  */
-void
-FinishPreparedTransaction(const char *gid, bool isCommit)
+static void FinishGXact(GlobalTransaction gxact, bool isCommit)
 {
-	GlobalTransaction gxact;
 	PGPROC	   *proc;
 	PGXACT	   *pgxact;
 	TransactionId xid;
@@ -1337,11 +1389,6 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	SharedInvalidationMessage *invalmsgs;
 	int			i;
 
-	/*
-	 * Validate the GID, and lock the GXACT to ensure that two backends do not
-	 * try to commit the same GID at once.
-	 */
-	gxact = LockGXact(gid, GetUserId());
 	proc = &ProcGlobal->allProcs[gxact->pgprocno];
 	pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 	xid = pgxact->xid;
@@ -1385,16 +1432,19 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	 * progress), then run the post-commit or post-abort callbacks. The
 	 * callbacks will release the locks the transaction held.
 	 */
-	if (isCommit)
-		RecordTransactionCommitPrepared(xid,
+	if (!RecoveryInProgress())
+	{
+		if (isCommit)
+			RecordTransactionCommitPrepared(xid,
 										hdr->nsubxacts, children,
 										hdr->ncommitrels, commitrels,
 										hdr->ninvalmsgs, invalmsgs,
 										hdr->initfileinval);
-	else
-		RecordTransactionAbortPrepared(xid,
+		else
+			RecordTransactionAbortPrepared(xid,
 									   hdr->nsubxacts, children,
 									   hdr->nabortrels, abortrels);
+	}
 
 	ProcArrayRemove(proc, latestXid);
 
@@ -1425,12 +1475,15 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 		delrels = abortrels;
 		ndelrels = hdr->nabortrels;
 	}
-	for (i = 0; i < ndelrels; i++)
+	if (!RecoveryInProgress())
 	{
-		SMgrRelation srel = smgropen(delrels[i], InvalidBackendId);
+		for (i = 0; i < ndelrels; i++)
+		{
+			SMgrRelation srel = smgropen(delrels[i], InvalidBackendId);
 
-		smgrdounlink(srel, false);
-		smgrclose(srel);
+			smgrdounlink(srel, false);
+			smgrclose(srel);
+		}
 	}
 
 	/*
@@ -1439,11 +1492,14 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	 * Relcache init file invalidation requires processing both before and
 	 * after we send the SI messages. See AtEOXact_Inval()
 	 */
-	if (hdr->initfileinval)
-		RelationCacheInitFilePreInvalidate();
-	SendSharedInvalidMessages(invalmsgs, hdr->ninvalmsgs);
-	if (hdr->initfileinval)
-		RelationCacheInitFilePostInvalidate();
+	if (!RecoveryInProgress())
+	{
+		if (hdr->initfileinval)
+			RelationCacheInitFilePreInvalidate();
+		SendSharedInvalidMessages(invalmsgs, hdr->ninvalmsgs);
+		if (hdr->initfileinval)
+			RelationCacheInitFilePostInvalidate();
+	}
 
 	/* And now do the callbacks */
 	if (isCommit)
@@ -1469,6 +1525,49 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 }
 
 /*
+ * FinishPreparedTransaction: execute COMMIT PREPARED or ROLLBACK PREPARED
+ */
+void
+FinishPreparedTransaction(const char *gid, bool isCommit)
+{
+	GlobalTransaction gxact;
+
+	/*
+	 * Validate the GID, and lock the GXACT to ensure that two backends do not
+	 * try to commit the same GID at once.
+	 */
+	gxact = LockGXact(gid, GetUserId());
+	FinishGXact(gxact, isCommit);
+}
+
+/*
+ * XlogRedoFinishPrepared()
+ *
+ * This function is called during replay when xlog reader faces 2pc commit or
+ * abort record. That function should clean up memory state that was created
+ * while replaying prepare xlog record.
+ */
+void
+XlogRedoFinishPrepared(TransactionId xid, bool isCommit)
+{
+	GlobalTransaction gxact;
+
+	Assert(RecoveryInProgress());
+
+	gxact = LockGXactByXid(xid);
+
+	/*
+	 * If requested xid wasn't found that means that prepare record was moved
+	 * to files before our replay started. That's okay and we have nothing to
+	 * clean/finish.
+	 */
+	if (!gxact)
+		return;
+
+	FinishGXact(gxact, isCommit);
+}
+
+/*
  * Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
  */
 static void
@@ -1690,7 +1789,48 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 	TransactionId *xids = NULL;
 	int			nxids = 0;
 	int			allocsize = 0;
+	int			i;
 
+	/*
+	 * We need to check the PGXACT array for prepared transactions that doesn't
+	 * have any state file in case of a slave restart with the master being off.
+	 */
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (!gxact->valid)
+			continue;
+
+		if (TransactionIdPrecedes(pgxact->xid, result))
+			result = pgxact->xid;
+
+		if (xids_p)
+		{
+			if (nxids == allocsize)
+			{
+				if (nxids == 0)
+				{
+					allocsize = 10;
+					xids = palloc(allocsize * sizeof(TransactionId));
+				}
+				else
+				{
+					allocsize = allocsize * 2;
+					xids = repalloc(xids, allocsize * sizeof(TransactionId));
+				}
+			}
+			xids[nxids++] = pgxact->xid;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+
+	/*
+	 * And now scan files in pg_twophase directory
+	 */
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
 	{
@@ -1701,7 +1841,6 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 			char	   *buf;
 			TwoPhaseFileHeader *hdr;
 			TransactionId *subxids;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
@@ -1809,102 +1948,105 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 }
 
 /*
- * StandbyRecoverPreparedTransactions
+ * RecoverPreparedFromBuffer
+ *
+ * Parse data in given buffer (that can be a pointer to WAL record or file)
+ * and load shared-memory state for that prepared transaction.
  *
- * Scan the pg_twophase directory and setup all the required information to
- * allow standby queries to treat prepared transactions as still active.
- * This is never called at the end of recovery - we use
- * RecoverPreparedTransactions() at that point.
+ * It's a caller responsibility to call MarkAsPrepared() on returned gxact.
  *
- * Currently we simply call SubTransSetParent() for any subxids of prepared
- * transactions. If overwriteOK is true, it's OK if some XIDs have already
- * been marked in pg_subtrans.
  */
-void
-StandbyRecoverPreparedTransactions(bool overwriteOK)
+static GlobalTransaction
+RecoverPreparedFromBuffer(char *buf, bool forceOverwriteOK)
 {
-	DIR		   *cldir;
-	struct dirent *clde;
+	char			*bufptr;
+	const char		*gid;
+	TransactionId	*subxids;
+	bool			overwriteOK = false;
+	int				i;
+	GlobalTransaction gxact;
+	TwoPhaseFileHeader	*hdr;
 
-	cldir = AllocateDir(TWOPHASE_DIR);
-	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
-	{
-		if (strlen(clde->d_name) == 8 &&
-			strspn(clde->d_name, "0123456789ABCDEF") == 8)
-		{
-			TransactionId xid;
-			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
+	/* Deconstruct header */
+	hdr = (TwoPhaseFileHeader *) buf;
+	bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+	gid = (const char *) bufptr;
+	bufptr += MAXALIGN(hdr->gidlen);
+	subxids = (TransactionId *) bufptr;
+	bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
+	bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
 
-			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
+	/*
+	 * It's possible that SubTransSetParent has been set before, if
+	 * the prepared transaction generated xid assignment records. Test
+	 * here must match one used in AssignTransactionId().
+	 */
+	if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
+						 XLogLogicalInfoActive()))
+		overwriteOK = true;
 
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+	/*
+	 * Caller can also force overwriteOK.
+	 */
+	if (forceOverwriteOK)
+		overwriteOK = true;
 
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+	/*
+	 * Reconstruct subtrans state for the transaction --- needed
+	 * because pg_subtrans is not preserved over a restart.  Note that
+	 * we are linking all the subtransactions directly to the
+	 * top-level XID; there may originally have been a more complex
+	 * hierarchy, but there's no need to restore that exactly.
+	 */
+	for (i = 0; i < hdr->nsubxacts; i++)
+		SubTransSetParent(subxids[i], hdr->xid, overwriteOK);
 
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				pfree(buf);
-				continue;
-			}
+	/*
+	 * Recreate its GXACT and dummy PGPROC
+	 */
+	gxact = MarkAsPreparing(hdr->xid, gid,
+							hdr->prepared_at,
+							hdr->owner, hdr->database);
+	GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
 
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID.
-			 */
-			subxids = (TransactionId *)
-				(buf + MAXALIGN(sizeof(TwoPhaseFileHeader)));
-			for (i = 0; i < hdr->nsubxacts; i++)
-			{
-				TransactionId subxid = subxids[i];
+	/*
+	 * Recover other state (notably locks) using resource managers
+	 */
+	ProcessRecords(bufptr, hdr->xid, twophase_recover_callbacks);
 
-				Assert(TransactionIdFollows(subxid, xid));
-				SubTransSetParent(xid, subxid, overwriteOK);
-			}
-		}
-	}
-	FreeDir(cldir);
+	/*
+	 * Release locks held by the standby process after we process each
+	 * prepared transaction. As a result, we don't need too many
+	 * additional locks at any one time.
+	 */
+	if (InHotStandby)
+		StandbyReleaseLockTree(hdr->xid, hdr->nsubxacts, subxids);
+
+	/*
+	 * We're done with recovering this transaction. Clear
+	 * MyLockedGxact, like we do in PrepareTransaction() during normal
+	 * operation.
+	 */
+	PostPrepare_Twophase();
+
+	return gxact;
 }
 
 /*
- * RecoverPreparedTransactions
+ * RecoverPreparedFromFiles
  *
  * Scan the pg_twophase directory and reload shared-memory state for each
  * prepared transaction (reacquire locks, etc).  This is run during database
  * startup.
  */
 void
-RecoverPreparedTransactions(void)
+RecoverPreparedFromFiles(bool forceOverwriteOK)
 {
 	char		dir[MAXPGPATH];
 	DIR		   *cldir;
 	struct dirent *clde;
-	bool		overwriteOK = false;
 
 	snprintf(dir, MAXPGPATH, "%s", TWOPHASE_DIR);
 
@@ -1916,15 +2058,27 @@ RecoverPreparedTransactions(void)
 		{
 			TransactionId xid;
 			char	   *buf;
-			char	   *bufptr;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
 			GlobalTransaction gxact;
-			const char	*gid;
 			int			i;
+			PGXACT	   *pgxact;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
+			/* Already recovered from WAL? */
+			LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+			for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+			{
+				gxact = TwoPhaseState->prepXacts[i];
+				pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+				if (TransactionIdEquals(xid, pgxact->xid))
+				{
+					LWLockRelease(TwoPhaseStateLock);
+					goto next_file;
+				}
+			}
+			LWLockRelease(TwoPhaseStateLock);
+
 			/* Already processed? */
 			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
 			{
@@ -1949,73 +2103,44 @@ RecoverPreparedTransactions(void)
 			ereport(LOG,
 					(errmsg("recovering prepared transaction %u", xid)));
 
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			Assert(TransactionIdEquals(hdr->xid, xid));
-			bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
-			gid = (const char *) bufptr;
-			bufptr += MAXALIGN(hdr->gidlen);
-			subxids = (TransactionId *) bufptr;
-			bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
-			bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
-
-			/*
-			 * It's possible that SubTransSetParent has been set before, if
-			 * the prepared transaction generated xid assignment records. Test
-			 * here must match one used in AssignTransactionId().
-			 */
-			if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
-								 XLogLogicalInfoActive()))
-				overwriteOK = true;
-
-			/*
-			 * Reconstruct subtrans state for the transaction --- needed
-			 * because pg_subtrans is not preserved over a restart.  Note that
-			 * we are linking all the subtransactions directly to the
-			 * top-level XID; there may originally have been a more complex
-			 * hierarchy, but there's no need to restore that exactly.
-			 */
-			for (i = 0; i < hdr->nsubxacts; i++)
-				SubTransSetParent(subxids[i], xid, overwriteOK);
-
-			/*
-			 * Recreate its GXACT and dummy PGPROC
-			 */
-			gxact = MarkAsPreparing(xid, gid,
-									hdr->prepared_at,
-									hdr->owner, hdr->database);
+			gxact = RecoverPreparedFromBuffer(buf, forceOverwriteOK);
 			gxact->ondisk = true;
-			GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
 			MarkAsPrepared(gxact);
 
-			/*
-			 * Recover other state (notably locks) using resource managers
-			 */
-			ProcessRecords(bufptr, xid, twophase_recover_callbacks);
-
-			/*
-			 * Release locks held by the standby process after we process each
-			 * prepared transaction. As a result, we don't need too many
-			 * additional locks at any one time.
-			 */
-			if (InHotStandby)
-				StandbyReleaseLockTree(xid, hdr->nsubxacts, subxids);
-
-			/*
-			 * We're done with recovering this transaction. Clear
-			 * MyLockedGxact, like we do in PrepareTransaction() during normal
-			 * operation.
-			 */
-			PostPrepare_Twophase();
-
 			pfree(buf);
 		}
+
+next_file:
+		continue;
+
 	}
 	FreeDir(cldir);
 }
 
+
+/*
+ * RecoverPreparedFromXLOG
+ *
+ * To avoid creation of state files during replay we registering
+ * prepare xlog records in shared memory in the same way as it happens
+ * while not in recovery. If replay faces commit xlog record before
+ * checkpoint/restartpoint happens then we avoid using files at all.
+ *
+ * We need this behaviour because the speed of the 2PC replay on the replica
+ * should be at least the same as the 2PC transaction speed of the master.
+ */
+void
+RecoverPreparedFromXLOG(XLogReaderState *record)
+{
+	GlobalTransaction gxact;
+
+	gxact = RecoverPreparedFromBuffer((char *) XLogRecGetData(record), false);
+	gxact->prepare_start_lsn = record->ReadRecPtr;
+	gxact->prepare_end_lsn = record->EndRecPtr;
+	MarkAsPrepared(gxact);
+}
+
+
 /*
  *	RecordTransactionCommitPrepared
  *
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 7e37331..9323ba2 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -5588,7 +5588,7 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+			XlogRedoFinishPrepared(parsed.twophase_xid, true);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5608,14 +5608,12 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+			XlogRedoFinishPrepared(parsed.twophase_xid, false);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		RecoverPreparedFromXLOG(record);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index f9644db..11c06b4 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6630,7 +6630,7 @@ StartupXLOG(void)
 
 				ProcArrayApplyRecoveryInfo(&running);
 
-				StandbyRecoverPreparedTransactions(false);
+				RecoverPreparedFromFiles(false);
 			}
 		}
 
@@ -7383,7 +7383,7 @@ StartupXLOG(void)
 	TrimMultiXact();
 
 	/* Reload shared-memory state for prepared transactions */
-	RecoverPreparedTransactions();
+	RecoverPreparedFromFiles(false);
 
 	/*
 	 * Shutdown the recovery environment. This must occur after
@@ -9297,7 +9297,7 @@ xlog_redo(XLogReaderState *record)
 
 			ProcArrayApplyRecoveryInfo(&running);
 
-			StandbyRecoverPreparedTransactions(true);
+			RecoverPreparedFromFiles(true);
 		}
 
 		/* ControlFile->checkPointCopy always tracks the latest ckpt XID */
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b7ce0c6..416ef5e 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -17,6 +17,7 @@
 #include "access/xlogdefs.h"
 #include "datatype/timestamp.h"
 #include "storage/lock.h"
+#include "access/xlogreader.h"
 
 /*
  * GlobalTransactionData is defined in twophase.c; other places have no
@@ -46,8 +47,8 @@ extern bool StandbyTransactionIdIsPrepared(TransactionId xid);
 
 extern TransactionId PrescanPreparedTransactions(TransactionId **xids_p,
 							int *nxids_p);
-extern void StandbyRecoverPreparedTransactions(bool overwriteOK);
-extern void RecoverPreparedTransactions(void);
+extern void RecoverPreparedFromFiles(bool overwriteOK);
+extern void RecoverPreparedFromXLOG(XLogReaderState *record);
 
 extern void RecreateTwoPhaseFile(TransactionId xid, void *content, int len);
 extern void RemoveTwoPhaseFile(TransactionId xid, bool giveWarning);
@@ -56,4 +57,5 @@ extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void XlogRedoFinishPrepared(TransactionId xid, bool isCommit);
 #endif   /* TWOPHASE_H */
diff --git a/src/test/recovery/t/007_twophase.pl b/src/test/recovery/t/007_twophase.pl
new file mode 100644
index 0000000..cfc7316
--- /dev/null
+++ b/src/test/recovery/t/007_twophase.pl
@@ -0,0 +1,236 @@
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 11;
+
+# Setup master node
+my $node_master = get_new_node("master");
+$node_master->init(allows_streaming => 1);
+$node_master->append_conf('postgresql.conf', qq(
+max_prepared_transactions = 10
+));
+$node_master->start;
+$node_master->backup('master_backup');
+$node_master->psql('postgres', "create table t(id int)");
+
+# Setup master node
+my $node_slave = get_new_node('slave');
+$node_slave->init_from_backup($node_master, 'master_backup', has_streaming => 1);
+$node_slave->start;
+
+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+	synchronous_standby_names = '*'
+));
+$node_master->psql('postgres', "select pg_reload_conf()");
+
+my $psql_out = '';
+my $psql_rc = '';
+
+###############################################################################
+# Check that we can commit and abort tx after soft restart.
+# Here checkpoint happens before shutdown and no WAL replay will not occur
+# during start. So postgres should re-create memory state from files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	prepare transaction 'y';");
+$node_master->stop;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared tx after restart.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared tx after restart.');
+
+###############################################################################
+# Check that we can commit and abort after hard restart.
+# On startup WAL replay will re-create memory for global transactions that
+# happend after the last checkpoint.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	prepare transaction 'y';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared tx after teardown.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared tx after teardown.');
+
+###############################################################################
+# Check that we can replay several tx with same name.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Replay several tx with same name.');
+
+###############################################################################
+# Check that WAL replay will cleanup it's memory state and release locks while
+# replaying commit.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_master->teardown_node;
+$node_master->start;
+$psql_rc = $node_master->psql('postgres',"
+	begin;
+	insert into t values (42);
+	-- This prepare can fail due to 2pc identifier or locks conflicts if replay
+	-- didn't fully cleanup it's state on commit.
+	prepare transaction 'x';");
+is($psql_rc, '0', "Check that replay will cleanup it's memory state");
+
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that WAL replay will cleanup it's memory state on running slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';
+	");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;", stdout => \$psql_out);
+is($psql_out, '0', "Check that replay will cleanup it's memory state on running slave");
+
+###############################################################################
+# The same as in previous case, but let's force checkpoint on slave between
+# prepare and commit.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	");
+$node_slave->psql('postgres',"checkpoint;");
+$node_master->psql('postgres', "commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;", stdout => \$psql_out);
+is($psql_out, '0', "Check that replay will cleanup it's memory state on slave after checkpoint");
+
+###############################################################################
+# Check that we can commit transaction on promoted slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	");
+$node_master->teardown_node;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$psql_rc = $node_slave->psql('postgres', "commit prepared 'x';");
+is($psql_rc, '0', "Restore prepared transaction on promoted slave.");
+
+# change roles
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+
+###############################################################################
+# Check that we restore prepared xacts after slave soft restart while master is
+# down. Since slave knows that master is down it uses different code path on
+# start.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->restart;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
+is($psql_out, '1', "Restore prepared xacts after slave soft restart while master is down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres',"commit prepared 'x'");
+
+###############################################################################
+# Check that we restore prepared xacts after slave hard restart while master is
+# down.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (242);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->teardown_node;
+$node_slave->start;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
+is($psql_out, '1', "Restore prepared xacts after slave hard restart while master is down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres',"commit prepared 'x'");
+
+
+###############################################################################
+# Commit prepared on master while slave is down.
+###############################################################################
+
+# Switch to asynchronous replication
+#$node_master->append_conf('postgresql.conf', qq(
+#	synchronous_standby_names = ''
+#));
+#$node_master->psql('postgres', "select pg_reload_conf()");
+
+
diff --git a/src/test/recovery/t/twophase_recovery_bug.pl b/src/test/recovery/t/twophase_recovery_bug.pl
new file mode 100644
index 0000000..9b03878
--- /dev/null
+++ b/src/test/recovery/t/twophase_recovery_bug.pl
@@ -0,0 +1,45 @@
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 2;
+
+# Setup master node
+my $node_master = get_new_node("master");
+$node_master->init(allows_streaming => 1);
+$node_master->append_conf('postgresql.conf', qq(
+max_prepared_transactions = 10
+));
+$node_master->start;
+$node_master->backup('master_backup');
+$node_master->psql('postgres', "create table t(id int)");
+
+# Setup slave node
+my $node_slave = get_new_node('slave');
+$node_slave->init_from_backup($node_master, 'master_backup', has_streaming => 1);
+$node_slave->start;
+
+my $psql_out = '';
+my $psql_rc = '';
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values(0);
+	create table t1(id int);
+	insert into t1 values(1);
+	create table t2(id int);
+	insert into t2 values(2);
+	savepoint s1;
+	drop table t1;
+	select * from t for update;
+	select * from t2 for share;
+	prepare transaction 'x';
+");
+sleep 2; # wait for changes to arrive on slave
+$node_slave->teardown_node;
+$node_master->psql('postgres',"commit prepared 'x'");
+$node_slave->start;
+$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
+is($psql_out, '0', "Commit prepared on master while slave is down.");
+$node_slave->psql('postgres',"select sum(id) from t2", stdout => \$psql_out);
+is($psql_out, '2', "Check that tx changes are visible.");
#73Michael Paquier
michael.paquier@gmail.com
In reply to: Stas Kelvich (#72)
2 attachment(s)
Re: Speedup twophase transactions

On Mon, Apr 11, 2016 at 7:16 PM, Stas Kelvich wrote:

Michael, it looks like that you are the only one person who can reproduce that bug. I’ve tried on bunch of OS’s and didn’t observe that behaviour, also looking at your backtraces I can’t get who is holding this lock (and all of that happens before first prepare record is replayed).

Where did you try it. FWIW, I can reproduce that on Linux and OSX, and
only manually though:
1) Create a master and a streaming standby.
2) Run the following on master:
BEGIN;
CREATE TABLE foo (a int);
PREPARE TRANSACTION 'tx';
3) stop -m immediate the standby
4) COMMIT PREPARED 'tx' on master
5) Restart standby, and the node will wait for a lock

Can you investigate it more? Particularly find out who holds the lock?

OK, so if I look at this backtrace that's a standby LOCK record, but
we already know that:
frame #9: 0x0000000107600383
postgres`LockAcquireExtended(locktag=0x00007fff58a4b228, lockmode=8,
sessionLock='\x01', dontWait='\0', reportMemoryError='\0') + 2819 at
lock.c:998
frame #10: 0x00000001075f9cd6
postgres`StandbyAcquireAccessExclusiveLock(xid=867, dbOid=16384,
relOid=16385) + 358 at standby.c:627
* frame #11: 0x00000001075fa23b
postgres`standby_redo(record=0x00007f90a9841e38) + 251 at
standby.c:809
frame #12: 0x0000000107298d37 postgres`StartupXLOG + 9351 at xlog.c:6871

Here is the record pointer:
(lldb) p *record
(XLogReaderState) $4 = {
read_page = 0x000000010729b3c0 (postgres`XLogPageRead at xlog.c:10973)
system_identifier = 6272572355656465658
private_data = 0x00007fff58a4bf40
ReadRecPtr = 50424128
EndRecPtr = 50424176
decoded_record = 0x00007f90a9843178
main_data = 0x00007f90a9804b78 "\x01"
main_data_len = 16
main_data_bufsz = 784
record_origin = 0
blocks = {
[0]: = { in_use = '\0' rnode = (spcNode = 1663, dbNode = 16384, relNode = 2674) forknum = MAIN_FORKNUM blkno = 22 flags = '\x10' has_image = '\0' bkp_image = 0x00007f90a984826b "\x01" hole_offset = 892 hole_length = 2076 bimg_len = 6116 bimg_info = '\x01' has_data = '\0' data = 0x00007f90a98595d8 "\a" data_len = 0 data_bufsz = 154 }
in_use = '\0'
rnode = (spcNode = 1663, dbNode = 16384, relNode = 2674)
forknum = MAIN_FORKNUM
blkno = 22
flags = '\x10'
has_image = '\0'
bkp_image = 0x00007f90a984826b "\x01"
hole_offset = 892
hole_length = 2076
bimg_len = 6116
bimg_info = '\x01'
has_data = '\0'
data = 0x00007f90a98595d8 "\a"
data_len = 0
data_bufsz = 154
}

And in our case this corresponds to the record with LSN 0/03016940
that cannot take an exclusive LOCK:
rmgr: Transaction len (rec/tot): 784/ 813, tx: 867, lsn:
0/03016610, prev 0/030165D8, desc: PREPARE
rmgr: Standby len (rec/tot): 16/ 42, tx: 0, lsn:
0/03016940, prev 0/03016610, desc: LOCK xid 867 db 16384 rel 16385
rmgr: Standby len (rec/tot): 28/ 54, tx: 0, lsn:
0/03016970, prev 0/03016940, desc: RUNNING_XACTS nextXid 868
latestCompletedXid 866 oldestRunningXid 867; 1 xacts: 867

There are two XID locks taken before that:
rmgr: Standby len (rec/tot): 16/ 42, tx: 867, lsn:
0/03016578, prev 0/03014D40, desc: LOCK xid 867 db 16384 rel 16385
rmgr: Standby len (rec/tot): 16/ 42, tx: 0, lsn:
0/030165A8, prev 0/03016578, desc: LOCK xid 867 db 16384 rel 16385

And pg_locks on the standby is reporting that another lock has been
taken but not released:
=# select locktype, pid, mode, granted, fastpath from pg_locks where
relation = 16385;
locktype | pid | mode | granted | fastpath
----------+-------+---------------------+---------+----------
relation | 68955 | AccessExclusiveLock | f | f
relation | null | AccessExclusiveLock | t | f
(2 rows)
In this case 68955 is the startup process trying to take the lock for
the LOCK record and it is not granted yet:
ioltas 68955 0.0 0.0 2593064 3228 ?? TXs 4:44PM 0:00.05
postgres: startup process recovering 000000010000000000000003
waiting

Now there is already a lock that has been taken and granted,
conflicting here... As the relation is only PREPARE'd yet and not
COMMIT PREPARED at this stage of the replay, isn't this lock taken
during the PREPARE phase, then it is not released by your new code
paths, no?

[One LOCK_DEBUG build later...]

It looks to be the case... The PREPARE phase replayed after the
standby is restarted in recovery creates a series of exclusive locks
on the table created and those are not taken on HEAD. Once those are
replayed the LOCK_STANDBY record is conflicting with it. In the case
of the failure, the COMMIT PREPARED record cannot be fetched from
master via the WAL stream so the relation never becomes visible.

Attached as two log files that are the result of a couple of runs,
those are the logs of the standby after being restarted in crash
recovery
- 2pc_master_logs.log, for HEAD.
- 2pc_patch_logs.log, with your last patch applied.
Feel free to have a look at them.
Regards,
--
Michael

Attachments:

2pc_master_logs.logapplication/octet-stream; name=2pc_master_logs.logDownload
2pc_patch_logs.logapplication/octet-stream; name=2pc_patch_logs.logDownload
#74Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#73)
Re: Speedup twophase transactions

On 12 Apr 2016, at 15:47, Michael Paquier <michael.paquier@gmail.com> wrote:

On Mon, Apr 11, 2016 at 7:16 PM, Stas Kelvich wrote:

Michael, it looks like that you are the only one person who can reproduce that bug. I’ve tried on bunch of OS’s and didn’t observe that behaviour, also looking at your backtraces I can’t get who is holding this lock (and all of that happens before first prepare record is replayed).

Where did you try it. FWIW, I can reproduce that on Linux and OSX, and
only manually though:

Thanks a lot, Michael! Now I was able to reproduce that. Seems that when
you was performing manual setup, master instance issued checkpoint, but in
my script that didn’t happened because of shorter timing. There are tests
with checkpoint between prepare/commit in proposed test suite, but none of
them was issuing ddl.

It looks to be the case... The PREPARE phase replayed after the
standby is restarted in recovery creates a series of exclusive locks
on the table created and those are not taken on HEAD. Once those are
replayed the LOCK_STANDBY record is conflicting with it. In the case
of the failure, the COMMIT PREPARED record cannot be fetched from
master via the WAL stream so the relation never becomes visible.

Yep, it is. It is okay for prepared xact hold a locks for created/changed tables,
but code in standby_redo() was written in assumption that there are no prepared
xacts at the time of recovery. I’ll look closer at checkpointer code and will send
updated patch.

And thanks again.

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#75Michael Paquier
michael.paquier@gmail.com
In reply to: Stas Kelvich (#74)
Re: Speedup twophase transactions

On Wed, Apr 13, 2016 at 1:53 AM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

On 12 Apr 2016, at 15:47, Michael Paquier <michael.paquier@gmail.com> wrote:

It looks to be the case... The PREPARE phase replayed after the
standby is restarted in recovery creates a series of exclusive locks
on the table created and those are not taken on HEAD. Once those are
replayed the LOCK_STANDBY record is conflicting with it. In the case
of the failure, the COMMIT PREPARED record cannot be fetched from
master via the WAL stream so the relation never becomes visible.

Yep, it is. It is okay for prepared xact hold a locks for created/changed tables,
but code in standby_redo() was written in assumption that there are no prepared
xacts at the time of recovery. I’ll look closer at checkpointer code and will send
updated patch.

And thanks again.

That's too late for 9.6 unfortunately, don't forget to add that in the next CF!
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#76Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#75)
1 attachment(s)
Re: Speedup twophase transactions

On 13 Apr 2016, at 01:04, Michael Paquier <michael.paquier@gmail.com> wrote:

On Wed, Apr 13, 2016 at 1:53 AM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

On 12 Apr 2016, at 15:47, Michael Paquier <michael.paquier@gmail.com> wrote:

It looks to be the case... The PREPARE phase replayed after the
standby is restarted in recovery creates a series of exclusive locks
on the table created and those are not taken on HEAD. Once those are
replayed the LOCK_STANDBY record is conflicting with it. In the case
of the failure, the COMMIT PREPARED record cannot be fetched from
master via the WAL stream so the relation never becomes visible.

Yep, it is. It is okay for prepared xact hold a locks for created/changed tables,
but code in standby_redo() was written in assumption that there are no prepared
xacts at the time of recovery. I’ll look closer at checkpointer code and will send
updated patch.

And thanks again.

That's too late for 9.6 unfortunately, don't forget to add that in the next CF!

Fixed patch attached. There already was infrastructure to skip currently
held locks during replay of standby_redo() and I’ve extended that with check for
prepared xids.

The reason why I’m still active on this thread is because I see real problems
in deploying 9.6 in current state. Let me stress my concern: current state of things
_WILL_BREAK_ async replication in case of substantial load of two phase
transactions on master. And a lot of J2EE apps falls into that category, as they
wrap most of their transactions in prepare/commit. Slave server just will always
increase it lag and will never catch up. It is possible to deal with that by switching
to synchronous replication or inserting triggers with pg_sleep on master, but it
doesn’t looks like normal behaviour of system.

Attachments:

twophase_replay.v5.patchapplication/octet-stream; name=twophase_replay.v5.patchDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index a65048b..bc9bffa 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,8 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *
+ *		The same procedure happens during replication and crash recovery.
  *
  *-------------------------------------------------------------------------
  */
@@ -578,6 +578,37 @@ LockGXact(const char *gid, Oid user)
 }
 
 /*
+ * LockGXactByXid
+ *
+ * Find prepared transaction by xid and lock corresponding gxact.
+ * This is used during recovery as an alternative to LockGXact().
+ */
+static GlobalTransaction
+LockGXactByXid(TransactionId xid)
+{
+	int i;
+	GlobalTransaction gxact = NULL;
+	PGXACT	   *pgxact;
+
+	LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		gxact = TwoPhaseState->prepXacts[i];
+		pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (TransactionIdEquals(xid, pgxact->xid))
+		{
+			gxact->locking_backend = MyBackendId;
+			MyLockedGxact = gxact;
+			break;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+	return gxact;
+}
+
+/*
  * RemoveGXact
  *		Remove the prepared transaction from the shared memory array.
  *
@@ -1241,9 +1272,9 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
  * Reads 2PC data from xlog. During checkpoint this data will be moved to
  * twophase files and ReadTwoPhaseFile should be used instead.
  *
- * Note clearly that this function accesses WAL during normal operation, similarly
- * to the way WALSender or Logical Decoding would do. It does not run during
- * crash recovery or standby processing.
+ * Note clearly that this function access WAL not only during recovery/replay
+ * but also during normal operation, similarly to the way WALSender or
+ * Logical Decoding would do.
  */
 static void
 XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
@@ -1252,8 +1283,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1296,12 +1325,30 @@ StandbyTransactionIdIsPrepared(TransactionId xid)
 	char	   *buf;
 	TwoPhaseFileHeader *hdr;
 	bool		result;
+	int			i;
 
 	Assert(TransactionIdIsValid(xid));
 
 	if (max_prepared_xacts <= 0)
 		return false;			/* nothing to do */
 
+	/*
+	 * At first check prepared tx that wasn't yet moved to disk.
+	 */
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		PGXACT *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (TransactionIdEquals(pgxact->xid, xid))
+		{
+			LWLockRelease(TwoPhaseStateLock);
+			return true;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
 	/* Read and validate file */
 	buf = ReadTwoPhaseFile(xid, false);
 	if (buf == NULL)
@@ -1316,12 +1363,17 @@ StandbyTransactionIdIsPrepared(TransactionId xid)
 }
 
 /*
- * FinishPreparedTransaction: execute COMMIT PREPARED or ROLLBACK PREPARED
+ * FinishGXact
+ *
+ * Do the actual finish of COMMIT/ABORT PREPARED. It is a caller
+ * responsibility to properly lock corresponding gxact.
+ *
+ * This function can be called during replay to clean memory state
+ * for previously prepared xact. In that case actions are the same
+ * as in normal mode but without any writes to WAL or files.
  */
-void
-FinishPreparedTransaction(const char *gid, bool isCommit)
+static void FinishGXact(GlobalTransaction gxact, bool isCommit)
 {
-	GlobalTransaction gxact;
 	PGPROC	   *proc;
 	PGXACT	   *pgxact;
 	TransactionId xid;
@@ -1337,11 +1389,6 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	SharedInvalidationMessage *invalmsgs;
 	int			i;
 
-	/*
-	 * Validate the GID, and lock the GXACT to ensure that two backends do not
-	 * try to commit the same GID at once.
-	 */
-	gxact = LockGXact(gid, GetUserId());
 	proc = &ProcGlobal->allProcs[gxact->pgprocno];
 	pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 	xid = pgxact->xid;
@@ -1385,16 +1432,19 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	 * progress), then run the post-commit or post-abort callbacks. The
 	 * callbacks will release the locks the transaction held.
 	 */
-	if (isCommit)
-		RecordTransactionCommitPrepared(xid,
+	if (!RecoveryInProgress())
+	{
+		if (isCommit)
+			RecordTransactionCommitPrepared(xid,
 										hdr->nsubxacts, children,
 										hdr->ncommitrels, commitrels,
 										hdr->ninvalmsgs, invalmsgs,
 										hdr->initfileinval);
-	else
-		RecordTransactionAbortPrepared(xid,
+		else
+			RecordTransactionAbortPrepared(xid,
 									   hdr->nsubxacts, children,
 									   hdr->nabortrels, abortrels);
+	}
 
 	ProcArrayRemove(proc, latestXid);
 
@@ -1425,12 +1475,15 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 		delrels = abortrels;
 		ndelrels = hdr->nabortrels;
 	}
-	for (i = 0; i < ndelrels; i++)
+	if (!RecoveryInProgress())
 	{
-		SMgrRelation srel = smgropen(delrels[i], InvalidBackendId);
+		for (i = 0; i < ndelrels; i++)
+		{
+			SMgrRelation srel = smgropen(delrels[i], InvalidBackendId);
 
-		smgrdounlink(srel, false);
-		smgrclose(srel);
+			smgrdounlink(srel, false);
+			smgrclose(srel);
+		}
 	}
 
 	/*
@@ -1439,11 +1492,14 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	 * Relcache init file invalidation requires processing both before and
 	 * after we send the SI messages. See AtEOXact_Inval()
 	 */
-	if (hdr->initfileinval)
-		RelationCacheInitFilePreInvalidate();
-	SendSharedInvalidMessages(invalmsgs, hdr->ninvalmsgs);
-	if (hdr->initfileinval)
-		RelationCacheInitFilePostInvalidate();
+	if (!RecoveryInProgress())
+	{
+		if (hdr->initfileinval)
+			RelationCacheInitFilePreInvalidate();
+		SendSharedInvalidMessages(invalmsgs, hdr->ninvalmsgs);
+		if (hdr->initfileinval)
+			RelationCacheInitFilePostInvalidate();
+	}
 
 	/* And now do the callbacks */
 	if (isCommit)
@@ -1469,6 +1525,49 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 }
 
 /*
+ * FinishPreparedTransaction: execute COMMIT PREPARED or ROLLBACK PREPARED
+ */
+void
+FinishPreparedTransaction(const char *gid, bool isCommit)
+{
+	GlobalTransaction gxact;
+
+	/*
+	 * Validate the GID, and lock the GXACT to ensure that two backends do not
+	 * try to commit the same GID at once.
+	 */
+	gxact = LockGXact(gid, GetUserId());
+	FinishGXact(gxact, isCommit);
+}
+
+/*
+ * XlogRedoFinishPrepared()
+ *
+ * This function is called during replay when xlog reader faces 2pc commit or
+ * abort record. That function should clean up memory state that was created
+ * while replaying prepare xlog record.
+ */
+void
+XlogRedoFinishPrepared(TransactionId xid, bool isCommit)
+{
+	GlobalTransaction gxact;
+
+	Assert(RecoveryInProgress());
+
+	gxact = LockGXactByXid(xid);
+
+	/*
+	 * If requested xid wasn't found that means that prepare record was moved
+	 * to files before our replay started. That's okay and we have nothing to
+	 * clean/finish.
+	 */
+	if (!gxact)
+		return;
+
+	FinishGXact(gxact, isCommit);
+}
+
+/*
  * Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
  */
 static void
@@ -1690,7 +1789,48 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 	TransactionId *xids = NULL;
 	int			nxids = 0;
 	int			allocsize = 0;
+	int			i;
 
+	/*
+	 * We need to check the PGXACT array for prepared transactions that doesn't
+	 * have any state file in case of a slave restart with the master being off.
+	 */
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (!gxact->valid)
+			continue;
+
+		if (TransactionIdPrecedes(pgxact->xid, result))
+			result = pgxact->xid;
+
+		if (xids_p)
+		{
+			if (nxids == allocsize)
+			{
+				if (nxids == 0)
+				{
+					allocsize = 10;
+					xids = palloc(allocsize * sizeof(TransactionId));
+				}
+				else
+				{
+					allocsize = allocsize * 2;
+					xids = repalloc(xids, allocsize * sizeof(TransactionId));
+				}
+			}
+			xids[nxids++] = pgxact->xid;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+
+	/*
+	 * And now scan files in pg_twophase directory
+	 */
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
 	{
@@ -1701,7 +1841,6 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 			char	   *buf;
 			TwoPhaseFileHeader *hdr;
 			TransactionId *subxids;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
@@ -1809,102 +1948,105 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 }
 
 /*
- * StandbyRecoverPreparedTransactions
+ * RecoverPreparedFromBuffer
+ *
+ * Parse data in given buffer (that can be a pointer to WAL record or file)
+ * and load shared-memory state for that prepared transaction.
  *
- * Scan the pg_twophase directory and setup all the required information to
- * allow standby queries to treat prepared transactions as still active.
- * This is never called at the end of recovery - we use
- * RecoverPreparedTransactions() at that point.
+ * It's a caller responsibility to call MarkAsPrepared() on returned gxact.
  *
- * Currently we simply call SubTransSetParent() for any subxids of prepared
- * transactions. If overwriteOK is true, it's OK if some XIDs have already
- * been marked in pg_subtrans.
  */
-void
-StandbyRecoverPreparedTransactions(bool overwriteOK)
+static GlobalTransaction
+RecoverPreparedFromBuffer(char *buf, bool forceOverwriteOK)
 {
-	DIR		   *cldir;
-	struct dirent *clde;
+	char			*bufptr;
+	const char		*gid;
+	TransactionId	*subxids;
+	bool			overwriteOK = false;
+	int				i;
+	GlobalTransaction gxact;
+	TwoPhaseFileHeader	*hdr;
 
-	cldir = AllocateDir(TWOPHASE_DIR);
-	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
-	{
-		if (strlen(clde->d_name) == 8 &&
-			strspn(clde->d_name, "0123456789ABCDEF") == 8)
-		{
-			TransactionId xid;
-			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
+	/* Deconstruct header */
+	hdr = (TwoPhaseFileHeader *) buf;
+	bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+	gid = (const char *) bufptr;
+	bufptr += MAXALIGN(hdr->gidlen);
+	subxids = (TransactionId *) bufptr;
+	bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
+	bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
 
-			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
+	/*
+	 * It's possible that SubTransSetParent has been set before, if
+	 * the prepared transaction generated xid assignment records. Test
+	 * here must match one used in AssignTransactionId().
+	 */
+	if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
+						 XLogLogicalInfoActive()))
+		overwriteOK = true;
 
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+	/*
+	 * Caller can also force overwriteOK.
+	 */
+	if (forceOverwriteOK)
+		overwriteOK = true;
 
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+	/*
+	 * Reconstruct subtrans state for the transaction --- needed
+	 * because pg_subtrans is not preserved over a restart.  Note that
+	 * we are linking all the subtransactions directly to the
+	 * top-level XID; there may originally have been a more complex
+	 * hierarchy, but there's no need to restore that exactly.
+	 */
+	for (i = 0; i < hdr->nsubxacts; i++)
+		SubTransSetParent(subxids[i], hdr->xid, overwriteOK);
 
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				pfree(buf);
-				continue;
-			}
+	/*
+	 * Recreate its GXACT and dummy PGPROC
+	 */
+	gxact = MarkAsPreparing(hdr->xid, gid,
+							hdr->prepared_at,
+							hdr->owner, hdr->database);
+	GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
 
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID.
-			 */
-			subxids = (TransactionId *)
-				(buf + MAXALIGN(sizeof(TwoPhaseFileHeader)));
-			for (i = 0; i < hdr->nsubxacts; i++)
-			{
-				TransactionId subxid = subxids[i];
+	/*
+	 * Recover other state (notably locks) using resource managers
+	 */
+	ProcessRecords(bufptr, hdr->xid, twophase_recover_callbacks);
 
-				Assert(TransactionIdFollows(subxid, xid));
-				SubTransSetParent(xid, subxid, overwriteOK);
-			}
-		}
-	}
-	FreeDir(cldir);
+	/*
+	 * Release locks held by the standby process after we process each
+	 * prepared transaction. As a result, we don't need too many
+	 * additional locks at any one time.
+	 */
+	if (InHotStandby)
+		StandbyReleaseLockTree(hdr->xid, hdr->nsubxacts, subxids);
+
+	/*
+	 * We're done with recovering this transaction. Clear
+	 * MyLockedGxact, like we do in PrepareTransaction() during normal
+	 * operation.
+	 */
+	PostPrepare_Twophase();
+
+	return gxact;
 }
 
 /*
- * RecoverPreparedTransactions
+ * RecoverPreparedFromFiles
  *
  * Scan the pg_twophase directory and reload shared-memory state for each
  * prepared transaction (reacquire locks, etc).  This is run during database
  * startup.
  */
 void
-RecoverPreparedTransactions(void)
+RecoverPreparedFromFiles(bool forceOverwriteOK)
 {
 	char		dir[MAXPGPATH];
 	DIR		   *cldir;
 	struct dirent *clde;
-	bool		overwriteOK = false;
 
 	snprintf(dir, MAXPGPATH, "%s", TWOPHASE_DIR);
 
@@ -1916,15 +2058,27 @@ RecoverPreparedTransactions(void)
 		{
 			TransactionId xid;
 			char	   *buf;
-			char	   *bufptr;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
 			GlobalTransaction gxact;
-			const char	*gid;
 			int			i;
+			PGXACT	   *pgxact;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
+			/* Already recovered from WAL? */
+			LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+			for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+			{
+				gxact = TwoPhaseState->prepXacts[i];
+				pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+				if (TransactionIdEquals(xid, pgxact->xid))
+				{
+					LWLockRelease(TwoPhaseStateLock);
+					goto next_file;
+				}
+			}
+			LWLockRelease(TwoPhaseStateLock);
+
 			/* Already processed? */
 			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
 			{
@@ -1949,73 +2103,44 @@ RecoverPreparedTransactions(void)
 			ereport(LOG,
 					(errmsg("recovering prepared transaction %u", xid)));
 
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			Assert(TransactionIdEquals(hdr->xid, xid));
-			bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
-			gid = (const char *) bufptr;
-			bufptr += MAXALIGN(hdr->gidlen);
-			subxids = (TransactionId *) bufptr;
-			bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
-			bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
-
-			/*
-			 * It's possible that SubTransSetParent has been set before, if
-			 * the prepared transaction generated xid assignment records. Test
-			 * here must match one used in AssignTransactionId().
-			 */
-			if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
-								 XLogLogicalInfoActive()))
-				overwriteOK = true;
-
-			/*
-			 * Reconstruct subtrans state for the transaction --- needed
-			 * because pg_subtrans is not preserved over a restart.  Note that
-			 * we are linking all the subtransactions directly to the
-			 * top-level XID; there may originally have been a more complex
-			 * hierarchy, but there's no need to restore that exactly.
-			 */
-			for (i = 0; i < hdr->nsubxacts; i++)
-				SubTransSetParent(subxids[i], xid, overwriteOK);
-
-			/*
-			 * Recreate its GXACT and dummy PGPROC
-			 */
-			gxact = MarkAsPreparing(xid, gid,
-									hdr->prepared_at,
-									hdr->owner, hdr->database);
+			gxact = RecoverPreparedFromBuffer(buf, forceOverwriteOK);
 			gxact->ondisk = true;
-			GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
 			MarkAsPrepared(gxact);
 
-			/*
-			 * Recover other state (notably locks) using resource managers
-			 */
-			ProcessRecords(bufptr, xid, twophase_recover_callbacks);
-
-			/*
-			 * Release locks held by the standby process after we process each
-			 * prepared transaction. As a result, we don't need too many
-			 * additional locks at any one time.
-			 */
-			if (InHotStandby)
-				StandbyReleaseLockTree(xid, hdr->nsubxacts, subxids);
-
-			/*
-			 * We're done with recovering this transaction. Clear
-			 * MyLockedGxact, like we do in PrepareTransaction() during normal
-			 * operation.
-			 */
-			PostPrepare_Twophase();
-
 			pfree(buf);
 		}
+
+next_file:
+		continue;
+
 	}
 	FreeDir(cldir);
 }
 
+
+/*
+ * RecoverPreparedFromXLOG
+ *
+ * To avoid creation of state files during replay we registering
+ * prepare xlog records in shared memory in the same way as it happens
+ * while not in recovery. If replay faces commit xlog record before
+ * checkpoint/restartpoint happens then we avoid using files at all.
+ *
+ * We need this behaviour because the speed of the 2PC replay on the replica
+ * should be at least the same as the 2PC transaction speed of the master.
+ */
+void
+RecoverPreparedFromXLOG(XLogReaderState *record)
+{
+	GlobalTransaction gxact;
+
+	gxact = RecoverPreparedFromBuffer((char *) XLogRecGetData(record), false);
+	gxact->prepare_start_lsn = record->ReadRecPtr;
+	gxact->prepare_end_lsn = record->EndRecPtr;
+	MarkAsPrepared(gxact);
+}
+
+
 /*
  *	RecordTransactionCommitPrepared
  *
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 7e37331..9323ba2 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -5588,7 +5588,7 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+			XlogRedoFinishPrepared(parsed.twophase_xid, true);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5608,14 +5608,12 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+			XlogRedoFinishPrepared(parsed.twophase_xid, false);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		RecoverPreparedFromXLOG(record);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index f9644db..11c06b4 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6630,7 +6630,7 @@ StartupXLOG(void)
 
 				ProcArrayApplyRecoveryInfo(&running);
 
-				StandbyRecoverPreparedTransactions(false);
+				RecoverPreparedFromFiles(false);
 			}
 		}
 
@@ -7383,7 +7383,7 @@ StartupXLOG(void)
 	TrimMultiXact();
 
 	/* Reload shared-memory state for prepared transactions */
-	RecoverPreparedTransactions();
+	RecoverPreparedFromFiles(false);
 
 	/*
 	 * Shutdown the recovery environment. This must occur after
@@ -9297,7 +9297,7 @@ xlog_redo(XLogReaderState *record)
 
 			ProcArrayApplyRecoveryInfo(&running);
 
-			StandbyRecoverPreparedTransactions(true);
+			RecoverPreparedFromFiles(true);
 		}
 
 		/* ControlFile->checkPointCopy always tracks the latest ckpt XID */
diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c
index 6a9bf84..ce97dbc 100644
--- a/src/backend/storage/ipc/standby.c
+++ b/src/backend/storage/ipc/standby.c
@@ -607,7 +607,8 @@ StandbyAcquireAccessExclusiveLock(TransactionId xid, Oid dbOid, Oid relOid)
 	/* Already processed? */
 	if (!TransactionIdIsValid(xid) ||
 		TransactionIdDidCommit(xid) ||
-		TransactionIdDidAbort(xid))
+		TransactionIdDidAbort(xid) ||
+		StandbyTransactionIdIsPrepared(xid))
 		return;
 
 	elog(trace_recovery(DEBUG4),
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b7ce0c6..416ef5e 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -17,6 +17,7 @@
 #include "access/xlogdefs.h"
 #include "datatype/timestamp.h"
 #include "storage/lock.h"
+#include "access/xlogreader.h"
 
 /*
  * GlobalTransactionData is defined in twophase.c; other places have no
@@ -46,8 +47,8 @@ extern bool StandbyTransactionIdIsPrepared(TransactionId xid);
 
 extern TransactionId PrescanPreparedTransactions(TransactionId **xids_p,
 							int *nxids_p);
-extern void StandbyRecoverPreparedTransactions(bool overwriteOK);
-extern void RecoverPreparedTransactions(void);
+extern void RecoverPreparedFromFiles(bool overwriteOK);
+extern void RecoverPreparedFromXLOG(XLogReaderState *record);
 
 extern void RecreateTwoPhaseFile(TransactionId xid, void *content, int len);
 extern void RemoveTwoPhaseFile(TransactionId xid, bool giveWarning);
@@ -56,4 +57,5 @@ extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void XlogRedoFinishPrepared(TransactionId xid, bool isCommit);
 #endif   /* TWOPHASE_H */
diff --git a/src/test/recovery/t/008_twophase.pl b/src/test/recovery/t/008_twophase.pl
new file mode 100644
index 0000000..3d563cf
--- /dev/null
+++ b/src/test/recovery/t/008_twophase.pl
@@ -0,0 +1,243 @@
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 12;
+
+# Setup master node
+my $node_master = get_new_node("master");
+$node_master->init(allows_streaming => 1);
+$node_master->append_conf('postgresql.conf', qq(
+	max_prepared_transactions = 10
+));
+$node_master->start;
+$node_master->backup('master_backup');
+$node_master->psql('postgres', "create table t(id int)");
+
+# Setup master node
+my $node_slave = get_new_node('slave');
+$node_slave->init_from_backup($node_master, 'master_backup', has_streaming => 1);
+$node_slave->start;
+
+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+	synchronous_standby_names = '*'
+));
+$node_master->psql('postgres', "select pg_reload_conf()");
+
+my $psql_out = '';
+my $psql_rc = '';
+
+###############################################################################
+# Check that we can commit and abort tx after soft restart.
+# Here checkpoint happens before shutdown and no WAL replay will not occur
+# during start. So postgres should re-create memory state from files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	prepare transaction 'y';");
+$node_master->stop;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared tx after restart.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared tx after restart.');
+
+###############################################################################
+# Check that we can commit and abort after hard restart.
+# On startup WAL replay will re-create memory for global transactions that
+# happend after the last checkpoint.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	prepare transaction 'y';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared tx after teardown.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared tx after teardown.');
+
+###############################################################################
+# Check that we can replay several tx with same name.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Replay several tx with same name.');
+
+###############################################################################
+# Check that WAL replay will cleanup it's memory state and release locks while
+# replaying commit.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_master->teardown_node;
+$node_master->start;
+$psql_rc = $node_master->psql('postgres',"
+	begin;
+	insert into t values (42);
+	-- This prepare can fail due to 2pc identifier or locks conflicts if replay
+	-- didn't fully cleanup it's state on commit.
+	prepare transaction 'x';");
+is($psql_rc, '0', "Check that replay will cleanup it's memory state");
+
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that WAL replay will cleanup it's memory state on running slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';
+	");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;", stdout => \$psql_out);
+is($psql_out, '0', "Check that replay will cleanup it's memory state on running slave");
+
+###############################################################################
+# The same as in previous case, but let's force checkpoint on slave between
+# prepare and commit.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	");
+$node_slave->psql('postgres',"checkpoint;");
+$node_master->psql('postgres', "commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;", stdout => \$psql_out);
+is($psql_out, '0', "Check that replay will cleanup it's memory state on slave after checkpoint");
+
+###############################################################################
+# Check that we can commit transaction on promoted slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	");
+$node_master->teardown_node;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$psql_rc = $node_slave->psql('postgres', "commit prepared 'x';");
+is($psql_rc, '0', "Restore prepared transaction on promoted slave.");
+
+# change roles
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+
+###############################################################################
+# Check that we restore prepared xacts after slave soft restart while master is
+# down. Since slave knows that master is down it uses different code path on
+# start.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->restart;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
+is($psql_out, '1', "Restore prepared xacts after slave soft restart while master is down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres',"commit prepared 'x'");
+
+###############################################################################
+# Check that we restore prepared xacts after slave hard restart while master is
+# down.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (242);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->teardown_node;
+$node_slave->start;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
+is($psql_out, '1', "Restore prepared xacts after slave hard restart while master is down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres',"commit prepared 'x'");
+
+
+###############################################################################
+# Check for a lock confcict between prepared tx with DDL inside and replay of
+# XLOG_STANDBY_LOCK wal record.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	create table t2(id int);
+	prepare transaction 'x';
+	-- checkpoint will issue XLOG_STANDBY_LOCK that can conflict with lock
+	-- held by 'create table' statement
+	checkpoint;
+	commit prepared 'x';
+");
+
+$node_slave->psql('postgres',"select count(*) from pg_prepared_xacts", stdout => \$psql_out);
+is($psql_out, '0', "Replay tx with DDL");
+
#77Jesper Pedersen
jesper.pedersen@redhat.com
In reply to: Stas Kelvich (#76)
Re: Speedup twophase transactions

Hi,

On 04/13/2016 10:31 AM, Stas Kelvich wrote:

On 13 Apr 2016, at 01:04, Michael Paquier <michael.paquier@gmail.com> wrote:
That's too late for 9.6 unfortunately, don't forget to add that in the next CF!

Fixed patch attached. There already was infrastructure to skip currently
held locks during replay of standby_redo() and I�ve extended that with check for
prepared xids.

The reason why I�m still active on this thread is because I see real problems
in deploying 9.6 in current state. Let me stress my concern: current state of things
_WILL_BREAK_ async replication in case of substantial load of two phase
transactions on master. And a lot of J2EE apps falls into that category, as they
wrap most of their transactions in prepare/commit. Slave server just will always
increase it lag and will never catch up. It is possible to deal with that by switching
to synchronous replication or inserting triggers with pg_sleep on master, but it
doesn�t looks like normal behaviour of system.

Discussed with Noah off-list I think we should revisit this for 9.6 due
to the async replica lag as shown in [1]/messages/by-id/E7497864-DE11-4099-83F5-89FB97AF5073@postgrespro.ru. The performance improvement
for the master node is shown in [2]/messages/by-id/5693F703.3000009@redhat.com.

As I see it there are 3 options to resolve this (in one way or another)

* Leave as is, document the behaviour in release notes/documentation
* Apply the patch for slaves
* Revert the changes done to the twophase.c during the 9.6 cycle.

All have pros/cons for the release.

Latest slave patch by Stas is on [3]https://commitfest.postgresql.org/10/523/.

Thoughts from others on the matter would be appreciated.

[1]: /messages/by-id/E7497864-DE11-4099-83F5-89FB97AF5073@postgrespro.ru
/messages/by-id/E7497864-DE11-4099-83F5-89FB97AF5073@postgrespro.ru
[2]: /messages/by-id/5693F703.3000009@redhat.com
[3]: https://commitfest.postgresql.org/10/523/

Best regards,
Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#78Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Jesper Pedersen (#77)
Re: Speedup twophase transactions

Jesper Pedersen wrote:

Discussed with Noah off-list I think we should revisit this for 9.6 due to
the async replica lag as shown in [1]. The performance improvement for the
master node is shown in [2].

I gave a very quick look and it seems to me far more invasive than we
would normally consider in the beta period. I would just put it to
sleep till release 10 opens up.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#79Michael Paquier
michael.paquier@gmail.com
In reply to: Alvaro Herrera (#78)
Re: Speedup twophase transactions

On Fri, May 20, 2016 at 12:46 PM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:

Jesper Pedersen wrote:

Discussed with Noah off-list I think we should revisit this for 9.6 due to
the async replica lag as shown in [1]. The performance improvement for the
master node is shown in [2].

I gave a very quick look and it seems to me far more invasive than we
would normally consider in the beta period. I would just put it to
sleep till release 10 opens up.

I share the same opinion. Improving 2PC is definitely a huge win
thanks to the first patch that got committed when WAL is generated,
but considering how the second patch is invasive really concerns me,
and I looked at the patch in an extended way a couple of weeks back.
As we care about stability now regarding 9.6, let's bump the second
portion to 10.0 as well as keep the improvement for WAL generation in
9.6.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#80Simon Riggs
simon@2ndquadrant.com
In reply to: Stas Kelvich (#76)
Re: Speedup twophase transactions

On 13 April 2016 at 15:31, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Fixed patch attached. There already was infrastructure to skip currently
held locks during replay of standby_redo() and I’ve extended that with check for
prepared xids.

Please confirm that everything still works on current HEAD for the new
CF, so review can start.

Thanks

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#81Michael Paquier
michael.paquier@gmail.com
In reply to: Simon Riggs (#80)
Re: Speedup twophase transactions

On Fri, Sep 2, 2016 at 5:06 AM, Simon Riggs <simon@2ndquadrant.com> wrote:

On 13 April 2016 at 15:31, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Fixed patch attached. There already was infrastructure to skip currently
held locks during replay of standby_redo() and I’ve extended that with check for
prepared xids.

Please confirm that everything still works on current HEAD for the new
CF, so review can start.

The patch does not apply cleanly. Stas, could you rebase? I am
switching the patch to "waiting on author" for now.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#82Michael Paquier
michael.paquier@gmail.com
In reply to: Michael Paquier (#81)
1 attachment(s)
Re: Speedup twophase transactions

On Sat, Sep 3, 2016 at 10:26 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:

On Fri, Sep 2, 2016 at 5:06 AM, Simon Riggs <simon@2ndquadrant.com> wrote:

On 13 April 2016 at 15:31, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Fixed patch attached. There already was infrastructure to skip currently
held locks during replay of standby_redo() and I’ve extended that with check for
prepared xids.

Please confirm that everything still works on current HEAD for the new
CF, so review can start.

The patch does not apply cleanly. Stas, could you rebase? I am
switching the patch to "waiting on author" for now.

So, I have just done the rebase myself and did an extra round of
reviews of the patch. Here are couple of comments after this extra
lookup.

LockGXactByXid() is aimed to be used only in recovery, so it seems
adapted to be to add an assertion with RecoveryInProgress(). Using
this routine in non-recovery code paths is risky because it assumes
that a PGXACT could be missing, which is fine in recovery as prepared
transactions could be moved to twophase files because of a checkpoint,
but not in normal cases. We could also add an assertion of the kind
gxact->locking_backend == InvalidBackendId before locking the PGXACT
but as this routine is just normally used by the startup process (in
non-standalone mode!) that's fine without.

The handling of invalidation messages and removal of relfilenodes done
in FinishGXact can be grouped together, checking only once for
!RecoveryInProgress().

+ *
+ *     The same procedure happens during replication and crash recovery.
  *
"during WAL replay" is more generic and applies here.
+
+next_file:
+       continue;
+
That can be removed and replaced by a "continue;".
+   /*
+    * At first check prepared tx that wasn't yet moved to disk.
+    */
+   LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+   for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+   {
+       GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+       PGXACT *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+       if (TransactionIdEquals(pgxact->xid, xid))
+       {
+           LWLockRelease(TwoPhaseStateLock);
+           return true;
+       }
+   }
+   LWLockRelease(TwoPhaseStateLock);
This overlaps with TwoPhaseGetGXact but I'd rather keep both things
separated: it does not seem worth complicating the existing interface,
and putting that in cache during recovery has no meaning.

I have also reworked the test format, and fixed many typos and grammar
problems in the patch as well as in the tests.

After review the result is attached. Perhaps a committer could get a look at it?
--
Michael

Attachments:

twophase_replay.v6.patchtext/x-diff; charset=US-ASCII; name=twophase_replay.v6.patchDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 9f55adc..eb7c339 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,8 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *
+ *		The same procedure happens during WAL replay.
  *
  *-------------------------------------------------------------------------
  */
@@ -578,6 +578,45 @@ LockGXact(const char *gid, Oid user)
 }
 
 /*
+ * LockGXactByXid
+ *
+ * Find prepared transaction by xid and lock corresponding GXACT.
+ * This is used during recovery as an alternative to LockGXact(), and
+ * should only be used in recovery. No entries found means that a checkpoint
+ * has moved the searched prepared transaction data to a twophase file.
+ *
+ * Returns the transaction data if found, or NULL if nothing has been locked.
+ */
+static GlobalTransaction
+LockGXactByXid(TransactionId xid)
+{
+	int		i;
+	GlobalTransaction gxact = NULL;
+
+	Assert(RecoveryInProgress());
+
+	LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		PGXACT	   *pgxact;
+
+		gxact = TwoPhaseState->prepXacts[i];
+		pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (TransactionIdEquals(xid, pgxact->xid))
+		{
+			/* ok to lock it */
+			gxact->locking_backend = MyBackendId;
+			MyLockedGxact = gxact;
+			break;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+	return gxact;
+}
+
+/*
  * RemoveGXact
  *		Remove the prepared transaction from the shared memory array.
  *
@@ -1241,9 +1280,9 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
  * Reads 2PC data from xlog. During checkpoint this data will be moved to
  * twophase files and ReadTwoPhaseFile should be used instead.
  *
- * Note clearly that this function accesses WAL during normal operation, similarly
- * to the way WALSender or Logical Decoding would do. It does not run during
- * crash recovery or standby processing.
+ * Note that this function accesses WAL not only during recovery but also
+ * during normal operation, similarly to the way WALSender or Logical
+ * Decoding would do.
  */
 static void
 XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
@@ -1252,8 +1291,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1296,13 +1333,35 @@ StandbyTransactionIdIsPrepared(TransactionId xid)
 	char	   *buf;
 	TwoPhaseFileHeader *hdr;
 	bool		result;
+	int			i;
 
 	Assert(TransactionIdIsValid(xid));
 
 	if (max_prepared_xacts <= 0)
 		return false;			/* nothing to do */
 
-	/* Read and validate file */
+	/*
+	 * First check if this prepared transaction has its information in
+	 * shared memory, and use it.
+	 */
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		PGXACT *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (TransactionIdEquals(pgxact->xid, xid))
+		{
+			LWLockRelease(TwoPhaseStateLock);
+			return true;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+	/*
+	 * Nothing in shared memory? Then just read its corresponding twophase
+	 * file and validate it.
+	 */
 	buf = ReadTwoPhaseFile(xid, false);
 	if (buf == NULL)
 		return false;
@@ -1316,12 +1375,17 @@ StandbyTransactionIdIsPrepared(TransactionId xid)
 }
 
 /*
- * FinishPreparedTransaction: execute COMMIT PREPARED or ROLLBACK PREPARED
+ * FinishGXact
+ *
+ * Do the actual finish of COMMIT/ABORT PREPARED. Calls is responsible
+ * for locking the transaction this routine is working on.
+ *
+ * This function can be called during replay to clean memory state for
+ * previously prepared xact. In that case actions are the same as in
+ * normal operations but without any writes to WAL or files.
  */
-void
-FinishPreparedTransaction(const char *gid, bool isCommit)
+static void FinishGXact(GlobalTransaction gxact, bool isCommit)
 {
-	GlobalTransaction gxact;
 	PGPROC	   *proc;
 	PGXACT	   *pgxact;
 	TransactionId xid;
@@ -1332,16 +1396,9 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	TransactionId *children;
 	RelFileNode *commitrels;
 	RelFileNode *abortrels;
-	RelFileNode *delrels;
-	int			ndelrels;
 	SharedInvalidationMessage *invalmsgs;
 	int			i;
 
-	/*
-	 * Validate the GID, and lock the GXACT to ensure that two backends do not
-	 * try to commit the same GID at once.
-	 */
-	gxact = LockGXact(gid, GetUserId());
 	proc = &ProcGlobal->allProcs[gxact->pgprocno];
 	pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 	xid = pgxact->xid;
@@ -1383,17 +1440,23 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	 * TransactionIdIsInProgress will stop saying the prepared xact is in
 	 * progress), then run the post-commit or post-abort callbacks. The
 	 * callbacks will release the locks the transaction held.
+	 *
+	 * In recovery nothing needs to happen here as this generates WAL
+	 * records.
 	 */
-	if (isCommit)
-		RecordTransactionCommitPrepared(xid,
+	if (!RecoveryInProgress())
+	{
+		if (isCommit)
+			RecordTransactionCommitPrepared(xid,
 										hdr->nsubxacts, children,
 										hdr->ncommitrels, commitrels,
 										hdr->ninvalmsgs, invalmsgs,
 										hdr->initfileinval);
-	else
-		RecordTransactionAbortPrepared(xid,
+		else
+			RecordTransactionAbortPrepared(xid,
 									   hdr->nsubxacts, children,
 									   hdr->nabortrels, abortrels);
+	}
 
 	ProcArrayRemove(proc, latestXid);
 
@@ -1408,41 +1471,50 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	gxact->valid = false;
 
 	/*
-	 * We have to remove any files that were supposed to be dropped. For
-	 * consistency with the regular xact.c code paths, must do this before
-	 * releasing locks, so do it before running the callbacks.
-	 *
-	 * NB: this code knows that we couldn't be dropping any temp rels ...
+	 * Perform actions needed only during normal operation, but *not* recovery.
 	 */
-	if (isCommit)
-	{
-		delrels = commitrels;
-		ndelrels = hdr->ncommitrels;
-	}
-	else
+	if (!RecoveryInProgress())
 	{
-		delrels = abortrels;
-		ndelrels = hdr->nabortrels;
-	}
-	for (i = 0; i < ndelrels; i++)
-	{
-		SMgrRelation srel = smgropen(delrels[i], InvalidBackendId);
+		RelFileNode *delrels;
+		int			ndelrels;
 
-		smgrdounlink(srel, false);
-		smgrclose(srel);
-	}
+		/*
+		 * We have to remove any files that were supposed to be dropped. For
+		 * consistency with the regular xact.c code paths, must do this before
+		 * releasing locks, so do it before running the callbacks.
+		 *
+		 * NB: this code knows that we couldn't be dropping any temp rels ...
+		 */
+		if (isCommit)
+		{
+			delrels = commitrels;
+			ndelrels = hdr->ncommitrels;
+		}
+		else
+		{
+			delrels = abortrels;
+			ndelrels = hdr->nabortrels;
+		}
+		for (i = 0; i < ndelrels; i++)
+		{
+			SMgrRelation srel = smgropen(delrels[i], InvalidBackendId);
 
-	/*
-	 * Handle cache invalidation messages.
-	 *
-	 * Relcache init file invalidation requires processing both before and
-	 * after we send the SI messages. See AtEOXact_Inval()
-	 */
-	if (hdr->initfileinval)
-		RelationCacheInitFilePreInvalidate();
-	SendSharedInvalidMessages(invalmsgs, hdr->ninvalmsgs);
-	if (hdr->initfileinval)
-		RelationCacheInitFilePostInvalidate();
+			smgrdounlink(srel, false);
+			smgrclose(srel);
+		}
+
+		/*
+		 * Handle cache invalidation messages.
+		 *
+		 * Relcache init file invalidation requires processing both before and
+		 * after we send the SI messages. See AtEOXact_Inval()
+		 */
+		if (hdr->initfileinval)
+			RelationCacheInitFilePreInvalidate();
+		SendSharedInvalidMessages(invalmsgs, hdr->ninvalmsgs);
+		if (hdr->initfileinval)
+			RelationCacheInitFilePostInvalidate();
+	}
 
 	/* And now do the callbacks */
 	if (isCommit)
@@ -1468,6 +1540,50 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 }
 
 /*
+ * FinishPreparedTransaction: execute COMMIT PREPARED or ROLLBACK PREPARED
+ */
+void
+FinishPreparedTransaction(const char *gid, bool isCommit)
+{
+	GlobalTransaction gxact;
+
+	/*
+	 * Validate the GID, and lock the GXACT to ensure that two backends do not
+	 * try to commit the same GID at once.
+	 */
+	gxact = LockGXact(gid, GetUserId());
+	FinishGXact(gxact, isCommit);
+}
+
+/*
+ * XlogRedoFinishPrepared()
+ *
+ * This function is called during recovery for WAL records working on COMMIT
+ * PREPARED or ABORT PREPARED. That function cleans up memory state that was
+ * created while replaying its corresponding PREPARE record if its information
+ * was not on disk in a twophase file.
+ */
+void
+XlogRedoFinishPrepared(TransactionId xid, bool isCommit)
+{
+	GlobalTransaction gxact;
+
+	Assert(RecoveryInProgress());
+
+	gxact = LockGXactByXid(xid);
+
+	/*
+	 * If requested xid was not found that means that the PREPARE record was
+	 * moved to a twophase file because of a checkpoint or a restart point.
+	 * There is nothing else to do in this case.
+	 */
+	if (!gxact)
+		return;
+
+	FinishGXact(gxact, isCommit);
+}
+
+/*
  * Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
  */
 static void
@@ -1690,7 +1806,47 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 	TransactionId *xids = NULL;
 	int			nxids = 0;
 	int			allocsize = 0;
+	int			i;
+
+	/*
+	 * We need to check the PGXACT array for prepared transactions that doesn't
+	 * have any state file in case of a slave restart with the master being off.
+	 */
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (!gxact->valid)
+			continue;
+
+		if (TransactionIdPrecedes(pgxact->xid, result))
+			result = pgxact->xid;
+
+		if (xids_p)
+		{
+			if (nxids == allocsize)
+			{
+				if (nxids == 0)
+				{
+					allocsize = 10;
+					xids = palloc(allocsize * sizeof(TransactionId));
+				}
+				else
+				{
+					allocsize = allocsize * 2;
+					xids = repalloc(xids, allocsize * sizeof(TransactionId));
+				}
+			}
+			xids[nxids++] = pgxact->xid;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
 
+	/*
+	 * And now scan files in pg_twophase directory
+	 */
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
 	{
@@ -1701,7 +1857,6 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 			char	   *buf;
 			TwoPhaseFileHeader *hdr;
 			TransactionId *subxids;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
@@ -1809,102 +1964,105 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 }
 
 /*
- * StandbyRecoverPreparedTransactions
+ * RecoverPreparedFromBuffer
  *
- * Scan the pg_twophase directory and setup all the required information to
- * allow standby queries to treat prepared transactions as still active.
- * This is never called at the end of recovery - we use
- * RecoverPreparedTransactions() at that point.
+ * Parse data in given buffer (that can be a pointer to WAL record holding
+ * this information or data read from a twophase file) and build the
+ * shared-memory state for that prepared transaction.
  *
- * Currently we simply call SubTransSetParent() for any subxids of prepared
- * transactions. If overwriteOK is true, it's OK if some XIDs have already
- * been marked in pg_subtrans.
+ * Caller is responsible for calling MarkAsPrepared() on the returned gxact.
  */
-void
-StandbyRecoverPreparedTransactions(bool overwriteOK)
+static GlobalTransaction
+RecoverPreparedFromBuffer(char *buf, bool forceOverwriteOK)
 {
-	DIR		   *cldir;
-	struct dirent *clde;
+	char			*bufptr;
+	const char		*gid;
+	TransactionId	*subxids;
+	bool			overwriteOK = false;
+	int				i;
+	GlobalTransaction gxact;
+	TwoPhaseFileHeader	*hdr;
 
-	cldir = AllocateDir(TWOPHASE_DIR);
-	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
-	{
-		if (strlen(clde->d_name) == 8 &&
-			strspn(clde->d_name, "0123456789ABCDEF") == 8)
-		{
-			TransactionId xid;
-			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
+	/* Deconstruct header */
+	hdr = (TwoPhaseFileHeader *) buf;
+	bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+	gid = (const char *) bufptr;
+	bufptr += MAXALIGN(hdr->gidlen);
+	subxids = (TransactionId *) bufptr;
+	bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
+	bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
 
-			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
+	/*
+	 * It's possible that SubTransSetParent has been set before, if
+	 * the prepared transaction generated xid assignment records. Test
+	 * here must match one used in AssignTransactionId().
+	 */
+	if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
+						 XLogLogicalInfoActive()))
+		overwriteOK = true;
 
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+	/*
+	 * Caller can also force overwriteOK.
+	 */
+	if (forceOverwriteOK)
+		overwriteOK = true;
 
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+	/*
+	 * Reconstruct subtrans state for the transaction --- needed
+	 * because pg_subtrans is not preserved over a restart.  Note that
+	 * we are linking all the subtransactions directly to the
+	 * top-level XID; there may originally have been a more complex
+	 * hierarchy, but there's no need to restore that exactly.
+	 */
+	for (i = 0; i < hdr->nsubxacts; i++)
+		SubTransSetParent(subxids[i], hdr->xid, overwriteOK);
 
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				pfree(buf);
-				continue;
-			}
+	/*
+	 * Recreate its GXACT and dummy PGPROC
+	 */
+	gxact = MarkAsPreparing(hdr->xid, gid,
+							hdr->prepared_at,
+							hdr->owner, hdr->database);
+	GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
 
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID.
-			 */
-			subxids = (TransactionId *)
-				(buf + MAXALIGN(sizeof(TwoPhaseFileHeader)));
-			for (i = 0; i < hdr->nsubxacts; i++)
-			{
-				TransactionId subxid = subxids[i];
+	/*
+	 * Recover other state (notably locks) using resource managers
+	 */
+	ProcessRecords(bufptr, hdr->xid, twophase_recover_callbacks);
 
-				Assert(TransactionIdFollows(subxid, xid));
-				SubTransSetParent(xid, subxid, overwriteOK);
-			}
-		}
-	}
-	FreeDir(cldir);
+	/*
+	 * Release locks held by the standby process after we process each
+	 * prepared transaction. As a result, we don't need too many
+	 * additional locks at any one time.
+	 */
+	if (InHotStandby)
+		StandbyReleaseLockTree(hdr->xid, hdr->nsubxacts, subxids);
+
+	/*
+	 * We're done with recovering this transaction. Clear
+	 * MyLockedGxact, like we do in PrepareTransaction() during normal
+	 * operation.
+	 */
+	PostPrepare_Twophase();
+
+	return gxact;
 }
 
 /*
- * RecoverPreparedTransactions
+ * RecoverPreparedFromFiles
  *
  * Scan the pg_twophase directory and reload shared-memory state for each
  * prepared transaction (reacquire locks, etc).  This is run during database
  * startup.
  */
 void
-RecoverPreparedTransactions(void)
+RecoverPreparedFromFiles(bool forceOverwriteOK)
 {
 	char		dir[MAXPGPATH];
 	DIR		   *cldir;
 	struct dirent *clde;
-	bool		overwriteOK = false;
 
 	snprintf(dir, MAXPGPATH, "%s", TWOPHASE_DIR);
 
@@ -1916,15 +2074,30 @@ RecoverPreparedTransactions(void)
 		{
 			TransactionId xid;
 			char	   *buf;
-			char	   *bufptr;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
 			GlobalTransaction gxact;
-			const char *gid;
 			int			i;
+			bool		recovered = false;
+			PGXACT	   *pgxact;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
+			/* Already recovered from WAL? */
+			LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+			for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+			{
+				gxact = TwoPhaseState->prepXacts[i];
+				pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+				if (TransactionIdEquals(xid, pgxact->xid))
+				{
+					recovered = true;
+					break;
+				}
+			}
+			LWLockRelease(TwoPhaseStateLock);
+			if (recovered)
+				continue;
+
 			/* Already processed? */
 			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
 			{
@@ -1949,73 +2122,39 @@ RecoverPreparedTransactions(void)
 			ereport(LOG,
 					(errmsg("recovering prepared transaction %u", xid)));
 
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			Assert(TransactionIdEquals(hdr->xid, xid));
-			bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
-			gid = (const char *) bufptr;
-			bufptr += MAXALIGN(hdr->gidlen);
-			subxids = (TransactionId *) bufptr;
-			bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
-			bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
-
-			/*
-			 * It's possible that SubTransSetParent has been set before, if
-			 * the prepared transaction generated xid assignment records. Test
-			 * here must match one used in AssignTransactionId().
-			 */
-			if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
-								 XLogLogicalInfoActive()))
-				overwriteOK = true;
-
-			/*
-			 * Reconstruct subtrans state for the transaction --- needed
-			 * because pg_subtrans is not preserved over a restart.  Note that
-			 * we are linking all the subtransactions directly to the
-			 * top-level XID; there may originally have been a more complex
-			 * hierarchy, but there's no need to restore that exactly.
-			 */
-			for (i = 0; i < hdr->nsubxacts; i++)
-				SubTransSetParent(subxids[i], xid, overwriteOK);
-
-			/*
-			 * Recreate its GXACT and dummy PGPROC
-			 */
-			gxact = MarkAsPreparing(xid, gid,
-									hdr->prepared_at,
-									hdr->owner, hdr->database);
+			gxact = RecoverPreparedFromBuffer(buf, forceOverwriteOK);
 			gxact->ondisk = true;
-			GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
 			MarkAsPrepared(gxact);
 
-			/*
-			 * Recover other state (notably locks) using resource managers
-			 */
-			ProcessRecords(bufptr, xid, twophase_recover_callbacks);
-
-			/*
-			 * Release locks held by the standby process after we process each
-			 * prepared transaction. As a result, we don't need too many
-			 * additional locks at any one time.
-			 */
-			if (InHotStandby)
-				StandbyReleaseLockTree(xid, hdr->nsubxacts, subxids);
-
-			/*
-			 * We're done with recovering this transaction. Clear
-			 * MyLockedGxact, like we do in PrepareTransaction() during normal
-			 * operation.
-			 */
-			PostPrepare_Twophase();
-
 			pfree(buf);
 		}
 	}
 	FreeDir(cldir);
 }
 
+
+/*
+ * RecoverPreparedFromXLOG
+ *
+ * To avoid the creation of twophase state files during replay we register
+ * WAL records for prepared transactions in shared memory in the same way
+ * during normal operations. If replay faces a WAL record for a COMMIT
+ * PREPARED transaction before a checkpoint or restartpoint happens then
+ * no files are used, limiting the I/O impact of such operations during
+ * recovery.
+ */
+void
+RecoverPreparedFromXLOG(XLogReaderState *record)
+{
+	GlobalTransaction gxact;
+
+	gxact = RecoverPreparedFromBuffer((char *) XLogRecGetData(record), false);
+	gxact->prepare_start_lsn = record->ReadRecPtr;
+	gxact->prepare_end_lsn = record->EndRecPtr;
+	MarkAsPrepared(gxact);
+}
+
+
 /*
  *	RecordTransactionCommitPrepared
  *
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index e11b229..6a40425 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -5602,7 +5602,7 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+			XlogRedoFinishPrepared(parsed.twophase_xid, true);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5622,14 +5622,12 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+			XlogRedoFinishPrepared(parsed.twophase_xid, false);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		RecoverPreparedFromXLOG(record);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 2189c22..613097f 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6704,7 +6704,7 @@ StartupXLOG(void)
 
 				ProcArrayApplyRecoveryInfo(&running);
 
-				StandbyRecoverPreparedTransactions(false);
+				RecoverPreparedFromFiles(false);
 			}
 		}
 
@@ -7463,7 +7463,7 @@ StartupXLOG(void)
 	TrimMultiXact();
 
 	/* Reload shared-memory state for prepared transactions */
-	RecoverPreparedTransactions();
+	RecoverPreparedFromFiles(false);
 
 	/*
 	 * Shutdown the recovery environment. This must occur after
@@ -9377,7 +9377,7 @@ xlog_redo(XLogReaderState *record)
 
 			ProcArrayApplyRecoveryInfo(&running);
 
-			StandbyRecoverPreparedTransactions(true);
+			RecoverPreparedFromFiles(true);
 		}
 
 		/* ControlFile->checkPointCopy always tracks the latest ckpt XID */
diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c
index 547f1a8..5ea2530 100644
--- a/src/backend/storage/ipc/standby.c
+++ b/src/backend/storage/ipc/standby.c
@@ -608,7 +608,8 @@ StandbyAcquireAccessExclusiveLock(TransactionId xid, Oid dbOid, Oid relOid)
 	/* Already processed? */
 	if (!TransactionIdIsValid(xid) ||
 		TransactionIdDidCommit(xid) ||
-		TransactionIdDidAbort(xid))
+		TransactionIdDidAbort(xid) ||
+		StandbyTransactionIdIsPrepared(xid))
 		return;
 
 	elog(trace_recovery(DEBUG4),
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b7ce0c6..416ef5e 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -17,6 +17,7 @@
 #include "access/xlogdefs.h"
 #include "datatype/timestamp.h"
 #include "storage/lock.h"
+#include "access/xlogreader.h"
 
 /*
  * GlobalTransactionData is defined in twophase.c; other places have no
@@ -46,8 +47,8 @@ extern bool StandbyTransactionIdIsPrepared(TransactionId xid);
 
 extern TransactionId PrescanPreparedTransactions(TransactionId **xids_p,
 							int *nxids_p);
-extern void StandbyRecoverPreparedTransactions(bool overwriteOK);
-extern void RecoverPreparedTransactions(void);
+extern void RecoverPreparedFromFiles(bool overwriteOK);
+extern void RecoverPreparedFromXLOG(XLogReaderState *record);
 
 extern void RecreateTwoPhaseFile(TransactionId xid, void *content, int len);
 extern void RemoveTwoPhaseFile(TransactionId xid, bool giveWarning);
@@ -56,4 +57,5 @@ extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void XlogRedoFinishPrepared(TransactionId xid, bool isCommit);
 #endif   /* TWOPHASE_H */
diff --git a/src/test/recovery/t/008_twophase.pl b/src/test/recovery/t/008_twophase.pl
new file mode 100644
index 0000000..3c203cd
--- /dev/null
+++ b/src/test/recovery/t/008_twophase.pl
@@ -0,0 +1,249 @@
+# Tests dedicated to two-phase commit in recovery
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 12;
+
+# Setup master node
+my $node_master = get_new_node("master");
+$node_master->init(allows_streaming => 1);
+$node_master->append_conf('postgresql.conf', qq(
+	max_prepared_transactions = 10
+));
+$node_master->start;
+$node_master->backup('master_backup');
+$node_master->psql('postgres', "create table t(id int)");
+
+# Setup master node
+my $node_slave = get_new_node('slave');
+$node_slave->init_from_backup($node_master, 'master_backup', has_streaming => 1);
+$node_slave->start;
+
+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+	synchronous_standby_names = '*'
+));
+$node_master->psql('postgres', "select pg_reload_conf()");
+
+my $psql_out = '';
+my $psql_rc = '';
+
+###############################################################################
+# Check that we can commit and abort tx after soft restart.
+# Here checkpoint happens before shutdown and no WAL replay will occur at next
+# startup. In this case postgres re-create shared-memory state from twophase
+# files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	prepare transaction 'y';");
+$node_master->stop;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared transaction after restart.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared transaction after restart.');
+
+###############################################################################
+# Check that we can commit and abort after hard restart.
+# At next startup, WAL replay will re-create shared memory state for preaped
+# using dedicated WAL records.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	prepare transaction 'y';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared tx after teardown.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared transaction after teardown.');
+
+###############################################################################
+# Check that WAL replay can handle several transactions with same name GID.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Replay several transactions with same GID.');
+
+###############################################################################
+# Check that WAL replay cleans up its shared memory state and releases locks
+# while replaying transaction commits.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_master->teardown_node;
+$node_master->start;
+$psql_rc = $node_master->psql('postgres', "begin;
+	insert into t values (42);
+	-- This prepare can fail due to conflicting GID or locks conflicts if
+	-- replay did not fully cleanup its state on previous commit.
+	prepare transaction 'x';");
+is($psql_rc, '0', "Cleanup of shared memory state for 2PC commit");
+
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that WAL replay will cleanup its shared memory state on running slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;",
+				  stdout => \$psql_out);
+is($psql_out, '0',
+   "Cleanup of shared memory state on running standby without checkpoint.");
+
+###############################################################################
+# Same as in previous case, but let's force checkpoint on slave between
+# prepare and commit to use on-disk twophase files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';");
+$node_slave->psql('postgres', "checkpoint;");
+$node_master->psql('postgres', "commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;",
+				  stdout => \$psql_out);
+is($psql_out, '0',
+   "Cleanup of shared memory state on running standby after checkpoint.");
+
+###############################################################################
+# Check that prepared transactions can be committed on promoted slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$psql_rc = $node_slave->psql('postgres', "commit prepared 'x';");
+is($psql_rc, '0', "Restore of prepared transaction on promoted slave.");
+
+# change roles
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+
+###############################################################################
+# Check that prepared transactions are replayed after soft restart of standby
+# while master is down. Since standby knows that master is down it uses
+# different code path on start to be sure that the status of transactions is
+# consistent.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	prepare transaction 'x';");
+$node_master->stop;
+$node_slave->restart;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+				  stdout => \$psql_out);
+is($psql_out, '1',
+   "Restore prepared transactions from files with master down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that prepared transactions are correctly replayed after slave hard
+# restart while master is down.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (242);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->teardown_node;
+$node_slave->start;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres',
+							  "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+				  stdout => \$psql_out);
+is($psql_out, '1',
+   "Restore prepared transactions from records with master down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres', "commit prepared 'x'");
+
+
+###############################################################################
+# Check for a lock confcict between prepared tx with DDL inside and replay of
+# XLOG_STANDBY_LOCK wal record.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	create table t2(id int);
+	prepare transaction 'x';
+	-- checkpoint will issue XLOG_STANDBY_LOCK that can conflict with lock
+	-- held by 'create table' statement
+	checkpoint;
+	commit prepared 'x';");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+				  stdout => \$psql_out);
+is($psql_out, '0', "Replay prepared transaction with DDL.");
#83Simon Riggs
simon@2ndquadrant.com
In reply to: Michael Paquier (#82)
Re: Speedup twophase transactions

On 6 September 2016 at 02:41, Michael Paquier <michael.paquier@gmail.com> wrote:

After review the result is attached. Perhaps a committer could get a look at it?

Yes, will do, but it will be a few more days yet.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#84Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#82)
Re: Speedup twophase transactions

On 06 Sep 2016, at 04:41, Michael Paquier <michael.paquier@gmail.com> wrote:

On Sat, Sep 3, 2016 at 10:26 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:

On Fri, Sep 2, 2016 at 5:06 AM, Simon Riggs <simon@2ndquadrant.com> wrote:

On 13 April 2016 at 15:31, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Fixed patch attached. There already was infrastructure to skip currently
held locks during replay of standby_redo() and I’ve extended that with check for
prepared xids.

Please confirm that everything still works on current HEAD for the new
CF, so review can start.

The patch does not apply cleanly. Stas, could you rebase? I am
switching the patch to "waiting on author" for now.

So, I have just done the rebase myself and did an extra round of
reviews of the patch. Here are couple of comments after this extra
lookup.

LockGXactByXid() is aimed to be used only in recovery, so it seems
adapted to be to add an assertion with RecoveryInProgress(). Using
this routine in non-recovery code paths is risky because it assumes
that a PGXACT could be missing, which is fine in recovery as prepared
transactions could be moved to twophase files because of a checkpoint,
but not in normal cases. We could also add an assertion of the kind
gxact->locking_backend == InvalidBackendId before locking the PGXACT
but as this routine is just normally used by the startup process (in
non-standalone mode!) that's fine without.

The handling of invalidation messages and removal of relfilenodes done
in FinishGXact can be grouped together, checking only once for
!RecoveryInProgress().

+ *
+ *     The same procedure happens during replication and crash recovery.
*
"during WAL replay" is more generic and applies here.
+
+next_file:
+       continue;
+
That can be removed and replaced by a "continue;".
+   /*
+    * At first check prepared tx that wasn't yet moved to disk.
+    */
+   LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+   for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+   {
+       GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+       PGXACT *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+       if (TransactionIdEquals(pgxact->xid, xid))
+       {
+           LWLockRelease(TwoPhaseStateLock);
+           return true;
+       }
+   }
+   LWLockRelease(TwoPhaseStateLock);
This overlaps with TwoPhaseGetGXact but I'd rather keep both things
separated: it does not seem worth complicating the existing interface,
and putting that in cache during recovery has no meaning.

Oh, I was preparing new version of patch, after fresh look on it. Probably, I should
said that in this topic. I’ve found a bug in sub transaction handling and now working
on fix.

I have also reworked the test format, and fixed many typos and grammar
problems in the patch as well as in the tests.

Thanks!

After review the result is attached. Perhaps a committer could get a look at it?

I'll check it against my failure scenario with subtransactions and post results or updated patch here.

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#85Michael Paquier
michael.paquier@gmail.com
In reply to: Stas Kelvich (#84)
Re: Speedup twophase transactions

On Tue, Sep 6, 2016 at 5:58 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

On 06 Sep 2016, at 04:41, Michael Paquier <michael.paquier@gmail.com> wrote:
This overlaps with TwoPhaseGetGXact but I'd rather keep both things
separated: it does not seem worth complicating the existing interface,
and putting that in cache during recovery has no meaning.

Oh, I was preparing new version of patch, after fresh look on it. Probably, I should
said that in this topic. I’ve found a bug in sub transaction handling and now working
on fix.

What's the problem actually?
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#86Simon Riggs
simon@2ndquadrant.com
In reply to: Stas Kelvich (#84)
Re: Speedup twophase transactions

On 6 September 2016 at 09:58, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

On 06 Sep 2016, at 04:41, Michael Paquier <michael.paquier@gmail.com> wrote:

On Sat, Sep 3, 2016 at 10:26 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:

On Fri, Sep 2, 2016 at 5:06 AM, Simon Riggs <simon@2ndquadrant.com> wrote:

On 13 April 2016 at 15:31, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Fixed patch attached. There already was infrastructure to skip currently
held locks during replay of standby_redo() and I’ve extended that with check for
prepared xids.

Please confirm that everything still works on current HEAD for the new
CF, so review can start.

The patch does not apply cleanly. Stas, could you rebase? I am
switching the patch to "waiting on author" for now.

So, I have just done the rebase myself and did an extra round of
reviews of the patch. Here are couple of comments after this extra
lookup.

Oh, I was preparing new version of patch, after fresh look on it. Probably, I should
said that in this topic. I’ve found a bug in sub transaction handling and now working
on fix.

Not replying has wasted time and effort.

After review the result is attached. Perhaps a committer could get a look at it?

I'll check it against my failure scenario with subtransactions and post results or updated patch here.

Make sure tests are added for that. It would have been better to say
you knew there were bugs in it earlier.

This has been buggy so far, so I am hesitant about this now. I suggest
we add significant docs to explain how it works, so everybody can
double-check the concepts. Please also do what you can to reduce the
patch complexity.

I'll look at this again in two weeks time. Help me to make sure it
gets committed that time by doing a full and complete patch. Thanks.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#87Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#85)
Re: Speedup twophase transactions

On 06 Sep 2016, at 12:09, Simon Riggs <simon@2ndquadrant.com> wrote:

On 6 September 2016 at 09:58, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

I'll check it against my failure scenario with subtransactions and post results or updated patch here.

Make sure tests are added for that. It would have been better to say
you knew there were bugs in it earlier.

I’ve spotted that yesterday during rebase. Sorry. Next time in same situation i’ll write right away
to save everyone’s time.

On 06 Sep 2016, at 12:03, Michael Paquier <michael.paquier@gmail.com> wrote:

On Tue, Sep 6, 2016 at 5:58 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Oh, I was preparing new version of patch, after fresh look on it. Probably, I should
said that in this topic. I’ve found a bug in sub transaction handling and now working
on fix.

What's the problem actually?

Handling of xids_p array in PrescanPreparedTransactions() is wrong for prepared tx's in memory.

Also I want to double-check and add comments to RecoveryInProgress() checks in FinishGXact.

I’ll post reworked patch in several days.

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#88Michael Paquier
michael.paquier@gmail.com
In reply to: Stas Kelvich (#87)
Re: Speedup twophase transactions

On 06 Sep 2016, at 12:03, Michael Paquier <michael.paquier@gmail.com> wrote:

On Tue, Sep 6, 2016 at 5:58 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Oh, I was preparing new version of patch, after fresh look on it. Probably, I should
said that in this topic. I’ve found a bug in sub transaction handling and now working
on fix.

What's the problem actually?

Handling of xids_p array in PrescanPreparedTransactions() is wrong for prepared tx's in memory.
Also I want to double-check and add comments to RecoveryInProgress() checks in FinishGXact.

I’ll post reworked patch in several days.

Could you use as a base the version I just sent yesterday then? I
noticed many mistakes in the comments, for example many s/it's/its/
and did a couple of adjustments around the code, the goto next_file
was particularly ugly. That will be that much work not do to again
later.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#89Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#88)
Re: Speedup twophase transactions

On 07 Sep 2016, at 03:09, Michael Paquier <michael.paquier@gmail.com> wrote:

On 06 Sep 2016, at 12:03, Michael Paquier <michael.paquier@gmail.com> wrote:

On Tue, Sep 6, 2016 at 5:58 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Oh, I was preparing new version of patch, after fresh look on it. Probably, I should
said that in this topic. I’ve found a bug in sub transaction handling and now working
on fix.

What's the problem actually?

Handling of xids_p array in PrescanPreparedTransactions() is wrong for prepared tx's in memory.
Also I want to double-check and add comments to RecoveryInProgress() checks in FinishGXact.

I’ll post reworked patch in several days.

Could you use as a base the version I just sent yesterday then? I
noticed many mistakes in the comments, for example many s/it's/its/
and did a couple of adjustments around the code, the goto next_file
was particularly ugly. That will be that much work not do to again
later.

Yes. Already merged branch with your version.

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#90Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Stas Kelvich (#89)
1 attachment(s)
Re: Speedup twophase transactions

On 07 Sep 2016, at 11:07, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

On 07 Sep 2016, at 03:09, Michael Paquier <michael.paquier@gmail.com> wrote:

On 06 Sep 2016, at 12:03, Michael Paquier <michael.paquier@gmail.com> wrote:

On Tue, Sep 6, 2016 at 5:58 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Oh, I was preparing new version of patch, after fresh look on it. Probably, I should
said that in this topic. I’ve found a bug in sub transaction handling and now working
on fix.

What's the problem actually?

Handling of xids_p array in PrescanPreparedTransactions() is wrong for prepared tx's in memory.
Also I want to double-check and add comments to RecoveryInProgress() checks in FinishGXact.

I’ll post reworked patch in several days.

Could you use as a base the version I just sent yesterday then? I
noticed many mistakes in the comments, for example many s/it's/its/
and did a couple of adjustments around the code, the goto next_file
was particularly ugly. That will be that much work not do to again
later.

Yes. Already merged branch with your version.

Here is updated version of patch.

Looking through old version i’ve noted few things that were bothering me:

* In case of crash replay PREPARE redo accesses SUBTRANS, but StartupSUBTRANS() isn’t called yet
in StartupXLOG().
* Several functions in twophase.c have to walk through both PGPROC and pg_twophase directory.

Now I slightly changed order of things in StartupXLOG() so twophase state loaded from from file and
StartupSUBTRANS called before actual recovery starts. So in all other functions we can be sure that
file were already loaded in memory.

Also since 2pc transactions now are dumped to files only on checkpoint, we can get rid of
PrescanPreparedTransactions() — all necessary info can reside in checkpoint itself. I’ve changed
behaviour of oldestActiveXid write in checkpoint and that’s probably discussable topic, but ISTM
that simplifies a lot recovery logic in both twophase.c and xlog.c. More comments on that in patch itself.

Attachments:

twophase_replay.v7.patchapplication/octet-stream; name=twophase_replay.v7.patchDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 5415604..d03ab07 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,8 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *
+ *		The same procedure happens during WAL replay.
  *
  *-------------------------------------------------------------------------
  */
@@ -578,6 +578,46 @@ LockGXact(const char *gid, Oid user)
 }
 
 /*
+ * LockGXactByXid
+ *
+ * Find prepared transaction by xid and lock corresponding GXACT.
+ * This is used during recovery as an alternative to LockGXact(), and
+ * should only be used in recovery. No entries found is sane situation,
+ * see comments inside XlogRedoFinishPrepared().
+ *
+ * Returns the transaction data if found, or NULL if nothing has been locked.
+ */
+static GlobalTransaction
+LockGXactByXid(TransactionId xid)
+{
+	int		i;
+	GlobalTransaction gxact = NULL;
+
+	Assert(RecoveryInProgress());
+
+	LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		PGXACT	   *pgxact;
+
+		gxact = TwoPhaseState->prepXacts[i];
+		pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (TransactionIdEquals(xid, pgxact->xid))
+		{
+			/* ok to lock it */
+			gxact->locking_backend = MyBackendId;
+			MyLockedGxact = gxact;
+			LWLockRelease(TwoPhaseStateLock);
+			return gxact;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+	return NULL;
+}
+
+/*
  * RemoveGXact
  *		Remove the prepared transaction from the shared memory array.
  *
@@ -1241,9 +1281,9 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
  * Reads 2PC data from xlog. During checkpoint this data will be moved to
  * twophase files and ReadTwoPhaseFile should be used instead.
  *
- * Note clearly that this function accesses WAL during normal operation, similarly
- * to the way WALSender or Logical Decoding would do. It does not run during
- * crash recovery or standby processing.
+ * Note that this function accesses WAL not only during recovery but also
+ * during normal operation, similarly to the way WALSender or Logical
+ * Decoding would do.
  */
 static void
 XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
@@ -1252,8 +1292,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1288,40 +1326,54 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 
 
 /*
- * Confirms an xid is prepared, during recovery
+ * Confirms an xid is prepared, during recovery.
+ *
+ * Here we know that all info about old prepared transactions is already
+ * loaded by RecoverPreparedFromFiles() so we can check only PGPROC.
  */
 bool
 StandbyTransactionIdIsPrepared(TransactionId xid)
 {
-	char	   *buf;
-	TwoPhaseFileHeader *hdr;
-	bool		result;
+	int			i;
 
 	Assert(TransactionIdIsValid(xid));
 
 	if (max_prepared_xacts <= 0)
 		return false;			/* nothing to do */
 
-	/* Read and validate file */
-	buf = ReadTwoPhaseFile(xid, false);
-	if (buf == NULL)
-		return false;
+	/*
+	 * Check if this prepared transaction has its information in
+	 * shared memory, and use it.
+	 */
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		PGXACT *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 
-	/* Check header also */
-	hdr = (TwoPhaseFileHeader *) buf;
-	result = TransactionIdEquals(hdr->xid, xid);
-	pfree(buf);
+		if (TransactionIdEquals(pgxact->xid, xid))
+		{
+			LWLockRelease(TwoPhaseStateLock);
+			return true;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
 
-	return result;
+	return false;
 }
 
 /*
- * FinishPreparedTransaction: execute COMMIT PREPARED or ROLLBACK PREPARED
+ * FinishGXact
+ *
+ * Do the actual finish of COMMIT/ABORT PREPARED. Calls is responsible
+ * for locking the transaction this routine is working on.
+ *
+ * This function can be called during replay to clean memory state for
+ * previously prepared xact. In that case actions are the same as in
+ * normal operations but without any writes to WAL or files.
  */
-void
-FinishPreparedTransaction(const char *gid, bool isCommit)
+static void FinishGXact(GlobalTransaction gxact, bool isCommit)
 {
-	GlobalTransaction gxact;
 	PGPROC	   *proc;
 	PGXACT	   *pgxact;
 	TransactionId xid;
@@ -1332,16 +1384,9 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	TransactionId *children;
 	RelFileNode *commitrels;
 	RelFileNode *abortrels;
-	RelFileNode *delrels;
-	int			ndelrels;
 	SharedInvalidationMessage *invalmsgs;
 	int			i;
 
-	/*
-	 * Validate the GID, and lock the GXACT to ensure that two backends do not
-	 * try to commit the same GID at once.
-	 */
-	gxact = LockGXact(gid, GetUserId());
 	proc = &ProcGlobal->allProcs[gxact->pgprocno];
 	pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 	xid = pgxact->xid;
@@ -1383,17 +1428,23 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	 * TransactionIdIsInProgress will stop saying the prepared xact is in
 	 * progress), then run the post-commit or post-abort callbacks. The
 	 * callbacks will release the locks the transaction held.
+	 *
+	 * In recovery nothing needs to happen here as this generates WAL
+	 * records.
 	 */
-	if (isCommit)
-		RecordTransactionCommitPrepared(xid,
+	if (!RecoveryInProgress())
+	{
+		if (isCommit)
+			RecordTransactionCommitPrepared(xid,
 										hdr->nsubxacts, children,
 										hdr->ncommitrels, commitrels,
 										hdr->ninvalmsgs, invalmsgs,
 										hdr->initfileinval);
-	else
-		RecordTransactionAbortPrepared(xid,
+		else
+			RecordTransactionAbortPrepared(xid,
 									   hdr->nsubxacts, children,
 									   hdr->nabortrels, abortrels);
+	}
 
 	ProcArrayRemove(proc, latestXid);
 
@@ -1408,41 +1459,50 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 	gxact->valid = false;
 
 	/*
-	 * We have to remove any files that were supposed to be dropped. For
-	 * consistency with the regular xact.c code paths, must do this before
-	 * releasing locks, so do it before running the callbacks.
-	 *
-	 * NB: this code knows that we couldn't be dropping any temp rels ...
+	 * Perform actions needed only during normal operation, but *not* recovery.
 	 */
-	if (isCommit)
+	if (!RecoveryInProgress())
 	{
-		delrels = commitrels;
-		ndelrels = hdr->ncommitrels;
-	}
-	else
-	{
-		delrels = abortrels;
-		ndelrels = hdr->nabortrels;
-	}
-	for (i = 0; i < ndelrels; i++)
-	{
-		SMgrRelation srel = smgropen(delrels[i], InvalidBackendId);
+		RelFileNode *delrels;
+		int			ndelrels;
 
-		smgrdounlink(srel, false);
-		smgrclose(srel);
-	}
+		/*
+		 * We have to remove any files that were supposed to be dropped. For
+		 * consistency with the regular xact.c code paths, must do this before
+		 * releasing locks, so do it before running the callbacks.
+		 *
+		 * NB: this code knows that we couldn't be dropping any temp rels ...
+		 */
+		if (isCommit)
+		{
+			delrels = commitrels;
+			ndelrels = hdr->ncommitrels;
+		}
+		else
+		{
+			delrels = abortrels;
+			ndelrels = hdr->nabortrels;
+		}
+		for (i = 0; i < ndelrels; i++)
+		{
+			SMgrRelation srel = smgropen(delrels[i], InvalidBackendId);
 
-	/*
-	 * Handle cache invalidation messages.
-	 *
-	 * Relcache init file invalidation requires processing both before and
-	 * after we send the SI messages. See AtEOXact_Inval()
-	 */
-	if (hdr->initfileinval)
-		RelationCacheInitFilePreInvalidate();
-	SendSharedInvalidMessages(invalmsgs, hdr->ninvalmsgs);
-	if (hdr->initfileinval)
-		RelationCacheInitFilePostInvalidate();
+			smgrdounlink(srel, false);
+			smgrclose(srel);
+		}
+
+		/*
+		 * Handle cache invalidation messages.
+		 *
+		 * Relcache init file invalidation requires processing both before and
+		 * after we send the SI messages. See AtEOXact_Inval()
+		 */
+		if (hdr->initfileinval)
+			RelationCacheInitFilePreInvalidate();
+		SendSharedInvalidMessages(invalmsgs, hdr->ninvalmsgs);
+		if (hdr->initfileinval)
+			RelationCacheInitFilePostInvalidate();
+	}
 
 	/* And now do the callbacks */
 	if (isCommit)
@@ -1468,6 +1528,51 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
 }
 
 /*
+ * FinishPreparedTransaction: execute COMMIT PREPARED or ROLLBACK PREPARED
+ */
+void
+FinishPreparedTransaction(const char *gid, bool isCommit)
+{
+	GlobalTransaction gxact;
+
+	/*
+	 * Validate the GID, and lock the GXACT to ensure that two backends do not
+	 * try to commit the same GID at once.
+	 */
+	gxact = LockGXact(gid, GetUserId());
+	FinishGXact(gxact, isCommit);
+}
+
+/*
+ * XlogRedoFinishPrepared()
+ *
+ * This function is called during recovery for WAL records working on COMMIT
+ * PREPARED or ABORT PREPARED. That function cleans up memory state that was
+ * created while replaying its corresponding PREPARE record if its information
+ * was not on disk in a twophase file.
+ */
+void
+XlogRedoFinishPrepared(TransactionId xid, bool isCommit)
+{
+	GlobalTransaction gxact;
+
+	Assert(RecoveryInProgress());
+
+	gxact = LockGXactByXid(xid);
+
+	/*
+	 * If requested xid was not found that means that PREPARE record was before
+	 * checkpoint that we are replaying from and COMMIT managed to remove
+	 * twophase file before crash. So we know that no memory state for prepared
+	 * tx was created and no files left and everything is fine.
+	 */
+	if (!gxact)
+		return;
+
+	FinishGXact(gxact, isCommit);
+}
+
+/*
  * Scan 2PC state data in memory and call the indicated callbacks for each 2PC record.
  */
 static void
@@ -1654,43 +1759,21 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
 }
 
 /*
- * PrescanPreparedTransactions
- *
- * Scan the pg_twophase directory and determine the range of valid XIDs
- * present.  This is run during database startup, after we have completed
- * reading WAL.  ShmemVariableCache->nextXid has been set to one more than
- * the highest XID for which evidence exists in WAL.
- *
  * We throw away any prepared xacts with main XID beyond nextXid --- if any
  * are present, it suggests that the DBA has done a PITR recovery to an
  * earlier point in time without cleaning out pg_twophase.  We dare not
  * try to recover such prepared xacts since they likely depend on database
  * state that doesn't exist now.
- *
- * However, we will advance nextXid beyond any subxact XIDs belonging to
- * valid prepared xacts.  We need to do this since subxact commit doesn't
- * write a WAL entry, and so there might be no evidence in WAL of those
- * subxact XIDs.
- *
- * Our other responsibility is to determine and return the oldest valid XID
- * among the prepared xacts (if none, return ShmemVariableCache->nextXid).
- * This is needed to synchronize pg_subtrans startup properly.
- *
- * If xids_p and nxids_p are not NULL, pointer to a palloc'd array of all
- * top-level xids is stored in *xids_p. The number of entries in the array
- * is returned in *nxids_p.
  */
-TransactionId
-PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
+void
+CleanupPreparedTransactionsOnPITR(void)
 {
-	TransactionId origNextXid = ShmemVariableCache->nextXid;
-	TransactionId result = origNextXid;
 	DIR		   *cldir;
 	struct dirent *clde;
-	TransactionId *xids = NULL;
-	int			nxids = 0;
-	int			allocsize = 0;
 
+	/*
+	 * Scan files in pg_twophase directory
+	 */
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
 	{
@@ -1698,15 +1781,11 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 			strspn(clde->d_name, "0123456789ABCDEF") == 8)
 		{
 			TransactionId xid;
-			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
 			/* Reject XID if too new */
-			if (TransactionIdFollowsOrEquals(xid, origNextXid))
+			if (TransactionIdFollowsOrEquals(xid, ShmemVariableCache->nextXid))
 			{
 				ereport(WARNING,
 						(errmsg("removing future two-phase state file \"%s\"",
@@ -1714,201 +1793,163 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 				RemoveTwoPhaseFile(xid, true);
 				continue;
 			}
-
-			/*
-			 * Note: we can't check if already processed because clog
-			 * subsystem isn't up yet.
-			 */
-
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
-
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				pfree(buf);
-				continue;
-			}
-
-			/*
-			 * OK, we think this file is valid.  Incorporate xid into the
-			 * running-minimum result.
-			 */
-			if (TransactionIdPrecedes(xid, result))
-				result = xid;
-
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID, and they may force us to advance nextXid.
-			 *
-			 * We don't expect anyone else to modify nextXid, hence we don't
-			 * need to hold a lock while examining it.  We still acquire the
-			 * lock to modify it, though.
-			 */
-			subxids = (TransactionId *) (buf +
-								MAXALIGN(sizeof(TwoPhaseFileHeader)) +
-								MAXALIGN(hdr->gidlen));
-			for (i = 0; i < hdr->nsubxacts; i++)
-			{
-				TransactionId subxid = subxids[i];
-
-				Assert(TransactionIdFollows(subxid, xid));
-				if (TransactionIdFollowsOrEquals(subxid,
-												 ShmemVariableCache->nextXid))
-				{
-					LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
-					ShmemVariableCache->nextXid = subxid;
-					TransactionIdAdvance(ShmemVariableCache->nextXid);
-					LWLockRelease(XidGenLock);
-				}
-			}
-
-
-			if (xids_p)
-			{
-				if (nxids == allocsize)
-				{
-					if (nxids == 0)
-					{
-						allocsize = 10;
-						xids = palloc(allocsize * sizeof(TransactionId));
-					}
-					else
-					{
-						allocsize = allocsize * 2;
-						xids = repalloc(xids, allocsize * sizeof(TransactionId));
-					}
-				}
-				xids[nxids++] = xid;
-			}
-
-			pfree(buf);
 		}
 	}
 	FreeDir(cldir);
+}
+
+
+/*
+ * Returns array of currently prepared two-phase transactions.
+ *
+ * If we faced shutdown checkpoint during replay we need that to create
+ * RunningTransactionsData structure and pass it to ProcArrayApplyRecoveryInfo.
+ */
+void
+GetPreparedTransactions(TransactionId **xids_p, int *nxids_p)
+{
+	TransactionId *xids = NULL;
+	int			nxids = 0;
+	int			allocsize = 0;
+	int			i;
+
+	Assert(RecoveryInProgress());
+
+	/*
+	 * We need to check the PGXACT array for prepared transactions that doesn't
+	 * have any state file in case of a slave restart with the master being off.
+	 */
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
+
+		if (!gxact->valid)
+			continue;
+
+		if (xids_p)
+		{
+			if (nxids == allocsize)
+			{
+				if (nxids == 0)
+				{
+					allocsize = 10;
+					xids = palloc(allocsize * sizeof(TransactionId));
+				}
+				else
+				{
+					allocsize = allocsize * 2;
+					xids = repalloc(xids, allocsize * sizeof(TransactionId));
+				}
+			}
+			xids[nxids++] = pgxact->xid;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
 
 	if (xids_p)
 	{
 		*xids_p = xids;
 		*nxids_p = nxids;
 	}
-
-	return result;
 }
 
+
 /*
- * StandbyRecoverPreparedTransactions
+ * RecoverPreparedFromBuffer
  *
- * Scan the pg_twophase directory and setup all the required information to
- * allow standby queries to treat prepared transactions as still active.
- * This is never called at the end of recovery - we use
- * RecoverPreparedTransactions() at that point.
+ * Parse data in given buffer (that can be a pointer to WAL record holding
+ * this information or data read from a twophase file) and build the
+ * shared-memory state for that prepared transaction.
  *
- * Currently we simply call SubTransSetParent() for any subxids of prepared
- * transactions. If overwriteOK is true, it's OK if some XIDs have already
- * been marked in pg_subtrans.
+ * Caller is responsible for calling MarkAsPrepared() on the returned gxact.
  */
-void
-StandbyRecoverPreparedTransactions(bool overwriteOK)
+static GlobalTransaction
+RecoverPreparedFromBuffer(char *buf)
 {
-	DIR		   *cldir;
-	struct dirent *clde;
+	char			*bufptr;
+	const char		*gid;
+	TransactionId	*subxids;
+	bool			overwriteOK = false;
+	int				i;
+	GlobalTransaction gxact;
+	TwoPhaseFileHeader	*hdr;
 
-	cldir = AllocateDir(TWOPHASE_DIR);
-	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
-	{
-		if (strlen(clde->d_name) == 8 &&
-			strspn(clde->d_name, "0123456789ABCDEF") == 8)
-		{
-			TransactionId xid;
-			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
+	/* Deconstruct header */
+	hdr = (TwoPhaseFileHeader *) buf;
+	bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+	gid = (const char *) bufptr;
+	bufptr += MAXALIGN(hdr->gidlen);
+	subxids = (TransactionId *) bufptr;
+	bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
+	bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
 
-			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
+	/*
+	 * It's possible that SubTransSetParent has been set before, if
+	 * the prepared transaction generated xid assignment records. Test
+	 * here must match one used in AssignTransactionId().
+	 */
+	if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
+						 XLogLogicalInfoActive()))
+		overwriteOK = true;
 
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+	/*
+	 * Reconstruct subtrans state for the transaction --- needed
+	 * because pg_subtrans is not preserved over a restart.  Note that
+	 * we are linking all the subtransactions directly to the
+	 * top-level XID; there may originally have been a more complex
+	 * hierarchy, but there's no need to restore that exactly.
+	 */
+	for (i = 0; i < hdr->nsubxacts; i++)
+		SubTransSetParent(subxids[i], hdr->xid, overwriteOK);
 
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+	/*
+	 * Recreate its GXACT and dummy PGPROC
+	 */
+	gxact = MarkAsPreparing(hdr->xid, gid,
+							hdr->prepared_at,
+							hdr->owner, hdr->database);
+	GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
 
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				pfree(buf);
-				continue;
-			}
+	/*
+	 * Recover other state (notably locks) using resource managers
+	 */
+	ProcessRecords(bufptr, hdr->xid, twophase_recover_callbacks);
 
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID.
-			 */
-			subxids = (TransactionId *) (buf +
-								MAXALIGN(sizeof(TwoPhaseFileHeader)) +
-								MAXALIGN(hdr->gidlen));
-			for (i = 0; i < hdr->nsubxacts; i++)
-			{
-				TransactionId subxid = subxids[i];
+	/*
+	 * Release locks held by the standby process after we process each
+	 * prepared transaction. As a result, we don't need too many
+	 * additional locks at any one time.
+	 */
+	if (InHotStandby)
+		StandbyReleaseLockTree(hdr->xid, hdr->nsubxacts, subxids);
 
-				Assert(TransactionIdFollows(subxid, xid));
-				SubTransSetParent(xid, subxid, overwriteOK);
-			}
+	/*
+	 * We're done with recovering this transaction. Clear
+	 * MyLockedGxact, like we do in PrepareTransaction() during normal
+	 * operation.
+	 */
+	PostPrepare_Twophase();
 
-			pfree(buf);
-		}
-	}
-	FreeDir(cldir);
+	return gxact;
 }
 
 /*
- * RecoverPreparedTransactions
+ * RecoverPreparedFromFiles
  *
  * Scan the pg_twophase directory and reload shared-memory state for each
  * prepared transaction (reacquire locks, etc).  This is run during database
- * startup.
+ * startup before actual WAL replay started.
  */
 void
-RecoverPreparedTransactions(void)
+RecoverPreparedFromFiles(void)
 {
 	char		dir[MAXPGPATH];
 	DIR		   *cldir;
 	struct dirent *clde;
-	bool		overwriteOK = false;
 
 	snprintf(dir, MAXPGPATH, "%s", TWOPHASE_DIR);
 
@@ -1920,25 +1961,10 @@ RecoverPreparedTransactions(void)
 		{
 			TransactionId xid;
 			char	   *buf;
-			char	   *bufptr;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
 			GlobalTransaction gxact;
-			const char *gid;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
-
 			/* Read and validate file */
 			buf = ReadTwoPhaseFile(xid, true);
 			if (buf == NULL)
@@ -1953,73 +1979,39 @@ RecoverPreparedTransactions(void)
 			ereport(LOG,
 					(errmsg("recovering prepared transaction %u", xid)));
 
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			Assert(TransactionIdEquals(hdr->xid, xid));
-			bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
-			gid = (const char *) bufptr;
-			bufptr += MAXALIGN(hdr->gidlen);
-			subxids = (TransactionId *) bufptr;
-			bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
-			bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
-
-			/*
-			 * It's possible that SubTransSetParent has been set before, if
-			 * the prepared transaction generated xid assignment records. Test
-			 * here must match one used in AssignTransactionId().
-			 */
-			if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
-								 XLogLogicalInfoActive()))
-				overwriteOK = true;
-
-			/*
-			 * Reconstruct subtrans state for the transaction --- needed
-			 * because pg_subtrans is not preserved over a restart.  Note that
-			 * we are linking all the subtransactions directly to the
-			 * top-level XID; there may originally have been a more complex
-			 * hierarchy, but there's no need to restore that exactly.
-			 */
-			for (i = 0; i < hdr->nsubxacts; i++)
-				SubTransSetParent(subxids[i], xid, overwriteOK);
-
-			/*
-			 * Recreate its GXACT and dummy PGPROC
-			 */
-			gxact = MarkAsPreparing(xid, gid,
-									hdr->prepared_at,
-									hdr->owner, hdr->database);
+			gxact = RecoverPreparedFromBuffer(buf);
 			gxact->ondisk = true;
-			GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
 			MarkAsPrepared(gxact);
 
-			/*
-			 * Recover other state (notably locks) using resource managers
-			 */
-			ProcessRecords(bufptr, xid, twophase_recover_callbacks);
-
-			/*
-			 * Release locks held by the standby process after we process each
-			 * prepared transaction. As a result, we don't need too many
-			 * additional locks at any one time.
-			 */
-			if (InHotStandby)
-				StandbyReleaseLockTree(xid, hdr->nsubxacts, subxids);
-
-			/*
-			 * We're done with recovering this transaction. Clear
-			 * MyLockedGxact, like we do in PrepareTransaction() during normal
-			 * operation.
-			 */
-			PostPrepare_Twophase();
-
 			pfree(buf);
 		}
 	}
 	FreeDir(cldir);
 }
 
+
+/*
+ * RecoverPreparedFromXLOG
+ *
+ * To avoid the creation of twophase state files during replay we register
+ * WAL records for prepared transactions in shared memory in the same way
+ * during normal operations. If replay faces a WAL record for a COMMIT
+ * PREPARED transaction before a checkpoint or restartpoint happens then
+ * no files are used, limiting the I/O impact of such operations during
+ * recovery.
+ */
+void
+RecoverPreparedFromXLOG(XLogReaderState *record)
+{
+	GlobalTransaction gxact;
+
+	gxact = RecoverPreparedFromBuffer((char *) XLogRecGetData(record));
+	gxact->prepare_start_lsn = record->ReadRecPtr;
+	gxact->prepare_end_lsn = record->EndRecPtr;
+	MarkAsPrepared(gxact);
+}
+
+
 /*
  *	RecordTransactionCommitPrepared
  *
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index e11b229..6a40425 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -5602,7 +5602,7 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+			XlogRedoFinishPrepared(parsed.twophase_xid, true);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5622,14 +5622,12 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+			XlogRedoFinishPrepared(parsed.twophase_xid, false);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		RecoverPreparedFromXLOG(record);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 2189c22..1ba6f14 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -5993,7 +5993,6 @@ StartupXLOG(void)
 	TimeLineID	EndOfLogTLI;
 	TimeLineID	PrevTimeLineID;
 	XLogRecord *record;
-	TransactionId oldestActiveXID;
 	bool		backupEndRequired = false;
 	bool		backupFromStandby = false;
 	DBState		dbstate_at_startup;
@@ -6513,6 +6512,24 @@ StartupXLOG(void)
 		InRecovery = true;
 	}
 
+	/*
+	 * Recover two-phase state of old prepared transactions.
+	 *
+	 * Long-running prepared transaction are moved to filesystem during
+	 * checkpoint as commit will need info from prepare record that will
+	 * be unavalable. So here we are following logical order of things and
+	 * restoring transactions that are behind checkpoint horizon before replay
+	 * started.
+	 *
+	 * RecoverPreparedFromFiles and cosequent replay of prepare records
+	 * will need access to SUBTRANCE structures, so init it here using
+	 * checkpoint's oldestActiveXid as it certainly not higher than latest
+	 * prepared xid.
+	 */
+	StartupCLOG();
+	StartupSUBTRANS(checkPoint.oldestActiveXid);
+	RecoverPreparedFromFiles();
+
 	/* REDO */
 	if (InRecovery)
 	{
@@ -6649,32 +6666,17 @@ StartupXLOG(void)
 		 */
 		if (ArchiveRecoveryRequested && EnableHotStandby)
 		{
-			TransactionId *xids;
-			int			nxids;
-
 			ereport(DEBUG1,
 					(errmsg("initializing for hot standby")));
 
 			InitRecoveryTransactionEnvironment();
 
-			if (wasShutdown)
-				oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
-			else
-				oldestActiveXID = checkPoint.oldestActiveXid;
-			Assert(TransactionIdIsValid(oldestActiveXID));
+			Assert(TransactionIdIsValid(checkPoint.oldestActiveXid));
 
 			/* Tell procarray about the range of xids it has to deal with */
 			ProcArrayInitRecovery(ShmemVariableCache->nextXid);
 
 			/*
-			 * Startup commit log and subtrans only.  MultiXact and commit
-			 * timestamp have already been started up and other SLRUs are not
-			 * maintained during recovery and need not be started yet.
-			 */
-			StartupCLOG();
-			StartupSUBTRANS(oldestActiveXID);
-
-			/*
 			 * If we're beginning at a shutdown checkpoint, we know that
 			 * nothing was running on the master at this point. So fake-up an
 			 * empty running-xacts record and use that here and now. Recover
@@ -6683,7 +6685,11 @@ StartupXLOG(void)
 			if (wasShutdown)
 			{
 				RunningTransactionsData running;
-				TransactionId latestCompletedXid;
+				TransactionId   latestCompletedXid;
+				TransactionId  *xids;
+				int				nxids;
+
+				GetPreparedTransactions(&xids, &nxids);
 
 				/*
 				 * Construct a RunningTransactions snapshot representing a
@@ -6695,7 +6701,7 @@ StartupXLOG(void)
 				running.subxcnt = 0;
 				running.subxid_overflow = false;
 				running.nextXid = checkPoint.nextXid;
-				running.oldestRunningXid = oldestActiveXID;
+				running.oldestRunningXid = checkPoint.oldestActiveXid;
 				latestCompletedXid = checkPoint.nextXid;
 				TransactionIdRetreat(latestCompletedXid);
 				Assert(TransactionIdIsNormal(latestCompletedXid));
@@ -6703,8 +6709,6 @@ StartupXLOG(void)
 				running.xids = xids;
 
 				ProcArrayApplyRecoveryInfo(&running);
-
-				StandbyRecoverPreparedTransactions(false);
 			}
 		}
 
@@ -7012,6 +7016,8 @@ StartupXLOG(void)
 					ereport(FATAL,
 							(errmsg("requested recovery stop point is before consistent recovery point")));
 
+				CleanupPreparedTransactionsOnPITR();
+
 				/*
 				 * This is the last point where we can restart recovery with a
 				 * new recovery target, if we shutdown and begin again. After
@@ -7272,9 +7278,6 @@ StartupXLOG(void)
 	XLogCtl->LogwrtRqst.Write = EndOfLog;
 	XLogCtl->LogwrtRqst.Flush = EndOfLog;
 
-	/* Pre-scan prepared transactions to find out the range of XIDs present */
-	oldestActiveXID = PrescanPreparedTransactions(NULL, NULL);
-
 	/*
 	 * Update full_page_writes in shared memory and write an XLOG_FPW_CHANGE
 	 * record before resource manager writes cleanup WAL records or checkpoint
@@ -7447,24 +7450,11 @@ StartupXLOG(void)
 	LWLockRelease(ProcArrayLock);
 
 	/*
-	 * Start up the commit log and subtrans, if not already done for hot
-	 * standby.  (commit timestamps are started below, if necessary.)
-	 */
-	if (standbyState == STANDBY_DISABLED)
-	{
-		StartupCLOG();
-		StartupSUBTRANS(oldestActiveXID);
-	}
-
-	/*
 	 * Perform end of recovery actions for any SLRUs that need it.
 	 */
 	TrimCLOG();
 	TrimMultiXact();
 
-	/* Reload shared-memory state for prepared transactions */
-	RecoverPreparedTransactions();
-
 	/*
 	 * Shutdown the recovery environment. This must occur after
 	 * RecoverPreparedTransactions(), see notes for lock_twophase_recover()
@@ -8311,14 +8301,16 @@ CreateCheckPoint(int flags)
 	checkPoint.time = (pg_time_t) time(NULL);
 
 	/*
-	 * For Hot Standby, derive the oldestActiveXid before we fix the redo
-	 * pointer. This allows us to begin accumulating changes to assemble our
-	 * starting snapshot of locks and transactions.
+	 * Before recovery we need to start SUBTRANS in a way that it will be
+	 * capable of storing info about possible two-phase transactions that we
+	 * will prepare during xlog replay. We can get here value that is less
+	 * then actual latest prepared transaction, but that is okay.
+	 *
+	 * Also for Hot Standby, this allows us to derive the oldestActiveXid
+	 * before we fix the redo pointer and to begin accumulating changes to
+	 * assemble our starting snapshot of locks and transactions.
 	 */
-	if (!shutdown && XLogStandbyInfoActive())
-		checkPoint.oldestActiveXid = GetOldestActiveTransactionId();
-	else
-		checkPoint.oldestActiveXid = InvalidTransactionId;
+	checkPoint.oldestActiveXid = GetOldestActiveTransactionId();
 
 	/*
 	 * We must block concurrent insertions while examining insert state to
@@ -9352,11 +9344,10 @@ xlog_redo(XLogReaderState *record)
 		{
 			TransactionId *xids;
 			int			nxids;
-			TransactionId oldestActiveXID;
 			TransactionId latestCompletedXid;
 			RunningTransactionsData running;
 
-			oldestActiveXID = PrescanPreparedTransactions(&xids, &nxids);
+			GetPreparedTransactions(&xids, &nxids);
 
 			/*
 			 * Construct a RunningTransactions snapshot representing a shut
@@ -9368,7 +9359,7 @@ xlog_redo(XLogReaderState *record)
 			running.subxcnt = 0;
 			running.subxid_overflow = false;
 			running.nextXid = checkPoint.nextXid;
-			running.oldestRunningXid = oldestActiveXID;
+			running.oldestRunningXid = checkPoint.oldestActiveXid;
 			latestCompletedXid = checkPoint.nextXid;
 			TransactionIdRetreat(latestCompletedXid);
 			Assert(TransactionIdIsNormal(latestCompletedXid));
@@ -9376,8 +9367,6 @@ xlog_redo(XLogReaderState *record)
 			running.xids = xids;
 
 			ProcArrayApplyRecoveryInfo(&running);
-
-			StandbyRecoverPreparedTransactions(true);
 		}
 
 		/* ControlFile->checkPointCopy always tracks the latest ckpt XID */
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index e5d487d..ec2371c 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -2078,7 +2078,8 @@ GetRunningTransactionData(void)
  * We look at all databases, though there is no need to include WALSender
  * since this has no effect on hot standby conflicts.
  *
- * This is never executed during recovery so there is no need to look at
+ * During recovery we need only minimum anong prepared transaction and since
+ * they are registering dummy PGPROC during replay there is no need to look at
  * KnownAssignedXids.
  *
  * We don't worry about updating other counters, we want to keep this as
@@ -2092,8 +2093,6 @@ GetOldestActiveTransactionId(void)
 	TransactionId oldestRunningXid;
 	int			index;
 
-	Assert(!RecoveryInProgress());
-
 	LWLockAcquire(ProcArrayLock, LW_SHARED);
 
 	/*
diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c
index 547f1a8..5ea2530 100644
--- a/src/backend/storage/ipc/standby.c
+++ b/src/backend/storage/ipc/standby.c
@@ -608,7 +608,8 @@ StandbyAcquireAccessExclusiveLock(TransactionId xid, Oid dbOid, Oid relOid)
 	/* Already processed? */
 	if (!TransactionIdIsValid(xid) ||
 		TransactionIdDidCommit(xid) ||
-		TransactionIdDidAbort(xid))
+		TransactionIdDidAbort(xid) ||
+		StandbyTransactionIdIsPrepared(xid))
 		return;
 
 	elog(trace_recovery(DEBUG4),
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b7ce0c6..f6fbf78 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -17,6 +17,7 @@
 #include "access/xlogdefs.h"
 #include "datatype/timestamp.h"
 #include "storage/lock.h"
+#include "access/xlogreader.h"
 
 /*
  * GlobalTransactionData is defined in twophase.c; other places have no
@@ -44,10 +45,11 @@ extern void StartPrepare(GlobalTransaction gxact);
 extern void EndPrepare(GlobalTransaction gxact);
 extern bool StandbyTransactionIdIsPrepared(TransactionId xid);
 
-extern TransactionId PrescanPreparedTransactions(TransactionId **xids_p,
+extern void CleanupPreparedTransactionsOnPITR(void);
+extern void GetPreparedTransactions(TransactionId **xids_p,
 							int *nxids_p);
-extern void StandbyRecoverPreparedTransactions(bool overwriteOK);
-extern void RecoverPreparedTransactions(void);
+extern void RecoverPreparedFromFiles(void);
+extern void RecoverPreparedFromXLOG(XLogReaderState *record);
 
 extern void RecreateTwoPhaseFile(TransactionId xid, void *content, int len);
 extern void RemoveTwoPhaseFile(TransactionId xid, bool giveWarning);
@@ -56,4 +58,5 @@ extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void XlogRedoFinishPrepared(TransactionId xid, bool isCommit);
 #endif   /* TWOPHASE_H */
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 0bc41ab..c42e05e 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -50,14 +50,7 @@ typedef struct CheckPoint
 										 * timestamp */
 	TransactionId newestCommitTsXid;	/* newest Xid with valid commit
 										 * timestamp */
-
-	/*
-	 * Oldest XID still running. This is only needed to initialize hot standby
-	 * mode from an online checkpoint, so we only bother calculating this for
-	 * online checkpoints and only when wal_level is replica. Otherwise it's
-	 * set to InvalidTransactionId.
-	 */
-	TransactionId oldestActiveXid;
+	TransactionId oldestActiveXid;		/* Oldest XID still running. */
 } CheckPoint;
 
 /* XLOG info values for XLOG rmgr */
diff --git a/src/test/recovery/t/008_twophase.pl b/src/test/recovery/t/008_twophase.pl
new file mode 100644
index 0000000..7ada751
--- /dev/null
+++ b/src/test/recovery/t/008_twophase.pl
@@ -0,0 +1,314 @@
+# Tests dedicated to two-phase commit in recovery
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 13;
+
+# Setup master node
+my $node_master = get_new_node("master");
+$node_master->init(allows_streaming => 1);
+$node_master->append_conf('postgresql.conf', qq(
+	max_prepared_transactions = 10
+));
+$node_master->start;
+$node_master->backup('master_backup');
+$node_master->psql('postgres', "create table t(id int)");
+
+# Setup master node
+my $node_slave = get_new_node('slave');
+$node_slave->init_from_backup($node_master, 'master_backup', has_streaming => 1);
+$node_slave->start;
+
+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+	synchronous_standby_names = '*'
+));
+$node_master->psql('postgres', "select pg_reload_conf()");
+
+my $psql_out = '';
+my $psql_rc = '';
+
+###############################################################################
+# Check that we can commit and abort tx after soft restart.
+# Here checkpoint happens before shutdown and no WAL replay will occur at next
+# startup. In this case postgres re-create shared-memory state from twophase
+# files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	savepoint s1;
+	insert into t values (143);
+	prepare transaction 'y';");
+$node_master->stop;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared transaction after restart.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared transaction after restart.');
+
+###############################################################################
+# Check that we can commit and abort after hard restart.
+# At next startup, WAL replay will re-create shared memory state for prepared
+# transaction using dedicated WAL records.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	savepoint s1;
+	insert into t values (143);
+	prepare transaction 'y';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared tx after teardown.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared transaction after teardown.');
+
+###############################################################################
+# Check that WAL replay can handle several transactions with same name GID.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Replay several transactions with same GID.');
+
+###############################################################################
+# Check that WAL replay cleans up its shared memory state and releases locks
+# while replaying transaction commits.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_master->teardown_node;
+$node_master->start;
+$psql_rc = $node_master->psql('postgres', "begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	-- This prepare can fail due to conflicting GID or locks conflicts if
+	-- replay did not fully cleanup its state on previous commit.
+	prepare transaction 'x';");
+is($psql_rc, '0', "Cleanup of shared memory state for 2PC commit");
+
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that WAL replay will cleanup its shared memory state on running slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;",
+	  stdout => \$psql_out);
+is($psql_out, '0',
+   "Cleanup of shared memory state on running standby without checkpoint.");
+
+###############################################################################
+# Same as in previous case, but let's force checkpoint on slave between
+# prepare and commit to use on-disk twophase files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_slave->psql('postgres', "checkpoint;");
+$node_master->psql('postgres', "commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;",
+	  stdout => \$psql_out);
+is($psql_out, '0',
+   "Cleanup of shared memory state on running standby after checkpoint.");
+
+###############################################################################
+# Check that prepared transactions can be committed on promoted slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$psql_rc = $node_slave->psql('postgres', "commit prepared 'x';");
+is($psql_rc, '0', "Restore of prepared transaction on promoted slave.");
+
+# change roles
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+
+###############################################################################
+# Check that prepared transactions are replayed after soft restart of standby
+# while master is down. Since standby knows that master is down it uses
+# different code path on start to be sure that the status of transactions is
+# consistent.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->stop;
+$node_slave->restart;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '1',
+   "Restore prepared transactions from files with master down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that prepared transactions are correctly replayed after slave hard
+# restart while master is down.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (242);
+	savepoint s1;
+	insert into t values (243);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->teardown_node;
+$node_slave->start;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres',
+	  "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '1',
+   "Restore prepared transactions from records with master down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres', "commit prepared 'x'");
+
+
+###############################################################################
+# Check for a lock confcict between prepared tx with DDL inside and replay of
+# XLOG_STANDBY_LOCK wal record.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	create table t2(id int);
+	savepoint s1;
+	insert into t2 values (42);
+	prepare transaction 'x';
+	-- checkpoint will issue XLOG_STANDBY_LOCK that can conflict with lock
+	-- held by 'create table' statement
+	checkpoint;
+	commit prepared 'x';");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '0', "Replay prepared transaction with DDL.");
+
+
+###############################################################################
+# Check that replay will correctly set SUBTRANS and properly andvance nextXid
+# so it won't conflict with savepoint xids.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	delete from t;
+	insert into t values (43);
+	savepoint s1;
+	insert into t values (43);
+	savepoint s2;
+	insert into t values (43);
+	savepoint s3;
+	insert into t values (43);
+	savepoint s4;
+	insert into t values (43);
+	savepoint s5;
+	insert into t values (43);
+	prepare transaction 'x';
+	checkpoint;");
+
+$node_master->stop;
+$node_master->start;
+$node_master->psql('postgres', "
+	-- here we can get xid of previous savepoint if nextXid
+	-- wasn't properly advanced
+	begin;
+	insert into t values (142);
+	abort;
+	commit prepared 'x';");
+
+$node_master->psql('postgres', "select count(*) from t",
+	  stdout => \$psql_out);
+is($psql_out, '6', "Check nextXid handling for prepared subtransactions");
\ No newline at end of file
#91Michael Paquier
michael.paquier@gmail.com
In reply to: Stas Kelvich (#90)
Re: Speedup twophase transactions

On Sat, Sep 17, 2016 at 2:45 AM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Looking through old version i’ve noted few things that were bothering me:

* In case of crash replay PREPARE redo accesses SUBTRANS, but StartupSUBTRANS() isn’t called yet
in StartupXLOG().
* Several functions in twophase.c have to walk through both PGPROC and pg_twophase directory.

Now I slightly changed order of things in StartupXLOG() so twophase state loaded from from file and
StartupSUBTRANS called before actual recovery starts. So in all other functions we can be sure that
file were already loaded in memory.

Also since 2pc transactions now are dumped to files only on checkpoint, we can get rid of
PrescanPreparedTransactions() — all necessary info can reside in checkpoint itself. I’ve changed
behaviour of oldestActiveXid write in checkpoint and that’s probably discussable topic, but ISTM
that simplifies a lot recovery logic in both twophase.c and xlog.c. More comments on that in patch itself.

Finally I am back on this one..

First be more careful about typos in the comments of your code. Just
reading through the patch I found quite a couple of places that needed
to be fixed. cosequent, unavalable, SUBTRANCE are some examples among
many other things..

+   StartupCLOG();
+   StartupSUBTRANS(checkPoint.oldestActiveXid);
+   RecoverPreparedFromFiles();
[...]
            /*
-            * Startup commit log and subtrans only.  MultiXact and commit
-            * timestamp have already been started up and other SLRUs are not
-            * maintained during recovery and need not be started yet.
-            */
-           StartupCLOG();
-           StartupSUBTRANS(oldestActiveXID);
Something is definitely wrong in this patch if we begin to do that.
There is no need to move those two calls normally, and I think that we
had better continue to do that only for hot standbys just to improve
2PC handling...

CleanupPreparedTransactionsOnPITR() cleans up pg_twophase, but isn't
it necessary to do something as well for what is in memory?

I have been thinking about this patch quite a bit, and the approach
taken looks really over-complicated to me. Basically what we are
trying to do here is to reuse as much as possible code paths that are
being used by non-recovery code paths during recovery, and then try to
adapt a bunch of things like SUBTRANS structures, CLOG, XID handling
and PGXACT things in sync to handle the 2PC information in memory
correctly. I am getting to think that this is *really* bug-prone in
the future and really going to make maintenance hard.

Taking one step back, and I know that I am a noisy one, but looking at
this patch is making me aware of the fact that it is trying more and
more to patch things around the more we dig into issues, and I'd
suspect that trying to adapt for example sub-transaction and clog
handling is just the tip of the iceberg and that there are more things
that need to be taken care of if we continue to move on with this
approach. Coming to the point: why don't we simplify things? In short:
- Just use a static list and allocate a chunk of shared memory as
needed. DSM could come into play here to adapt to the size of a 2PC
status file, this way there is no need to allocate a chunk of memory
with an upper bound. Still we could use an hardcoded upper-bound of
say 2k with max_prepared_transactions, and just write the 2PC status
file to disk if its size is higher than what can stick into memory.
- save 2PC information in memory instead of writing it to a 2PC file
when XLOG_XACT_PREPARE shows up.
- flush information to disk when there is a valid restart point in
RecoveryRestartPoint(). We would need as well to tweak
PrescanPreparedTransactions accordingly than everything we are trying
to do here.
That would be way more simple than what's proposed here, and we'd
still keep all the performance benefits.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#92Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#91)
Re: Speedup twophase transactions

On 20 Sep 2016, at 09:40, Michael Paquier <michael.paquier@gmail.com> wrote:

Thanks for digging into this.

+   StartupCLOG();
+   StartupSUBTRANS(checkPoint.oldestActiveXid);
+   RecoverPreparedFromFiles();
[...]
/*
-            * Startup commit log and subtrans only.  MultiXact and commit
-            * timestamp have already been started up and other SLRUs are not
-            * maintained during recovery and need not be started yet.
-            */
-           StartupCLOG();
-           StartupSUBTRANS(oldestActiveXID);
Something is definitely wrong in this patch if we begin to do that.

Putting that before actual WAL replay is just following historical order of events.
Prepared files are pieces of WAL that happened before checkpoint, so ISTM
there is no conceptual problem in restoring their state before replay.

Moreover I think that this approach is better then oldest one.
There is kind of circular dependency between StartupSUBTRANS() and restoring
old prepared transaction: StartupSUBTRANS requires oldestActiveXID, but you
can get it only after recovering 2pc files, but that recovery requires working SUBTRANS.

In old code that was resolved by two passes through 2pc files: first one finds
oldestActiveXmin but doesn’t restore their state, then subtrans was started, and
only after that RecoverPreparedTransactions() is called. And that logic was repeated
three times in xlog.c with help of following functions:
PrescanPreparedTransactions() — 3 calls
StandbyRecoverPreparedTransactions() — 2 calls
RecoverPreparedTransactions() — 1 call

Now, since we know that 2pc files are written only on checkpoint we can boil all
that cases down to one: get oldestActiveXID from checkpoint and restore it before
WAL replay. So only one call of RecoverPreparedFromFiles() and one call of
CleanupPreparedTransactionsOnPITR() in case of PITR. And each of them does
only what stated on function name without side-effects like advancing nextXid in
previous versions.

So, to summarise, i think keeping old interfaces here is bigger evil because in each of
mentioned functions we will need to deal with both memory and file 2pc states.

There is no need to move those two calls normally, and I think that we
had better continue to do that only for hot standbys just to improve
2PC handling…

We can simulate old interface, it’s not a big problem. But that will complicate 2pc code and
keep useless code in xlog.c because it was written in assumption that 2pc file can be created
before checkpoint and now it isn’t true.

CleanupPreparedTransactionsOnPITR() cleans up pg_twophase, but isn't
it necessary to do something as well for what is in memory?

Yes, that is necessary. Thanks, will fix. And probably add prepared tx to PITR test.

I have been thinking about this patch quite a bit, and the approach
taken looks really over-complicated to me. Basically what we are
trying to do here is to reuse as much as possible code paths that are
being used by non-recovery code paths during recovery, and then try to
adapt a bunch of things like SUBTRANS structures, CLOG, XID handling
and PGXACT things in sync to handle the 2PC information in memory
correctly. I am getting to think that this is *really* bug-prone in
the future and really going to make maintenance hard.

With this patch we are reusing one infrastructure for normal work, recovery and replay.
I don’t think that we will win a lot reliability if we split that to a different code paths.

Taking one step back, and I know that I am a noisy one, but looking at
this patch is making me aware of the fact that it is trying more and
more to patch things around the more we dig into issues, and I'd
suspect that trying to adapt for example sub-transaction and clog
handling is just the tip of the iceberg and that there are more things
that need to be taken care of if we continue to move on with this
approach.

Again, it isn’t patching around to fix issues, it’s totally possible to keep old interface.
However it’s possible that current approach is wrong because of some aspect
that i didn’t think of, but now I don’t see good counterarguments.

Coming to the point: why don't we simplify things? In short:

- Just use a static list and allocate a chunk of shared memory as
needed. DSM could come into play here to adapt to the size of a 2PC
status file, this way there is no need to allocate a chunk of memory
with an upper bound. Still we could use an hardcoded upper-bound of
say 2k with max_prepared_transactions, and just write the 2PC status
file to disk if its size is higher than what can stick into memory.
- save 2PC information in memory instead of writing it to a 2PC file
when XLOG_XACT_PREPARE shows up.
- flush information to disk when there is a valid restart point in
RecoveryRestartPoint(). We would need as well to tweak
PrescanPreparedTransactions accordingly than everything we are trying
to do here.
That would be way more simple than what's proposed here, and we'd
still keep all the performance benefits.

So file arrives to replica through WAL and instead of directly reading it you suggest
to copy it do DSM if it is small enough, and to filesystem if not. Probably that will
allow us to stay only around reading/writing files, but:

* That will be measurably slower than proposed approach because of unnecessary
copying between WAL and DSM. Not to mention prepare records of arbitrary length.
* Different behaviour on replica and master — less tested code for replica.
* Almost the same behaviour can be achieved by delaying fsync on 2pc files till
checkpoint.

But if reword you comment in a way that it is better to avoid replaying prepare record at all,
like it happens now in master, then I see the point. And that is possible even without DSM, it possible
to allocate static sized array storing some info about tx, whether it is in the WAL or in file, xid, gid.
Some sort of PGXACT doppelganger only for replay purposes instead of using normal one.

So taking into account my comments what do you think? Should we keep current approach
or try to avoid replaying prepare records at all?

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#93Michael Paquier
michael.paquier@gmail.com
In reply to: Stas Kelvich (#92)
Re: Speedup twophase transactions

On Tue, Sep 20, 2016 at 11:13 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

On 20 Sep 2016, at 09:40, Michael Paquier <michael.paquier@gmail.com> wrote:
+   StartupCLOG();
+   StartupSUBTRANS(checkPoint.oldestActiveXid);
+   RecoverPreparedFromFiles();
[...]
/*
-            * Startup commit log and subtrans only.  MultiXact and commit
-            * timestamp have already been started up and other SLRUs are not
-            * maintained during recovery and need not be started yet.
-            */
-           StartupCLOG();
-           StartupSUBTRANS(oldestActiveXID);
Something is definitely wrong in this patch if we begin to do that.

Putting that before actual WAL replay is just following historical order of events.
Prepared files are pieces of WAL that happened before checkpoint, so ISTM
there is no conceptual problem in restoring their state before replay.

For wal_level = minimal there is no need to call that to begin with..
And I'd think that it is better to continue with things the same way.

I have been thinking about this patch quite a bit, and the approach
taken looks really over-complicated to me. Basically what we are
trying to do here is to reuse as much as possible code paths that are
being used by non-recovery code paths during recovery, and then try to
adapt a bunch of things like SUBTRANS structures, CLOG, XID handling
and PGXACT things in sync to handle the 2PC information in memory
correctly. I am getting to think that this is *really* bug-prone in
the future and really going to make maintenance hard.

With this patch we are reusing one infrastructure for normal work, recovery and replay.

The mix between normal work and recovery is the scary part. Normal
work and recovery are entirely two different things.

Taking one step back, and I know that I am a noisy one, but looking at
this patch is making me aware of the fact that it is trying more and
more to patch things around the more we dig into issues, and I'd
suspect that trying to adapt for example sub-transaction and clog
handling is just the tip of the iceberg and that there are more things
that need to be taken care of if we continue to move on with this
approach.

Again, it isn’t patching around to fix issues, it’s totally possible to keep old interface.
However it’s possible that current approach is wrong because of some aspect
that i didn’t think of, but now I don’t see good counterarguments.

Any missing points could be costly here. If we have a way to make
things with the same performance

Coming to the point: why don't we simplify things? In short:

- Just use a static list and allocate a chunk of shared memory as
needed. DSM could come into play here to adapt to the size of a 2PC
status file, this way there is no need to allocate a chunk of memory
with an upper bound. Still we could use an hardcoded upper-bound of
say 2k with max_prepared_transactions, and just write the 2PC status
file to disk if its size is higher than what can stick into memory.
- save 2PC information in memory instead of writing it to a 2PC file
when XLOG_XACT_PREPARE shows up.
- flush information to disk when there is a valid restart point in
RecoveryRestartPoint(). We would need as well to tweak
PrescanPreparedTransactions accordingly than everything we are trying
to do here.
That would be way more simple than what's proposed here, and we'd
still keep all the performance benefits.

So file arrives to replica through WAL and instead of directly reading it you suggest
to copy it do DSM if it is small enough, and to filesystem if not. Probably that will
allow us to stay only around reading/writing files, but:

* That will be measurably slower than proposed approach because of unnecessary
copying between WAL and DSM. Not to mention prepare records of arbitrary length.
* Different behaviour on replica and master — less tested code for replica.

Well, the existing code on HEAD is battery-tested as well. This
reduces the likeliness of bugs at replay with new features.

* Almost the same behaviour can be achieved by delaying fsync on 2pc files till
checkpoint.

Oh, that's an idea here. It may be interesting to see if this meets
your performance goals... And that could result in a far smaller
patch.

But if reword you comment in a way that it is better to avoid replaying prepare record at all,
like it happens now in master, then I see the point. And that is possible even without DSM, it possible
to allocate static sized array storing some info about tx, whether it is in the WAL or in file, xid, gid.
Some sort of PGXACT doppelganger only for replay purposes instead of using normal one.

That's exactly what I meant. The easiest approach is just to allocate
a bunk of shared memory made of 2PC_STATUS_SIZE * max_prepared_xacts
with 2PC_STATUS_SIZE set up at an arbitrary size that we find
appropriate to store the information of the file. DSM may be useful to
take care of the case where the status file is bigger than one slot,
but with a sufficiently wise upper bound, we can get away without it.

So taking into account my comments what do you think? Should we keep current approach
or try to avoid replaying prepare records at all?

I'd really like to give a try to the idea you just mentioned, aka
delay the fsync of the 2PC files from RecreateTwoPhaseFile to
CreateRestartPoint, or get things into memory.. I could write one, or
both of those patches. I don't mind.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#94Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#93)
Re: Speedup twophase transactions

On 21 Sep 2016, at 10:32, Michael Paquier <michael.paquier@gmail.com> wrote:

On Tue, Sep 20, 2016 at 11:13 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Putting that before actual WAL replay is just following historical order of events.
Prepared files are pieces of WAL that happened before checkpoint, so ISTM
there is no conceptual problem in restoring their state before replay.

For wal_level = minimal there is no need to call that to begin with..
And I'd think that it is better to continue with things the same way.

Hm, why?

With this patch we are reusing one infrastructure for normal work, recovery and replay.

The mix between normal work and recovery is the scary part. Normal
work and recovery are entirely two different things.

Okay, agreed. Commit procedure that checks whether recovery is active or not
is quite hacky solution.

So taking into account my comments what do you think? Should we keep current approach
or try to avoid replaying prepare records at all?

I'd really like to give a try to the idea you just mentioned, aka
delay the fsync of the 2PC files from RecreateTwoPhaseFile to
CreateRestartPoint, or get things into memory.. I could write one, or
both of those patches. I don't mind.

I’m not giving up yet, i’ll write them) I still have in mind several other patches to 2pc handling in
postgres during this release cycle — logical decoding and partitioned hash instead of
TwoPhaseState list.

My bet that relative speed of that patches will depend on used filesystem. Like it was with the
first patch in that mail thread it is totally possible sometimes to hit filesystem limits on file
creation speed. Otherwise both approaches should be more or less equal, i suppose.

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#95Michael Paquier
michael.paquier@gmail.com
In reply to: Stas Kelvich (#94)
Re: Speedup twophase transactions

On Thu, Sep 22, 2016 at 12:30 AM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

I’m not giving up yet, i’ll write them) I still have in mind several other patches to 2pc handling in
postgres during this release cycle — logical decoding and partitioned hash instead of
TwoPhaseState list.

My bet that relative speed of that patches will depend on used filesystem. Like it was with the
first patch in that mail thread it is totally possible sometimes to hit filesystem limits on file
creation speed. Otherwise both approaches should be more or less equal, i suppose.

OK. I am marking this patch as returned with feedback then. Looking
forward to seeing the next investigations.. At least this review has
taught us one thing or two.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#96Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#95)
1 attachment(s)
Re: Speedup twophase transactions

On 27 Sep 2016, at 03:30, Michael Paquier <michael.paquier@gmail.com> wrote:

OK. I am marking this patch as returned with feedback then. Looking
forward to seeing the next investigations.. At least this review has
taught us one thing or two.

So, here is brand new implementation of the same thing.

Now instead of creating pgproc entry for prepared transaction during recovery,
I just store recptr/xid correspondence in separate 2L-list and deleting entries in that
list if redo process faced commit/abort. In case of checkpoint or end of recovery
transactions remaining in that list are dumped to files in pg_twophase.

Seems that current approach is way more simpler and patch has two times less
LOCs then previous one.

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Attachments:

twophase_recovery_list.diffapplication/octet-stream; name=twophase_recovery_list.diff; x-unix-mode=0644Download
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 5415604..fb69646 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,8 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *		* Simplified version of the same scenario happens during recovery and
+ *		  replication. See comments to KnownPreparedXact structure.
  *
  *-------------------------------------------------------------------------
  */
@@ -181,6 +181,35 @@ static GlobalTransaction MyLockedGxact = NULL;
 
 static bool twophaseExitRegistered = false;
 
+/*
+ * During replay and replication KnownPreparedList holds info about active prepared
+ * transactions that weren't moved to files yet. We will need that info by the end of
+ * recovery (including promote) to restore memory state of that transactions.
+ *
+ * Naive approach here is to move each PREPARE record to disk, fsync it and don't have
+ * that list at all, but that provokes a lot of unnecessary fsyncs on small files
+ * causing replica to be slower than master.
+ *
+ * Replay of twophase records happens by the following rules:
+ *		* On PREPARE redo KnownPreparedAdd() is called to add that transaction to
+ *		  KnownPreparedList and no more actions are taken.
+ *		* On checkpoint redo we iterate through KnownPreparedList and move all prepare
+ *		  records that behind redo_horizon to files and deleting them from list.
+ *		* On COMMIT/ABORT we delete file or entry in KnownPreparedList.
+ *		* At the end of recovery we move all known prepared transactions to disk
+ *		  to allow RecoverPreparedTransactions/StandbyRecoverPreparedTransactions
+ *		  do their work.
+ */
+typedef struct KnownPreparedXact
+{
+	TransactionId	xid;
+	XLogRecPtr		prepare_start_lsn;
+	XLogRecPtr		prepare_end_lsn;
+	dlist_node		list_node;
+} KnownPreparedXact;
+
+static dlist_head KnownPreparedList = DLIST_STATIC_INIT(KnownPreparedList);
+
 static void RecordTransactionCommitPrepared(TransactionId xid,
 								int nchildren,
 								TransactionId *children,
@@ -1241,9 +1270,9 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
  * Reads 2PC data from xlog. During checkpoint this data will be moved to
  * twophase files and ReadTwoPhaseFile should be used instead.
  *
- * Note clearly that this function accesses WAL during normal operation, similarly
- * to the way WALSender or Logical Decoding would do. It does not run during
- * crash recovery or standby processing.
+ * Note clearly that this function can access WAL during normal operation, similarly
+ * to the way WALSender or Logical Decoding would do.
+ *
  */
 static void
 XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
@@ -1252,8 +1281,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1691,6 +1718,15 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 	int			nxids = 0;
 	int			allocsize = 0;
 
+	/*
+	 * Move prepared transactions from KnownPreparedList to files, if any.
+	 * It is possible to skip that step and teach subsequent code about
+	 * KnownPreparedList, but whole PrescanPreparedTransactions() happens
+	 * once during end of recovery or promote, so probably it isn't worth
+	 * complications.
+	 */
+	KnownPreparedRecreateFiles(InvalidXLogRecPtr);
+
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
 	{
@@ -2162,3 +2198,111 @@ RecordTransactionAbortPrepared(TransactionId xid,
 	 */
 	SyncRepWaitForLSN(recptr, false);
 }
+
+/*
+ * KnownPreparedAdd.
+ *
+ * Store correspondence of start/end lsn and xid in KnownPreparedList.
+ * This is called during redo of prepare record to have list of prepared
+ * transactions that aren't yet moved to 2PC files by the end of recovery.
+ */
+void
+KnownPreparedAdd(XLogReaderState *record)
+{
+	KnownPreparedXact *xact;
+	TwoPhaseFileHeader *hdr = (TwoPhaseFileHeader *) XLogRecGetData(record);
+
+	Assert(RecoveryInProgress());
+
+	xact = (KnownPreparedXact *) palloc(sizeof(KnownPreparedXact));
+	xact->xid = hdr->xid;
+	xact->prepare_start_lsn = record->ReadRecPtr;
+	xact->prepare_end_lsn = record->EndRecPtr;
+
+	dlist_push_tail(&KnownPreparedList, &xact->list_node);
+}
+
+/*
+ * KnownPreparedRemoveByXid
+ *
+ * Forget about prepared transaction. Called during commit/abort redo.
+ */
+void
+KnownPreparedRemoveByXid(TransactionId xid)
+{
+	dlist_mutable_iter miter;
+
+	Assert(RecoveryInProgress());
+
+	dlist_foreach_modify(miter, &KnownPreparedList)
+	{
+		KnownPreparedXact   *xact = dlist_container(KnownPreparedXact,
+														list_node, miter.cur);
+
+		if (xact->xid == xid)
+		{
+			dlist_delete(miter.cur);
+			/*
+			 * Since we found entry in KnownPreparedList we know that file isn't
+			 * on disk yet and we can end up here.
+			 */
+			return;
+		}
+	}
+
+	/*
+	 * Here we know that file should be moved to disk. But aborting recovery because
+	 * of absence of unnecessary file doesn't seems to be a good idea, so call remove
+	 * with giveWarning=false.
+	 */
+	RemoveTwoPhaseFile(xid, false);
+}
+
+/*
+ * KnownPreparedRecreateFiles
+ *
+ * Moves prepare records from WAL to files. Called during checkpoint replay
+ * or PrescanPreparedTransactions.
+ *
+ * redo_horizon = InvalidXLogRecPtr indicates that all transactions from
+ *		KnownPreparedList should be moved to disk.
+ */
+void
+KnownPreparedRecreateFiles(XLogRecPtr redo_horizon)
+{
+	dlist_mutable_iter miter;
+	int			serialized_xacts = 0;
+
+	Assert(RecoveryInProgress());
+
+	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_START();
+
+	dlist_foreach_modify(miter, &KnownPreparedList)
+	{
+		KnownPreparedXact   *xact = dlist_container(KnownPreparedXact,
+														list_node, miter.cur);
+
+		if (xact->prepare_end_lsn <= redo_horizon || redo_horizon == InvalidXLogRecPtr)
+		{
+			char	   *buf;
+			int			len;
+
+			XlogReadTwoPhaseData(xact->prepare_start_lsn, &buf, &len);
+			RecreateTwoPhaseFile(xact->xid, buf, len);
+			pfree(buf);
+			dlist_delete(miter.cur);
+			serialized_xacts++;
+		}
+	}
+
+	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_DONE();
+
+	if (log_checkpoints && serialized_xacts > 0)
+		ereport(LOG,
+				(errmsg_plural("%u two-phase state file was written "
+							   "for long-running prepared transactions",
+							   "%u two-phase state files were written "
+							   "for long-running prepared transactions",
+							   serialized_xacts,
+							   serialized_xacts)));
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index d643216..b3e0238 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -5604,7 +5604,9 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete KnownPrepared entry or 2PC file. */
+			KnownPreparedRemoveByXid(parsed.twophase_xid);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5624,14 +5626,20 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete KnownPrepared entry or 2PC file. */
+			KnownPreparedRemoveByXid(parsed.twophase_xid);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		/*
+		 * If that transaction will not be committed by the end of recovery then we
+		 * will need 2PC file (the record contents is exactly the 2PC file) to be able
+		 * to commit that later.
+		 * For now store xid and pointers to that record in KnownPreparedList.
+		 */
+		KnownPreparedAdd(record);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 084401d..9ac1fd7 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -9488,6 +9488,7 @@ xlog_redo(XLogReaderState *record)
 					(errmsg("unexpected timeline ID %u (should be %u) in checkpoint record",
 							checkPoint.ThisTimeLineID, ThisTimeLineID)));
 
+		KnownPreparedRecreateFiles(checkPoint.redo);
 		RecoveryRestartPoint(&checkPoint);
 	}
 	else if (info == XLOG_END_OF_RECOVERY)
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b7ce0c6..23be08f 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -15,6 +15,7 @@
 #define TWOPHASE_H
 
 #include "access/xlogdefs.h"
+#include "access/xlogreader.h"
 #include "datatype/timestamp.h"
 #include "storage/lock.h"
 
@@ -56,4 +57,8 @@ extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void KnownPreparedAdd(XLogReaderState *record);
+extern void KnownPreparedRemoveByXid(TransactionId xid);
+extern void KnownPreparedRecreateFiles(XLogRecPtr redo_horizon);
+
 #endif   /* TWOPHASE_H */
diff --git a/src/test/recovery/t/009_twophase.pl b/src/test/recovery/t/009_twophase.pl
new file mode 100644
index 0000000..27bccb3
--- /dev/null
+++ b/src/test/recovery/t/009_twophase.pl
@@ -0,0 +1,315 @@
+# Tests dedicated to two-phase commit in recovery
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 13;
+
+# Setup master node
+my $node_master = get_new_node("master");
+$node_master->init(allows_streaming => 1);
+$node_master->append_conf('postgresql.conf', qq(
+	max_prepared_transactions = 10
+	log_checkpoints = true
+));
+$node_master->start;
+$node_master->backup('master_backup');
+$node_master->psql('postgres', "create table t(id int)");
+
+# Setup master node
+my $node_slave = get_new_node('slave');
+$node_slave->init_from_backup($node_master, 'master_backup', has_streaming => 1);
+$node_slave->start;
+
+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+	synchronous_standby_names = '*'
+));
+$node_master->psql('postgres', "select pg_reload_conf()");
+
+my $psql_out = '';
+my $psql_rc = '';
+
+###############################################################################
+# Check that we can commit and abort tx after soft restart.
+# Here checkpoint happens before shutdown and no WAL replay will occur at next
+# startup. In this case postgres re-create shared-memory state from twophase
+# files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	savepoint s1;
+	insert into t values (143);
+	prepare transaction 'y';");
+$node_master->stop;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared transaction after restart.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared transaction after restart.');
+
+###############################################################################
+# Check that we can commit and abort after hard restart.
+# At next startup, WAL replay will re-create shared memory state for prepared
+# transaction using dedicated WAL records.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	savepoint s1;
+	insert into t values (143);
+	prepare transaction 'y';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared tx after teardown.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared transaction after teardown.');
+
+###############################################################################
+# Check that WAL replay can handle several transactions with same name GID.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Replay several transactions with same GID.');
+
+###############################################################################
+# Check that WAL replay cleans up its shared memory state and releases locks
+# while replaying transaction commits.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_master->teardown_node;
+$node_master->start;
+$psql_rc = $node_master->psql('postgres', "begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	-- This prepare can fail due to conflicting GID or locks conflicts if
+	-- replay did not fully cleanup its state on previous commit.
+	prepare transaction 'x';");
+is($psql_rc, '0', "Cleanup of shared memory state for 2PC commit");
+
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that WAL replay will cleanup its shared memory state on running slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;",
+	  stdout => \$psql_out);
+is($psql_out, '0',
+   "Cleanup of shared memory state on running standby without checkpoint.");
+
+###############################################################################
+# Same as in previous case, but let's force checkpoint on slave between
+# prepare and commit to use on-disk twophase files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_slave->psql('postgres', "checkpoint;");
+$node_master->psql('postgres', "commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;",
+	  stdout => \$psql_out);
+is($psql_out, '0',
+   "Cleanup of shared memory state on running standby after checkpoint.");
+
+###############################################################################
+# Check that prepared transactions can be committed on promoted slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$psql_rc = $node_slave->psql('postgres', "commit prepared 'x';");
+is($psql_rc, '0', "Restore of prepared transaction on promoted slave.");
+
+# change roles
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+
+###############################################################################
+# Check that prepared transactions are replayed after soft restart of standby
+# while master is down. Since standby knows that master is down it uses
+# different code path on start to be sure that the status of transactions is
+# consistent.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->stop;
+$node_slave->restart;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '1',
+   "Restore prepared transactions from files with master down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that prepared transactions are correctly replayed after slave hard
+# restart while master is down.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (242);
+	savepoint s1;
+	insert into t values (243);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->teardown_node;
+$node_slave->start;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres',
+	  "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '1',
+   "Restore prepared transactions from records with master down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres', "commit prepared 'x'");
+
+
+###############################################################################
+# Check for a lock conflict between prepared tx with DDL inside and replay of
+# XLOG_STANDBY_LOCK wal record.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	create table t2(id int);
+	savepoint s1;
+	insert into t2 values (42);
+	prepare transaction 'x';
+	-- checkpoint will issue XLOG_STANDBY_LOCK that can conflict with lock
+	-- held by 'create table' statement
+	checkpoint;
+	commit prepared 'x';");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '0', "Replay prepared transaction with DDL.");
+
+
+###############################################################################
+# Check that replay will correctly set SUBTRANS and properly advance nextXid
+# so it won't conflict with savepoint xids.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	delete from t;
+	insert into t values (43);
+	savepoint s1;
+	insert into t values (43);
+	savepoint s2;
+	insert into t values (43);
+	savepoint s3;
+	insert into t values (43);
+	savepoint s4;
+	insert into t values (43);
+	savepoint s5;
+	insert into t values (43);
+	prepare transaction 'x';
+	checkpoint;");
+
+$node_master->stop;
+$node_master->start;
+$node_master->psql('postgres', "
+	-- here we can get xid of previous savepoint if nextXid
+	-- wasn't properly advanced
+	begin;
+	insert into t values (142);
+	abort;
+	commit prepared 'x';");
+
+$node_master->psql('postgres', "select count(*) from t",
+	  stdout => \$psql_out);
+is($psql_out, '6', "Check nextXid handling for prepared subtransactions");
#97Bruce Momjian
bruce@momjian.us
In reply to: Stas Kelvich (#96)
Re: Speedup twophase transactions

On Fri, Dec 16, 2016 at 02:00:46PM +0300, Stas Kelvich wrote:

On 27 Sep 2016, at 03:30, Michael Paquier <michael.paquier@gmail.com> wrote:

OK. I am marking this patch as returned with feedback then. Looking
forward to seeing the next investigations.. At least this review has
taught us one thing or two.

So, here is brand new implementation of the same thing.

Now instead of creating pgproc entry for prepared transaction during recovery,
I just store recptr/xid correspondence in separate 2L-list and deleting entries in that
list if redo process faced commit/abort. In case of checkpoint or end of recovery
transactions remaining in that list are dumped to files in pg_twophase.

Seems that current approach is way more simpler and patch has two times less
LOCs then previous one.

Uh, did you mean to attached patch here?

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#98Michael Paquier
michael.paquier@gmail.com
In reply to: Bruce Momjian (#97)
1 attachment(s)
Re: Speedup twophase transactions

On Sun, Dec 18, 2016 at 6:42 AM, Bruce Momjian <bruce@momjian.us> wrote:

Uh, did you mean to attached patch here?

Strange. I can confirm that I have received the patch as attached, but
it is not on the archives.
--
Michael

Attachments:

twophase_recovery_list.difftext/plain; charset=US-ASCII; name=twophase_recovery_list.diffDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 5415604..fb69646 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,8 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *		* Simplified version of the same scenario happens during recovery and
+ *		  replication. See comments to KnownPreparedXact structure.
  *
  *-------------------------------------------------------------------------
  */
@@ -181,6 +181,35 @@ static GlobalTransaction MyLockedGxact = NULL;
 
 static bool twophaseExitRegistered = false;
 
+/*
+ * During replay and replication KnownPreparedList holds info about active prepared
+ * transactions that weren't moved to files yet. We will need that info by the end of
+ * recovery (including promote) to restore memory state of that transactions.
+ *
+ * Naive approach here is to move each PREPARE record to disk, fsync it and don't have
+ * that list at all, but that provokes a lot of unnecessary fsyncs on small files
+ * causing replica to be slower than master.
+ *
+ * Replay of twophase records happens by the following rules:
+ *		* On PREPARE redo KnownPreparedAdd() is called to add that transaction to
+ *		  KnownPreparedList and no more actions are taken.
+ *		* On checkpoint redo we iterate through KnownPreparedList and move all prepare
+ *		  records that behind redo_horizon to files and deleting them from list.
+ *		* On COMMIT/ABORT we delete file or entry in KnownPreparedList.
+ *		* At the end of recovery we move all known prepared transactions to disk
+ *		  to allow RecoverPreparedTransactions/StandbyRecoverPreparedTransactions
+ *		  do their work.
+ */
+typedef struct KnownPreparedXact
+{
+	TransactionId	xid;
+	XLogRecPtr		prepare_start_lsn;
+	XLogRecPtr		prepare_end_lsn;
+	dlist_node		list_node;
+} KnownPreparedXact;
+
+static dlist_head KnownPreparedList = DLIST_STATIC_INIT(KnownPreparedList);
+
 static void RecordTransactionCommitPrepared(TransactionId xid,
 								int nchildren,
 								TransactionId *children,
@@ -1241,9 +1270,9 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
  * Reads 2PC data from xlog. During checkpoint this data will be moved to
  * twophase files and ReadTwoPhaseFile should be used instead.
  *
- * Note clearly that this function accesses WAL during normal operation, similarly
- * to the way WALSender or Logical Decoding would do. It does not run during
- * crash recovery or standby processing.
+ * Note clearly that this function can access WAL during normal operation, similarly
+ * to the way WALSender or Logical Decoding would do.
+ *
  */
 static void
 XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
@@ -1252,8 +1281,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1691,6 +1718,15 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 	int			nxids = 0;
 	int			allocsize = 0;
 
+	/*
+	 * Move prepared transactions from KnownPreparedList to files, if any.
+	 * It is possible to skip that step and teach subsequent code about
+	 * KnownPreparedList, but whole PrescanPreparedTransactions() happens
+	 * once during end of recovery or promote, so probably it isn't worth
+	 * complications.
+	 */
+	KnownPreparedRecreateFiles(InvalidXLogRecPtr);
+
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
 	{
@@ -2162,3 +2198,111 @@ RecordTransactionAbortPrepared(TransactionId xid,
 	 */
 	SyncRepWaitForLSN(recptr, false);
 }
+
+/*
+ * KnownPreparedAdd.
+ *
+ * Store correspondence of start/end lsn and xid in KnownPreparedList.
+ * This is called during redo of prepare record to have list of prepared
+ * transactions that aren't yet moved to 2PC files by the end of recovery.
+ */
+void
+KnownPreparedAdd(XLogReaderState *record)
+{
+	KnownPreparedXact *xact;
+	TwoPhaseFileHeader *hdr = (TwoPhaseFileHeader *) XLogRecGetData(record);
+
+	Assert(RecoveryInProgress());
+
+	xact = (KnownPreparedXact *) palloc(sizeof(KnownPreparedXact));
+	xact->xid = hdr->xid;
+	xact->prepare_start_lsn = record->ReadRecPtr;
+	xact->prepare_end_lsn = record->EndRecPtr;
+
+	dlist_push_tail(&KnownPreparedList, &xact->list_node);
+}
+
+/*
+ * KnownPreparedRemoveByXid
+ *
+ * Forget about prepared transaction. Called during commit/abort redo.
+ */
+void
+KnownPreparedRemoveByXid(TransactionId xid)
+{
+	dlist_mutable_iter miter;
+
+	Assert(RecoveryInProgress());
+
+	dlist_foreach_modify(miter, &KnownPreparedList)
+	{
+		KnownPreparedXact   *xact = dlist_container(KnownPreparedXact,
+														list_node, miter.cur);
+
+		if (xact->xid == xid)
+		{
+			dlist_delete(miter.cur);
+			/*
+			 * Since we found entry in KnownPreparedList we know that file isn't
+			 * on disk yet and we can end up here.
+			 */
+			return;
+		}
+	}
+
+	/*
+	 * Here we know that file should be moved to disk. But aborting recovery because
+	 * of absence of unnecessary file doesn't seems to be a good idea, so call remove
+	 * with giveWarning=false.
+	 */
+	RemoveTwoPhaseFile(xid, false);
+}
+
+/*
+ * KnownPreparedRecreateFiles
+ *
+ * Moves prepare records from WAL to files. Called during checkpoint replay
+ * or PrescanPreparedTransactions.
+ *
+ * redo_horizon = InvalidXLogRecPtr indicates that all transactions from
+ *		KnownPreparedList should be moved to disk.
+ */
+void
+KnownPreparedRecreateFiles(XLogRecPtr redo_horizon)
+{
+	dlist_mutable_iter miter;
+	int			serialized_xacts = 0;
+
+	Assert(RecoveryInProgress());
+
+	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_START();
+
+	dlist_foreach_modify(miter, &KnownPreparedList)
+	{
+		KnownPreparedXact   *xact = dlist_container(KnownPreparedXact,
+														list_node, miter.cur);
+
+		if (xact->prepare_end_lsn <= redo_horizon || redo_horizon == InvalidXLogRecPtr)
+		{
+			char	   *buf;
+			int			len;
+
+			XlogReadTwoPhaseData(xact->prepare_start_lsn, &buf, &len);
+			RecreateTwoPhaseFile(xact->xid, buf, len);
+			pfree(buf);
+			dlist_delete(miter.cur);
+			serialized_xacts++;
+		}
+	}
+
+	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_DONE();
+
+	if (log_checkpoints && serialized_xacts > 0)
+		ereport(LOG,
+				(errmsg_plural("%u two-phase state file was written "
+							   "for long-running prepared transactions",
+							   "%u two-phase state files were written "
+							   "for long-running prepared transactions",
+							   serialized_xacts,
+							   serialized_xacts)));
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index d643216..b3e0238 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -5604,7 +5604,9 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete KnownPrepared entry or 2PC file. */
+			KnownPreparedRemoveByXid(parsed.twophase_xid);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5624,14 +5626,20 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete KnownPrepared entry or 2PC file. */
+			KnownPreparedRemoveByXid(parsed.twophase_xid);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		/*
+		 * If that transaction will not be committed by the end of recovery then we
+		 * will need 2PC file (the record contents is exactly the 2PC file) to be able
+		 * to commit that later.
+		 * For now store xid and pointers to that record in KnownPreparedList.
+		 */
+		KnownPreparedAdd(record);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 084401d..9ac1fd7 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -9488,6 +9488,7 @@ xlog_redo(XLogReaderState *record)
 					(errmsg("unexpected timeline ID %u (should be %u) in checkpoint record",
 							checkPoint.ThisTimeLineID, ThisTimeLineID)));
 
+		KnownPreparedRecreateFiles(checkPoint.redo);
 		RecoveryRestartPoint(&checkPoint);
 	}
 	else if (info == XLOG_END_OF_RECOVERY)
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b7ce0c6..23be08f 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -15,6 +15,7 @@
 #define TWOPHASE_H
 
 #include "access/xlogdefs.h"
+#include "access/xlogreader.h"
 #include "datatype/timestamp.h"
 #include "storage/lock.h"
 
@@ -56,4 +57,8 @@ extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void KnownPreparedAdd(XLogReaderState *record);
+extern void KnownPreparedRemoveByXid(TransactionId xid);
+extern void KnownPreparedRecreateFiles(XLogRecPtr redo_horizon);
+
 #endif   /* TWOPHASE_H */
diff --git a/src/test/recovery/t/009_twophase.pl b/src/test/recovery/t/009_twophase.pl
new file mode 100644
index 0000000..27bccb3
--- /dev/null
+++ b/src/test/recovery/t/009_twophase.pl
@@ -0,0 +1,315 @@
+# Tests dedicated to two-phase commit in recovery
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 13;
+
+# Setup master node
+my $node_master = get_new_node("master");
+$node_master->init(allows_streaming => 1);
+$node_master->append_conf('postgresql.conf', qq(
+	max_prepared_transactions = 10
+	log_checkpoints = true
+));
+$node_master->start;
+$node_master->backup('master_backup');
+$node_master->psql('postgres', "create table t(id int)");
+
+# Setup master node
+my $node_slave = get_new_node('slave');
+$node_slave->init_from_backup($node_master, 'master_backup', has_streaming => 1);
+$node_slave->start;
+
+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+	synchronous_standby_names = '*'
+));
+$node_master->psql('postgres', "select pg_reload_conf()");
+
+my $psql_out = '';
+my $psql_rc = '';
+
+###############################################################################
+# Check that we can commit and abort tx after soft restart.
+# Here checkpoint happens before shutdown and no WAL replay will occur at next
+# startup. In this case postgres re-create shared-memory state from twophase
+# files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	savepoint s1;
+	insert into t values (143);
+	prepare transaction 'y';");
+$node_master->stop;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared transaction after restart.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared transaction after restart.');
+
+###############################################################################
+# Check that we can commit and abort after hard restart.
+# At next startup, WAL replay will re-create shared memory state for prepared
+# transaction using dedicated WAL records.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	savepoint s1;
+	insert into t values (143);
+	prepare transaction 'y';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared tx after teardown.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared transaction after teardown.');
+
+###############################################################################
+# Check that WAL replay can handle several transactions with same name GID.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Replay several transactions with same GID.');
+
+###############################################################################
+# Check that WAL replay cleans up its shared memory state and releases locks
+# while replaying transaction commits.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_master->teardown_node;
+$node_master->start;
+$psql_rc = $node_master->psql('postgres', "begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	-- This prepare can fail due to conflicting GID or locks conflicts if
+	-- replay did not fully cleanup its state on previous commit.
+	prepare transaction 'x';");
+is($psql_rc, '0', "Cleanup of shared memory state for 2PC commit");
+
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that WAL replay will cleanup its shared memory state on running slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;",
+	  stdout => \$psql_out);
+is($psql_out, '0',
+   "Cleanup of shared memory state on running standby without checkpoint.");
+
+###############################################################################
+# Same as in previous case, but let's force checkpoint on slave between
+# prepare and commit to use on-disk twophase files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_slave->psql('postgres', "checkpoint;");
+$node_master->psql('postgres', "commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;",
+	  stdout => \$psql_out);
+is($psql_out, '0',
+   "Cleanup of shared memory state on running standby after checkpoint.");
+
+###############################################################################
+# Check that prepared transactions can be committed on promoted slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$psql_rc = $node_slave->psql('postgres', "commit prepared 'x';");
+is($psql_rc, '0', "Restore of prepared transaction on promoted slave.");
+
+# change roles
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+
+###############################################################################
+# Check that prepared transactions are replayed after soft restart of standby
+# while master is down. Since standby knows that master is down it uses
+# different code path on start to be sure that the status of transactions is
+# consistent.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->stop;
+$node_slave->restart;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '1',
+   "Restore prepared transactions from files with master down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that prepared transactions are correctly replayed after slave hard
+# restart while master is down.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (242);
+	savepoint s1;
+	insert into t values (243);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->teardown_node;
+$node_slave->start;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres',
+	  "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '1',
+   "Restore prepared transactions from records with master down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres', "commit prepared 'x'");
+
+
+###############################################################################
+# Check for a lock conflict between prepared tx with DDL inside and replay of
+# XLOG_STANDBY_LOCK wal record.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	create table t2(id int);
+	savepoint s1;
+	insert into t2 values (42);
+	prepare transaction 'x';
+	-- checkpoint will issue XLOG_STANDBY_LOCK that can conflict with lock
+	-- held by 'create table' statement
+	checkpoint;
+	commit prepared 'x';");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '0', "Replay prepared transaction with DDL.");
+
+
+###############################################################################
+# Check that replay will correctly set SUBTRANS and properly advance nextXid
+# so it won't conflict with savepoint xids.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	delete from t;
+	insert into t values (43);
+	savepoint s1;
+	insert into t values (43);
+	savepoint s2;
+	insert into t values (43);
+	savepoint s3;
+	insert into t values (43);
+	savepoint s4;
+	insert into t values (43);
+	savepoint s5;
+	insert into t values (43);
+	prepare transaction 'x';
+	checkpoint;");
+
+$node_master->stop;
+$node_master->start;
+$node_master->psql('postgres', "
+	-- here we can get xid of previous savepoint if nextXid
+	-- wasn't properly advanced
+	begin;
+	insert into t values (142);
+	abort;
+	commit prepared 'x';");
+
+$node_master->psql('postgres', "select count(*) from t",
+	  stdout => \$psql_out);
+is($psql_out, '6', "Check nextXid handling for prepared subtransactions");
#99Bruce Momjian
bruce@momjian.us
In reply to: Michael Paquier (#98)
Re: Speedup twophase transactions

On Sun, Dec 18, 2016 at 07:41:50AM +0900, Michael Paquier wrote:

On Sun, Dec 18, 2016 at 6:42 AM, Bruce Momjian <bruce@momjian.us> wrote:

Uh, did you mean to attached patch here?

Strange. I can confirm that I have received the patch as attached, but
it is not on the archives.

It must have been stripped by our email system. You were a direct CC so
you received it.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#100Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Bruce Momjian (#99)
Re: Speedup twophase transactions

On 18 Dec 2016, at 01:54, Bruce Momjian <bruce@momjian.us> wrote:

On Sun, Dec 18, 2016 at 07:41:50AM +0900, Michael Paquier wrote:

Strange. I can confirm that I have received the patch as attached, but
it is not on the archives.

It must have been stripped by our email system. You were a direct CC so
you received it.

Then, probably, my mail client did something strange. I’ll check.

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#101David Fetter
david@fetter.org
In reply to: Bruce Momjian (#99)
Re: Speedup twophase transactions

On Sat, Dec 17, 2016 at 05:54:04PM -0500, Bruce Momjian wrote:

On Sun, Dec 18, 2016 at 07:41:50AM +0900, Michael Paquier wrote:

On Sun, Dec 18, 2016 at 6:42 AM, Bruce Momjian <bruce@momjian.us> wrote:

Uh, did you mean to attached patch here?

Strange. I can confirm that I have received the patch as attached, but
it is not on the archives.

It must have been stripped by our email system. You were a direct
CC so you received it.

I was neither, and I received it, so I don't thing PostgreSQL's email
system stripped it. It's pretty mystifying, though.

Best,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#102Andres Freund
andres@anarazel.de
In reply to: David Fetter (#101)
Re: Speedup twophase transactions

On 2016-12-17 15:30:08 -0800, David Fetter wrote:

On Sat, Dec 17, 2016 at 05:54:04PM -0500, Bruce Momjian wrote:

On Sun, Dec 18, 2016 at 07:41:50AM +0900, Michael Paquier wrote:

On Sun, Dec 18, 2016 at 6:42 AM, Bruce Momjian <bruce@momjian.us> wrote:

Uh, did you mean to attached patch here?

Strange. I can confirm that I have received the patch as attached, but
it is not on the archives.

It must have been stripped by our email system. You were a direct
CC so you received it.

I was neither, and I received it, so I don't thing PostgreSQL's email
system stripped it. It's pretty mystifying, though.

The mime construction in the email is weird. The attachement is below a
multipart/alternative and multipart/mixed, besides the text/html version
of the plain text email. Because the attachement is below the
multipart/alternative (i.e. the switch between plain text / html), it'll
not be considered a proper attachement by many mail readers.

It seems quite borked to put an attachement there - weird that apple
mail does so.

Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#103Bruce Momjian
bruce@momjian.us
In reply to: Andres Freund (#102)
Re: Speedup twophase transactions

On Sat, Dec 17, 2016 at 03:47:34PM -0800, Andres Freund wrote:

On 2016-12-17 15:30:08 -0800, David Fetter wrote:

On Sat, Dec 17, 2016 at 05:54:04PM -0500, Bruce Momjian wrote:

On Sun, Dec 18, 2016 at 07:41:50AM +0900, Michael Paquier wrote:

On Sun, Dec 18, 2016 at 6:42 AM, Bruce Momjian <bruce@momjian.us> wrote:

Uh, did you mean to attached patch here?

Strange. I can confirm that I have received the patch as attached, but
it is not on the archives.

It must have been stripped by our email system. You were a direct
CC so you received it.

I was neither, and I received it, so I don't thing PostgreSQL's email
system stripped it. It's pretty mystifying, though.

The mime construction in the email is weird. The attachement is below a
multipart/alternative and multipart/mixed, besides the text/html version
of the plain text email. Because the attachement is below the
multipart/alternative (i.e. the switch between plain text / html), it'll
not be considered a proper attachement by many mail readers.

It seems quite borked to put an attachement there - weird that apple
mail does so.

Oh, I now see the attachment _is_ in the original email, it is just that
mutt doesn't show it. Here is the heirarchy of the email as shown by
mutt:

I 1 <no description> [multipa/alternativ, 7bit, 27K]
I 2 ├─><no description> [text/plain, quoted, us-ascii, 0.8K]
I 3 └─><no description> [multipa/mixed, 7bit, 26K]
I 4 ├─><no description> [text/html, quoted, us-ascii, 5.4K]
A 5 ├─>twophase_recovery_list.diff [applica/octet-stre, 7bit, 20K]
I 6 └─><no description> [text/html, 7bit, us-ascii, 0.4K]

As Andres already stated, the problem is that there is a text/plain and
text/html of the same email, and the diff is _inside_ the multipa/mixed
HTML block. I think it needs to be outside on its own.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#104Bruce Momjian
bruce@momjian.us
In reply to: Bruce Momjian (#103)
Re: Speedup twophase transactions

On Sat, Dec 17, 2016 at 07:38:46PM -0500, Bruce Momjian wrote:

As Andres already stated, the problem is that there is a text/plain and
text/html of the same email, and the diff is _inside_ the multipa/mixed
HTML block. I think it needs to be outside on its own.

Mutt shows text/plain by default. I bet email readers that prefer to
display HTML see the patch. Mystery solved by Andres! :-)

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#105Michael Paquier
michael.paquier@gmail.com
In reply to: Stas Kelvich (#96)
Re: Speedup twophase transactions

On Fri, Dec 16, 2016 at 8:00 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

So, here is brand new implementation of the same thing.

Now instead of creating pgproc entry for prepared transaction during
recovery,
I just store recptr/xid correspondence in separate 2L-list and deleting
entries in that
list if redo process faced commit/abort. In case of checkpoint or end of
recovery
transactions remaining in that list are dumped to files in pg_twophase.

Seems that current approach is way more simpler and patch has two times less
LOCs then previous one.

That's indeed way simpler than before. Have you as well looked at the
most simple approach discussed? That would be just roughly replacing
the pg_fsync() calls currently in RecreateTwoPhaseFile() by a save
into a list as you are doing, then issue them all checkpoint. Even for
2PC files that are created and then removed before the next
checkpoint, those will likely be in system cache. This removes as well
the need to have XlogReadTwoPhaseData() work in crash recovery, which
makes me a bit nervous. And this saves lookups at the WAL segments
still present in pg_xlog, making the operation at checkpoint much
faster with many 2PC files to process.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#106Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#105)
Re: Speedup twophase transactions

On 21 Dec 2016, at 19:56, Michael Paquier <michael.paquier@gmail.com> wrote:

That's indeed way simpler than before. Have you as well looked at the
most simple approach discussed? That would be just roughly replacing
the pg_fsync() calls currently in RecreateTwoPhaseFile() by a save
into a list as you are doing, then issue them all checkpoint.Even for
2PC files that are created and then removed before the next
checkpoint, those will likely be in system cache.

Yes, I tried that as well. But in such approach another bottleneck arises —
new file creation isn’t very cheap operation itself. Dual xeon with 100 backends
quickly hit that, and OS routines about file creation occupies first places in
perf top. Probably that depends on filesystem (I used ext4), but avoiding
file creation when it isn’t necessary seems like cleaner approach.
On the other hand it is possible to skip file creation by reusing files, for example
naming them by dummy PGPROC offset, but that would require some changes
to places that right now looks only at filenames.

This removes as well
the need to have XlogReadTwoPhaseData() work in crash recovery, which
makes me a bit nervous.

Hm, do you have any particular bad scenario for that case in you mind?

And this saves lookups at the WAL segments
still present in pg_xlog, making the operation at checkpoint much
faster with many 2PC files to process.

ISTM your reasoning about filesystem cache applies here as well, but just without
spending time on file creation.

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

#107Michael Paquier
michael.paquier@gmail.com
In reply to: Stas Kelvich (#106)
Re: Speedup twophase transactions

On Wed, Dec 21, 2016 at 10:37 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

On 21 Dec 2016, at 19:56, Michael Paquier <michael.paquier@gmail.com> wrote:

That's indeed way simpler than before. Have you as well looked at the
most simple approach discussed? That would be just roughly replacing
the pg_fsync() calls currently in RecreateTwoPhaseFile() by a save
into a list as you are doing, then issue them all checkpoint.Even for

2PC files that are created and then removed before the next
checkpoint, those will likely be in system cache.

Yes, I tried that as well. But in such approach another bottleneck arises

new file creation isn’t very cheap operation itself. Dual xeon with 100
backends
quickly hit that, and OS routines about file creation occupies first places
in perf top. Probably that depends on filesystem (I used ext4), but avoiding
file creation when it isn’t necessary seems like cleaner approach.
On the other hand it is possible to skip file creation by reusing files,
for example
naming them by dummy PGPROC offset, but that would require some changes
to places that right now looks only at filenames.

Interesting. Thanks for looking at it. Your latest approach looks more
promising based on that then.

And this saves lookups at the WAL segments
still present in pg_xlog, making the operation at checkpoint much
faster with many 2PC files to process.

ISTM your reasoning about filesystem cache applies here as well, but just
without spending time on file creation.

True. The more spread the checkpoints and 2PC files, the more risk to
require access to disk. Memory's cheap anyway. What was the system
memory? How many checkpoints did you trigger for how many 2PC files
created? Perhaps it would be a good idea to look for the 2PC files
from WAL records in a specific order. Did you try to use
dlist_push_head instead of dlist_push_tail? This may make a difference
on systems where WAL segments don't fit in system cache as the latest
files generated would be looked at first for 2PC data.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#108Michael Paquier
michael.paquier@gmail.com
In reply to: Michael Paquier (#107)
Re: Speedup twophase transactions

On Thu, Dec 22, 2016 at 7:35 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:

On Wed, Dec 21, 2016 at 10:37 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

ISTM your reasoning about filesystem cache applies here as well, but just
without spending time on file creation.

True. The more spread the checkpoints and 2PC files, the more risk to
require access to disk. Memory's cheap anyway. What was the system
memory? How many checkpoints did you trigger for how many 2PC files
created? Perhaps it would be a good idea to look for the 2PC files
from WAL records in a specific order. Did you try to use
dlist_push_head instead of dlist_push_tail? This may make a difference
on systems where WAL segments don't fit in system cache as the latest
files generated would be looked at first for 2PC data.

Stas, have you tested as well tested the impact on recovery time when
WAL segments are very likely evicted from the OS cache? This could be
a plausible scenario if a standby instance is heavily used for
read-only transactions (say pgbench -S), and that the data quantity is
higher than the amount of RAM available. It would not be complicated
to test that: just drop_caches before beginning recovery. The maximum
amount of 2PC transactions that need to have access to the past WAL
segments is linearly related to the volume of WAL between two
checkpoints, so max_wal_size does not really matter. What matters is
the time it takes to recover the same amount of WAL. Increasing
max_wal_size would give more room to reduce the overall noise between
two measurements though.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#109Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#108)
Re: Speedup twophase transactions

On 22 Dec 2016, at 05:35, Michael Paquier <michael.paquier@gmail.com> wrote:

True. The more spread the checkpoints and 2PC files, the more risk to
require access to disk. Memory's cheap anyway. What was the system
memory? How many checkpoints did you trigger for how many 2PC files
created?

Standard config with increased shared_buffers. I think the most significant
impact on the recovery speed here is on the client side, namely time between
prepare and commit. Right now I’m using pgbench script that issues commit
right after prepare. It’s also possible to put sleep between prepare and commit
and increase number of connections to thousands. That will be probably the
worst case — majority of prepared tx will be moved to files.

Perhaps it would be a good idea to look for the 2PC files
from WAL records in a specific order. Did you try to use
dlist_push_head instead of dlist_push_tail? This may make a difference
on systems where WAL segments don't fit in system cache as the latest
files generated would be looked at first for 2PC data.

Ouch! Good catch. I didn’t actually noticed that list populated in opposite order
with respect to traversal. I’ll fix that.

On 27 Dec 2016, at 08:33, Michael Paquier <michael.paquier@gmail.com> wrote:

Stas, have you tested as well tested the impact on recovery time when
WAL segments are very likely evicted from the OS cache? This could be
a plausible scenario if a standby instance is heavily used for
read-only transactions (say pgbench -S), and that the data quantity is
higher than the amount of RAM available. It would not be complicated
to test that: just drop_caches before beginning recovery. The maximum
amount of 2PC transactions that need to have access to the past WAL
segments is linearly related to the volume of WAL between two
checkpoints, so max_wal_size does not really matter. What matters is
the time it takes to recover the same amount of WAL. Increasing
max_wal_size would give more room to reduce the overall noise between
two measurements though.

Okay, i’ll perform such testing.

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

#110Michael Paquier
michael.paquier@gmail.com
In reply to: Stas Kelvich (#109)
Re: Speedup twophase transactions

On Tue, Dec 27, 2016 at 12:59 PM, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

Standard config with increased shared_buffers. I think the most significant
impact on the recovery speed here is on the client side, namely time between
prepare and commit. Right now I’m using pgbench script that issues commit
right after prepare. It’s also possible to put sleep between prepare and
commit
and increase number of connections to thousands. That will be probably the
worst case — majority of prepared tx will be moved to files.

I think that it would be a good idea to actually test that in pure
recovery time, aka no client, and just use a base backup and make it
recover X prepared transactions that have created Y checkpoints after
dropping cache (or restarting server).
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#111Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#110)
5 attachment(s)
Re: Speedup twophase transactions

On 27 Dec 2016, at 07:31, Michael Paquier <michael.paquier@gmail.com> wrote:

I think that it would be a good idea to actually test that in pure
recovery time, aka no client, and just use a base backup and make it
recover X prepared transactions that have created Y checkpoints after
dropping cache (or restarting server).

I did tests with following setup:

* Start postgres initialised with pgbench
* Start pg_receivexlog
* Take basebackup
* Perform 1.5 M transactions
* Stop everything and apply wal files stored by pg_receivexlog to base backup.

All tests performed on a laptop with nvme ssd
number of transactions: 1.5M
start segment: 0x4

-master non-2pc:
last segment: 0x1b
average recovery time per 16 wal files: 11.8s
average total recovery time: 17.0s

-master 2pc:
last segment: 0x44
average recovery time per 16 wal files: 142s
average total recovery time: 568s

-patched 2pc (previous patch):
last segment: 0x44
average recovery time per 16 wal files: 5.3s
average total recovery time: 21.2s

-patched2 2pc (dlist_push_tail changed to dlist_push_head):
last segment: 0x44
average recovery time per 16 wal files: 5.2s
average total recovery time: 20.8s

So skipping unnecessary fsyncs gave us x25 speed increase even on ssd (on hdd difference should be bigger).
Pushing to list's head didn’t yield measurable results, but anyway seems to be conceptually better.

PS:
I’ve faced situation when pg_basebackup freezes until checkpoint happens (automatic or user-issued).
Is that expected behaviour?

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Attachments:

twophase_recovery_list_2.diffapplication/octet-stream; name=twophase_recovery_list_2.diff; x-unix-mode=0644Download
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 5b72c1d..07fc7f8 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,8 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *		* Simplified version of the same scenario happens during recovery and
+ *		  replication. See comments to KnownPreparedXact structure.
  *
  *-------------------------------------------------------------------------
  */
@@ -181,6 +181,35 @@ static GlobalTransaction MyLockedGxact = NULL;
 
 static bool twophaseExitRegistered = false;
 
+/*
+ * During replay and replication KnownPreparedList holds info about active prepared
+ * transactions that weren't moved to files yet. We will need that info by the end of
+ * recovery (including promote) to restore memory state of that transactions.
+ *
+ * Naive approach here is to move each PREPARE record to disk, fsync it and don't have
+ * that list at all, but that provokes a lot of unnecessary fsyncs on small files
+ * causing replica to be slower than master.
+ *
+ * Replay of twophase records happens by the following rules:
+ *		* On PREPARE redo KnownPreparedAdd() is called to add that transaction to
+ *		  KnownPreparedList and no more actions are taken.
+ *		* On checkpoint redo we iterate through KnownPreparedList and move all prepare
+ *		  records that behind redo_horizon to files and deleting them from list.
+ *		* On COMMIT/ABORT we delete file or entry in KnownPreparedList.
+ *		* At the end of recovery we move all known prepared transactions to disk
+ *		  to allow RecoverPreparedTransactions/StandbyRecoverPreparedTransactions
+ *		  do their work.
+ */
+typedef struct KnownPreparedXact
+{
+	TransactionId	xid;
+	XLogRecPtr		prepare_start_lsn;
+	XLogRecPtr		prepare_end_lsn;
+	dlist_node		list_node;
+} KnownPreparedXact;
+
+static dlist_head KnownPreparedList = DLIST_STATIC_INIT(KnownPreparedList);
+
 static void RecordTransactionCommitPrepared(TransactionId xid,
 								int nchildren,
 								TransactionId *children,
@@ -1241,9 +1270,9 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
  * Reads 2PC data from xlog. During checkpoint this data will be moved to
  * twophase files and ReadTwoPhaseFile should be used instead.
  *
- * Note clearly that this function accesses WAL during normal operation, similarly
- * to the way WALSender or Logical Decoding would do. It does not run during
- * crash recovery or standby processing.
+ * Note clearly that this function can access WAL during normal operation, similarly
+ * to the way WALSender or Logical Decoding would do.
+ *
  */
 static void
 XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
@@ -1252,8 +1281,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1691,6 +1718,15 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 	int			nxids = 0;
 	int			allocsize = 0;
 
+	/*
+	 * Move prepared transactions from KnownPreparedList to files, if any.
+	 * It is possible to skip that step and teach subsequent code about
+	 * KnownPreparedList, but whole PrescanPreparedTransactions() happens
+	 * once during end of recovery or promote, so probably it isn't worth
+	 * complications.
+	 */
+	KnownPreparedRecreateFiles(InvalidXLogRecPtr);
+
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
 	{
@@ -2162,3 +2198,111 @@ RecordTransactionAbortPrepared(TransactionId xid,
 	 */
 	SyncRepWaitForLSN(recptr, false);
 }
+
+/*
+ * KnownPreparedAdd.
+ *
+ * Store correspondence of start/end lsn and xid in KnownPreparedList.
+ * This is called during redo of prepare record to have list of prepared
+ * transactions that aren't yet moved to 2PC files by the end of recovery.
+ */
+void
+KnownPreparedAdd(XLogReaderState *record)
+{
+	KnownPreparedXact *xact;
+	TwoPhaseFileHeader *hdr = (TwoPhaseFileHeader *) XLogRecGetData(record);
+
+	Assert(RecoveryInProgress());
+
+	xact = (KnownPreparedXact *) palloc(sizeof(KnownPreparedXact));
+	xact->xid = hdr->xid;
+	xact->prepare_start_lsn = record->ReadRecPtr;
+	xact->prepare_end_lsn = record->EndRecPtr;
+
+	dlist_push_head(&KnownPreparedList, &xact->list_node);
+}
+
+/*
+ * KnownPreparedRemoveByXid
+ *
+ * Forget about prepared transaction. Called during commit/abort redo.
+ */
+void
+KnownPreparedRemoveByXid(TransactionId xid)
+{
+	dlist_mutable_iter miter;
+
+	Assert(RecoveryInProgress());
+
+	dlist_foreach_modify(miter, &KnownPreparedList)
+	{
+		KnownPreparedXact   *xact = dlist_container(KnownPreparedXact,
+														list_node, miter.cur);
+
+		if (xact->xid == xid)
+		{
+			dlist_delete(miter.cur);
+			/*
+			 * Since we found entry in KnownPreparedList we know that file isn't
+			 * on disk yet and we can end up here.
+			 */
+			return;
+		}
+	}
+
+	/*
+	 * Here we know that file should be moved to disk. But aborting recovery because
+	 * of absence of unnecessary file doesn't seems to be a good idea, so call remove
+	 * with giveWarning=false.
+	 */
+	RemoveTwoPhaseFile(xid, false);
+}
+
+/*
+ * KnownPreparedRecreateFiles
+ *
+ * Moves prepare records from WAL to files. Called during checkpoint replay
+ * or PrescanPreparedTransactions.
+ *
+ * redo_horizon = InvalidXLogRecPtr indicates that all transactions from
+ *		KnownPreparedList should be moved to disk.
+ */
+void
+KnownPreparedRecreateFiles(XLogRecPtr redo_horizon)
+{
+	dlist_mutable_iter miter;
+	int			serialized_xacts = 0;
+
+	Assert(RecoveryInProgress());
+
+	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_START();
+
+	dlist_foreach_modify(miter, &KnownPreparedList)
+	{
+		KnownPreparedXact   *xact = dlist_container(KnownPreparedXact,
+														list_node, miter.cur);
+
+		if (xact->prepare_end_lsn <= redo_horizon || redo_horizon == InvalidXLogRecPtr)
+		{
+			char	   *buf;
+			int			len;
+
+			XlogReadTwoPhaseData(xact->prepare_start_lsn, &buf, &len);
+			RecreateTwoPhaseFile(xact->xid, buf, len);
+			pfree(buf);
+			dlist_delete(miter.cur);
+			serialized_xacts++;
+		}
+	}
+
+	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_DONE();
+
+	if (log_checkpoints && serialized_xacts > 0)
+		ereport(LOG,
+				(errmsg_plural("%u two-phase state file was written "
+							   "for long-running prepared transactions",
+							   "%u two-phase state files were written "
+							   "for long-running prepared transactions",
+							   serialized_xacts,
+							   serialized_xacts)));
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index f6f136d..6c2ae7f 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -5606,7 +5606,9 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete KnownPrepared entry or 2PC file. */
+			KnownPreparedRemoveByXid(parsed.twophase_xid);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5626,14 +5628,20 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete KnownPrepared entry or 2PC file. */
+			KnownPreparedRemoveByXid(parsed.twophase_xid);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		/*
+		 * If that transaction will not be committed by the end of recovery then we
+		 * will need 2PC file (the record contents is exactly the 2PC file) to be able
+		 * to commit that later.
+		 * For now store xid and pointers to that record in KnownPreparedList.
+		 */
+		KnownPreparedAdd(record);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 2f5d603..2c15ed1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -9573,6 +9573,7 @@ xlog_redo(XLogReaderState *record)
 					(errmsg("unexpected timeline ID %u (should be %u) in checkpoint record",
 							checkPoint.ThisTimeLineID, ThisTimeLineID)));
 
+		KnownPreparedRecreateFiles(checkPoint.redo);
 		RecoveryRestartPoint(&checkPoint);
 	}
 	else if (info == XLOG_END_OF_RECOVERY)
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b2b7848..817bb9e 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -15,6 +15,7 @@
 #define TWOPHASE_H
 
 #include "access/xlogdefs.h"
+#include "access/xlogreader.h"
 #include "datatype/timestamp.h"
 #include "storage/lock.h"
 
@@ -56,4 +57,8 @@ extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void KnownPreparedAdd(XLogReaderState *record);
+extern void KnownPreparedRemoveByXid(TransactionId xid);
+extern void KnownPreparedRecreateFiles(XLogRecPtr redo_horizon);
+
 #endif   /* TWOPHASE_H */
diff --git a/src/test/recovery/t/009_twophase.pl b/src/test/recovery/t/009_twophase.pl
new file mode 100644
index 0000000..27bccb3
--- /dev/null
+++ b/src/test/recovery/t/009_twophase.pl
@@ -0,0 +1,315 @@
+# Tests dedicated to two-phase commit in recovery
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 13;
+
+# Setup master node
+my $node_master = get_new_node("master");
+$node_master->init(allows_streaming => 1);
+$node_master->append_conf('postgresql.conf', qq(
+	max_prepared_transactions = 10
+	log_checkpoints = true
+));
+$node_master->start;
+$node_master->backup('master_backup');
+$node_master->psql('postgres', "create table t(id int)");
+
+# Setup master node
+my $node_slave = get_new_node('slave');
+$node_slave->init_from_backup($node_master, 'master_backup', has_streaming => 1);
+$node_slave->start;
+
+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+	synchronous_standby_names = '*'
+));
+$node_master->psql('postgres', "select pg_reload_conf()");
+
+my $psql_out = '';
+my $psql_rc = '';
+
+###############################################################################
+# Check that we can commit and abort tx after soft restart.
+# Here checkpoint happens before shutdown and no WAL replay will occur at next
+# startup. In this case postgres re-create shared-memory state from twophase
+# files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	savepoint s1;
+	insert into t values (143);
+	prepare transaction 'y';");
+$node_master->stop;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared transaction after restart.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared transaction after restart.');
+
+###############################################################################
+# Check that we can commit and abort after hard restart.
+# At next startup, WAL replay will re-create shared memory state for prepared
+# transaction using dedicated WAL records.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	savepoint s1;
+	insert into t values (143);
+	prepare transaction 'y';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared tx after teardown.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared transaction after teardown.');
+
+###############################################################################
+# Check that WAL replay can handle several transactions with same name GID.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Replay several transactions with same GID.');
+
+###############################################################################
+# Check that WAL replay cleans up its shared memory state and releases locks
+# while replaying transaction commits.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_master->teardown_node;
+$node_master->start;
+$psql_rc = $node_master->psql('postgres', "begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	-- This prepare can fail due to conflicting GID or locks conflicts if
+	-- replay did not fully cleanup its state on previous commit.
+	prepare transaction 'x';");
+is($psql_rc, '0', "Cleanup of shared memory state for 2PC commit");
+
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that WAL replay will cleanup its shared memory state on running slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;",
+	  stdout => \$psql_out);
+is($psql_out, '0',
+   "Cleanup of shared memory state on running standby without checkpoint.");
+
+###############################################################################
+# Same as in previous case, but let's force checkpoint on slave between
+# prepare and commit to use on-disk twophase files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_slave->psql('postgres', "checkpoint;");
+$node_master->psql('postgres', "commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;",
+	  stdout => \$psql_out);
+is($psql_out, '0',
+   "Cleanup of shared memory state on running standby after checkpoint.");
+
+###############################################################################
+# Check that prepared transactions can be committed on promoted slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$psql_rc = $node_slave->psql('postgres', "commit prepared 'x';");
+is($psql_rc, '0', "Restore of prepared transaction on promoted slave.");
+
+# change roles
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+
+###############################################################################
+# Check that prepared transactions are replayed after soft restart of standby
+# while master is down. Since standby knows that master is down it uses
+# different code path on start to be sure that the status of transactions is
+# consistent.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->stop;
+$node_slave->restart;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '1',
+   "Restore prepared transactions from files with master down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that prepared transactions are correctly replayed after slave hard
+# restart while master is down.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (242);
+	savepoint s1;
+	insert into t values (243);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->teardown_node;
+$node_slave->start;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres',
+	  "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '1',
+   "Restore prepared transactions from records with master down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres', "commit prepared 'x'");
+
+
+###############################################################################
+# Check for a lock conflict between prepared tx with DDL inside and replay of
+# XLOG_STANDBY_LOCK wal record.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	create table t2(id int);
+	savepoint s1;
+	insert into t2 values (42);
+	prepare transaction 'x';
+	-- checkpoint will issue XLOG_STANDBY_LOCK that can conflict with lock
+	-- held by 'create table' statement
+	checkpoint;
+	commit prepared 'x';");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '0', "Replay prepared transaction with DDL.");
+
+
+###############################################################################
+# Check that replay will correctly set SUBTRANS and properly advance nextXid
+# so it won't conflict with savepoint xids.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	delete from t;
+	insert into t values (43);
+	savepoint s1;
+	insert into t values (43);
+	savepoint s2;
+	insert into t values (43);
+	savepoint s3;
+	insert into t values (43);
+	savepoint s4;
+	insert into t values (43);
+	savepoint s5;
+	insert into t values (43);
+	prepare transaction 'x';
+	checkpoint;");
+
+$node_master->stop;
+$node_master->start;
+$node_master->psql('postgres', "
+	-- here we can get xid of previous savepoint if nextXid
+	-- wasn't properly advanced
+	begin;
+	insert into t values (142);
+	abort;
+	commit prepared 'x';");
+
+$node_master->psql('postgres', "select count(*) from t",
+	  stdout => \$psql_out);
+is($psql_out, '6', "Check nextXid handling for prepared subtransactions");
master-non2pc.svgimage/svg+xml; name=master-non2pc.svg; x-unix-mode=0644Download
patched-2pc.svgimage/svg+xml; name=patched-2pc.svg; x-unix-mode=0644Download
patched2-2pc.svgimage/svg+xml; name=patched2-2pc.svg; x-unix-mode=0644Download
master-2pc.svgimage/svg+xml; name=master-2pc.svg; x-unix-mode=0644Download
#112Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Stas Kelvich (#111)
1 attachment(s)
Re: Speedup twophase transactions

Hi Stas,

So, here is brand new implementation of the same thing.

Now instead of creating pgproc entry for prepared transaction during recovery,
I just store recptr/xid correspondence in separate 2L-list and
deleting entries in that
list if redo process faced commit/abort. In case of checkpoint or end
of recovery
transactions remaining in that list are dumped to files in pg_twophase.

Seems that current approach is way more simpler and patch has two times less
LOCs then previous one.

The proposed approach and patch does appear to be much simpler than
the previous one.

I have some comments/questions on the patch (twophase_recovery_list_2.diff):

1)
+ /*

+ * Move prepared transactions from KnownPreparedList to files, if any.

+ * It is possible to skip that step and teach subsequent code about

+ * KnownPreparedList, but whole PrescanPreparedTransactions() happens

+ * once during end of recovery or promote, so probably it isn't worth

+ * complications.

+ */

+ KnownPreparedRecreateFiles(InvalidXLogRecPtr);

+

Speeding up recovery or failover activity via a faster promote is a
desirable thing. So, maybe, we should look at teaching the relevant
code about using "KnownPreparedList"? I know that would increase the
size of this patch and would mean more testing, but this seems to be
last remaining optimization in this code path.

2)
+ /*

+ * Here we know that file should be moved to disk. But
aborting recovery because

+ * of absence of unnecessary file doesn't seems to be a good
idea, so call remove

+ * with giveWarning=false.

+ */

+ RemoveTwoPhaseFile(xid, false);

We are going to call the above in case of COMMIT/ABORT. If so, we
should always find the "xid" entry either in the KnownPreparedList or
as a file. Does it make sense to call the above function with "false"
then?

3) We are pushing the fsyncing of 2PC files to the checkpoint replay
activity with this patch. Now, typically, we would assume that PREPARE
followed by COMMIT/ABORT would happen within a checkpoint replay
cycle, if not, this would make checkpoint replays slower since the
earlier spread out fsyncs are now getting bunched up. I had concerns
about this but clearly your latest performance measurements don't show
any negatives around this directly.

4) Minor nit-pick on existing code.

(errmsg_plural("%u two-phase state file was written "
"for
long-running prepared transactions",
"%u
two-phase state files were written "
"for
long-running prepared transactions",
serialized_xacts,
serialized_xacts)

Shouldn’t the singular part of the message above be:
"%u two-phase state file was written for a long-running prepared transaction"

But, then, English is not my native language, so I might be off here :-)

5) Why isn't KnownPreparedRecreateFiles() tracking which entries from
KnownPreparedList have been written to disk in an earlier iteration
like in the normal code path? Why is it removing the entry from
KnownPreparedList and not just marking it as saved on disk?

6) Do we need to hold TwoPhaseStateLock lock in shared mode while
calling KnownPreparedRecreateFiles()? I think, it does not matter in
the recovery/replay code path.

7) I fixed some typos and comments. PFA, patch attached.

Other than this, I ran TAP tests and they succeed as needed.

Regards,
Nikhils

Show quoted text

On 23 January 2017 at 16:56, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

On 27 Dec 2016, at 07:31, Michael Paquier <michael.paquier@gmail.com> wrote:

I think that it would be a good idea to actually test that in pure
recovery time, aka no client, and just use a base backup and make it
recover X prepared transactions that have created Y checkpoints after
dropping cache (or restarting server).

I did tests with following setup:

* Start postgres initialised with pgbench
* Start pg_receivexlog
* Take basebackup
* Perform 1.5 M transactions
* Stop everything and apply wal files stored by pg_receivexlog to base backup.

All tests performed on a laptop with nvme ssd
number of transactions: 1.5M
start segment: 0x4

-master non-2pc:
last segment: 0x1b
average recovery time per 16 wal files: 11.8s
average total recovery time: 17.0s

-master 2pc:
last segment: 0x44
average recovery time per 16 wal files: 142s
average total recovery time: 568s

-patched 2pc (previous patch):
last segment: 0x44
average recovery time per 16 wal files: 5.3s
average total recovery time: 21.2s

-patched2 2pc (dlist_push_tail changed to dlist_push_head):
last segment: 0x44
average recovery time per 16 wal files: 5.2s
average total recovery time: 20.8s

So skipping unnecessary fsyncs gave us x25 speed increase even on ssd (on hdd difference should be bigger).
Pushing to list's head didn’t yield measurable results, but anyway seems to be conceptually better.

PS:
I’ve faced situation when pg_basebackup freezes until checkpoint happens (automatic or user-issued).
Is that expected behaviour?

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Attachments:

twophase_recovery_list_2_23017.patchapplication/octet-stream; name=twophase_recovery_list_2_23017.patchDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 5b72c1d..a0c99ef 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,8 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *		* Simplified version of the same scenario happens during recovery and
+ *		  replication. See comments on the KnownPreparedXact structure.
  *
  *-------------------------------------------------------------------------
  */
@@ -181,6 +181,35 @@ static GlobalTransaction MyLockedGxact = NULL;
 
 static bool twophaseExitRegistered = false;
 
+/*
+ * During replay and replication, KnownPreparedList holds information about
+ * active prepared transactions that haven't been moved to disk yet. We will
+ * need this information at the end of recovery (including standby promote) to
+ * restore the state of these transactions.
+ *
+ * Replay of twophase records happens by the following rules:
+ *
+ *    * On PREPARE redo KnownPreparedAdd() is called to add that transaction
+ *      to KnownPreparedList.
+ *    * On checkpoint redo we iterate through KnownPreparedList and move all
+ *      prepare records that are behind the redo_horizon to disk. We also
+ *      delete them from KnownPreparedList.
+ *    * On COMMIT/ABORT we delete the entry from KnownPreparedList. If an
+ *      entry is not found, we delete the corresponding entry from the disk.
+ *    * At the end of recovery we move all known prepared transactions to disk.
+ *      This allows RecoverPreparedTransactions() and
+ *      StandbyRecoverPreparedTransactions() to do their work.
+ */
+typedef struct KnownPreparedXact
+{
+	TransactionId	xid;
+	XLogRecPtr		prepare_start_lsn;
+	XLogRecPtr		prepare_end_lsn;
+	dlist_node		list_node;
+} KnownPreparedXact;
+
+static dlist_head KnownPreparedList = DLIST_STATIC_INIT(KnownPreparedList);
+
 static void RecordTransactionCommitPrepared(TransactionId xid,
 								int nchildren,
 								TransactionId *children,
@@ -1241,9 +1270,9 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
  * Reads 2PC data from xlog. During checkpoint this data will be moved to
  * twophase files and ReadTwoPhaseFile should be used instead.
  *
- * Note clearly that this function accesses WAL during normal operation, similarly
- * to the way WALSender or Logical Decoding would do. It does not run during
- * crash recovery or standby processing.
+ * Note clearly that this function can access WAL during normal operation, similarly
+ * to the way WALSender or Logical Decoding would do.
+ *
  */
 static void
 XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
@@ -1252,8 +1281,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1691,6 +1718,15 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 	int			nxids = 0;
 	int			allocsize = 0;
 
+	/*
+	 * Move prepared transactions, if any, from KnownPreparedList to files.
+	 * It is possible to skip this step and teach subsequent code about
+	 * KnownPreparedList, but PrescanPreparedTransactions() happens once
+	 * during end of recovery or on promote, so probably it isn't worth
+	 * the additional code.
+	 */
+	KnownPreparedRecreateFiles(InvalidXLogRecPtr);
+
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
 	{
@@ -2162,3 +2198,113 @@ RecordTransactionAbortPrepared(TransactionId xid,
 	 */
 	SyncRepWaitForLSN(recptr, false);
 }
+
+/*
+ *  KnownPreparedAdd
+ *
+ * Store pointers to the start/end of the WAL record along with the xid in
+ * KnownPreparedList.
+ *
+ * This is called during redo of prepare record.
+ */
+void
+KnownPreparedAdd(XLogReaderState *record)
+{
+	KnownPreparedXact *xact;
+	TwoPhaseFileHeader *hdr = (TwoPhaseFileHeader *) XLogRecGetData(record);
+
+	Assert(RecoveryInProgress());
+
+	xact = (KnownPreparedXact *) palloc(sizeof(KnownPreparedXact));
+	xact->xid = hdr->xid;
+	xact->prepare_start_lsn = record->ReadRecPtr;
+	xact->prepare_end_lsn = record->EndRecPtr;
+
+	dlist_push_head(&KnownPreparedList, &xact->list_node);
+}
+
+/*
+ *  KnownPreparedRemoveByXid
+ *
+ * Forget about a prepared transaction. Called during commit/abort redo.
+ */
+void
+KnownPreparedRemoveByXid(TransactionId xid)
+{
+	dlist_mutable_iter miter;
+
+	Assert(RecoveryInProgress());
+
+	dlist_foreach_modify(miter, &KnownPreparedList)
+	{
+		KnownPreparedXact   *xact = dlist_container(KnownPreparedXact,
+														list_node, miter.cur);
+
+		if (xact->xid == xid)
+		{
+			dlist_delete(miter.cur);
+			/*
+			 * Found the entry in KnownPreparedList and we know that the file
+			 * is not on disk.
+			 */
+			return;
+		}
+	}
+
+	/*
+	 * Entry should be on disk. But aborting recovery because of absence of
+	 * unnecessary file doesn't seems to be a good idea, so call remove with
+	 * giveWarning=false. TODO: is this true?
+	 */
+	RemoveTwoPhaseFile(xid, false);
+}
+
+/*
+ *  KnownPreparedRecreateFiles
+ *
+ * Move prepare records from WAL to files. Called during checkpoint replay
+ * or in PrescanPreparedTransactions().
+ *
+ * redo_horizon = InvalidXLogRecPtr indicates that all transactions from
+ * KnownPreparedList should be moved to disk.
+ */
+void
+KnownPreparedRecreateFiles(XLogRecPtr redo_horizon)
+{
+	dlist_mutable_iter miter;
+	int			serialized_xacts = 0;
+
+	Assert(RecoveryInProgress());
+
+	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_START();
+
+	dlist_foreach_modify(miter, &KnownPreparedList)
+	{
+		KnownPreparedXact   *xact = dlist_container(KnownPreparedXact,
+										list_node, miter.cur);
+
+		if (xact->prepare_end_lsn <= redo_horizon ||
+							redo_horizon == InvalidXLogRecPtr)
+		{
+			char	   *buf;
+			int			len;
+
+			XlogReadTwoPhaseData(xact->prepare_start_lsn, &buf, &len);
+			RecreateTwoPhaseFile(xact->xid, buf, len);
+			pfree(buf);
+			dlist_delete(miter.cur);
+			serialized_xacts++;
+		}
+	}
+
+	TRACE_POSTGRESQL_TWOPHASE_CHECKPOINT_DONE();
+
+	if (log_checkpoints && serialized_xacts > 0)
+		ereport(LOG,
+				(errmsg_plural("%u two-phase state file was written "
+							   "for long-running prepared transactions",
+							   "%u two-phase state files were written "
+							   "for long-running prepared transactions",
+							   serialized_xacts,
+							   serialized_xacts)));
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index f6f136d..f87069b 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -5606,7 +5606,9 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete KnownPreparedList entry or 2PC file. */
+			KnownPreparedRemoveByXid(parsed.twophase_xid);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5626,14 +5628,18 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete KnownPreparedList entry or 2PC file. */
+			KnownPreparedRemoveByXid(parsed.twophase_xid);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		/*
+		 * Store xid and start/end pointers of the WAL record in
+		 * KnownPreparedList.
+		 */
+		KnownPreparedAdd(record);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 2f5d603..53c65e6 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -9573,6 +9573,15 @@ xlog_redo(XLogReaderState *record)
 					(errmsg("unexpected timeline ID %u (should be %u) in checkpoint record",
 							checkPoint.ThisTimeLineID, ThisTimeLineID)));
 
+
+		/*
+		 * Move prepared transactions, if any, from KnownPreparedList to files.
+		 * It is possible to skip this step and teach subsequent code about
+		 * KnownPreparedList, but PrescanPreparedTransactions() happens once
+		 * during end of recovery or on promote, so probably it isn't worth
+		 * the additional code.
+		 */
+		KnownPreparedRecreateFiles(checkPoint.redo);
 		RecoveryRestartPoint(&checkPoint);
 	}
 	else if (info == XLOG_END_OF_RECOVERY)
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b2b7848..817bb9e 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -15,6 +15,7 @@
 #define TWOPHASE_H
 
 #include "access/xlogdefs.h"
+#include "access/xlogreader.h"
 #include "datatype/timestamp.h"
 #include "storage/lock.h"
 
@@ -56,4 +57,8 @@ extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void KnownPreparedAdd(XLogReaderState *record);
+extern void KnownPreparedRemoveByXid(TransactionId xid);
+extern void KnownPreparedRecreateFiles(XLogRecPtr redo_horizon);
+
 #endif   /* TWOPHASE_H */
diff --git a/src/test/recovery/t/009_twophase.pl b/src/test/recovery/t/009_twophase.pl
new file mode 100755
index 0000000..fff7025
--- /dev/null
+++ b/src/test/recovery/t/009_twophase.pl
@@ -0,0 +1,315 @@
+# Tests dedicated to two-phase commit in recovery
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 13;
+
+# Setup master node
+my $node_master = get_new_node("master");
+$node_master->init(allows_streaming => 1);
+$node_master->append_conf('postgresql.conf', qq(
+	max_prepared_transactions = 10
+	log_checkpoints = true
+));
+$node_master->start;
+$node_master->backup('master_backup');
+$node_master->psql('postgres', "create table t(id int)");
+
+# Setup slave node
+my $node_slave = get_new_node('slave');
+$node_slave->init_from_backup($node_master, 'master_backup', has_streaming => 1);
+$node_slave->start;
+
+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+	synchronous_standby_names = '*'
+));
+$node_master->psql('postgres', "select pg_reload_conf()");
+
+my $psql_out = '';
+my $psql_rc = '';
+
+###############################################################################
+# Check that we can commit and abort tx after soft restart.
+# Here checkpoint happens before shutdown and no WAL replay will occur at next
+# startup. In this case postgres re-creates shared-memory state from twophase
+# files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	savepoint s1;
+	insert into t values (143);
+	prepare transaction 'y';");
+$node_master->stop;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared transaction after restart.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared transaction after restart.');
+
+###############################################################################
+# Check that we can commit and abort after hard restart.
+# At next startup, WAL replay will re-create shared memory state for prepared
+# transaction using dedicated WAL records.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	savepoint s1;
+	insert into t values (143);
+	prepare transaction 'y';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared transaction after teardown.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared transaction after teardown.');
+
+###############################################################################
+# Check that WAL replay can handle several transactions with same GID name.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Replay several transactions with same GID.');
+
+###############################################################################
+# Check that WAL replay cleans up its shared memory state and releases locks
+# while replaying transaction commits.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_master->teardown_node;
+$node_master->start;
+$psql_rc = $node_master->psql('postgres', "begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	-- This prepare can fail due to conflicting GID or locks conflicts if
+	-- replay did not fully cleanup its state on previous commit.
+	prepare transaction 'x';");
+is($psql_rc, '0', "Cleanup of shared memory state for 2PC commit");
+
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that WAL replay will cleanup its shared memory state on running slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;",
+	  stdout => \$psql_out);
+is($psql_out, '0',
+   "Cleanup of shared memory state on running standby without checkpoint.");
+
+###############################################################################
+# Same as in previous case, but let's force checkpoint on slave between
+# prepare and commit to use on-disk twophase files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_slave->psql('postgres', "checkpoint;");
+$node_master->psql('postgres', "commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;",
+	  stdout => \$psql_out);
+is($psql_out, '0',
+   "Cleanup of shared memory state on running standby after checkpoint.");
+
+###############################################################################
+# Check that prepared transactions can be committed on promoted slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$psql_rc = $node_slave->psql('postgres', "commit prepared 'x';");
+is($psql_rc, '0', "Restore of prepared transaction on promoted slave.");
+
+# change roles
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+
+###############################################################################
+# Check that prepared transactions are replayed after soft restart of standby
+# while master is down. Since standby knows that master is down it uses
+# different code path on start to be sure that the status of transactions is
+# consistent.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->stop;
+$node_slave->restart;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '1',
+   "Restore prepared transactions from files with master down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that prepared transactions are correctly replayed after slave hard
+# restart while master is down.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (242);
+	savepoint s1;
+	insert into t values (243);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->teardown_node;
+$node_slave->start;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres',
+	  "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '1',
+   "Restore prepared transactions from records with master down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres', "commit prepared 'x'");
+
+
+###############################################################################
+# Check for a lock conflict between prepared tx with DDL inside and replay of
+# XLOG_STANDBY_LOCK wal record.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	create table t2(id int);
+	savepoint s1;
+	insert into t2 values (42);
+	prepare transaction 'x';
+	-- checkpoint will issue XLOG_STANDBY_LOCK that can conflict with lock
+	-- held by 'create table' statement
+	checkpoint;
+	commit prepared 'x';");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '0', "Replay prepared transaction with DDL.");
+
+
+###############################################################################
+# Check that replay will correctly set SUBTRANS and properly advance nextXid
+# so it won't conflict with savepoint xids.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	delete from t;
+	insert into t values (43);
+	savepoint s1;
+	insert into t values (43);
+	savepoint s2;
+	insert into t values (43);
+	savepoint s3;
+	insert into t values (43);
+	savepoint s4;
+	insert into t values (43);
+	savepoint s5;
+	insert into t values (43);
+	prepare transaction 'x';
+	checkpoint;");
+
+$node_master->stop;
+$node_master->start;
+$node_master->psql('postgres', "
+	-- here we can get xid of previous savepoint if nextXid
+	-- wasn't properly advanced
+	begin;
+	insert into t values (142);
+	abort;
+	commit prepared 'x';");
+
+$node_master->psql('postgres', "select count(*) from t",
+	  stdout => \$psql_out);
+is($psql_out, '6', "Check nextXid handling for prepared subtransactions");
#113Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#112)
Re: Speedup twophase transactions

On Mon, Jan 23, 2017 at 9:00 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

Hi Stas,

So, here is brand new implementation of the same thing.

Now instead of creating pgproc entry for prepared transaction during recovery,
I just store recptr/xid correspondence in separate 2L-list and
deleting entries in that
list if redo process faced commit/abort. In case of checkpoint or end
of recovery
transactions remaining in that list are dumped to files in pg_twophase.

Seems that current approach is way more simpler and patch has two times less
LOCs then previous one.

The proposed approach and patch does appear to be much simpler than
the previous one.

I have some comments/questions on the patch (twophase_recovery_list_2.diff):

1)
+       /*
+        * Move prepared transactions from KnownPreparedList to files, if any.
+        * It is possible to skip that step and teach subsequent code about
+        * KnownPreparedList, but whole PrescanPreparedTransactions() happens
+        * once during end of recovery or promote, so probably it isn't worth
+        * complications.
+        */
+       KnownPreparedRecreateFiles(InvalidXLogRecPtr);
+

Speeding up recovery or failover activity via a faster promote is a
desirable thing. So, maybe, we should look at teaching the relevant
code about using "KnownPreparedList"? I know that would increase the
size of this patch and would mean more testing, but this seems to be
last remaining optimization in this code path.

That's a good idea, worth having in this patch. Actually we may not
want to call KnownPreparedRecreateFiles() here as promotion is not
synonym of end-of-recovery checkpoint for a couple of releases now.

3) We are pushing the fsyncing of 2PC files to the checkpoint replay
activity with this patch. Now, typically, we would assume that PREPARE
followed by COMMIT/ABORT would happen within a checkpoint replay
cycle, if not, this would make checkpoint replays slower since the
earlier spread out fsyncs are now getting bunched up. I had concerns
about this but clearly your latest performance measurements don't show
any negatives around this directly.

Most of the 2PC syncs just won't happen, such transactions normally
don't last long, and the number you would get during a checkpoint is
largely lower than what would happen between two checkpoints. When
working on Postgres-XC, the number of 2PC was really more funky.

6) Do we need to hold TwoPhaseStateLock lock in shared mode while
calling KnownPreparedRecreateFiles()? I think, it does not matter in
the recovery/replay code path.

It does not matter as the only code path tracking that would be the
checkpointer in TwoPhaseCheckpoint(), and this 2PC recovery patch does
not touch anymore TwoPhaseStateData. Actually the patch makes no use
of it.

7) I fixed some typos and comments. PFA, patch attached.

Other than this, I ran TAP tests and they succeed as needed.

On 23 January 2017 at 16:56, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:

I did tests with following setup:

* Start postgres initialised with pgbench
* Start pg_receivexlog
* Take basebackup
* Perform 1.5 M transactions
* Stop everything and apply wal files stored by pg_receivexlog to base backup.

All tests performed on a laptop with nvme ssd
number of transactions: 1.5M
start segment: 0x4

-master non-2pc:
last segment: 0x1b
average recovery time per 16 wal files: 11.8s
average total recovery time: 17.0s

-master 2pc:
last segment: 0x44
average recovery time per 16 wal files: 142s
average total recovery time: 568s

-patched 2pc (previous patch):
last segment: 0x44
average recovery time per 16 wal files: 5.3s
average total recovery time: 21.2s

-patched2 2pc (dlist_push_tail changed to dlist_push_head):
last segment: 0x44
average recovery time per 16 wal files: 5.2s
average total recovery time: 20.8s

The difference between those two is likely noise.

By the way, in those measurements, the OS cache is still filled with
the past WAL segments, which is a rather best case, no? What happens
if you do the same kind of tests on a box where memory is busy doing
something else and replayed WAL segments get evicted from the OS cache
more aggressively once the startup process switches to a new segment?
This could be tested for example on a VM with few memory (say 386MB or
less) so as the startup process needs to access again the past WAL
segments to recover the 2PC information it needs to get them back
directly from disk... One trick that you could use here would be to
tweak the startup process so as it drops the OS cache once a segment
is finished replaying, and see the effects of an aggressive OS cache
eviction. This patch is showing really nice improvements with the OS
cache backing up the data, still it would make sense to test things
with a worse test case and see if things could be done better. The
startup process now only reads records sequentially, not randomly
which is a concept that this patch introduces.

Anyway, perhaps this does not matter much, the non-recovery code path
does the same thing as this patch, and the improvement is too much to
be ignored. So for consistency's sake we could go with the approach
proposed which has the advantage to not put any restriction on the
size of the 2PC file contrary to what an implementation saving the
contents of the 2PC files into memory would need to do.

So skipping unnecessary fsyncs gave us x25 speed increase even on ssd (on hdd difference should be bigger).
Pushing to list's head didn’t yield measurable results, but anyway seems to be conceptually better.

That's in the logic of more recent segments getting priority as
startup process reads the records sequentially, so it is definitely
better.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#114Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#113)
Re: Speedup twophase transactions

Speeding up recovery or failover activity via a faster promote is a
desirable thing. So, maybe, we should look at teaching the relevant
code about using "KnownPreparedList"? I know that would increase the
size of this patch and would mean more testing, but this seems to be
last remaining optimization in this code path.

That's a good idea, worth having in this patch. Actually we may not
want to call KnownPreparedRecreateFiles() here as promotion is not
synonym of end-of-recovery checkpoint for a couple of releases now.

Once implemented, a good way to performance test this could be to set
checkpoint_timeout to a a large value like an hour. Then, generate
enough 2PC WAL while ensuring that a checkpoint does not happen
automatically or otherwise.

We could then measure the time taken to recover on startup to see the efficacy.

Most of the 2PC syncs just won't happen, such transactions normally
don't last long, and the number you would get during a checkpoint is
largely lower than what would happen between two checkpoints. When
working on Postgres-XC, the number of 2PC was really more funky.

Yes, postgres-xl is full of 2PC, so hopefully this optimization should
help a lot in that case as well.

Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#115Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#113)
Re: Speedup twophase transactions

On 24 Jan 2017, at 09:42, Michael Paquier <michael.paquier@gmail.com> wrote:

On Mon, Jan 23, 2017 at 9:00 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

Speeding up recovery or failover activity via a faster promote is a
desirable thing. So, maybe, we should look at teaching the relevant
code about using "KnownPreparedList"? I know that would increase the
size of this patch and would mean more testing, but this seems to be
last remaining optimization in this code path.

That's a good idea, worth having in this patch. Actually we may not
want to call KnownPreparedRecreateFiles() here as promotion is not
synonym of end-of-recovery checkpoint for a couple of releases now.

Thanks for review, Nikhil and Michael.

I don’t follow here. We are moving data away from WAL to files on checkpoint because after checkpoint
there is no guaranty that WAL segment with our prepared tx will be still available.

The difference between those two is likely noise.

By the way, in those measurements, the OS cache is still filled with
the past WAL segments, which is a rather best case, no? What happens
if you do the same kind of tests on a box where memory is busy doing
something else and replayed WAL segments get evicted from the OS cache
more aggressively once the startup process switches to a new segment?
This could be tested for example on a VM with few memory (say 386MB or
less) so as the startup process needs to access again the past WAL
segments to recover the 2PC information it needs to get them back
directly from disk... One trick that you could use here would be to
tweak the startup process so as it drops the OS cache once a segment
is finished replaying, and see the effects of an aggressive OS cache
eviction. This patch is showing really nice improvements with the OS
cache backing up the data, still it would make sense to test things
with a worse test case and see if things could be done better. The
startup process now only reads records sequentially, not randomly
which is a concept that this patch introduces.

Anyway, perhaps this does not matter much, the non-recovery code path
does the same thing as this patch, and the improvement is too much to
be ignored. So for consistency's sake we could go with the approach
proposed which has the advantage to not put any restriction on the
size of the 2PC file contrary to what an implementation saving the
contents of the 2PC files into memory would need to do.

Maybe i’m missing something, but I don’t see how OS cache can affect something here.

Total WAL size was 0x44 * 16 = 1088 MB, recovery time is about 20s. Sequential reading 1GB of data
is order of magnitude faster even on the old hdd, not speaking of ssd. Also you can take a look on flame graphs
attached to previous message — majority of time during recovery spent in pg_qsort while replaying
PageRepairFragmentation, while whole xact_redo_commit() takes about 1% of time. That amount can
grow in case of uncached disk read but taking into account total recovery time this should not affect much.

If you are talking about uncached access only during checkpoint than here we are restricted with
max_prepared_transaction, so at max we will read about hundred of small files (usually fitting into one filesystem page) which will also
be barely noticeable comparing to recovery time between checkpoints. Also wal segments cache eviction during
replay doesn’t seems to me as standard scenario.

Anyway i took the machine with hdd to slow down read speed and run tests again. During one of the runs i
launched in parallel bash loop that was dropping os cache each second (while wal fragment replay takes
also about one second).

1.5M transactions
start segment: 0x06
last segment: 0x47

patched, with constant cache_drop:
total recovery time: 86s

patched, without constant cache_drop:
total recovery time: 68s

(while difference is significant, i bet that happens mostly because of database file segments should be re-read after cache drop)

master, without constant cache_drop:
time to recover 35 segments: 2h 25m (after that i tired to wait)
expected total recovery time: 4.5 hours

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#116Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Stas Kelvich (#115)
Re: Speedup twophase transactions

Thanks for review, Nikhil and Michael.

I don’t follow here. We are moving data away from WAL to files on checkpoint because after checkpoint
there is no guaranty that WAL segment with our prepared tx will be still available.

We are talking about the recovery/promote code path. Specifically this
call to KnownPreparedRecreateFiles() in PrescanPreparedTransactions().

We write the files to disk and they get immediately read up in the
following code. We could not write the files to disk and read
KnownPreparedList in the code path that follows as well as elsewhere.

Regards,
Nikhils

The difference between those two is likely noise.

By the way, in those measurements, the OS cache is still filled with
the past WAL segments, which is a rather best case, no? What happens
if you do the same kind of tests on a box where memory is busy doing
something else and replayed WAL segments get evicted from the OS cache
more aggressively once the startup process switches to a new segment?
This could be tested for example on a VM with few memory (say 386MB or
less) so as the startup process needs to access again the past WAL
segments to recover the 2PC information it needs to get them back
directly from disk... One trick that you could use here would be to
tweak the startup process so as it drops the OS cache once a segment
is finished replaying, and see the effects of an aggressive OS cache
eviction. This patch is showing really nice improvements with the OS
cache backing up the data, still it would make sense to test things
with a worse test case and see if things could be done better. The
startup process now only reads records sequentially, not randomly
which is a concept that this patch introduces.

Anyway, perhaps this does not matter much, the non-recovery code path
does the same thing as this patch, and the improvement is too much to
be ignored. So for consistency's sake we could go with the approach
proposed which has the advantage to not put any restriction on the
size of the 2PC file contrary to what an implementation saving the
contents of the 2PC files into memory would need to do.

Maybe i’m missing something, but I don’t see how OS cache can affect something here.

Total WAL size was 0x44 * 16 = 1088 MB, recovery time is about 20s. Sequential reading 1GB of data
is order of magnitude faster even on the old hdd, not speaking of ssd. Also you can take a look on flame graphs
attached to previous message — majority of time during recovery spent in pg_qsort while replaying
PageRepairFragmentation, while whole xact_redo_commit() takes about 1% of time. That amount can
grow in case of uncached disk read but taking into account total recovery time this should not affect much.

If you are talking about uncached access only during checkpoint than here we are restricted with
max_prepared_transaction, so at max we will read about hundred of small files (usually fitting into one filesystem page) which will also
be barely noticeable comparing to recovery time between checkpoints. Also wal segments cache eviction during
replay doesn’t seems to me as standard scenario.

Anyway i took the machine with hdd to slow down read speed and run tests again. During one of the runs i
launched in parallel bash loop that was dropping os cache each second (while wal fragment replay takes
also about one second).

1.5M transactions
start segment: 0x06
last segment: 0x47

patched, with constant cache_drop:
total recovery time: 86s

patched, without constant cache_drop:
total recovery time: 68s

(while difference is significant, i bet that happens mostly because of database file segments should be re-read after cache drop)

master, without constant cache_drop:
time to recover 35 segments: 2h 25m (after that i tired to wait)
expected total recovery time: 4.5 hours

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#117Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Nikhil Sontakke (#116)
Re: Speedup twophase transactions

We are talking about the recovery/promote code path. Specifically this
call to KnownPreparedRecreateFiles() in PrescanPreparedTransactions().

We write the files to disk and they get immediately read up in the
following code. We could not write the files to disk and read
KnownPreparedList in the code path that follows as well as elsewhere.

Thinking more on this.

The only optimization that's really remaining is handling of prepared
transactions that have not been committed or will linger around for
long. The short lived 2PC transactions have been optimized already via
this patch.

The question remains whether saving off a few fsyncs/reads for these
long-lived prepared transactions is worth the additional code churn.
Even if we add code to go through the KnownPreparedList, we still will
have to go through the other on-disk 2PC transactions anyways. So,
maybe not.

Regards,
Nikhils

Regards,
Nikhils

The difference between those two is likely noise.

By the way, in those measurements, the OS cache is still filled with
the past WAL segments, which is a rather best case, no? What happens
if you do the same kind of tests on a box where memory is busy doing
something else and replayed WAL segments get evicted from the OS cache
more aggressively once the startup process switches to a new segment?
This could be tested for example on a VM with few memory (say 386MB or
less) so as the startup process needs to access again the past WAL
segments to recover the 2PC information it needs to get them back
directly from disk... One trick that you could use here would be to
tweak the startup process so as it drops the OS cache once a segment
is finished replaying, and see the effects of an aggressive OS cache
eviction. This patch is showing really nice improvements with the OS
cache backing up the data, still it would make sense to test things
with a worse test case and see if things could be done better. The
startup process now only reads records sequentially, not randomly
which is a concept that this patch introduces.

Anyway, perhaps this does not matter much, the non-recovery code path
does the same thing as this patch, and the improvement is too much to
be ignored. So for consistency's sake we could go with the approach
proposed which has the advantage to not put any restriction on the
size of the 2PC file contrary to what an implementation saving the
contents of the 2PC files into memory would need to do.

Maybe i’m missing something, but I don’t see how OS cache can affect something here.

Total WAL size was 0x44 * 16 = 1088 MB, recovery time is about 20s. Sequential reading 1GB of data
is order of magnitude faster even on the old hdd, not speaking of ssd. Also you can take a look on flame graphs
attached to previous message — majority of time during recovery spent in pg_qsort while replaying
PageRepairFragmentation, while whole xact_redo_commit() takes about 1% of time. That amount can
grow in case of uncached disk read but taking into account total recovery time this should not affect much.

If you are talking about uncached access only during checkpoint than here we are restricted with
max_prepared_transaction, so at max we will read about hundred of small files (usually fitting into one filesystem page) which will also
be barely noticeable comparing to recovery time between checkpoints. Also wal segments cache eviction during
replay doesn’t seems to me as standard scenario.

Anyway i took the machine with hdd to slow down read speed and run tests again. During one of the runs i
launched in parallel bash loop that was dropping os cache each second (while wal fragment replay takes
also about one second).

1.5M transactions
start segment: 0x06
last segment: 0x47

patched, with constant cache_drop:
total recovery time: 86s

patched, without constant cache_drop:
total recovery time: 68s

(while difference is significant, i bet that happens mostly because of database file segments should be re-read after cache drop)

master, without constant cache_drop:
time to recover 35 segments: 2h 25m (after that i tired to wait)
expected total recovery time: 4.5 hours

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#118Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#117)
Re: Speedup twophase transactions

On Wed, Jan 25, 2017 at 11:55 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

We are talking about the recovery/promote code path. Specifically this
call to KnownPreparedRecreateFiles() in PrescanPreparedTransactions().

We write the files to disk and they get immediately read up in the
following code. We could not write the files to disk and read
KnownPreparedList in the code path that follows as well as elsewhere.

Thinking more on this.

The only optimization that's really remaining is handling of prepared
transactions that have not been committed or will linger around for
long. The short lived 2PC transactions have been optimized already via
this patch.

The question remains whether saving off a few fsyncs/reads for these
long-lived prepared transactions is worth the additional code churn.
Even if we add code to go through the KnownPreparedList, we still will
have to go through the other on-disk 2PC transactions anyways. So,
maybe not.

We should really try to do things right now, or we'll never come back
to it. 9.3 (if my memory does not fail me?) has reduced the time to do
promotion by removing the need of the end-of-recovery checkpoint,
while I agree that there should not be that many 2PC transactions at
this point, if there are for a reason or another, the time it takes to
complete promotion would be impacted. So let's refactor
PrescanPreparedTransactions() so as it is able to handle 2PC data from
a buffer extracted by XlogReadTwoPhaseData(), and we are good to go.

--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -9573,6 +9573,15 @@ xlog_redo(XLogReaderState *record)
                    (errmsg("unexpected timeline ID %u (should be %u)
in checkpoint record",
                            checkPoint.ThisTimeLineID, ThisTimeLineID)));
+
+       /*
+        * Move prepared transactions, if any, from KnownPreparedList to files.
+        * It is possible to skip this step and teach subsequent code about
+        * KnownPreparedList, but PrescanPreparedTransactions() happens once
+        * during end of recovery or on promote, so probably it isn't worth
+        * the additional code.
+        */
+       KnownPreparedRecreateFiles(checkPoint.redo);
        RecoveryRestartPoint(&checkPoint);
Looking again at this code, I think that this is incorrect. The
checkpointer should be in charge of doing this work and not the
startup process, so this should go into CheckpointTwoPhase() instead.
At the end, we should be able to just live without
KnownPreparedRecreateFiles() and just rip it off from the patch.
-- 
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#119Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#118)
Re: Speedup twophase transactions

The question remains whether saving off a few fsyncs/reads for these
long-lived prepared transactions is worth the additional code churn.
Even if we add code to go through the KnownPreparedList, we still will
have to go through the other on-disk 2PC transactions anyways. So,
maybe not.

We should really try to do things right now, or we'll never come back
to it. 9.3 (if my memory does not fail me?) has reduced the time to do
promotion by removing the need of the end-of-recovery checkpoint,
while I agree that there should not be that many 2PC transactions at
this point, if there are for a reason or another, the time it takes to
complete promotion would be impacted. So let's refactor
PrescanPreparedTransactions() so as it is able to handle 2PC data from
a buffer extracted by XlogReadTwoPhaseData(), and we are good to go.

Not quite. If we modify PrescanPreparedTransactions(), we also need to
make RecoverPreparedTransactions() and
StandbyRecoverPreparedTransactions() handle 2PC data via
XlogReadTwoPhaseData().

+       /*
+        * Move prepared transactions, if any, from KnownPreparedList to files.
+        * It is possible to skip this step and teach subsequent code about
+        * KnownPreparedList, but PrescanPreparedTransactions() happens once
+        * during end of recovery or on promote, so probably it isn't worth
+        * the additional code.
+        */

This comment is misplaced. Does not make sense before this specific call.

+ KnownPreparedRecreateFiles(checkPoint.redo);
RecoveryRestartPoint(&checkPoint);
Looking again at this code, I think that this is incorrect. The
checkpointer should be in charge of doing this work and not the
startup process, so this should go into CheckpointTwoPhase() instead.

I don't see a function by the above name in the code?

Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#120Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#119)
Re: Speedup twophase transactions

On Thu, Jan 26, 2017 at 1:38 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

We should really try to do things right now, or we'll never come back
to it. 9.3 (if my memory does not fail me?) has reduced the time to do
promotion by removing the need of the end-of-recovery checkpoint,
while I agree that there should not be that many 2PC transactions at
this point, if there are for a reason or another, the time it takes to
complete promotion would be impacted. So let's refactor
PrescanPreparedTransactions() so as it is able to handle 2PC data from
a buffer extracted by XlogReadTwoPhaseData(), and we are good to go.

Not quite. If we modify PrescanPreparedTransactions(), we also need to
make RecoverPreparedTransactions() and
StandbyRecoverPreparedTransactions() handle 2PC data via
XlogReadTwoPhaseData().

Ah, right for both, even for RecoverPreparedTransactions() that
happens at the end of recovery. Thanks for noticing. The patch
mentions that as well:
+ *    * At the end of recovery we move all known prepared transactions to disk.
+ *      This allows RecoverPreparedTransactions() and
+ *      StandbyRecoverPreparedTransactions() to do their work.
I need some strong coffee..

+ KnownPreparedRecreateFiles(checkPoint.redo);
RecoveryRestartPoint(&checkPoint);
Looking again at this code, I think that this is incorrect. The
checkpointer should be in charge of doing this work and not the
startup process, so this should go into CheckpointTwoPhase() instead.

I don't see a function by the above name in the code?

I look at this patch from you and that's present for me:
/messages/by-id/CAMGcDxf8Bn9ZPBBJZba9wiyQq-Qk5uqq=VjoMnRnW5s+fKST3w@mail.gmail.com
If I look as well at the last version of Stas it is here:
/messages/by-id/BECC988A-DB74-48D5-B5D5-A54551A6242A@postgrespro.ru

As this change:
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -9573,6 +9573,7 @@ xlog_redo(XLogReaderState *record)
                     (errmsg("unexpected timeline ID %u (should be %u)
in checkpoint record",
                             checkPoint.ThisTimeLineID, ThisTimeLineID)));
+        KnownPreparedRecreateFiles(checkPoint.redo);
         RecoveryRestartPoint(&checkPoint);
     }
And actually, when a XLOG_CHECKPOINT_SHUTDOWN record is taken, 2PC
files are not flushed to disk with this patch. This is a problem as a
new restart point is created... Having the flush in CheckpointTwoPhase
really makes the most sense.
-- 
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#121Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#120)
Re: Speedup twophase transactions

I look at this patch from you and that's present for me:
/messages/by-id/CAMGcDxf8Bn9ZPBBJZba9wiyQq-&gt;Qk5uqq=VjoMnRnW5s+fKST3w@mail.gmail.com

--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -9573,6 +9573,7 @@ xlog_redo(XLogReaderState *record)
(errmsg("unexpected timeline ID %u (should be %u)
in checkpoint record",
checkPoint.ThisTimeLineID, ThisTimeLineID)));

+ KnownPreparedRecreateFiles(checkPoint.redo);
RecoveryRestartPoint(&checkPoint);
}

Oh, sorry. I was asking about CheckpointTwoPhase(). I don't see a
function by this name. And now I see, the name is CheckPointTwoPhase()
:-)

And actually, when a XLOG_CHECKPOINT_SHUTDOWN record is taken, 2PC
files are not flushed to disk with this patch. This is a problem as a
new restart point is created... Having the flush in CheckpointTwoPhase
really makes the most sense.

Umm, AFAICS, CheckPointTwoPhase() does not get called in the "standby
promote" code path.

Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#122Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#121)
Re: Speedup twophase transactions

On Thu, Jan 26, 2017 at 4:09 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

I look at this patch from you and that's present for me:
/messages/by-id/CAMGcDxf8Bn9ZPBBJZba9wiyQq-&gt;Qk5uqq=VjoMnRnW5s+fKST3w@mail.gmail.com

--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -9573,6 +9573,7 @@ xlog_redo(XLogReaderState *record)
(errmsg("unexpected timeline ID %u (should be %u)
in checkpoint record",
checkPoint.ThisTimeLineID, ThisTimeLineID)));

+ KnownPreparedRecreateFiles(checkPoint.redo);
RecoveryRestartPoint(&checkPoint);
}

Oh, sorry. I was asking about CheckpointTwoPhase(). I don't see a
function by this name. And now I see, the name is CheckPointTwoPhase()
:-)

My mistake then :D

And actually, when a XLOG_CHECKPOINT_SHUTDOWN record is taken, 2PC
files are not flushed to disk with this patch. This is a problem as a
new restart point is created... Having the flush in CheckpointTwoPhase
really makes the most sense.

Umm, AFAICS, CheckPointTwoPhase() does not get called in the "standby
promote" code path.

CreateRestartPoint() calls it via CheckPointGuts() while in recovery.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#123Stas Kelvich
s.kelvich@postgrespro.ru
In reply to: Michael Paquier (#122)
Re: Speedup twophase transactions

On 26 Jan 2017, at 10:34, Michael Paquier <michael.paquier@gmail.com> wrote:

On Thu, Jan 26, 2017 at 4:09 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

I look at this patch from you and that's present for me:
/messages/by-id/CAMGcDxf8Bn9ZPBBJZba9wiyQq-&gt;Qk5uqq=VjoMnRnW5s+fKST3w@mail.gmail.com

--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -9573,6 +9573,7 @@ xlog_redo(XLogReaderState *record)
(errmsg("unexpected timeline ID %u (should be %u)
in checkpoint record",
checkPoint.ThisTimeLineID, ThisTimeLineID)));

+ KnownPreparedRecreateFiles(checkPoint.redo);
RecoveryRestartPoint(&checkPoint);
}

Oh, sorry. I was asking about CheckpointTwoPhase(). I don't see a
function by this name. And now I see, the name is CheckPointTwoPhase()
:-)

My mistake then :D

And actually, when a XLOG_CHECKPOINT_SHUTDOWN record is taken, 2PC
files are not flushed to disk with this patch. This is a problem as a
new restart point is created... Having the flush in CheckpointTwoPhase
really makes the most sense.

Umm, AFAICS, CheckPointTwoPhase() does not get called in the "standby
promote" code path.

CreateRestartPoint() calls it via CheckPointGuts() while in recovery.

Huh, glad that this tread received a lot of attention.

On 24 Jan 2017, at 17:26, Nikhil Sontakke <nikhils@2ndquadrant.com> wrote:

We are talking about the recovery/promote code path. Specifically this
call to KnownPreparedRecreateFiles() in PrescanPreparedTransactions().

We write the files to disk and they get immediately read up in the
following code. We could not write the files to disk and read
KnownPreparedList in the code path that follows as well as elsewhere.

Thanks Nikhil, now I got that. Since we are talking about promotion we are on different timescale and 1-10 second
lag matters a lot.

I think I have in my mind realistic scenario when proposed recovery code path will hit the worst case: Google cloud.
They have quite fast storage, but fsync time is really big and can go up to 10-100ms (i suppose it is network-attacheble).
Having say 300 prepared tx, we can delay promotion up to half minute.

So i think it worth of examination.

--
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#124Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#122)
Re: Speedup twophase transactions

Umm, AFAICS, CheckPointTwoPhase() does not get called in the "standby
promote" code path.

CreateRestartPoint() calls it via CheckPointGuts() while in recovery.

May be that I am missing something.

But, I put the recovery process and the checkpointer process of the
standby under gdb with breakpoints on these functions, but both did
not hit CreateRestartPoint() as well as CheckPointGuts() when I issued
a promote :-|

Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#125Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#124)
Re: Speedup twophase transactions

On Thu, Jan 26, 2017 at 5:02 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

Umm, AFAICS, CheckPointTwoPhase() does not get called in the "standby
promote" code path.

CreateRestartPoint() calls it via CheckPointGuts() while in recovery.

May be that I am missing something.

But, I put the recovery process and the checkpointer process of the
standby under gdb with breakpoints on these functions, but both did
not hit CreateRestartPoint() as well as CheckPointGuts() when I issued
a promote :-|

No end-of-recovery checkpoints happen at promotion since 9.3. You can
still use fallback_promote as promote file to trigger the pre-9.2 (9.2
included) behavior.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#126Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#125)
Re: Speedup twophase transactions

But, I put the recovery process and the checkpointer process of the
standby under gdb with breakpoints on these functions, but both did
not hit CreateRestartPoint() as well as CheckPointGuts() when I issued
a promote :-|

No end-of-recovery checkpoints happen at promotion since 9.3. You can
still use fallback_promote as promote file to trigger the pre-9.2 (9.2
included) behavior.

Ok, so that means, we also need to fsync out these 2PC XIDs at promote
time as well for their durability.

Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#127Simon Riggs
simon@2ndquadrant.com
In reply to: Nikhil Sontakke (#126)
Re: Speedup twophase transactions

On 27 January 2017 at 09:59, Nikhil Sontakke <nikhils@2ndquadrant.com> wrote:

But, I put the recovery process and the checkpointer process of the
standby under gdb with breakpoints on these functions, but both did
not hit CreateRestartPoint() as well as CheckPointGuts() when I issued
a promote :-|

No end-of-recovery checkpoints happen at promotion since 9.3. You can
still use fallback_promote as promote file to trigger the pre-9.2 (9.2
included) behavior.

Ok, so that means, we also need to fsync out these 2PC XIDs at promote
time as well for their durability.

Why? The data files haven't been fsynced either at that point.

If there is a bug there it doesn't just affect 2PC.

What sequence of actions would cause data loss?

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#128Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Simon Riggs (#127)
Re: Speedup twophase transactions

On 27 January 2017 at 15:37, Simon Riggs <simon@2ndquadrant.com> wrote:

On 27 January 2017 at 09:59, Nikhil Sontakke <nikhils@2ndquadrant.com> wrote:

But, I put the recovery process and the checkpointer process of the
standby under gdb with breakpoints on these functions, but both did
not hit CreateRestartPoint() as well as CheckPointGuts() when I issued
a promote :-|

No end-of-recovery checkpoints happen at promotion since 9.3. You can
still use fallback_promote as promote file to trigger the pre-9.2 (9.2
included) behavior.

Ok, so that means, we also need to fsync out these 2PC XIDs at promote
time as well for their durability.

Why?

The xact_redo code will add prepared transactions to the
KnownPreparedList in memory. Earlier it used to create the on-disk 2PC
file.

At standby promote, the surviving (yet uncommitted) prepared
transactions from KnownPreparedList need to be persisted, right?

Regards,
Nikhils

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#129Simon Riggs
simon@2ndquadrant.com
In reply to: Nikhil Sontakke (#128)
Re: Speedup twophase transactions

On 27 January 2017 at 11:01, Nikhil Sontakke <nikhils@2ndquadrant.com> wrote:

On 27 January 2017 at 15:37, Simon Riggs <simon@2ndquadrant.com> wrote:

On 27 January 2017 at 09:59, Nikhil Sontakke <nikhils@2ndquadrant.com> wrote:

But, I put the recovery process and the checkpointer process of the
standby under gdb with breakpoints on these functions, but both did
not hit CreateRestartPoint() as well as CheckPointGuts() when I issued
a promote :-|

No end-of-recovery checkpoints happen at promotion since 9.3. You can
still use fallback_promote as promote file to trigger the pre-9.2 (9.2
included) behavior.

Ok, so that means, we also need to fsync out these 2PC XIDs at promote
time as well for their durability.

Why?

The xact_redo code will add prepared transactions to the
KnownPreparedList in memory. Earlier it used to create the on-disk 2PC
file.

At standby promote, the surviving (yet uncommitted) prepared
transactions from KnownPreparedList need to be persisted, right?

I don't see why, so please explain or show the error that can be
caused if we don't.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#130Michael Paquier
michael.paquier@gmail.com
In reply to: Simon Riggs (#129)
Re: Speedup twophase transactions

On Fri, Jan 27, 2017 at 8:23 PM, Simon Riggs <simon@2ndquadrant.com> wrote:

On 27 January 2017 at 11:01, Nikhil Sontakke <nikhils@2ndquadrant.com> wrote:

The xact_redo code will add prepared transactions to the
KnownPreparedList in memory. Earlier it used to create the on-disk 2PC
file.

At standby promote, the surviving (yet uncommitted) prepared
transactions from KnownPreparedList need to be persisted, right?

I don't see why, so please explain or show the error that can be
caused if we don't.

I agree with Simon here. There is no point to fsync the 2PC files are
in case of an immediate crash after promotion replay will happen from
the last checkpoint, aka the one before the promotion has been
triggered. So there is no point to flush them at promotion, they would
be replayed anyway.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#131Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#130)
Re: Speedup twophase transactions

The xact_redo code will add prepared transactions to the
KnownPreparedList in memory. Earlier it used to create the on-disk 2PC
file.

At standby promote, the surviving (yet uncommitted) prepared
transactions from KnownPreparedList need to be persisted, right?

I don't see why, so please explain or show the error that can be
caused if we don't.

I agree with Simon here. There is no point to fsync the 2PC files are
in case of an immediate crash after promotion replay will happen from
the last checkpoint, aka the one before the promotion has been
triggered. So there is no point to flush them at promotion, they would
be replayed anyway.

1) start master
2) start streaming replica
3) on master

begin;

create table test1(g int);

prepare transaction 'test';

4) Promote standby via trigger file or via "pg_ctl promote"

I thought if we don't fsync the KnownPreparedList then we might not
create the 2PC state properly on the standby.

However, I do realize that we will be calling
RecoverPreparedTransactions() eventually on promote. This function
will be modified to go through the KnownPreparedList to reload shmem
appropriately for 2PC. So, all good in that case.

Apologies for the digression.

Regards,
Nikhils

--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#132Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#120)
Re: Speedup twophase transactions
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -9573,6 +9573,7 @@ xlog_redo(XLogReaderState *record)
(errmsg("unexpected timeline ID %u (should be %u)
in checkpoint record",
checkPoint.ThisTimeLineID, ThisTimeLineID)));

+ KnownPreparedRecreateFiles(checkPoint.redo);
RecoveryRestartPoint(&checkPoint);
}
And actually, when a XLOG_CHECKPOINT_SHUTDOWN record is taken, 2PC
files are not flushed to disk with this patch. This is a problem as a
new restart point is created... Having the flush in CheckpointTwoPhase
really makes the most sense.

Having CheckPointTwoPhase() do the flush would mean shifting the data
from KnownPreparedList into TwoPhaseState shmem.

I wonder what's the best location for this in the common case when we
do shutdown of standby. We could add code in XLOG_CHECKPOINT_SHUTDOWN
and XLOG_CHECKPOINT_ONLINE xlog_redo code path.

Regards,
Nikhils

--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#133Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#132)
Re: Speedup twophase transactions

On Tue, Jan 31, 2017 at 3:45 AM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -9573,6 +9573,7 @@ xlog_redo(XLogReaderState *record)
(errmsg("unexpected timeline ID %u (should be %u)
in checkpoint record",
checkPoint.ThisTimeLineID, ThisTimeLineID)));

+ KnownPreparedRecreateFiles(checkPoint.redo);
RecoveryRestartPoint(&checkPoint);
}
And actually, when a XLOG_CHECKPOINT_SHUTDOWN record is taken, 2PC
files are not flushed to disk with this patch. This is a problem as a
new restart point is created... Having the flush in CheckpointTwoPhase
really makes the most sense.

Having CheckPointTwoPhase() do the flush would mean shifting the data
from KnownPreparedList into TwoPhaseState shmem.

Er, no. For CheckPointTwoPhase(), at recovery what needs to be done is
to have all the entries in KnownPreparedList() flushed to disk and
have those entries removed while holding a shared memory lock. And for
the rest we need to be careful to have PrescanPreparedTransactions,
RecoverPreparedTransactions and StandbyRecoverPreparedTransactions
aware of entries that are in KnownPreparedList(). Let's leave the
business of putting the information from KnownPreparedList to
TwoPhaseState in RecoverPreparedTransactions, which need to be aware
of entries in KnownPreparedList() anyway. The only thing that differs
is how the 2PC information is fetched: from the segments or from the
files in pg_twophase.

I wonder what's the best location for this in the common case when we
do shutdown of standby. We could add code in XLOG_CHECKPOINT_SHUTDOWN
and XLOG_CHECKPOINT_ONLINE xlog_redo code path.

ShutdownXLOG() calls CreateRestartPoint() when a standby shuts down,
so doing all the durability work in CheckPointTwoPhase() would take
care of any problems.

(moving patch to CF 2017-03)
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#134Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#133)
Re: Speedup twophase transactions

Having CheckPointTwoPhase() do the flush would mean shifting the data
from KnownPreparedList into TwoPhaseState shmem.

Er, no. For CheckPointTwoPhase(), at recovery what needs to be done is
to have all the entries in KnownPreparedList() flushed to disk and
have those entries removed while holding a shared memory lock.

The KnownPreparedList is constructed by the recovery process.
CheckPointTwoPhase() gets called by the checkpointer process. The
checkpointer does not have access to this valid KnownPreparedList.

And for
the rest we need to be careful to have PrescanPreparedTransactions,
RecoverPreparedTransactions and StandbyRecoverPreparedTransactions
aware of entries that are in KnownPreparedList().

Yeah, that part is straightforward. It does involve duplication of the
earlier while loops to work against KnownPreparedList. A smart single
while loop which loops over the 2PC files followed by the list would
help here :-)

Let's leave the
business of putting the information from KnownPreparedList to
TwoPhaseState in RecoverPreparedTransactions, which need to be aware
of entries in KnownPreparedList() anyway. The only thing that differs
is how the 2PC information is fetched: from the segments or from the
files in pg_twophase.

Yeah. This part is also ok. We also got to be careful to mark the
shmem gxact entry with "ondisk = false" and need to set
prepare_start_lsn/prepare_end_lsn properly as well.

I wonder what's the best location for this in the common case when we
do shutdown of standby. We could add code in XLOG_CHECKPOINT_SHUTDOWN
and XLOG_CHECKPOINT_ONLINE xlog_redo code path.

ShutdownXLOG() calls CreateRestartPoint() when a standby shuts down,
so doing all the durability work in CheckPointTwoPhase() would take
care of any problems.

ShutdownXLOG() gets called from the checkpointer process. See comments
above about the checkpointer not having access to the proper
KnownPreparedList.

The following test sequence will trigger the issue:

1) start master
2) start replica
3) prepare a transaction on master
4) shutdown master
5) shutdown replica

CheckPointTwoPhase() in (5) does not sync this prepared transaction
because the checkpointer's KnownPreparedList is empty.

Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#135Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#134)
Re: Speedup twophase transactions

On Tue, Jan 31, 2017 at 2:34 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

I wonder what's the best location for this in the common case when we
do shutdown of standby. We could add code in XLOG_CHECKPOINT_SHUTDOWN
and XLOG_CHECKPOINT_ONLINE xlog_redo code path.

ShutdownXLOG() calls CreateRestartPoint() when a standby shuts down,
so doing all the durability work in CheckPointTwoPhase() would take
care of any problems.

ShutdownXLOG() gets called from the checkpointer process. See comments
above about the checkpointer not having access to the proper
KnownPreparedList.

The following test sequence will trigger the issue:

1) start master
2) start replica
3) prepare a transaction on master
4) shutdown master
5) shutdown replica

CheckPointTwoPhase() in (5) does not sync this prepared transaction
because the checkpointer's KnownPreparedList is empty.

And that's why this needs to be stored in shared memory with a number
of elements made of max_prepared_xacts...
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#136Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#135)
Re: Speedup twophase transactions

CheckPointTwoPhase() in (5) does not sync this prepared transaction
because the checkpointer's KnownPreparedList is empty.

And that's why this needs to be stored in shared memory with a number
of elements made of max_prepared_xacts...

Yeah. Was thinking about this yesterday. How about adding entries in
TwoPhaseState itself (which become valid later)? Only if it does not
cause a lot of code churn.

The current non-shmem list patch only needs to handle standby
shutdowns correctly. Other aspects like standby promotion/recovery are
handled ok AFAICS.

Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#137Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#136)
Re: Speedup twophase transactions

On Tue, Jan 31, 2017 at 2:58 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

CheckPointTwoPhase() in (5) does not sync this prepared transaction
because the checkpointer's KnownPreparedList is empty.

And that's why this needs to be stored in shared memory with a number
of elements made of max_prepared_xacts...

Yeah. Was thinking about this yesterday. How about adding entries in
TwoPhaseState itself (which become valid later)? Only if it does not
cause a lot of code churn.

That's possible as well, yes.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#138Robert Haas
robertmhaas@gmail.com
In reply to: Nikhil Sontakke (#112)
Re: Speedup twophase transactions

On Mon, Jan 23, 2017 at 7:00 AM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

4) Minor nit-pick on existing code.

(errmsg_plural("%u two-phase state file was written "
"for
long-running prepared transactions",
"%u
two-phase state files were written "
"for
long-running prepared transactions",
serialized_xacts,
serialized_xacts)

Shouldn’t the singular part of the message above be:
"%u two-phase state file was written for a long-running prepared transaction"

But, then, English is not my native language, so I might be off here :-)

If there's one file per long-running prepared transaction, which I
think is true, then I agree with you.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#139Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Robert Haas (#138)
1 attachment(s)
Re: Speedup twophase transactions

Shouldn’t the singular part of the message above be:
"%u two-phase state file was written for a long-running prepared transaction"

But, then, English is not my native language, so I might be off here :-)

If there's one file per long-running prepared transaction, which I
think is true, then I agree with you.

PFA, small patch to fix this, then.

Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Attachments:

twophase_typo.patchapplication/octet-stream; name=twophase_typo.patchDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 5b72c1d..2027975 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -1646,7 +1646,7 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
 	if (log_checkpoints && serialized_xacts > 0)
 		ereport(LOG,
 				(errmsg_plural("%u two-phase state file was written "
-							   "for long-running prepared transactions",
+							   "for a long-running prepared transaction",
 							   "%u two-phase state files were written "
 							   "for long-running prepared transactions",
 							   serialized_xacts,
#140Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#137)
1 attachment(s)
Re: Speedup twophase transactions

Yeah. Was thinking about this yesterday. How about adding entries in
TwoPhaseState itself (which become valid later)? Only if it does not
cause a lot of code churn.

That's possible as well, yes.

PFA a patch which does the above. It re-uses the TwoPhaseState gxact
entries to track 2PC PREPARE/COMMIT in shared memory. The advantage
being that CheckPointTwoPhase() becomes the only place where the fsync
of 2PC files happens.

A minor annoyance in the patch is the duplication of the code to add
the 2nd while loop to go through these shared memory entries in
PrescanPreparedTransactions, RecoverPreparedTransactions and
StandbyRecoverPreparedTransactions.

Other than this, I ran TAP tests and they succeed as needed.

Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Attachments:

twophase_recovery_shmem_020217.patchapplication/octet-stream; name=twophase_recovery_shmem_020217.patchDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 6fde2bd..b91922e 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,26 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *
+ *      During replay and replication, TwoPhaseState also holds information
+ *      about active prepared transactions that haven't been moved to disk yet.
+ *
+ *      Replay of twophase records happens by the following rules:
+ *
+ *      * On PREPARE redo we add the transaction to TwoPhaseState->prepXacts.
+ *        We set gxact->inredo to true for such entries.
+ *
+ *      * On Checkpoint we iterate through TwoPhaseState->prepXacts entries
+ *        that have gxact->inredo set and are behind the redo_horizon. We
+ *        save them to disk and also set gxact->ondisk to true.
+ *
+ *      * On COMMIT/ABORT we delete the entry from TwoPhaseState->prepXacts.
+ *        If gxact->ondisk is true, we delete the corresponding entry from
+ *        the disk as well.
+ *
+ *      * RecoverPreparedTransactions(), StandbyRecoverPreparedTransactions()
+ *        and PrescanPreparedTransactions() have been modified to go through
+ *        gxact->inredo entries that have not made to disk yet.
  *
  *-------------------------------------------------------------------------
  */
@@ -148,11 +166,13 @@ typedef struct GlobalTransactionData
 	 */
 	XLogRecPtr	prepare_start_lsn;		/* XLOG offset of prepare record start */
 	XLogRecPtr	prepare_end_lsn;	/* XLOG offset of prepare record end */
+	TransactionId	xid;			/* The GXACT id */
 
 	Oid			owner;			/* ID of user that executed the xact */
 	BackendId	locking_backend;	/* backend currently working on the xact */
 	bool		valid;			/* TRUE if PGPROC entry is in proc array */
 	bool		ondisk;			/* TRUE if prepare state file is on disk */
+	bool		inredo;			/* TRUE if entry was added via xlog_redo */
 	char		gid[GIDSIZE];	/* The GID assigned to the prepared xact */
 }	GlobalTransactionData;
 
@@ -350,12 +370,14 @@ PostPrepare_Twophase(void)
  */
 GlobalTransaction
 MarkAsPreparing(TransactionId xid, const char *gid,
-				TimestampTz prepared_at, Oid owner, Oid databaseid)
+				TimestampTz prepared_at, Oid owner, Oid databaseid,
+				XLogRecPtr prepare_start_lsn, XLogRecPtr prepare_end_lsn)
 {
 	GlobalTransaction gxact;
 	PGPROC	   *proc;
 	PGXACT	   *pgxact;
 	int			i;
+	bool		found = false;
 
 	if (strlen(gid) >= GIDSIZE)
 		ereport(ERROR,
@@ -385,22 +407,32 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 		gxact = TwoPhaseState->prepXacts[i];
 		if (strcmp(gxact->gid, gid) == 0)
 		{
-			ereport(ERROR,
+			/* It's ok to find an entry in the redo/recovery case */
+			if (!gxact->inredo)
+				ereport(ERROR,
 					(errcode(ERRCODE_DUPLICATE_OBJECT),
 					 errmsg("transaction identifier \"%s\" is already in use",
 							gid)));
+			else
+			{
+				found = true;
+				break;
+			}
 		}
 	}
 
 	/* Get a free gxact from the freelist */
-	if (TwoPhaseState->freeGXacts == NULL)
+	if (!found && TwoPhaseState->freeGXacts == NULL)
 		ereport(ERROR,
 				(errcode(ERRCODE_OUT_OF_MEMORY),
 				 errmsg("maximum number of prepared transactions reached"),
 				 errhint("Increase max_prepared_transactions (currently %d).",
 						 max_prepared_xacts)));
-	gxact = TwoPhaseState->freeGXacts;
-	TwoPhaseState->freeGXacts = gxact->next;
+	if (!found)
+	{
+		gxact = TwoPhaseState->freeGXacts;
+		TwoPhaseState->freeGXacts = gxact->next;
+	}
 
 	proc = &ProcGlobal->allProcs[gxact->pgprocno];
 	pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
@@ -432,18 +464,24 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 	pgxact->nxids = 0;
 
 	gxact->prepared_at = prepared_at;
-	/* initialize LSN to InvalidXLogRecPtr */
-	gxact->prepare_start_lsn = InvalidXLogRecPtr;
-	gxact->prepare_end_lsn = InvalidXLogRecPtr;
+	/* initialize LSN to passed in values */
+	gxact->prepare_start_lsn = prepare_start_lsn;
+	gxact->prepare_end_lsn = prepare_end_lsn;
+	gxact->xid = xid;
 	gxact->owner = owner;
 	gxact->locking_backend = MyBackendId;
 	gxact->valid = false;
 	gxact->ondisk = false;
+	gxact->inredo = false;
 	strcpy(gxact->gid, gid);
 
 	/* And insert it into the active array */
-	Assert(TwoPhaseState->numPrepXacts < max_prepared_xacts);
-	TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts++] = gxact;
+	if (!found)
+	{
+		Assert(TwoPhaseState->numPrepXacts < max_prepared_xacts);
+		TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts++] = gxact;
+	}
+
 
 	/*
 	 * Remember that we have this GlobalTransaction entry locked for us. If we
@@ -457,6 +495,92 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 }
 
 /*
+ * MarkAsPreparingInRedo
+ *		Reserve the GID for the given transaction in the redo code path.
+ *
+ * Internally, this creates a gxact struct and puts it into the active array.
+ *
+ * In redo, this struct is mainly used to track PREPARE/COMMIT entries
+ * in shared memory. Hence, we only fill up the bare minimum contents here.
+ * The gxact also gets marked with gxact->inredo set to true to indicate
+ * that it got added in the redo phase
+ */
+GlobalTransaction
+MarkAsPreparingInRedo(TransactionId xid, const char *gid,
+				TimestampTz prepared_at, Oid owner, Oid databaseid,
+				XLogRecPtr prepare_start_lsn, XLogRecPtr prepare_end_lsn)
+{
+	GlobalTransaction gxact;
+	int			i;
+
+	if (strlen(gid) >= GIDSIZE)
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("transaction identifier \"%s\" is too long",
+						gid)));
+
+	/* fail immediately if feature is disabled */
+	if (max_prepared_xacts == 0)
+		ereport(ERROR,
+				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+				 errmsg("prepared transactions are disabled"),
+			  errhint("Set max_prepared_transactions to a nonzero value.")));
+
+	/* on first call, register the exit hook */
+	if (!twophaseExitRegistered)
+	{
+		before_shmem_exit(AtProcExit_Twophase, 0);
+		twophaseExitRegistered = true;
+	}
+
+	LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+	/* Check for conflicting GID */
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		gxact = TwoPhaseState->prepXacts[i];
+		if (strcmp(gxact->gid, gid) == 0)
+		{
+			ereport(ERROR,
+					(errcode(ERRCODE_DUPLICATE_OBJECT),
+					 errmsg("transaction identifier \"%s\" is already in use",
+							gid)));
+		}
+	}
+
+	/* Get a free gxact from the freelist */
+	if (TwoPhaseState->freeGXacts == NULL)
+		ereport(ERROR,
+				(errcode(ERRCODE_OUT_OF_MEMORY),
+				 errmsg("maximum number of prepared transactions reached"),
+				 errhint("Increase max_prepared_transactions (currently %d).",
+						 max_prepared_xacts)));
+	gxact = TwoPhaseState->freeGXacts;
+	TwoPhaseState->freeGXacts = gxact->next;
+
+
+	gxact->prepared_at = prepared_at;
+	/* initialize LSN to passed in values */
+	gxact->prepare_start_lsn = prepare_start_lsn;
+	gxact->prepare_end_lsn = prepare_end_lsn;
+	gxact->xid = xid;
+	gxact->owner = owner;
+	gxact->locking_backend = InvalidBackendId;
+	gxact->valid = false;
+	gxact->ondisk = false;
+	gxact->inredo = true; /* yes, added in redo */
+	strcpy(gxact->gid, gid);
+
+	/* And insert it into the active array */
+	Assert(TwoPhaseState->numPrepXacts < max_prepared_xacts);
+	TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts++] = gxact;
+
+	LWLockRelease(TwoPhaseStateLock);
+
+	return gxact;
+}
+
+/*
  * GXactLoadSubxactData
  *
  * If the transaction being persisted had any subtransactions, this must
@@ -1242,9 +1366,9 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
  * Reads 2PC data from xlog. During checkpoint this data will be moved to
  * twophase files and ReadTwoPhaseFile should be used instead.
  *
- * Note clearly that this function accesses WAL during normal operation, similarly
- * to the way WALSender or Logical Decoding would do. It does not run during
- * crash recovery or standby processing.
+ * Note clearly that this function can access WAL during normal operation, similarly
+ * to the way WALSender or Logical Decoding would do.
+ *
  */
 static void
 XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
@@ -1253,8 +1377,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1624,9 +1746,8 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
 	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
 	{
 		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
-		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 
-		if (gxact->valid &&
+		if ((gxact->valid || gxact->inredo) &&
 			!gxact->ondisk &&
 			gxact->prepare_end_lsn <= redo_horizon)
 		{
@@ -1634,7 +1755,7 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
 			int			len;
 
 			XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, &len);
-			RecreateTwoPhaseFile(pgxact->xid, buf, len);
+			RecreateTwoPhaseFile(gxact->xid, buf, len);
 			gxact->ondisk = true;
 			pfree(buf);
 			serialized_xacts++;
@@ -1691,6 +1812,7 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 	TransactionId *xids = NULL;
 	int			nxids = 0;
 	int			allocsize = 0;
+	int			i;
 
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
@@ -1702,7 +1824,6 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 			char	   *buf;
 			TwoPhaseFileHeader *hdr;
 			TransactionId *subxids;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
@@ -1801,6 +1922,90 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 	}
 	FreeDir(cldir);
 
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		TransactionId xid;
+		int			len;
+		int			j;
+		char	   *buf;
+		TwoPhaseFileHeader *hdr;
+		TransactionId *subxids;
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+
+		/* only look at entries added by redo and not already on disk */
+		if (!gxact->inredo || gxact->ondisk)
+			continue;
+
+		xid = gxact->xid;
+		/* Reject XID if too new */
+		if (TransactionIdFollowsOrEquals(xid, origNextXid))
+		{
+			ereport(WARNING,
+					(errmsg("removing future two-phase state data from memory \"%u\"",
+							xid)));
+			PrepareRedoRemove(xid);
+			continue;
+		}
+
+		/* Read xlog data */
+		XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, &len);
+
+		/* Deconstruct header */
+		hdr = (TwoPhaseFileHeader *) buf;
+		Assert(TransactionIdEquals(hdr->xid, xid));
+
+		if (TransactionIdPrecedes(xid, result))
+			result = xid;
+
+		/*
+		 * Examine subtransaction XIDs ... they should all follow main
+		 * XID, and they may force us to advance nextXid.
+		 *
+		 * We don't expect anyone else to modify nextXid, hence we don't
+		 * need to hold a lock while examining it.  We still acquire the
+		 * lock to modify it, though.
+		 */
+		subxids = (TransactionId *) (buf +
+									 MAXALIGN(sizeof(TwoPhaseFileHeader)) +
+									 MAXALIGN(hdr->gidlen));
+		for (j = 0; j < hdr->nsubxacts; j++)
+		{
+			TransactionId subxid = subxids[j];
+
+			Assert(TransactionIdFollows(subxid, xid));
+			if (TransactionIdFollowsOrEquals(subxid,
+											 ShmemVariableCache->nextXid))
+			{
+				LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
+				ShmemVariableCache->nextXid = subxid;
+				TransactionIdAdvance(ShmemVariableCache->nextXid);
+				LWLockRelease(XidGenLock);
+			}
+		}
+
+		if (xids_p)
+		{
+			if (nxids == allocsize)
+			{
+				if (nxids == 0)
+				{
+					allocsize = 10;
+					xids = palloc(allocsize * sizeof(TransactionId));
+				}
+				else
+				{
+					allocsize = allocsize * 2;
+					xids = repalloc(xids, allocsize * sizeof(TransactionId));
+				}
+			}
+			xids[nxids++] = xid;
+		}
+
+		pfree(buf);
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
 	if (xids_p)
 	{
 		*xids_p = xids;
@@ -1827,6 +2032,7 @@ StandbyRecoverPreparedTransactions(bool overwriteOK)
 {
 	DIR		   *cldir;
 	struct dirent *clde;
+	int			i;
 
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
@@ -1838,7 +2044,6 @@ StandbyRecoverPreparedTransactions(bool overwriteOK)
 			char	   *buf;
 			TwoPhaseFileHeader *hdr;
 			TransactionId *subxids;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
@@ -1894,6 +2099,56 @@ StandbyRecoverPreparedTransactions(bool overwriteOK)
 		}
 	}
 	FreeDir(cldir);
+
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		TransactionId xid;
+		int			len;
+		char	   *buf;
+		TwoPhaseFileHeader *hdr;
+		TransactionId *subxids;
+		int			j;
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+
+		/* only look at entries added by redo and not already on disk */
+		if (!gxact->inredo || gxact->ondisk)
+			continue;
+
+		xid = gxact->xid;
+		/* Already processed? */
+		if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
+		{
+			ereport(WARNING,
+					(errmsg("removing stale 2PC data from shared memory")));
+			PrepareRedoRemove(xid);
+			continue;
+		}
+
+		/* Read xlog data */
+		XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, &len);
+
+		/* Deconstruct header */
+		hdr = (TwoPhaseFileHeader *) buf;
+		Assert(TransactionIdEquals(hdr->xid, xid));
+
+		/*
+		 * Examine subtransaction XIDs ... they should all follow main
+		 * XID
+		 */
+		subxids = (TransactionId *) (buf +
+									 MAXALIGN(sizeof(TwoPhaseFileHeader)) +
+									 MAXALIGN(hdr->gidlen));
+		for (j = 0; j < hdr->nsubxacts; j++)
+		{
+			TransactionId subxid = subxids[j];
+
+			Assert(TransactionIdFollows(subxid, xid));
+			SubTransSetParent(xid, subxid, overwriteOK);
+		}
+		pfree(buf);
+	}
+	LWLockRelease(TwoPhaseStateLock);
 }
 
 /*
@@ -1910,6 +2165,7 @@ RecoverPreparedTransactions(void)
 	DIR		   *cldir;
 	struct dirent *clde;
 	bool		overwriteOK = false;
+	int			i;
 
 	snprintf(dir, MAXPGPATH, "%s", TWOPHASE_DIR);
 
@@ -1926,7 +2182,6 @@ RecoverPreparedTransactions(void)
 			TransactionId *subxids;
 			GlobalTransaction gxact;
 			const char *gid;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
@@ -1990,7 +2245,8 @@ RecoverPreparedTransactions(void)
 			 */
 			gxact = MarkAsPreparing(xid, gid,
 									hdr->prepared_at,
-									hdr->owner, hdr->database);
+									hdr->owner, hdr->database,
+									InvalidXLogRecPtr, InvalidXLogRecPtr);
 			gxact->ondisk = true;
 			GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
 			MarkAsPrepared(gxact);
@@ -2019,6 +2275,109 @@ RecoverPreparedTransactions(void)
 		}
 	}
 	FreeDir(cldir);
+
+	/*
+	 * Don't need a lock in the recovery phase.
+	 */
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		TransactionId xid;
+		char	   *buf;
+		char	   *bufptr;
+		TwoPhaseFileHeader *hdr;
+		TransactionId *subxids;
+		const char *gid;
+		int			len;
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		GlobalTransaction gxactnew;
+
+		/* only look at entries added by redo and not already on disk */
+		if (!gxact->inredo || gxact->ondisk)
+			continue;
+
+		xid = gxact->xid;
+
+		/* Already processed? */
+		if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
+		{
+			ereport(WARNING,
+					(errmsg("removing stale 2PC data from shared memory %u", xid)));
+			PrepareRedoRemove(xid);
+			continue;
+		}
+
+		/* Read xlog data */
+		XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, &len);
+
+		ereport(LOG,
+				(errmsg("recovering prepared transaction %u from shared memory", xid)));
+
+		/* Deconstruct header */
+		hdr = (TwoPhaseFileHeader *) buf;
+		Assert(TransactionIdEquals(hdr->xid, xid));
+		bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+		gid = (const char *) bufptr;
+		bufptr += MAXALIGN(hdr->gidlen);
+		subxids = (TransactionId *) bufptr;
+		bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
+		bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
+		bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
+		bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+		/*
+		 * It's possible that SubTransSetParent has been set before, if
+		 * the prepared transaction generated xid assignment records. Test
+		 * here must match one used in AssignTransactionId().
+		 */
+		if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
+							 XLogLogicalInfoActive()))
+			overwriteOK = true;
+
+		/*
+		 * Reconstruct subtrans state for the transaction --- needed
+		 * because pg_subtrans is not preserved over a restart.  Note that
+		 * we are linking all the subtransactions directly to the
+		 * top-level XID; there may originally have been a more complex
+		 * hierarchy, but there's no need to restore that exactly.
+		 */
+		for (i = 0; i < hdr->nsubxacts; i++)
+			SubTransSetParent(subxids[i], xid, overwriteOK);
+
+		/*
+		 * Recreate its GXACT and dummy PGPROC
+		 */
+		gxactnew = MarkAsPreparing(xid, gid,
+								hdr->prepared_at,
+								hdr->owner, hdr->database,
+								gxact->prepare_start_lsn,
+								gxact->prepare_end_lsn);
+
+		Assert(gxactnew == gxact);
+		GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
+		MarkAsPrepared(gxact);
+
+		/*
+		 * Recover other state (notably locks) using resource managers
+		 */
+		ProcessRecords(bufptr, xid, twophase_recover_callbacks);
+
+		/*
+		 * Release locks held by the standby process after we process each
+		 * prepared transaction. As a result, we don't need too many
+		 * additional locks at any one time.
+		 */
+		if (InHotStandby)
+			StandbyReleaseLockTree(xid, hdr->nsubxacts, subxids);
+
+		/*
+		 * We're done with recovering this transaction. Clear
+		 * MyLockedGxact, like we do in PrepareTransaction() during normal
+		 * operation.
+		 */
+		PostPrepare_Twophase();
+
+		pfree(buf);
+	}
 }
 
 /*
@@ -2163,3 +2522,83 @@ RecordTransactionAbortPrepared(TransactionId xid,
 	 */
 	SyncRepWaitForLSN(recptr, false);
 }
+
+/*
+ * PrepareRedoAdd
+ *
+ * Store pointers to the start/end of the WAL record along with the xid in
+ * a gxact entry in shared memory TwoPhaseState structure
+ */
+void
+PrepareRedoAdd(XLogReaderState *record)
+{
+	char	          *buf = XLogRecGetData(record);
+	TwoPhaseFileHeader *hdr = (TwoPhaseFileHeader *) buf;
+	char			  *bufptr;
+	const char		  *gid;
+	GlobalTransaction gxact;
+
+	Assert(RecoveryInProgress());
+
+	bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+	gid = (const char *) bufptr;
+
+	/*
+	 * Add a GXACT entry
+	 */
+	gxact = MarkAsPreparingInRedo(hdr->xid, gid,
+							hdr->prepared_at,
+							hdr->owner, hdr->database,
+							record->ReadRecPtr,
+							record->EndRecPtr);
+
+	elog(DEBUG2, "Adding 2PC data to shared memory %u", hdr->xid);
+}
+
+/*
+ * PrepareRedoRemove
+ *
+ * Remove the corresponding gxact entry from TwoPhaseState. Also
+ * remove the 2PC file.
+ */
+void
+PrepareRedoRemove(TransactionId xid)
+{
+	GlobalTransaction gxact;
+	int			i;
+	bool		found = false;
+
+	Assert(RecoveryInProgress());
+
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		gxact = TwoPhaseState->prepXacts[i];
+
+		if (gxact->xid == xid)
+		{
+			Assert(gxact->inredo);
+			found = true;
+			break;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+	if (found)
+	{
+		/*
+		 * And now we can clean up any files we may have left.
+		 */
+		if (gxact->ondisk)
+			RemoveTwoPhaseFile(xid, true);
+		RemoveGXact(gxact);
+		elog(DEBUG2, "Removing 2PC data from shared memory %u", xid);
+	}
+	else
+	{
+		/*
+		 * Entry could be on disk. Call with giveWarning=false
+		 * since it can be expected during replay.
+		 */
+		RemoveTwoPhaseFile(xid, false);
+	}
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index f6f136d..8f027a9 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -2294,7 +2294,8 @@ PrepareTransaction(void)
 	 * GID is invalid or already in use.
 	 */
 	gxact = MarkAsPreparing(xid, prepareGID, prepared_at,
-							GetUserId(), MyDatabaseId);
+							GetUserId(), MyDatabaseId,
+							InvalidXLogRecPtr, InvalidXLogRecPtr);
 	prepareGID = NULL;
 
 	/*
@@ -5606,7 +5607,9 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete TwoPhaseState gxact entry and/or 2PC file. */
+			PrepareRedoRemove(parsed.twophase_xid);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5626,14 +5629,18 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete TwoPhaseState gxact entry and/or 2PC file. */
+			PrepareRedoRemove(parsed.twophase_xid);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		/*
+		 * Store xid and start/end pointers of the WAL record in
+		 * TwoPhaseState gxact entry.
+		 */
+		PrepareRedoAdd(record);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b2b7848..063b946 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -15,6 +15,7 @@
 #define TWOPHASE_H
 
 #include "access/xlogdefs.h"
+#include "access/xlogreader.h"
 #include "datatype/timestamp.h"
 #include "storage/lock.h"
 
@@ -38,7 +39,12 @@ extern BackendId TwoPhaseGetDummyBackendId(TransactionId xid);
 
 extern GlobalTransaction MarkAsPreparing(TransactionId xid, const char *gid,
 				TimestampTz prepared_at,
-				Oid owner, Oid databaseid);
+				Oid owner, Oid databaseid,
+				XLogRecPtr prepare_start_lsn, XLogRecPtr prepare_end_lsn);
+extern GlobalTransaction MarkAsPreparingInRedo(TransactionId xid, const char *gid,
+				TimestampTz prepared_at,
+				Oid owner, Oid databaseid,
+				XLogRecPtr prepare_start_lsn, XLogRecPtr prepare_end_lsn);
 
 extern void StartPrepare(GlobalTransaction gxact);
 extern void EndPrepare(GlobalTransaction gxact);
@@ -56,4 +62,6 @@ extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void PrepareRedoAdd(XLogReaderState *record);
+extern void PrepareRedoRemove(TransactionId xid);
 #endif   /* TWOPHASE_H */
diff --git a/src/test/recovery/t/009_twophase.pl b/src/test/recovery/t/009_twophase.pl
new file mode 100755
index 0000000..dd2c708
--- /dev/null
+++ b/src/test/recovery/t/009_twophase.pl
@@ -0,0 +1,315 @@
+# Tests dedicated to two-phase commit in recovery
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 13;
+
+# Setup master node
+my $node_master = get_new_node("master");
+$node_master->init(allows_streaming => 1);
+$node_master->append_conf('postgresql.conf', qq(
+	max_prepared_transactions = 10
+	log_checkpoints = true
+));
+$node_master->start;
+$node_master->backup('master_backup');
+$node_master->psql('postgres', "create table t(id int)");
+
+# Setup slave node
+my $node_slave = get_new_node('slave');
+$node_slave->init_from_backup($node_master, 'master_backup', has_streaming => 1);
+$node_slave->start;
+
+# Switch to synchronous replication
+$node_master->append_conf('postgresql.conf', qq(
+	synchronous_standby_names = '*'
+));
+$node_master->psql('postgres', "select pg_reload_conf()");
+
+my $psql_out = '';
+my $psql_rc = '';
+
+###############################################################################
+# Check that we can commit and abort transaction after soft restart.
+# Here checkpoint happens before shutdown and no WAL replay will occur at next
+# startup. In this case postgres re-creates shared-memory state from twophase
+# files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	savepoint s1;
+	insert into t values (143);
+	prepare transaction 'y';");
+$node_master->stop;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared transaction after restart.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared transaction after restart.');
+
+###############################################################################
+# Check that we can commit and abort after a hard restart.
+# At next startup, WAL replay will re-create shared memory state for prepared
+# transaction using dedicated WAL records.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	begin;
+	insert into t values (142);
+	savepoint s1;
+	insert into t values (143);
+	prepare transaction 'y';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Commit prepared transaction after teardown.');
+
+$psql_rc = $node_master->psql('postgres', "rollback prepared 'y'");
+is($psql_rc, '0', 'Rollback prepared transaction after teardown.');
+
+###############################################################################
+# Check that WAL replay can handle several transactions with same GID name.
+###############################################################################
+
+$node_master->psql('postgres', "
+	checkpoint;
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_master->start;
+
+$psql_rc = $node_master->psql('postgres', "commit prepared 'x'");
+is($psql_rc, '0', 'Replay several transactions with same GID.');
+
+###############################################################################
+# Check that WAL replay cleans up its shared memory state and releases locks
+# while replaying transaction commits.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_master->teardown_node;
+$node_master->start;
+$psql_rc = $node_master->psql('postgres', "begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	-- This prepare can fail due to conflicting GID or locks conflicts if
+	-- replay did not fully cleanup its state on previous commit.
+	prepare transaction 'x';");
+is($psql_rc, '0', "Cleanup of shared memory state for 2PC commit");
+
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that WAL replay will cleanup its shared memory state on running slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';
+	commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;",
+	  stdout => \$psql_out);
+is($psql_out, '0',
+   "Cleanup of shared memory state on running standby without checkpoint.");
+
+###############################################################################
+# Same as in previous case, but let's force checkpoint on slave between
+# prepare and commit to use on-disk twophase files.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_slave->psql('postgres', "checkpoint;");
+$node_master->psql('postgres', "commit prepared 'x';");
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts;",
+	  stdout => \$psql_out);
+is($psql_out, '0',
+   "Cleanup of shared memory state on running standby after checkpoint.");
+
+###############################################################################
+# Check that prepared transactions can be committed on promoted slave.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->teardown_node;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$psql_rc = $node_slave->psql('postgres', "commit prepared 'x';");
+is($psql_rc, '0', "Restore of prepared transaction on promoted slave.");
+
+# change roles
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+
+###############################################################################
+# Check that prepared transactions are replayed after soft restart of standby
+# while master is down. Since standby knows that master is down it uses a
+# different code path on startup to ensure that the status of transactions is
+# consistent.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (42);
+	savepoint s1;
+	insert into t values (43);
+	prepare transaction 'x';");
+$node_master->stop;
+$node_slave->restart;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '1',
+   "Restore prepared transactions from files with master down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres', "commit prepared 'x'");
+
+###############################################################################
+# Check that prepared transactions are correctly replayed after slave hard
+# restart while master is down.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	insert into t values (242);
+	savepoint s1;
+	insert into t values (243);
+	prepare transaction 'x';
+	");
+$node_master->stop;
+$node_slave->teardown_node;
+$node_slave->start;
+$node_slave->promote;
+$node_slave->poll_query_until('postgres',
+	  "SELECT pg_is_in_recovery() <> true");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '1',
+   "Restore prepared transactions from records with master down.");
+
+# restore state
+($node_master, $node_slave) = ($node_slave, $node_master);
+$node_slave->enable_streaming($node_master);
+$node_slave->append_conf('recovery.conf', qq(
+recovery_target_timeline='latest'
+));
+$node_slave->start;
+$node_master->psql('postgres', "commit prepared 'x'");
+
+
+###############################################################################
+# Check for a lock conflict between prepared transaction with DDL inside and replay of
+# XLOG_STANDBY_LOCK wal record.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	create table t2(id int);
+	savepoint s1;
+	insert into t2 values (42);
+	prepare transaction 'x';
+	-- checkpoint will issue XLOG_STANDBY_LOCK that can conflict with lock
+	-- held by 'create table' statement
+	checkpoint;
+	commit prepared 'x';");
+
+$node_slave->psql('postgres', "select count(*) from pg_prepared_xacts",
+	  stdout => \$psql_out);
+is($psql_out, '0', "Replay prepared transaction with DDL.");
+
+
+###############################################################################
+# Check that replay will correctly set SUBTRANS and properly advance nextXid
+# so that it won't conflict with savepoint xids.
+###############################################################################
+
+$node_master->psql('postgres', "
+	begin;
+	delete from t;
+	insert into t values (43);
+	savepoint s1;
+	insert into t values (43);
+	savepoint s2;
+	insert into t values (43);
+	savepoint s3;
+	insert into t values (43);
+	savepoint s4;
+	insert into t values (43);
+	savepoint s5;
+	insert into t values (43);
+	prepare transaction 'x';
+	checkpoint;");
+
+$node_master->stop;
+$node_master->start;
+$node_master->psql('postgres', "
+	-- here we can get xid of previous savepoint if nextXid
+	-- wasn't properly advanced
+	begin;
+	insert into t values (142);
+	abort;
+	commit prepared 'x';");
+
+$node_master->psql('postgres', "select count(*) from t",
+	  stdout => \$psql_out);
+is($psql_out, '6', "Check nextXid handling for prepared subtransactions");
#141Robert Haas
robertmhaas@gmail.com
In reply to: Nikhil Sontakke (#139)
Re: Speedup twophase transactions

On Wed, Feb 1, 2017 at 3:29 AM, Nikhil Sontakke <nikhils@2ndquadrant.com> wrote:

Shouldn’t the singular part of the message above be:
"%u two-phase state file was written for a long-running prepared transaction"

But, then, English is not my native language, so I might be off here :-)

If there's one file per long-running prepared transaction, which I
think is true, then I agree with you.

PFA, small patch to fix this, then.

Committed.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#142Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Nikhil Sontakke (#140)
Re: Speedup twophase transactions

I have pushed the TAP test file, which is already passing, at least for
me. Let's see what the buildfarm says.

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#143Michael Paquier
michael.paquier@gmail.com
In reply to: Alvaro Herrera (#142)
Re: Speedup twophase transactions

On Wed, Feb 22, 2017 at 7:03 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:

I have pushed the TAP test file, which is already passing, at least for
me. Let's see what the buildfarm says.

Thanks. I still have on my sheet to look at the latest 2PC patch.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#144Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#140)
Re: Speedup twophase transactions

On Thu, Feb 2, 2017 at 3:07 PM, Nikhil Sontakke <nikhils@2ndquadrant.com> wrote:

Yeah. Was thinking about this yesterday. How about adding entries in
TwoPhaseState itself (which become valid later)? Only if it does not
cause a lot of code churn.

That's possible as well, yes.

PFA a patch which does the above. It re-uses the TwoPhaseState gxact
entries to track 2PC PREPARE/COMMIT in shared memory. The advantage
being that CheckPointTwoPhase() becomes the only place where the fsync
of 2PC files happens.

A minor annoyance in the patch is the duplication of the code to add
the 2nd while loop to go through these shared memory entries in
PrescanPreparedTransactions, RecoverPreparedTransactions and
StandbyRecoverPreparedTransactions.

Other than this, I ran TAP tests and they succeed as needed.

Thanks for the new patch. Finally I am looking at it... The regression
tests already committed are all able to pass.

twophase.c: In function ‘PrepareRedoAdd’:
twophase.c:2539:20: warning: variable ‘gxact’ set but not used
[-Wunused-but-set-variable]
GlobalTransaction gxact;
There is a warning at compilation.

The comment at the top of PrescanPreparedTransactions() needs to be
updated. Not only does this routine look for the contents of
pg_twophase, but it does also look at the shared memory contents, at
least those ones marked with inredo and not on_disk.

+           ereport(WARNING,
+                   (errmsg("removing future two-phase state data from
memory \"%u\"",
+                           xid)));
+           PrepareRedoRemove(xid);
+           continue
Those are not normal (partially because unlink is atomic, but not
durable)... But they match the correct coding pattern regarding
incorrect 2PC entries... I'd really like to see those switched to a
FATAL with unlink() made durable for those calls.
+       /* Deconstruct header */
+       hdr = (TwoPhaseFileHeader *) buf;
+       Assert(TransactionIdEquals(hdr->xid, xid));
+
+       if (TransactionIdPrecedes(xid, result))
+           result = xid;
This portion is repeated three times and could be easily refactored.
You could just have a routine that returns the oldes transaction ID
used, and ignore the result for StandbyRecoverPreparedTransactions()
by casting a (void) to let any kind of static analyzer understand that
we don't care about the result for example. Handling for subxids is
necessary as well depending on the code path. Spliting things into a
could of sub-routines may be more readable as well. There are really a
couple of small parts that can be gathered and strengthened.
+       /*
+        * Recreate its GXACT and dummy PGPROC
+        */
+       gxactnew = MarkAsPreparing(xid, gid,
+                               hdr->prepared_at,
+                               hdr->owner, hdr->database,
+                               gxact->prepare_start_lsn,
+                               gxact->prepare_end_lsn);
MarkAsPreparing() does not need to be extended with two new arguments.
RecoverPreparedTransactions() is used only at the end of recovery,
where it is not necessary to look at the 2PC state files from the
records. In this code path inredo is also set to false :)
+   {
+       /*
+        * Entry could be on disk. Call with giveWarning=false
+        * since it can be expected during replay.
+        */
+       RemoveTwoPhaseFile(xid, false);
+   }
This would be removed at the end of recovery anyway as a stale entry,
so that's not necessary.
+           /* Delete TwoPhaseState gxact entry and/or 2PC file. */
+           PrepareRedoRemove(parsed.twophase_xid);
Both things should not be present, no? If the file is pushed to disk
it means that the checkpoint horizon has already moved.
-           ereport(ERROR,
+           /* It's ok to find an entry in the redo/recovery case */
+           if (!gxact->inredo)
+               ereport(ERROR,
                    (errcode(ERRCODE_DUPLICATE_OBJECT),
                     errmsg("transaction identifier \"%s\" is already in use",
                            gid)));
+           else
+           {
+               found = true;
+               break;
+           }
I would not have thought so.

MarkAsPreparing and MarkAsPreparingInRedo really share the same code.
What if the setup of the dummy PGPROC entry is made conditional?
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#145Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#144)
Re: Speedup twophase transactions

Hi Michael,

Thanks for taking a look at the patch.

twophase.c: In function ‘PrepareRedoAdd’:
twophase.c:2539:20: warning: variable ‘gxact’ set but not used
[-Wunused-but-set-variable]
GlobalTransaction gxact;
There is a warning at compilation.

Will fix.

The comment at the top of PrescanPreparedTransactions() needs to be
updated. Not only does this routine look for the contents of
pg_twophase, but it does also look at the shared memory contents, at
least those ones marked with inredo and not on_disk.

Oh yes. Will change comments at top of
StandbyRecoverPreparedTransactions(), RecoverPreparedTransactions() as well.

+           ereport(WARNING,
+                   (errmsg("removing future two-phase state data from
memory \"%u\"",
+                           xid)));
+           PrepareRedoRemove(xid);
+           continue
Those are not normal (partially because unlink is atomic, but not
durable)... But they match the correct coding pattern regarding
incorrect 2PC entries... I'd really like to see those switched to a
FATAL with unlink() made durable for those calls.

Hmm, not sure what exactly we need to do here. If you look at the prior
checks, there we already skip on-disk entries. So, typically, the entries
that we encounter here will be in shmem only.

+       /* Deconstruct header */
+       hdr = (TwoPhaseFileHeader *) buf;
+       Assert(TransactionIdEquals(hdr->xid, xid));
+
+       if (TransactionIdPrecedes(xid, result))
+           result = xid;
This portion is repeated three times and could be easily refactored.
You could just have a routine that returns the oldes transaction ID
used, and ignore the result for StandbyRecoverPreparedTransactions()
by casting a (void) to let any kind of static analyzer understand that
we don't care about the result for example. Handling for subxids is
necessary as well depending on the code path. Spliting things into a
could of sub-routines may be more readable as well. There are really a
couple of small parts that can be gathered and strengthened.

I will see if we can reduce this to a couple of function calls.

+       /*
+        * Recreate its GXACT and dummy PGPROC
+        */
+       gxactnew = MarkAsPreparing(xid, gid,
+                               hdr->prepared_at,
+                               hdr->owner, hdr->database,
+                               gxact->prepare_start_lsn,
+                               gxact->prepare_end_lsn);
MarkAsPreparing() does not need to be extended with two new arguments.
RecoverPreparedTransactions() is used only at the end of recovery,
where it is not necessary to look at the 2PC state files from the
records. In this code path inredo is also set to false :)

That's not true. We will have entries with inredo set at the end of
recovery as well. Infact the MarkAsPreparing() call
from RecoverPreparedTransactions() is the one which will remove these
inredo entries and convert them into regular entries. We have optimized the
recovery code path as well.

+   {
+       /*
+        * Entry could be on disk. Call with giveWarning=false
+        * since it can be expected during replay.
+        */
+       RemoveTwoPhaseFile(xid, false);
+   }
This would be removed at the end of recovery anyway as a stale entry,
so that's not necessary.

Ok, will remove this.

+           /* Delete TwoPhaseState gxact entry and/or 2PC file. */
+           PrepareRedoRemove(parsed.twophase_xid);
Both things should not be present, no? If the file is pushed to disk
it means that the checkpoint horizon has already moved.

PREPARE in redo, followed by a checkpoint, followed by a COMMIT/ROLLBACK.
We can have both the bits set in this case.

-           ereport(ERROR,
+           /* It's ok to find an entry in the redo/recovery case */
+           if (!gxact->inredo)
+               ereport(ERROR,
(errcode(ERRCODE_DUPLICATE_OBJECT),
errmsg("transaction identifier \"%s\" is already in
use",
gid)));
+           else
+           {
+               found = true;
+               break;
+           }
I would not have thought so.

Since we are using the TwoPhaseState structure to track redo entries, at

end of recovery, we will find existing entries. Please see my comments
above for RecoverPreparedTransactions()

MarkAsPreparing and MarkAsPreparingInRedo really share the same code.
What if the setup of the dummy PGPROC entry is made conditional?

I thought it was cleaner this ways. We can definitely add a bunch of
if-else in MarkAsPreparing() but it won't look pretty.

Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services

#146David Steele
david@pgmasters.net
In reply to: Nikhil Sontakke (#145)
Re: Speedup twophase transactions

Nikhil,

On 2/27/17 12:19 AM, Nikhil Sontakke wrote:

Hi Michael,

Thanks for taking a look at the patch.

twophase.c: In function ‘PrepareRedoAdd’:
twophase.c:2539:20: warning: variable ‘gxact’ set but not used
[-Wunused-but-set-variable]
GlobalTransaction gxact;
There is a warning at compilation.

Will fix.

<...>

Do you know when you will have a new patch ready?

It would be great to get this thread closed out after 14 months and many
commits.

Thanks,
--
-David
david@pgmasters.net

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#147Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: David Steele (#146)
1 attachment(s)
Re: Speedup twophase transactions

Hi David and Michael,

It would be great to get this thread closed out after 14 months and many
commits.

PFA, latest patch which addresses Michael's comments.

twophase.c: In function ‘PrepareRedoAdd’:

twophase.c:2539:20: warning: variable ‘gxact’ set but not used
[-Wunused-but-set-variable]
GlobalTransaction gxact;
There is a warning at compilation.

Fixed.

The comment at the top of PrescanPreparedTransactions() needs to be
updated. Not only does this routine look for the contents of
pg_twophase, but it does also look at the shared memory contents, at
least those ones marked with inredo and not on_disk.

Changed comments at top of PrescanPreparedTransactions() ,
StandbyRecoverPreparedTransactions(), and RecoverPreparedTransactions().

+           ereport(WARNING,
+                   (errmsg("removing future two-phase state data from
memory \"%u\"",
+                           xid)));
+           PrepareRedoRemove(xid);
+           continue
Those are not normal (partially because unlink is atomic, but not
durable)... But they match the correct coding pattern regarding
incorrect 2PC entries... I'd really like to see those switched to a
FATAL with unlink() made durable for those calls.

Hmm, not sure what exactly we need to do here. If you look at the prior
checks, there we already skip on-disk entries. So, typically, the entries
that we encounter here will be in shmem only.

+       /* Deconstruct header */
+       hdr = (TwoPhaseFileHeader *) buf;
+       Assert(TransactionIdEquals(hdr->xid, xid));
+
+       if (TransactionIdPrecedes(xid, result))
+           result = xid;
This portion is repeated three times and could be easily refactored.
You could just have a routine that returns the oldes transaction ID
used, and ignore the result for StandbyRecoverPreparedTransactions()
by casting a (void) to let any kind of static analyzer understand that
we don't care about the result for example. Handling for subxids is
necessary as well depending on the code path. Spliting things into a
could of sub-routines may be more readable as well. There are really a
couple of small parts that can be gathered and strengthened.

Have added a new function to do this now. It reads either from disk or
shared memory and produces error/log messages accordingly as well now.

+       /*
+        * Recreate its GXACT and dummy PGPROC
+        */
+       gxactnew = MarkAsPreparing(xid, gid,
+                               hdr->prepared_at,
+                               hdr->owner, hdr->database,
+                               gxact->prepare_start_lsn,
+                               gxact->prepare_end_lsn);
MarkAsPreparing() does not need to be extended with two new arguments.
RecoverPreparedTransactions() is used only at the end of recovery,
where it is not necessary to look at the 2PC state files from the
records. In this code path inredo is also set to false :)

That's not true. We will have entries with inredo set at the end of
recovery as well. Infact the MarkAsPreparing() call
from RecoverPreparedTransactions() is the one which will remove these
inredo entries and convert them into regular entries. We have optimized the
recovery code path as well.

+           /* Delete TwoPhaseState gxact entry and/or 2PC file. */
+           PrepareRedoRemove(parsed.twophase_xid);
Both things should not be present, no? If the file is pushed to disk
it means that the checkpoint horizon has already moved.

PREPARE in redo, followed by a checkpoint, followed by a COMMIT/ROLLBACK.
We can have both the bits set in this case.

-           ereport(ERROR,
+           /* It's ok to find an entry in the redo/recovery case */
+           if (!gxact->inredo)
+               ereport(ERROR,
(errcode(ERRCODE_DUPLICATE_OBJECT),
errmsg("transaction identifier \"%s\" is already in
use",
gid)));
+           else
+           {
+               found = true;
+               break;
+           }
I would not have thought so.

Since we are using the TwoPhaseState structure to track redo entries, at

end of recovery, we will find existing entries. Please see my comments
above for RecoverPreparedTransactions()

MarkAsPreparing and MarkAsPreparingInRedo really share the same code.
What if the setup of the dummy PGPROC entry is made conditional?

I realized that MarkAsPreparingInRedo() does not need to do all the sanity
checking since it's going to be invoked during redo and everything that
comes in is kosher already. So its contents are much simplified in this
latest patch.

Tests pass with this latest patch.

Regards,
Nikhils

Attachments:

twophase_recovery_shmem_110317.patchapplication/octet-stream; name=twophase_recovery_shmem_110317.patchDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 0a8edb9..75ddd04 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,26 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *
+ *      During replay and replication, TwoPhaseState also holds information
+ *      about active prepared transactions that haven't been moved to disk yet.
+ *
+ *      Replay of twophase records happens by the following rules:
+ *
+ *      * On PREPARE redo we add the transaction to TwoPhaseState->prepXacts.
+ *        We set gxact->inredo to true for such entries.
+ *
+ *      * On Checkpoint we iterate through TwoPhaseState->prepXacts entries
+ *        that have gxact->inredo set and are behind the redo_horizon. We
+ *        save them to disk and also set gxact->ondisk to true.
+ *
+ *      * On COMMIT/ABORT we delete the entry from TwoPhaseState->prepXacts.
+ *        If gxact->ondisk is true, we delete the corresponding entry from
+ *        the disk as well.
+ *
+ *      * RecoverPreparedTransactions(), StandbyRecoverPreparedTransactions()
+ *        and PrescanPreparedTransactions() have been modified to go through
+ *        gxact->inredo entries that have not made to disk yet.
  *
  *-------------------------------------------------------------------------
  */
@@ -147,11 +165,13 @@ typedef struct GlobalTransactionData
 	 */
 	XLogRecPtr	prepare_start_lsn;		/* XLOG offset of prepare record start */
 	XLogRecPtr	prepare_end_lsn;	/* XLOG offset of prepare record end */
+	TransactionId	xid;			/* The GXACT id */
 
 	Oid			owner;			/* ID of user that executed the xact */
 	BackendId	locking_backend;	/* backend currently working on the xact */
 	bool		valid;			/* TRUE if PGPROC entry is in proc array */
 	bool		ondisk;			/* TRUE if prepare state file is on disk */
+	bool		inredo;			/* TRUE if entry was added via xlog_redo */
 	char		gid[GIDSIZE];	/* The GID assigned to the prepared xact */
 }	GlobalTransactionData;
 
@@ -198,6 +218,10 @@ static void ProcessRecords(char *bufptr, TransactionId xid,
 static void RemoveGXact(GlobalTransaction gxact);
 
 static void XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len);
+static char *ProcessTwoPhaseBufferAndReturn(TransactionId xid,
+							XLogRecPtr	prepare_start_lsn,
+							bool fromdisk, bool overwriteOK, bool setParent,
+							TransactionId *result, TransactionId *maxsubxid);
 
 /*
  * Initialization of shared memory
@@ -349,12 +373,14 @@ PostPrepare_Twophase(void)
  */
 GlobalTransaction
 MarkAsPreparing(TransactionId xid, const char *gid,
-				TimestampTz prepared_at, Oid owner, Oid databaseid)
+				TimestampTz prepared_at, Oid owner, Oid databaseid,
+				XLogRecPtr prepare_start_lsn, XLogRecPtr prepare_end_lsn)
 {
 	GlobalTransaction gxact;
 	PGPROC	   *proc;
 	PGXACT	   *pgxact;
 	int			i;
+	bool		found = false;
 
 	if (strlen(gid) >= GIDSIZE)
 		ereport(ERROR,
@@ -384,22 +410,32 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 		gxact = TwoPhaseState->prepXacts[i];
 		if (strcmp(gxact->gid, gid) == 0)
 		{
-			ereport(ERROR,
+			/* It's ok to find an entry in the redo/recovery case */
+			if (!gxact->inredo)
+				ereport(ERROR,
 					(errcode(ERRCODE_DUPLICATE_OBJECT),
 					 errmsg("transaction identifier \"%s\" is already in use",
 							gid)));
+			else
+			{
+				found = true;
+				break;
+			}
 		}
 	}
 
 	/* Get a free gxact from the freelist */
-	if (TwoPhaseState->freeGXacts == NULL)
+	if (!found && TwoPhaseState->freeGXacts == NULL)
 		ereport(ERROR,
 				(errcode(ERRCODE_OUT_OF_MEMORY),
 				 errmsg("maximum number of prepared transactions reached"),
 				 errhint("Increase max_prepared_transactions (currently %d).",
 						 max_prepared_xacts)));
-	gxact = TwoPhaseState->freeGXacts;
-	TwoPhaseState->freeGXacts = gxact->next;
+	if (!found)
+	{
+		gxact = TwoPhaseState->freeGXacts;
+		TwoPhaseState->freeGXacts = gxact->next;
+	}
 
 	proc = &ProcGlobal->allProcs[gxact->pgprocno];
 	pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
@@ -431,18 +467,24 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 	pgxact->nxids = 0;
 
 	gxact->prepared_at = prepared_at;
-	/* initialize LSN to InvalidXLogRecPtr */
-	gxact->prepare_start_lsn = InvalidXLogRecPtr;
-	gxact->prepare_end_lsn = InvalidXLogRecPtr;
+	/* initialize LSN to passed in values */
+	gxact->prepare_start_lsn = prepare_start_lsn;
+	gxact->prepare_end_lsn = prepare_end_lsn;
+	gxact->xid = xid;
 	gxact->owner = owner;
 	gxact->locking_backend = MyBackendId;
 	gxact->valid = false;
 	gxact->ondisk = false;
+	gxact->inredo = false;
 	strcpy(gxact->gid, gid);
 
 	/* And insert it into the active array */
-	Assert(TwoPhaseState->numPrepXacts < max_prepared_xacts);
-	TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts++] = gxact;
+	if (!found)
+	{
+		Assert(TwoPhaseState->numPrepXacts < max_prepared_xacts);
+		TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts++] = gxact;
+	}
+
 
 	/*
 	 * Remember that we have this GlobalTransaction entry locked for us. If we
@@ -456,6 +498,58 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 }
 
 /*
+ * MarkAsPreparingInRedo
+ *		Reserve the GID for the given transaction in the redo code path.
+ *
+ * Internally, this creates a gxact struct and puts it into the active array.
+ *
+ * In redo, this struct is mainly used to track PREPARE/COMMIT entries
+ * in shared memory. Hence, we only fill up the bare minimum contents here.
+ * The gxact also gets marked with gxact->inredo set to true to indicate
+ * that it got added in the redo phase
+ */
+GlobalTransaction
+MarkAsPreparingInRedo(TransactionId xid, const char *gid,
+				TimestampTz prepared_at, Oid owner, Oid databaseid,
+				XLogRecPtr prepare_start_lsn, XLogRecPtr prepare_end_lsn)
+{
+	GlobalTransaction gxact;
+
+	LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+
+	/* Get a free gxact from the freelist */
+	if (TwoPhaseState->freeGXacts == NULL)
+		ereport(ERROR,
+				(errcode(ERRCODE_OUT_OF_MEMORY),
+				 errmsg("maximum number of prepared transactions reached"),
+				 errhint("Increase max_prepared_transactions (currently %d).",
+						 max_prepared_xacts)));
+	gxact = TwoPhaseState->freeGXacts;
+	TwoPhaseState->freeGXacts = gxact->next;
+
+
+	gxact->prepared_at = prepared_at;
+	/* initialize LSN to passed in values */
+	gxact->prepare_start_lsn = prepare_start_lsn;
+	gxact->prepare_end_lsn = prepare_end_lsn;
+	gxact->xid = xid;
+	gxact->owner = owner;
+	gxact->locking_backend = InvalidBackendId;
+	gxact->valid = false;
+	gxact->ondisk = false;
+	gxact->inredo = true; /* yes, added in redo */
+	strcpy(gxact->gid, gid);
+
+	/* And insert it into the active array */
+	Assert(TwoPhaseState->numPrepXacts < max_prepared_xacts);
+	TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts++] = gxact;
+
+	LWLockRelease(TwoPhaseStateLock);
+
+	return gxact;
+}
+
+/*
  * GXactLoadSubxactData
  *
  * If the transaction being persisted had any subtransactions, this must
@@ -1241,9 +1335,9 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
  * Reads 2PC data from xlog. During checkpoint this data will be moved to
  * twophase files and ReadTwoPhaseFile should be used instead.
  *
- * Note clearly that this function accesses WAL during normal operation, similarly
- * to the way WALSender or Logical Decoding would do. It does not run during
- * crash recovery or standby processing.
+ * Note clearly that this function can access WAL during normal operation, similarly
+ * to the way WALSender or Logical Decoding would do.
+ *
  */
 static void
 XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
@@ -1252,8 +1346,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1623,9 +1715,8 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
 	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
 	{
 		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
-		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 
-		if (gxact->valid &&
+		if ((gxact->valid || gxact->inredo) &&
 			!gxact->ondisk &&
 			gxact->prepare_end_lsn <= redo_horizon)
 		{
@@ -1633,7 +1724,7 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
 			int			len;
 
 			XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, &len);
-			RecreateTwoPhaseFile(pgxact->xid, buf, len);
+			RecreateTwoPhaseFile(gxact->xid, buf, len);
 			gxact->ondisk = true;
 			pfree(buf);
 			serialized_xacts++;
@@ -1661,6 +1752,9 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
  * reading WAL.  ShmemVariableCache->nextXid has been set to one more than
  * the highest XID for which evidence exists in WAL.
  *
+ * Additionally, scan TwoPhaseState shmem entries that are marked as added
+ * in redo and are not already on disk.
+ *
  * We throw away any prepared xacts with main XID beyond nextXid --- if any
  * are present, it suggests that the DBA has done a PITR recovery to an
  * earlier point in time without cleaning out pg_twophase.  We dare not
@@ -1685,11 +1779,13 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 {
 	TransactionId origNextXid = ShmemVariableCache->nextXid;
 	TransactionId result = origNextXid;
+	TransactionId maxsubxid = origNextXid;
 	DIR		   *cldir;
 	struct dirent *clde;
 	TransactionId *xids = NULL;
 	int			nxids = 0;
 	int			allocsize = 0;
+	int			i;
 
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
@@ -1699,83 +1795,16 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 		{
 			TransactionId xid;
 			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
-			/* Reject XID if too new */
-			if (TransactionIdFollowsOrEquals(xid, origNextXid))
-			{
-				ereport(WARNING,
-						(errmsg("removing future two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
 
-			/*
-			 * Note: we can't check if already processed because clog
-			 * subsystem isn't up yet.
-			 */
+			buf = ProcessTwoPhaseBufferAndReturn(xid, InvalidXLogRecPtr,
+												 true, false, false,
+												 &result, &maxsubxid);
 
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
 			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
 				continue;
-			}
-
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				pfree(buf);
-				continue;
-			}
-
-			/*
-			 * OK, we think this file is valid.  Incorporate xid into the
-			 * running-minimum result.
-			 */
-			if (TransactionIdPrecedes(xid, result))
-				result = xid;
-
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID, and they may force us to advance nextXid.
-			 *
-			 * We don't expect anyone else to modify nextXid, hence we don't
-			 * need to hold a lock while examining it.  We still acquire the
-			 * lock to modify it, though.
-			 */
-			subxids = (TransactionId *) (buf +
-								MAXALIGN(sizeof(TwoPhaseFileHeader)) +
-								MAXALIGN(hdr->gidlen));
-			for (i = 0; i < hdr->nsubxacts; i++)
-			{
-				TransactionId subxid = subxids[i];
-
-				Assert(TransactionIdFollows(subxid, xid));
-				if (TransactionIdFollowsOrEquals(subxid,
-												 ShmemVariableCache->nextXid))
-				{
-					LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
-					ShmemVariableCache->nextXid = subxid;
-					TransactionIdAdvance(ShmemVariableCache->nextXid);
-					LWLockRelease(XidGenLock);
-				}
-			}
-
 
 			if (xids_p)
 			{
@@ -1800,12 +1829,63 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 	}
 	FreeDir(cldir);
 
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		TransactionId xid;
+		char	   *buf;
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+
+		/* only look at entries added by redo and not already on disk */
+		if (!gxact->inredo || gxact->ondisk)
+			continue;
+
+		xid = gxact->xid;
+
+		buf = ProcessTwoPhaseBufferAndReturn(xid, gxact->prepare_start_lsn,
+											 false, false, false,
+											 &result, &maxsubxid);
+
+		if (buf == NULL)
+			continue;
+
+		if (xids_p)
+		{
+			if (nxids == allocsize)
+			{
+				if (nxids == 0)
+				{
+					allocsize = 10;
+					xids = palloc(allocsize * sizeof(TransactionId));
+				}
+				else
+				{
+					allocsize = allocsize * 2;
+					xids = repalloc(xids, allocsize * sizeof(TransactionId));
+				}
+			}
+			xids[nxids++] = xid;
+		}
+
+		pfree(buf);
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
 	if (xids_p)
 	{
 		*xids_p = xids;
 		*nxids_p = nxids;
 	}
 
+	/* update nextXid if needed */
+	if (TransactionIdFollowsOrEquals(maxsubxid, ShmemVariableCache->nextXid))
+	{
+		LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
+		ShmemVariableCache->nextXid = maxsubxid;
+		TransactionIdAdvance(ShmemVariableCache->nextXid);
+		LWLockRelease(XidGenLock);
+	}
+
 	return result;
 }
 
@@ -1814,6 +1894,10 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
  *
  * Scan the pg_twophase directory and setup all the required information to
  * allow standby queries to treat prepared transactions as still active.
+ *
+ * Additionally, scan TwoPhaseState shmem entries that are marked as added
+ * in redo and are not already on disk.
+ *
  * This is never called at the end of recovery - we use
  * RecoverPreparedTransactions() at that point.
  *
@@ -1826,6 +1910,7 @@ StandbyRecoverPreparedTransactions(bool overwriteOK)
 {
 	DIR		   *cldir;
 	struct dirent *clde;
+	int			i;
 
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
@@ -1835,72 +1920,50 @@ StandbyRecoverPreparedTransactions(bool overwriteOK)
 		{
 			TransactionId xid;
 			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
-
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
-
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
+			buf = ProcessTwoPhaseBufferAndReturn(xid, InvalidXLogRecPtr,
+												 true, overwriteOK, true,
+												 NULL, NULL);
+			if (buf != NULL)
 				pfree(buf);
-				continue;
-			}
+		}
+	}
+	FreeDir(cldir);
 
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID.
-			 */
-			subxids = (TransactionId *) (buf +
-								MAXALIGN(sizeof(TwoPhaseFileHeader)) +
-								MAXALIGN(hdr->gidlen));
-			for (i = 0; i < hdr->nsubxacts; i++)
-			{
-				TransactionId subxid = subxids[i];
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		TransactionId xid;
+		char	   *buf;
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
 
-				Assert(TransactionIdFollows(subxid, xid));
-				SubTransSetParent(xid, subxid, overwriteOK);
-			}
+		/* only look at entries added by redo and not already on disk */
+		if (!gxact->inredo || gxact->ondisk)
+			continue;
+
+		xid = gxact->xid;
 
+		buf = ProcessTwoPhaseBufferAndReturn(xid, gxact->prepare_start_lsn,
+											 false, overwriteOK, true,
+											 NULL, NULL);
+		if (buf != NULL)
 			pfree(buf);
-		}
 	}
-	FreeDir(cldir);
+	LWLockRelease(TwoPhaseStateLock);
 }
 
 /*
  * RecoverPreparedTransactions
  *
  * Scan the pg_twophase directory and reload shared-memory state for each
- * prepared transaction (reacquire locks, etc).  This is run during database
- * startup.
+ * prepared transaction (reacquire locks, etc).
+ *
+ * Additionally, scan TwoPhaseState shmem entries that are marked as added
+ * in redo and are not already on disk.
+ *
+ * This is run during database startup.
  */
 void
 RecoverPreparedTransactions(void)
@@ -1909,6 +1972,7 @@ RecoverPreparedTransactions(void)
 	DIR		   *cldir;
 	struct dirent *clde;
 	bool		overwriteOK = false;
+	int			i;
 
 	snprintf(dir, MAXPGPATH, "%s", TWOPHASE_DIR);
 
@@ -1925,35 +1989,17 @@ RecoverPreparedTransactions(void)
 			TransactionId *subxids;
 			GlobalTransaction gxact;
 			const char *gid;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
-
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
+			buf = ProcessTwoPhaseBufferAndReturn(xid, InvalidXLogRecPtr,
+												 true, false, false,
+												 NULL, NULL);
 			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
 				continue;
-			}
 
 			ereport(LOG,
-					(errmsg("recovering prepared transaction %u", xid)));
-
-			/* Deconstruct header */
+				(errmsg("recovering prepared transaction %u from state file", xid)));
 			hdr = (TwoPhaseFileHeader *) buf;
 			Assert(TransactionIdEquals(hdr->xid, xid));
 			bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
@@ -1989,7 +2035,8 @@ RecoverPreparedTransactions(void)
 			 */
 			gxact = MarkAsPreparing(xid, gid,
 									hdr->prepared_at,
-									hdr->owner, hdr->database);
+									hdr->owner, hdr->database,
+									InvalidXLogRecPtr, InvalidXLogRecPtr);
 			gxact->ondisk = true;
 			GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
 			MarkAsPrepared(gxact);
@@ -2018,6 +2065,249 @@ RecoverPreparedTransactions(void)
 		}
 	}
 	FreeDir(cldir);
+
+	/*
+	 * Don't need a lock in the recovery phase.
+	 */
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		TransactionId xid;
+		char	   *buf;
+		char	   *bufptr;
+		TwoPhaseFileHeader *hdr;
+		TransactionId *subxids;
+		const char *gid;
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		GlobalTransaction gxactnew;
+
+		/* only look at entries added by redo and not already on disk */
+		if (!gxact->inredo || gxact->ondisk)
+			continue;
+
+		xid = gxact->xid;
+
+		buf = ProcessTwoPhaseBufferAndReturn(xid, gxact->prepare_start_lsn,
+											 false, false, false,
+											 NULL, NULL);
+		if (buf == NULL)
+			continue;
+
+		ereport(LOG,
+				(errmsg("recovering prepared transaction %u from shared memory", xid)));
+
+		hdr = (TwoPhaseFileHeader *) buf;
+		Assert(TransactionIdEquals(hdr->xid, xid));
+		bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+		gid = (const char *) bufptr;
+		bufptr += MAXALIGN(hdr->gidlen);
+		subxids = (TransactionId *) bufptr;
+		bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
+		bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
+		bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
+		bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+		/*
+		 * It's possible that SubTransSetParent has been set before, if
+		 * the prepared transaction generated xid assignment records. Test
+		 * here must match one used in AssignTransactionId().
+		 */
+		if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
+							 XLogLogicalInfoActive()))
+			overwriteOK = true;
+
+		/*
+		 * Reconstruct subtrans state for the transaction --- needed
+		 * because pg_subtrans is not preserved over a restart.  Note that
+		 * we are linking all the subtransactions directly to the
+		 * top-level XID; there may originally have been a more complex
+		 * hierarchy, but there's no need to restore that exactly.
+		 */
+		for (i = 0; i < hdr->nsubxacts; i++)
+			SubTransSetParent(subxids[i], xid, overwriteOK);
+
+		/*
+		 * Recreate its GXACT and dummy PGPROC
+		 */
+		gxactnew = MarkAsPreparing(xid, gid,
+								hdr->prepared_at,
+								hdr->owner, hdr->database,
+								gxact->prepare_start_lsn,
+								gxact->prepare_end_lsn);
+
+		Assert(gxactnew == gxact);
+		GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
+		MarkAsPrepared(gxact);
+
+		/*
+		 * Recover other state (notably locks) using resource managers
+		 */
+		ProcessRecords(bufptr, xid, twophase_recover_callbacks);
+
+		/*
+		 * Release locks held by the standby process after we process each
+		 * prepared transaction. As a result, we don't need too many
+		 * additional locks at any one time.
+		 */
+		if (InHotStandby)
+			StandbyReleaseLockTree(xid, hdr->nsubxacts, subxids);
+
+		/*
+		 * We're done with recovering this transaction. Clear
+		 * MyLockedGxact, like we do in PrepareTransaction() during normal
+		 * operation.
+		 */
+		PostPrepare_Twophase();
+
+		pfree(buf);
+	}
+}
+
+/*
+ * Given a transaction id, read it either from disk or read it directly
+ * via shmem xlog record pointer using the provided "prepare_start_lsn"
+ *
+ * If setParent is true, then use the overwriteOK parameter to set up
+ * subtransaction parent linkages
+ *
+ * If result and maxsubxid are not NULL, fill them up with smallest
+ * running transaction id (lesser than ShmemVariableCache->nextXid)
+ * and largest subtransaction id for this transaction respectively
+ */
+static char *
+ProcessTwoPhaseBufferAndReturn(TransactionId xid,
+							XLogRecPtr	prepare_start_lsn,
+							bool fromdisk, bool overwriteOK,
+							bool setParent, TransactionId *result,
+							TransactionId *maxsubxid)
+{
+	TransactionId origNextXid = ShmemVariableCache->nextXid;
+	TransactionId res;
+	TransactionId maxsub;
+	TransactionId *subxids;
+	char	   *buf;
+	TwoPhaseFileHeader *hdr;
+	int			i;
+
+	if (!fromdisk)
+		Assert(prepare_start_lsn != InvalidXLogRecPtr);
+
+	if (result)
+		res = *result;
+	if (maxsubxid)
+		maxsub = *maxsubxid;
+
+	/* Already processed? */
+	if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
+	{
+		if (fromdisk)
+		{
+			ereport(WARNING,
+					(errmsg("removing stale two-phase state file for \"%u\"",
+							xid)));
+			RemoveTwoPhaseFile(xid, true);
+		}
+		else
+		{
+			ereport(WARNING,
+					(errmsg("removing stale two-phase state from"
+							" shared memory for \"%u\"", xid)));
+			PrepareRedoRemove(xid);
+		}
+		return NULL;
+	}
+
+	/* Reject XID if too new */
+	if (TransactionIdFollowsOrEquals(xid, origNextXid))
+	{
+		if (fromdisk)
+		{
+			ereport(WARNING,
+					(errmsg("removing future two-phase state file for \"%u\"",
+							xid)));
+			RemoveTwoPhaseFile(xid, true);
+		}
+		else
+		{
+			ereport(WARNING,
+					(errmsg("removing future two-phase state from memory for \"%u\"",
+							xid)));
+			PrepareRedoRemove(xid);
+		}
+		return NULL;
+	}
+
+	if (fromdisk)
+	{
+		/* Read and validate file */
+		buf = ReadTwoPhaseFile(xid, true);
+		if (buf == NULL)
+		{
+			ereport(WARNING,
+					(errmsg("removing corrupt two-phase state file for \"%u\"",
+							xid)));
+			RemoveTwoPhaseFile(xid, true);
+			return NULL;
+		}
+	}
+	else
+	{
+		/* Read xlog data */
+		XlogReadTwoPhaseData(prepare_start_lsn, &buf, NULL);
+	}
+
+	/* Deconstruct header */
+	hdr = (TwoPhaseFileHeader *) buf;
+	if (!TransactionIdEquals(hdr->xid, xid))
+	{
+		if (fromdisk)
+		{
+			ereport(WARNING,
+					(errmsg("removing corrupt two-phase state file for \"%u\"",
+							xid)));
+			RemoveTwoPhaseFile(xid, true);
+		}
+		else
+		{
+			ereport(WARNING,
+					(errmsg("removing corrupt two-phase state from memory for \"%u\"",
+							xid)));
+			PrepareRedoRemove(xid);
+		}
+		pfree(buf);
+		return NULL;
+	}
+
+	/*
+	 * OK, we think this buffer is valid.  Incorporate xid into the
+	 * running-minimum result.
+	 */
+	if (TransactionIdPrecedes(xid, res))
+		res = xid;
+
+	/*
+	 * Examine subtransaction XIDs ... they should all follow main
+	 * XID, and they may force us to advance nextXid.
+	 */
+	subxids = (TransactionId *) (buf +
+								 MAXALIGN(sizeof(TwoPhaseFileHeader)) +
+								 MAXALIGN(hdr->gidlen));
+	for (i = 0; i < hdr->nsubxacts; i++)
+	{
+		TransactionId subxid = subxids[i];
+
+		Assert(TransactionIdFollows(subxid, xid));
+		if (TransactionIdFollowsOrEquals(subxid, maxsub))
+			maxsub = subxid;
+		if (setParent)
+			SubTransSetParent(xid, subxid, overwriteOK);
+	}
+
+	if (result)
+		*result = res;
+	if (maxsubxid)
+		*maxsubxid = maxsub;
+
+	return buf;
 }
 
 /*
@@ -2162,3 +2452,83 @@ RecordTransactionAbortPrepared(TransactionId xid,
 	 */
 	SyncRepWaitForLSN(recptr, false);
 }
+
+/*
+ * PrepareRedoAdd
+ *
+ * Store pointers to the start/end of the WAL record along with the xid in
+ * a gxact entry in shared memory TwoPhaseState structure
+ */
+void
+PrepareRedoAdd(XLogReaderState *record)
+{
+	char	          *buf = XLogRecGetData(record);
+	TwoPhaseFileHeader *hdr = (TwoPhaseFileHeader *) buf;
+	char			  *bufptr;
+	const char		  *gid;
+	GlobalTransaction gxact;
+
+	Assert(RecoveryInProgress());
+
+	bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+	gid = (const char *) bufptr;
+
+	/*
+	 * Add a GXACT entry
+	 */
+	gxact = MarkAsPreparingInRedo(hdr->xid, gid,
+							hdr->prepared_at,
+							hdr->owner, hdr->database,
+							record->ReadRecPtr,
+							record->EndRecPtr);
+
+	elog(DEBUG2, "Adding 2PC data to shared memory %u", gxact->xid);
+}
+
+/*
+ * PrepareRedoRemove
+ *
+ * Remove the corresponding gxact entry from TwoPhaseState. Also
+ * remove the 2PC file.
+ */
+void
+PrepareRedoRemove(TransactionId xid)
+{
+	GlobalTransaction gxact;
+	int			i;
+	bool		found = false;
+
+	Assert(RecoveryInProgress());
+
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		gxact = TwoPhaseState->prepXacts[i];
+
+		if (gxact->xid == xid)
+		{
+			Assert(gxact->inredo);
+			found = true;
+			break;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+	if (found)
+	{
+		/*
+		 * And now we can clean up any files we may have left.
+		 */
+		if (gxact->ondisk)
+			RemoveTwoPhaseFile(xid, true);
+		RemoveGXact(gxact);
+		elog(DEBUG2, "Removing 2PC data from shared memory %u", xid);
+	}
+	else
+	{
+		/*
+		 * Entry could be on disk. Call with giveWarning=false
+		 * since it can be expected during replay.
+		 */
+		RemoveTwoPhaseFile(xid, false);
+	}
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 82f9a3c..2357048 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -2294,7 +2294,8 @@ PrepareTransaction(void)
 	 * GID is invalid or already in use.
 	 */
 	gxact = MarkAsPreparing(xid, prepareGID, prepared_at,
-							GetUserId(), MyDatabaseId);
+							GetUserId(), MyDatabaseId,
+							InvalidXLogRecPtr, InvalidXLogRecPtr);
 	prepareGID = NULL;
 
 	/*
@@ -5606,7 +5607,9 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete TwoPhaseState gxact entry and/or 2PC file. */
+			PrepareRedoRemove(parsed.twophase_xid);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5626,14 +5629,18 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete TwoPhaseState gxact entry and/or 2PC file. */
+			PrepareRedoRemove(parsed.twophase_xid);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		/*
+		 * Store xid and start/end pointers of the WAL record in
+		 * TwoPhaseState gxact entry.
+		 */
+		PrepareRedoAdd(record);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b2b7848..063b946 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -15,6 +15,7 @@
 #define TWOPHASE_H
 
 #include "access/xlogdefs.h"
+#include "access/xlogreader.h"
 #include "datatype/timestamp.h"
 #include "storage/lock.h"
 
@@ -38,7 +39,12 @@ extern BackendId TwoPhaseGetDummyBackendId(TransactionId xid);
 
 extern GlobalTransaction MarkAsPreparing(TransactionId xid, const char *gid,
 				TimestampTz prepared_at,
-				Oid owner, Oid databaseid);
+				Oid owner, Oid databaseid,
+				XLogRecPtr prepare_start_lsn, XLogRecPtr prepare_end_lsn);
+extern GlobalTransaction MarkAsPreparingInRedo(TransactionId xid, const char *gid,
+				TimestampTz prepared_at,
+				Oid owner, Oid databaseid,
+				XLogRecPtr prepare_start_lsn, XLogRecPtr prepare_end_lsn);
 
 extern void StartPrepare(GlobalTransaction gxact);
 extern void EndPrepare(GlobalTransaction gxact);
@@ -56,4 +62,6 @@ extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void PrepareRedoAdd(XLogReaderState *record);
+extern void PrepareRedoRemove(TransactionId xid);
 #endif   /* TWOPHASE_H */
#148Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#147)
Re: Speedup twophase transactions

On Sat, Mar 11, 2017 at 7:26 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

Hi David and Michael,

It would be great to get this thread closed out after 14 months and many
commits.

PFA, latest patch which addresses Michael's comments.

Thanks for the new version. Let's head toward a final patch.

+           ereport(WARNING,
+                   (errmsg("removing future two-phase state data from
memory \"%u\"",
+                           xid)));
+           PrepareRedoRemove(xid);
+           continue
Those are not normal (partially because unlink is atomic, but not
durable)... But they match the correct coding pattern regarding
incorrect 2PC entries... I'd really like to see those switched to a
FATAL with unlink() made durable for those calls.

Hmm, not sure what exactly we need to do here. If you look at the prior
checks, there we already skip on-disk entries. So, typically, the entries
that we encounter here will be in shmem only.

As long as we don't have an alternative to offer a durable unlink,
let's do nothing then. This is as well consistent with the other code
paths handling corrupted or incorrect 2PC entries.

+       /* Deconstruct header */
+       hdr = (TwoPhaseFileHeader *) buf;
+       Assert(TransactionIdEquals(hdr->xid, xid));
+
+       if (TransactionIdPrecedes(xid, result))
+           result = xid;
This portion is repeated three times and could be easily refactored.
You could just have a routine that returns the oldes transaction ID
used, and ignore the result for StandbyRecoverPreparedTransactions()
by casting a (void) to let any kind of static analyzer understand that
we don't care about the result for example. Handling for subxids is
necessary as well depending on the code path. Spliting things into a
could of sub-routines may be more readable as well. There are really a
couple of small parts that can be gathered and strengthened.

Have added a new function to do this now. It reads either from disk or
shared memory and produces error/log messages accordingly as well now.

And that's ProcessTwoPhaseBufferAndReturn in the patch.
ProcessTwoPhaseBuffer may be a better name.

+       /*
+        * Recreate its GXACT and dummy PGPROC
+        */
+       gxactnew = MarkAsPreparing(xid, gid,
+                               hdr->prepared_at,
+                               hdr->owner, hdr->database,
+                               gxact->prepare_start_lsn,
+                               gxact->prepare_end_lsn);
MarkAsPreparing() does not need to be extended with two new arguments.
RecoverPreparedTransactions() is used only at the end of recovery,
where it is not necessary to look at the 2PC state files from the
records. In this code path inredo is also set to false :)

That's not true. We will have entries with inredo set at the end of recovery
as well. Infact the MarkAsPreparing() call from
RecoverPreparedTransactions() is the one which will remove these inredo
entries and convert them into regular entries. We have optimized the
recovery code path as well.

+           /* Delete TwoPhaseState gxact entry and/or 2PC file. */
+           PrepareRedoRemove(parsed.twophase_xid);
Both things should not be present, no? If the file is pushed to disk
it means that the checkpoint horizon has already moved.

PREPARE in redo, followed by a checkpoint, followed by a COMMIT/ROLLBACK. We
can have both the bits set in this case.

Oh, I see where our thoughts don't overlap. I actually thought that
the shared memory entry and the on-disk file cannot co-exist (or if
you want a file flushed at checkpoint should have its shmem entry
removed). But you are right and I am wrong. In order to have the error
handling done properly if the maximum amount of 2PC transactions is
reached. Still....

-           ereport(ERROR,
+           /* It's ok to find an entry in the redo/recovery case */
+           if (!gxact->inredo)
+               ereport(ERROR,
(errcode(ERRCODE_DUPLICATE_OBJECT),
errmsg("transaction identifier \"%s\" is already in
use",
gid)));
+           else
+           {
+               found = true;
+               break;
+           }
I would not have thought so.

Since we are using the TwoPhaseState structure to track redo entries, at end
of recovery, we will find existing entries. Please see my comments above for
RecoverPreparedTransactions()

This is absolutely not good, because it is a direct result of the
interactions of the first loop of RecoverPreparedTransaction() with
its second loop, and because MarkAsPreparing() can finished by being
called *twice* from the same transaction. I really think that this
portion should be removed and that RecoverPreparedTransactions()
should be more careful when scanning the entries in pg_twophase by
looking up at what exists as well in shared memory, instead of doing
that in MarkAsPreparing().

Here are some more comments:

+       /*
+        * Recreate its GXACT and dummy PGPROC
+        */
+       gxactnew = MarkAsPreparing(xid, gid,
+                               hdr->prepared_at,
+                               hdr->owner, hdr->database,
+                               gxact->prepare_start_lsn,
+                               gxact->prepare_end_lsn);
+
+       Assert(gxactnew == gxact);
Here it would be better to set ondisk to false. This makes the code
more consistent with the previous loop, and the intention clear.

The first loop of RecoverPreparedTransactions() has a lot in common
with its second loop. You may want to refactor a little bit more here.

+/*
+ * PrepareRedoRemove
+ *
+ * Remove the corresponding gxact entry from TwoPhaseState. Also
+ * remove the 2PC file.
+ */
This could be a bit more expanded. The removal of the 2PC does not
happen after removing the in-memory data, it would happen if the
in-memory data is not found.
+MarkAsPreparingInRedo(TransactionId xid, const char *gid,
+               TimestampTz prepared_at, Oid owner, Oid databaseid,
+               XLogRecPtr prepare_start_lsn, XLogRecPtr prepare_end_lsn)
+{
+   GlobalTransaction gxact;
+
+   LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
MarkAsPreparingInRedo is internal to twophase.c. There is no need to
expose it externally and it is just used in PrepareRedoAdd so you
could just group both.
    bool        valid;          /* TRUE if PGPROC entry is in proc array */
    bool        ondisk;         /* TRUE if prepare state file is on disk */
+   bool        inredo;         /* TRUE if entry was added via xlog_redo */
We could have a set of flags here, that's the 3rd boolean of the
structure used for a status.
-- 
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#149Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#148)
1 attachment(s)
Re: Speedup twophase transactions

Thanks for the new version. Let's head toward a final patch.

:-)

Have added a new function to do this now. It reads either from disk or

shared memory and produces error/log messages accordingly as well now.

And that's ProcessTwoPhaseBufferAndReturn in the patch.
ProcessTwoPhaseBuffer may be a better name.

Renamed to ProcessTwoPhaseBuffer()

Since we are using the TwoPhaseState structure to track redo entries, at
end

of recovery, we will find existing entries. Please see my comments above

for

RecoverPreparedTransactions()

This is absolutely not good, because it is a direct result of the
interactions of the first loop of RecoverPreparedTransaction() with
its second loop, and because MarkAsPreparing() can finished by being
called *twice* from the same transaction. I really think that this
portion should be removed and that RecoverPreparedTransactions()
should be more careful when scanning the entries in pg_twophase by
looking up at what exists as well in shared memory, instead of doing
that in MarkAsPreparing().

Ok. Modified MarkAsPreparing() to call a new MarkAsPreparingGuts()
function. This function takes in a "gxact' and works on it.

RecoverPreparedTransaction() now calls a newly added
RecoverFromTwoPhaseBuffer() function which checks if an entry already
exists via redo and calls the MarkAsPreparingGuts() function by passing in
that gxact. Otherwise the existing MarkAsPreparing() gets called.

Here are some more comments:

+       /*
+        * Recreate its GXACT and dummy PGPROC
+        */
+       gxactnew = MarkAsPreparing(xid, gid,
+                               hdr->prepared_at,
+                               hdr->owner, hdr->database,
+                               gxact->prepare_start_lsn,
+                               gxact->prepare_end_lsn);
+
+       Assert(gxactnew == gxact);
Here it would be better to set ondisk to false. This makes the code
more consistent with the previous loop, and the intention clear.

Done.

The first loop of RecoverPreparedTransactions() has a lot in common
with its second loop. You may want to refactor a little bit more here.

Done. Added the new function RecoverFromTwoPhaseBuffer() as mentioned above.

+/*
+ * PrepareRedoRemove
+ *
+ * Remove the corresponding gxact entry from TwoPhaseState. Also
+ * remove the 2PC file.
+ */
This could be a bit more expanded. The removal of the 2PC does not
happen after removing the in-memory data, it would happen if the
in-memory data is not found.

Done

+MarkAsPreparingInRedo(TransactionId xid, const char *gid,
+               TimestampTz prepared_at, Oid owner, Oid databaseid,
+               XLogRecPtr prepare_start_lsn, XLogRecPtr prepare_end_lsn)
+{
+   GlobalTransaction gxact;
+
+   LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
MarkAsPreparingInRedo is internal to twophase.c. There is no need to
expose it externally and it is just used in PrepareRedoAdd so you
could just group both.

Removed this MarkAsPreparingInRedo() function and inlined the code in
PrepareRedoAdd().

bool valid; /* TRUE if PGPROC entry is in proc array */

bool ondisk; /* TRUE if prepare state file is on disk */
+ bool inredo; /* TRUE if entry was added via xlog_redo */
We could have a set of flags here, that's the 3rd boolean of the
structure used for a status.

This is more of a cleanup and does not need to be part of this patch. This
can be a follow-on cleanup patch.

I also managed to do some perf testing.

Modified Stas' earlier scripts slightly:

\set naccounts 100000 * :scale

\set from_aid random(1, :naccounts)

\set to_aid random(1, :naccounts)

\set delta random(1, 100)

\set scale :scale+1

BEGIN;

UPDATE pgbench_accounts SET abalance = abalance - :delta WHERE aid =
:from_aid;

UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid =
:to_aid;

PREPARE TRANSACTION ':client_id.:scale';

COMMIT PREPARED ':client_id.:scale';

Created a base backup with scale factor 125 on an AWS t2.large instance.
Set up archiving and did a 20 minute run with the above script saving the
WALs in the archive.

Then used recovery.conf to point to this WAL location and used the base
backup to recover.

With this patch applied: 20s

Without patch: Stopped measuring after 5 minutes ;-)

Regards,

Nikhils

--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services

Attachments:

twophase_recovery_shmem_150317.patchapplication/octet-stream; name=twophase_recovery_shmem_150317.patchDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 5cefc43..9639958 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,26 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *
+ *      During replay and replication, TwoPhaseState also holds information
+ *      about active prepared transactions that haven't been moved to disk yet.
+ *
+ *      Replay of twophase records happens by the following rules:
+ *
+ *      * On PREPARE redo we add the transaction to TwoPhaseState->prepXacts.
+ *        We set gxact->inredo to true for such entries.
+ *
+ *      * On Checkpoint we iterate through TwoPhaseState->prepXacts entries
+ *        that have gxact->inredo set and are behind the redo_horizon. We
+ *        save them to disk and also set gxact->ondisk to true.
+ *
+ *      * On COMMIT/ABORT we delete the entry from TwoPhaseState->prepXacts.
+ *        If gxact->ondisk is true, we delete the corresponding entry from
+ *        the disk as well.
+ *
+ *      * RecoverPreparedTransactions(), StandbyRecoverPreparedTransactions()
+ *        and PrescanPreparedTransactions() have been modified to go through
+ *        gxact->inredo entries that have not made to disk yet.
  *
  *-------------------------------------------------------------------------
  */
@@ -147,11 +165,13 @@ typedef struct GlobalTransactionData
 	 */
 	XLogRecPtr	prepare_start_lsn;		/* XLOG offset of prepare record start */
 	XLogRecPtr	prepare_end_lsn;	/* XLOG offset of prepare record end */
+	TransactionId	xid;			/* The GXACT id */
 
 	Oid			owner;			/* ID of user that executed the xact */
 	BackendId	locking_backend;	/* backend currently working on the xact */
 	bool		valid;			/* TRUE if PGPROC entry is in proc array */
 	bool		ondisk;			/* TRUE if prepare state file is on disk */
+	bool		inredo;			/* TRUE if entry was added via xlog_redo */
 	char		gid[GIDSIZE];	/* The GID assigned to the prepared xact */
 }	GlobalTransactionData;
 
@@ -198,6 +218,16 @@ static void ProcessRecords(char *bufptr, TransactionId xid,
 static void RemoveGXact(GlobalTransaction gxact);
 
 static void XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len);
+static char *ProcessTwoPhaseBuffer(TransactionId xid,
+							XLogRecPtr	prepare_start_lsn,
+							bool fromdisk, bool overwriteOK, bool setParent,
+							TransactionId *result, TransactionId *maxsubxid);
+static void MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid,
+				const char *gid, TimestampTz prepared_at, Oid owner,
+				Oid databaseid, XLogRecPtr prepare_start_lsn,
+				XLogRecPtr prepare_end_lsn);
+static void RecoverFromTwoPhaseBuffer(char *buf, GlobalTransaction gxact,
+									  TransactionId xid);
 
 /*
  * Initialization of shared memory
@@ -342,18 +372,13 @@ PostPrepare_Twophase(void)
 /*
  * MarkAsPreparing
  *		Reserve the GID for the given transaction.
- *
- * Internally, this creates a gxact struct and puts it into the active array.
- * NOTE: this is also used when reloading a gxact after a crash; so avoid
- * assuming that we can use very much backend context.
  */
 GlobalTransaction
 MarkAsPreparing(TransactionId xid, const char *gid,
-				TimestampTz prepared_at, Oid owner, Oid databaseid)
+				TimestampTz prepared_at, Oid owner, Oid databaseid,
+				XLogRecPtr prepare_start_lsn, XLogRecPtr prepare_end_lsn)
 {
 	GlobalTransaction gxact;
-	PGPROC	   *proc;
-	PGXACT	   *pgxact;
 	int			i;
 
 	if (strlen(gid) >= GIDSIZE)
@@ -401,6 +426,38 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 	gxact = TwoPhaseState->freeGXacts;
 	TwoPhaseState->freeGXacts = gxact->next;
 
+	MarkAsPreparingGuts(gxact, xid, gid, prepared_at, owner, databaseid,
+						prepare_start_lsn, prepare_end_lsn);
+
+	/* And insert it into the active array */
+	Assert(TwoPhaseState->numPrepXacts < max_prepared_xacts);
+	TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts++] = gxact;
+
+	LWLockRelease(TwoPhaseStateLock);
+
+	return gxact;
+}
+
+/*
+ * MarkAsPreparingGuts
+ *
+ * This uses a gxact struct and puts it into the active array.
+ * NOTE: this is also used when reloading a gxact after a crash; so avoid
+ * assuming that we can use very much backend context.
+ *
+ * Note: This function should be called with appropriate locks held
+ */
+
+static void
+MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
+				TimestampTz prepared_at, Oid owner, Oid databaseid,
+				XLogRecPtr prepare_start_lsn, XLogRecPtr prepare_end_lsn)
+{
+	PGPROC	   *proc;
+	PGXACT	   *pgxact;
+	int			i;
+
+	Assert(gxact != NULL);
 	proc = &ProcGlobal->allProcs[gxact->pgprocno];
 	pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 
@@ -431,28 +488,23 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 	pgxact->nxids = 0;
 
 	gxact->prepared_at = prepared_at;
-	/* initialize LSN to InvalidXLogRecPtr */
-	gxact->prepare_start_lsn = InvalidXLogRecPtr;
-	gxact->prepare_end_lsn = InvalidXLogRecPtr;
+	/* initialize LSN to passed in values */
+	gxact->prepare_start_lsn = prepare_start_lsn;
+	gxact->prepare_end_lsn = prepare_end_lsn;
+	gxact->xid = xid;
 	gxact->owner = owner;
 	gxact->locking_backend = MyBackendId;
 	gxact->valid = false;
 	gxact->ondisk = false;
+	gxact->inredo = false;
 	strcpy(gxact->gid, gid);
 
-	/* And insert it into the active array */
-	Assert(TwoPhaseState->numPrepXacts < max_prepared_xacts);
-	TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts++] = gxact;
-
 	/*
 	 * Remember that we have this GlobalTransaction entry locked for us. If we
 	 * abort after this, we must release it.
 	 */
 	MyLockedGxact = gxact;
-
-	LWLockRelease(TwoPhaseStateLock);
-
-	return gxact;
+	return;
 }
 
 /*
@@ -1241,9 +1293,9 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
  * Reads 2PC data from xlog. During checkpoint this data will be moved to
  * twophase files and ReadTwoPhaseFile should be used instead.
  *
- * Note clearly that this function accesses WAL during normal operation, similarly
- * to the way WALSender or Logical Decoding would do. It does not run during
- * crash recovery or standby processing.
+ * Note clearly that this function can access WAL during normal operation, similarly
+ * to the way WALSender or Logical Decoding would do.
+ *
  */
 static void
 XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
@@ -1252,8 +1304,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1623,9 +1673,8 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
 	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
 	{
 		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
-		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 
-		if (gxact->valid &&
+		if ((gxact->valid || gxact->inredo) &&
 			!gxact->ondisk &&
 			gxact->prepare_end_lsn <= redo_horizon)
 		{
@@ -1633,7 +1682,7 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
 			int			len;
 
 			XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, &len);
-			RecreateTwoPhaseFile(pgxact->xid, buf, len);
+			RecreateTwoPhaseFile(gxact->xid, buf, len);
 			gxact->ondisk = true;
 			pfree(buf);
 			serialized_xacts++;
@@ -1661,6 +1710,9 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
  * reading WAL.  ShmemVariableCache->nextXid has been set to one more than
  * the highest XID for which evidence exists in WAL.
  *
+ * Additionally, scan TwoPhaseState shmem entries that are marked as added
+ * in redo and are not already on disk.
+ *
  * We throw away any prepared xacts with main XID beyond nextXid --- if any
  * are present, it suggests that the DBA has done a PITR recovery to an
  * earlier point in time without cleaning out pg_twophase.  We dare not
@@ -1685,11 +1737,13 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 {
 	TransactionId origNextXid = ShmemVariableCache->nextXid;
 	TransactionId result = origNextXid;
+	TransactionId maxsubxid = origNextXid;
 	DIR		   *cldir;
 	struct dirent *clde;
 	TransactionId *xids = NULL;
 	int			nxids = 0;
 	int			allocsize = 0;
+	int			i;
 
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
@@ -1699,83 +1753,16 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 		{
 			TransactionId xid;
 			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
-			/* Reject XID if too new */
-			if (TransactionIdFollowsOrEquals(xid, origNextXid))
-			{
-				ereport(WARNING,
-						(errmsg("removing future two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
 
-			/*
-			 * Note: we can't check if already processed because clog
-			 * subsystem isn't up yet.
-			 */
+			buf = ProcessTwoPhaseBuffer(xid, InvalidXLogRecPtr,
+												 true, false, false,
+												 &result, &maxsubxid);
 
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
 			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
-
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				pfree(buf);
 				continue;
-			}
-
-			/*
-			 * OK, we think this file is valid.  Incorporate xid into the
-			 * running-minimum result.
-			 */
-			if (TransactionIdPrecedes(xid, result))
-				result = xid;
-
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID, and they may force us to advance nextXid.
-			 *
-			 * We don't expect anyone else to modify nextXid, hence we don't
-			 * need to hold a lock while examining it.  We still acquire the
-			 * lock to modify it, though.
-			 */
-			subxids = (TransactionId *) (buf +
-								MAXALIGN(sizeof(TwoPhaseFileHeader)) +
-								MAXALIGN(hdr->gidlen));
-			for (i = 0; i < hdr->nsubxacts; i++)
-			{
-				TransactionId subxid = subxids[i];
-
-				Assert(TransactionIdFollows(subxid, xid));
-				if (TransactionIdFollowsOrEquals(subxid,
-												 ShmemVariableCache->nextXid))
-				{
-					LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
-					ShmemVariableCache->nextXid = subxid;
-					TransactionIdAdvance(ShmemVariableCache->nextXid);
-					LWLockRelease(XidGenLock);
-				}
-			}
-
 
 			if (xids_p)
 			{
@@ -1800,12 +1787,63 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 	}
 	FreeDir(cldir);
 
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		TransactionId xid;
+		char	   *buf;
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+
+		/* only look at entries added by redo and not already on disk */
+		if (!gxact->inredo || gxact->ondisk)
+			continue;
+
+		xid = gxact->xid;
+
+		buf = ProcessTwoPhaseBuffer(xid, gxact->prepare_start_lsn,
+											 false, false, false,
+											 &result, &maxsubxid);
+
+		if (buf == NULL)
+			continue;
+
+		if (xids_p)
+		{
+			if (nxids == allocsize)
+			{
+				if (nxids == 0)
+				{
+					allocsize = 10;
+					xids = palloc(allocsize * sizeof(TransactionId));
+				}
+				else
+				{
+					allocsize = allocsize * 2;
+					xids = repalloc(xids, allocsize * sizeof(TransactionId));
+				}
+			}
+			xids[nxids++] = xid;
+		}
+
+		pfree(buf);
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
 	if (xids_p)
 	{
 		*xids_p = xids;
 		*nxids_p = nxids;
 	}
 
+	/* update nextXid if needed */
+	if (TransactionIdFollowsOrEquals(maxsubxid, ShmemVariableCache->nextXid))
+	{
+		LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
+		ShmemVariableCache->nextXid = maxsubxid;
+		TransactionIdAdvance(ShmemVariableCache->nextXid);
+		LWLockRelease(XidGenLock);
+	}
+
 	return result;
 }
 
@@ -1814,6 +1852,10 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
  *
  * Scan the pg_twophase directory and setup all the required information to
  * allow standby queries to treat prepared transactions as still active.
+ *
+ * Additionally, scan TwoPhaseState shmem entries that are marked as added
+ * in redo and are not already on disk.
+ *
  * This is never called at the end of recovery - we use
  * RecoverPreparedTransactions() at that point.
  *
@@ -1826,6 +1868,7 @@ StandbyRecoverPreparedTransactions(bool overwriteOK)
 {
 	DIR		   *cldir;
 	struct dirent *clde;
+	int			i;
 
 	cldir = AllocateDir(TWOPHASE_DIR);
 	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
@@ -1835,72 +1878,50 @@ StandbyRecoverPreparedTransactions(bool overwriteOK)
 		{
 			TransactionId xid;
 			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
-
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
-
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
+			buf = ProcessTwoPhaseBuffer(xid, InvalidXLogRecPtr,
+												 true, overwriteOK, true,
+												 NULL, NULL);
+			if (buf != NULL)
 				pfree(buf);
-				continue;
-			}
+		}
+	}
+	FreeDir(cldir);
 
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID.
-			 */
-			subxids = (TransactionId *) (buf +
-								MAXALIGN(sizeof(TwoPhaseFileHeader)) +
-								MAXALIGN(hdr->gidlen));
-			for (i = 0; i < hdr->nsubxacts; i++)
-			{
-				TransactionId subxid = subxids[i];
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		TransactionId xid;
+		char	   *buf;
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
 
-				Assert(TransactionIdFollows(subxid, xid));
-				SubTransSetParent(xid, subxid, overwriteOK);
-			}
+		/* only look at entries added by redo and not already on disk */
+		if (!gxact->inredo || gxact->ondisk)
+			continue;
 
+		xid = gxact->xid;
+
+		buf = ProcessTwoPhaseBuffer(xid, gxact->prepare_start_lsn,
+											 false, overwriteOK, true,
+											 NULL, NULL);
+		if (buf != NULL)
 			pfree(buf);
-		}
 	}
-	FreeDir(cldir);
+	LWLockRelease(TwoPhaseStateLock);
 }
 
 /*
  * RecoverPreparedTransactions
  *
  * Scan the pg_twophase directory and reload shared-memory state for each
- * prepared transaction (reacquire locks, etc).  This is run during database
- * startup.
+ * prepared transaction (reacquire locks, etc).
+ *
+ * Additionally, scan TwoPhaseState shmem entries that are marked as added
+ * in redo and are not already on disk.
+ *
+ * This is run during database startup.
  */
 void
 RecoverPreparedTransactions(void)
@@ -1908,7 +1929,7 @@ RecoverPreparedTransactions(void)
 	char		dir[MAXPGPATH];
 	DIR		   *cldir;
 	struct dirent *clde;
-	bool		overwriteOK = false;
+	int			i;
 
 	snprintf(dir, MAXPGPATH, "%s", TWOPHASE_DIR);
 
@@ -1920,104 +1941,307 @@ RecoverPreparedTransactions(void)
 		{
 			TransactionId xid;
 			char	   *buf;
-			char	   *bufptr;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			GlobalTransaction gxact;
-			const char *gid;
-			int			i;
 
 			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
 
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
-
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
+			buf = ProcessTwoPhaseBuffer(xid, InvalidXLogRecPtr,
+												 true, false, false,
+												 NULL, NULL);
 			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
 				continue;
-			}
 
 			ereport(LOG,
-					(errmsg("recovering prepared transaction %u", xid)));
-
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			Assert(TransactionIdEquals(hdr->xid, xid));
-			bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
-			gid = (const char *) bufptr;
-			bufptr += MAXALIGN(hdr->gidlen);
-			subxids = (TransactionId *) bufptr;
-			bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
-			bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+				(errmsg("recovering prepared transaction %u from state file", xid)));
 
-			/*
-			 * It's possible that SubTransSetParent has been set before, if
-			 * the prepared transaction generated xid assignment records. Test
-			 * here must match one used in AssignTransactionId().
-			 */
-			if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
-								 XLogLogicalInfoActive()))
-				overwriteOK = true;
+			RecoverFromTwoPhaseBuffer(buf, NULL, xid);
 
-			/*
-			 * Reconstruct subtrans state for the transaction --- needed
-			 * because pg_subtrans is not preserved over a restart.  Note that
-			 * we are linking all the subtransactions directly to the
-			 * top-level XID; there may originally have been a more complex
-			 * hierarchy, but there's no need to restore that exactly.
-			 */
-			for (i = 0; i < hdr->nsubxacts; i++)
-				SubTransSetParent(subxids[i], xid, overwriteOK);
+			pfree(buf);
+		}
+	}
+	FreeDir(cldir);
 
-			/*
-			 * Recreate its GXACT and dummy PGPROC
-			 */
-			gxact = MarkAsPreparing(xid, gid,
-									hdr->prepared_at,
-									hdr->owner, hdr->database);
-			gxact->ondisk = true;
-			GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
-			MarkAsPrepared(gxact);
+	/*
+	 * Don't need a lock in the recovery phase.
+	 */
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		TransactionId xid;
+		char	   *buf;
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
 
-			/*
-			 * Recover other state (notably locks) using resource managers
-			 */
-			ProcessRecords(bufptr, xid, twophase_recover_callbacks);
+		/* only look at entries added by redo and not already on disk */
+		if (!gxact->inredo || gxact->ondisk)
+			continue;
 
-			/*
-			 * Release locks held by the standby process after we process each
-			 * prepared transaction. As a result, we don't need too many
-			 * additional locks at any one time.
-			 */
-			if (InHotStandby)
-				StandbyReleaseLockTree(xid, hdr->nsubxacts, subxids);
+		xid = gxact->xid;
 
-			/*
-			 * We're done with recovering this transaction. Clear
-			 * MyLockedGxact, like we do in PrepareTransaction() during normal
-			 * operation.
-			 */
-			PostPrepare_Twophase();
+		buf = ProcessTwoPhaseBuffer(xid, gxact->prepare_start_lsn,
+											 false, false, false,
+											 NULL, NULL);
+		if (buf == NULL)
+			continue;
 
-			pfree(buf);
+		ereport(LOG,
+				(errmsg("recovering prepared transaction %u from shared memory", xid)));
+
+		RecoverFromTwoPhaseBuffer(buf, gxact, xid);
+
+		pfree(buf);
+	}
+}
+
+/*
+ * Given a transaction id, read it either from disk or read it directly
+ * via shmem xlog record pointer using the provided "prepare_start_lsn"
+ *
+ * If setParent is true, then use the overwriteOK parameter to set up
+ * subtransaction parent linkages
+ *
+ * If result and maxsubxid are not NULL, fill them up with smallest
+ * running transaction id (lesser than ShmemVariableCache->nextXid)
+ * and largest subtransaction id for this transaction respectively
+ */
+static char *
+ProcessTwoPhaseBuffer(TransactionId xid,
+							XLogRecPtr	prepare_start_lsn,
+							bool fromdisk, bool overwriteOK,
+							bool setParent, TransactionId *result,
+							TransactionId *maxsubxid)
+{
+	TransactionId origNextXid = ShmemVariableCache->nextXid;
+	TransactionId res;
+	TransactionId maxsub;
+	TransactionId *subxids;
+	char	   *buf;
+	TwoPhaseFileHeader *hdr;
+	int			i;
+
+	if (!fromdisk)
+		Assert(prepare_start_lsn != InvalidXLogRecPtr);
+
+	if (result)
+		res = *result;
+	if (maxsubxid)
+		maxsub = *maxsubxid;
+
+	/* Already processed? */
+	if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
+	{
+		if (fromdisk)
+		{
+			ereport(WARNING,
+					(errmsg("removing stale two-phase state file for \"%u\"",
+							xid)));
+			RemoveTwoPhaseFile(xid, true);
 		}
+		else
+		{
+			ereport(WARNING,
+					(errmsg("removing stale two-phase state from"
+							" shared memory for \"%u\"", xid)));
+			PrepareRedoRemove(xid);
+		}
+		return NULL;
 	}
-	FreeDir(cldir);
+
+	/* Reject XID if too new */
+	if (TransactionIdFollowsOrEquals(xid, origNextXid))
+	{
+		if (fromdisk)
+		{
+			ereport(WARNING,
+					(errmsg("removing future two-phase state file for \"%u\"",
+							xid)));
+			RemoveTwoPhaseFile(xid, true);
+		}
+		else
+		{
+			ereport(WARNING,
+					(errmsg("removing future two-phase state from memory for \"%u\"",
+							xid)));
+			PrepareRedoRemove(xid);
+		}
+		return NULL;
+	}
+
+	if (fromdisk)
+	{
+		/* Read and validate file */
+		buf = ReadTwoPhaseFile(xid, true);
+		if (buf == NULL)
+		{
+			ereport(WARNING,
+					(errmsg("removing corrupt two-phase state file for \"%u\"",
+							xid)));
+			RemoveTwoPhaseFile(xid, true);
+			return NULL;
+		}
+	}
+	else
+	{
+		/* Read xlog data */
+		XlogReadTwoPhaseData(prepare_start_lsn, &buf, NULL);
+	}
+
+	/* Deconstruct header */
+	hdr = (TwoPhaseFileHeader *) buf;
+	if (!TransactionIdEquals(hdr->xid, xid))
+	{
+		if (fromdisk)
+		{
+			ereport(WARNING,
+					(errmsg("removing corrupt two-phase state file for \"%u\"",
+							xid)));
+			RemoveTwoPhaseFile(xid, true);
+		}
+		else
+		{
+			ereport(WARNING,
+					(errmsg("removing corrupt two-phase state from memory for \"%u\"",
+							xid)));
+			PrepareRedoRemove(xid);
+		}
+		pfree(buf);
+		return NULL;
+	}
+
+	/*
+	 * OK, we think this buffer is valid.  Incorporate xid into the
+	 * running-minimum result.
+	 */
+	if (TransactionIdPrecedes(xid, res))
+		res = xid;
+
+	/*
+	 * Examine subtransaction XIDs ... they should all follow main
+	 * XID, and they may force us to advance nextXid.
+	 */
+	subxids = (TransactionId *) (buf +
+								 MAXALIGN(sizeof(TwoPhaseFileHeader)) +
+								 MAXALIGN(hdr->gidlen));
+	for (i = 0; i < hdr->nsubxacts; i++)
+	{
+		TransactionId subxid = subxids[i];
+
+		Assert(TransactionIdFollows(subxid, xid));
+		if (TransactionIdFollowsOrEquals(subxid, maxsub))
+			maxsub = subxid;
+		if (setParent)
+			SubTransSetParent(xid, subxid, overwriteOK);
+	}
+
+	if (result)
+		*result = res;
+	if (maxsubxid)
+		*maxsubxid = maxsub;
+
+	return buf;
+}
+
+static void
+RecoverFromTwoPhaseBuffer(char *buf, GlobalTransaction gxin, TransactionId xid)
+{
+	char	   *bufptr;
+	TwoPhaseFileHeader *hdr;
+	TransactionId *subxids;
+	GlobalTransaction gxact = gxin;
+	const char *gid;
+	bool		overwriteOK = false;
+	int			i;
+
+	hdr = (TwoPhaseFileHeader *) buf;
+	Assert(TransactionIdEquals(hdr->xid, xid));
+	bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+	gid = (const char *) bufptr;
+	bufptr += MAXALIGN(hdr->gidlen);
+	subxids = (TransactionId *) bufptr;
+	bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
+	bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
+	bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+
+	/*
+	 * It's possible that SubTransSetParent has been set before, if
+	 * the prepared transaction generated xid assignment records. Test
+	 * here must match one used in AssignTransactionId().
+	 */
+	if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
+						 XLogLogicalInfoActive()))
+		overwriteOK = true;
+
+	/*
+	 * Reconstruct subtrans state for the transaction --- needed
+	 * because pg_subtrans is not preserved over a restart.  Note that
+	 * we are linking all the subtransactions directly to the
+	 * top-level XID; there may originally have been a more complex
+	 * hierarchy, but there's no need to restore that exactly.
+	 */
+	for (i = 0; i < hdr->nsubxacts; i++)
+		SubTransSetParent(subxids[i], xid, overwriteOK);
+
+	/*
+	 * Recreate its GXACT and dummy PGPROC. But, check whether
+	 * it was added in redo and already has a shmem entry for
+	 * it
+	 */
+	if (gxact == NULL)
+	{
+		LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+		/* Check for GID */
+		for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+		{
+			gxact = TwoPhaseState->prepXacts[i];
+			if (strcmp(gxact->gid, gid) == 0)
+				break;
+			else
+				gxact = NULL;
+		}
+		LWLockRelease(TwoPhaseStateLock);
+	}
+
+	if (gxact == NULL)
+	{
+		gxact = MarkAsPreparing(xid, gid,
+							hdr->prepared_at,
+							hdr->owner, hdr->database,
+							InvalidXLogRecPtr, InvalidXLogRecPtr);
+		gxact->ondisk = true;
+	}
+	else
+	{
+		Assert(gxact->inredo);
+		LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+		MarkAsPreparingGuts(gxact, xid, gid,
+							hdr->prepared_at,
+							hdr->owner, hdr->database,
+							gxact->prepare_start_lsn, gxact->prepare_end_lsn);
+		gxact->inredo = false; /* recovered, set to false now */
+		LWLockRelease(TwoPhaseStateLock);
+	}
+	GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
+	MarkAsPrepared(gxact);
+
+	/*
+	 * Recover other state (notably locks) using resource managers
+	 */
+	ProcessRecords(bufptr, xid, twophase_recover_callbacks);
+
+	/*
+	 * Release locks held by the standby process after we process each
+	 * prepared transaction. As a result, we don't need too many
+	 * additional locks at any one time.
+	 */
+	if (InHotStandby)
+		StandbyReleaseLockTree(xid, hdr->nsubxacts, subxids);
+
+	/*
+	 * We're done with recovering this transaction. Clear
+	 * MyLockedGxact, like we do in PrepareTransaction() during normal
+	 * operation.
+	 */
+	PostPrepare_Twophase();
+
+	return;
 }
 
 /*
@@ -2162,3 +2386,114 @@ RecordTransactionAbortPrepared(TransactionId xid,
 	 */
 	SyncRepWaitForLSN(recptr, false);
 }
+
+/*
+ * PrepareRedoAdd
+ *
+ * Store pointers to the start/end of the WAL record along with the xid in
+ * a gxact entry in shared memory TwoPhaseState structure
+ */
+void
+PrepareRedoAdd(XLogReaderState *record)
+{
+	char	          *buf = XLogRecGetData(record);
+	TwoPhaseFileHeader *hdr = (TwoPhaseFileHeader *) buf;
+	char			  *bufptr;
+	const char		  *gid;
+	GlobalTransaction gxact;
+
+	Assert(RecoveryInProgress());
+
+	bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+	gid = (const char *) bufptr;
+
+	/*
+	 * Reserve the GID for the given transaction in the redo code path.
+	 *
+	 * This creates a gxact struct and puts it into the active array.
+	 *
+	 * In redo, this struct is mainly used to track PREPARE/COMMIT entries
+	 * in shared memory. Hence, we only fill up the bare minimum contents here.
+	 * The gxact also gets marked with gxact->inredo set to true to indicate
+	 * that it got added in the redo phase
+	 */
+
+	LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+	/* Get a free gxact from the freelist */
+	if (TwoPhaseState->freeGXacts == NULL)
+		ereport(ERROR,
+				(errcode(ERRCODE_OUT_OF_MEMORY),
+				 errmsg("maximum number of prepared transactions reached"),
+				 errhint("Increase max_prepared_transactions (currently %d).",
+						 max_prepared_xacts)));
+	gxact = TwoPhaseState->freeGXacts;
+	TwoPhaseState->freeGXacts = gxact->next;
+
+	gxact->prepared_at = hdr->prepared_at;
+	gxact->prepare_start_lsn = record->ReadRecPtr;
+	gxact->prepare_end_lsn = record->EndRecPtr;
+	gxact->xid = hdr->xid;
+	gxact->owner = hdr->owner;
+	gxact->locking_backend = InvalidBackendId;
+	gxact->valid = false;
+	gxact->ondisk = false;
+	gxact->inredo = true; /* yes, added in redo */
+	strcpy(gxact->gid, gid);
+
+	/* And insert it into the active array */
+	Assert(TwoPhaseState->numPrepXacts < max_prepared_xacts);
+	TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts++] = gxact;
+
+	LWLockRelease(TwoPhaseStateLock);
+
+	elog(DEBUG2, "Adding 2PC data to shared memory %u", gxact->xid);
+}
+
+/*
+ * PrepareRedoRemove
+ *
+ * Remove the corresponding gxact entry from TwoPhaseState. Also
+ * remove the 2PC file if a prepared transaction was saved via
+ * an earlier checkpoint.
+ */
+void
+PrepareRedoRemove(TransactionId xid)
+{
+	GlobalTransaction gxact;
+	int			i;
+	bool		found = false;
+
+	Assert(RecoveryInProgress());
+
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		gxact = TwoPhaseState->prepXacts[i];
+
+		if (gxact->xid == xid)
+		{
+			Assert(gxact->inredo);
+			found = true;
+			break;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+	if (found)
+	{
+		/*
+		 * And now we can clean up any files we may have left.
+		 */
+		if (gxact->ondisk)
+			RemoveTwoPhaseFile(xid, true);
+		RemoveGXact(gxact);
+		elog(DEBUG2, "Removing 2PC data from shared memory %u", xid);
+	}
+	else
+	{
+		/*
+		 * Entry could be on disk. Call with giveWarning=false
+		 * since it can be expected during replay.
+		 */
+		RemoveTwoPhaseFile(xid, false);
+	}
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 82f9a3c..2357048 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -2294,7 +2294,8 @@ PrepareTransaction(void)
 	 * GID is invalid or already in use.
 	 */
 	gxact = MarkAsPreparing(xid, prepareGID, prepared_at,
-							GetUserId(), MyDatabaseId);
+							GetUserId(), MyDatabaseId,
+							InvalidXLogRecPtr, InvalidXLogRecPtr);
 	prepareGID = NULL;
 
 	/*
@@ -5606,7 +5607,9 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete TwoPhaseState gxact entry and/or 2PC file. */
+			PrepareRedoRemove(parsed.twophase_xid);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5626,14 +5629,18 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete TwoPhaseState gxact entry and/or 2PC file. */
+			PrepareRedoRemove(parsed.twophase_xid);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		/*
+		 * Store xid and start/end pointers of the WAL record in
+		 * TwoPhaseState gxact entry.
+		 */
+		PrepareRedoAdd(record);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b2b7848..defeae7 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -15,6 +15,7 @@
 #define TWOPHASE_H
 
 #include "access/xlogdefs.h"
+#include "access/xlogreader.h"
 #include "datatype/timestamp.h"
 #include "storage/lock.h"
 
@@ -38,7 +39,8 @@ extern BackendId TwoPhaseGetDummyBackendId(TransactionId xid);
 
 extern GlobalTransaction MarkAsPreparing(TransactionId xid, const char *gid,
 				TimestampTz prepared_at,
-				Oid owner, Oid databaseid);
+				Oid owner, Oid databaseid,
+				XLogRecPtr prepare_start_lsn, XLogRecPtr prepare_end_lsn);
 
 extern void StartPrepare(GlobalTransaction gxact);
 extern void EndPrepare(GlobalTransaction gxact);
@@ -56,4 +58,6 @@ extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void PrepareRedoAdd(XLogReaderState *record);
+extern void PrepareRedoRemove(TransactionId xid);
 #endif   /* TWOPHASE_H */
#150Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#149)
Re: Speedup twophase transactions

On Wed, Mar 15, 2017 at 4:48 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

bool valid; /* TRUE if PGPROC entry is in proc array
*/
bool ondisk; /* TRUE if prepare state file is on disk
*/
+ bool inredo; /* TRUE if entry was added via xlog_redo
*/
We could have a set of flags here, that's the 3rd boolean of the
structure used for a status.

This is more of a cleanup and does not need to be part of this patch. This
can be a follow-on cleanup patch.

OK, that's fine for me. This patch is complicated enough anyway.

After some time thinking about it, I have finally put my finger on
what was itching me about this patch, and the answer is here:

+ *      Replay of twophase records happens by the following rules:
+ *
+ *      * On PREPARE redo we add the transaction to TwoPhaseState->prepXacts.
+ *        We set gxact->inredo to true for such entries.
+ *
+ *      * On Checkpoint we iterate through TwoPhaseState->prepXacts entries
+ *        that have gxact->inredo set and are behind the redo_horizon. We
+ *        save them to disk and also set gxact->ondisk to true.
+ *
+ *      * On COMMIT/ABORT we delete the entry from TwoPhaseState->prepXacts.
+ *        If gxact->ondisk is true, we delete the corresponding entry from
+ *        the disk as well.
+ *
+ *      * RecoverPreparedTransactions(), StandbyRecoverPreparedTransactions()
+ *        and PrescanPreparedTransactions() have been modified to go through
+ *        gxact->inredo entries that have not made to disk yet.

It seems to me that there should be an initial scan of pg_twophase at
the beginning of recovery, discarding on the way with a WARNING
entries that are older than the checkpoint redo horizon. This should
fill in shmem entries using something close to PrepareRedoAdd(), and
mark those entries as inredo. Then, at the end of recovery,
PrescanPreparedTransactions does not need to look at the entries in
pg_twophase. And that's the case as well of
RecoverPreparedTransaction(). I think that you could get the patch
much simplified this way, as any 2PC data can be fetched directly from
WAL segments and there is no need to rely on scans of pg_twophase,
this is replaced by scans of entries in TwoPhaseState.

I also managed to do some perf testing.

Modified Stas' earlier scripts slightly:

\set naccounts 100000 * :scale
\set from_aid random(1, :naccounts)
\set to_aid random(1, :naccounts)
\set delta random(1, 100)
\set scale :scale+1
BEGIN;
UPDATE pgbench_accounts SET abalance = abalance - :delta WHERE aid =
:from_aid;
UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid =
:to_aid;
PREPARE TRANSACTION ':client_id.:scale';
COMMIT PREPARED ':client_id.:scale';

Created a base backup with scale factor 125 on an AWS t2.large instance. Set
up archiving and did a 20 minute run with the above script saving the WALs
in the archive.

Then used recovery.conf to point to this WAL location and used the base
backup to recover.

With this patch applied: 20s
Without patch: Stopped measuring after 5 minutes ;-)

And that's really nice.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#151Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#150)
Re: Speedup twophase transactions
+ *      * RecoverPreparedTransactions(), StandbyRecoverPreparedTransact
ions()
+ *        and PrescanPreparedTransactions() have been modified to go
throug
+ *        gxact->inredo entries that have not made to disk yet.

It seems to me that there should be an initial scan of pg_twophase at
the beginning of recovery, discarding on the way with a WARNING
entries that are older than the checkpoint redo horizon. This should
fill in shmem entries using something close to PrepareRedoAdd(), and
mark those entries as inredo. Then, at the end of recovery,
PrescanPreparedTransactions does not need to look at the entries in
pg_twophase. And that's the case as well of
RecoverPreparedTransaction(). I think that you could get the patch
much simplified this way, as any 2PC data can be fetched directly from
WAL segments and there is no need to rely on scans of pg_twophase,
this is replaced by scans of entries in TwoPhaseState.

I don't think this will work. We cannot replace pg_twophase with shmem
entries + WAL pointers. This is because we cannot expect to have WAL
entries around for long running prepared queries which survive across
checkpoints.

Regards,
Nikhils

#152Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#151)
Re: Speedup twophase transactions

On Thu, Mar 16, 2017 at 7:18 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

+ *      * RecoverPreparedTransactions(),
StandbyRecoverPreparedTransactions()
+ *        and PrescanPreparedTransactions() have been modified to go
throug
+ *        gxact->inredo entries that have not made to disk yet.

It seems to me that there should be an initial scan of pg_twophase at
the beginning of recovery, discarding on the way with a WARNING
entries that are older than the checkpoint redo horizon. This should
fill in shmem entries using something close to PrepareRedoAdd(), and
mark those entries as inredo. Then, at the end of recovery,
PrescanPreparedTransactions does not need to look at the entries in
pg_twophase. And that's the case as well of
RecoverPreparedTransaction(). I think that you could get the patch
much simplified this way, as any 2PC data can be fetched directly from
WAL segments and there is no need to rely on scans of pg_twophase,
this is replaced by scans of entries in TwoPhaseState.

I don't think this will work. We cannot replace pg_twophase with shmem
entries + WAL pointers. This is because we cannot expect to have WAL entries
around for long running prepared queries which survive across checkpoints.

But at the beginning of recovery, we can mark such entries with ondisk
and inredo, in which case the WAL pointers stored in the shmem entries
do not matter because the data is already on disk.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#153Michael Paquier
michael.paquier@gmail.com
In reply to: Michael Paquier (#152)
Re: Speedup twophase transactions

On Thu, Mar 16, 2017 at 9:25 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:

On Thu, Mar 16, 2017 at 7:18 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

+ *      * RecoverPreparedTransactions(),
StandbyRecoverPreparedTransactions()
+ *        and PrescanPreparedTransactions() have been modified to go
throug
+ *        gxact->inredo entries that have not made to disk yet.

It seems to me that there should be an initial scan of pg_twophase at
the beginning of recovery, discarding on the way with a WARNING
entries that are older than the checkpoint redo horizon. This should
fill in shmem entries using something close to PrepareRedoAdd(), and
mark those entries as inredo. Then, at the end of recovery,
PrescanPreparedTransactions does not need to look at the entries in
pg_twophase. And that's the case as well of
RecoverPreparedTransaction(). I think that you could get the patch
much simplified this way, as any 2PC data can be fetched directly from
WAL segments and there is no need to rely on scans of pg_twophase,
this is replaced by scans of entries in TwoPhaseState.

I don't think this will work. We cannot replace pg_twophase with shmem
entries + WAL pointers. This is because we cannot expect to have WAL entries
around for long running prepared queries which survive across checkpoints.

But at the beginning of recovery, we can mark such entries with ondisk
and inredo, in which case the WAL pointers stored in the shmem entries
do not matter because the data is already on disk.

Nikhil, do you mind if I try something like that? As we already know
what is the first XID when beginning redo via
ShmemVariableCache->nextXid it is possible to discard 2PC files that
should not be here. What makes me worry is the control of the maximum
number of entries in shared memory. If there are legit 2PC files that
are flushed on disk at checkpoint, you would finish with potentially
more 2PC transactions than what should be possible (even if updates of
max_prepared_xacts are WAL-logged).
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#154Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#153)
Re: Speedup twophase transactions

Nikhil, do you mind if I try something like that? As we already know
what is the first XID when beginning redo via
ShmemVariableCache->nextXid it is possible to discard 2PC files that
should not be here.

Yeah, that is ok.

What makes me worry is the control of the maximum
number of entries in shared memory. If there are legit 2PC files that
are flushed on disk at checkpoint, you would finish with potentially
more 2PC transactions than what should be possible (even if updates of
max_prepared_xacts are WAL-logged).

The max_prepared_xacts number restricts the number of pending PREPARED
transactions *across* the 2PC files and shmem inredo entries. We can never
have more entries than this value.

Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services

#155Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#152)
Re: Speedup twophase transactions

I don't think this will work. We cannot replace pg_twophase with shmem
entries + WAL pointers. This is because we cannot expect to have WAL

entries

around for long running prepared queries which survive across

checkpoints.

But at the beginning of recovery, we can mark such entries with ondisk
and inredo, in which case the WAL pointers stored in the shmem entries
do not matter because the data is already on disk.

Ok, we can do that and then yes, RecoverPreparedTransaction() can just have
one loop going through the shmem entries. BUT, we cannot ignore
"inredo"+"ondisk" entries. For such entries, we will have to read and
recover from the corresponding 2PC files.

Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services

#156Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#155)
Re: Speedup twophase transactions

On Fri, Mar 17, 2017 at 4:42 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

I don't think this will work. We cannot replace pg_twophase with shmem
entries + WAL pointers. This is because we cannot expect to have WAL
entries
around for long running prepared queries which survive across
checkpoints.

But at the beginning of recovery, we can mark such entries with ondisk
and inredo, in which case the WAL pointers stored in the shmem entries
do not matter because the data is already on disk.

Ok, we can do that and then yes, RecoverPreparedTransaction() can just have
one loop going through the shmem entries. BUT, we cannot ignore
"inredo"+"ondisk" entries. For such entries, we will have to read and
recover from the corresponding 2PC files.

Yes. About other things I found... In CheckPointTwoPhase(), I am
actually surprised that prepare_start_lsn and prepare_end_lsn are not
reset to InvalidXLogRecPtr when a shmem entry is flushed to disk after
ondisk is set to true, there is no need for them as the data does not
need to be fetched from WAL segments so we had better be consistent
(regression tests fail if I do that). And the two extra arguments in
MarkAsPreparing() are really unnecessary, they get set all the time to
InvalidXLogRecPtr.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#157Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#156)
Re: Speedup twophase transactions

Ok, we can do that and then yes, RecoverPreparedTransaction() can just

have

one loop going through the shmem entries. BUT, we cannot ignore
"inredo"+"ondisk" entries. For such entries, we will have to read and
recover from the corresponding 2PC files.

Yes. About other things I found... In CheckPointTwoPhase(), I am
actually surprised that prepare_start_lsn and prepare_end_lsn are not
reset to InvalidXLogRecPtr when a shmem entry is flushed to disk after
ondisk is set to true, there is no need for them as the data does not
need to be fetched from WAL segments so we had better be consistent
(regression tests fail if I do that). And the two extra arguments in
MarkAsPreparing() are really unnecessary, they get set all the time to
InvalidXLogRecPtr.

Micheal, it looks like you are working on a final version of this patch? I
will wait to review it from my end, then.

Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services

#158Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#157)
Re: Speedup twophase transactions

On Fri, Mar 17, 2017 at 5:00 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

Ok, we can do that and then yes, RecoverPreparedTransaction() can just
have
one loop going through the shmem entries. BUT, we cannot ignore
"inredo"+"ondisk" entries. For such entries, we will have to read and
recover from the corresponding 2PC files.

Yes. About other things I found... In CheckPointTwoPhase(), I am
actually surprised that prepare_start_lsn and prepare_end_lsn are not
reset to InvalidXLogRecPtr when a shmem entry is flushed to disk after
ondisk is set to true, there is no need for them as the data does not
need to be fetched from WAL segments so we had better be consistent
(regression tests fail if I do that). And the two extra arguments in
MarkAsPreparing() are really unnecessary, they get set all the time to
InvalidXLogRecPtr.

Micheal, it looks like you are working on a final version of this patch? I
will wait to review it from my end, then.

I have to admit that I am beginning to get drawn into it...
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#159Michael Paquier
michael.paquier@gmail.com
In reply to: Michael Paquier (#158)
1 attachment(s)
Re: Speedup twophase transactions

On Fri, Mar 17, 2017 at 5:15 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:

On Fri, Mar 17, 2017 at 5:00 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

Micheal, it looks like you are working on a final version of this patch? I
will wait to review it from my end, then.

I have to admit that I am beginning to get drawn into it...

And here is what I got. I have found a couple of inconsistencies in
the patch, roughly:
- During recovery entries marked with ondisk = true should have their
start and end LSN reset to InvalidXLogRecPtr. This was actually
leading to some inconsistencies in MarkAsPreparing() for 2PC
transactions staying around for more than 2 checkpoints.
- RecoverPreparedTransactions(), StandbyRecoverPreparedTransactions()
and PrescanPreparedTransactions() doing both a scan of pg_twophase and
the shared memory entries was way too complicated. I have changed
things so as only memory entries are scanned by those routines, but an
initial scan of pg_twophase is done before recovery.
- Some inconsistencies in the comments and some typos found on the way.
- Simplification of some routines used in redo, as well as simplified
the set of routines made available to users.

Tests are passing for me, an extra lookup would be nice.
--
Michael

Attachments:

twophase_recovery_shmem_michael.patchapplication/octet-stream; name=twophase_recovery_shmem_michael.patchDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index f09941d0ec..8e251a05ce 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,27 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *
+ *		During replay and replication, TwoPhaseState also holds information
+ *		about active prepared transactions that haven't been moved to disk yet.
+ *
+ *		Replay of twophase records happens by the following rules:
+ *
+ *		* At the beginning of recovery, pg_twophase is scanned once, filling
+ *		  TwoPhaseState with entries marked with gxact->inredo and
+ *		  gxact->ondisk.  Two-phase file data older than the XID horizon of
+ *		  the redo position are discarded.
+ *		* On PREPARE redo, the transaction is added TwoPhaseState->prepXacts.
+ *		  gxact->inredo is set to true for such entries.
+ *		* On Checkpoint we iterate through TwoPhaseState->prepXacts entries
+ *		  that have gxact->inredo set and are behind the redo_horizon. We
+ *		  save them to disk and then switch gxact->ondisk to true.
+ *		* On COMMIT/ABORT we delete the entry from TwoPhaseState->prepXacts.
+ *		  If gxact->ondisk is true, the corresponding entry from the disk
+ *		  is additionally deleted.
+ *		* RecoverPreparedTransactions(), StandbyRecoverPreparedTransactions()
+ *		  and PrescanPreparedTransactions() have been modified to go through
+ *		  gxact->inredo entries that have not made it to disk.
  *
  *-------------------------------------------------------------------------
  */
@@ -147,11 +166,13 @@ typedef struct GlobalTransactionData
 	 */
 	XLogRecPtr	prepare_start_lsn;		/* XLOG offset of prepare record start */
 	XLogRecPtr	prepare_end_lsn;	/* XLOG offset of prepare record end */
+	TransactionId	xid;			/* The GXACT id */
 
 	Oid			owner;			/* ID of user that executed the xact */
 	BackendId	locking_backend;	/* backend currently working on the xact */
 	bool		valid;			/* TRUE if PGPROC entry is in proc array */
 	bool		ondisk;			/* TRUE if prepare state file is on disk */
+	bool		inredo;			/* TRUE if entry was added via xlog_redo */
 	char		gid[GIDSIZE];	/* The GID assigned to the prepared xact */
 }	GlobalTransactionData;
 
@@ -198,6 +219,15 @@ static void ProcessRecords(char *bufptr, TransactionId xid,
 static void RemoveGXact(GlobalTransaction gxact);
 
 static void XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len);
+static char *ProcessTwoPhaseBuffer(TransactionId xid,
+							XLogRecPtr	prepare_start_lsn,
+							bool fromdisk, bool overwriteOK, bool setParent,
+							TransactionId *result, TransactionId *maxsubxid);
+static void MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid,
+				const char *gid, TimestampTz prepared_at, Oid owner,
+				Oid databaseid);
+static void RemoveTwoPhaseFile(TransactionId xid, bool giveWarning);
+static void RecreateTwoPhaseFile(TransactionId xid, void *content, int len);
 
 /*
  * Initialization of shared memory
@@ -342,18 +372,12 @@ PostPrepare_Twophase(void)
 /*
  * MarkAsPreparing
  *		Reserve the GID for the given transaction.
- *
- * Internally, this creates a gxact struct and puts it into the active array.
- * NOTE: this is also used when reloading a gxact after a crash; so avoid
- * assuming that we can use very much backend context.
  */
 GlobalTransaction
 MarkAsPreparing(TransactionId xid, const char *gid,
 				TimestampTz prepared_at, Oid owner, Oid databaseid)
 {
 	GlobalTransaction gxact;
-	PGPROC	   *proc;
-	PGXACT	   *pgxact;
 	int			i;
 
 	if (strlen(gid) >= GIDSIZE)
@@ -401,6 +425,37 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 	gxact = TwoPhaseState->freeGXacts;
 	TwoPhaseState->freeGXacts = gxact->next;
 
+	MarkAsPreparingGuts(gxact, xid, gid, prepared_at, owner, databaseid);
+
+	gxact->ondisk = false;
+
+	/* And insert it into the active array */
+	Assert(TwoPhaseState->numPrepXacts < max_prepared_xacts);
+	TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts++] = gxact;
+
+	LWLockRelease(TwoPhaseStateLock);
+
+	return gxact;
+}
+
+/*
+ * MarkAsPreparingGuts
+ *
+ * This uses a gxact struct and puts it into the active array.
+ * NOTE: this is also used when reloading a gxact after a crash; so avoid
+ * assuming that we can use very much backend context.
+ *
+ * Note: This function should be called with appropriate locks held.
+ */
+static void
+MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
+					TimestampTz prepared_at, Oid owner, Oid databaseid)
+{
+	PGPROC	   *proc;
+	PGXACT	   *pgxact;
+	int			i;
+
+	Assert(gxact != NULL);
 	proc = &ProcGlobal->allProcs[gxact->pgprocno];
 	pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 
@@ -431,28 +486,18 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 	pgxact->nxids = 0;
 
 	gxact->prepared_at = prepared_at;
-	/* initialize LSN to InvalidXLogRecPtr */
-	gxact->prepare_start_lsn = InvalidXLogRecPtr;
-	gxact->prepare_end_lsn = InvalidXLogRecPtr;
+	gxact->xid = xid;
 	gxact->owner = owner;
 	gxact->locking_backend = MyBackendId;
 	gxact->valid = false;
-	gxact->ondisk = false;
+	gxact->inredo = false;
 	strcpy(gxact->gid, gid);
 
-	/* And insert it into the active array */
-	Assert(TwoPhaseState->numPrepXacts < max_prepared_xacts);
-	TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts++] = gxact;
-
 	/*
 	 * Remember that we have this GlobalTransaction entry locked for us. If we
 	 * abort after this, we must release it.
 	 */
 	MyLockedGxact = gxact;
-
-	LWLockRelease(TwoPhaseStateLock);
-
-	return gxact;
 }
 
 /*
@@ -1244,9 +1289,9 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
  * Reads 2PC data from xlog. During checkpoint this data will be moved to
  * twophase files and ReadTwoPhaseFile should be used instead.
  *
- * Note clearly that this function accesses WAL during normal operation, similarly
- * to the way WALSender or Logical Decoding would do. It does not run during
- * crash recovery or standby processing.
+ * Note clearly that this function can access WAL during normal operation,
+ * similarly to the way WALSender or Logical Decoding would do.
+ *
  */
 static void
 XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
@@ -1255,8 +1300,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1501,7 +1544,7 @@ ProcessRecords(char *bufptr, TransactionId xid,
  * If giveWarning is false, do not complain about file-not-present;
  * this is an expected case during WAL replay.
  */
-void
+static void
 RemoveTwoPhaseFile(TransactionId xid, bool giveWarning)
 {
 	char		path[MAXPGPATH];
@@ -1521,7 +1564,7 @@ RemoveTwoPhaseFile(TransactionId xid, bool giveWarning)
  *
  * Note: content and len don't include CRC.
  */
-void
+static void
 RecreateTwoPhaseFile(TransactionId xid, void *content, int len)
 {
 	char		path[MAXPGPATH];
@@ -1587,9 +1630,11 @@ RecreateTwoPhaseFile(TransactionId xid, void *content, int len)
 /*
  * CheckPointTwoPhase -- handle 2PC component of checkpointing.
  *
- * We must fsync the state file of any GXACT that is valid and has a PREPARE
- * LSN <= the checkpoint's redo horizon.  (If the gxact isn't valid yet or
- * has a later LSN, this checkpoint is not responsible for fsyncing it.)
+ * We must fsync the state file of any GXACT that is valid or has been
+ * generated during redo and has a PREPARE LSN <= the checkpoint's redo
+ * horizon.  (If the gxact isn't valid yet, has not been generated in
+ * redo, or has a later LSN, this checkpoint is not responsible for
+ * fsyncing it.)
  *
  * This is deliberately run as late as possible in the checkpoint sequence,
  * because GXACTs ordinarily have short lifespans, and so it is quite
@@ -1632,9 +1677,8 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
 	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
 	{
 		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
-		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 
-		if (gxact->valid &&
+		if ((gxact->valid || gxact->inredo) &&
 			!gxact->ondisk &&
 			gxact->prepare_end_lsn <= redo_horizon)
 		{
@@ -1642,8 +1686,10 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
 			int			len;
 
 			XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, &len);
-			RecreateTwoPhaseFile(pgxact->xid, buf, len);
+			RecreateTwoPhaseFile(gxact->xid, buf, len);
 			gxact->ondisk = true;
+			gxact->prepare_start_lsn = InvalidXLogRecPtr;
+			gxact->prepare_end_lsn = InvalidXLogRecPtr;
 			pfree(buf);
 			serialized_xacts++;
 		}
@@ -1663,12 +1709,49 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
 }
 
 /*
+ * restoreTwoPhaseData
+ *
+ * Scan pg_twophase and fill TwoPhaseState depending on the on-disk data.
+ * This is called once at the beginning of recovery, saving any extra
+ * lookups in the future.  Two-phase files that are newer than the
+ * minimum XID horizon are discarded on the way.
+ */
+void
+restoreTwoPhaseData(void)
+{
+	DIR			   *cldir;
+	struct dirent  *clde;
+
+	cldir = AllocateDir(TWOPHASE_DIR);
+	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
+	{
+		if (strlen(clde->d_name) == 8 &&
+			strspn(clde->d_name, "0123456789ABCDEF") == 8)
+		{
+			TransactionId xid;
+			char	   *buf;
+
+			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
+
+			buf = ProcessTwoPhaseBuffer(xid, InvalidXLogRecPtr,
+										true, false, false,
+										NULL, NULL);
+			if (buf == NULL)
+				continue;
+
+			PrepareRedoAdd(buf, InvalidXLogRecPtr, InvalidXLogRecPtr);
+		}
+	}
+	FreeDir(cldir);
+}
+
+/*
  * PrescanPreparedTransactions
  *
- * Scan the pg_twophase directory and determine the range of valid XIDs
- * present.  This is run during database startup, after we have completed
- * reading WAL.  ShmemVariableCache->nextXid has been set to one more than
- * the highest XID for which evidence exists in WAL.
+ * Scan the shared memory entries of TwoPhaseState and determine the range
+ * of valid XIDs present.  This is run during database startup, after we
+ * have completed reading WAL.  ShmemVariableCache->nextXid has been set to
+ * one more than the highest XID for which evidence exists in WAL.
  *
  * We throw away any prepared xacts with main XID beyond nextXid --- if any
  * are present, it suggests that the DBA has done a PITR recovery to an
@@ -1694,120 +1777,52 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 {
 	TransactionId origNextXid = ShmemVariableCache->nextXid;
 	TransactionId result = origNextXid;
-	DIR		   *cldir;
-	struct dirent *clde;
+	TransactionId maxsubxid = origNextXid;
 	TransactionId *xids = NULL;
 	int			nxids = 0;
 	int			allocsize = 0;
+	int			i;
 
-	cldir = AllocateDir(TWOPHASE_DIR);
-	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
 	{
-		if (strlen(clde->d_name) == 8 &&
-			strspn(clde->d_name, "0123456789ABCDEF") == 8)
-		{
-			TransactionId xid;
-			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
-
-			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
-
-			/* Reject XID if too new */
-			if (TransactionIdFollowsOrEquals(xid, origNextXid))
-			{
-				ereport(WARNING,
-						(errmsg("removing future two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+		TransactionId xid;
+		char	   *buf;
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
 
-			/*
-			 * Note: we can't check if already processed because clog
-			 * subsystem isn't up yet.
-			 */
+		Assert(gxact->inredo);
 
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+		xid = gxact->xid;
 
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				pfree(buf);
-				continue;
-			}
+		buf = ProcessTwoPhaseBuffer(xid,
+				gxact->prepare_start_lsn,
+				gxact->ondisk, false, false,
+				&result, &maxsubxid);
 
-			/*
-			 * OK, we think this file is valid.  Incorporate xid into the
-			 * running-minimum result.
-			 */
-			if (TransactionIdPrecedes(xid, result))
-				result = xid;
+		if (buf == NULL)
+			continue;
 
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID, and they may force us to advance nextXid.
-			 *
-			 * We don't expect anyone else to modify nextXid, hence we don't
-			 * need to hold a lock while examining it.  We still acquire the
-			 * lock to modify it, though.
-			 */
-			subxids = (TransactionId *) (buf +
-								MAXALIGN(sizeof(TwoPhaseFileHeader)) +
-								MAXALIGN(hdr->gidlen));
-			for (i = 0; i < hdr->nsubxacts; i++)
+		if (xids_p)
+		{
+			if (nxids == allocsize)
 			{
-				TransactionId subxid = subxids[i];
-
-				Assert(TransactionIdFollows(subxid, xid));
-				if (TransactionIdFollowsOrEquals(subxid,
-												 ShmemVariableCache->nextXid))
+				if (nxids == 0)
 				{
-					LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
-					ShmemVariableCache->nextXid = subxid;
-					TransactionIdAdvance(ShmemVariableCache->nextXid);
-					LWLockRelease(XidGenLock);
+					allocsize = 10;
+					xids = palloc(allocsize * sizeof(TransactionId));
 				}
-			}
-
-
-			if (xids_p)
-			{
-				if (nxids == allocsize)
+				else
 				{
-					if (nxids == 0)
-					{
-						allocsize = 10;
-						xids = palloc(allocsize * sizeof(TransactionId));
-					}
-					else
-					{
-						allocsize = allocsize * 2;
-						xids = repalloc(xids, allocsize * sizeof(TransactionId));
-					}
+					allocsize = allocsize * 2;
+					xids = repalloc(xids, allocsize * sizeof(TransactionId));
 				}
-				xids[nxids++] = xid;
 			}
-
-			pfree(buf);
+			xids[nxids++] = xid;
 		}
+
+		pfree(buf);
 	}
-	FreeDir(cldir);
+	LWLockRelease(TwoPhaseStateLock);
 
 	if (xids_p)
 	{
@@ -1815,14 +1830,25 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 		*nxids_p = nxids;
 	}
 
+	/* update nextXid if needed */
+	if (TransactionIdFollowsOrEquals(maxsubxid, ShmemVariableCache->nextXid))
+	{
+		LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
+		ShmemVariableCache->nextXid = maxsubxid;
+		TransactionIdAdvance(ShmemVariableCache->nextXid);
+		LWLockRelease(XidGenLock);
+	}
+
 	return result;
 }
 
 /*
  * StandbyRecoverPreparedTransactions
  *
- * Scan the pg_twophase directory and setup all the required information to
- * allow standby queries to treat prepared transactions as still active.
+ * Scan the shared memory entries of TwoPhaseState and setup all the required
+ * information to allow standby queries to treat prepared transactions as still
+ * active.
+ *
  * This is never called at the end of recovery - we use
  * RecoverPreparedTransactions() at that point.
  *
@@ -1833,202 +1859,292 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 void
 StandbyRecoverPreparedTransactions(bool overwriteOK)
 {
-	DIR		   *cldir;
-	struct dirent *clde;
+	int			i;
 
-	cldir = AllocateDir(TWOPHASE_DIR);
-	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
 	{
-		if (strlen(clde->d_name) == 8 &&
-			strspn(clde->d_name, "0123456789ABCDEF") == 8)
-		{
-			TransactionId xid;
-			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
-
-			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
-
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
-
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+		TransactionId xid;
+		char	   *buf;
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
 
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				pfree(buf);
-				continue;
-			}
+		Assert(gxact->inredo);
 
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID.
-			 */
-			subxids = (TransactionId *) (buf +
-								MAXALIGN(sizeof(TwoPhaseFileHeader)) +
-								MAXALIGN(hdr->gidlen));
-			for (i = 0; i < hdr->nsubxacts; i++)
-			{
-				TransactionId subxid = subxids[i];
-
-				Assert(TransactionIdFollows(subxid, xid));
-				SubTransSetParent(xid, subxid, overwriteOK);
-			}
+		xid = gxact->xid;
 
+		buf = ProcessTwoPhaseBuffer(xid,
+				gxact->prepare_start_lsn,
+				gxact->ondisk, overwriteOK, true,
+				NULL, NULL);
+		if (buf != NULL)
 			pfree(buf);
-		}
 	}
-	FreeDir(cldir);
+	LWLockRelease(TwoPhaseStateLock);
 }
 
 /*
  * RecoverPreparedTransactions
  *
- * Scan the pg_twophase directory and reload shared-memory state for each
- * prepared transaction (reacquire locks, etc).  This is run during database
- * startup.
+ * Scan the shared memory entries of TwoPhaseState and reload the state for
+ * each prepared transaction (reacquire locks, etc).
+ *
+ * This is run during database startup.
  */
 void
 RecoverPreparedTransactions(void)
 {
-	char		dir[MAXPGPATH];
-	DIR		   *cldir;
-	struct dirent *clde;
-	bool		overwriteOK = false;
-
-	snprintf(dir, MAXPGPATH, "%s", TWOPHASE_DIR);
+	int			i;
 
-	cldir = AllocateDir(dir);
-	while ((clde = ReadDir(cldir, dir)) != NULL)
+	/*
+	 * Don't need a lock in the recovery phase.
+	 */
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
 	{
-		if (strlen(clde->d_name) == 8 &&
-			strspn(clde->d_name, "0123456789ABCDEF") == 8)
-		{
-			TransactionId xid;
-			char	   *buf;
-			char	   *bufptr;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			GlobalTransaction gxact;
-			const char *gid;
-			int			i;
+		TransactionId xid;
+		char	   *buf;
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		char	   *bufptr;
+		TwoPhaseFileHeader *hdr;
+		TransactionId *subxids;
+		const char *gid;
+		bool		overwriteOK = false;
+		int			i;
 
-			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
+		xid = gxact->xid;
 
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+		buf = ProcessTwoPhaseBuffer(xid,
+				gxact->prepare_start_lsn,
+				gxact->ondisk, false, false,
+				NULL, NULL);
+		if (buf == NULL)
+			continue;
 
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+		ereport(LOG,
+				(errmsg("recovering prepared transaction %u from shared memory", xid)));
+
+		hdr = (TwoPhaseFileHeader *) buf;
+		Assert(TransactionIdEquals(hdr->xid, xid));
+		bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+		gid = (const char *) bufptr;
+		bufptr += MAXALIGN(hdr->gidlen);
+		subxids = (TransactionId *) bufptr;
+		bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
+		bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
+		bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
+		bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
 
-			ereport(LOG,
-					(errmsg("recovering prepared transaction %u", xid)));
-
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			Assert(TransactionIdEquals(hdr->xid, xid));
-			bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
-			gid = (const char *) bufptr;
-			bufptr += MAXALIGN(hdr->gidlen);
-			subxids = (TransactionId *) bufptr;
-			bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
-			bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+		/*
+		 * It's possible that SubTransSetParent has been set before, if
+		 * the prepared transaction generated xid assignment records. Test
+		 * here must match one used in AssignTransactionId().
+		 */
+		if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
+							 XLogLogicalInfoActive()))
+			overwriteOK = true;
 
-			/*
-			 * It's possible that SubTransSetParent has been set before, if
-			 * the prepared transaction generated xid assignment records. Test
-			 * here must match one used in AssignTransactionId().
-			 */
-			if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
-								 XLogLogicalInfoActive()))
-				overwriteOK = true;
+		/*
+		 * Reconstruct subtrans state for the transaction --- needed
+		 * because pg_subtrans is not preserved over a restart.  Note that
+		 * we are linking all the subtransactions directly to the
+		 * top-level XID; there may originally have been a more complex
+		 * hierarchy, but there's no need to restore that exactly.
+		 */
+		for (i = 0; i < hdr->nsubxacts; i++)
+			SubTransSetParent(subxids[i], xid, overwriteOK);
 
-			/*
-			 * Reconstruct subtrans state for the transaction --- needed
-			 * because pg_subtrans is not preserved over a restart.  Note that
-			 * we are linking all the subtransactions directly to the
-			 * top-level XID; there may originally have been a more complex
-			 * hierarchy, but there's no need to restore that exactly.
-			 */
-			for (i = 0; i < hdr->nsubxacts; i++)
-				SubTransSetParent(subxids[i], xid, overwriteOK);
+		/*
+		 * Recreate its GXACT and dummy PGPROC. But, check whether
+		 * it was added in redo and already has a shmem entry for
+		 * it.
+		 */
+		LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+		MarkAsPreparingGuts(gxact, xid, gid,
+							hdr->prepared_at,
+							hdr->owner, hdr->database);
 
-			/*
-			 * Recreate its GXACT and dummy PGPROC
-			 */
-			gxact = MarkAsPreparing(xid, gid,
-									hdr->prepared_at,
-									hdr->owner, hdr->database);
-			gxact->ondisk = true;
-			GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
-			MarkAsPrepared(gxact);
+		/* recovered, so reset the flag for entries generated by redo */
+		gxact->inredo = false;
 
-			/*
-			 * Recover other state (notably locks) using resource managers
-			 */
-			ProcessRecords(bufptr, xid, twophase_recover_callbacks);
+		LWLockRelease(TwoPhaseStateLock);
 
-			/*
-			 * Release locks held by the standby process after we process each
-			 * prepared transaction. As a result, we don't need too many
-			 * additional locks at any one time.
-			 */
-			if (InHotStandby)
-				StandbyReleaseLockTree(xid, hdr->nsubxacts, subxids);
+		GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
+		MarkAsPrepared(gxact);
 
-			/*
-			 * We're done with recovering this transaction. Clear
-			 * MyLockedGxact, like we do in PrepareTransaction() during normal
-			 * operation.
-			 */
-			PostPrepare_Twophase();
+		/*
+		 * Recover other state (notably locks) using resource managers
+		 */
+		ProcessRecords(bufptr, xid, twophase_recover_callbacks);
 
-			pfree(buf);
+		/*
+		 * Release locks held by the standby process after we process each
+		 * prepared transaction. As a result, we don't need too many
+		 * additional locks at any one time.
+		 */
+		if (InHotStandby)
+			StandbyReleaseLockTree(xid, hdr->nsubxacts, subxids);
+
+		/*
+		 * We're done with recovering this transaction. Clear
+		 * MyLockedGxact, like we do in PrepareTransaction() during normal
+		 * operation.
+		 */
+		PostPrepare_Twophase();
+
+		pfree(buf);
+	}
+}
+
+/*
+ * ProcessTwoPhaseBuffer
+ *
+ * Given a transaction id, read it either from disk or read it directly
+ * via shmem xlog record pointer using the provided "prepare_start_lsn".
+ *
+ * If setParent is true, then use the overwriteOK parameter to set up
+ * subtransaction parent linkages.
+ *
+ * If result and maxsubxid are not NULL, fill them up with smallest
+ * running transaction id (lesser than ShmemVariableCache->nextXid)
+ * and largest subtransaction id for this transaction respectively.
+ */
+static char *
+ProcessTwoPhaseBuffer(TransactionId xid,
+					  XLogRecPtr prepare_start_lsn,
+					  bool fromdisk, bool overwriteOK,
+					  bool setParent, TransactionId *result,
+					  TransactionId *maxsubxid)
+{
+	TransactionId origNextXid = ShmemVariableCache->nextXid;
+	TransactionId res;
+	TransactionId maxsub;
+	TransactionId *subxids;
+	char	   *buf;
+	TwoPhaseFileHeader *hdr;
+	int			i;
+
+	if (!fromdisk)
+		Assert(prepare_start_lsn != InvalidXLogRecPtr);
+
+	if (result)
+		res = *result;
+	if (maxsubxid)
+		maxsub = *maxsubxid;
+
+	/* Already processed? */
+	if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
+	{
+		if (fromdisk)
+		{
+			ereport(WARNING,
+					(errmsg("removing stale two-phase state file for \"%u\"",
+							xid)));
+			RemoveTwoPhaseFile(xid, true);
 		}
+		else
+		{
+			ereport(WARNING,
+					(errmsg("removing stale two-phase state from"
+							" shared memory for \"%u\"", xid)));
+			PrepareRedoRemove(xid, true);
+		}
+		return NULL;
 	}
-	FreeDir(cldir);
+
+	/* Reject XID if too new */
+	if (TransactionIdFollowsOrEquals(xid, origNextXid))
+	{
+		if (fromdisk)
+		{
+			ereport(WARNING,
+					(errmsg("removing future two-phase state file for \"%u\"",
+							xid)));
+			RemoveTwoPhaseFile(xid, true);
+		}
+		else
+		{
+			ereport(WARNING,
+					(errmsg("removing future two-phase state from memory for \"%u\"",
+							xid)));
+			PrepareRedoRemove(xid, true);
+		}
+		return NULL;
+	}
+
+	if (fromdisk)
+	{
+		/* Read and validate file */
+		buf = ReadTwoPhaseFile(xid, true);
+		if (buf == NULL)
+		{
+			ereport(WARNING,
+					(errmsg("removing corrupt two-phase state file for \"%u\"",
+							xid)));
+			RemoveTwoPhaseFile(xid, true);
+			return NULL;
+		}
+	}
+	else
+	{
+		/* Read xlog data */
+		XlogReadTwoPhaseData(prepare_start_lsn, &buf, NULL);
+	}
+
+	/* Deconstruct header */
+	hdr = (TwoPhaseFileHeader *) buf;
+	if (!TransactionIdEquals(hdr->xid, xid))
+	{
+		if (fromdisk)
+		{
+			ereport(WARNING,
+					(errmsg("removing corrupt two-phase state file for \"%u\"",
+							xid)));
+			RemoveTwoPhaseFile(xid, true);
+		}
+		else
+		{
+			ereport(WARNING,
+					(errmsg("removing corrupt two-phase state from memory for \"%u\"",
+							xid)));
+			PrepareRedoRemove(xid, true);
+		}
+		pfree(buf);
+		return NULL;
+	}
+
+	/*
+	 * OK, we think this buffer is valid.  Incorporate xid into the
+	 * running-minimum result.
+	 */
+	if (TransactionIdPrecedes(xid, res))
+		res = xid;
+
+	/*
+	 * Examine subtransaction XIDs ... they should all follow main
+	 * XID, and they may force us to advance nextXid.
+	 */
+	subxids = (TransactionId *) (buf +
+								 MAXALIGN(sizeof(TwoPhaseFileHeader)) +
+								 MAXALIGN(hdr->gidlen));
+	for (i = 0; i < hdr->nsubxacts; i++)
+	{
+		TransactionId subxid = subxids[i];
+
+		Assert(TransactionIdFollows(subxid, xid));
+		if (TransactionIdFollowsOrEquals(subxid, maxsub))
+			maxsub = subxid;
+		if (setParent)
+			SubTransSetParent(xid, subxid, overwriteOK);
+	}
+
+	if (result)
+		*result = res;
+	if (maxsubxid)
+		*maxsubxid = maxsub;
+
+	return buf;
 }
 
+
 /*
  *	RecordTransactionCommitPrepared
  *
@@ -2171,3 +2287,111 @@ RecordTransactionAbortPrepared(TransactionId xid,
 	 */
 	SyncRepWaitForLSN(recptr, false);
 }
+
+/*
+ * PrepareRedoAdd
+ *
+ * Store pointers to the start/end of the WAL record along with the xid in
+ * a gxact entry in shared memory TwoPhaseState structure.  If caller
+ * specifies InvalidXLogRecPtr as WAL position to fetch the two-phase
+ * data, the entry is marked as located on disk.
+ */
+void
+PrepareRedoAdd(char *buf, XLogRecPtr start_lsn, XLogRecPtr end_lsn)
+{
+	TwoPhaseFileHeader *hdr = (TwoPhaseFileHeader *) buf;
+	char			  *bufptr;
+	const char		  *gid;
+	GlobalTransaction gxact;
+
+	Assert(RecoveryInProgress());
+
+	bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+	gid = (const char *) bufptr;
+
+	/*
+	 * Reserve the GID for the given transaction in the redo code path.
+	 *
+	 * This creates a gxact struct and puts it into the active array.
+	 *
+	 * In redo, this struct is mainly used to track PREPARE/COMMIT entries
+	 * in shared memory. Hence, we only fill up the bare minimum contents here.
+	 * The gxact also gets marked with gxact->inredo set to true to indicate
+	 * that it got added in the redo phase
+	 */
+
+	LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+	/* Get a free gxact from the freelist */
+	if (TwoPhaseState->freeGXacts == NULL)
+		ereport(ERROR,
+				(errcode(ERRCODE_OUT_OF_MEMORY),
+				 errmsg("maximum number of prepared transactions reached"),
+				 errhint("Increase max_prepared_transactions (currently %d).",
+						 max_prepared_xacts)));
+	gxact = TwoPhaseState->freeGXacts;
+	TwoPhaseState->freeGXacts = gxact->next;
+
+	gxact->prepared_at = hdr->prepared_at;
+	gxact->prepare_start_lsn = start_lsn;
+	gxact->prepare_end_lsn = end_lsn;
+	gxact->xid = hdr->xid;
+	gxact->owner = hdr->owner;
+	gxact->locking_backend = InvalidBackendId;
+	gxact->valid = false;
+	gxact->ondisk = XLogRecPtrIsInvalid(start_lsn);
+	gxact->inredo = true; /* yes, added in redo */
+	strcpy(gxact->gid, gid);
+
+	/* And insert it into the active array */
+	Assert(TwoPhaseState->numPrepXacts < max_prepared_xacts);
+	TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts++] = gxact;
+
+	LWLockRelease(TwoPhaseStateLock);
+
+	elog(DEBUG2, "Adding 2PC data to shared memory %u", gxact->xid);
+}
+
+/*
+ * PrepareRedoRemove
+ *
+ * Remove the corresponding gxact entry from TwoPhaseState. Also
+ * remove the 2PC file if a prepared transaction was saved via
+ * an earlier checkpoint.
+ */
+void
+PrepareRedoRemove(TransactionId xid, bool giveWarning)
+{
+	GlobalTransaction gxact = NULL;
+	int			i;
+
+	Assert(RecoveryInProgress());
+
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		gxact = TwoPhaseState->prepXacts[i];
+
+		if (gxact->xid == xid)
+		{
+			Assert(gxact->inredo);
+			break;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+	/*
+	 * Just leave if there is nothing, this is expected during WAL replay.
+	 */
+	if (gxact == NULL)
+		return;
+
+	/*
+	 * And now we can clean up any files we may have left.
+	 */
+	elog(DEBUG2, "Removing 2PC data from shared memory %u", xid);
+	if (gxact->ondisk)
+		RemoveTwoPhaseFile(xid, giveWarning);
+	RemoveGXact(gxact);
+
+	return;
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 02e0779f32..e411b413ae 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -5606,7 +5606,9 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete TwoPhaseState gxact entry and/or 2PC file. */
+			PrepareRedoRemove(parsed.twophase_xid, false);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5626,14 +5628,20 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete TwoPhaseState gxact entry and/or 2PC file. */
+			PrepareRedoRemove(parsed.twophase_xid, false);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		/*
+		 * Store xid and start/end pointers of the WAL record in
+		 * TwoPhaseState gxact entry.
+		 */
+		PrepareRedoAdd(XLogRecGetData(record),
+					   record->ReadRecPtr,
+					   record->EndRecPtr);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 9480377611..af6dc7fc3a 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6691,6 +6691,16 @@ StartupXLOG(void)
 	 */
 	restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
 
+	/*
+	 * Before running in recovery, scan pg_twophase and fill in its status
+	 * to be able to work on entries generated by redo.  Doing a scan before
+	 * taking any recovery action has the merit to discard any 2PC files that
+	 * are newer than the first record to replay, saving from any conflicts at
+	 * replay.  This avoids as well any subsequent scans when doing recovery
+	 * of the on-disk two-phase data.
+	 */
+	restoreTwoPhaseData();
+
 	lastFullPageWrites = checkPoint.fullPageWrites;
 
 	RedoRecPtr = XLogCtl->RedoRecPtr = XLogCtl->Insert.RedoRecPtr = checkPoint.redo;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b2b7848fad..4d547c5553 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -49,11 +49,12 @@ extern TransactionId PrescanPreparedTransactions(TransactionId **xids_p,
 extern void StandbyRecoverPreparedTransactions(bool overwriteOK);
 extern void RecoverPreparedTransactions(void);
 
-extern void RecreateTwoPhaseFile(TransactionId xid, void *content, int len);
-extern void RemoveTwoPhaseFile(TransactionId xid, bool giveWarning);
-
 extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
+						   XLogRecPtr end_lsn);
+extern void PrepareRedoRemove(TransactionId xid, bool giveWarning);
+extern void restoreTwoPhaseData(void);
 #endif   /* TWOPHASE_H */
#160Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#159)
Re: Speedup twophase transactions

Thanks Michael,

I was away for a bit. I will take a look at this patch and get back to you
soon.

Regards,
Nikhils

On 22 March 2017 at 07:40, Michael Paquier <michael.paquier@gmail.com>
wrote:

On Fri, Mar 17, 2017 at 5:15 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:

On Fri, Mar 17, 2017 at 5:00 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

Micheal, it looks like you are working on a final version of this

patch? I

will wait to review it from my end, then.

I have to admit that I am beginning to get drawn into it...

And here is what I got. I have found a couple of inconsistencies in
the patch, roughly:
- During recovery entries marked with ondisk = true should have their
start and end LSN reset to InvalidXLogRecPtr. This was actually
leading to some inconsistencies in MarkAsPreparing() for 2PC
transactions staying around for more than 2 checkpoints.
- RecoverPreparedTransactions(), StandbyRecoverPreparedTransactions()
and PrescanPreparedTransactions() doing both a scan of pg_twophase and
the shared memory entries was way too complicated. I have changed
things so as only memory entries are scanned by those routines, but an
initial scan of pg_twophase is done before recovery.
- Some inconsistencies in the comments and some typos found on the way.
- Simplification of some routines used in redo, as well as simplified
the set of routines made available to users.

Tests are passing for me, an extra lookup would be nice.
--
Michael

--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services

#161Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#160)
Re: Speedup twophase transactions

On Sun, Mar 26, 2017 at 4:50 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

I was away for a bit. I will take a look at this patch and get back to you
soon.

No problem. Thanks for your time!
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#162Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#161)
1 attachment(s)
Re: Speedup twophase transactions

Hi Micheal,

The latest patch looks good. By now doing a single scan of shmem two phase
data, we have removed the double loops in all the affected functions which
is good.

My only question is if the added call to restoreTwoPhaseData() is good
enough to handle all the 3
functions PrescanPreparedTransactions(), StandbyRecoverPreparedTransactions()
and RecoverPreparedTransactions() appropriately? It looks as if it does,
but we need to be doubly sure..

PFA, revised patch with a very minor typo fix and rebase against latest
master. The test cases pass as needed.

Oh, btw, while running TAP tests, I got a few errors in unrelated tests.

"# testing connection parameter "target_session_attrs"

not ok 5 - connect to node master if mode "read-write" and master,standby_1
listed

# Failed test 'connect to node master if mode "read-write" and
master,standby_1 listed'

# at t/001_stream_rep.pl line 93.

# got: ''

# expected: '1'

not ok 6 - connect to node master if mode "read-write" and standby_1,master
listed

# Failed test 'connect to node master if mode "read-write" and
standby_1,master listed'

# at t/001_stream_rep.pl line 93.

# got: ''

# expected: '1'

not ok 7 - connect to node master if mode "any" and master,standby_1 listed

# Failed test 'connect to node master if mode "any" and master,standby_1
listed'

# at t/001_stream_rep.pl line 93.

# got: ''

# expected: '1'

not ok 8 - connect to node standby_1 if mode "any" and standby_1,master
listed"

Again, not related to this recovery code path, but not sure if others see
this as well.

Regards,
Nikhils

On 27 March 2017 at 05:35, Michael Paquier <michael.paquier@gmail.com>
wrote:

On Sun, Mar 26, 2017 at 4:50 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

I was away for a bit. I will take a look at this patch and get back to

you

soon.

No problem. Thanks for your time!
--
Michael

--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services

Attachments:

twophase_recovery_shmem_280317.patchapplication/octet-stream; name=twophase_recovery_shmem_280317.patchDownload
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 83169cc..aebd55f 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -45,8 +45,27 @@
  *		  fsynced
  *		* If COMMIT happens after checkpoint then backend reads state data from
  *		  files
- *		* In case of crash replay will move data from xlog to files, if that
- *		  hasn't happened before. XXX TODO - move to shmem in replay also
+ *
+ *		During replay and replication, TwoPhaseState also holds information
+ *		about active prepared transactions that haven't been moved to disk yet.
+ *
+ *		Replay of twophase records happens by the following rules:
+ *
+ *		* At the beginning of recovery, pg_twophase is scanned once, filling
+ *		  TwoPhaseState with entries marked with gxact->inredo and
+ *		  gxact->ondisk.  Two-phase file data older than the XID horizon of
+ *		  the redo position are discarded.
+ *		* On PREPARE redo, the transaction is added to TwoPhaseState->prepXacts.
+ *		  gxact->inredo is set to true for such entries.
+ *		* On Checkpoint we iterate through TwoPhaseState->prepXacts entries
+ *		  that have gxact->inredo set and are behind the redo_horizon. We
+ *		  save them to disk and then switch gxact->ondisk to true.
+ *		* On COMMIT/ABORT we delete the entry from TwoPhaseState->prepXacts.
+ *		  If gxact->ondisk is true, the corresponding entry from the disk
+ *		  is additionally deleted.
+ *		* RecoverPreparedTransactions(), StandbyRecoverPreparedTransactions()
+ *		  and PrescanPreparedTransactions() have been modified to go through
+ *		  gxact->inredo entries that have not made it to disk.
  *
  *-------------------------------------------------------------------------
  */
@@ -147,11 +166,13 @@ typedef struct GlobalTransactionData
 	 */
 	XLogRecPtr	prepare_start_lsn;		/* XLOG offset of prepare record start */
 	XLogRecPtr	prepare_end_lsn;	/* XLOG offset of prepare record end */
+	TransactionId	xid;			/* The GXACT id */
 
 	Oid			owner;			/* ID of user that executed the xact */
 	BackendId	locking_backend;	/* backend currently working on the xact */
 	bool		valid;			/* TRUE if PGPROC entry is in proc array */
 	bool		ondisk;			/* TRUE if prepare state file is on disk */
+	bool		inredo;			/* TRUE if entry was added via xlog_redo */
 	char		gid[GIDSIZE];	/* The GID assigned to the prepared xact */
 }	GlobalTransactionData;
 
@@ -198,6 +219,15 @@ static void ProcessRecords(char *bufptr, TransactionId xid,
 static void RemoveGXact(GlobalTransaction gxact);
 
 static void XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len);
+static char *ProcessTwoPhaseBuffer(TransactionId xid,
+							XLogRecPtr	prepare_start_lsn,
+							bool fromdisk, bool overwriteOK, bool setParent,
+							TransactionId *result, TransactionId *maxsubxid);
+static void MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid,
+				const char *gid, TimestampTz prepared_at, Oid owner,
+				Oid databaseid);
+static void RemoveTwoPhaseFile(TransactionId xid, bool giveWarning);
+static void RecreateTwoPhaseFile(TransactionId xid, void *content, int len);
 
 /*
  * Initialization of shared memory
@@ -342,18 +372,12 @@ PostPrepare_Twophase(void)
 /*
  * MarkAsPreparing
  *		Reserve the GID for the given transaction.
- *
- * Internally, this creates a gxact struct and puts it into the active array.
- * NOTE: this is also used when reloading a gxact after a crash; so avoid
- * assuming that we can use very much backend context.
  */
 GlobalTransaction
 MarkAsPreparing(TransactionId xid, const char *gid,
 				TimestampTz prepared_at, Oid owner, Oid databaseid)
 {
 	GlobalTransaction gxact;
-	PGPROC	   *proc;
-	PGXACT	   *pgxact;
 	int			i;
 
 	if (strlen(gid) >= GIDSIZE)
@@ -401,6 +425,37 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 	gxact = TwoPhaseState->freeGXacts;
 	TwoPhaseState->freeGXacts = gxact->next;
 
+	MarkAsPreparingGuts(gxact, xid, gid, prepared_at, owner, databaseid);
+
+	gxact->ondisk = false;
+
+	/* And insert it into the active array */
+	Assert(TwoPhaseState->numPrepXacts < max_prepared_xacts);
+	TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts++] = gxact;
+
+	LWLockRelease(TwoPhaseStateLock);
+
+	return gxact;
+}
+
+/*
+ * MarkAsPreparingGuts
+ *
+ * This uses a gxact struct and puts it into the active array.
+ * NOTE: this is also used when reloading a gxact after a crash; so avoid
+ * assuming that we can use very much backend context.
+ *
+ * Note: This function should be called with appropriate locks held.
+ */
+static void
+MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
+					TimestampTz prepared_at, Oid owner, Oid databaseid)
+{
+	PGPROC	   *proc;
+	PGXACT	   *pgxact;
+	int			i;
+
+	Assert(gxact != NULL);
 	proc = &ProcGlobal->allProcs[gxact->pgprocno];
 	pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 
@@ -431,28 +486,18 @@ MarkAsPreparing(TransactionId xid, const char *gid,
 	pgxact->nxids = 0;
 
 	gxact->prepared_at = prepared_at;
-	/* initialize LSN to InvalidXLogRecPtr */
-	gxact->prepare_start_lsn = InvalidXLogRecPtr;
-	gxact->prepare_end_lsn = InvalidXLogRecPtr;
+	gxact->xid = xid;
 	gxact->owner = owner;
 	gxact->locking_backend = MyBackendId;
 	gxact->valid = false;
-	gxact->ondisk = false;
+	gxact->inredo = false;
 	strcpy(gxact->gid, gid);
 
-	/* And insert it into the active array */
-	Assert(TwoPhaseState->numPrepXacts < max_prepared_xacts);
-	TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts++] = gxact;
-
 	/*
 	 * Remember that we have this GlobalTransaction entry locked for us. If we
 	 * abort after this, we must release it.
 	 */
 	MyLockedGxact = gxact;
-
-	LWLockRelease(TwoPhaseStateLock);
-
-	return gxact;
 }
 
 /*
@@ -1244,9 +1289,9 @@ ReadTwoPhaseFile(TransactionId xid, bool give_warnings)
  * Reads 2PC data from xlog. During checkpoint this data will be moved to
  * twophase files and ReadTwoPhaseFile should be used instead.
  *
- * Note clearly that this function accesses WAL during normal operation, similarly
- * to the way WALSender or Logical Decoding would do. It does not run during
- * crash recovery or standby processing.
+ * Note clearly that this function can access WAL during normal operation,
+ * similarly to the way WALSender or Logical Decoding would do.
+ *
  */
 static void
 XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
@@ -1255,8 +1300,6 @@ XlogReadTwoPhaseData(XLogRecPtr lsn, char **buf, int *len)
 	XLogReaderState *xlogreader;
 	char	   *errormsg;
 
-	Assert(!RecoveryInProgress());
-
 	xlogreader = XLogReaderAllocate(&read_local_xlog_page, NULL);
 	if (!xlogreader)
 		ereport(ERROR,
@@ -1501,7 +1544,7 @@ ProcessRecords(char *bufptr, TransactionId xid,
  * If giveWarning is false, do not complain about file-not-present;
  * this is an expected case during WAL replay.
  */
-void
+static void
 RemoveTwoPhaseFile(TransactionId xid, bool giveWarning)
 {
 	char		path[MAXPGPATH];
@@ -1521,7 +1564,7 @@ RemoveTwoPhaseFile(TransactionId xid, bool giveWarning)
  *
  * Note: content and len don't include CRC.
  */
-void
+static void
 RecreateTwoPhaseFile(TransactionId xid, void *content, int len)
 {
 	char		path[MAXPGPATH];
@@ -1587,9 +1630,11 @@ RecreateTwoPhaseFile(TransactionId xid, void *content, int len)
 /*
  * CheckPointTwoPhase -- handle 2PC component of checkpointing.
  *
- * We must fsync the state file of any GXACT that is valid and has a PREPARE
- * LSN <= the checkpoint's redo horizon.  (If the gxact isn't valid yet or
- * has a later LSN, this checkpoint is not responsible for fsyncing it.)
+ * We must fsync the state file of any GXACT that is valid or has been
+ * generated during redo and has a PREPARE LSN <= the checkpoint's redo
+ * horizon.  (If the gxact isn't valid yet, has not been generated in
+ * redo, or has a later LSN, this checkpoint is not responsible for
+ * fsyncing it.)
  *
  * This is deliberately run as late as possible in the checkpoint sequence,
  * because GXACTs ordinarily have short lifespans, and so it is quite
@@ -1632,9 +1677,8 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
 	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
 	{
 		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
-		PGXACT	   *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
 
-		if (gxact->valid &&
+		if ((gxact->valid || gxact->inredo) &&
 			!gxact->ondisk &&
 			gxact->prepare_end_lsn <= redo_horizon)
 		{
@@ -1642,8 +1686,10 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
 			int			len;
 
 			XlogReadTwoPhaseData(gxact->prepare_start_lsn, &buf, &len);
-			RecreateTwoPhaseFile(pgxact->xid, buf, len);
+			RecreateTwoPhaseFile(gxact->xid, buf, len);
 			gxact->ondisk = true;
+			gxact->prepare_start_lsn = InvalidXLogRecPtr;
+			gxact->prepare_end_lsn = InvalidXLogRecPtr;
 			pfree(buf);
 			serialized_xacts++;
 		}
@@ -1671,12 +1717,49 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon)
 }
 
 /*
+ * restoreTwoPhaseData
+ *
+ * Scan pg_twophase and fill TwoPhaseState depending on the on-disk data.
+ * This is called once at the beginning of recovery, saving any extra
+ * lookups in the future.  Two-phase files that are newer than the
+ * minimum XID horizon are discarded on the way.
+ */
+void
+restoreTwoPhaseData(void)
+{
+	DIR			   *cldir;
+	struct dirent  *clde;
+
+	cldir = AllocateDir(TWOPHASE_DIR);
+	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
+	{
+		if (strlen(clde->d_name) == 8 &&
+			strspn(clde->d_name, "0123456789ABCDEF") == 8)
+		{
+			TransactionId xid;
+			char	   *buf;
+
+			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
+
+			buf = ProcessTwoPhaseBuffer(xid, InvalidXLogRecPtr,
+										true, false, false,
+										NULL, NULL);
+			if (buf == NULL)
+				continue;
+
+			PrepareRedoAdd(buf, InvalidXLogRecPtr, InvalidXLogRecPtr);
+		}
+	}
+	FreeDir(cldir);
+}
+
+/*
  * PrescanPreparedTransactions
  *
- * Scan the pg_twophase directory and determine the range of valid XIDs
- * present.  This is run during database startup, after we have completed
- * reading WAL.  ShmemVariableCache->nextXid has been set to one more than
- * the highest XID for which evidence exists in WAL.
+ * Scan the shared memory entries of TwoPhaseState and determine the range
+ * of valid XIDs present.  This is run during database startup, after we
+ * have completed reading WAL.  ShmemVariableCache->nextXid has been set to
+ * one more than the highest XID for which evidence exists in WAL.
  *
  * We throw away any prepared xacts with main XID beyond nextXid --- if any
  * are present, it suggests that the DBA has done a PITR recovery to an
@@ -1702,120 +1785,52 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 {
 	TransactionId origNextXid = ShmemVariableCache->nextXid;
 	TransactionId result = origNextXid;
-	DIR		   *cldir;
-	struct dirent *clde;
+	TransactionId maxsubxid = origNextXid;
 	TransactionId *xids = NULL;
 	int			nxids = 0;
 	int			allocsize = 0;
+	int			i;
 
-	cldir = AllocateDir(TWOPHASE_DIR);
-	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
 	{
-		if (strlen(clde->d_name) == 8 &&
-			strspn(clde->d_name, "0123456789ABCDEF") == 8)
-		{
-			TransactionId xid;
-			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
-
-			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
-
-			/* Reject XID if too new */
-			if (TransactionIdFollowsOrEquals(xid, origNextXid))
-			{
-				ereport(WARNING,
-						(errmsg("removing future two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+		TransactionId xid;
+		char	   *buf;
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
 
-			/*
-			 * Note: we can't check if already processed because clog
-			 * subsystem isn't up yet.
-			 */
+		Assert(gxact->inredo);
 
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+		xid = gxact->xid;
 
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				pfree(buf);
-				continue;
-			}
+		buf = ProcessTwoPhaseBuffer(xid,
+				gxact->prepare_start_lsn,
+				gxact->ondisk, false, false,
+				&result, &maxsubxid);
 
-			/*
-			 * OK, we think this file is valid.  Incorporate xid into the
-			 * running-minimum result.
-			 */
-			if (TransactionIdPrecedes(xid, result))
-				result = xid;
+		if (buf == NULL)
+			continue;
 
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID, and they may force us to advance nextXid.
-			 *
-			 * We don't expect anyone else to modify nextXid, hence we don't
-			 * need to hold a lock while examining it.  We still acquire the
-			 * lock to modify it, though.
-			 */
-			subxids = (TransactionId *) (buf +
-								MAXALIGN(sizeof(TwoPhaseFileHeader)) +
-								MAXALIGN(hdr->gidlen));
-			for (i = 0; i < hdr->nsubxacts; i++)
+		if (xids_p)
+		{
+			if (nxids == allocsize)
 			{
-				TransactionId subxid = subxids[i];
-
-				Assert(TransactionIdFollows(subxid, xid));
-				if (TransactionIdFollowsOrEquals(subxid,
-												 ShmemVariableCache->nextXid))
+				if (nxids == 0)
 				{
-					LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
-					ShmemVariableCache->nextXid = subxid;
-					TransactionIdAdvance(ShmemVariableCache->nextXid);
-					LWLockRelease(XidGenLock);
+					allocsize = 10;
+					xids = palloc(allocsize * sizeof(TransactionId));
 				}
-			}
-
-
-			if (xids_p)
-			{
-				if (nxids == allocsize)
+				else
 				{
-					if (nxids == 0)
-					{
-						allocsize = 10;
-						xids = palloc(allocsize * sizeof(TransactionId));
-					}
-					else
-					{
-						allocsize = allocsize * 2;
-						xids = repalloc(xids, allocsize * sizeof(TransactionId));
-					}
+					allocsize = allocsize * 2;
+					xids = repalloc(xids, allocsize * sizeof(TransactionId));
 				}
-				xids[nxids++] = xid;
 			}
-
-			pfree(buf);
+			xids[nxids++] = xid;
 		}
+
+		pfree(buf);
 	}
-	FreeDir(cldir);
+	LWLockRelease(TwoPhaseStateLock);
 
 	if (xids_p)
 	{
@@ -1823,14 +1838,25 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 		*nxids_p = nxids;
 	}
 
+	/* update nextXid if needed */
+	if (TransactionIdFollowsOrEquals(maxsubxid, ShmemVariableCache->nextXid))
+	{
+		LWLockAcquire(XidGenLock, LW_EXCLUSIVE);
+		ShmemVariableCache->nextXid = maxsubxid;
+		TransactionIdAdvance(ShmemVariableCache->nextXid);
+		LWLockRelease(XidGenLock);
+	}
+
 	return result;
 }
 
 /*
  * StandbyRecoverPreparedTransactions
  *
- * Scan the pg_twophase directory and setup all the required information to
- * allow standby queries to treat prepared transactions as still active.
+ * Scan the shared memory entries of TwoPhaseState and setup all the required
+ * information to allow standby queries to treat prepared transactions as still
+ * active.
+ *
  * This is never called at the end of recovery - we use
  * RecoverPreparedTransactions() at that point.
  *
@@ -1841,202 +1867,292 @@ PrescanPreparedTransactions(TransactionId **xids_p, int *nxids_p)
 void
 StandbyRecoverPreparedTransactions(bool overwriteOK)
 {
-	DIR		   *cldir;
-	struct dirent *clde;
+	int			i;
 
-	cldir = AllocateDir(TWOPHASE_DIR);
-	while ((clde = ReadDir(cldir, TWOPHASE_DIR)) != NULL)
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
 	{
-		if (strlen(clde->d_name) == 8 &&
-			strspn(clde->d_name, "0123456789ABCDEF") == 8)
-		{
-			TransactionId xid;
-			char	   *buf;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			int			i;
-
-			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
-
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
-
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+		TransactionId xid;
+		char	   *buf;
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
 
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			if (!TransactionIdEquals(hdr->xid, xid))
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				pfree(buf);
-				continue;
-			}
+		Assert(gxact->inredo);
 
-			/*
-			 * Examine subtransaction XIDs ... they should all follow main
-			 * XID.
-			 */
-			subxids = (TransactionId *) (buf +
-								MAXALIGN(sizeof(TwoPhaseFileHeader)) +
-								MAXALIGN(hdr->gidlen));
-			for (i = 0; i < hdr->nsubxacts; i++)
-			{
-				TransactionId subxid = subxids[i];
-
-				Assert(TransactionIdFollows(subxid, xid));
-				SubTransSetParent(xid, subxid, overwriteOK);
-			}
+		xid = gxact->xid;
 
+		buf = ProcessTwoPhaseBuffer(xid,
+				gxact->prepare_start_lsn,
+				gxact->ondisk, overwriteOK, true,
+				NULL, NULL);
+		if (buf != NULL)
 			pfree(buf);
-		}
 	}
-	FreeDir(cldir);
+	LWLockRelease(TwoPhaseStateLock);
 }
 
 /*
  * RecoverPreparedTransactions
  *
- * Scan the pg_twophase directory and reload shared-memory state for each
- * prepared transaction (reacquire locks, etc).  This is run during database
- * startup.
+ * Scan the shared memory entries of TwoPhaseState and reload the state for
+ * each prepared transaction (reacquire locks, etc).
+ *
+ * This is run during database startup.
  */
 void
 RecoverPreparedTransactions(void)
 {
-	char		dir[MAXPGPATH];
-	DIR		   *cldir;
-	struct dirent *clde;
-	bool		overwriteOK = false;
-
-	snprintf(dir, MAXPGPATH, "%s", TWOPHASE_DIR);
+	int			i;
 
-	cldir = AllocateDir(dir);
-	while ((clde = ReadDir(cldir, dir)) != NULL)
+	/*
+	 * Don't need a lock in the recovery phase.
+	 */
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
 	{
-		if (strlen(clde->d_name) == 8 &&
-			strspn(clde->d_name, "0123456789ABCDEF") == 8)
-		{
-			TransactionId xid;
-			char	   *buf;
-			char	   *bufptr;
-			TwoPhaseFileHeader *hdr;
-			TransactionId *subxids;
-			GlobalTransaction gxact;
-			const char *gid;
-			int			i;
+		TransactionId xid;
+		char	   *buf;
+		GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
+		char	   *bufptr;
+		TwoPhaseFileHeader *hdr;
+		TransactionId *subxids;
+		const char *gid;
+		bool		overwriteOK = false;
+		int			i;
 
-			xid = (TransactionId) strtoul(clde->d_name, NULL, 16);
+		xid = gxact->xid;
 
-			/* Already processed? */
-			if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
-			{
-				ereport(WARNING,
-						(errmsg("removing stale two-phase state file \"%s\"",
-								clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+		buf = ProcessTwoPhaseBuffer(xid,
+				gxact->prepare_start_lsn,
+				gxact->ondisk, false, false,
+				NULL, NULL);
+		if (buf == NULL)
+			continue;
 
-			/* Read and validate file */
-			buf = ReadTwoPhaseFile(xid, true);
-			if (buf == NULL)
-			{
-				ereport(WARNING,
-					  (errmsg("removing corrupt two-phase state file \"%s\"",
-							  clde->d_name)));
-				RemoveTwoPhaseFile(xid, true);
-				continue;
-			}
+		ereport(LOG,
+				(errmsg("recovering prepared transaction %u from shared memory", xid)));
+
+		hdr = (TwoPhaseFileHeader *) buf;
+		Assert(TransactionIdEquals(hdr->xid, xid));
+		bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+		gid = (const char *) bufptr;
+		bufptr += MAXALIGN(hdr->gidlen);
+		subxids = (TransactionId *) bufptr;
+		bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
+		bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
+		bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
+		bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
 
-			ereport(LOG,
-					(errmsg("recovering prepared transaction %u", xid)));
-
-			/* Deconstruct header */
-			hdr = (TwoPhaseFileHeader *) buf;
-			Assert(TransactionIdEquals(hdr->xid, xid));
-			bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
-			gid = (const char *) bufptr;
-			bufptr += MAXALIGN(hdr->gidlen);
-			subxids = (TransactionId *) bufptr;
-			bufptr += MAXALIGN(hdr->nsubxacts * sizeof(TransactionId));
-			bufptr += MAXALIGN(hdr->ncommitrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
-			bufptr += MAXALIGN(hdr->ninvalmsgs * sizeof(SharedInvalidationMessage));
+		/*
+		 * It's possible that SubTransSetParent has been set before, if
+		 * the prepared transaction generated xid assignment records. Test
+		 * here must match one used in AssignTransactionId().
+		 */
+		if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
+							 XLogLogicalInfoActive()))
+			overwriteOK = true;
 
-			/*
-			 * It's possible that SubTransSetParent has been set before, if
-			 * the prepared transaction generated xid assignment records. Test
-			 * here must match one used in AssignTransactionId().
-			 */
-			if (InHotStandby && (hdr->nsubxacts >= PGPROC_MAX_CACHED_SUBXIDS ||
-								 XLogLogicalInfoActive()))
-				overwriteOK = true;
+		/*
+		 * Reconstruct subtrans state for the transaction --- needed
+		 * because pg_subtrans is not preserved over a restart.  Note that
+		 * we are linking all the subtransactions directly to the
+		 * top-level XID; there may originally have been a more complex
+		 * hierarchy, but there's no need to restore that exactly.
+		 */
+		for (i = 0; i < hdr->nsubxacts; i++)
+			SubTransSetParent(subxids[i], xid, overwriteOK);
 
-			/*
-			 * Reconstruct subtrans state for the transaction --- needed
-			 * because pg_subtrans is not preserved over a restart.  Note that
-			 * we are linking all the subtransactions directly to the
-			 * top-level XID; there may originally have been a more complex
-			 * hierarchy, but there's no need to restore that exactly.
-			 */
-			for (i = 0; i < hdr->nsubxacts; i++)
-				SubTransSetParent(subxids[i], xid, overwriteOK);
+		/*
+		 * Recreate its GXACT and dummy PGPROC. But, check whether
+		 * it was added in redo and already has a shmem entry for
+		 * it.
+		 */
+		LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+		MarkAsPreparingGuts(gxact, xid, gid,
+							hdr->prepared_at,
+							hdr->owner, hdr->database);
 
-			/*
-			 * Recreate its GXACT and dummy PGPROC
-			 */
-			gxact = MarkAsPreparing(xid, gid,
-									hdr->prepared_at,
-									hdr->owner, hdr->database);
-			gxact->ondisk = true;
-			GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
-			MarkAsPrepared(gxact);
+		/* recovered, so reset the flag for entries generated by redo */
+		gxact->inredo = false;
 
-			/*
-			 * Recover other state (notably locks) using resource managers
-			 */
-			ProcessRecords(bufptr, xid, twophase_recover_callbacks);
+		LWLockRelease(TwoPhaseStateLock);
 
-			/*
-			 * Release locks held by the standby process after we process each
-			 * prepared transaction. As a result, we don't need too many
-			 * additional locks at any one time.
-			 */
-			if (InHotStandby)
-				StandbyReleaseLockTree(xid, hdr->nsubxacts, subxids);
+		GXactLoadSubxactData(gxact, hdr->nsubxacts, subxids);
+		MarkAsPrepared(gxact);
 
-			/*
-			 * We're done with recovering this transaction. Clear
-			 * MyLockedGxact, like we do in PrepareTransaction() during normal
-			 * operation.
-			 */
-			PostPrepare_Twophase();
+		/*
+		 * Recover other state (notably locks) using resource managers
+		 */
+		ProcessRecords(bufptr, xid, twophase_recover_callbacks);
 
-			pfree(buf);
+		/*
+		 * Release locks held by the standby process after we process each
+		 * prepared transaction. As a result, we don't need too many
+		 * additional locks at any one time.
+		 */
+		if (InHotStandby)
+			StandbyReleaseLockTree(xid, hdr->nsubxacts, subxids);
+
+		/*
+		 * We're done with recovering this transaction. Clear
+		 * MyLockedGxact, like we do in PrepareTransaction() during normal
+		 * operation.
+		 */
+		PostPrepare_Twophase();
+
+		pfree(buf);
+	}
+}
+
+/*
+ * ProcessTwoPhaseBuffer
+ *
+ * Given a transaction id, read it either from disk or read it directly
+ * via shmem xlog record pointer using the provided "prepare_start_lsn".
+ *
+ * If setParent is true, then use the overwriteOK parameter to set up
+ * subtransaction parent linkages.
+ *
+ * If result and maxsubxid are not NULL, fill them up with smallest
+ * running transaction id (lesser than ShmemVariableCache->nextXid)
+ * and largest subtransaction id for this transaction respectively.
+ */
+static char *
+ProcessTwoPhaseBuffer(TransactionId xid,
+					  XLogRecPtr prepare_start_lsn,
+					  bool fromdisk, bool overwriteOK,
+					  bool setParent, TransactionId *result,
+					  TransactionId *maxsubxid)
+{
+	TransactionId origNextXid = ShmemVariableCache->nextXid;
+	TransactionId res;
+	TransactionId maxsub;
+	TransactionId *subxids;
+	char	   *buf;
+	TwoPhaseFileHeader *hdr;
+	int			i;
+
+	if (!fromdisk)
+		Assert(prepare_start_lsn != InvalidXLogRecPtr);
+
+	if (result)
+		res = *result;
+	if (maxsubxid)
+		maxsub = *maxsubxid;
+
+	/* Already processed? */
+	if (TransactionIdDidCommit(xid) || TransactionIdDidAbort(xid))
+	{
+		if (fromdisk)
+		{
+			ereport(WARNING,
+					(errmsg("removing stale two-phase state file for \"%u\"",
+							xid)));
+			RemoveTwoPhaseFile(xid, true);
 		}
+		else
+		{
+			ereport(WARNING,
+					(errmsg("removing stale two-phase state from"
+							" shared memory for \"%u\"", xid)));
+			PrepareRedoRemove(xid, true);
+		}
+		return NULL;
 	}
-	FreeDir(cldir);
+
+	/* Reject XID if too new */
+	if (TransactionIdFollowsOrEquals(xid, origNextXid))
+	{
+		if (fromdisk)
+		{
+			ereport(WARNING,
+					(errmsg("removing future two-phase state file for \"%u\"",
+							xid)));
+			RemoveTwoPhaseFile(xid, true);
+		}
+		else
+		{
+			ereport(WARNING,
+					(errmsg("removing future two-phase state from memory for \"%u\"",
+							xid)));
+			PrepareRedoRemove(xid, true);
+		}
+		return NULL;
+	}
+
+	if (fromdisk)
+	{
+		/* Read and validate file */
+		buf = ReadTwoPhaseFile(xid, true);
+		if (buf == NULL)
+		{
+			ereport(WARNING,
+					(errmsg("removing corrupt two-phase state file for \"%u\"",
+							xid)));
+			RemoveTwoPhaseFile(xid, true);
+			return NULL;
+		}
+	}
+	else
+	{
+		/* Read xlog data */
+		XlogReadTwoPhaseData(prepare_start_lsn, &buf, NULL);
+	}
+
+	/* Deconstruct header */
+	hdr = (TwoPhaseFileHeader *) buf;
+	if (!TransactionIdEquals(hdr->xid, xid))
+	{
+		if (fromdisk)
+		{
+			ereport(WARNING,
+					(errmsg("removing corrupt two-phase state file for \"%u\"",
+							xid)));
+			RemoveTwoPhaseFile(xid, true);
+		}
+		else
+		{
+			ereport(WARNING,
+					(errmsg("removing corrupt two-phase state from memory for \"%u\"",
+							xid)));
+			PrepareRedoRemove(xid, true);
+		}
+		pfree(buf);
+		return NULL;
+	}
+
+	/*
+	 * OK, we think this buffer is valid.  Incorporate xid into the
+	 * running-minimum result.
+	 */
+	if (TransactionIdPrecedes(xid, res))
+		res = xid;
+
+	/*
+	 * Examine subtransaction XIDs ... they should all follow main
+	 * XID, and they may force us to advance nextXid.
+	 */
+	subxids = (TransactionId *) (buf +
+								 MAXALIGN(sizeof(TwoPhaseFileHeader)) +
+								 MAXALIGN(hdr->gidlen));
+	for (i = 0; i < hdr->nsubxacts; i++)
+	{
+		TransactionId subxid = subxids[i];
+
+		Assert(TransactionIdFollows(subxid, xid));
+		if (TransactionIdFollowsOrEquals(subxid, maxsub))
+			maxsub = subxid;
+		if (setParent)
+			SubTransSetParent(xid, subxid, overwriteOK);
+	}
+
+	if (result)
+		*result = res;
+	if (maxsubxid)
+		*maxsubxid = maxsub;
+
+	return buf;
 }
 
+
 /*
  *	RecordTransactionCommitPrepared
  *
@@ -2187,3 +2303,111 @@ RecordTransactionAbortPrepared(TransactionId xid,
 	 */
 	SyncRepWaitForLSN(recptr, false);
 }
+
+/*
+ * PrepareRedoAdd
+ *
+ * Store pointers to the start/end of the WAL record along with the xid in
+ * a gxact entry in shared memory TwoPhaseState structure.  If caller
+ * specifies InvalidXLogRecPtr as WAL position to fetch the two-phase
+ * data, the entry is marked as located on disk.
+ */
+void
+PrepareRedoAdd(char *buf, XLogRecPtr start_lsn, XLogRecPtr end_lsn)
+{
+	TwoPhaseFileHeader *hdr = (TwoPhaseFileHeader *) buf;
+	char			  *bufptr;
+	const char		  *gid;
+	GlobalTransaction gxact;
+
+	Assert(RecoveryInProgress());
+
+	bufptr = buf + MAXALIGN(sizeof(TwoPhaseFileHeader));
+	gid = (const char *) bufptr;
+
+	/*
+	 * Reserve the GID for the given transaction in the redo code path.
+	 *
+	 * This creates a gxact struct and puts it into the active array.
+	 *
+	 * In redo, this struct is mainly used to track PREPARE/COMMIT entries
+	 * in shared memory. Hence, we only fill up the bare minimum contents here.
+	 * The gxact also gets marked with gxact->inredo set to true to indicate
+	 * that it got added in the redo phase
+	 */
+
+	LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
+	/* Get a free gxact from the freelist */
+	if (TwoPhaseState->freeGXacts == NULL)
+		ereport(ERROR,
+				(errcode(ERRCODE_OUT_OF_MEMORY),
+				 errmsg("maximum number of prepared transactions reached"),
+				 errhint("Increase max_prepared_transactions (currently %d).",
+						 max_prepared_xacts)));
+	gxact = TwoPhaseState->freeGXacts;
+	TwoPhaseState->freeGXacts = gxact->next;
+
+	gxact->prepared_at = hdr->prepared_at;
+	gxact->prepare_start_lsn = start_lsn;
+	gxact->prepare_end_lsn = end_lsn;
+	gxact->xid = hdr->xid;
+	gxact->owner = hdr->owner;
+	gxact->locking_backend = InvalidBackendId;
+	gxact->valid = false;
+	gxact->ondisk = XLogRecPtrIsInvalid(start_lsn);
+	gxact->inredo = true; /* yes, added in redo */
+	strcpy(gxact->gid, gid);
+
+	/* And insert it into the active array */
+	Assert(TwoPhaseState->numPrepXacts < max_prepared_xacts);
+	TwoPhaseState->prepXacts[TwoPhaseState->numPrepXacts++] = gxact;
+
+	LWLockRelease(TwoPhaseStateLock);
+
+	elog(DEBUG2, "Adding 2PC data to shared memory %u", gxact->xid);
+}
+
+/*
+ * PrepareRedoRemove
+ *
+ * Remove the corresponding gxact entry from TwoPhaseState. Also
+ * remove the 2PC file if a prepared transaction was saved via
+ * an earlier checkpoint.
+ */
+void
+PrepareRedoRemove(TransactionId xid, bool giveWarning)
+{
+	GlobalTransaction gxact = NULL;
+	int			i;
+
+	Assert(RecoveryInProgress());
+
+	LWLockAcquire(TwoPhaseStateLock, LW_SHARED);
+	for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
+	{
+		gxact = TwoPhaseState->prepXacts[i];
+
+		if (gxact->xid == xid)
+		{
+			Assert(gxact->inredo);
+			break;
+		}
+	}
+	LWLockRelease(TwoPhaseStateLock);
+
+	/*
+	 * Just leave if there is nothing, this is expected during WAL replay.
+	 */
+	if (gxact == NULL)
+		return;
+
+	/*
+	 * And now we can clean up any files we may have left.
+	 */
+	elog(DEBUG2, "Removing 2PC data from shared memory %u", xid);
+	if (gxact->ondisk)
+		RemoveTwoPhaseFile(xid, giveWarning);
+	RemoveGXact(gxact);
+
+	return;
+}
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index c8751c6..6f614e4 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -5615,7 +5615,9 @@ xact_redo(XLogReaderState *record)
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_commit(&parsed, parsed.twophase_xid,
 							 record->EndRecPtr, XLogRecGetOrigin(record));
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete TwoPhaseState gxact entry and/or 2PC file. */
+			PrepareRedoRemove(parsed.twophase_xid, false);
 		}
 	}
 	else if (info == XLOG_XACT_ABORT || info == XLOG_XACT_ABORT_PREPARED)
@@ -5635,14 +5637,20 @@ xact_redo(XLogReaderState *record)
 		{
 			Assert(TransactionIdIsValid(parsed.twophase_xid));
 			xact_redo_abort(&parsed, parsed.twophase_xid);
-			RemoveTwoPhaseFile(parsed.twophase_xid, false);
+
+			/* Delete TwoPhaseState gxact entry and/or 2PC file. */
+			PrepareRedoRemove(parsed.twophase_xid, false);
 		}
 	}
 	else if (info == XLOG_XACT_PREPARE)
 	{
-		/* the record contents are exactly the 2PC file */
-		RecreateTwoPhaseFile(XLogRecGetXid(record),
-						  XLogRecGetData(record), XLogRecGetDataLen(record));
+		/*
+		 * Store xid and start/end pointers of the WAL record in
+		 * TwoPhaseState gxact entry.
+		 */
+		PrepareRedoAdd(XLogRecGetData(record),
+					   record->ReadRecPtr,
+					   record->EndRecPtr);
 	}
 	else if (info == XLOG_XACT_ASSIGNMENT)
 	{
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 61ca81d..1e0343f 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6696,6 +6696,16 @@ StartupXLOG(void)
 	 */
 	restoreTimeLineHistoryFiles(ThisTimeLineID, recoveryTargetTLI);
 
+	/*
+	 * Before running in recovery, scan pg_twophase and fill in its status
+	 * to be able to work on entries generated by redo.  Doing a scan before
+	 * taking any recovery action has the merit to discard any 2PC files that
+	 * are newer than the first record to replay, saving from any conflicts at
+	 * replay.  This avoids as well any subsequent scans when doing recovery
+	 * of the on-disk two-phase data.
+	 */
+	restoreTwoPhaseData();
+
 	lastFullPageWrites = checkPoint.fullPageWrites;
 
 	RedoRecPtr = XLogCtl->RedoRecPtr = XLogCtl->Insert.RedoRecPtr = checkPoint.redo;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index b2b7848..4d547c5 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -49,11 +49,12 @@ extern TransactionId PrescanPreparedTransactions(TransactionId **xids_p,
 extern void StandbyRecoverPreparedTransactions(bool overwriteOK);
 extern void RecoverPreparedTransactions(void);
 
-extern void RecreateTwoPhaseFile(TransactionId xid, void *content, int len);
-extern void RemoveTwoPhaseFile(TransactionId xid, bool giveWarning);
-
 extern void CheckPointTwoPhase(XLogRecPtr redo_horizon);
 
 extern void FinishPreparedTransaction(const char *gid, bool isCommit);
 
+extern void PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
+						   XLogRecPtr end_lsn);
+extern void PrepareRedoRemove(TransactionId xid, bool giveWarning);
+extern void restoreTwoPhaseData(void);
 #endif   /* TWOPHASE_H */
#163Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Nikhil Sontakke (#162)
Re: Speedup twophase transactions

Please ignore reports about errors in other tests. Seem spurious..

Regards,
Nikhils

On 28 March 2017 at 10:40, Nikhil Sontakke <nikhils@2ndquadrant.com> wrote:

Hi Micheal,

The latest patch looks good. By now doing a single scan of shmem two phase
data, we have removed the double loops in all the affected functions which
is good.

My only question is if the added call to restoreTwoPhaseData() is good
enough to handle all the 3 functions PrescanPreparedTransactions(),
StandbyRecoverPreparedTransactions() and RecoverPreparedTransactions()
appropriately? It looks as if it does, but we need to be doubly sure..

PFA, revised patch with a very minor typo fix and rebase against latest
master. The test cases pass as needed.

Oh, btw, while running TAP tests, I got a few errors in unrelated tests.

"# testing connection parameter "target_session_attrs"

not ok 5 - connect to node master if mode "read-write" and
master,standby_1 listed

# Failed test 'connect to node master if mode "read-write" and
master,standby_1 listed'

# at t/001_stream_rep.pl line 93.

# got: ''

# expected: '1'

not ok 6 - connect to node master if mode "read-write" and
standby_1,master listed

# Failed test 'connect to node master if mode "read-write" and
standby_1,master listed'

# at t/001_stream_rep.pl line 93.

# got: ''

# expected: '1'

not ok 7 - connect to node master if mode "any" and master,standby_1 listed

# Failed test 'connect to node master if mode "any" and master,standby_1
listed'

# at t/001_stream_rep.pl line 93.

# got: ''

# expected: '1'

not ok 8 - connect to node standby_1 if mode "any" and standby_1,master
listed"

Again, not related to this recovery code path, but not sure if others see
this as well.

Regards,
Nikhils

On 27 March 2017 at 05:35, Michael Paquier <michael.paquier@gmail.com>
wrote:

On Sun, Mar 26, 2017 at 4:50 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

I was away for a bit. I will take a look at this patch and get back to

you

soon.

No problem. Thanks for your time!
--
Michael

--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services

--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services

#164Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#162)
Re: Speedup twophase transactions

On Tue, Mar 28, 2017 at 2:10 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

The latest patch looks good. By now doing a single scan of shmem two phase
data, we have removed the double loops in all the affected functions which
is good.

Yup.

My only question is if the added call to restoreTwoPhaseData() is good
enough to handle all the 3 functions PrescanPreparedTransactions(),
StandbyRecoverPreparedTransactions() and RecoverPreparedTransactions()
appropriately? It looks as if it does, but we need to be doubly sure..

Yeah, I have spent a bit of time thinking about that. But as
restoreTwoPhaseData() is basically what those other three routines do
but at an earlier stage I cannot see a problem with it. I don't
discard being in shortage of imagination of course.

PFA, revised patch with a very minor typo fix and rebase against latest
master. The test cases pass as needed.

Thanks!

Oh, btw, while running TAP tests, I got a few errors in unrelated tests.
[...]
Again, not related to this recovery code path, but not sure if others see
this as well.

Definitely not related to this patch, and I am unable to see anything
like that. Even spurious errors merit attention, but even by running
those tests multiple times daily I have not seen anything like that.
That's mainly on OSX 10.11 though.

I don't have anything else to say about this patch, so should we mark
that as ready for committer? There are still a couple of days left
until the end of the CF, and quite a lot has happened, so this could
get on time into PG10.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#165Nikhil Sontakke
nikhils@2ndquadrant.com
In reply to: Michael Paquier (#164)
Re: Speedup twophase transactions

I don't have anything else to say about this patch, so should we mark
that as ready for committer? There are still a couple of days left
until the end of the CF, and quite a lot has happened, so this could
get on time into PG10.

Yes, let's mark it "ready for committer". This patch/feature has been a
long journey! :)

Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services

#166Michael Paquier
michael.paquier@gmail.com
In reply to: Nikhil Sontakke (#165)
Re: Speedup twophase transactions

On Tue, Mar 28, 2017 at 3:08 PM, Nikhil Sontakke
<nikhils@2ndquadrant.com> wrote:

I don't have anything else to say about this patch, so should we mark
that as ready for committer? There are still a couple of days left
until the end of the CF, and quite a lot has happened, so this could
get on time into PG10.

Yes, let's mark it "ready for committer". This patch/feature has been a long
journey! :)

OK, done. I have just noticed that Simon has marked himself as a
committer of this patch 24 hours ago.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#167Michael Paquier
michael.paquier@gmail.com
In reply to: Michael Paquier (#166)
Re: Speedup twophase transactions

On Tue, Mar 28, 2017 at 3:10 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:

OK, done. I have just noticed that Simon has marked himself as a
committer of this patch 24 hours ago.

For the archive's sake, this has been committed as 728bd991. Thanks Simon!
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers