POC: Sharing record typmods between backends

Started by Thomas Munroalmost 9 years ago50 messages
#1Thomas Munro
thomas.munro@enterprisedb.com
2 attachment(s)

Hi hackers,

Tuples can have type RECORDOID and a typmod number that identifies a
"blessed" TupleDesc in a backend-private cache. To support the
sharing of such tuples through shared memory and temporary files, I
think we need a typmod registry in shared memory. Here's a
proof-of-concept patch for discussion. I'd be grateful for any
feedback and/or flames.

This is a problem I ran into in my parallel hash join project. Robert
pointed it out to me and told me to go read tqueue.c for details, and
my first reaction was: I'll code around this by teaching the planner
to avoid sharing tuples from paths that produce transient record types
based on tlist analysis[1]/messages/by-id/CAEepm=2+zf7L_-eZ5hPW5=US+utdo=9tMVD4wt7ZSM-uOoSxWg@mail.gmail.com. Aside from being a cop-out, that approach
doesn't work because the planner doesn't actually know what types the
executor might come up with since some amount of substitution for
structurally-similar records seems to be allowed[2]/messages/by-id/CA+TgmoZMH6mJyXX=YLSOvJ8jULFqGgXWZCr_rbkc1nJ+177VSQ@mail.gmail.com (though I'm not
sure I can explain that). So... we're gonna need a bigger boat.

The patch uses typcache.c's backend-private cache still, but if the
backend is currently "attached" to a shared registry then it functions
as a write though cache. There is no cache-invalidation problem
because registered typmods are never unregistered. parallel.c exports
the leader's existing record typmods into a shared registry, and
attaches to it in workers. A DSM detach hook returns backends to
private cache mode when parallelism ends.

Some thoughts:

* Maybe it would be better to have just one DSA area, rather than the
one controlled by execParallel.c (for executor nodes to use) and this
new one controlled by parallel.c (for the ParallelContext). Those
scopes are approximately the same at least in the parallel query case,
but...

* It would be nice for the SharedRecordTypeRegistry to be able to
survive longer than a single parallel query, perhaps in a per-session
DSM segment. Perhaps eventually we will want to consider a
query-scoped area, a transaction-scoped area and a session-scoped
area? I didn't investigate that for this POC.

* It seemed to be a reasonable goal to avoid allocating an extra DSM
segment for every parallel query, so the new DSA area is created
in-place. 192KB turns out to be enough to hold an empty
SharedRecordTypmodRegistry due to dsa.c's superblock allocation scheme
(that's two 64KB size class superblocks + some DSA control
information). It'll create a new DSM segment as soon as you start
using blessed records, and will do so for every parallel query you
start from then on with the same backend. Erm, maybe adding 192KB to
every parallel query DSM segment won't be popular...

* Perhaps simplehash + an LWLock would be better than dht, but I
haven't looked into that. Can it be convinced to work in DSA memory
and to grow on demand?

Here's one way to hit the new code path, so that record types blessed
in a worker are accessed from the leader:

CREATE TABLE foo AS SELECT generate_series(1, 10) AS x;
CREATE OR REPLACE FUNCTION make_record(n int)
RETURNS RECORD LANGUAGE plpgsql PARALLEL SAFE AS
$$
BEGIN
RETURN CASE n
WHEN 1 THEN ROW(1)
WHEN 2 THEN ROW(1, 2)
WHEN 3 THEN ROW(1, 2, 3)
WHEN 4 THEN ROW(1, 2, 3, 4)
ELSE ROW(1, 2, 3, 4, 5)
END;
END;
$$;
SET force_parallel_mode = 1;
SELECT make_record(x) FROM foo;

PATCH

1. Apply dht-v3.patch[3]/messages/by-id/CAEepm=3d8o8XdVwYT6O=bHKsKAM2pu2D6sV1S_=4d+jStVCE7w@mail.gmail.com.
2. Apply shared-record-typmod-registry-v1.patch.
3. Apply rip-out-tqueue-remapping-v1.patch.

[1]: /messages/by-id/CAEepm=2+zf7L_-eZ5hPW5=US+utdo=9tMVD4wt7ZSM-uOoSxWg@mail.gmail.com
[2]: /messages/by-id/CA+TgmoZMH6mJyXX=YLSOvJ8jULFqGgXWZCr_rbkc1nJ+177VSQ@mail.gmail.com
[3]: /messages/by-id/CAEepm=3d8o8XdVwYT6O=bHKsKAM2pu2D6sV1S_=4d+jStVCE7w@mail.gmail.com

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

rip-out-tqueue-remapping-v1.patchapplication/octet-stream; name=rip-out-tqueue-remapping-v1.patchDownload
diff --git a/src/backend/executor/tqueue.c b/src/backend/executor/tqueue.c
index 8d7e711b3bc..d7bfc3deb21 100644
--- a/src/backend/executor/tqueue.c
+++ b/src/backend/executor/tqueue.c
@@ -3,25 +3,10 @@
  * tqueue.c
  *	  Use shm_mq to send & receive tuples between parallel backends
  *
- * Most of the complexity in this module arises from transient RECORD types,
- * which all have type RECORDOID and are distinguished by typmod numbers
- * that are managed per-backend (see src/backend/utils/cache/typcache.c).
- * The sender's set of RECORD typmod assignments probably doesn't match the
- * receiver's.  To deal with this, we make the sender send a description
- * of each transient RECORD type appearing in the data it sends.  The
- * receiver finds or creates a matching type in its own typcache, and then
- * maps the sender's typmod for that type to its own typmod.
- *
  * A DestReceiver of type DestTupleQueue, which is a TQueueDestReceiver
- * under the hood, writes tuples from the executor to a shm_mq.  If
- * necessary, it also writes control messages describing transient
- * record types used within the tuple.
+ * under the hood, writes tuples from the executor to a shm_mq.
  *
- * A TupleQueueReader reads tuples, and control messages if any are sent,
- * from a shm_mq and returns the tuples.  If transient record types are
- * in use, it registers those types locally based on the control messages
- * and rewrites the typmods sent by the remote side to the corresponding
- * local record typmods.
+ * A TupleQueueReader reads tuples from a shm_mq and returns the tuples.
  *
  * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
  * Portions Copyright (c) 1994, Regents of the University of California
@@ -35,186 +20,33 @@
 #include "postgres.h"
 
 #include "access/htup_details.h"
-#include "catalog/pg_type.h"
 #include "executor/tqueue.h"
-#include "funcapi.h"
-#include "lib/stringinfo.h"
-#include "miscadmin.h"
-#include "utils/array.h"
-#include "utils/lsyscache.h"
-#include "utils/memutils.h"
-#include "utils/rangetypes.h"
-#include "utils/syscache.h"
-#include "utils/typcache.h"
-
-
-/*
- * The data transferred through the shm_mq is divided into messages.
- * One-byte messages are mode-switch messages, telling the receiver to switch
- * between "control" and "data" modes.  (We always start up in "data" mode.)
- * Otherwise, when in "data" mode, each message is a tuple.  When in "control"
- * mode, each message defines one transient-typmod-to-tupledesc mapping to
- * let us interpret future tuples.  Both of those cases certainly require
- * more than one byte, so no confusion is possible.
- */
-#define TUPLE_QUEUE_MODE_CONTROL	'c' /* mode-switch message contents */
-#define TUPLE_QUEUE_MODE_DATA		'd'
-
-/*
- * Both the sender and receiver build trees of TupleRemapInfo nodes to help
- * them identify which (sub) fields of transmitted tuples are composite and
- * may thus need remap processing.  We might need to look within arrays and
- * ranges, not only composites, to find composite sub-fields.  A NULL
- * TupleRemapInfo pointer indicates that it is known that the described field
- * is not composite and has no composite substructure.
- *
- * Note that we currently have to look at each composite field at runtime,
- * even if we believe it's of a named composite type (i.e., not RECORD).
- * This is because we allow the actual value to be a compatible transient
- * RECORD type.  That's grossly inefficient, and it would be good to get
- * rid of the requirement, but it's not clear what would need to change.
- *
- * Also, we allow the top-level tuple structure, as well as the actual
- * structure of composite subfields, to change from one tuple to the next
- * at runtime.  This may well be entirely historical, but it's mostly free
- * to support given the previous requirement; and other places in the system
- * also permit this, so it's not entirely clear if we could drop it.
- */
-
-typedef enum
-{
-	TQUEUE_REMAP_ARRAY,			/* array */
-	TQUEUE_REMAP_RANGE,			/* range */
-	TQUEUE_REMAP_RECORD			/* composite type, named or transient */
-} TupleRemapClass;
-
-typedef struct TupleRemapInfo TupleRemapInfo;
-
-typedef struct ArrayRemapInfo
-{
-	int16		typlen;			/* array element type's storage properties */
-	bool		typbyval;
-	char		typalign;
-	TupleRemapInfo *element_remap;		/* array element type's remap info */
-} ArrayRemapInfo;
-
-typedef struct RangeRemapInfo
-{
-	TypeCacheEntry *typcache;	/* range type's typcache entry */
-	TupleRemapInfo *bound_remap;	/* range bound type's remap info */
-} RangeRemapInfo;
-
-typedef struct RecordRemapInfo
-{
-	/* Original (remote) type ID info last seen for this composite field */
-	Oid			rectypid;
-	int32		rectypmod;
-	/* Local RECORD typmod, or -1 if unset; not used on sender side */
-	int32		localtypmod;
-	/* If no fields of the record require remapping, these are NULL: */
-	TupleDesc	tupledesc;		/* copy of record's tupdesc */
-	TupleRemapInfo **field_remap;		/* each field's remap info */
-} RecordRemapInfo;
-
-struct TupleRemapInfo
-{
-	TupleRemapClass remapclass;
-	union
-	{
-		ArrayRemapInfo arr;
-		RangeRemapInfo rng;
-		RecordRemapInfo rec;
-	}			u;
-};
 
 /*
  * DestReceiver object's private contents
  *
  * queue and tupledesc are pointers to data supplied by DestReceiver's caller.
- * The recordhtab and remap info are owned by the DestReceiver and are kept
- * in mycontext.  tmpcontext is a tuple-lifespan context to hold cruft
- * created while traversing each tuple to find record subfields.
  */
 typedef struct TQueueDestReceiver
 {
 	DestReceiver pub;			/* public fields */
 	shm_mq_handle *queue;		/* shm_mq to send to */
-	MemoryContext mycontext;	/* context containing TQueueDestReceiver */
-	MemoryContext tmpcontext;	/* per-tuple context, if needed */
-	HTAB	   *recordhtab;		/* table of transmitted typmods, if needed */
-	char		mode;			/* current message mode */
 	TupleDesc	tupledesc;		/* current top-level tuple descriptor */
-	TupleRemapInfo **field_remapinfo;	/* current top-level remap info */
 } TQueueDestReceiver;
 
 /*
- * Hash table entries for mapping remote to local typmods.
- */
-typedef struct RecordTypmodMap
-{
-	int32		remotetypmod;	/* hash key (must be first!) */
-	int32		localtypmod;
-} RecordTypmodMap;
-
-/*
  * TupleQueueReader object's private contents
  *
  * queue and tupledesc are pointers to data supplied by reader's caller.
- * The typmodmap and remap info are owned by the TupleQueueReader and
- * are kept in mycontext.
  *
  * "typedef struct TupleQueueReader TupleQueueReader" is in tqueue.h
  */
 struct TupleQueueReader
 {
 	shm_mq_handle *queue;		/* shm_mq to receive from */
-	MemoryContext mycontext;	/* context containing TupleQueueReader */
-	HTAB	   *typmodmap;		/* RecordTypmodMap hashtable, if needed */
-	char		mode;			/* current message mode */
 	TupleDesc	tupledesc;		/* current top-level tuple descriptor */
-	TupleRemapInfo **field_remapinfo;	/* current top-level remap info */
 };
 
-/* Local function prototypes */
-static void TQExamine(TQueueDestReceiver *tqueue,
-		  TupleRemapInfo *remapinfo,
-		  Datum value);
-static void TQExamineArray(TQueueDestReceiver *tqueue,
-			   ArrayRemapInfo *remapinfo,
-			   Datum value);
-static void TQExamineRange(TQueueDestReceiver *tqueue,
-			   RangeRemapInfo *remapinfo,
-			   Datum value);
-static void TQExamineRecord(TQueueDestReceiver *tqueue,
-				RecordRemapInfo *remapinfo,
-				Datum value);
-static void TQSendRecordInfo(TQueueDestReceiver *tqueue, int32 typmod,
-				 TupleDesc tupledesc);
-static void TupleQueueHandleControlMessage(TupleQueueReader *reader,
-							   Size nbytes, char *data);
-static HeapTuple TupleQueueHandleDataMessage(TupleQueueReader *reader,
-							Size nbytes, HeapTupleHeader data);
-static HeapTuple TQRemapTuple(TupleQueueReader *reader,
-			 TupleDesc tupledesc,
-			 TupleRemapInfo **field_remapinfo,
-			 HeapTuple tuple);
-static Datum TQRemap(TupleQueueReader *reader, TupleRemapInfo *remapinfo,
-		Datum value, bool *changed);
-static Datum TQRemapArray(TupleQueueReader *reader, ArrayRemapInfo *remapinfo,
-			 Datum value, bool *changed);
-static Datum TQRemapRange(TupleQueueReader *reader, RangeRemapInfo *remapinfo,
-			 Datum value, bool *changed);
-static Datum TQRemapRecord(TupleQueueReader *reader, RecordRemapInfo *remapinfo,
-			  Datum value, bool *changed);
-static TupleRemapInfo *BuildTupleRemapInfo(Oid typid, MemoryContext mycontext);
-static TupleRemapInfo *BuildArrayRemapInfo(Oid elemtypid,
-					MemoryContext mycontext);
-static TupleRemapInfo *BuildRangeRemapInfo(Oid rngtypid,
-					MemoryContext mycontext);
-static TupleRemapInfo **BuildFieldRemapInfo(TupleDesc tupledesc,
-					MemoryContext mycontext);
-
-
 /*
  * Receive a tuple from a query, and send it to the designated shm_mq.
  *
@@ -228,81 +60,8 @@ tqueueReceiveSlot(TupleTableSlot *slot, DestReceiver *self)
 	HeapTuple	tuple;
 	shm_mq_result result;
 
-	/*
-	 * If first time through, compute remapping info for the top-level fields.
-	 * On later calls, if the tupledesc has changed, set up for the new
-	 * tupledesc.  (This is a strange test both because the executor really
-	 * shouldn't change the tupledesc, and also because it would be unsafe if
-	 * the old tupledesc could be freed and a new one allocated at the same
-	 * address.  But since some very old code in printtup.c uses a similar
-	 * approach, we adopt it here as well.)
-	 *
-	 * Here and elsewhere in this module, when replacing remapping info we
-	 * pfree the top-level object because that's easy, but we don't bother to
-	 * recursively free any substructure.  This would lead to query-lifespan
-	 * memory leaks if the mapping info actually changed frequently, but since
-	 * we don't expect that to happen, it doesn't seem worth expending code to
-	 * prevent it.
-	 */
 	if (tqueue->tupledesc != tupledesc)
-	{
-		/* Is it worth trying to free substructure of the remap tree? */
-		if (tqueue->field_remapinfo != NULL)
-			pfree(tqueue->field_remapinfo);
-		tqueue->field_remapinfo = BuildFieldRemapInfo(tupledesc,
-													  tqueue->mycontext);
 		tqueue->tupledesc = tupledesc;
-	}
-
-	/*
-	 * When, because of the types being transmitted, no record typmod mapping
-	 * can be needed, we can skip a good deal of work.
-	 */
-	if (tqueue->field_remapinfo != NULL)
-	{
-		TupleRemapInfo **remapinfo = tqueue->field_remapinfo;
-		int			i;
-		MemoryContext oldcontext = NULL;
-
-		/* Deform the tuple so we can examine fields, if not done already. */
-		slot_getallattrs(slot);
-
-		/* Iterate over each attribute and search it for transient typmods. */
-		for (i = 0; i < tupledesc->natts; i++)
-		{
-			/* Ignore nulls and types that don't need special handling. */
-			if (slot->tts_isnull[i] || remapinfo[i] == NULL)
-				continue;
-
-			/* Switch to temporary memory context to avoid leaking. */
-			if (oldcontext == NULL)
-			{
-				if (tqueue->tmpcontext == NULL)
-					tqueue->tmpcontext =
-						AllocSetContextCreate(tqueue->mycontext,
-											  "tqueue sender temp context",
-											  ALLOCSET_DEFAULT_SIZES);
-				oldcontext = MemoryContextSwitchTo(tqueue->tmpcontext);
-			}
-
-			/* Examine the value. */
-			TQExamine(tqueue, remapinfo[i], slot->tts_values[i]);
-		}
-
-		/* If we used the temp context, reset it and restore prior context. */
-		if (oldcontext != NULL)
-		{
-			MemoryContextSwitchTo(oldcontext);
-			MemoryContextReset(tqueue->tmpcontext);
-		}
-
-		/* If we entered control mode, switch back to data mode. */
-		if (tqueue->mode != TUPLE_QUEUE_MODE_DATA)
-		{
-			tqueue->mode = TUPLE_QUEUE_MODE_DATA;
-			shm_mq_send(tqueue->queue, sizeof(char), &tqueue->mode, false);
-		}
-	}
 
 	/* Send the tuple itself. */
 	tuple = ExecMaterializeSlot(slot);
@@ -320,248 +79,6 @@ tqueueReceiveSlot(TupleTableSlot *slot, DestReceiver *self)
 }
 
 /*
- * Examine the given datum and send any necessary control messages for
- * transient record types contained in it.
- *
- * remapinfo is previously-computed remapping info about the datum's type.
- *
- * This function just dispatches based on the remap class.
- */
-static void
-TQExamine(TQueueDestReceiver *tqueue, TupleRemapInfo *remapinfo, Datum value)
-{
-	/* This is recursive, so it could be driven to stack overflow. */
-	check_stack_depth();
-
-	switch (remapinfo->remapclass)
-	{
-		case TQUEUE_REMAP_ARRAY:
-			TQExamineArray(tqueue, &remapinfo->u.arr, value);
-			break;
-		case TQUEUE_REMAP_RANGE:
-			TQExamineRange(tqueue, &remapinfo->u.rng, value);
-			break;
-		case TQUEUE_REMAP_RECORD:
-			TQExamineRecord(tqueue, &remapinfo->u.rec, value);
-			break;
-	}
-}
-
-/*
- * Examine a record datum and send any necessary control messages for
- * transient record types contained in it.
- */
-static void
-TQExamineRecord(TQueueDestReceiver *tqueue, RecordRemapInfo *remapinfo,
-				Datum value)
-{
-	HeapTupleHeader tup;
-	Oid			typid;
-	int32		typmod;
-	TupleDesc	tupledesc;
-
-	/* Extract type OID and typmod from tuple. */
-	tup = DatumGetHeapTupleHeader(value);
-	typid = HeapTupleHeaderGetTypeId(tup);
-	typmod = HeapTupleHeaderGetTypMod(tup);
-
-	/*
-	 * If first time through, or if this isn't the same composite type as last
-	 * time, consider sending a control message, and then look up the
-	 * necessary information for examining the fields.
-	 */
-	if (typid != remapinfo->rectypid || typmod != remapinfo->rectypmod)
-	{
-		/* Free any old data. */
-		if (remapinfo->tupledesc != NULL)
-			FreeTupleDesc(remapinfo->tupledesc);
-		/* Is it worth trying to free substructure of the remap tree? */
-		if (remapinfo->field_remap != NULL)
-			pfree(remapinfo->field_remap);
-
-		/* Look up tuple descriptor in typcache. */
-		tupledesc = lookup_rowtype_tupdesc(typid, typmod);
-
-		/*
-		 * If this is a transient record type, send the tupledesc in a control
-		 * message.  (TQSendRecordInfo is smart enough to do this only once
-		 * per typmod.)
-		 */
-		if (typid == RECORDOID)
-			TQSendRecordInfo(tqueue, typmod, tupledesc);
-
-		/* Figure out whether fields need recursive processing. */
-		remapinfo->field_remap = BuildFieldRemapInfo(tupledesc,
-													 tqueue->mycontext);
-		if (remapinfo->field_remap != NULL)
-		{
-			/*
-			 * We need to inspect the record contents, so save a copy of the
-			 * tupdesc.  (We could possibly just reference the typcache's
-			 * copy, but then it's problematic when to release the refcount.)
-			 */
-			MemoryContext oldcontext = MemoryContextSwitchTo(tqueue->mycontext);
-
-			remapinfo->tupledesc = CreateTupleDescCopy(tupledesc);
-			MemoryContextSwitchTo(oldcontext);
-		}
-		else
-		{
-			/* No fields of the record require remapping. */
-			remapinfo->tupledesc = NULL;
-		}
-		remapinfo->rectypid = typid;
-		remapinfo->rectypmod = typmod;
-
-		/* Release reference count acquired by lookup_rowtype_tupdesc. */
-		DecrTupleDescRefCount(tupledesc);
-	}
-
-	/*
-	 * If field remapping is required, deform the tuple and examine each
-	 * field.
-	 */
-	if (remapinfo->field_remap != NULL)
-	{
-		Datum	   *values;
-		bool	   *isnull;
-		HeapTupleData tdata;
-		int			i;
-
-		/* Deform the tuple so we can check each column within. */
-		tupledesc = remapinfo->tupledesc;
-		values = (Datum *) palloc(tupledesc->natts * sizeof(Datum));
-		isnull = (bool *) palloc(tupledesc->natts * sizeof(bool));
-		tdata.t_len = HeapTupleHeaderGetDatumLength(tup);
-		ItemPointerSetInvalid(&(tdata.t_self));
-		tdata.t_tableOid = InvalidOid;
-		tdata.t_data = tup;
-		heap_deform_tuple(&tdata, tupledesc, values, isnull);
-
-		/* Recursively check each interesting non-NULL attribute. */
-		for (i = 0; i < tupledesc->natts; i++)
-		{
-			if (!isnull[i] && remapinfo->field_remap[i])
-				TQExamine(tqueue, remapinfo->field_remap[i], values[i]);
-		}
-
-		/* Need not clean up, since we're in a short-lived context. */
-	}
-}
-
-/*
- * Examine an array datum and send any necessary control messages for
- * transient record types contained in it.
- */
-static void
-TQExamineArray(TQueueDestReceiver *tqueue, ArrayRemapInfo *remapinfo,
-			   Datum value)
-{
-	ArrayType  *arr = DatumGetArrayTypeP(value);
-	Oid			typid = ARR_ELEMTYPE(arr);
-	Datum	   *elem_values;
-	bool	   *elem_nulls;
-	int			num_elems;
-	int			i;
-
-	/* Deconstruct the array. */
-	deconstruct_array(arr, typid, remapinfo->typlen,
-					  remapinfo->typbyval, remapinfo->typalign,
-					  &elem_values, &elem_nulls, &num_elems);
-
-	/* Examine each element. */
-	for (i = 0; i < num_elems; i++)
-	{
-		if (!elem_nulls[i])
-			TQExamine(tqueue, remapinfo->element_remap, elem_values[i]);
-	}
-}
-
-/*
- * Examine a range datum and send any necessary control messages for
- * transient record types contained in it.
- */
-static void
-TQExamineRange(TQueueDestReceiver *tqueue, RangeRemapInfo *remapinfo,
-			   Datum value)
-{
-	RangeType  *range = DatumGetRangeType(value);
-	RangeBound	lower;
-	RangeBound	upper;
-	bool		empty;
-
-	/* Extract the lower and upper bounds. */
-	range_deserialize(remapinfo->typcache, range, &lower, &upper, &empty);
-
-	/* Nothing to do for an empty range. */
-	if (empty)
-		return;
-
-	/* Examine each bound, if present. */
-	if (!upper.infinite)
-		TQExamine(tqueue, remapinfo->bound_remap, upper.val);
-	if (!lower.infinite)
-		TQExamine(tqueue, remapinfo->bound_remap, lower.val);
-}
-
-/*
- * Send tuple descriptor information for a transient typmod, unless we've
- * already done so previously.
- */
-static void
-TQSendRecordInfo(TQueueDestReceiver *tqueue, int32 typmod, TupleDesc tupledesc)
-{
-	StringInfoData buf;
-	bool		found;
-	int			i;
-
-	/* Initialize hash table if not done yet. */
-	if (tqueue->recordhtab == NULL)
-	{
-		HASHCTL		ctl;
-
-		MemSet(&ctl, 0, sizeof(ctl));
-		/* Hash table entries are just typmods */
-		ctl.keysize = sizeof(int32);
-		ctl.entrysize = sizeof(int32);
-		ctl.hcxt = tqueue->mycontext;
-		tqueue->recordhtab = hash_create("tqueue sender record type hashtable",
-										 100, &ctl,
-									  HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-	}
-
-	/* Have we already seen this record type?  If not, must report it. */
-	hash_search(tqueue->recordhtab, &typmod, HASH_ENTER, &found);
-	if (found)
-		return;
-
-	elog(DEBUG3, "sending tqueue control message for record typmod %d", typmod);
-
-	/* If message queue is in data mode, switch to control mode. */
-	if (tqueue->mode != TUPLE_QUEUE_MODE_CONTROL)
-	{
-		tqueue->mode = TUPLE_QUEUE_MODE_CONTROL;
-		shm_mq_send(tqueue->queue, sizeof(char), &tqueue->mode, false);
-	}
-
-	/* Assemble a control message. */
-	initStringInfo(&buf);
-	appendBinaryStringInfo(&buf, (char *) &typmod, sizeof(int32));
-	appendBinaryStringInfo(&buf, (char *) &tupledesc->natts, sizeof(int));
-	appendBinaryStringInfo(&buf, (char *) &tupledesc->tdhasoid, sizeof(bool));
-	for (i = 0; i < tupledesc->natts; i++)
-	{
-		appendBinaryStringInfo(&buf, (char *) tupledesc->attrs[i],
-							   sizeof(FormData_pg_attribute));
-	}
-
-	/* Send control message. */
-	shm_mq_send(tqueue->queue, buf.len, buf.data, false);
-
-	/* We assume it's OK to leak buf because we're in a short-lived context. */
-}
-
-/*
  * Prepare to receive tuples from executor.
  */
 static void
@@ -587,15 +104,6 @@ tqueueShutdownReceiver(DestReceiver *self)
 static void
 tqueueDestroyReceiver(DestReceiver *self)
 {
-	TQueueDestReceiver *tqueue = (TQueueDestReceiver *) self;
-
-	if (tqueue->tmpcontext != NULL)
-		MemoryContextDelete(tqueue->tmpcontext);
-	if (tqueue->recordhtab != NULL)
-		hash_destroy(tqueue->recordhtab);
-	/* Is it worth trying to free substructure of the remap tree? */
-	if (tqueue->field_remapinfo != NULL)
-		pfree(tqueue->field_remapinfo);
 	pfree(self);
 }
 
@@ -615,13 +123,8 @@ CreateTupleQueueDestReceiver(shm_mq_handle *handle)
 	self->pub.rDestroy = tqueueDestroyReceiver;
 	self->pub.mydest = DestTupleQueue;
 	self->queue = handle;
-	self->mycontext = CurrentMemoryContext;
-	self->tmpcontext = NULL;
-	self->recordhtab = NULL;
-	self->mode = TUPLE_QUEUE_MODE_DATA;
 	/* Top-level tupledesc is not known yet */
 	self->tupledesc = NULL;
-	self->field_remapinfo = NULL;
 
 	return (DestReceiver *) self;
 }
@@ -635,11 +138,7 @@ CreateTupleQueueReader(shm_mq_handle *handle, TupleDesc tupledesc)
 	TupleQueueReader *reader = palloc0(sizeof(TupleQueueReader));
 
 	reader->queue = handle;
-	reader->mycontext = CurrentMemoryContext;
-	reader->typmodmap = NULL;
-	reader->mode = TUPLE_QUEUE_MODE_DATA;
 	reader->tupledesc = tupledesc;
-	reader->field_remapinfo = BuildFieldRemapInfo(tupledesc, reader->mycontext);
 
 	return reader;
 }
@@ -651,11 +150,6 @@ void
 DestroyTupleQueueReader(TupleQueueReader *reader)
 {
 	shm_mq_detach(shm_mq_get_queue(reader->queue));
-	if (reader->typmodmap != NULL)
-		hash_destroy(reader->typmodmap);
-	/* Is it worth trying to free substructure of the remap tree? */
-	if (reader->field_remapinfo != NULL)
-		pfree(reader->field_remapinfo);
 	pfree(reader);
 }
 
@@ -667,9 +161,6 @@ DestroyTupleQueueReader(TupleQueueReader *reader)
  * is set to true when there are no remaining tuples and otherwise to false.
  *
  * The returned tuple, if any, is allocated in CurrentMemoryContext.
- * That should be a short-lived (tuple-lifespan) context, because we are
- * pretty cavalier about leaking memory in that context if we have to do
- * tuple remapping.
  *
  * Even when shm_mq_receive() returns SHM_MQ_WOULD_BLOCK, this can still
  * accumulate bytes from a partially-read message, so it's useful to call
@@ -678,64 +169,29 @@ DestroyTupleQueueReader(TupleQueueReader *reader)
 HeapTuple
 TupleQueueReaderNext(TupleQueueReader *reader, bool nowait, bool *done)
 {
+	HeapTupleData htup;
 	shm_mq_result result;
+	Size		nbytes;
+	void	   *data;
 
 	if (done != NULL)
 		*done = false;
 
-	for (;;)
-	{
-		Size		nbytes;
-		void	   *data;
-
-		/* Attempt to read a message. */
-		result = shm_mq_receive(reader->queue, &nbytes, &data, nowait);
-
-		/* If queue is detached, set *done and return NULL. */
-		if (result == SHM_MQ_DETACHED)
-		{
-			if (done != NULL)
-				*done = true;
-			return NULL;
-		}
-
-		/* In non-blocking mode, bail out if no message ready yet. */
-		if (result == SHM_MQ_WOULD_BLOCK)
-			return NULL;
-		Assert(result == SHM_MQ_SUCCESS);
+	/* Attempt to read a message. */
+	result = shm_mq_receive(reader->queue, &nbytes, &data, nowait);
 
-		/*
-		 * We got a message (see message spec at top of file).  Process it.
-		 */
-		if (nbytes == 1)
-		{
-			/* Mode switch message. */
-			reader->mode = ((char *) data)[0];
-		}
-		else if (reader->mode == TUPLE_QUEUE_MODE_DATA)
-		{
-			/* Tuple data. */
-			return TupleQueueHandleDataMessage(reader, nbytes, data);
-		}
-		else if (reader->mode == TUPLE_QUEUE_MODE_CONTROL)
-		{
-			/* Control message, describing a transient record type. */
-			TupleQueueHandleControlMessage(reader, nbytes, data);
-		}
-		else
-			elog(ERROR, "unrecognized tqueue mode: %d", (int) reader->mode);
+	/* If queue is detached, set *done and return NULL. */
+	if (result == SHM_MQ_DETACHED)
+	{
+		if (done != NULL)
+			*done = true;
+		return NULL;
 	}
-}
 
-/*
- * Handle a data message - that is, a tuple - from the remote side.
- */
-static HeapTuple
-TupleQueueHandleDataMessage(TupleQueueReader *reader,
-							Size nbytes,
-							HeapTupleHeader data)
-{
-	HeapTupleData htup;
+	/* In non-blocking mode, bail out if no message ready yet. */
+	if (result == SHM_MQ_WOULD_BLOCK)
+		return NULL;
+	Assert(result == SHM_MQ_SUCCESS);
 
 	/*
 	 * Set up a dummy HeapTupleData pointing to the data from the shm_mq
@@ -746,531 +202,5 @@ TupleQueueHandleDataMessage(TupleQueueReader *reader,
 	htup.t_len = nbytes;
 	htup.t_data = data;
 
-	/*
-	 * Either just copy the data into a regular palloc'd tuple, or remap it,
-	 * as required.
-	 */
-	return TQRemapTuple(reader,
-						reader->tupledesc,
-						reader->field_remapinfo,
-						&htup);
-}
-
-/*
- * Copy the given tuple, remapping any transient typmods contained in it.
- */
-static HeapTuple
-TQRemapTuple(TupleQueueReader *reader,
-			 TupleDesc tupledesc,
-			 TupleRemapInfo **field_remapinfo,
-			 HeapTuple tuple)
-{
-	Datum	   *values;
-	bool	   *isnull;
-	bool		changed = false;
-	int			i;
-
-	/*
-	 * If no remapping is necessary, just copy the tuple into a single
-	 * palloc'd chunk, as caller will expect.
-	 */
-	if (field_remapinfo == NULL)
-		return heap_copytuple(tuple);
-
-	/* Deform tuple so we can remap record typmods for individual attrs. */
-	values = (Datum *) palloc(tupledesc->natts * sizeof(Datum));
-	isnull = (bool *) palloc(tupledesc->natts * sizeof(bool));
-	heap_deform_tuple(tuple, tupledesc, values, isnull);
-
-	/* Recursively process each interesting non-NULL attribute. */
-	for (i = 0; i < tupledesc->natts; i++)
-	{
-		if (isnull[i] || field_remapinfo[i] == NULL)
-			continue;
-		values[i] = TQRemap(reader, field_remapinfo[i], values[i], &changed);
-	}
-
-	/* Reconstruct the modified tuple, if anything was modified. */
-	if (changed)
-		return heap_form_tuple(tupledesc, values, isnull);
-	else
-		return heap_copytuple(tuple);
-}
-
-/*
- * Process the given datum and replace any transient record typmods
- * contained in it.  Set *changed to TRUE if we actually changed the datum.
- *
- * remapinfo is previously-computed remapping info about the datum's type.
- *
- * This function just dispatches based on the remap class.
- */
-static Datum
-TQRemap(TupleQueueReader *reader, TupleRemapInfo *remapinfo,
-		Datum value, bool *changed)
-{
-	/* This is recursive, so it could be driven to stack overflow. */
-	check_stack_depth();
-
-	switch (remapinfo->remapclass)
-	{
-		case TQUEUE_REMAP_ARRAY:
-			return TQRemapArray(reader, &remapinfo->u.arr, value, changed);
-
-		case TQUEUE_REMAP_RANGE:
-			return TQRemapRange(reader, &remapinfo->u.rng, value, changed);
-
-		case TQUEUE_REMAP_RECORD:
-			return TQRemapRecord(reader, &remapinfo->u.rec, value, changed);
-	}
-
-	elog(ERROR, "unrecognized tqueue remap class: %d",
-		 (int) remapinfo->remapclass);
-	return (Datum) 0;
-}
-
-/*
- * Process the given array datum and replace any transient record typmods
- * contained in it.  Set *changed to TRUE if we actually changed the datum.
- */
-static Datum
-TQRemapArray(TupleQueueReader *reader, ArrayRemapInfo *remapinfo,
-			 Datum value, bool *changed)
-{
-	ArrayType  *arr = DatumGetArrayTypeP(value);
-	Oid			typid = ARR_ELEMTYPE(arr);
-	bool		element_changed = false;
-	Datum	   *elem_values;
-	bool	   *elem_nulls;
-	int			num_elems;
-	int			i;
-
-	/* Deconstruct the array. */
-	deconstruct_array(arr, typid, remapinfo->typlen,
-					  remapinfo->typbyval, remapinfo->typalign,
-					  &elem_values, &elem_nulls, &num_elems);
-
-	/* Remap each element. */
-	for (i = 0; i < num_elems; i++)
-	{
-		if (!elem_nulls[i])
-			elem_values[i] = TQRemap(reader,
-									 remapinfo->element_remap,
-									 elem_values[i],
-									 &element_changed);
-	}
-
-	if (element_changed)
-	{
-		/* Reconstruct and return the array.  */
-		*changed = true;
-		arr = construct_md_array(elem_values, elem_nulls,
-							   ARR_NDIM(arr), ARR_DIMS(arr), ARR_LBOUND(arr),
-								 typid, remapinfo->typlen,
-								 remapinfo->typbyval, remapinfo->typalign);
-		return PointerGetDatum(arr);
-	}
-
-	/* Else just return the value as-is. */
-	return value;
-}
-
-/*
- * Process the given range datum and replace any transient record typmods
- * contained in it.  Set *changed to TRUE if we actually changed the datum.
- */
-static Datum
-TQRemapRange(TupleQueueReader *reader, RangeRemapInfo *remapinfo,
-			 Datum value, bool *changed)
-{
-	RangeType  *range = DatumGetRangeType(value);
-	bool		bound_changed = false;
-	RangeBound	lower;
-	RangeBound	upper;
-	bool		empty;
-
-	/* Extract the lower and upper bounds. */
-	range_deserialize(remapinfo->typcache, range, &lower, &upper, &empty);
-
-	/* Nothing to do for an empty range. */
-	if (empty)
-		return value;
-
-	/* Remap each bound, if present. */
-	if (!upper.infinite)
-		upper.val = TQRemap(reader, remapinfo->bound_remap,
-							upper.val, &bound_changed);
-	if (!lower.infinite)
-		lower.val = TQRemap(reader, remapinfo->bound_remap,
-							lower.val, &bound_changed);
-
-	if (bound_changed)
-	{
-		/* Reserialize.  */
-		*changed = true;
-		range = range_serialize(remapinfo->typcache, &lower, &upper, empty);
-		return RangeTypeGetDatum(range);
-	}
-
-	/* Else just return the value as-is. */
-	return value;
-}
-
-/*
- * Process the given record datum and replace any transient record typmods
- * contained in it.  Set *changed to TRUE if we actually changed the datum.
- */
-static Datum
-TQRemapRecord(TupleQueueReader *reader, RecordRemapInfo *remapinfo,
-			  Datum value, bool *changed)
-{
-	HeapTupleHeader tup;
-	Oid			typid;
-	int32		typmod;
-	bool		changed_typmod;
-	TupleDesc	tupledesc;
-
-	/* Extract type OID and typmod from tuple. */
-	tup = DatumGetHeapTupleHeader(value);
-	typid = HeapTupleHeaderGetTypeId(tup);
-	typmod = HeapTupleHeaderGetTypMod(tup);
-
-	/*
-	 * If first time through, or if this isn't the same composite type as last
-	 * time, identify the required typmod mapping, and then look up the
-	 * necessary information for processing the fields.
-	 */
-	if (typid != remapinfo->rectypid || typmod != remapinfo->rectypmod)
-	{
-		/* Free any old data. */
-		if (remapinfo->tupledesc != NULL)
-			FreeTupleDesc(remapinfo->tupledesc);
-		/* Is it worth trying to free substructure of the remap tree? */
-		if (remapinfo->field_remap != NULL)
-			pfree(remapinfo->field_remap);
-
-		/* If transient record type, look up matching local typmod. */
-		if (typid == RECORDOID)
-		{
-			RecordTypmodMap *mapent;
-
-			Assert(reader->typmodmap != NULL);
-			mapent = hash_search(reader->typmodmap, &typmod,
-								 HASH_FIND, NULL);
-			if (mapent == NULL)
-				elog(ERROR, "tqueue received unrecognized remote typmod %d",
-					 typmod);
-			remapinfo->localtypmod = mapent->localtypmod;
-		}
-		else
-			remapinfo->localtypmod = -1;
-
-		/* Look up tuple descriptor in typcache. */
-		tupledesc = lookup_rowtype_tupdesc(typid, remapinfo->localtypmod);
-
-		/* Figure out whether fields need recursive processing. */
-		remapinfo->field_remap = BuildFieldRemapInfo(tupledesc,
-													 reader->mycontext);
-		if (remapinfo->field_remap != NULL)
-		{
-			/*
-			 * We need to inspect the record contents, so save a copy of the
-			 * tupdesc.  (We could possibly just reference the typcache's
-			 * copy, but then it's problematic when to release the refcount.)
-			 */
-			MemoryContext oldcontext = MemoryContextSwitchTo(reader->mycontext);
-
-			remapinfo->tupledesc = CreateTupleDescCopy(tupledesc);
-			MemoryContextSwitchTo(oldcontext);
-		}
-		else
-		{
-			/* No fields of the record require remapping. */
-			remapinfo->tupledesc = NULL;
-		}
-		remapinfo->rectypid = typid;
-		remapinfo->rectypmod = typmod;
-
-		/* Release reference count acquired by lookup_rowtype_tupdesc. */
-		DecrTupleDescRefCount(tupledesc);
-	}
-
-	/* If transient record, replace remote typmod with local typmod. */
-	if (typid == RECORDOID && typmod != remapinfo->localtypmod)
-	{
-		typmod = remapinfo->localtypmod;
-		changed_typmod = true;
-	}
-	else
-		changed_typmod = false;
-
-	/*
-	 * If we need to change the typmod, or if there are any potentially
-	 * remappable fields, replace the tuple.
-	 */
-	if (changed_typmod || remapinfo->field_remap != NULL)
-	{
-		HeapTupleData htup;
-		HeapTuple	atup;
-
-		/* For now, assume we always need to change the tuple in this case. */
-		*changed = true;
-
-		/* Copy tuple, possibly remapping contained fields. */
-		ItemPointerSetInvalid(&htup.t_self);
-		htup.t_tableOid = InvalidOid;
-		htup.t_len = HeapTupleHeaderGetDatumLength(tup);
-		htup.t_data = tup;
-		atup = TQRemapTuple(reader,
-							remapinfo->tupledesc,
-							remapinfo->field_remap,
-							&htup);
-
-		/* Apply the correct labeling for a local Datum. */
-		HeapTupleHeaderSetTypeId(atup->t_data, typid);
-		HeapTupleHeaderSetTypMod(atup->t_data, typmod);
-		HeapTupleHeaderSetDatumLength(atup->t_data, htup.t_len);
-
-		/* And return the results. */
-		return HeapTupleHeaderGetDatum(atup->t_data);
-	}
-
-	/* Else just return the value as-is. */
-	return value;
-}
-
-/*
- * Handle a control message from the tuple queue reader.
- *
- * Control messages are sent when the remote side is sending tuples that
- * contain transient record types.  We need to arrange to bless those
- * record types locally and translate between remote and local typmods.
- */
-static void
-TupleQueueHandleControlMessage(TupleQueueReader *reader, Size nbytes,
-							   char *data)
-{
-	int32		remotetypmod;
-	int			natts;
-	bool		hasoid;
-	Size		offset = 0;
-	Form_pg_attribute *attrs;
-	TupleDesc	tupledesc;
-	RecordTypmodMap *mapent;
-	bool		found;
-	int			i;
-
-	/* Extract remote typmod. */
-	memcpy(&remotetypmod, &data[offset], sizeof(int32));
-	offset += sizeof(int32);
-
-	/* Extract attribute count. */
-	memcpy(&natts, &data[offset], sizeof(int));
-	offset += sizeof(int);
-
-	/* Extract hasoid flag. */
-	memcpy(&hasoid, &data[offset], sizeof(bool));
-	offset += sizeof(bool);
-
-	/* Extract attribute details. The tupledesc made here is just transient. */
-	attrs = palloc(natts * sizeof(Form_pg_attribute));
-	for (i = 0; i < natts; i++)
-	{
-		attrs[i] = palloc(sizeof(FormData_pg_attribute));
-		memcpy(attrs[i], &data[offset], sizeof(FormData_pg_attribute));
-		offset += sizeof(FormData_pg_attribute);
-	}
-
-	/* We should have read the whole message. */
-	Assert(offset == nbytes);
-
-	/* Construct TupleDesc, and assign a local typmod. */
-	tupledesc = CreateTupleDesc(natts, hasoid, attrs);
-	tupledesc = BlessTupleDesc(tupledesc);
-
-	/* Create mapping hashtable if it doesn't exist already. */
-	if (reader->typmodmap == NULL)
-	{
-		HASHCTL		ctl;
-
-		MemSet(&ctl, 0, sizeof(ctl));
-		ctl.keysize = sizeof(int32);
-		ctl.entrysize = sizeof(RecordTypmodMap);
-		ctl.hcxt = reader->mycontext;
-		reader->typmodmap = hash_create("tqueue receiver record type hashtable",
-										100, &ctl,
-									  HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
-	}
-
-	/* Create map entry. */
-	mapent = hash_search(reader->typmodmap, &remotetypmod, HASH_ENTER,
-						 &found);
-	if (found)
-		elog(ERROR, "duplicate tqueue control message for typmod %d",
-			 remotetypmod);
-	mapent->localtypmod = tupledesc->tdtypmod;
-
-	elog(DEBUG3, "tqueue mapping remote typmod %d to local typmod %d",
-		 remotetypmod, mapent->localtypmod);
-}
-
-/*
- * Build remap info for the specified data type, storing it in mycontext.
- * Returns NULL if neither the type nor any subtype could require remapping.
- */
-static TupleRemapInfo *
-BuildTupleRemapInfo(Oid typid, MemoryContext mycontext)
-{
-	HeapTuple	tup;
-	Form_pg_type typ;
-
-	/* This is recursive, so it could be driven to stack overflow. */
-	check_stack_depth();
-
-restart:
-	tup = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typid));
-	if (!HeapTupleIsValid(tup))
-		elog(ERROR, "cache lookup failed for type %u", typid);
-	typ = (Form_pg_type) GETSTRUCT(tup);
-
-	/* Look through domains to underlying base type. */
-	if (typ->typtype == TYPTYPE_DOMAIN)
-	{
-		typid = typ->typbasetype;
-		ReleaseSysCache(tup);
-		goto restart;
-	}
-
-	/* If it's a true array type, deal with it that way. */
-	if (OidIsValid(typ->typelem) && typ->typlen == -1)
-	{
-		typid = typ->typelem;
-		ReleaseSysCache(tup);
-		return BuildArrayRemapInfo(typid, mycontext);
-	}
-
-	/* Similarly, deal with ranges appropriately. */
-	if (typ->typtype == TYPTYPE_RANGE)
-	{
-		ReleaseSysCache(tup);
-		return BuildRangeRemapInfo(typid, mycontext);
-	}
-
-	/*
-	 * If it's a composite type (including RECORD), set up for remapping.  We
-	 * don't attempt to determine the status of subfields here, since we do
-	 * not have enough information yet; just mark everything invalid.
-	 */
-	if (typ->typtype == TYPTYPE_COMPOSITE || typid == RECORDOID)
-	{
-		TupleRemapInfo *remapinfo;
-
-		remapinfo = (TupleRemapInfo *)
-			MemoryContextAlloc(mycontext, sizeof(TupleRemapInfo));
-		remapinfo->remapclass = TQUEUE_REMAP_RECORD;
-		remapinfo->u.rec.rectypid = InvalidOid;
-		remapinfo->u.rec.rectypmod = -1;
-		remapinfo->u.rec.localtypmod = -1;
-		remapinfo->u.rec.tupledesc = NULL;
-		remapinfo->u.rec.field_remap = NULL;
-		ReleaseSysCache(tup);
-		return remapinfo;
-	}
-
-	/* Nothing else can possibly need remapping attention. */
-	ReleaseSysCache(tup);
-	return NULL;
-}
-
-static TupleRemapInfo *
-BuildArrayRemapInfo(Oid elemtypid, MemoryContext mycontext)
-{
-	TupleRemapInfo *remapinfo;
-	TupleRemapInfo *element_remapinfo;
-
-	/* See if element type requires remapping. */
-	element_remapinfo = BuildTupleRemapInfo(elemtypid, mycontext);
-	/* If not, the array doesn't either. */
-	if (element_remapinfo == NULL)
-		return NULL;
-	/* OK, set up to remap the array. */
-	remapinfo = (TupleRemapInfo *)
-		MemoryContextAlloc(mycontext, sizeof(TupleRemapInfo));
-	remapinfo->remapclass = TQUEUE_REMAP_ARRAY;
-	get_typlenbyvalalign(elemtypid,
-						 &remapinfo->u.arr.typlen,
-						 &remapinfo->u.arr.typbyval,
-						 &remapinfo->u.arr.typalign);
-	remapinfo->u.arr.element_remap = element_remapinfo;
-	return remapinfo;
-}
-
-static TupleRemapInfo *
-BuildRangeRemapInfo(Oid rngtypid, MemoryContext mycontext)
-{
-	TupleRemapInfo *remapinfo;
-	TupleRemapInfo *bound_remapinfo;
-	TypeCacheEntry *typcache;
-
-	/*
-	 * Get range info from the typcache.  We assume this pointer will stay
-	 * valid for the duration of the query.
-	 */
-	typcache = lookup_type_cache(rngtypid, TYPECACHE_RANGE_INFO);
-	if (typcache->rngelemtype == NULL)
-		elog(ERROR, "type %u is not a range type", rngtypid);
-
-	/* See if range bound type requires remapping. */
-	bound_remapinfo = BuildTupleRemapInfo(typcache->rngelemtype->type_id,
-										  mycontext);
-	/* If not, the range doesn't either. */
-	if (bound_remapinfo == NULL)
-		return NULL;
-	/* OK, set up to remap the range. */
-	remapinfo = (TupleRemapInfo *)
-		MemoryContextAlloc(mycontext, sizeof(TupleRemapInfo));
-	remapinfo->remapclass = TQUEUE_REMAP_RANGE;
-	remapinfo->u.rng.typcache = typcache;
-	remapinfo->u.rng.bound_remap = bound_remapinfo;
-	return remapinfo;
-}
-
-/*
- * Build remap info for fields of the type described by the given tupdesc.
- * Returns an array of TupleRemapInfo pointers, or NULL if no field
- * requires remapping.  Data is allocated in mycontext.
- */
-static TupleRemapInfo **
-BuildFieldRemapInfo(TupleDesc tupledesc, MemoryContext mycontext)
-{
-	TupleRemapInfo **remapinfo;
-	bool		noop = true;
-	int			i;
-
-	/* Recursively determine the remapping status of each field. */
-	remapinfo = (TupleRemapInfo **)
-		MemoryContextAlloc(mycontext,
-						   tupledesc->natts * sizeof(TupleRemapInfo *));
-	for (i = 0; i < tupledesc->natts; i++)
-	{
-		Form_pg_attribute attr = tupledesc->attrs[i];
-
-		if (attr->attisdropped)
-		{
-			remapinfo[i] = NULL;
-			continue;
-		}
-		remapinfo[i] = BuildTupleRemapInfo(attr->atttypid, mycontext);
-		if (remapinfo[i] != NULL)
-			noop = false;
-	}
-
-	/* If no fields require remapping, report that by returning NULL. */
-	if (noop)
-	{
-		pfree(remapinfo);
-		remapinfo = NULL;
-	}
-
-	return remapinfo;
+	return heap_copytuple(&htup);
 }
shared-record-typmod-registry-v1.patchapplication/octet-stream; name=shared-record-typmod-registry-v1.patchDownload
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 9fd7b4e019b..b72f5363045 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -343,11 +343,68 @@ DecrTupleDescRefCount(TupleDesc tupdesc)
 }
 
 /*
- * Compare two TupleDesc structures for logical equality
+ * Compare two TupleDesc attributes for logical equality
  *
  * Note: we deliberately do not check the attrelid and tdtypmod fields.
  * This allows typcache.c to use this routine to see if a cached record type
  * matches a requested type, and is harmless for relcache.c's uses.
+ */
+bool
+equalTupleDescAttrs(Form_pg_attribute attr1, Form_pg_attribute attr2)
+{
+	/*
+	 * We do not need to check every single field here: we can disregard
+	 * attrelid and attnum (which were used to place the row in the attrs
+	 * array in the first place).  It might look like we could dispense
+	 * with checking attlen/attbyval/attalign, since these are derived
+	 * from atttypid; but in the case of dropped columns we must check
+	 * them (since atttypid will be zero for all dropped columns) and in
+	 * general it seems safer to check them always.
+	 *
+	 * attcacheoff must NOT be checked since it's possibly not set in both
+	 * copies.
+	 */
+	if (strcmp(NameStr(attr1->attname), NameStr(attr2->attname)) != 0)
+		return false;
+	if (attr1->atttypid != attr2->atttypid)
+		return false;
+	if (attr1->attstattarget != attr2->attstattarget)
+		return false;
+	if (attr1->attlen != attr2->attlen)
+		return false;
+	if (attr1->attndims != attr2->attndims)
+		return false;
+	if (attr1->atttypmod != attr2->atttypmod)
+		return false;
+	if (attr1->attbyval != attr2->attbyval)
+		return false;
+	if (attr1->attstorage != attr2->attstorage)
+		return false;
+	if (attr1->attalign != attr2->attalign)
+		return false;
+	if (attr1->attnotnull != attr2->attnotnull)
+		return false;
+	if (attr1->atthasdef != attr2->atthasdef)
+		return false;
+	if (attr1->attidentity != attr2->attidentity)
+		return false;
+	if (attr1->attisdropped != attr2->attisdropped)
+		return false;
+	if (attr1->attislocal != attr2->attislocal)
+		return false;
+	if (attr1->attinhcount != attr2->attinhcount)
+		return false;
+	if (attr1->attcollation != attr2->attcollation)
+		return false;
+	/* attacl, attoptions and attfdwoptions are not even present... */
+
+	return true;
+}
+
+/*
+ * Compare two TupleDesc structures for logical equality
+ *
+ * Note: see equalTupleDescAttrs for the note on fields that we don't compare.
  * We don't compare tdrefcount, either.
  */
 bool
@@ -369,51 +426,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
 		Form_pg_attribute attr1 = tupdesc1->attrs[i];
 		Form_pg_attribute attr2 = tupdesc2->attrs[i];
 
-		/*
-		 * We do not need to check every single field here: we can disregard
-		 * attrelid and attnum (which were used to place the row in the attrs
-		 * array in the first place).  It might look like we could dispense
-		 * with checking attlen/attbyval/attalign, since these are derived
-		 * from atttypid; but in the case of dropped columns we must check
-		 * them (since atttypid will be zero for all dropped columns) and in
-		 * general it seems safer to check them always.
-		 *
-		 * attcacheoff must NOT be checked since it's possibly not set in both
-		 * copies.
-		 */
-		if (strcmp(NameStr(attr1->attname), NameStr(attr2->attname)) != 0)
-			return false;
-		if (attr1->atttypid != attr2->atttypid)
-			return false;
-		if (attr1->attstattarget != attr2->attstattarget)
-			return false;
-		if (attr1->attlen != attr2->attlen)
-			return false;
-		if (attr1->attndims != attr2->attndims)
-			return false;
-		if (attr1->atttypmod != attr2->atttypmod)
-			return false;
-		if (attr1->attbyval != attr2->attbyval)
-			return false;
-		if (attr1->attstorage != attr2->attstorage)
-			return false;
-		if (attr1->attalign != attr2->attalign)
-			return false;
-		if (attr1->attnotnull != attr2->attnotnull)
-			return false;
-		if (attr1->atthasdef != attr2->atthasdef)
-			return false;
-		if (attr1->attidentity != attr2->attidentity)
-			return false;
-		if (attr1->attisdropped != attr2->attisdropped)
-			return false;
-		if (attr1->attislocal != attr2->attislocal)
-			return false;
-		if (attr1->attinhcount != attr2->attinhcount)
-			return false;
-		if (attr1->attcollation != attr2->attcollation)
+		if (!equalTupleDescAttrs(attr1, attr2))
 			return false;
-		/* attacl, attoptions and attfdwoptions are not even present... */
 	}
 
 	if (tupdesc1->constr != NULL)
diff --git a/src/backend/access/transam/parallel.c b/src/backend/access/transam/parallel.c
index b3d3853fbc2..29532cf379c 100644
--- a/src/backend/access/transam/parallel.c
+++ b/src/backend/access/transam/parallel.c
@@ -35,6 +35,7 @@
 #include "utils/memutils.h"
 #include "utils/resowner.h"
 #include "utils/snapmgr.h"
+#include "utils/typcache.h"
 
 
 /*
@@ -46,6 +47,14 @@
  */
 #define PARALLEL_ERROR_QUEUE_SIZE			16384
 
+/*
+ * We want to create a DSA area to store shared state that has the same extent
+ * as a parallel context, to hold the record type registry.  We don't want it
+ * to have to create any DSM segments just yet in common cases, so we'll give
+ * it enough space to hold an empty SharedRecordTypmodRegistry.
+ */
+#define PARALLEL_CONTEXT_DSA_SIZE			0x30000
+
 /* Magic number for parallel context TOC. */
 #define PARALLEL_MAGIC						0x50477c7c
 
@@ -62,6 +71,8 @@
 #define PARALLEL_KEY_ACTIVE_SNAPSHOT		UINT64CONST(0xFFFFFFFFFFFF0007)
 #define PARALLEL_KEY_TRANSACTION_STATE		UINT64CONST(0xFFFFFFFFFFFF0008)
 #define PARALLEL_KEY_EXTENSION_TRAMPOLINE	UINT64CONST(0xFFFFFFFFFFFF0009)
+#define PARALLEL_KEY_CONTEXT_DSA			UINT64CONST(0xFFFFFFFFFFFF000A)
+#define PARALLEL_KEY_RECORD_TYPMOD_REGISTRY	UINT64CONST(0xFFFFFFFFFFFF000B)
 
 /* Fixed-size parallel state. */
 typedef struct FixedParallelState
@@ -202,6 +213,7 @@ InitializeParallelDSM(ParallelContext *pcxt)
 	Size		library_len = 0;
 	Size		guc_len = 0;
 	Size		combocidlen = 0;
+	Size		typmod_registry_size = 0;
 	Size		tsnaplen = 0;
 	Size		asnaplen = 0;
 	Size		tstatelen = 0;
@@ -237,8 +249,11 @@ InitializeParallelDSM(ParallelContext *pcxt)
 		shm_toc_estimate_chunk(&pcxt->estimator, asnaplen);
 		tstatelen = EstimateTransactionStateSpace();
 		shm_toc_estimate_chunk(&pcxt->estimator, tstatelen);
+		shm_toc_estimate_chunk(&pcxt->estimator, PARALLEL_CONTEXT_DSA_SIZE);
+		typmod_registry_size = SharedRecordTypmodRegistryEstimate();
+		shm_toc_estimate_chunk(&pcxt->estimator, typmod_registry_size);
 		/* If you add more chunks here, you probably need to add keys. */
-		shm_toc_estimate_keys(&pcxt->estimator, 6);
+		shm_toc_estimate_keys(&pcxt->estimator, 8);
 
 		/* Estimate space need for error queues. */
 		StaticAssertStmt(BUFFERALIGN(PARALLEL_ERROR_QUEUE_SIZE) ==
@@ -312,6 +327,8 @@ InitializeParallelDSM(ParallelContext *pcxt)
 		char	   *asnapspace;
 		char	   *tstatespace;
 		char	   *error_queue_space;
+		char	   *typemod_registry_space;
+		char	   *context_dsa_space;
 
 		/* Serialize shared libraries we have loaded. */
 		libraryspace = shm_toc_allocate(pcxt->toc, library_len);
@@ -328,6 +345,27 @@ InitializeParallelDSM(ParallelContext *pcxt)
 		SerializeComboCIDState(combocidlen, combocidspace);
 		shm_toc_insert(pcxt->toc, PARALLEL_KEY_COMBO_CID, combocidspace);
 
+		/*
+		 * Make a DSA area for dynamically-sized shared state that has the
+		 * same scope as this ParallelContext.
+		 */
+		context_dsa_space = shm_toc_allocate(pcxt->toc,
+											 PARALLEL_CONTEXT_DSA_SIZE);
+		pcxt->context_dsa = dsa_create_in_place(context_dsa_space,
+												PARALLEL_CONTEXT_DSA_SIZE,
+												LWTRANCHE_PARALLEL_CONTEXT_DSA,
+												pcxt->seg);
+		shm_toc_insert(pcxt->toc, PARALLEL_KEY_CONTEXT_DSA, context_dsa_space);
+
+		/* Set up shared record type registry. */
+		typemod_registry_space = shm_toc_allocate(pcxt->toc,
+												  typmod_registry_size);
+		SharedRecordTypmodRegistryInit((SharedRecordTypmodRegistry *)
+									   typemod_registry_space,
+									   pcxt->context_dsa, pcxt->seg);
+		shm_toc_insert(pcxt->toc, PARALLEL_KEY_RECORD_TYPMOD_REGISTRY,
+					   typemod_registry_space);
+
 		/* Serialize transaction snapshot and active snapshot. */
 		tsnapspace = shm_toc_allocate(pcxt->toc, tsnaplen);
 		SerializeSnapshot(transaction_snapshot, tsnapspace);
@@ -633,6 +671,13 @@ DestroyParallelContext(ParallelContext *pcxt)
 		}
 	}
 
+	/* Detach from the context-scope DSA area, if there is one. */
+	if (pcxt->context_dsa != NULL)
+	{
+		dsa_detach(pcxt->context_dsa);
+		pcxt->context_dsa = NULL;
+	}
+
 	/*
 	 * If we have allocated a shared memory segment, detach it.  This will
 	 * implicitly detach the error queues, and any other shared memory queues,
@@ -947,6 +992,9 @@ ParallelWorkerMain(Datum main_arg)
 	char	   *asnapspace;
 	char	   *tstatespace;
 	StringInfoData msgbuf;
+	char	   *typmod_registry_space;
+	char	   *context_dsa_space;
+	dsa_area   *context_dsa;
 
 	/* Set flag to indicate that we're initializing a parallel worker. */
 	InitializingParallelWorker = true;
@@ -1066,6 +1114,20 @@ ParallelWorkerMain(Datum main_arg)
 	Assert(combocidspace != NULL);
 	RestoreComboCIDState(combocidspace);
 
+	/* Attach to the DSA area. */
+	context_dsa_space = shm_toc_lookup(toc, PARALLEL_KEY_CONTEXT_DSA);
+	Assert(context_dsa_space != NULL);
+	context_dsa = dsa_attach_in_place(context_dsa_space, seg);
+
+	/* Attach to shared record type registry. */
+	typmod_registry_space =
+		shm_toc_lookup(toc, PARALLEL_KEY_RECORD_TYPMOD_REGISTRY);
+	Assert(typmod_registry_space != NULL);
+	SharedRecordTypmodRegistryAttach((SharedRecordTypmodRegistry *)
+									 typmod_registry_space,
+									 context_dsa,
+									 seg);
+
 	/* Restore transaction snapshot. */
 	tsnapspace = shm_toc_lookup(toc, PARALLEL_KEY_TRANSACTION_SNAPSHOT);
 	Assert(tsnapspace != NULL);
@@ -1114,6 +1176,9 @@ ParallelWorkerMain(Datum main_arg)
 	/* Must pop active snapshot so resowner.c doesn't complain. */
 	PopActiveSnapshot();
 
+	/* Detach from context-scoped DSA area. */
+	dsa_detach(context_dsa);
+
 	/* Shut down the parallel-worker transaction. */
 	EndParallelWorkerTransaction();
 
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 3e133941f47..76e813d59b7 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -494,7 +494,7 @@ RegisterLWLockTranches(void)
 
 	if (LWLockTrancheArray == NULL)
 	{
-		LWLockTranchesAllocated = 64;
+		LWLockTranchesAllocated = 128;
 		LWLockTrancheArray = (char **)
 			MemoryContextAllocZero(TopMemoryContext,
 						  LWLockTranchesAllocated * sizeof(char *));
@@ -510,7 +510,13 @@ RegisterLWLockTranches(void)
 						  "predicate_lock_manager");
 	LWLockRegisterTranche(LWTRANCHE_PARALLEL_QUERY_DSA,
 						  "parallel_query_dsa");
+	LWLockRegisterTranche(LWTRANCHE_PARALLEL_CONTEXT_DSA,
+						  "parallel_context_dsa");
 	LWLockRegisterTranche(LWTRANCHE_TBM, "tbm");
+	LWLockRegisterTranche(LWTRANCHE_SHARED_RECORD_ATTS_INDEX,
+						  "shared_record_atts_index");
+	LWLockRegisterTranche(LWTRANCHE_SHARED_RECORD_TYPMOD_INDEX,
+						  "shared_record_typmods_index");
 
 	/* Register named tranches. */
 	for (i = 0; i < NamedLWLockTrancheRequests; i++)
diff --git a/src/backend/utils/cache/typcache.c b/src/backend/utils/cache/typcache.c
index 0cf5001a758..f26b0a7a58f 100644
--- a/src/backend/utils/cache/typcache.c
+++ b/src/backend/utils/cache/typcache.c
@@ -55,7 +55,9 @@
 #include "catalog/pg_type.h"
 #include "commands/defrem.h"
 #include "executor/executor.h"
+#include "lib/dht.h"
 #include "optimizer/planner.h"
+#include "storage/lwlock.h"
 #include "utils/builtins.h"
 #include "utils/catcache.h"
 #include "utils/fmgroids.h"
@@ -67,7 +69,6 @@
 #include "utils/syscache.h"
 #include "utils/typcache.h"
 
-
 /* The main type cache hashtable searched by lookup_type_cache */
 static HTAB *TypeCacheHash = NULL;
 
@@ -148,12 +149,95 @@ typedef struct RecordCacheEntry
 	List	   *tupdescs;
 } RecordCacheEntry;
 
+/*
+ * A mechanism for sharing record typmods between backends.
+ */
+struct SharedRecordTypmodRegistry
+{
+	dht_hash_table_handle atts_index_handle;
+	dht_hash_table_handle typmod_index_handle;
+	pg_atomic_uint32 next_typmod;
+};
+
+/*
+ * A flattened/serialized representation of a TupleDesc for use in shared
+ * memory.  Can be converted to and from regular TupleDesc format.  Doesn't
+ * support constraints and doesn't store the actual type OID, because this is
+ * only for use with RECORD types as created by CreateTupleDesc().  These are
+ * arranged into a linked list, in the hash table entry corresponding to the
+ * OIDs of the first 16 attributes, so we'd expect to get more than one entry
+ * in the list when named and other properties differ.
+ */
+typedef struct SerializedTupleDesc
+{
+	dsa_pointer next;			/* next with the same same attribute OIDs */
+	int			natts;			/* number of attributes in the tuple */
+	int32		typmod;			/* typmod for tuple type */
+	bool		hasoid;			/* tuple has oid attribute in its header */
+
+	/*
+	 * The attributes follow.  We only ever access the first
+	 * ATTRIBUTE_FIXED_PART_SIZE bytes of each element, like the code in
+	 * tupdesc.c.
+	 */
+	FormData_pg_attribute attributes[FLEXIBLE_ARRAY_MEMBER];
+} SerializedTupleDesc;
+
+/*
+ * An entry in SharedRecordTypmodRegistry's attribute index.  The key is the
+ * first REC_HASH_KEYS attribute OIDs.  That means that collisions are
+ * possible, but that's OK because SerializedTupleDesc objects are arranged
+ * into a list.
+ */
+typedef struct SRTRAttsIndexEntry
+{
+	Oid			leading_attr_oids[REC_HASH_KEYS];
+	dsa_pointer serialized_tupdesc;
+} SRTRAttsIndexEntry;
+
+/*
+ * An entry in SharedRecordTypmodRegistry's typmod index.  Points to a single
+ * SerializedTupleDesc in shared memory.
+ */
+typedef struct SRTRTypmodIndexEntry
+{
+	uint32		typmod;
+	dsa_pointer serialized_tupdesc;
+} SRTRTypmodIndexEntry;
+
+/* Parameters for SharedRecordTypmodRegistry's attributes hash table. */
+const static dht_parameters srtr_atts_index_params = {
+	sizeof(Oid) * REC_HASH_KEYS,
+	sizeof(SRTRAttsIndexEntry),
+	memcmp,
+	tag_hash,
+	LWTRANCHE_SHARED_RECORD_ATTS_INDEX
+};
+
+/* Parameters for SharedRecordTypmodRegistry's typmod hash table. */
+const static dht_parameters srtr_typmod_index_params = {
+	sizeof(uint32),
+	sizeof(SRTRTypmodIndexEntry),
+	memcmp,
+	tag_hash,
+	LWTRANCHE_SHARED_RECORD_TYPMOD_INDEX
+};
+
 static HTAB *RecordCacheHash = NULL;
 
 static TupleDesc *RecordCacheArray = NULL;
 static int32 RecordCacheArrayLen = 0;	/* allocated length of array */
 static int32 NextRecordTypmod = 0;		/* number of entries used */
 
+/* Current SharedRecordTypmodRegistry, if attached. */
+static struct
+{
+	SharedRecordTypmodRegistry *shared;
+	dht_hash_table *atts_index;
+	dht_hash_table *typmod_index;
+	dsa_area *area;
+} CurrentSharedRecordTypmodRegistry;
+
 static void load_typcache_tupdesc(TypeCacheEntry *typentry);
 static void load_rangetype_info(TypeCacheEntry *typentry);
 static void load_domaintype_info(TypeCacheEntry *typentry);
@@ -174,6 +258,13 @@ static void TypeCacheConstrCallback(Datum arg, int cacheid, uint32 hashvalue);
 static void load_enum_cache_data(TypeCacheEntry *tcache);
 static EnumItem *find_enumitem(TypeCacheEnumData *enumdata, Oid arg);
 static int	enum_oid_cmp(const void *left, const void *right);
+static void shared_record_typmod_registry_detach(dsm_segment *segment,
+												 Datum datum);
+static int32 find_or_allocate_shared_record_typmod(TupleDesc tupdesc);
+static TupleDesc deserialize_tupledesc(const SerializedTupleDesc *serialized);
+static dsa_pointer serialize_tupledesc(dsa_area *area,
+									   const TupleDesc tupdesc);
+
 
 
 /*
@@ -1199,6 +1290,33 @@ cache_record_field_properties(TypeCacheEntry *typentry)
 	typentry->flags |= TCFLAGS_CHECKED_FIELD_PROPERTIES;
 }
 
+/*
+ * Make sure that RecordCacheArray is large enough to store 'typmod'.
+ */
+static void
+ensure_record_cache_typmod_slot_exists(int32 typmod)
+{
+	if (RecordCacheArray == NULL)
+	{
+		RecordCacheArray = (TupleDesc *)
+			MemoryContextAllocZero(CacheMemoryContext, 64 * sizeof(TupleDesc));
+		RecordCacheArrayLen = 64;
+	}
+
+	if (typmod >= RecordCacheArrayLen)
+	{
+		int32		newlen = RecordCacheArrayLen * 2;
+
+		while (typmod >= newlen)
+			newlen *= 2;
+
+		RecordCacheArray = (TupleDesc *) repalloc(RecordCacheArray,
+												  newlen * sizeof(TupleDesc));
+		memset(RecordCacheArray + RecordCacheArrayLen, 0,
+			   (newlen - RecordCacheArrayLen) * sizeof(TupleDesc *));
+		RecordCacheArrayLen = newlen;
+	}
+}
 
 /*
  * lookup_rowtype_tupdesc_internal --- internal routine to lookup a rowtype
@@ -1229,15 +1347,49 @@ lookup_rowtype_tupdesc_internal(Oid type_id, int32 typmod, bool noError)
 		/*
 		 * It's a transient record type, so look in our record-type table.
 		 */
-		if (typmod < 0 || typmod >= NextRecordTypmod)
+		if (typmod >= 0)
 		{
-			if (!noError)
-				ereport(ERROR,
-						(errcode(ERRCODE_WRONG_OBJECT_TYPE),
-						 errmsg("record type has not been registered")));
-			return NULL;
+			/* It is already in our local cache? */
+			if (typmod < RecordCacheArrayLen &&
+				RecordCacheArray[typmod] != NULL)
+				return RecordCacheArray[typmod];
+
+			/* Are we attached to a SharedRecordTypmodRegistry? */
+			if (CurrentSharedRecordTypmodRegistry.shared != NULL)
+			{
+				SRTRTypmodIndexEntry *entry;
+
+				/* Try to find it in the shared typmod index. */
+				entry = dht_find(CurrentSharedRecordTypmodRegistry.typmod_index,
+								 &typmod, false);
+				if (entry != NULL)
+				{
+					SerializedTupleDesc *serialized;
+
+					serialized = (SerializedTupleDesc *)
+						dsa_get_address(CurrentSharedRecordTypmodRegistry.area,
+										entry->serialized_tupdesc);
+					Assert(typmod == serialized->typmod);
+
+					/* We may need to extend the local RecordCacheArray. */
+					ensure_record_cache_typmod_slot_exists(typmod);
+
+					/* Produce and cache a TupleDesc. */
+					RecordCacheArray[typmod] =
+						deserialize_tupledesc(serialized);
+					dht_release(CurrentSharedRecordTypmodRegistry.typmod_index,
+								entry);
+
+					return RecordCacheArray[typmod];
+				}
+			}
 		}
-		return RecordCacheArray[typmod];
+
+		if (!noError)
+			ereport(ERROR,
+					(errcode(ERRCODE_WRONG_OBJECT_TYPE),
+					 errmsg("record type has not been registered")));
+		return NULL;
 	}
 }
 
@@ -1362,30 +1514,27 @@ assign_record_type_typmod(TupleDesc tupDesc)
 		}
 	}
 
+	/* Look in the SharedRecordTypmodRegistry, if attached */
+	newtypmod = find_or_allocate_shared_record_typmod(tupDesc);
+
 	/* Not present, so need to manufacture an entry */
 	oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
 
-	if (RecordCacheArray == NULL)
-	{
-		RecordCacheArray = (TupleDesc *) palloc(64 * sizeof(TupleDesc));
-		RecordCacheArrayLen = 64;
-	}
-	else if (NextRecordTypmod >= RecordCacheArrayLen)
-	{
-		int32		newlen = RecordCacheArrayLen * 2;
-
-		RecordCacheArray = (TupleDesc *) repalloc(RecordCacheArray,
-												  newlen * sizeof(TupleDesc));
-		RecordCacheArrayLen = newlen;
-	}
+	/*
+	 * Whether we just got a new typmod from a SharedRecordTypmodRegistry or
+	 * we're allocating one locally, make sure the RecordCacheArray is big
+	 * enough.
+	 */
+	ensure_record_cache_typmod_slot_exists(Max(NextRecordTypmod, newtypmod));
 
 	/* if fail in subrs, no damage except possibly some wasted memory... */
 	entDesc = CreateTupleDescCopy(tupDesc);
 	recentry->tupdescs = lcons(entDesc, recentry->tupdescs);
 	/* mark it as a reference-counted tupdesc */
 	entDesc->tdrefcount = 1;
-	/* now it's safe to advance NextRecordTypmod */
-	newtypmod = NextRecordTypmod++;
+	/* now it's safe to advance NextRecordTypmod, if allocating locally */
+	if (newtypmod == -1)
+		newtypmod = NextRecordTypmod++;
 	entDesc->tdtypmod = newtypmod;
 	RecordCacheArray[newtypmod] = entDesc;
 
@@ -1396,6 +1545,176 @@ assign_record_type_typmod(TupleDesc tupDesc)
 }
 
 /*
+ * Return the amout of shmem required to hold a SharedRecordTypmodRegistry.
+ * This exists only to avoid exposing private innards of
+ * SharedRecordTypmodRegistry in a header.
+ */
+size_t
+SharedRecordTypmodRegistryEstimate(void)
+{
+	return sizeof(SharedRecordTypmodRegistry);
+}
+
+/*
+ * Initialize 'registry' in a pre-existing shared memory region, which must be
+ * maximally aligned and have space for SharedRecordTypmodRegistryEstimate()
+ * bytes.
+ *
+ * 'area' will be used to allocate shared memory space as required for the
+ * typemod registration.  The current process, expected to be a leader process
+ * in a parallel query, will be attached automatically and its current record
+ * types will be loaded into the *registry.  While attached, all calls to
+ * assign_record_type_typmod will use the shared registry.  Other backends
+ * will need to attach explicitly.
+ *
+ * An on-detach callback will be installed for 'segment', so that normal
+ * private record type cache behavior can be restored when the DSM segment
+ * goes away.
+ */
+void
+SharedRecordTypmodRegistryInit(SharedRecordTypmodRegistry *registry,
+							 dsa_area *area,
+							 dsm_segment *segment)
+{
+	dht_hash_table *atts_index;
+	dht_hash_table *typmod_index;
+	int32 typmod;
+
+	/* We can't already be attached to a shared registry. */
+	Assert(CurrentSharedRecordTypmodRegistry.shared == NULL);
+	Assert(CurrentSharedRecordTypmodRegistry.atts_index == NULL);
+	Assert(CurrentSharedRecordTypmodRegistry.typmod_index == NULL);
+	Assert(CurrentSharedRecordTypmodRegistry.area == NULL);
+
+	/* Create the hash table indexed by attribute OIDs. */
+	atts_index = dht_create(area, &srtr_atts_index_params);
+	registry->atts_index_handle = dht_get_hash_table_handle(atts_index);
+
+	/* Create the hash table indexed by typmod. */
+	typmod_index = dht_create(area, &srtr_typmod_index_params);
+	registry->typmod_index_handle = dht_get_hash_table_handle(typmod_index);
+
+	/* Initialize the 'next typmode' to this backend's next value. */
+	pg_atomic_init_u32(&registry->next_typmod, NextRecordTypmod);
+
+	/*
+	 * Copy all entries from this backend's private registry into the shared
+	 * registry.
+	 */
+	for (typmod = 0; typmod < NextRecordTypmod; ++typmod)
+	{
+		SRTRTypmodIndexEntry *typmod_index_entry;
+		SRTRAttsIndexEntry *atts_index_entry;
+		SerializedTupleDesc *serialized;
+		dsa_pointer serialized_dp;
+		TupleDesc tupdesc;
+		Oid atts_key[REC_HASH_KEYS];
+		bool found;
+		int i;
+
+		tupdesc = RecordCacheArray[typmod];
+		if (tupdesc == NULL)
+			continue;
+
+		/* Serialize the TupleDesc into shared memory. */
+		serialized_dp = serialize_tupledesc(area, tupdesc);
+
+		/* Insert into the typmod index. */
+		typmod_index_entry = dht_find_or_insert(typmod_index,
+												&tupdesc->tdtypmod,
+												&found);
+		if (found)
+			elog(ERROR, "cannot create duplicate shared record typmod");
+		typmod_index_entry->typmod = tupdesc->tdtypmod;
+		typmod_index_entry->serialized_tupdesc = serialized_dp;
+		dht_release(typmod_index, typmod_index_entry);
+
+		/* Insert into the attributes index. */
+		memset(atts_key, 0, sizeof(atts_key));
+		for (i = 0; i < Min(tupdesc->natts, REC_HASH_KEYS); ++i)
+			atts_key[i] = tupdesc->attrs[i]->atttypid;
+		atts_index_entry = dht_find_or_insert(atts_index, &atts_key, &found);
+		if (!found)
+		{
+			memcpy(atts_index_entry->leading_attr_oids,
+				   atts_key,
+				   sizeof(atts_key));
+			atts_index_entry->serialized_tupdesc = InvalidDsaPointer;
+		}
+
+		/* Push onto list. */
+		serialized = (SerializedTupleDesc *)
+			dsa_get_address(area, serialized_dp);
+		serialized->next = atts_index_entry->serialized_tupdesc;
+		atts_index_entry->serialized_tupdesc = serialized_dp;
+		dht_release(atts_index, atts_index_entry);
+	}
+
+	/* Set up our detach hook so that we can return to private cache mode. */
+	on_dsm_detach(segment, shared_record_typmod_registry_detach,
+				  PointerGetDatum(registry));
+
+	/*
+	 * Set up the global state that will tell assign_record_type_typmod and
+	 * lookup_rowtype_tupdesc_internal about the shared registry.
+	 */
+	CurrentSharedRecordTypmodRegistry.shared = registry;
+	CurrentSharedRecordTypmodRegistry.atts_index = atts_index;
+	CurrentSharedRecordTypmodRegistry.typmod_index = typmod_index;
+	CurrentSharedRecordTypmodRegistry.area = area;
+}
+
+/*
+ * Attach to 'registry', which must have been initialized already by another
+ * backend.  Future calls to assign_record_type_typmod and
+ * lookup_rowtype_tupdesc_internal will use the shared registry, until
+ * 'segment' is detached.
+ */
+void
+SharedRecordTypmodRegistryAttach(SharedRecordTypmodRegistry *registry,
+							   dsa_area *area,
+							   dsm_segment *segment)
+{
+	dht_hash_table *atts_index;
+	dht_hash_table *typmod_index;
+
+	/* We can't already be attached to a shared registry. */
+	Assert(CurrentSharedRecordTypmodRegistry.shared == NULL);
+	Assert(CurrentSharedRecordTypmodRegistry.atts_index == NULL);
+	Assert(CurrentSharedRecordTypmodRegistry.typmod_index == NULL);
+	Assert(CurrentSharedRecordTypmodRegistry.area == NULL);
+
+	/*
+	 * We can't already have typmods in our local cache, because they'd clash
+	 * with those imported by SharedRecordTypmodRegistryInit.  This should be a
+	 * freshly started parallel worker.  If we ever support worker recycling,
+	 * a worker would need to zap its local cache in between servicing
+	 * different queries, in order to be able to call this and synchronize
+	 * typmods with a new leader.
+	 */
+	Assert(NextRecordTypmod == 0);
+
+	/* Attach to the two hash tables. */
+	atts_index = dht_attach(area, &srtr_atts_index_params,
+							registry->atts_index_handle);
+	typmod_index = dht_attach(area, &srtr_typmod_index_params,
+							  registry->typmod_index_handle);
+
+	/* Set up our detach hook so that we can return to private cache mode. */
+	on_dsm_detach(segment, shared_record_typmod_registry_detach,
+				  PointerGetDatum(registry));
+
+	/*
+	 * Set up the global state that will tell assign_record_type_typmod and
+	 * lookup_rowtype_tupdesc_internal about the shared registry.
+	 */
+	CurrentSharedRecordTypmodRegistry.shared = registry;
+	CurrentSharedRecordTypmodRegistry.atts_index = atts_index;
+	CurrentSharedRecordTypmodRegistry.typmod_index = typmod_index;
+	CurrentSharedRecordTypmodRegistry.area = area;
+}
+
+/*
  * TypeCacheRelCallback
  *		Relcache inval callback function
  *
@@ -1809,3 +2128,225 @@ enum_oid_cmp(const void *left, const void *right)
 	else
 		return 0;
 }
+
+/*
+ * Serialize a TupleDesc into a SerializedTupleDesc in DSA area 'area', and
+ * return a dsa_pointer.
+ */
+static dsa_pointer
+serialize_tupledesc(dsa_area *area, const TupleDesc tupdesc)
+{
+	SerializedTupleDesc *serialized;
+	dsa_pointer serialized_dp;
+	size_t size;
+	int i;
+
+	size = offsetof(SerializedTupleDesc, attributes) +
+		sizeof(FormData_pg_attribute) * tupdesc->natts;
+	serialized_dp = dsa_allocate(area, size);
+	serialized = (SerializedTupleDesc *) dsa_get_address(area, serialized_dp);
+
+	serialized->natts = tupdesc->natts;
+	serialized->typmod = tupdesc->tdtypmod;
+	serialized->hasoid = tupdesc->tdhasoid;
+	for (i = 0; i < tupdesc->natts; ++i)
+		memcpy(&serialized->attributes[i], tupdesc->attrs[i],
+			   ATTRIBUTE_FIXED_PART_SIZE);
+
+	return serialized_dp;
+}
+
+/*
+ * Deserialize a SerializedTupleDesc to produce a TupleDesc.  The result is
+ * allocated in CacheMemoryContext and has a refcount of 1.
+ */
+static TupleDesc
+deserialize_tupledesc(const SerializedTupleDesc *serialized)
+{
+	Form_pg_attribute *attributes;
+	MemoryContext oldctxt;
+	TupleDesc tupdesc;
+	int i;
+
+	/*
+	 * We have an array of FormData_pg_attribute but we need an array of
+	 * pointers to FormData_pg_attribute.
+	 */
+	oldctxt = MemoryContextSwitchTo(CacheMemoryContext);
+	attributes = palloc(sizeof(Form_pg_attribute) * serialized->natts);
+	for (i = 0; i < serialized->natts; ++i)
+	{
+		attributes[i] = palloc(ATTRIBUTE_FIXED_PART_SIZE);
+		memcpy(attributes[i], &serialized->attributes[i],
+			   ATTRIBUTE_FIXED_PART_SIZE);
+	}
+	tupdesc =
+		CreateTupleDesc(serialized->natts, serialized->hasoid, attributes);
+	tupdesc->tdtypmod = serialized->typmod;
+	tupdesc->tdrefcount = 1;
+	MemoryContextSwitchTo(oldctxt);
+
+	return tupdesc;
+}
+
+/*
+ * We can't use equalTupleDescs to compare a SerializedTupleDesc with a
+ * TupleDesc, but we don't want to allocate memory just to compare.  This
+ * function produces the same result without deserializing first.
+ */
+static bool
+serialized_tupledesc_matches(SerializedTupleDesc *serialized,
+							 TupleDesc tupdesc)
+{
+	int i;
+
+	if (serialized->natts != tupdesc->natts ||
+		serialized->hasoid != tupdesc->tdhasoid ||
+		tupdesc->constr != NULL)
+		return false;
+
+	for (i = 0; i < serialized->natts; ++i)
+	{
+		if (!equalTupleDescAttrs(&serialized->attributes[i],
+								 tupdesc->attrs[i]))
+			return false;
+	}
+
+	return true;
+}
+
+/*
+ * If we are attached to a SharedRecordTypmodRegistry, find or create a
+ * SerializedTupleDesc that matches 'tupdesc', and return its typmod.
+ * Otherwise return -1.
+ */
+static int32
+find_or_allocate_shared_record_typmod(TupleDesc tupdesc)
+{
+	SRTRAttsIndexEntry *atts_index_entry;
+	SRTRTypmodIndexEntry *typmod_index_entry;
+	SerializedTupleDesc *serialized;
+	dsa_pointer serialized_dp;
+	Oid			hashkey[REC_HASH_KEYS];
+	bool		found;
+	int32		typmod;
+	int			i;
+
+	/* If not even attached, nothing to do. */
+	if (CurrentSharedRecordTypmodRegistry.shared == NULL)
+		return -1;
+
+	/* Try to find a match. */
+	memset(hashkey, 0, sizeof(hashkey));
+	for (i = 0; i < tupdesc->natts; ++i)
+		hashkey[i] = tupdesc->attrs[i]->atttypid;
+	atts_index_entry = (SRTRAttsIndexEntry *)
+		dht_find_or_insert(CurrentSharedRecordTypmodRegistry.atts_index,
+						   hashkey,
+						   &found);
+	if (!found)
+	{
+		/* Making a new entry. */
+		memcpy(atts_index_entry->leading_attr_oids,
+			   hashkey,
+			   sizeof(hashkey));
+		atts_index_entry->serialized_tupdesc = InvalidDsaPointer;
+	}
+
+	/* Scan the list we found for a matching serialized one. */
+	serialized_dp = atts_index_entry->serialized_tupdesc;
+	while (DsaPointerIsValid(serialized_dp))
+	{
+		serialized =
+			dsa_get_address(CurrentSharedRecordTypmodRegistry.area,
+							serialized_dp);
+		if (serialized_tupledesc_matches(serialized, tupdesc))
+		{
+			/* Found a match, we are finished. */
+			typmod = serialized->typmod;
+			dht_release(CurrentSharedRecordTypmodRegistry.atts_index,
+						atts_index_entry);
+			return typmod;
+		}
+		serialized_dp = serialized->next;
+	}
+
+	/* We didn't find a matching entry, so let's allocate a new one. */
+	typmod = (int)
+		pg_atomic_fetch_add_u32(&CurrentSharedRecordTypmodRegistry.shared->next_typmod,
+								1);
+
+	/* Allocate shared memory and serialize the TupleDesc. */
+	serialized_dp = serialize_tupledesc(CurrentSharedRecordTypmodRegistry.area,
+										tupdesc);
+	serialized = (SerializedTupleDesc *)
+		dsa_get_address(CurrentSharedRecordTypmodRegistry.area, serialized_dp);
+	serialized->typmod = typmod;
+
+	/*
+	 * While we still hold the atts_index entry locked, add this to
+	 * typmod_index.  That's important because we don't want anyone to be able
+	 * to find a typmod via the former that can't yet be looked up in the
+	 * latter.
+	 */
+	typmod_index_entry =
+		dht_find_or_insert(CurrentSharedRecordTypmodRegistry.typmod_index,
+						   &typmod, &found);
+	if (found)
+		elog(ERROR, "cannot create duplicate shared record typmod");
+	typmod_index_entry->typmod = typmod;
+	typmod_index_entry->serialized_tupdesc = serialized_dp;
+	dht_release(CurrentSharedRecordTypmodRegistry.typmod_index,
+				typmod_index_entry);
+
+	/* Push onto the front of list in atts_index_entry. */
+	serialized->next = atts_index_entry->serialized_tupdesc;
+	atts_index_entry->serialized_tupdesc = serialized_dp;
+
+	dht_release(CurrentSharedRecordTypmodRegistry.atts_index,
+				atts_index_entry);
+
+	return typmod;
+}
+
+/*
+ * DSM segment detach hook used to disconnect this backend's record typmod
+ * cache from the shared registry.  Detaching from the
+ * SharedRecordTypmodRegistry returns this backend to local typmod cache mode
+ * until such time as another parallel query runs.
+ */
+static void
+shared_record_typmod_registry_detach(dsm_segment *segment, Datum datum)
+{
+	SharedRecordTypmodRegistry *shared;
+
+	shared = (SharedRecordTypmodRegistry *) DatumGetPointer(datum);
+
+	/*
+	 * XXX Should we now copy all entries from shared memory into the
+	 * backend's local cache?  That depends on whether you think that there is
+	 * any chance this backend could see any shared tuples created by other
+	 * backends after this detach operation.  If tuples somehow survived from
+	 * query to query, that would be true.  But presently I don't think they
+	 * do, if we assume that all mechanisms that allow us to receive tuples
+	 * from other backends are linked to DSM segment mapping lifetime (tuple
+	 * queues, shared hash tables, shared temporary files).
+	 *
+	 * The only thing we need to synchronize to return to local-typmod-cache
+	 * mode is NextRecordTypmod.  That means that we can resume generating new
+	 * backend-local entries that don't clash.
+	 */
+	NextRecordTypmod = pg_atomic_read_u32(&shared->next_typmod);
+
+	/*
+	 * We don't free the SharedRecordTypmodRegistry's DSM memory, though we
+	 * could using a reference counting scheme if we wanted to.  There doesn't
+	 * seem to be any point because the whole DSA area will be going away
+	 * automatically when the DSM segment containing it is destroyed,
+	 * conceptually like a MemoryContext.
+	 */
+	CurrentSharedRecordTypmodRegistry.shared = NULL;
+	CurrentSharedRecordTypmodRegistry.atts_index = NULL;
+	CurrentSharedRecordTypmodRegistry.typmod_index = NULL;
+	CurrentSharedRecordTypmodRegistry.area = NULL;
+}
diff --git a/src/include/access/parallel.h b/src/include/access/parallel.h
index 5065a3830cf..a0c3b8eb955 100644
--- a/src/include/access/parallel.h
+++ b/src/include/access/parallel.h
@@ -19,6 +19,7 @@
 #include "postmaster/bgworker.h"
 #include "storage/shm_mq.h"
 #include "storage/shm_toc.h"
+#include "utils/dsa.h"
 
 typedef void (*parallel_worker_main_type) (dsm_segment *seg, shm_toc *toc);
 
@@ -41,6 +42,7 @@ typedef struct ParallelContext
 	ErrorContextCallback *error_context_stack;
 	shm_toc_estimator estimator;
 	dsm_segment *seg;
+	dsa_area   *context_dsa;
 	void	   *private_memory;
 	shm_toc    *toc;
 	ParallelWorkerInfo *worker;
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index b48f839028b..97a73b1483e 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -110,6 +110,9 @@ extern void DecrTupleDescRefCount(TupleDesc tupdesc);
 			DecrTupleDescRefCount(tupdesc); \
 	} while (0)
 
+extern bool equalTupleDescAttrs(Form_pg_attribute attr1,
+								Form_pg_attribute attr2);
+
 extern bool equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2);
 
 extern void TupleDescInitEntry(TupleDesc desc,
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index 0cd45bb6d8e..c0117f7bafd 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -212,7 +212,10 @@ typedef enum BuiltinTrancheIds
 	LWTRANCHE_LOCK_MANAGER,
 	LWTRANCHE_PREDICATE_LOCK_MANAGER,
 	LWTRANCHE_PARALLEL_QUERY_DSA,
+	LWTRANCHE_PARALLEL_CONTEXT_DSA,
 	LWTRANCHE_TBM,
+	LWTRANCHE_SHARED_RECORD_ATTS_INDEX,
+	LWTRANCHE_SHARED_RECORD_TYPMOD_INDEX,
 	LWTRANCHE_FIRST_USER_DEFINED
 }	BuiltinTrancheIds;
 
diff --git a/src/include/utils/typcache.h b/src/include/utils/typcache.h
index 1bf94e2548d..861610a2835 100644
--- a/src/include/utils/typcache.h
+++ b/src/include/utils/typcache.h
@@ -18,6 +18,8 @@
 
 #include "access/tupdesc.h"
 #include "fmgr.h"
+#include "storage/dsm.h"
+#include "utils/dsa.h"
 
 
 /* DomainConstraintCache is an opaque struct known only within typcache.c */
@@ -139,6 +141,7 @@ typedef struct DomainConstraintRef
 	MemoryContextCallback callback;		/* used to release refcount when done */
 } DomainConstraintRef;
 
+typedef struct SharedRecordTypmodRegistry SharedRecordTypmodRegistry;
 
 extern TypeCacheEntry *lookup_type_cache(Oid type_id, int flags);
 
@@ -160,4 +163,12 @@ extern void assign_record_type_typmod(TupleDesc tupDesc);
 
 extern int	compare_values_of_enum(TypeCacheEntry *tcache, Oid arg1, Oid arg2);
 
+extern size_t SharedRecordTypmodRegistryEstimate(void);
+extern void SharedRecordTypmodRegistryInit(SharedRecordTypmodRegistry *,
+										   dsa_area *area,
+										   dsm_segment *seg);
+extern void SharedRecordTypmodRegistryAttach(SharedRecordTypmodRegistry *,
+											 dsa_area *area,
+											 dsm_segment *seg);
+
 #endif   /* TYPCACHE_H */
#2Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Thomas Munro (#1)
1 attachment(s)
Re: POC: Sharing record typmods between backends

On Fri, Apr 7, 2017 at 5:21 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

* It would be nice for the SharedRecordTypeRegistry to be able to
survive longer than a single parallel query, perhaps in a per-session
DSM segment. Perhaps eventually we will want to consider a
query-scoped area, a transaction-scoped area and a session-scoped
area? I didn't investigate that for this POC.

This seems like the right way to go. I think there should be one
extra patch in this patch stack, to create a per-session DSA area (and
perhaps a "SharedSessionState" struct?) that worker backends can
attach to. It could be created when you first run a parallel query,
and then reused for all parallel queries for the rest of your session.
So, after you've run one parallel query, all future record typmod
registrations would get pushed (write-through style) into shmem, for
use by other backends that you might start in future parallel queries.
That will avoid having to copy the leader's registered record typmods
into shmem for every query going forward (the behaviour of the current
POC patch).

* Perhaps simplehash + an LWLock would be better than dht, but I
haven't looked into that. Can it be convinced to work in DSA memory
and to grow on demand?

Any views on this?

1. Apply dht-v3.patch[3].
2. Apply shared-record-typmod-registry-v1.patch.
3. Apply rip-out-tqueue-remapping-v1.patch.

Here's a rebased version of the second patch (the other two still
apply). It's still POC code only and still uses a
per-parallel-context DSA area for space, not the per-session one I am
now proposing we develop, if people are in favour of the approach.

In case it wasn't clear from my earlier description, a nice side
effect of using a shared typmod registry is that you can delete 85% of
tqueue.c (see patch #3), so if you don't count the hash table
implementation we come out about even in terms of lines of code.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

shared-record-typmod-registry-v2.patchapplication/octet-stream; name=shared-record-typmod-registry-v2.patchDownload
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 9fd7b4e019b..b72f5363045 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -343,11 +343,68 @@ DecrTupleDescRefCount(TupleDesc tupdesc)
 }
 
 /*
- * Compare two TupleDesc structures for logical equality
+ * Compare two TupleDesc attributes for logical equality
  *
  * Note: we deliberately do not check the attrelid and tdtypmod fields.
  * This allows typcache.c to use this routine to see if a cached record type
  * matches a requested type, and is harmless for relcache.c's uses.
+ */
+bool
+equalTupleDescAttrs(Form_pg_attribute attr1, Form_pg_attribute attr2)
+{
+	/*
+	 * We do not need to check every single field here: we can disregard
+	 * attrelid and attnum (which were used to place the row in the attrs
+	 * array in the first place).  It might look like we could dispense
+	 * with checking attlen/attbyval/attalign, since these are derived
+	 * from atttypid; but in the case of dropped columns we must check
+	 * them (since atttypid will be zero for all dropped columns) and in
+	 * general it seems safer to check them always.
+	 *
+	 * attcacheoff must NOT be checked since it's possibly not set in both
+	 * copies.
+	 */
+	if (strcmp(NameStr(attr1->attname), NameStr(attr2->attname)) != 0)
+		return false;
+	if (attr1->atttypid != attr2->atttypid)
+		return false;
+	if (attr1->attstattarget != attr2->attstattarget)
+		return false;
+	if (attr1->attlen != attr2->attlen)
+		return false;
+	if (attr1->attndims != attr2->attndims)
+		return false;
+	if (attr1->atttypmod != attr2->atttypmod)
+		return false;
+	if (attr1->attbyval != attr2->attbyval)
+		return false;
+	if (attr1->attstorage != attr2->attstorage)
+		return false;
+	if (attr1->attalign != attr2->attalign)
+		return false;
+	if (attr1->attnotnull != attr2->attnotnull)
+		return false;
+	if (attr1->atthasdef != attr2->atthasdef)
+		return false;
+	if (attr1->attidentity != attr2->attidentity)
+		return false;
+	if (attr1->attisdropped != attr2->attisdropped)
+		return false;
+	if (attr1->attislocal != attr2->attislocal)
+		return false;
+	if (attr1->attinhcount != attr2->attinhcount)
+		return false;
+	if (attr1->attcollation != attr2->attcollation)
+		return false;
+	/* attacl, attoptions and attfdwoptions are not even present... */
+
+	return true;
+}
+
+/*
+ * Compare two TupleDesc structures for logical equality
+ *
+ * Note: see equalTupleDescAttrs for the note on fields that we don't compare.
  * We don't compare tdrefcount, either.
  */
 bool
@@ -369,51 +426,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
 		Form_pg_attribute attr1 = tupdesc1->attrs[i];
 		Form_pg_attribute attr2 = tupdesc2->attrs[i];
 
-		/*
-		 * We do not need to check every single field here: we can disregard
-		 * attrelid and attnum (which were used to place the row in the attrs
-		 * array in the first place).  It might look like we could dispense
-		 * with checking attlen/attbyval/attalign, since these are derived
-		 * from atttypid; but in the case of dropped columns we must check
-		 * them (since atttypid will be zero for all dropped columns) and in
-		 * general it seems safer to check them always.
-		 *
-		 * attcacheoff must NOT be checked since it's possibly not set in both
-		 * copies.
-		 */
-		if (strcmp(NameStr(attr1->attname), NameStr(attr2->attname)) != 0)
-			return false;
-		if (attr1->atttypid != attr2->atttypid)
-			return false;
-		if (attr1->attstattarget != attr2->attstattarget)
-			return false;
-		if (attr1->attlen != attr2->attlen)
-			return false;
-		if (attr1->attndims != attr2->attndims)
-			return false;
-		if (attr1->atttypmod != attr2->atttypmod)
-			return false;
-		if (attr1->attbyval != attr2->attbyval)
-			return false;
-		if (attr1->attstorage != attr2->attstorage)
-			return false;
-		if (attr1->attalign != attr2->attalign)
-			return false;
-		if (attr1->attnotnull != attr2->attnotnull)
-			return false;
-		if (attr1->atthasdef != attr2->atthasdef)
-			return false;
-		if (attr1->attidentity != attr2->attidentity)
-			return false;
-		if (attr1->attisdropped != attr2->attisdropped)
-			return false;
-		if (attr1->attislocal != attr2->attislocal)
-			return false;
-		if (attr1->attinhcount != attr2->attinhcount)
-			return false;
-		if (attr1->attcollation != attr2->attcollation)
+		if (!equalTupleDescAttrs(attr1, attr2))
 			return false;
-		/* attacl, attoptions and attfdwoptions are not even present... */
 	}
 
 	if (tupdesc1->constr != NULL)
diff --git a/src/backend/access/transam/parallel.c b/src/backend/access/transam/parallel.c
index 2dad3e8a655..54acd989101 100644
--- a/src/backend/access/transam/parallel.c
+++ b/src/backend/access/transam/parallel.c
@@ -36,6 +36,7 @@
 #include "utils/memutils.h"
 #include "utils/resowner.h"
 #include "utils/snapmgr.h"
+#include "utils/typcache.h"
 
 
 /*
@@ -47,6 +48,14 @@
  */
 #define PARALLEL_ERROR_QUEUE_SIZE			16384
 
+/*
+ * We want to create a DSA area to store shared state that has the same extent
+ * as a parallel context, to hold the record type registry.  We don't want it
+ * to have to create any DSM segments just yet in common cases, so we'll give
+ * it enough space to hold an empty SharedRecordTypmodRegistry.
+ */
+#define PARALLEL_CONTEXT_DSA_SIZE			0x30000
+
 /* Magic number for parallel context TOC. */
 #define PARALLEL_MAGIC						0x50477c7c
 
@@ -63,6 +72,9 @@
 #define PARALLEL_KEY_ACTIVE_SNAPSHOT		UINT64CONST(0xFFFFFFFFFFFF0007)
 #define PARALLEL_KEY_TRANSACTION_STATE		UINT64CONST(0xFFFFFFFFFFFF0008)
 #define PARALLEL_KEY_ENTRYPOINT				UINT64CONST(0xFFFFFFFFFFFF0009)
+#define PARALLEL_KEY_EXTENSION_TRAMPOLINE	UINT64CONST(0xFFFFFFFFFFFF000A)
+#define PARALLEL_KEY_CONTEXT_DSA			UINT64CONST(0xFFFFFFFFFFFF000B)
+#define PARALLEL_KEY_RECORD_TYPMOD_REGISTRY	UINT64CONST(0xFFFFFFFFFFFF000C)
 
 /* Fixed-size parallel state. */
 typedef struct FixedParallelState
@@ -191,6 +203,7 @@ InitializeParallelDSM(ParallelContext *pcxt)
 	Size		library_len = 0;
 	Size		guc_len = 0;
 	Size		combocidlen = 0;
+	Size		typmod_registry_size = 0;
 	Size		tsnaplen = 0;
 	Size		asnaplen = 0;
 	Size		tstatelen = 0;
@@ -226,8 +239,11 @@ InitializeParallelDSM(ParallelContext *pcxt)
 		shm_toc_estimate_chunk(&pcxt->estimator, asnaplen);
 		tstatelen = EstimateTransactionStateSpace();
 		shm_toc_estimate_chunk(&pcxt->estimator, tstatelen);
+		shm_toc_estimate_chunk(&pcxt->estimator, PARALLEL_CONTEXT_DSA_SIZE);
+		typmod_registry_size = SharedRecordTypmodRegistryEstimate();
+		shm_toc_estimate_chunk(&pcxt->estimator, typmod_registry_size);
 		/* If you add more chunks here, you probably need to add keys. */
-		shm_toc_estimate_keys(&pcxt->estimator, 6);
+		shm_toc_estimate_keys(&pcxt->estimator, 8);
 
 		/* Estimate space need for error queues. */
 		StaticAssertStmt(BUFFERALIGN(PARALLEL_ERROR_QUEUE_SIZE) ==
@@ -295,6 +311,8 @@ InitializeParallelDSM(ParallelContext *pcxt)
 		char	   *asnapspace;
 		char	   *tstatespace;
 		char	   *error_queue_space;
+		char	   *typemod_registry_space;
+		char	   *context_dsa_space;
 		char	   *entrypointstate;
 		Size		lnamelen;
 
@@ -313,6 +331,27 @@ InitializeParallelDSM(ParallelContext *pcxt)
 		SerializeComboCIDState(combocidlen, combocidspace);
 		shm_toc_insert(pcxt->toc, PARALLEL_KEY_COMBO_CID, combocidspace);
 
+		/*
+		 * Make a DSA area for dynamically-sized shared state that has the
+		 * same scope as this ParallelContext.
+		 */
+		context_dsa_space = shm_toc_allocate(pcxt->toc,
+											 PARALLEL_CONTEXT_DSA_SIZE);
+		pcxt->context_dsa = dsa_create_in_place(context_dsa_space,
+												PARALLEL_CONTEXT_DSA_SIZE,
+												LWTRANCHE_PARALLEL_CONTEXT_DSA,
+												pcxt->seg);
+		shm_toc_insert(pcxt->toc, PARALLEL_KEY_CONTEXT_DSA, context_dsa_space);
+
+		/* Set up shared record type registry. */
+		typemod_registry_space = shm_toc_allocate(pcxt->toc,
+												  typmod_registry_size);
+		SharedRecordTypmodRegistryInit((SharedRecordTypmodRegistry *)
+									   typemod_registry_space,
+									   pcxt->context_dsa, pcxt->seg);
+		shm_toc_insert(pcxt->toc, PARALLEL_KEY_RECORD_TYPMOD_REGISTRY,
+					   typemod_registry_space);
+
 		/* Serialize transaction snapshot and active snapshot. */
 		tsnapspace = shm_toc_allocate(pcxt->toc, tsnaplen);
 		SerializeSnapshot(transaction_snapshot, tsnapspace);
@@ -618,6 +657,13 @@ DestroyParallelContext(ParallelContext *pcxt)
 		}
 	}
 
+	/* Detach from the context-scope DSA area, if there is one. */
+	if (pcxt->context_dsa != NULL)
+	{
+		dsa_detach(pcxt->context_dsa);
+		pcxt->context_dsa = NULL;
+	}
+
 	/*
 	 * If we have allocated a shared memory segment, detach it.  This will
 	 * implicitly detach the error queues, and any other shared memory queues,
@@ -938,6 +984,9 @@ ParallelWorkerMain(Datum main_arg)
 	char	   *asnapspace;
 	char	   *tstatespace;
 	StringInfoData msgbuf;
+	char	   *typmod_registry_space;
+	char	   *context_dsa_space;
+	dsa_area   *context_dsa;
 
 	/* Set flag to indicate that we're initializing a parallel worker. */
 	InitializingParallelWorker = true;
@@ -1069,6 +1118,20 @@ ParallelWorkerMain(Datum main_arg)
 	Assert(combocidspace != NULL);
 	RestoreComboCIDState(combocidspace);
 
+	/* Attach to the DSA area. */
+	context_dsa_space = shm_toc_lookup(toc, PARALLEL_KEY_CONTEXT_DSA);
+	Assert(context_dsa_space != NULL);
+	context_dsa = dsa_attach_in_place(context_dsa_space, seg);
+
+	/* Attach to shared record type registry. */
+	typmod_registry_space =
+		shm_toc_lookup(toc, PARALLEL_KEY_RECORD_TYPMOD_REGISTRY);
+	Assert(typmod_registry_space != NULL);
+	SharedRecordTypmodRegistryAttach((SharedRecordTypmodRegistry *)
+									 typmod_registry_space,
+									 context_dsa,
+									 seg);
+
 	/* Restore transaction snapshot. */
 	tsnapspace = shm_toc_lookup(toc, PARALLEL_KEY_TRANSACTION_SNAPSHOT);
 	Assert(tsnapspace != NULL);
@@ -1114,6 +1177,9 @@ ParallelWorkerMain(Datum main_arg)
 	/* Must pop active snapshot so resowner.c doesn't complain. */
 	PopActiveSnapshot();
 
+	/* Detach from context-scoped DSA area. */
+	dsa_detach(context_dsa);
+
 	/* Shut down the parallel-worker transaction. */
 	EndParallelWorkerTransaction();
 
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 35536e47894..0d7996b5205 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -494,7 +494,7 @@ RegisterLWLockTranches(void)
 
 	if (LWLockTrancheArray == NULL)
 	{
-		LWLockTranchesAllocated = 64;
+		LWLockTranchesAllocated = 128;
 		LWLockTrancheArray = (char **)
 			MemoryContextAllocZero(TopMemoryContext,
 								   LWLockTranchesAllocated * sizeof(char *));
@@ -510,7 +510,13 @@ RegisterLWLockTranches(void)
 						  "predicate_lock_manager");
 	LWLockRegisterTranche(LWTRANCHE_PARALLEL_QUERY_DSA,
 						  "parallel_query_dsa");
+	LWLockRegisterTranche(LWTRANCHE_PARALLEL_CONTEXT_DSA,
+						  "parallel_context_dsa");
 	LWLockRegisterTranche(LWTRANCHE_TBM, "tbm");
+	LWLockRegisterTranche(LWTRANCHE_SHARED_RECORD_ATTS_INDEX,
+						  "shared_record_atts_index");
+	LWLockRegisterTranche(LWTRANCHE_SHARED_RECORD_TYPMOD_INDEX,
+						  "shared_record_typmods_index");
 
 	/* Register named tranches. */
 	for (i = 0; i < NamedLWLockTrancheRequests; i++)
diff --git a/src/backend/utils/cache/typcache.c b/src/backend/utils/cache/typcache.c
index 0cf5001a758..f26b0a7a58f 100644
--- a/src/backend/utils/cache/typcache.c
+++ b/src/backend/utils/cache/typcache.c
@@ -55,7 +55,9 @@
 #include "catalog/pg_type.h"
 #include "commands/defrem.h"
 #include "executor/executor.h"
+#include "lib/dht.h"
 #include "optimizer/planner.h"
+#include "storage/lwlock.h"
 #include "utils/builtins.h"
 #include "utils/catcache.h"
 #include "utils/fmgroids.h"
@@ -67,7 +69,6 @@
 #include "utils/syscache.h"
 #include "utils/typcache.h"
 
-
 /* The main type cache hashtable searched by lookup_type_cache */
 static HTAB *TypeCacheHash = NULL;
 
@@ -148,12 +149,95 @@ typedef struct RecordCacheEntry
 	List	   *tupdescs;
 } RecordCacheEntry;
 
+/*
+ * A mechanism for sharing record typmods between backends.
+ */
+struct SharedRecordTypmodRegistry
+{
+	dht_hash_table_handle atts_index_handle;
+	dht_hash_table_handle typmod_index_handle;
+	pg_atomic_uint32 next_typmod;
+};
+
+/*
+ * A flattened/serialized representation of a TupleDesc for use in shared
+ * memory.  Can be converted to and from regular TupleDesc format.  Doesn't
+ * support constraints and doesn't store the actual type OID, because this is
+ * only for use with RECORD types as created by CreateTupleDesc().  These are
+ * arranged into a linked list, in the hash table entry corresponding to the
+ * OIDs of the first 16 attributes, so we'd expect to get more than one entry
+ * in the list when named and other properties differ.
+ */
+typedef struct SerializedTupleDesc
+{
+	dsa_pointer next;			/* next with the same same attribute OIDs */
+	int			natts;			/* number of attributes in the tuple */
+	int32		typmod;			/* typmod for tuple type */
+	bool		hasoid;			/* tuple has oid attribute in its header */
+
+	/*
+	 * The attributes follow.  We only ever access the first
+	 * ATTRIBUTE_FIXED_PART_SIZE bytes of each element, like the code in
+	 * tupdesc.c.
+	 */
+	FormData_pg_attribute attributes[FLEXIBLE_ARRAY_MEMBER];
+} SerializedTupleDesc;
+
+/*
+ * An entry in SharedRecordTypmodRegistry's attribute index.  The key is the
+ * first REC_HASH_KEYS attribute OIDs.  That means that collisions are
+ * possible, but that's OK because SerializedTupleDesc objects are arranged
+ * into a list.
+ */
+typedef struct SRTRAttsIndexEntry
+{
+	Oid			leading_attr_oids[REC_HASH_KEYS];
+	dsa_pointer serialized_tupdesc;
+} SRTRAttsIndexEntry;
+
+/*
+ * An entry in SharedRecordTypmodRegistry's typmod index.  Points to a single
+ * SerializedTupleDesc in shared memory.
+ */
+typedef struct SRTRTypmodIndexEntry
+{
+	uint32		typmod;
+	dsa_pointer serialized_tupdesc;
+} SRTRTypmodIndexEntry;
+
+/* Parameters for SharedRecordTypmodRegistry's attributes hash table. */
+const static dht_parameters srtr_atts_index_params = {
+	sizeof(Oid) * REC_HASH_KEYS,
+	sizeof(SRTRAttsIndexEntry),
+	memcmp,
+	tag_hash,
+	LWTRANCHE_SHARED_RECORD_ATTS_INDEX
+};
+
+/* Parameters for SharedRecordTypmodRegistry's typmod hash table. */
+const static dht_parameters srtr_typmod_index_params = {
+	sizeof(uint32),
+	sizeof(SRTRTypmodIndexEntry),
+	memcmp,
+	tag_hash,
+	LWTRANCHE_SHARED_RECORD_TYPMOD_INDEX
+};
+
 static HTAB *RecordCacheHash = NULL;
 
 static TupleDesc *RecordCacheArray = NULL;
 static int32 RecordCacheArrayLen = 0;	/* allocated length of array */
 static int32 NextRecordTypmod = 0;		/* number of entries used */
 
+/* Current SharedRecordTypmodRegistry, if attached. */
+static struct
+{
+	SharedRecordTypmodRegistry *shared;
+	dht_hash_table *atts_index;
+	dht_hash_table *typmod_index;
+	dsa_area *area;
+} CurrentSharedRecordTypmodRegistry;
+
 static void load_typcache_tupdesc(TypeCacheEntry *typentry);
 static void load_rangetype_info(TypeCacheEntry *typentry);
 static void load_domaintype_info(TypeCacheEntry *typentry);
@@ -174,6 +258,13 @@ static void TypeCacheConstrCallback(Datum arg, int cacheid, uint32 hashvalue);
 static void load_enum_cache_data(TypeCacheEntry *tcache);
 static EnumItem *find_enumitem(TypeCacheEnumData *enumdata, Oid arg);
 static int	enum_oid_cmp(const void *left, const void *right);
+static void shared_record_typmod_registry_detach(dsm_segment *segment,
+												 Datum datum);
+static int32 find_or_allocate_shared_record_typmod(TupleDesc tupdesc);
+static TupleDesc deserialize_tupledesc(const SerializedTupleDesc *serialized);
+static dsa_pointer serialize_tupledesc(dsa_area *area,
+									   const TupleDesc tupdesc);
+
 
 
 /*
@@ -1199,6 +1290,33 @@ cache_record_field_properties(TypeCacheEntry *typentry)
 	typentry->flags |= TCFLAGS_CHECKED_FIELD_PROPERTIES;
 }
 
+/*
+ * Make sure that RecordCacheArray is large enough to store 'typmod'.
+ */
+static void
+ensure_record_cache_typmod_slot_exists(int32 typmod)
+{
+	if (RecordCacheArray == NULL)
+	{
+		RecordCacheArray = (TupleDesc *)
+			MemoryContextAllocZero(CacheMemoryContext, 64 * sizeof(TupleDesc));
+		RecordCacheArrayLen = 64;
+	}
+
+	if (typmod >= RecordCacheArrayLen)
+	{
+		int32		newlen = RecordCacheArrayLen * 2;
+
+		while (typmod >= newlen)
+			newlen *= 2;
+
+		RecordCacheArray = (TupleDesc *) repalloc(RecordCacheArray,
+												  newlen * sizeof(TupleDesc));
+		memset(RecordCacheArray + RecordCacheArrayLen, 0,
+			   (newlen - RecordCacheArrayLen) * sizeof(TupleDesc *));
+		RecordCacheArrayLen = newlen;
+	}
+}
 
 /*
  * lookup_rowtype_tupdesc_internal --- internal routine to lookup a rowtype
@@ -1229,15 +1347,49 @@ lookup_rowtype_tupdesc_internal(Oid type_id, int32 typmod, bool noError)
 		/*
 		 * It's a transient record type, so look in our record-type table.
 		 */
-		if (typmod < 0 || typmod >= NextRecordTypmod)
+		if (typmod >= 0)
 		{
-			if (!noError)
-				ereport(ERROR,
-						(errcode(ERRCODE_WRONG_OBJECT_TYPE),
-						 errmsg("record type has not been registered")));
-			return NULL;
+			/* It is already in our local cache? */
+			if (typmod < RecordCacheArrayLen &&
+				RecordCacheArray[typmod] != NULL)
+				return RecordCacheArray[typmod];
+
+			/* Are we attached to a SharedRecordTypmodRegistry? */
+			if (CurrentSharedRecordTypmodRegistry.shared != NULL)
+			{
+				SRTRTypmodIndexEntry *entry;
+
+				/* Try to find it in the shared typmod index. */
+				entry = dht_find(CurrentSharedRecordTypmodRegistry.typmod_index,
+								 &typmod, false);
+				if (entry != NULL)
+				{
+					SerializedTupleDesc *serialized;
+
+					serialized = (SerializedTupleDesc *)
+						dsa_get_address(CurrentSharedRecordTypmodRegistry.area,
+										entry->serialized_tupdesc);
+					Assert(typmod == serialized->typmod);
+
+					/* We may need to extend the local RecordCacheArray. */
+					ensure_record_cache_typmod_slot_exists(typmod);
+
+					/* Produce and cache a TupleDesc. */
+					RecordCacheArray[typmod] =
+						deserialize_tupledesc(serialized);
+					dht_release(CurrentSharedRecordTypmodRegistry.typmod_index,
+								entry);
+
+					return RecordCacheArray[typmod];
+				}
+			}
 		}
-		return RecordCacheArray[typmod];
+
+		if (!noError)
+			ereport(ERROR,
+					(errcode(ERRCODE_WRONG_OBJECT_TYPE),
+					 errmsg("record type has not been registered")));
+		return NULL;
 	}
 }
 
@@ -1362,30 +1514,27 @@ assign_record_type_typmod(TupleDesc tupDesc)
 		}
 	}
 
+	/* Look in the SharedRecordTypmodRegistry, if attached */
+	newtypmod = find_or_allocate_shared_record_typmod(tupDesc);
+
 	/* Not present, so need to manufacture an entry */
 	oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
 
-	if (RecordCacheArray == NULL)
-	{
-		RecordCacheArray = (TupleDesc *) palloc(64 * sizeof(TupleDesc));
-		RecordCacheArrayLen = 64;
-	}
-	else if (NextRecordTypmod >= RecordCacheArrayLen)
-	{
-		int32		newlen = RecordCacheArrayLen * 2;
-
-		RecordCacheArray = (TupleDesc *) repalloc(RecordCacheArray,
-												  newlen * sizeof(TupleDesc));
-		RecordCacheArrayLen = newlen;
-	}
+	/*
+	 * Whether we just got a new typmod from a SharedRecordTypmodRegistry or
+	 * we're allocating one locally, make sure the RecordCacheArray is big
+	 * enough.
+	 */
+	ensure_record_cache_typmod_slot_exists(Max(NextRecordTypmod, newtypmod));
 
 	/* if fail in subrs, no damage except possibly some wasted memory... */
 	entDesc = CreateTupleDescCopy(tupDesc);
 	recentry->tupdescs = lcons(entDesc, recentry->tupdescs);
 	/* mark it as a reference-counted tupdesc */
 	entDesc->tdrefcount = 1;
-	/* now it's safe to advance NextRecordTypmod */
-	newtypmod = NextRecordTypmod++;
+	/* now it's safe to advance NextRecordTypmod, if allocating locally */
+	if (newtypmod == -1)
+		newtypmod = NextRecordTypmod++;
 	entDesc->tdtypmod = newtypmod;
 	RecordCacheArray[newtypmod] = entDesc;
 
@@ -1396,6 +1545,176 @@ assign_record_type_typmod(TupleDesc tupDesc)
 }
 
 /*
+ * Return the amout of shmem required to hold a SharedRecordTypmodRegistry.
+ * This exists only to avoid exposing private innards of
+ * SharedRecordTypmodRegistry in a header.
+ */
+size_t
+SharedRecordTypmodRegistryEstimate(void)
+{
+	return sizeof(SharedRecordTypmodRegistry);
+}
+
+/*
+ * Initialize 'registry' in a pre-existing shared memory region, which must be
+ * maximally aligned and have space for SharedRecordTypmodRegistryEstimate()
+ * bytes.
+ *
+ * 'area' will be used to allocate shared memory space as required for the
+ * typemod registration.  The current process, expected to be a leader process
+ * in a parallel query, will be attached automatically and its current record
+ * types will be loaded into the *registry.  While attached, all calls to
+ * assign_record_type_typmod will use the shared registry.  Other backends
+ * will need to attach explicitly.
+ *
+ * An on-detach callback will be installed for 'segment', so that normal
+ * private record type cache behavior can be restored when the DSM segment
+ * goes away.
+ */
+void
+SharedRecordTypmodRegistryInit(SharedRecordTypmodRegistry *registry,
+							 dsa_area *area,
+							 dsm_segment *segment)
+{
+	dht_hash_table *atts_index;
+	dht_hash_table *typmod_index;
+	int32 typmod;
+
+	/* We can't already be attached to a shared registry. */
+	Assert(CurrentSharedRecordTypmodRegistry.shared == NULL);
+	Assert(CurrentSharedRecordTypmodRegistry.atts_index == NULL);
+	Assert(CurrentSharedRecordTypmodRegistry.typmod_index == NULL);
+	Assert(CurrentSharedRecordTypmodRegistry.area == NULL);
+
+	/* Create the hash table indexed by attribute OIDs. */
+	atts_index = dht_create(area, &srtr_atts_index_params);
+	registry->atts_index_handle = dht_get_hash_table_handle(atts_index);
+
+	/* Create the hash table indexed by typmod. */
+	typmod_index = dht_create(area, &srtr_typmod_index_params);
+	registry->typmod_index_handle = dht_get_hash_table_handle(typmod_index);
+
+	/* Initialize the 'next typmode' to this backend's next value. */
+	pg_atomic_init_u32(&registry->next_typmod, NextRecordTypmod);
+
+	/*
+	 * Copy all entries from this backend's private registry into the shared
+	 * registry.
+	 */
+	for (typmod = 0; typmod < NextRecordTypmod; ++typmod)
+	{
+		SRTRTypmodIndexEntry *typmod_index_entry;
+		SRTRAttsIndexEntry *atts_index_entry;
+		SerializedTupleDesc *serialized;
+		dsa_pointer serialized_dp;
+		TupleDesc tupdesc;
+		Oid atts_key[REC_HASH_KEYS];
+		bool found;
+		int i;
+
+		tupdesc = RecordCacheArray[typmod];
+		if (tupdesc == NULL)
+			continue;
+
+		/* Serialize the TupleDesc into shared memory. */
+		serialized_dp = serialize_tupledesc(area, tupdesc);
+
+		/* Insert into the typmod index. */
+		typmod_index_entry = dht_find_or_insert(typmod_index,
+												&tupdesc->tdtypmod,
+												&found);
+		if (found)
+			elog(ERROR, "cannot create duplicate shared record typmod");
+		typmod_index_entry->typmod = tupdesc->tdtypmod;
+		typmod_index_entry->serialized_tupdesc = serialized_dp;
+		dht_release(typmod_index, typmod_index_entry);
+
+		/* Insert into the attributes index. */
+		memset(atts_key, 0, sizeof(atts_key));
+		for (i = 0; i < Min(tupdesc->natts, REC_HASH_KEYS); ++i)
+			atts_key[i] = tupdesc->attrs[i]->atttypid;
+		atts_index_entry = dht_find_or_insert(atts_index, &atts_key, &found);
+		if (!found)
+		{
+			memcpy(atts_index_entry->leading_attr_oids,
+				   atts_key,
+				   sizeof(atts_key));
+			atts_index_entry->serialized_tupdesc = InvalidDsaPointer;
+		}
+
+		/* Push onto list. */
+		serialized = (SerializedTupleDesc *)
+			dsa_get_address(area, serialized_dp);
+		serialized->next = atts_index_entry->serialized_tupdesc;
+		atts_index_entry->serialized_tupdesc = serialized_dp;
+		dht_release(atts_index, atts_index_entry);
+	}
+
+	/* Set up our detach hook so that we can return to private cache mode. */
+	on_dsm_detach(segment, shared_record_typmod_registry_detach,
+				  PointerGetDatum(registry));
+
+	/*
+	 * Set up the global state that will tell assign_record_type_typmod and
+	 * lookup_rowtype_tupdesc_internal about the shared registry.
+	 */
+	CurrentSharedRecordTypmodRegistry.shared = registry;
+	CurrentSharedRecordTypmodRegistry.atts_index = atts_index;
+	CurrentSharedRecordTypmodRegistry.typmod_index = typmod_index;
+	CurrentSharedRecordTypmodRegistry.area = area;
+}
+
+/*
+ * Attach to 'registry', which must have been initialized already by another
+ * backend.  Future calls to assign_record_type_typmod and
+ * lookup_rowtype_tupdesc_internal will use the shared registry, until
+ * 'segment' is detached.
+ */
+void
+SharedRecordTypmodRegistryAttach(SharedRecordTypmodRegistry *registry,
+							   dsa_area *area,
+							   dsm_segment *segment)
+{
+	dht_hash_table *atts_index;
+	dht_hash_table *typmod_index;
+
+	/* We can't already be attached to a shared registry. */
+	Assert(CurrentSharedRecordTypmodRegistry.shared == NULL);
+	Assert(CurrentSharedRecordTypmodRegistry.atts_index == NULL);
+	Assert(CurrentSharedRecordTypmodRegistry.typmod_index == NULL);
+	Assert(CurrentSharedRecordTypmodRegistry.area == NULL);
+
+	/*
+	 * We can't already have typmods in our local cache, because they'd clash
+	 * with those imported by SharedRecordTypmodRegistryInit.  This should be a
+	 * freshly started parallel worker.  If we ever support worker recycling,
+	 * a worker would need to zap its local cache in between servicing
+	 * different queries, in order to be able to call this and synchronize
+	 * typmods with a new leader.
+	 */
+	Assert(NextRecordTypmod == 0);
+
+	/* Attach to the two hash tables. */
+	atts_index = dht_attach(area, &srtr_atts_index_params,
+							registry->atts_index_handle);
+	typmod_index = dht_attach(area, &srtr_typmod_index_params,
+							  registry->typmod_index_handle);
+
+	/* Set up our detach hook so that we can return to private cache mode. */
+	on_dsm_detach(segment, shared_record_typmod_registry_detach,
+				  PointerGetDatum(registry));
+
+	/*
+	 * Set up the global state that will tell assign_record_type_typmod and
+	 * lookup_rowtype_tupdesc_internal about the shared registry.
+	 */
+	CurrentSharedRecordTypmodRegistry.shared = registry;
+	CurrentSharedRecordTypmodRegistry.atts_index = atts_index;
+	CurrentSharedRecordTypmodRegistry.typmod_index = typmod_index;
+	CurrentSharedRecordTypmodRegistry.area = area;
+}
+
+/*
  * TypeCacheRelCallback
  *		Relcache inval callback function
  *
@@ -1809,3 +2128,225 @@ enum_oid_cmp(const void *left, const void *right)
 	else
 		return 0;
 }
+
+/*
+ * Serialize a TupleDesc into a SerializedTupleDesc in DSA area 'area', and
+ * return a dsa_pointer.
+ */
+static dsa_pointer
+serialize_tupledesc(dsa_area *area, const TupleDesc tupdesc)
+{
+	SerializedTupleDesc *serialized;
+	dsa_pointer serialized_dp;
+	size_t size;
+	int i;
+
+	size = offsetof(SerializedTupleDesc, attributes) +
+		sizeof(FormData_pg_attribute) * tupdesc->natts;
+	serialized_dp = dsa_allocate(area, size);
+	serialized = (SerializedTupleDesc *) dsa_get_address(area, serialized_dp);
+
+	serialized->natts = tupdesc->natts;
+	serialized->typmod = tupdesc->tdtypmod;
+	serialized->hasoid = tupdesc->tdhasoid;
+	for (i = 0; i < tupdesc->natts; ++i)
+		memcpy(&serialized->attributes[i], tupdesc->attrs[i],
+			   ATTRIBUTE_FIXED_PART_SIZE);
+
+	return serialized_dp;
+}
+
+/*
+ * Deserialize a SerializedTupleDesc to produce a TupleDesc.  The result is
+ * allocated in CacheMemoryContext and has a refcount of 1.
+ */
+static TupleDesc
+deserialize_tupledesc(const SerializedTupleDesc *serialized)
+{
+	Form_pg_attribute *attributes;
+	MemoryContext oldctxt;
+	TupleDesc tupdesc;
+	int i;
+
+	/*
+	 * We have an array of FormData_pg_attribute but we need an array of
+	 * pointers to FormData_pg_attribute.
+	 */
+	oldctxt = MemoryContextSwitchTo(CacheMemoryContext);
+	attributes = palloc(sizeof(Form_pg_attribute) * serialized->natts);
+	for (i = 0; i < serialized->natts; ++i)
+	{
+		attributes[i] = palloc(ATTRIBUTE_FIXED_PART_SIZE);
+		memcpy(attributes[i], &serialized->attributes[i],
+			   ATTRIBUTE_FIXED_PART_SIZE);
+	}
+	tupdesc =
+		CreateTupleDesc(serialized->natts, serialized->hasoid, attributes);
+	tupdesc->tdtypmod = serialized->typmod;
+	tupdesc->tdrefcount = 1;
+	MemoryContextSwitchTo(oldctxt);
+
+	return tupdesc;
+}
+
+/*
+ * We can't use equalTupleDescs to compare a SerializedTupleDesc with a
+ * TupleDesc, but we don't want to allocate memory just to compare.  This
+ * function produces the same result without deserializing first.
+ */
+static bool
+serialized_tupledesc_matches(SerializedTupleDesc *serialized,
+							 TupleDesc tupdesc)
+{
+	int i;
+
+	if (serialized->natts != tupdesc->natts ||
+		serialized->hasoid != tupdesc->tdhasoid ||
+		tupdesc->constr != NULL)
+		return false;
+
+	for (i = 0; i < serialized->natts; ++i)
+	{
+		if (!equalTupleDescAttrs(&serialized->attributes[i],
+								 tupdesc->attrs[i]))
+			return false;
+	}
+
+	return true;
+}
+
+/*
+ * If we are attached to a SharedRecordTypmodRegistry, find or create a
+ * SerializedTupleDesc that matches 'tupdesc', and return its typmod.
+ * Otherwise return -1.
+ */
+static int32
+find_or_allocate_shared_record_typmod(TupleDesc tupdesc)
+{
+	SRTRAttsIndexEntry *atts_index_entry;
+	SRTRTypmodIndexEntry *typmod_index_entry;
+	SerializedTupleDesc *serialized;
+	dsa_pointer serialized_dp;
+	Oid			hashkey[REC_HASH_KEYS];
+	bool		found;
+	int32		typmod;
+	int			i;
+
+	/* If not even attached, nothing to do. */
+	if (CurrentSharedRecordTypmodRegistry.shared == NULL)
+		return -1;
+
+	/* Try to find a match. */
+	memset(hashkey, 0, sizeof(hashkey));
+	for (i = 0; i < tupdesc->natts; ++i)
+		hashkey[i] = tupdesc->attrs[i]->atttypid;
+	atts_index_entry = (SRTRAttsIndexEntry *)
+		dht_find_or_insert(CurrentSharedRecordTypmodRegistry.atts_index,
+						   hashkey,
+						   &found);
+	if (!found)
+	{
+		/* Making a new entry. */
+		memcpy(atts_index_entry->leading_attr_oids,
+			   hashkey,
+			   sizeof(hashkey));
+		atts_index_entry->serialized_tupdesc = InvalidDsaPointer;
+	}
+
+	/* Scan the list we found for a matching serialized one. */
+	serialized_dp = atts_index_entry->serialized_tupdesc;
+	while (DsaPointerIsValid(serialized_dp))
+	{
+		serialized =
+			dsa_get_address(CurrentSharedRecordTypmodRegistry.area,
+							serialized_dp);
+		if (serialized_tupledesc_matches(serialized, tupdesc))
+		{
+			/* Found a match, we are finished. */
+			typmod = serialized->typmod;
+			dht_release(CurrentSharedRecordTypmodRegistry.atts_index,
+						atts_index_entry);
+			return typmod;
+		}
+		serialized_dp = serialized->next;
+	}
+
+	/* We didn't find a matching entry, so let's allocate a new one. */
+	typmod = (int)
+		pg_atomic_fetch_add_u32(&CurrentSharedRecordTypmodRegistry.shared->next_typmod,
+								1);
+
+	/* Allocate shared memory and serialize the TupleDesc. */
+	serialized_dp = serialize_tupledesc(CurrentSharedRecordTypmodRegistry.area,
+										tupdesc);
+	serialized = (SerializedTupleDesc *)
+		dsa_get_address(CurrentSharedRecordTypmodRegistry.area, serialized_dp);
+	serialized->typmod = typmod;
+
+	/*
+	 * While we still hold the atts_index entry locked, add this to
+	 * typmod_index.  That's important because we don't want anyone to be able
+	 * to find a typmod via the former that can't yet be looked up in the
+	 * latter.
+	 */
+	typmod_index_entry =
+		dht_find_or_insert(CurrentSharedRecordTypmodRegistry.typmod_index,
+						   &typmod, &found);
+	if (found)
+		elog(ERROR, "cannot create duplicate shared record typmod");
+	typmod_index_entry->typmod = typmod;
+	typmod_index_entry->serialized_tupdesc = serialized_dp;
+	dht_release(CurrentSharedRecordTypmodRegistry.typmod_index,
+				typmod_index_entry);
+
+	/* Push onto the front of list in atts_index_entry. */
+	serialized->next = atts_index_entry->serialized_tupdesc;
+	atts_index_entry->serialized_tupdesc = serialized_dp;
+
+	dht_release(CurrentSharedRecordTypmodRegistry.atts_index,
+				atts_index_entry);
+
+	return typmod;
+}
+
+/*
+ * DSM segment detach hook used to disconnect this backend's record typmod
+ * cache from the shared registry.  Detaching from the
+ * SharedRecordTypmodRegistry returns this backend to local typmod cache mode
+ * until such time as another parallel query runs.
+ */
+static void
+shared_record_typmod_registry_detach(dsm_segment *segment, Datum datum)
+{
+	SharedRecordTypmodRegistry *shared;
+
+	shared = (SharedRecordTypmodRegistry *) DatumGetPointer(datum);
+
+	/*
+	 * XXX Should we now copy all entries from shared memory into the
+	 * backend's local cache?  That depends on whether you think that there is
+	 * any chance this backend could see any shared tuples created by other
+	 * backends after this detach operation.  If tuples somehow survived from
+	 * query to query, that would be true.  But presently I don't think they
+	 * do, if we assume that all mechanisms that allow us to receive tuples
+	 * from other backends are linked to DSM segment mapping lifetime (tuple
+	 * queues, shared hash tables, shared temporary files).
+	 *
+	 * The only thing we need to synchronize to return to local-typmod-cache
+	 * mode is NextRecordTypmod.  That means that we can resume generating new
+	 * backend-local entries that don't clash.
+	 */
+	NextRecordTypmod = pg_atomic_read_u32(&shared->next_typmod);
+
+	/*
+	 * We don't free the SharedRecordTypmodRegistry's DSM memory, though we
+	 * could using a reference counting scheme if we wanted to.  There doesn't
+	 * seem to be any point because the whole DSA area will be going away
+	 * automatically when the DSM segment containing it is destroyed,
+	 * conceptually like a MemoryContext.
+	 */
+	CurrentSharedRecordTypmodRegistry.shared = NULL;
+	CurrentSharedRecordTypmodRegistry.atts_index = NULL;
+	CurrentSharedRecordTypmodRegistry.typmod_index = NULL;
+	CurrentSharedRecordTypmodRegistry.area = NULL;
+}
diff --git a/src/include/access/parallel.h b/src/include/access/parallel.h
index 590e27a4845..b5781c28693 100644
--- a/src/include/access/parallel.h
+++ b/src/include/access/parallel.h
@@ -19,6 +19,7 @@
 #include "postmaster/bgworker.h"
 #include "storage/shm_mq.h"
 #include "storage/shm_toc.h"
+#include "utils/dsa.h"
 
 typedef void (*parallel_worker_main_type) (dsm_segment *seg, shm_toc *toc);
 
@@ -40,6 +41,7 @@ typedef struct ParallelContext
 	ErrorContextCallback *error_context_stack;
 	shm_toc_estimator estimator;
 	dsm_segment *seg;
+	dsa_area   *context_dsa;
 	void	   *private_memory;
 	shm_toc    *toc;
 	ParallelWorkerInfo *worker;
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index b48f839028b..97a73b1483e 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -110,6 +110,9 @@ extern void DecrTupleDescRefCount(TupleDesc tupdesc);
 			DecrTupleDescRefCount(tupdesc); \
 	} while (0)
 
+extern bool equalTupleDescAttrs(Form_pg_attribute attr1,
+								Form_pg_attribute attr2);
+
 extern bool equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2);
 
 extern void TupleDescInitEntry(TupleDesc desc,
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index 0cd45bb6d8e..c0117f7bafd 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -212,7 +212,10 @@ typedef enum BuiltinTrancheIds
 	LWTRANCHE_LOCK_MANAGER,
 	LWTRANCHE_PREDICATE_LOCK_MANAGER,
 	LWTRANCHE_PARALLEL_QUERY_DSA,
+	LWTRANCHE_PARALLEL_CONTEXT_DSA,
 	LWTRANCHE_TBM,
+	LWTRANCHE_SHARED_RECORD_ATTS_INDEX,
+	LWTRANCHE_SHARED_RECORD_TYPMOD_INDEX,
 	LWTRANCHE_FIRST_USER_DEFINED
 }	BuiltinTrancheIds;
 
diff --git a/src/include/utils/typcache.h b/src/include/utils/typcache.h
index 1bf94e2548d..861610a2835 100644
--- a/src/include/utils/typcache.h
+++ b/src/include/utils/typcache.h
@@ -18,6 +18,8 @@
 
 #include "access/tupdesc.h"
 #include "fmgr.h"
+#include "storage/dsm.h"
+#include "utils/dsa.h"
 
 
 /* DomainConstraintCache is an opaque struct known only within typcache.c */
@@ -139,6 +141,7 @@ typedef struct DomainConstraintRef
 	MemoryContextCallback callback;		/* used to release refcount when done */
 } DomainConstraintRef;
 
+typedef struct SharedRecordTypmodRegistry SharedRecordTypmodRegistry;
 
 extern TypeCacheEntry *lookup_type_cache(Oid type_id, int flags);
 
@@ -160,4 +163,12 @@ extern void assign_record_type_typmod(TupleDesc tupDesc);
 
 extern int	compare_values_of_enum(TypeCacheEntry *tcache, Oid arg1, Oid arg2);
 
+extern size_t SharedRecordTypmodRegistryEstimate(void);
+extern void SharedRecordTypmodRegistryInit(SharedRecordTypmodRegistry *,
+										   dsa_area *area,
+										   dsm_segment *seg);
+extern void SharedRecordTypmodRegistryAttach(SharedRecordTypmodRegistry *,
+											 dsa_area *area,
+											 dsm_segment *seg);
+
 #endif   /* TYPCACHE_H */
#3Dilip Kumar
dilipbalaut@gmail.com
In reply to: Thomas Munro (#2)
Re: POC: Sharing record typmods between backends

On Tue, May 30, 2017 at 1:09 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

* Perhaps simplehash + an LWLock would be better than dht, but I
haven't looked into that. Can it be convinced to work in DSA memory
and to grow on demand?

Simplehash provides an option to provide your own allocator function
to it. So in the allocator function, you can allocate memory from DSA.
After it reaches some threshold it expands the size (double) and it
will again call the allocator function to allocate the bigger memory.
You can refer pagetable_allocate in tidbitmap.c.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Robert Haas
robertmhaas@gmail.com
In reply to: Dilip Kumar (#3)
Re: POC: Sharing record typmods between backends

On Tue, May 30, 2017 at 2:45 AM, Dilip Kumar <dilipbalaut@gmail.com> wrote:

On Tue, May 30, 2017 at 1:09 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

* Perhaps simplehash + an LWLock would be better than dht, but I
haven't looked into that. Can it be convinced to work in DSA memory
and to grow on demand?

Simplehash provides an option to provide your own allocator function
to it. So in the allocator function, you can allocate memory from DSA.
After it reaches some threshold it expands the size (double) and it
will again call the allocator function to allocate the bigger memory.
You can refer pagetable_allocate in tidbitmap.c.

That only allows the pagetable to be shared, not the hash table itself.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#5Dilip Kumar
dilipbalaut@gmail.com
In reply to: Robert Haas (#4)
Re: POC: Sharing record typmods between backends

On Wed, May 31, 2017 at 10:57 AM, Robert Haas <robertmhaas@gmail.com> wrote:

Simplehash provides an option to provide your own allocator function
to it. So in the allocator function, you can allocate memory from DSA.
After it reaches some threshold it expands the size (double) and it
will again call the allocator function to allocate the bigger memory.
You can refer pagetable_allocate in tidbitmap.c.

That only allows the pagetable to be shared, not the hash table itself.

I agree with you. But, if I understand the use case correctly we need
to store the TupleDesc for the RECORD in shared hash so that it can be
shared across multiple processes. I think this can be achieved with
the simplehash as well.

For getting this done, we need some fixed shared memory for holding
static members of SH_TYPE and the process which creates the simplehash
will be responsible for copying these static members to the shared
location so that other processes can access the SH_TYPE. And, the
dynamic part (the actual hash entries) can be allocated using DSA by
registering SH_ALLOCATE function.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6Robert Haas
robertmhaas@gmail.com
In reply to: Dilip Kumar (#5)
Re: POC: Sharing record typmods between backends

On Wed, May 31, 2017 at 11:16 AM, Dilip Kumar <dilipbalaut@gmail.com> wrote:

I agree with you. But, if I understand the use case correctly we need
to store the TupleDesc for the RECORD in shared hash so that it can be
shared across multiple processes. I think this can be achieved with
the simplehash as well.

For getting this done, we need some fixed shared memory for holding
static members of SH_TYPE and the process which creates the simplehash
will be responsible for copying these static members to the shared
location so that other processes can access the SH_TYPE. And, the
dynamic part (the actual hash entries) can be allocated using DSA by
registering SH_ALLOCATE function.

Well, SH_TYPE's members SH_ELEMENT_TYPE *data and void *private_data
are not going to work in DSM, because they are pointers. You can
doubtless come up with a way around that problem, but I guess the
question is whether that's actually any better than just using DHT.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#7Dilip Kumar
dilipbalaut@gmail.com
In reply to: Robert Haas (#6)
Re: POC: Sharing record typmods between backends

On Wed, May 31, 2017 at 12:53 PM, Robert Haas <robertmhaas@gmail.com> wrote:

Well, SH_TYPE's members SH_ELEMENT_TYPE *data and void *private_data
are not going to work in DSM, because they are pointers. You can
doubtless come up with a way around that problem, but I guess the
question is whether that's actually any better than just using DHT.

Probably I misunderstood the question. I assumed that we need to bring
in DHT only for achieving this goal. But, if the question is simply
the comparison of DHT vs simplehash for this particular case then I
agree that DHT is a more appropriate choice.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#8Andres Freund
andres@anarazel.de
In reply to: Dilip Kumar (#7)
Re: POC: Sharing record typmods between backends

On 2017-05-31 13:27:28 -0400, Dilip Kumar wrote:

On Wed, May 31, 2017 at 12:53 PM, Robert Haas <robertmhaas@gmail.com> wrote:

Well, SH_TYPE's members SH_ELEMENT_TYPE *data and void *private_data
are not going to work in DSM, because they are pointers. You can
doubtless come up with a way around that problem, but I guess the
question is whether that's actually any better than just using DHT.

Probably I misunderstood the question. I assumed that we need to bring
in DHT only for achieving this goal. But, if the question is simply
the comparison of DHT vs simplehash for this particular case then I
agree that DHT is a more appropriate choice.

Yea, I don't think simplehash is the best choice here. It's worthwhile
to use it for performance critical bits, but using it for everything
would just increase code size without much benefit. I'd tentatively
assume that anonymous record type aren't going to be super common, and
that this is going to be the biggest bottleneck if you use them.

- Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#9Robert Haas
robertmhaas@gmail.com
In reply to: Andres Freund (#8)
Re: POC: Sharing record typmods between backends

On Wed, May 31, 2017 at 1:46 PM, Andres Freund <andres@anarazel.de> wrote:

On 2017-05-31 13:27:28 -0400, Dilip Kumar wrote:

On Wed, May 31, 2017 at 12:53 PM, Robert Haas <robertmhaas@gmail.com> wrote:

Well, SH_TYPE's members SH_ELEMENT_TYPE *data and void *private_data
are not going to work in DSM, because they are pointers. You can
doubtless come up with a way around that problem, but I guess the
question is whether that's actually any better than just using DHT.

Probably I misunderstood the question. I assumed that we need to bring
in DHT only for achieving this goal. But, if the question is simply
the comparison of DHT vs simplehash for this particular case then I
agree that DHT is a more appropriate choice.

Yea, I don't think simplehash is the best choice here. It's worthwhile
to use it for performance critical bits, but using it for everything
would just increase code size without much benefit. I'd tentatively
assume that anonymous record type aren't going to be super common, and
that this is going to be the biggest bottleneck if you use them.

Did you mean "not going to be"?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#10Andres Freund
andres@anarazel.de
In reply to: Robert Haas (#9)
Re: POC: Sharing record typmods between backends

On May 31, 2017 11:28:18 AM PDT, Robert Haas <robertmhaas@gmail.com> wrote:

On Wed, May 31, 2017 at 1:46 PM, Andres Freund <andres@anarazel.de>
wrote:

On 2017-05-31 13:27:28 -0400, Dilip Kumar wrote:

On Wed, May 31, 2017 at 12:53 PM, Robert Haas

<robertmhaas@gmail.com> wrote:

Well, SH_TYPE's members SH_ELEMENT_TYPE *data and void

*private_data

are not going to work in DSM, because they are pointers. You can
doubtless come up with a way around that problem, but I guess the
question is whether that's actually any better than just using

DHT.

Probably I misunderstood the question. I assumed that we need to

bring

in DHT only for achieving this goal. But, if the question is simply
the comparison of DHT vs simplehash for this particular case then I
agree that DHT is a more appropriate choice.

Yea, I don't think simplehash is the best choice here. It's

worthwhile

to use it for performance critical bits, but using it for everything
would just increase code size without much benefit. I'd tentatively
assume that anonymous record type aren't going to be super common,

and

that this is going to be the biggest bottleneck if you use them.

Did you mean "not going to be"?

Err, yes. Thanks
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#11Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#10)
1 attachment(s)
Re: POC: Sharing record typmods between backends

On Thu, Jun 1, 2017 at 6:29 AM, Andres Freund <andres@anarazel.de> wrote:

On May 31, 2017 11:28:18 AM PDT, Robert Haas <robertmhaas@gmail.com> wrote:

On 2017-05-31 13:27:28 -0400, Dilip Kumar wrote:

[ ... various discussion in support of using DHT ... ]

Ok, good.

Here's a new version that introduces a per-session DSM segment to hold
the shared record typmod registry (and maybe more things later). The
per-session segment is created the first time you run a parallel query
(though there is handling for failure to allocate that allows the
parallel query to continue with no workers) and lives until your
leader backend exits. When parallel workers start up, they see its
handle in the per-query segment and attach to it, which puts
typcache.c into write-through cache mode so their idea of record
typmods stays in sync with the leader (and each other).

I also noticed that I could delete even more of tqueue.c than before:
it doesn't seem to have any remaining reason to need to know the
TupleDesc.

One way to test this code is to apply just
0003-rip-out-tqueue-remapping-v3.patch and then try the example from
the first message in this thread to see it break, and then try again
with the other two patches applied. By adding debugging trace you can
see that the worker pushes a bunch of TupleDescs into shmem, they get
pulled out by the leader when it sees the tuples, and then on a second
invocation the (new) worker can reuse them: it finds matches already
in shmem from the first invocation.

I used a DSM segment with a TOC and a DSA area inside that, like the
existing per-query DSM segment, but obviously you could spin it
various different ways. One example: just have a DSA area and make a
new kind of TOC thing that deals in dsa_pointers. Better ideas?

I believe combo CIDs should also go in there, to enable parallel
write, but I'm not 100% sure: that's neither per-session nor per-query
data, that's per-transaction. So perhaps the per-session DSM could
hold a per-session DSA and a per-transaction DSA, where the latter is
reset for each transaction, just like TopTransactionContext (though
dsa.c doesn't have a 'reset thyself' function currently). That seems
like a good place to store a shared combo CID hash table using DHT.
Thoughts?

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

shared-record-typmod-registry-v3.patchset.tgzapplication/x-gzip; name=shared-record-typmod-registry-v3.patchset.tgzDownload
#12Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#11)
Re: POC: Sharing record typmods between backends

On 2017-07-10 21:39:09 +1200, Thomas Munro wrote:

Here's a new version that introduces a per-session DSM segment to hold
the shared record typmod registry (and maybe more things later).

You like to switch it up. *.patchset.tgz??? ;)

It does concern me that we're growing yet another somewhat different
hashtable implementation. Yet I don't quite see how we could avoid
it. dynahash relies on proper pointers, simplehash doesn't do locking
(and shouldn't) and also relies on pointers, although to a much lesser
degree. All the open coded tables aren't a good match either. So I
don't quite see an alternative, but I'd love one.

Regards,

Andres

diff --git a/src/backend/lib/dht.c b/src/backend/lib/dht.c
new file mode 100644
index 00000000000..2fec70f7742
--- /dev/null
+++ b/src/backend/lib/dht.c

FWIW, not a big fan of dht as a filename (nor of dsa.c). For one DHT
usually refers to distributed hash tables, which this is not, and for
another the abbreviation is so short it's not immediately
understandable, and likely to conflict further. I think it'd possibly
ok to have dht as symbol prefixes, but rename the file to be longer.

+ * To deal with currency, it has a fixed size set of partitions, each of which
+ * is independently locked.

s/currency/concurrency/ I presume.

+ * Each bucket maps to a partition; so insert, find
+ * and iterate operations normally only acquire one lock.  Therefore, good
+ * concurrency is achieved whenever they don't collide at the lock partition

s/they/operations/?

+ * level.  However, when a resize operation begins, all partition locks must
+ * be acquired simultaneously for a brief period.  This is only expected to
+ * happen a small number of times until a stable size is found, since growth is
+ * geometric.

I'm a bit doubtful that we need partitioning at this point, and that it
doesn't actually *degrade* performance for your typmod case.

+ * Resizing is done incrementally so that no individual insert operation pays
+ * for the potentially large cost of splitting all buckets.

I'm not sure this is a reasonable tradeoff for the use-case suggested so
far, it doesn't exactly make things simpler. We're not going to grow
much.

+/* The opaque type used for tracking iterator state. */
+struct dht_iterator;
+typedef struct dht_iterator dht_iterator;

Isn't it actually the iterator state? Rather than tracking it? Also, why
is it opaque given you're actually defining it below? Guess you'd moved
it at some point.

+/*
+ * The set of parameters needed to create or attach to a hash table.  The
+ * members tranche_id and tranche_name do not need to be initialized when
+ * attaching to an existing hash table.
+ */
+typedef struct
+{
+	Size key_size;				/* Size of the key (initial bytes of entry) */
+	Size entry_size;			/* Total size of entry */

Let's use size_t, like we kind of concluded in the thread you started:
http://archives.postgresql.org/message-id/25076.1489699457%40sss.pgh.pa.us
:)

+	dht_compare_function compare_function;	/* Compare function */
+	dht_hash_function hash_function;	/* Hash function */

Might be worth explaining that these need to provided when attaching
because they're possibly process local. Did you test this with
EXEC_BACKEND?

+	int tranche_id;				/* The tranche ID to use for locks. */
+} dht_parameters;
+struct dht_iterator
+{
+	dht_hash_table *hash_table;	/* The hash table we are iterating over. */
+	bool exclusive;				/* Whether to lock buckets exclusively. */
+	Size partition;				/* The index of the next partition to visit. */
+	Size bucket;				/* The index of the next bucket to visit. */
+	dht_hash_table_item *item;  /* The most recently returned item. */
+	dsa_pointer last_item_pointer; /* The last item visited. */
+	Size table_size_log2; 	/* The table size when we started iterating. */
+	bool locked;			/* Whether the current partition is locked. */

Haven't gotten to the actual code yet, but this kinda suggest we leave a
partition locked when iterating? Hm, that seems likely to result in a
fair bit of pain...

+/* Iterating over the whole hash table. */
+extern void dht_iterate_begin(dht_hash_table *hash_table,
+							  dht_iterator *iterator, bool exclusive);
+extern void *dht_iterate_next(dht_iterator *iterator);
+extern void dht_iterate_delete(dht_iterator *iterator);

s/delete/delete_current/? Otherwise it looks like it's part of
manipulating just the iterator.

+extern void dht_iterate_release(dht_iterator *iterator);

I'd add lock to to the name.

+/*
+ * An item in the hash table.  This wraps the user's entry object in an
+ * envelop that holds a pointer back to the bucket and a pointer to the next
+ * item in the bucket.
+ */
+struct dht_hash_table_item
+{
+	/* The hashed key, to avoid having to recompute it. */
+	dht_hash	hash;
+	/* The next item in the same bucket. */
+	dsa_pointer next;
+	/* The user's entry object follows here. */
+	char		entry[FLEXIBLE_ARRAY_MEMBER];

What's the point of using FLEXIBLE_ARRAY_MEMBER here? And isn't using a
char going to lead to alignment problems?

+/* The number of partitions for locking purposes. */
+#define DHT_NUM_PARTITIONS_LOG2 7

Could use some justification.

+/*
+ * The head object for a hash table.  This will be stored in dynamic shared
+ * memory.
+ */
+typedef struct
+{

Why anonymous? Not that it hurts much, but seems weird to deviate just
here.

+/*
+ * Create a new hash table backed by the given dynamic shared area, with the
+ * given parameters.
+ */
+dht_hash_table *
+dht_create(dsa_area *area, const dht_parameters *params)
+{
+	dht_hash_table *hash_table;
+	dsa_pointer control;
+
+	/* Allocate the backend-local object representing the hash table. */
+	hash_table = palloc(sizeof(dht_hash_table));

Should be documented that this uses caller's MemoryContext.

+	/* Set up the array of lock partitions. */
+	{
+		int			i;
+
+		for (i = 0; i < DHT_NUM_PARTITIONS; ++i)
+		{
+			LWLockInitialize(PARTITION_LOCK(hash_table, i),
+							 hash_table->control->lwlock_tranche_id);
+			hash_table->control->partitions[i].count = 0;
+		}

I'd store hash_table->control->lwlock_tranche_id and partitions[i] in
local vars. Possibly hash_table->control too.

+/*
+ * Detach from a hash table.  This frees backend-local resources associated
+ * with the hash table, but the hash table will continue to exist until it is
+ * either explicitly destroyed (by a backend that is still attached to it), or
+ * the area that backs it is returned to the operating system.
+ */
+void
+dht_detach(dht_hash_table *hash_table)
+{
+	/* The hash table may have been destroyed.  Just free local memory. */
+	pfree(hash_table);
+}

Somewhat inclined to add debugging refcount - seems like bugs around
that might be annoying to find. Maybe also add an assert ensuring that
no locks are held?

+/*
+ * Look up an entry, given a key.  Returns a pointer to an entry if one can be
+ * found with the given key.  Returns NULL if the key is not found.  If a
+ * non-NULL value is returned, the entry is locked and must be released by
+ * calling dht_release.  If an error is raised before dht_release is called,
+ * the lock will be released automatically, but the caller must take care to
+ * ensure that the entry is not left corrupted.  The lock mode is either
+ * shared or exclusive depending on 'exclusive'.

This API seems a bit fragile.

+/*
+ * Returns a pointer to an exclusively locked item which must be released with
+ * dht_release.  If the key is found in the hash table, 'found' is set to true
+ * and a pointer to the existing entry is returned.  If the key is not found,
+ * 'found' is set to false, and a pointer to a newly created entry is related.

"is related"?

+ */
+void *
+dht_find_or_insert(dht_hash_table *hash_table,
+				   const void *key,
+				   bool *found)
+{
+	size_t		hash;
+	size_t		partition_index;
+	dht_partition *partition;
+	dht_hash_table_item *item;
+
+	hash = hash_table->params.hash_function(key, hash_table->params.key_size);
+	partition_index = PARTITION_FOR_HASH(hash);
+	partition = &hash_table->control->partitions[partition_index];
+
+	Assert(hash_table->control->magic == DHT_MAGIC);
+	Assert(!hash_table->exclusively_locked);

Why just exclusively locked? Why'd shared be ok?

+/*
+ * Unlock an entry which was locked by dht_find or dht_find_or_insert.
+ */
+void
+dht_release(dht_hash_table *hash_table, void *entry)
+{
+	dht_hash_table_item *item = ITEM_FROM_ENTRY(entry);
+	size_t		partition_index = PARTITION_FOR_HASH(item->hash);
+	bool		deferred_resize_work = false;
+
+	Assert(hash_table->control->magic == DHT_MAGIC);

Assert lock held (LWLockHeldByMe())

+/*
+ * Begin iterating through the whole hash table.  The caller must supply a
+ * dht_iterator object, which can then be used to call dht_iterate_next to get
+ * values until the end is reached.
+ */
+void
+dht_iterate_begin(dht_hash_table *hash_table,
+				  dht_iterator *iterator,
+				  bool exclusive)
+{
+	Assert(hash_table->control->magic == DHT_MAGIC);
+
+	iterator->hash_table = hash_table;
+	iterator->exclusive = exclusive;
+	iterator->partition = 0;
+	iterator->bucket = 0;
+	iterator->item = NULL;
+	iterator->last_item_pointer = InvalidDsaPointer;
+	iterator->locked = false;
+
+	/* Snapshot the size (arbitrary lock to prevent size changing). */
+	LWLockAcquire(PARTITION_LOCK(hash_table, 0), LW_SHARED);
+	ensure_valid_bucket_pointers(hash_table);
+	iterator->table_size_log2 = hash_table->size_log2;
+	LWLockRelease(PARTITION_LOCK(hash_table, 0));

Hm. So we're introducing some additional contention on partition 0 -
probably ok.

+/*
+ * Move to the next item in the hash table.  Returns a pointer to an entry, or
+ * NULL if the end of the hash table has been reached.  The item is locked in
+ * exclusive or shared mode depending on the argument given to
+ * dht_iterate_begin.  The caller can optionally release the lock by calling
+ * dht_iterate_release, and then call dht_iterate_next again to move to the
+ * next entry.  If the iteration is in exclusive mode, client code can also
+ * call dht_iterate_delete.  When the end of the hash table is reached, or at
+ * any time, the client may call dht_iterate_end to abandon iteration.
+ */

I'd just shorten the end to "at any time the client may call
dht_iterate_end to ..."

+/*
+ * Release the most recently obtained lock.  This can optionally be called in
+ * between calls to dht_iterator_next to allow other processes to access the
+ * same partition of the hash table.
+ */
+void
+dht_iterate_release(dht_iterator *iterator)
+{
+	Assert(iterator->locked);
+	LWLockRelease(PARTITION_LOCK(iterator->hash_table, iterator->partition));
+	iterator->locked = false;
+}
+
+/*
+ * Terminate iteration.  This must be called after iteration completes,
+ * whether or not the end was reached.  The iterator object may then be reused
+ * for another iteration.
+ */
+void
+dht_iterate_end(dht_iterator *iterator)
+{
+	Assert(iterator->hash_table->control->magic == DHT_MAGIC);
+	if (iterator->locked)
+		LWLockRelease(PARTITION_LOCK(iterator->hash_table,
+									 iterator->partition));
+}
+
+/*
+ * Print out debugging information about the internal state of the hash table.
+ */
+void
+dht_dump(dht_hash_table *hash_table)
+{
+	size_t		i;
+	size_t		j;
+
+	Assert(hash_table->control->magic == DHT_MAGIC);
+
+	for (i = 0; i < DHT_NUM_PARTITIONS; ++i)
+		LWLockAcquire(PARTITION_LOCK(hash_table, i), LW_SHARED);

Should probably assert & document that no locks are held - otherwise
there's going to be ugly deadlocks. And that's an unlikely thing to try.

+	ensure_valid_bucket_pointers(hash_table);
+
+	fprintf(stderr,
+			"hash table size = %zu\n", (size_t) 1 << hash_table->size_log2);
+	for (i = 0; i < DHT_NUM_PARTITIONS; ++i)
+	{
+		dht_partition *partition = &hash_table->control->partitions[i];
+		size_t		begin = BUCKET_INDEX_FOR_PARTITION(i, hash_table->size_log2);
+		size_t		end = BUCKET_INDEX_FOR_PARTITION(i + 1, hash_table->size_log2);
+
+		fprintf(stderr, "  partition %zu\n", i);
+		fprintf(stderr,
+				"    active buckets (key count = %zu)\n", partition->count);
+
+		for (j = begin; j < end; ++j)
+		{
+			size_t		count = 0;
+			dsa_pointer bucket = hash_table->buckets[j];
+
+			while (DsaPointerIsValid(bucket))
+			{
+				dht_hash_table_item *item;
+
+				item = dsa_get_address(hash_table->area, bucket);
+
+				bucket = item->next;
+				++count;
+			}
+			fprintf(stderr, "      bucket %zu (key count = %zu)\n", j, count);
+		}
+		if (RESIZE_IN_PROGRESS(hash_table))
+		{
+			size_t		begin;
+			size_t		end;
+
+			begin = BUCKET_INDEX_FOR_PARTITION(i, hash_table->size_log2 - 1);
+			end = BUCKET_INDEX_FOR_PARTITION(i + 1,
+											 hash_table->size_log2 - 1);
+
+			fprintf(stderr, "    old buckets (key count = %zu)\n",
+					partition->old_count);
+
+			for (j = begin; j < end; ++j)
+			{
+				size_t		count = 0;
+				dsa_pointer bucket = hash_table->old_buckets[j];
+
+				while (DsaPointerIsValid(bucket))
+				{
+					dht_hash_table_item *item;
+
+					item = dsa_get_address(hash_table->area, bucket);
+
+					bucket = item->next;
+					++count;
+				}
+				fprintf(stderr,
+						"      bucket %zu (key count = %zu)\n", j, count);
+			}
+		}
+	}
+
+	for (i = 0; i < DHT_NUM_PARTITIONS; ++i)
+		LWLockRelease(PARTITION_LOCK(hash_table, i));
+}

I'd put this below actual "production" code.

- Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#13Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#11)
Re: POC: Sharing record typmods between backends

Hi,

diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 9fd7b4e019b..97c0125a4ba 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -337,17 +337,75 @@ DecrTupleDescRefCount(TupleDesc tupdesc)
 {
 	Assert(tupdesc->tdrefcount > 0);
-	ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc);
+	if (CurrentResourceOwner != NULL)
+		ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc);
 	if (--tupdesc->tdrefcount == 0)
 		FreeTupleDesc(tupdesc);
 }

What's this about? CurrentResourceOwner should always be valid here, no?
If so, why did that change? I don't think it's good to detach this from
the resowner infrastructure...

 /*
- * Compare two TupleDesc structures for logical equality
+ * Compare two TupleDescs' attributes for logical equality
  *
  * Note: we deliberately do not check the attrelid and tdtypmod fields.
  * This allows typcache.c to use this routine to see if a cached record type
  * matches a requested type, and is harmless for relcache.c's uses.
+ */
+bool
+equalTupleDescAttrs(Form_pg_attribute attr1, Form_pg_attribute attr2)
+{

comment not really accurate, this routine afaik isn't used by
typcache.c?

/*
- * Magic numbers for parallel state sharing.  Higher-level code should use
- * smaller values, leaving these very large ones for use by this module.
+ * Magic numbers for per-context parallel state sharing.  Higher-level code
+ * should use smaller values, leaving these very large ones for use by this
+ * module.
  */
 #define PARALLEL_KEY_FIXED					UINT64CONST(0xFFFFFFFFFFFF0001)
 #define PARALLEL_KEY_ERROR_QUEUE			UINT64CONST(0xFFFFFFFFFFFF0002)
@@ -63,6 +74,16 @@
 #define PARALLEL_KEY_ACTIVE_SNAPSHOT		UINT64CONST(0xFFFFFFFFFFFF0007)
 #define PARALLEL_KEY_TRANSACTION_STATE		UINT64CONST(0xFFFFFFFFFFFF0008)
 #define PARALLEL_KEY_ENTRYPOINT				UINT64CONST(0xFFFFFFFFFFFF0009)
+#define PARALLEL_KEY_SESSION_DSM			UINT64CONST(0xFFFFFFFFFFFF000A)
+
+/* Magic number for per-session DSM TOC. */
+#define PARALLEL_SESSION_MAGIC				0xabb0fbc9
+
+/*
+ * Magic numbers for parallel state sharing in the per-session DSM area.
+ */
+#define PARALLEL_KEY_SESSION_DSA			UINT64CONST(0xFFFFFFFFFFFF0001)
+#define PARALLEL_KEY_RECORD_TYPMOD_REGISTRY	UINT64CONST(0xFFFFFFFFFFFF0002)

Not this patch's fault, but this infrastructure really isn't great. We
should really replace it with a shmem.h style infrastructure, using a
dht hashtable as backing...

+/* The current per-session DSM segment, if attached. */
+static dsm_segment *current_session_segment = NULL;
+

I think it'd be better if we had a proper 'SessionState' and
'BackendSessionState' infrastructure that then contains the dsm segment
etc. I think we'll otherwise just end up with a bunch of parallel
infrastructures.

+/*
+ * A mechanism for sharing record typmods between backends.
+ */
+struct SharedRecordTypmodRegistry
+{
+	dht_hash_table_handle atts_index_handle;
+	dht_hash_table_handle typmod_index_handle;
+	pg_atomic_uint32 next_typmod;
+};
+

I think the code needs to explain better how these are intended to be
used. IIUC, atts_index is used to find typmods by "identity", and
typmod_index by the typmod, right? And we need both to avoid
all workers generating different tupledescs, right? Kinda guessable by
reading typecache.c, but that shouldn't be needed.

+/*
+ * A flattened/serialized representation of a TupleDesc for use in shared
+ * memory.  Can be converted to and from regular TupleDesc format.  Doesn't
+ * support constraints and doesn't store the actual type OID, because this is
+ * only for use with RECORD types as created by CreateTupleDesc().  These are
+ * arranged into a linked list, in the hash table entry corresponding to the
+ * OIDs of the first 16 attributes, so we'd expect to get more than one entry
+ * in the list when named and other properties differ.
+ */
+typedef struct SerializedTupleDesc
+{
+	dsa_pointer next;			/* next with the same same attribute OIDs */
+	int			natts;			/* number of attributes in the tuple */
+	int32		typmod;			/* typmod for tuple type */
+	bool		hasoid;			/* tuple has oid attribute in its header */
+
+	/*
+	 * The attributes follow.  We only ever access the first
+	 * ATTRIBUTE_FIXED_PART_SIZE bytes of each element, like the code in
+	 * tupdesc.c.
+	 */
+	FormData_pg_attribute attributes[FLEXIBLE_ARRAY_MEMBER];
+} SerializedTupleDesc;

Not a fan of a separate tupledesc representation, that's just going to
lead to divergence over time. I think we should rather change the normal
tupledesc representation to be compatible with this, and 'just' have a
wrapper struct for the parallel case (with next and such).

+/*
+ * An entry in SharedRecordTypmodRegistry's attribute index.  The key is the
+ * first REC_HASH_KEYS attribute OIDs.  That means that collisions are
+ * possible, but that's OK because SerializedTupleDesc objects are arranged
+ * into a list.
+ */
+/* Parameters for SharedRecordTypmodRegistry's attributes hash table. */
+const static dht_parameters srtr_atts_index_params = {
+	sizeof(Oid) * REC_HASH_KEYS,
+	sizeof(SRTRAttsIndexEntry),
+	memcmp,
+	tag_hash,
+	LWTRANCHE_SHARED_RECORD_ATTS_INDEX
+};
+
+/* Parameters for SharedRecordTypmodRegistry's typmod hash table. */
+const static dht_parameters srtr_typmod_index_params = {
+	sizeof(uint32),
+	sizeof(SRTRTypmodIndexEntry),
+	memcmp,
+	tag_hash,
+	LWTRANCHE_SHARED_RECORD_TYPMOD_INDEX
+};
+

I'm very much not a fan of this representation. I know you copied the
logic, but I think it's a bad idea. I think the key should just be a
dsa_pointer, and then we can have a proper tag_hash that hashes the
whole thing, and a proper comparator too. Just have

/*
* Combine two hash values, resulting in another hash value, with decent bit
* mixing.
*
* Similar to boost's hash_combine().
*/
static inline uint32
hash_combine(uint32 a, uint32 b)
{
a ^= b + 0x9e3779b9 + (a << 6) + (a >> 2);
return a;
}

and then hash everything.

+/*
+ * Make sure that RecordCacheArray is large enough to store 'typmod'.
+ */
+static void
+ensure_record_cache_typmod_slot_exists(int32 typmod)
+{
+	if (RecordCacheArray == NULL)
+	{
+		RecordCacheArray = (TupleDesc *)
+			MemoryContextAllocZero(CacheMemoryContext, 64 * sizeof(TupleDesc));
+		RecordCacheArrayLen = 64;
+	}
+
+	if (typmod >= RecordCacheArrayLen)
+	{
+		int32		newlen = RecordCacheArrayLen * 2;
+
+		while (typmod >= newlen)
+			newlen *= 2;
+
+		RecordCacheArray = (TupleDesc *) repalloc(RecordCacheArray,
+												  newlen * sizeof(TupleDesc));
+		memset(RecordCacheArray + RecordCacheArrayLen, 0,
+			   (newlen - RecordCacheArrayLen) * sizeof(TupleDesc *));
+		RecordCacheArrayLen = newlen;
+	}
+}

Do we really want to keep this? Could just have an equivalent dynahash
for the non-parallel case?

 /*
  * lookup_rowtype_tupdesc_internal --- internal routine to lookup a rowtype
@@ -1229,15 +1347,49 @@ lookup_rowtype_tupdesc_internal(Oid type_id, int32 typmod, bool noError)
 		/*
 		 * It's a transient record type, so look in our record-type table.
 		 */
-		if (typmod < 0 || typmod >= NextRecordTypmod)
+		if (typmod >= 0)
 		{
-			if (!noError)
-				ereport(ERROR,
-						(errcode(ERRCODE_WRONG_OBJECT_TYPE),
-						 errmsg("record type has not been registered")));
-			return NULL;
+			/* It is already in our local cache? */
+			if (typmod < RecordCacheArrayLen &&
+				RecordCacheArray[typmod] != NULL)
+				return RecordCacheArray[typmod];
+
+			/* Are we attached to a SharedRecordTypmodRegistry? */
+			if (CurrentSharedRecordTypmodRegistry.shared != NULL)

Why do we want to do lookups in both? I don't think it's a good idea to
have a chance that you could have the same typmod in both the local
registry (because it'd been created before the shared one) and in the
shared (because it was created in a worker). Ah, that's for caching
purposes? If so, see my above point that we shouldn't have a serialized
version of typdesc (yesyes, constraints will be a bit ugly).

+/*
+ * If we are attached to a SharedRecordTypmodRegistry, find or create a
+ * SerializedTupleDesc that matches 'tupdesc', and return its typmod.
+ * Otherwise return -1.
+ */
+static int32
+find_or_allocate_shared_record_typmod(TupleDesc tupdesc)
+{
+	SRTRAttsIndexEntry *atts_index_entry;
+	SRTRTypmodIndexEntry *typmod_index_entry;
+	SerializedTupleDesc *serialized;
+	dsa_pointer serialized_dp;
+	Oid			hashkey[REC_HASH_KEYS];
+	bool		found;
+	int32		typmod;
+	int			i;
+
+	/* If not even attached, nothing to do. */
+	if (CurrentSharedRecordTypmodRegistry.shared == NULL)
+		return -1;
+
+	/* Try to find a match. */
+	memset(hashkey, 0, sizeof(hashkey));
+	for (i = 0; i < tupdesc->natts; ++i)
+		hashkey[i] = tupdesc->attrs[i]->atttypid;
+	atts_index_entry = (SRTRAttsIndexEntry *)
+		dht_find_or_insert(CurrentSharedRecordTypmodRegistry.atts_index,
+						   hashkey,
+						   &found);
+	if (!found)
+	{
+		/* Making a new entry. */
+		memcpy(atts_index_entry->leading_attr_oids,
+			   hashkey,
+			   sizeof(hashkey));
+		atts_index_entry->serialized_tupdesc = InvalidDsaPointer;
+	}
+
+	/* Scan the list we found for a matching serialized one. */
+	serialized_dp = atts_index_entry->serialized_tupdesc;
+	while (DsaPointerIsValid(serialized_dp))
+	{
+		serialized =
+			dsa_get_address(CurrentSharedRecordTypmodRegistry.area,
+							serialized_dp);
+		if (serialized_tupledesc_matches(serialized, tupdesc))
+		{
+			/* Found a match, we are finished. */
+			typmod = serialized->typmod;
+			dht_release(CurrentSharedRecordTypmodRegistry.atts_index,
+						atts_index_entry);
+			return typmod;
+		}
+		serialized_dp = serialized->next;
+	}
+
+	/* We didn't find a matching entry, so let's allocate a new one. */
+	typmod = (int)
+		pg_atomic_fetch_add_u32(&CurrentSharedRecordTypmodRegistry.shared->next_typmod,
+								1);
+
+	/* Allocate shared memory and serialize the TupleDesc. */
+	serialized_dp = serialize_tupledesc(CurrentSharedRecordTypmodRegistry.area,
+										tupdesc);
+	serialized = (SerializedTupleDesc *)
+		dsa_get_address(CurrentSharedRecordTypmodRegistry.area, serialized_dp);
+	serialized->typmod = typmod;
+
+	/*
+	 * While we still hold the atts_index entry locked, add this to
+	 * typmod_index.  That's important because we don't want anyone to be able
+	 * to find a typmod via the former that can't yet be looked up in the
+	 * latter.
+	 */
+	typmod_index_entry =
+		dht_find_or_insert(CurrentSharedRecordTypmodRegistry.typmod_index,
+						   &typmod, &found);
+	if (found)
+		elog(ERROR, "cannot create duplicate shared record typmod");
+	typmod_index_entry->typmod = typmod;
+	typmod_index_entry->serialized_tupdesc = serialized_dp;
+	dht_release(CurrentSharedRecordTypmodRegistry.typmod_index,
+				typmod_index_entry);

What if we fail to allocate memory for the entry in typmod_index?

- Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#14Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#13)
1 attachment(s)
Re: POC: Sharing record typmods between backends

Hi,

Please find attached a new patch series. I apologise in advance for
0001 and note that the patchset now weighs in at ~75kB compressed.
Here are my in-line replies to your two reviews:

On Tue, Jul 25, 2017 at 10:09 PM, Andres Freund <andres@anarazel.de> wrote:

It does concern me that we're growing yet another somewhat different
hashtable implementation. Yet I don't quite see how we could avoid
it. dynahash relies on proper pointers, simplehash doesn't do locking
(and shouldn't) and also relies on pointers, although to a much lesser
degree. All the open coded tables aren't a good match either. So I
don't quite see an alternative, but I'd love one.

Yeah, I agree. To deal with data structures with different pointer
types, locking policy, inlined hash/eq functions etc, perhaps there is
a way we could eventually do 'policy based design' using the kind of
macro trickery you started where we generate N different hash table
variations but only have to maintain source code for one chaining hash
table implementation? Or perl scripts that effectively behave as a
cfront^H^H^H nevermind. I'm not planning to investigate that for this
cycle.

diff --git a/src/backend/lib/dht.c b/src/backend/lib/dht.c
new file mode 100644
index 00000000000..2fec70f7742
--- /dev/null
+++ b/src/backend/lib/dht.c

FWIW, not a big fan of dht as a filename (nor of dsa.c). For one DHT
usually refers to distributed hash tables, which this is not, and for
another the abbreviation is so short it's not immediately
understandable, and likely to conflict further. I think it'd possibly
ok to have dht as symbol prefixes, but rename the file to be longer.

OK. Now it's ds_hash_table.{c,h}, where "ds" stands for "dynamic
shared". Better? If we were to do other data structures in DSA
memory they could follow that style: ds_red_black_tree.c, ds_vector.c,
ds_deque.c etc and their identifier prefix would be drbt_, dv_, dd_
etc.

Do you want to see a separate patch to rename dsa.c? Got a better
name? You could have spoken up earlier :-) It does sound like a bit
like the thing from crypto or perhaps a scary secret government
department.

+ * To deal with currency, it has a fixed size set of partitions, each of which
+ * is independently locked.

s/currency/concurrency/ I presume.

Fixed.

+ * Each bucket maps to a partition; so insert, find
+ * and iterate operations normally only acquire one lock.  Therefore, good
+ * concurrency is achieved whenever they don't collide at the lock partition

s/they/operations/?

Fixed.

+ * level.  However, when a resize operation begins, all partition locks must
+ * be acquired simultaneously for a brief period.  This is only expected to
+ * happen a small number of times until a stable size is found, since growth is
+ * geometric.

I'm a bit doubtful that we need partitioning at this point, and that it
doesn't actually *degrade* performance for your typmod case.

Yeah, partitioning not needed for this case, but this is supposed to
be more generally useful. I thought about making the number of
partitions a construction parameter, but it doesn't really hurt does
it?

+ * Resizing is done incrementally so that no individual insert operation pays
+ * for the potentially large cost of splitting all buckets.

I'm not sure this is a reasonable tradeoff for the use-case suggested so
far, it doesn't exactly make things simpler. We're not going to grow
much.

Yeah, designed to be more generally useful. Are you saying you would
prefer to see the DHT patch split into an initial submission that does
the simplest thing possible, so that the unlucky guy who causes the
hash table to grow has to do all the work of moving buckets to a
bigger hash table? Then we could move the more complicated
incremental growth stuff to a later patch.

+/* The opaque type used for tracking iterator state. */
+struct dht_iterator;
+typedef struct dht_iterator dht_iterator;

Isn't it actually the iterator state? Rather than tracking it? Also, why
is it opaque given you're actually defining it below? Guess you'd moved
it at some point.

Improved comment. The iterator state is defined below in the .h, but
with a warning that client code mustn't access it; it exists in the
header only because it's very useful to be able to but dht_iterator on
the stack which requires the client code to have its definition, but I
want to reserve the right to change it arbitrarily in future.

+/*
+ * The set of parameters needed to create or attach to a hash table.  The
+ * members tranche_id and tranche_name do not need to be initialized when
+ * attaching to an existing hash table.
+ */
+typedef struct
+{
+       Size key_size;                          /* Size of the key (initial bytes of entry) */
+       Size entry_size;                        /* Total size of entry */

Let's use size_t, like we kind of concluded in the thread you started:
http://archives.postgresql.org/message-id/25076.1489699457%40sss.pgh.pa.us
:)

Sold.

+       dht_compare_function compare_function;  /* Compare function */
+       dht_hash_function hash_function;        /* Hash function */

Might be worth explaining that these need to provided when attaching
because they're possibly process local. Did you test this with
EXEC_BACKEND?

Added explanation. I haven't personally tested with EXEC_BACKEND but
I believe one of my colleagues had something else that uses DHT this
running on a Windows box and didn't shout at me, and I see no reason
to think it shouldn't work: as explained in the new comment, every
attacher has to supply the function pointers from their own process
space (and standard footgun rules apply if you don't supply compatible
functions).

+       int tranche_id;                         /* The tranche ID to use for locks. */
+} dht_parameters;
+struct dht_iterator
+{
+       dht_hash_table *hash_table;     /* The hash table we are iterating over. */
+       bool exclusive;                         /* Whether to lock buckets exclusively. */
+       Size partition;                         /* The index of the next partition to visit. */
+       Size bucket;                            /* The index of the next bucket to visit. */
+       dht_hash_table_item *item;  /* The most recently returned item. */
+       dsa_pointer last_item_pointer; /* The last item visited. */
+       Size table_size_log2;   /* The table size when we started iterating. */
+       bool locked;                    /* Whether the current partition is locked. */

Haven't gotten to the actual code yet, but this kinda suggest we leave a
partition locked when iterating? Hm, that seems likely to result in a
fair bit of pain...

By default yes, but you can release the lock with
dht_iterate_release_lock() and it'll be reacquired when you call
dht_iterate_next(). If you do that, then you'll continue iterating
after where you left off without visiting any item that you've already
visited, because the pointers are stored in pointer order (even though
the most recently visited item may have been freed rendering the
pointer invalid, we can still use its pointer to skip everything
already visited by numerical comparison without dereferencing it, and
it's indeterminate whether anything added while you were unlocked is
visible to you).

Maintaining linked lists in a certain order sucks, but DHT doesn't
allow duplicate keys and grows when load factor exceeds X so unless
your hash function is busted...

This is complicated, and in the category that I would normally want a
stack of heavy unit tests for. If you don't feel like making
decisions about this now, perhaps iteration (and incremental resize?)
could be removed, leaving only the most primitive get/put hash table
facilities -- just enough for this purpose? Then a later patch could
add them back, with a set of really convincing unit tests...

+/* Iterating over the whole hash table. */
+extern void dht_iterate_begin(dht_hash_table *hash_table,
+                                                         dht_iterator *iterator, bool exclusive);
+extern void *dht_iterate_next(dht_iterator *iterator);
+extern void dht_iterate_delete(dht_iterator *iterator);

s/delete/delete_current/? Otherwise it looks like it's part of
manipulating just the iterator.

Done.

+extern void dht_iterate_release(dht_iterator *iterator);

I'd add lock to to the name.

Done.

+/*
+ * An item in the hash table.  This wraps the user's entry object in an
+ * envelop that holds a pointer back to the bucket and a pointer to the next
+ * item in the bucket.
+ */
+struct dht_hash_table_item
+{
+       /* The hashed key, to avoid having to recompute it. */
+       dht_hash        hash;
+       /* The next item in the same bucket. */
+       dsa_pointer next;
+       /* The user's entry object follows here. */
+       char            entry[FLEXIBLE_ARRAY_MEMBER];

What's the point of using FLEXIBLE_ARRAY_MEMBER here? And isn't using a
char going to lead to alignment problems?

Fixed. No longer using a member 'entry', just a comment that user
data follows and a macro to find it based on
MAXALIGN(sizeof(dht_hash_table_item)).

+/* The number of partitions for locking purposes. */
+#define DHT_NUM_PARTITIONS_LOG2 7

Could use some justification.

Added. Short version: if it's good enough for the buffer pool...

+/*
+ * The head object for a hash table.  This will be stored in dynamic shared
+ * memory.
+ */
+typedef struct
+{

Why anonymous? Not that it hurts much, but seems weird to deviate just
here.

Fixed.

+/*
+ * Create a new hash table backed by the given dynamic shared area, with the
+ * given parameters.
+ */
+dht_hash_table *
+dht_create(dsa_area *area, const dht_parameters *params)
+{
+       dht_hash_table *hash_table;
+       dsa_pointer control;
+
+       /* Allocate the backend-local object representing the hash table. */
+       hash_table = palloc(sizeof(dht_hash_table));

Should be documented that this uses caller's MemoryContext.

Done.

+       /* Set up the array of lock partitions. */
+       {
+               int                     i;
+
+               for (i = 0; i < DHT_NUM_PARTITIONS; ++i)
+               {
+                       LWLockInitialize(PARTITION_LOCK(hash_table, i),
+                                                        hash_table->control->lwlock_tranche_id);
+                       hash_table->control->partitions[i].count = 0;
+               }

I'd store hash_table->control->lwlock_tranche_id and partitions[i] in
local vars. Possibly hash_table->control too.

Tidied up. I made local vars for partitions and tranche_id.

+/*
+ * Detach from a hash table.  This frees backend-local resources associated
+ * with the hash table, but the hash table will continue to exist until it is
+ * either explicitly destroyed (by a backend that is still attached to it), or
+ * the area that backs it is returned to the operating system.
+ */
+void
+dht_detach(dht_hash_table *hash_table)
+{
+       /* The hash table may have been destroyed.  Just free local memory. */
+       pfree(hash_table);
+}

Somewhat inclined to add debugging refcount - seems like bugs around
that might be annoying to find. Maybe also add an assert ensuring that
no locks are held?

Added assert that not locks are held.

In an earlier version I had reference counts. Then I realised that it
wasn't really helping anything. The state of being 'attached' to a
dht_hash_table isn't really the same as holding a heavyweight resource
like a DSM segment or a file which is backed by kernel resources.
'Attaching' is just something you have to do to get a backend-local
palloc()-ated object required to interact with the hash table, and
since it's just a bit of memory there is no strict requirement to
detach from it, if you're happy to let MemoryContext do that for you.
To put it in GC terms, there is no important finalizer here. Here I
am making the same distinction that we make between stuff managed by
resowner.c (files etc) and stuff managed by MemoryContext (memory); in
the former case it's an elog()-gable offence not to close things
explicitly in non-error paths, but in the latter you're free to do
that, or pfree earlier. If in future we create more things that can
live in DSA memory, I'd like them to be similarly free-and-easy. Make
sense?

In any case this use user of DHT remains attached for the backend's lifetime.

+/*
+ * Look up an entry, given a key.  Returns a pointer to an entry if one can be
+ * found with the given key.  Returns NULL if the key is not found.  If a
+ * non-NULL value is returned, the entry is locked and must be released by
+ * calling dht_release.  If an error is raised before dht_release is called,
+ * the lock will be released automatically, but the caller must take care to
+ * ensure that the entry is not left corrupted.  The lock mode is either
+ * shared or exclusive depending on 'exclusive'.

This API seems a bit fragile.

Do you mean "... the caller must take care to ensure that the entry is
not left corrupted"? This is the same as anything protected by
LWLocks including shared buffers. If you error out, locks are
released and you had better not have left things in a bad state. I
guess this comment is really just about what C++ people call "basic
exception safety".

Or something else?

+/*
+ * Returns a pointer to an exclusively locked item which must be released with
+ * dht_release.  If the key is found in the hash table, 'found' is set to true
+ * and a pointer to the existing entry is returned.  If the key is not found,
+ * 'found' is set to false, and a pointer to a newly created entry is related.

"is related"?

Fixed.

+ */
+void *
+dht_find_or_insert(dht_hash_table *hash_table,
+                                  const void *key,
+                                  bool *found)
+{
+       size_t          hash;
+       size_t          partition_index;
+       dht_partition *partition;
+       dht_hash_table_item *item;
+
+       hash = hash_table->params.hash_function(key, hash_table->params.key_size);
+       partition_index = PARTITION_FOR_HASH(hash);
+       partition = &hash_table->control->partitions[partition_index];
+
+       Assert(hash_table->control->magic == DHT_MAGIC);
+       Assert(!hash_table->exclusively_locked);

Why just exclusively locked? Why'd shared be ok?

It wouldn't be OK. I just didn't have the state required to assert
that. Fixed.

I think in future it should be allowed to lock more than one partition
(conceptually more than one entry) at a time, but only after figuring
out a decent API to support doing that in deadlock-avoiding order. I
don't have a need or a plan for that yet. For the same reason it's
not OK to use dht_find[_or_insert] while any iterator has locked a
partition, which wasn't documented (is now) and isn't currently
assertable.

+/*
+ * Unlock an entry which was locked by dht_find or dht_find_or_insert.
+ */
+void
+dht_release(dht_hash_table *hash_table, void *entry)
+{
+       dht_hash_table_item *item = ITEM_FROM_ENTRY(entry);
+       size_t          partition_index = PARTITION_FOR_HASH(item->hash);
+       bool            deferred_resize_work = false;
+
+       Assert(hash_table->control->magic == DHT_MAGIC);

Assert lock held (LWLockHeldByMe())

Added this, and a couple more.

+/*
+ * Begin iterating through the whole hash table.  The caller must supply a
+ * dht_iterator object, which can then be used to call dht_iterate_next to get
+ * values until the end is reached.
+ */
+void
+dht_iterate_begin(dht_hash_table *hash_table,
+                                 dht_iterator *iterator,
+                                 bool exclusive)
+{
+       Assert(hash_table->control->magic == DHT_MAGIC);
+
+       iterator->hash_table = hash_table;
+       iterator->exclusive = exclusive;
+       iterator->partition = 0;
+       iterator->bucket = 0;
+       iterator->item = NULL;
+       iterator->last_item_pointer = InvalidDsaPointer;
+       iterator->locked = false;
+
+       /* Snapshot the size (arbitrary lock to prevent size changing). */
+       LWLockAcquire(PARTITION_LOCK(hash_table, 0), LW_SHARED);
+       ensure_valid_bucket_pointers(hash_table);
+       iterator->table_size_log2 = hash_table->size_log2;
+       LWLockRelease(PARTITION_LOCK(hash_table, 0));

Hm. So we're introducing some additional contention on partition 0 -
probably ok.

It would be cute to use MyProcPid % DHT_NUM_PARTITIONS, but that might
be a deadlock hazard if you have multiple iterators on the go at once.
Otherwise iterators only ever lock partitions in order.

+/*
+ * Move to the next item in the hash table.  Returns a pointer to an entry, or
+ * NULL if the end of the hash table has been reached.  The item is locked in
+ * exclusive or shared mode depending on the argument given to
+ * dht_iterate_begin.  The caller can optionally release the lock by calling
+ * dht_iterate_release, and then call dht_iterate_next again to move to the
+ * next entry.  If the iteration is in exclusive mode, client code can also
+ * call dht_iterate_delete.  When the end of the hash table is reached, or at
+ * any time, the client may call dht_iterate_end to abandon iteration.
+ */

I'd just shorten the end to "at any time the client may call
dht_iterate_end to ..."

Done.

[snip]
+
+/*
+ * Print out debugging information about the internal state of the hash table.
+ */
+void
+dht_dump(dht_hash_table *hash_table)
+{
+       size_t          i;
+       size_t          j;
+
+       Assert(hash_table->control->magic == DHT_MAGIC);
+
+       for (i = 0; i < DHT_NUM_PARTITIONS; ++i)
+               LWLockAcquire(PARTITION_LOCK(hash_table, i), LW_SHARED);

Should probably assert & document that no locks are held - otherwise
there's going to be ugly deadlocks. And that's an unlikely thing to try.

OK.

[snip]
+}

I'd put this below actual "production" code.

Done.

On Tue, Aug 1, 2017 at 9:08 AM, Andres Freund <andres@anarazel.de> wrote:

Hi,

diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 9fd7b4e019b..97c0125a4ba 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -337,17 +337,75 @@ DecrTupleDescRefCount(TupleDesc tupdesc)
{
Assert(tupdesc->tdrefcount > 0);
-       ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc);
+       if (CurrentResourceOwner != NULL)
+               ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc);
if (--tupdesc->tdrefcount == 0)
FreeTupleDesc(tupdesc);
}

What's this about? CurrentResourceOwner should always be valid here, no?
If so, why did that change? I don't think it's good to detach this from
the resowner infrastructure...

The reason is that I install a detach hook
shared_record_typmod_registry_detach() in worker processes to clear
out their typmod registry. It runs at a time when there is no
CurrentResourceOwner. It's a theoretical concern only today, because
workers are not reused. If a workers lingered in a waiting room and
then attached to a new session DSM from a different leader, then it
needs to remember nothing of the previous leader's typmods.

/*
- * Compare two TupleDesc structures for logical equality
+ * Compare two TupleDescs' attributes for logical equality
*
* Note: we deliberately do not check the attrelid and tdtypmod fields.
* This allows typcache.c to use this routine to see if a cached record type
* matches a requested type, and is harmless for relcache.c's uses.
+ */
+bool
+equalTupleDescAttrs(Form_pg_attribute attr1, Form_pg_attribute attr2)
+{

comment not really accurate, this routine afaik isn't used by
typcache.c?

I removed this whole hunk and left equalTupleDescs() alone, because I
no longer needed to make that change in this new version. See below.

/*
- * Magic numbers for parallel state sharing.  Higher-level code should use
- * smaller values, leaving these very large ones for use by this module.
+ * Magic numbers for per-context parallel state sharing.  Higher-level code
+ * should use smaller values, leaving these very large ones for use by this
+ * module.
*/
#define PARALLEL_KEY_FIXED                                     UINT64CONST(0xFFFFFFFFFFFF0001)
#define PARALLEL_KEY_ERROR_QUEUE                       UINT64CONST(0xFFFFFFFFFFFF0002)
@@ -63,6 +74,16 @@
#define PARALLEL_KEY_ACTIVE_SNAPSHOT           UINT64CONST(0xFFFFFFFFFFFF0007)
#define PARALLEL_KEY_TRANSACTION_STATE         UINT64CONST(0xFFFFFFFFFFFF0008)
#define PARALLEL_KEY_ENTRYPOINT                                UINT64CONST(0xFFFFFFFFFFFF0009)
+#define PARALLEL_KEY_SESSION_DSM                       UINT64CONST(0xFFFFFFFFFFFF000A)
+
+/* Magic number for per-session DSM TOC. */
+#define PARALLEL_SESSION_MAGIC                         0xabb0fbc9
+
+/*
+ * Magic numbers for parallel state sharing in the per-session DSM area.
+ */
+#define PARALLEL_KEY_SESSION_DSA                       UINT64CONST(0xFFFFFFFFFFFF0001)
+#define PARALLEL_KEY_RECORD_TYPMOD_REGISTRY    UINT64CONST(0xFFFFFFFFFFFF0002)

Not this patch's fault, but this infrastructure really isn't great. We
should really replace it with a shmem.h style infrastructure, using a
dht hashtable as backing...

Well, I am trying to use the established programming style. We
already have a per-query DSM with a TOC indexed by magic numbers (and
executor node IDs). I add a per-session DSM with a TOC indexed by a
different set of magic numbers. We could always come up with
something better and fix it in both places later?

+/* The current per-session DSM segment, if attached. */
+static dsm_segment *current_session_segment = NULL;
+

I think it'd be better if we had a proper 'SessionState' and
'BackendSessionState' infrastructure that then contains the dsm segment
etc. I think we'll otherwise just end up with a bunch of parallel
infrastructures.

I'll have to come back to you on this one.

+/*
+ * A mechanism for sharing record typmods between backends.
+ */
+struct SharedRecordTypmodRegistry
+{
+       dht_hash_table_handle atts_index_handle;
+       dht_hash_table_handle typmod_index_handle;
+       pg_atomic_uint32 next_typmod;
+};
+

I think the code needs to explain better how these are intended to be
used. IIUC, atts_index is used to find typmods by "identity", and
typmod_index by the typmod, right? And we need both to avoid
all workers generating different tupledescs, right? Kinda guessable by
reading typecache.c, but that shouldn't be needed.

Fixed.

+/*
+ * A flattened/serialized representation of a TupleDesc for use in shared
+ * memory.  Can be converted to and from regular TupleDesc format.  Doesn't
+ * support constraints and doesn't store the actual type OID, because this is
+ * only for use with RECORD types as created by CreateTupleDesc().  These are
+ * arranged into a linked list, in the hash table entry corresponding to the
+ * OIDs of the first 16 attributes, so we'd expect to get more than one entry
+ * in the list when named and other properties differ.
+ */
+typedef struct SerializedTupleDesc
+{
+       dsa_pointer next;                       /* next with the same same attribute OIDs */
+       int                     natts;                  /* number of attributes in the tuple */
+       int32           typmod;                 /* typmod for tuple type */
+       bool            hasoid;                 /* tuple has oid attribute in its header */
+
+       /*
+        * The attributes follow.  We only ever access the first
+        * ATTRIBUTE_FIXED_PART_SIZE bytes of each element, like the code in
+        * tupdesc.c.
+        */
+       FormData_pg_attribute attributes[FLEXIBLE_ARRAY_MEMBER];
+} SerializedTupleDesc;

Not a fan of a separate tupledesc representation, that's just going to
lead to divergence over time. I think we should rather change the normal
tupledesc representation to be compatible with this, and 'just' have a
wrapper struct for the parallel case (with next and such).

OK. I killed this. Instead I flattened tupleDesc to make it usable
directly in shared memory as long as there are no constraints. There
is still a small wrapper SharedTupleDesc, but that's just to bolt a
'next' pointer to them so we can chain together TupleDescs with the
same OIDs.

The new 0001 patch changes tupdesc->attrs[i]->foo with
TupleDescAttr(tupdesc, i)->foo everywhere in the tree, so that the
change from attrs[i] to &attrs[i] can be hidden.

+/*
+ * An entry in SharedRecordTypmodRegistry's attribute index.  The key is the
+ * first REC_HASH_KEYS attribute OIDs.  That means that collisions are
+ * possible, but that's OK because SerializedTupleDesc objects are arranged
+ * into a list.
+ */
+/* Parameters for SharedRecordTypmodRegistry's attributes hash table. */
+const static dht_parameters srtr_atts_index_params = {
+       sizeof(Oid) * REC_HASH_KEYS,
+       sizeof(SRTRAttsIndexEntry),
+       memcmp,
+       tag_hash,
+       LWTRANCHE_SHARED_RECORD_ATTS_INDEX
+};
+
+/* Parameters for SharedRecordTypmodRegistry's typmod hash table. */
+const static dht_parameters srtr_typmod_index_params = {
+       sizeof(uint32),
+       sizeof(SRTRTypmodIndexEntry),
+       memcmp,
+       tag_hash,
+       LWTRANCHE_SHARED_RECORD_TYPMOD_INDEX
+};
+

I'm very much not a fan of this representation. I know you copied the
logic, but I think it's a bad idea. I think the key should just be a
dsa_pointer, and then we can have a proper tag_hash that hashes the
whole thing, and a proper comparator too. Just have

/*
* Combine two hash values, resulting in another hash value, with decent bit
* mixing.
*
* Similar to boost's hash_combine().
*/
static inline uint32
hash_combine(uint32 a, uint32 b)
{
a ^= b + 0x9e3779b9 + (a << 6) + (a >> 2);
return a;
}

and then hash everything.

Hmm. I'm not sure I understand. I know what hash_combine is for but
what do you mean when you say they key should just be a dsa_pointer?
What's wrong with providing the key size, whole entry size, compare
function and hash function like this?

+/*
+ * Make sure that RecordCacheArray is large enough to store 'typmod'.
+ */
+static void
+ensure_record_cache_typmod_slot_exists(int32 typmod)
+{
+       if (RecordCacheArray == NULL)
+       {
+               RecordCacheArray = (TupleDesc *)
+                       MemoryContextAllocZero(CacheMemoryContext, 64 * sizeof(TupleDesc));
+               RecordCacheArrayLen = 64;
+       }
+
+       if (typmod >= RecordCacheArrayLen)
+       {
+               int32           newlen = RecordCacheArrayLen * 2;
+
+               while (typmod >= newlen)
+                       newlen *= 2;
+
+               RecordCacheArray = (TupleDesc *) repalloc(RecordCacheArray,
+                                                                                                 newlen * sizeof(TupleDesc));
+               memset(RecordCacheArray + RecordCacheArrayLen, 0,
+                          (newlen - RecordCacheArrayLen) * sizeof(TupleDesc *));
+               RecordCacheArrayLen = newlen;
+       }
+}

Do we really want to keep this? Could just have an equivalent dynahash
for the non-parallel case?

Hmm. Well the plain old array makes a lot of sense in the
non-parallel case, since we allocate typmods starting from zero. What
don't you like about it? The reason for using an array for
backend-local lookup (aside from "that's how it is already") is that
it's actually the best data structure for the job; the reason for
using a hash table in the shared case is that it gives you locking and
coordinates growth for free. (For the OID index it has to be a hash
table in both cases.)

/*
* lookup_rowtype_tupdesc_internal --- internal routine to lookup a rowtype
@@ -1229,15 +1347,49 @@ lookup_rowtype_tupdesc_internal(Oid type_id, int32 typmod, bool noError)
/*
* It's a transient record type, so look in our record-type table.
*/
-               if (typmod < 0 || typmod >= NextRecordTypmod)
+               if (typmod >= 0)
{
-                       if (!noError)
-                               ereport(ERROR,
-                                               (errcode(ERRCODE_WRONG_OBJECT_TYPE),
-                                                errmsg("record type has not been registered")));
-                       return NULL;
+                       /* It is already in our local cache? */
+                       if (typmod < RecordCacheArrayLen &&
+                               RecordCacheArray[typmod] != NULL)
+                               return RecordCacheArray[typmod];
+
+                       /* Are we attached to a SharedRecordTypmodRegistry? */
+                       if (CurrentSharedRecordTypmodRegistry.shared != NULL)

Why do we want to do lookups in both? I don't think it's a good idea to
have a chance that you could have the same typmod in both the local
registry (because it'd been created before the shared one) and in the
shared (because it was created in a worker). Ah, that's for caching
purposes? If so, see my above point that we shouldn't have a serialized
version of typdesc (yesyes, constraints will be a bit ugly).

Right, that's what I've now done. It's basically a write-through
cache: we'll try to find it in the backend local structures and then
fall back to the shared one. But if we find it in shared memory,
we'll just copy the pointer it into our local data structures.

In the last version I'd build a new TupleDesc from the serialized
form, but now there is no serialized form, just TupleDesc objects
which are now shmem-friendly (except for constraints, which do not
survive the matter transfer into shmem; see TupleDescCopy).

+       /*
+        * While we still hold the atts_index entry locked, add this to
+        * typmod_index.  That's important because we don't want anyone to be able
+        * to find a typmod via the former that can't yet be looked up in the
+        * latter.
+        */
+       typmod_index_entry =
+               dht_find_or_insert(CurrentSharedRecordTypmodRegistry.typmod_index,
+                                                  &typmod, &found);
+       if (found)
+               elog(ERROR, "cannot create duplicate shared record typmod");
+       typmod_index_entry->typmod = typmod;
+       typmod_index_entry->serialized_tupdesc = serialized_dp;
+       dht_release(CurrentSharedRecordTypmodRegistry.typmod_index,
+                               typmod_index_entry);

What if we fail to allocate memory for the entry in typmod_index?

Well, I was careful to make sure that it was only pushed onto the list
in the atts_index entry until after we'd successfully allocated
entries in both indexes, so there was no way to exit from this
function leaving a TupleDesc in one index but not the other. In other
words, it might have created an atts_index entry but it'd have an
empty list. But yeah, on reflection we shouldn't leak shared_dp in
that case. In this version I added PG_CATCH() to dsa_free() it and
PG_RETHROW().

More testing and review needed.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

shared-record-typmods-v4.patchset.tgzapplication/x-gzip; name=shared-record-typmods-v4.patchset.tgzDownload
��j�Y��{{���8|��?��W��(��K�U$9��m��������b�`������s�]��FPv�������X������%�q#�F�#�0�[/C?>|�o�����E���>��v�=������*�s:�N��z�����F��9�����6q�F0��f����|^�\~�����o�?|M����]]��d�^_����$Q|�:L��+�y7��7��w��!��@3��K�?�w;����^�������5��/�p��|o������qg}�7���`��f#�����P���:\9�b�tFN��9����x��|�\��K7v^#(�/��E���X%"ZGA,�Y��/������^�M��m�a �3��7���u:0�����G�%�;{{tu���y:��abtRb|A����N:9696���G������D����� v��]g)<2��������YGb�Fn�7���-`���_�����s,d��F@��p�8�<�E�q�VI�/X]��\�ugq�^B=}���"~]i~~R0�$&�I��^���Ex�lbD�f��y��^��+���{����&���m;����T�7���s�h�G�������'A�m�`��s�$����O�a�\G-?]>=>:�����$>�\���w���_��_.��������w���_�W��'�����;���OK���5�0}pxx�����~�������<���9Nh���;b:�o������:F����L������v��k�Z�x
���F���f���[��[���{��{v����������k�K��������:x�K��^,�Q��m�����*��^��5�x+�_1��X������h�e����#����������<�OgQ���\�s���Q:F���`�t?�f[�'1@{�����-\	��B��QC����
V���	�c��!`�X%q������o�
�	/�}�#�oy_�F���f8}�����fQ��*�.{^���il��I���=;���k��?�M���~)��>�wY������$ta��r��
��_�=^�������Y&����_�9O�sw^���6�"�P}i�oAE�Q]	DY.e��O��v���?~�����?J7�<�tO�R9�������u�q�V|:B$@~q����w���_����u�����s���������sPH�i��uI���{�M��k��m@����E�Ez�l�yF�|vg����lu��M�,Wb��k�p���N
����B� �'��[���/�Hbx/=�d���IzC��K{��]���C��<��(�!lj��s�����9U}(�|
����
�O-�����y���-��%
�~��
������
l?Y�ny)��F��]��6bS�T�4!��Bx"x�a���v�g��>������K�$�d�/����U�B��_��q�q�M���m���H
���='���EU�b�;eO�L�PD^�{�a�,Qyj�3������*z'nT$WJ�R�aT�b+��(��OA�A�
E���t1��zf?����{��/��f>�P����a��\4��k�\_��d�I������Q�u=�`���Z=�)`Q >������<�1����s�����m�Yl)���'�P��H���8�S������[��
��Nw�5��[>�f!
�
�t�H�&An�����2�J�K=���B���]rP�R�	������������W����/�Yq�v����.����ly5r��
d2��B����N���uS��aY��
��zBG�Yw�+�;�qs[�H]�,_Y%��^�?-����e�
k/�|O,J{��������d����s5����7�$$��U��'.��m��(�'�n3."!SBDS"�����Z!����L�L��v�7�!�)X��K�x�`�����w�<qy�^��>-�����< ��w:�����x�Vk8���31��& ���Gp�x����v;�n���7A��9M�rWw��4�������$��2�����7��?D���	{J���������?FM�y����x7������3'p�pV8�<8����|p�����\���(��~������p��9��	�g�;{{{�`��y���_��[��x�_�V�g�yx_�xJ��������m���~����&L�4��*�z1������<3_��������CT�
fk�!����~�aJ�t�#$��N@�n�Kp��Z��^pI�.���<r���8�~)�S��t��F��������J��s����^/=
����GM������a f��1�������d���!�O{�)�cy��3.��;q�B3K�u7����I�;����_�J�/-_�d�=2X���1���?$��@q�a3I@G�?f;Qt8/��.4�K��� o{�{�&���&�O!;|��gB)k~yq��������r@+�~&��;�e��_�*�-C$P�
�oO����-*@�8�`�q�C���#�?����������������f��?\}��_D�S�q�z��h=V-�dv���I���)l���"��Y�S��u{������s������-&��hV���c�lE-q�^����T��vE��������������p^�|��F�4�X8�}2t��[[q&��P@��#�\��0���a��u:/w�P�{9�����VNi���"�tf���8���D~��y,�S���S{�x2F��nx7�{7�A����������M����pe��\\���`� ��H�y!�i�
���p���CnJ���:+��c�<H-(���w�Z/s������;[�9�M�Y���c�Z�9�-�O�@�O:�]��c���B[*�
���A��2�uk�����l���qV�P2%����o2p�5�M�]��z�b�T0���
���h��W�%[��p9e������/�}s|uv�fzt�����)���V%��g��A������7q�'�� ����%�/��=|!������������	>a��O��$�:�Zz�����^��W�����s]z	o�����Y�m�-��jH����b=���z�w���<�AX-�K
	5����vQ@�W��"v��/�~�GH����i��M�ST?���tF��
��?�o���0��������`�U��0�Z�fO�B�5��^�"k��^�N����6�y�X��A�M��#��'�����������&��E���c���[7�s�}'V��^���&qn��B.�1��������T�G�&�v�Zo���S�����X+�/@<��C]�������VH�!�g,
�[���e���7���f�O���m*���u'P�H��7
%m�%0�\,3�z��������wF���w��HL������b��l,[�*k�8�u��b��
6� ���E���d�.R�N�B�i���Y.�E�S��hv�WI�����@
��k��pU&�*D1_��A�FB��t�w�c��������.�:�<~���4��m#h:	��Il0�C�H�/�������M��sXJ�=X���p����Ep���D�7�~����a�\�r�j�6�2���
���!����3�x�r2a�Z%K��
���
j]W�����X�3��L���0�CK�|��b����]�]5�
����xh:0��t����><�Nn��d����`4N��L�2�4���Q��J����N�3��~����nw����c:R9[L��������M�s��"�`#���V_�����v�k$M�E���N��EcM��B�=��J�B���2�"
����[,���mC�F>su�����G+���~I/)��@��?�6��yK1B�Cg[(��;���v��i���no0���N���CepZ����N����n��W���V�Fj�v�0��y�!"i��C����h����}l�rU*�Z�VSj���D�18yMAt�N��?-� r�3����I$-�>}G\�q���J;k����	hM$�&�@����)!�ml�{;d+
#%��dlV��'�j�}���G-+LO���HW�f�F�sy 7��M�@
��nR�Pr0R�\���V������uGm�N3�v���p���4��*�M�%��`�<�����Fo�J�c]xg%"65Ga�45���@�k�5�BjX�M���_`�����2���I+��Q�jO���`�3�������R\�1D�	f=x�Lb����|I�A��7k�m4l@�8X�J��.<6Q��K-X �W���B�p�B��NW�u5hOH��W<��SEa4�@_G����D�'%�����$����I�/p�[U��MDN6!O������}��;9��t~$t��SB���e9"�6N����u)��D,���}���f���H����<���Tc�sr�>�s�2�\��Y�#�>��I{ �m����
��x<����}��yf�1iV]Z�|A�J7(����0[�����0)^A���gSX�������0�[�D�.�}6��<�T�c���P�2�.C��Ll3^Q��y#�i��4������"�'��
���?x����&���&��|�����83��������wF���3k�:�����O�^��[<�M>�mhs���IS�0L�C�[�dv�Q�"���A+�!�Ng���y�h�.�k ��9��kDk���,�%�����]�S��v���x��a�v��7GD�u;����������x�/�������W}{:}}v�S.������u�Q�"����l������i�^��v�gJ��b��#}Pq��&�����Mgo�����f���������^�8����l���/C��>�&�W:���DC}iM\��M���@�^ �a`$,�5��p��`���T����\��^vI���_q#>p����8��u�-���(lc��W/���J�����O���.�"�'N�Y�	��Lt{�;�$��mF����"�#�g%��������y�`5��o
��N�������k7��96�2cz��m�B��su#@���E�{����a��;z�zZ���a�[���b+���l����qfFc�e=�w,��/�&�[�O�k��A�[l:�����E��d�4^�����Z/���~c��R_�����F�����a���s����VW<�.ZSL���1���OJ�E��m���/����M���u0��~������":;��l�+�s�������{Ed{��":�":��1�6�� ����(�cPE?f9���0�;8���>��KC�������nG����,?���|Jl���ZoA�Tb�kX�`/u�J�6v`Uc����-�Jv6���K�D����`�!oB�P��/8��)��xS")*&>��c4���]]u��@�%�Ef�[�tr�Y���TM\o6���V�;���no8�8^��3�)}��L����hr}�Ik���a
�&���*���P�>�!���l_����ki;��S�J�5f�n���E��*����I����-��M�-<k:�i�I���t>{��v��2b���.���=O��LC��	p
D���cO_����`%�~���g��I�s;.�Q����Z��`���������*G��de[v~k��[��kJ�iL�F�:�X;6������]�mGB���w�z�@<q��ro/����_���v8���Pi�V��PH������H��	�_��%�v9gc�$C{��L=�g��7�����
&pb�R����mi�|��b&f����s<�z��8��e(&�-�(�oD��|��V�,H���q�3���������/���$�-�R����E�n:_�������fz&�4Yd�b�n0����kw��y�gw����|����nBn��c���G1Z Ii��d��%���Y��;g����8�2R�4S��V�6x�������!�g��q�<�n:�y��E"�R4�cZ�sB���T�-� p|)��Q8�eW����H.��L���1�����w���R68��|OKqd�����cicM4�H����d���h
�����
�#���7�������~
�)������|����"zW!C6��N�k�]=x��L`Frn
��F��_�����
��T�������`RI��P��%�;�w������������u_�����N����=o�n����'��{����t;c��KB��T��c���"�N�6��{�2�P��y>�er�*]L)�@��o�'�iJ�d�L������m��5W�V>k�����F�n���2���GT4A.���X����W�^��~w~v��S4�4u��0:�)���F�y��[��H
���0�Hr�&�(i��]S�}�O�P�����d1	Vh���b@I^�-h�}2�In�Et�<������f��+�
Gb0���`qg�K*�?U�-���UM���@�@o�K�F���]|��!�����o/�.��N��kS��h��&�ox��e�������>�u7���N��aw�O��;�V�W�7u����K�X�y��O���m�O~���i��T�ku��d��;�"�O�_������?�_%����!�)�����N{�����-t>��{�����RMI�`g�����h��3�x2��Ioa���Q���B��oK"�U��l	6����U=�h��w~J`<V�+o���99��Z��?-�n�|A��B���I��}�����A�RB�c��E��X����� ���4�yO)K1&�����v���O8��lo��q����"�?W�){>?J��V6{!����qi4&*`��,��c��Y���Y��n��\��y�u��c��F�mP;�������r������/_�"�
0��^���>?>z3}}���������FV�������f��:~��b!u��2����5��s_��o/O�G�����_�s��+x��@��B�/�t�|�AG�~���]	�,pl+r��K�@9�9��F�R��iP�X���&��Tl�!��H@I�k�;�9����Z�T�S+abv�����S�w#�������!�}2��@op8�~A�0��:�Ul�!�&`��D/��
S���cv�KSd7�]b�i��������A���@��q�:�����y����{�]e��\@1����VIe*Uei�Ki��������B����<��������Z�Tf��1
qd[+ ��|�|�y'0/���7a���m�)>I�U�6F�1Nzn��0{2�����L���s�e'_#��5���h+%�����i�����}��f{�	���!�wAt�D�Q
�o�(��
"
H�������>hv���.`L_:�t�C���p<yv�����S_	�"xw����j���y��G�N��0�`�����}����������������SD���<un�6*j��`%�k;9�	]S���!���b9�+�m����<� �C9��Eb."�I�>�����
sH�{[����bn�������t�
�f
�e���nV��U#�t��s%i�H�,/�&��Rg�~&��:�����(���M�V��S��|=��>3zSN>��lhf2��X_\��S���jdN�����y5�T����S��~�+�5n��{�/��q�$����9�F��`M�ga����9���b�����w��`/y
�U��z	���������r��N�h�_��~P_�������F��$�j���RV���������g��U	�����H���O*��}��Bq�p�
b�������������N���t�"`�+=�PM.a�s��i�h�vH!����}�?�8L����L�,��\�x����@e�����~��*c������d/�������7(I�srl����H��M�n)��V6B�Hr�(�$XS��SA;C�d.9��J�N�v�������*q��T������PZ���E�}n������M_����e/���R%���o��=b	�C�&�9�����L�:d�������p(�c����&�$/I�~���w����B5��A���s����������>�����s��L��>K�x�����N��H9&�K��G �H34S�����k��O������V�������t���H���iZb-�E��!H�.�8��������O�/�g"�T
r��'��9�fk[a���>1M��p5'���
��b��i>q���0�N
?$�]��e��UF@OHr��l7	#�����L��A��e;v��<�m3���*_a������5��$N����&8%��-2�-�NG�g��#|�P�@�!�z�"g�R��R�������"����~$����j���Z��A������Q	(�Y�37�Cn����������.>��4����`�4�����R�F8��h�	���s�'6�����Z������E�7�m�&�*t�l�����DH��Z���fRDg�J�h��������E�����9ueX^�.dY������6B�a�?�����1����)_`��4���������^��1k�����rgS�%x>e��LL���rU���	+�����733��>��;���>�{����NS��P�%=t7��$�b�}����4�l!����N��	�}y�	ZPb����i����[p5*��JR
�q�eh��F@KI,����Cv�:�,Vld"���"\]��d(%m�R-�CN
������T���"��������!��B��;��tNopc����
�d!VV�M����E����j��6���8���v�k�����Mm�8cSF������IS��>�������	�����,�	JJ���MG�=|���UMo�����c6��8��8��v�q��Q��i����8���r��w�a'�mO�k�f�Q�N����'���.��goNN�2�����]��r�������B�Qq��N`6+�65k���f�y$�U3�}'�&a{b�)�_�^�^�wG%�;������~�sJ����� tG:A����	�6��~���9�aw�E�U������?M�ut�����������3��3���~?S`���)�@}��E~'��L����az�XNv��~?^�5�z�/�F�x��T��!��!Co<�C�Hp�!C
<�~���Q�o���������r���j)�D�x���d.����'&nw8k�j��1�Y����X�wR�H�O)<f�n�����z��b1g�#(���'2L�U�t��]�/����Z����1������Oq-sfv�=���mLm���(��C56����T�9�%V�<d�p�������:s�AU�wY6��:k����[]������f����b�����Xk�pS�'�yD�#�	���\)X�b5���?p.a���hB���������N�4������:O8�j\i[�">j���>s�
2G��(L0j��W���W������������2
a�X�o�f������	's��������N�_8\��M�^�uB��Y8q���j��O�-e���j���`�(�m����A�$��s��-�-S*���W��2�4����C����������xwz�6��6k���
�'=�����d����''����\��n���*0G�bfxXFi�q����#�� x����
���+j�P���c���=�?)�B�<�R0��������A������E}�nE�����)e�UD������.����d��k�J
�8�������%gG+":5�,q�11�)�����)?�s�f��+��u/@:"�|��7	&�jd8��;X���)�f_�|9�cc��0��*�����6_^G�����s���P&�H'�/AE����\�������v����uAC���f��D�J�-o���m�/?R���M�lO��W��v�,�`%+6����by�_��"����ZVc�����$�qPo��tfK��	@����Z������W���F����.����+�� Rv�u��C�^�TK��5R3�����G�
�
���>6�p������b�0l�q�J���\��dp�G�\��&�B���7�d���l�2��D��
������p0�;���a
�M��Sn
o��r�2�N<��M�aPN�����9�9������j-���b��L��]�W������'#�>��y�n��{}y��l�q'�_��i�M�	���S�{���L�����eU�D0���HJ1B�gg]�v��� ��3��j�haM��
������!���V��y
�"X�o|>�%	W���S��������	<?V~�<�9��=��D*��^GB#�SU����&G!�M!	V�������<q�-.����z/��fj�7�$[T[�q�n.']����oMD�x�c�k�nX��ZD�"��RH���N���@y�2P��)&J/��jL����2��a
"R��+��Y!N���F�Y�)(�7�7_:h�����aWk�M�~+;0���[�je�0i�&|^+��l�xW&vZM�����;��L�y�5����d6�����=X��i7ca���]�4����������/���6���MoO\���Y���v<@��:�we���9����Jv.%��~���M���F���X��*�9���^o�T
�����C"!��8�I�-tA-�2t�y4{@���=n������X���[1Q����b�����X�kH4�
V_&�S�P��6u�2�P�=�]S��zbDu>��U����N�)#m��I$0�����y/vU���8�xib�J������E�aU�dIC�����$�M,��7QI�;�����C���x0������>���Q�l�q;��#�R$�h2�5�B6�b���l5X���T�Id50�R����4xb�����	��P��{��Uc�c��m��������`���D����f�{v�C�����]1��Um��6���C#-�b�D�>Q�c�����ddC4���}����*G)����'h��_�%F�
L��l�\���Z[��`��	���r�����������o���x�����sC:zr�Y��i�	�1��O��-k�H����~U��}sr
o;=�9���-���������0�k�EAX��x������#��2%����k��t�	bY�t�8F�bF��c�x��������Y�q)��f��(�V��2���J��}�}G������o*v+%w�@��]1�[�Y������p���"7^���i���������8!t3�:�������J�T�?�)2���1�c���u�y}���Wg_�iH--�8Dq�XSK3��DWPIQ���V9����Vl�=-E��g~�%z�������39v��m��{�O��I��4)�Ez��At���|�~0�4
@gzbm��Z��x��Hq�c��wYF�V��;���`��Z����I{4h���=X)���dHs[�F�n��8)��a��Z�sfR��D()��di���������F�����d��~A<U��#�HV&0j:������Y������//��,�F�����3������n�����������/�����k&�8�i�s_��kS�&yU��i�S��������`$�Su`�c&��(��y��p_���;�>f��r�B����M5��S�<g���S�n��BZa6a6����q�G�F���l�6���3n���`�ju����oT#�;?`5�����t��a�u@m��Xc�rvf��p���LA<���xx�^H���=Mh�����t����y��&��_�pBGj$�~����|D,��|��B;�B�0N�:.����+���w�}���?����J�BI<�Aq�h ��nk����M^�}U[������Ca/'��e�G�������_��FH��������}��d���0�n�$f)V����K&���6i�~�����d��pF��K���I��� C�$�Q�������M#�G��&@fn��H~�F0�]��?��Q�G���Fi^xx�Ty����L�S��.���?�������0j���0�2R�������v�;q�F1�=0���D���v�e���?
f�C��`abE��d������_�N/�����99����Q`��R��_�r���E�	]�19�
��\�)��(�	o�(���#*e��Z�:V+o�J(���������y������.`���h���qX�|�2T�0)�������	GM�l=�����]i�r�P�C}�%#�Wd����4y��V���l�r+�Dd��fO�
o0d}����1�\�hY����)�i@G��A���?)���W)p��!��j:&8�d�	����������������\������;��������e"��� �mbQn�"���n���Ck�N4sQ���,����"ZB�RN>���:�I"�k����$��E�Z+�_.w�����;������g���Ym�4�M�B�g
����
��M}��w�g�is�^��_t�Tq�����+����O�\�p��<�)�pyO�-?T���/!-�g��'�;m�D�����!�Q��k��w��;[p���������Z7�a��i[�:�6�VT������J
��%�R�m��f��|�[(�p+�Vu����B��i�� 4MB�g���eC�!GW5����yy_�B.H���������NU^����U���I)�l��'�A �J����[A�_�zmv$���&	�i��������Z�Fb���F�&��d�I���*xt��6��e���3�xM��pA�����%�L������;�z^���v�����i�p��Xz��o����qG�{x�����E���W��lYX��R/r
F�7����&<�S���*�:\�-�h}x���)��:��`��&o$��������lo�A����o�Sy18P&|��>�����J�:�GKn�j��W<9�
�� �D�:��#��6�vf�y{���I��j����~��:-���	����IL�����D?�G�!5�"7r��W�X��pg\x��#@��J�M=K�\��E�,���IO)��&S������t��m2)�K����A^�8�����������L�]\M�L��;��T�U��<�>#�|%�C�"_M����N�!�u����o��7�G,�c�S�5�e����?���!�k���W��T�eC��F09����
4�Ki��<�����M�z�$%�&��P�D	�����#u�!Olv��=OJ�{
�Ig�T`|;�Va��uQ�J������7�
�G8Y�[<����en���K�a�yt@�V��>)*�����
��)��._����f����#Z�;���\*�_������'�n��_)�^�4�� �8��*�Ro4�M&���[-�������
��&�*3
������y�[��*^�w�B��w'| �$MA���{y'b�_���Ba8)v�Nia�9���N�B-~�����lQiO7����'��}uv|��������]mi��$�b�M{xI�����������l[P���`QN�*��:��T���H��>+J�,��J>�G�.%"�:���c��)����Y:��W�iB�_��R��!�5��CE?���~�FE����#i�2-��Kc����fc�g�+�E)m�C"�`*&S�a��U7�CS�67&����9{����7l��n�g@G^$\Y��O'
oI�Q�~�#F��N��M�@{��X��1����V��C���
�m��+23��������R�[B\I;��_������)�Uq�$^�����RP�M��Y��H�|�P'�xa	/IE����4)��#$��VC�v�����RE1W9�����X�e��W�����1RD�bBO�x�!"\������+l�8sw���s��\�x�|�<���*���;�y�	��>�E������yd�:��
a������2s��b~�������;�����R�D�-��g��"+���2�|*�������O���}��v���?;N���ZpT����e�4���K4�/%�y�����YfV�a��9 pt�Y�Q�� �~#��6���K���+L�$�KzX�d�DSKl��(�h����+���)H^�1��G�|�X�29�l/d�8��6<���nz�0TMT��c�|9���#�5�B����p)�h2`M�J����<�}��U��hM�z�������lI}u<��S|�?Rc��8c�4�����)z+�oX}���B�� �%���T���Ci�,1S���J���
��?Y1wLFy��Oq���YU���� �:���|L��f�y/�;�:
�O���>����,v��r}g��|��9A�Y�lV+�vl7�k9��;U�o��w�lR��i8V6�B��Y>����i{�4�=� &e�j�R������L�l"{{m��a��f	���`HS�5T�D�!UT�1o4��0��d4��Q�����{e�@m���`(f����Z���3��<���Kc���h�����i �1����x\F2z�e�Ozz��q�_�s%��=���Q|�o3,�vd�n6�w6��Q)~gw�p� 6��Y<snv���Hr:?b*(^�(A�U-3�m;������f�z�����,�����N�;�5�67s�^&�1>��������)�5c�@9D��GDb�����m�z��������t�K��-��x�{��F���^#��kl6���0�5�]��:�hUTXa`�x#-�.��t�4|��R�� �2^��^��2��_Q��ya��b�uU�m�����_�4�,-�ni#�C2�)0����eN��y���%�|3v:M�l���re��)EQo��xe'�zD�:��m�����s`{��� ����k/-~���g@���zVE<4������[�w�p�ta�}�}��#_�%�@���ao�uf=����\����?��I�C���Ft���cUb}�au��J(0�h������9	a�D�m�����s�`>��^.>���D�6�����oPT}��F��*u*<F�Z��R����c�t���Z+����,���>���0j8�@��V��l�'���x�C�u}�}�CU�G���UF�P�+����:��F�6
P���).�L��2����!���N�.�k�}�V��%;���R���M;��U����)]�?W9'�0�w��*[Ot'��`�NZ��hw����k�������%� D����n})����{�F��9P��m�3��7��:;�':������o�O�v^�_�N�����i��s�m�+W���s������������w>�a�K=�������'�?mB����������\3]^��@s~5%���H�Ko����������P�����������)���L�(�YRo1�!o8���R�������W���d���x��{}w6����T��\���RMO���]<��di�/��E��l�������
3�Si/�#
���3��'�����Ud9V�B����v���Z���#R��q�����?�l������@����[b�oM�Qq?_��=���9��d��Y��@�v]w�YZ5O�qF�o�+k��7�y�J:e������z�a�A�z��C�/URs�=[�)���dg����z8�.�m�V��������N����*p��}��o\E�MMk�X���Pa�7~4�{)�`�p�i��*n��M��|�]8���i�G��h�_~�~����3�J����5��{��>W�R��9�\%����(\�lB�rM��u@.��y�h_
�F���)�R�!��X�������C�y	����5X���>�����N��?Us��q�>p-���[������8Fj~�#��������	���;�f����b����+��U��;��~���lw��}�����,T�����!n�:�iP��:��q���6��tI�WKQ^2����,������U�p�F��'���m���V����n��?�Dr�t�Y�[���<��3�i:�6bw2�|�/����,w������H�9��$s�:O����39g����X��zB��|X��,:��\l�(����^�Y%����L&�`wJ��/��dO��n�|�We�)\0ws�NHC��>�Kd�h�r��?2�C�I�����Q.3\��~�eN�{����&�.�����%A%��6K����*��9��:V H���|�{�n^�_�������g�N����>�*P��A��(����ji��d�[Uw���tC.�K�����������`�J�����qO�t2v��?�S)2����&��O�J�������%X�;�n �_��1�����v����\��i2�����1��:C3[d�����>���� .���g��`�nr-��#�)�������Z���(�%�,jA]�gID�.Ji1�F��]IX�����x;�i(s�j�P���d�W�E+d<"W�Q�O��GW����� �2��h��l�r�|�It!u�t��W�/�)��\a	[�
$��Rs��KD��m5%�t�����gE������?��LU�x�!S&}
�������&�6����g
���#�i�K���XS!��Z�[RtR��Y'j���,��Iu]�LF">=�(��z\29�U�h$3�*��b+vz�C
���a�������PR�g��[�`���|����0�h>�(F������7��gu(���)�VV5������b�E�������o��cls����"���
���<����1�����r5��FWQ�O�����e�#0+����=]�O��*�jh\���_A FI!�nv��'�Vk6����u������U�����_}.���W��y�%R�hAG��?���$0�^��@�2���i����X{`0%��#4����G��|J���j^)���x8����|���rH1��
��Z$[\���@7������pm|y��cGN?��,��5{9?^�9�	����D���������?�������N���������t����-_]��\[#�w��wp��p>o�G}1�����*VV��Ryr��H��Eb��ks.�B�t)�k&�1�����#�:� �5S��A���]?m�#]m�,�\��������n����9zs�<��q����S "~�
�5-h�b�;�(���5��)�4��V���?��#}��z&:�
�NFG�!~!6����U���M�T�����o/`��'�fQr������h���/����Z��`<�}�=rG5H���h�f���M������N��r�$`\&�]��I&��(��Uo}���:���9|A/=5#03�.�^�0K�y�j����"_��
e�rN����)�8�{{���&�g�C#���2s���Ij��JL���txD��~u�����Z>�<0���#�1����Yb����M��a�a-j�������)������B��P���$Q''i�x�
a��Q��]�IL�M������Oz$L�I&��g��T{�x�-���
��a�B1����X�}�;����
Vnc`�KW���t�8E!��!���Xio%F�a�u���,{{V���ZE�$���:r����f�����CFI�4qE��F�s&32��A�Uo�L�HV��w0��\e����v��(��E""��t���-�����7���}����>?9���{�}! ��p���
SN����7��������<���Iy��o����t���i.�������
���_���I~����@��zf�6p��������+;���T)���wg����j���:iz��
V�#�����r�T�?|�]���4L��&������)�u�3�GJ�%����������6k���1E$��D&p�@�!�u�4TL;�?���x;jB91<��>|�~���(��%�R�
�h��+�X���fS��qg)�%�����@��+�(h��8}�����)\�R�
����>�����{U�f0sU��x�h����N`��	�h[���7��VF���y���H���=�f��a����W�a�K�=H��������z��
��R����/_�N������K�o�������'Q�Bai^4���;��\�����>�"�^}#��,4e+������p���DK���i�}���������f=?�kU������J���j��==��|'��{�k��Ha�����h����3L���7�=���@���*k�����Nf ;/����~�oX���z��%|���"�dJ�k:�\�C��x���x����L�C���:R2�B�#KadD��WS� �
�J/e9�-|	�F�a^G��N/�����b�%a���
�p-�^���;5��,�2�1�f���7v��e"���T!?����J���`45��s������o/x�����������5R���p��Hz��$�
<�,:���F�m
���U����&:MA�
�''F��I&�A����]��Ki0�I�9j��G}R5��Z�<?�^aR�lD�LU�ol��b����Y�����~����7^�4�'��*�_r�|������:��l+t���j�D]�#[�7����zN�d�F50cv�~]���;3"h�WlqxM_���������%�{��~"�%��-y����%b�_���'7��!-������I��-4
q�-tQ�)M�
k��5����!�[�9�*w��i�cg0��3I����K�5�@�@�?F�|��
9Z�A�%H/)�4-��$��q1�R��y�@���~fA��<v$���X�(w�t��(�kM�a)���h�����R(�����^�{&m�df����.��7�k(��w������0>
�h��J
��F���I�l,��T�*��>!��F>�����M���Ks�a�����N��������_���������[o���C�=���$�w���vhjg����6���h����>��K"ZN�AV�t�i��$���#��O#U�����z�!�]���I�o{�u���<�,]�?����4���}c��y����c����X��[�;��:���9l9+�-t�4������Zmo0tf��[�s����4m�y����"	�rp���L7�rm�� +~�e������2����;��V�e��*�v��r�e����������sg?W�8e}����_gW�bt�/�`��`d��l�t�B������]m��_���!�U�%6��0�OH���I�;�6�]���8���0(8aUA^��*�C<��S���<2LW�� �*���0���vz>O��F�x�T�w�y)���N�j�;#`d��4Z����^t
3�k3��W>�q;��v��|����;��p�y�i�zn�]�;ow�2���c<=���BS�w��0T
n�"�No�o����/8a+�u`t)2����n%��a|�C�)�Q(�����?w������F�z�����8��8�[W1���
��7kd�����[����0)�����3���p�8>W��J�D(@��$aD?0�&G�-�#`2��'sw6�<Q���y�n�����dQ+� ���}�&���%"|��	��|�XZH���V.��;����"I��y79�4���b�b�22��N
�~�8�/5z�6�*a2O]z�lN8�R��H�gs�	#C��O��K�R���e�Ge���E���5[��1�6��Bx
��h�H�w�B��D�!\���t�RG�(��V��o��O�Qy�r���b�����:V��
��P�B��6[�	�_f���6�z
I�c��i��oI#�"DA�o�����e�3l�����j-�����H���(�����]�^SE�q];���Mt���1dT��������WZ	�A�H��"
��M�s��A�0���Y�4fG7)��Q��z����j�F��^�����$N����TM�8XyB����V��
U�4t�9��'	�?��J��a���r6c�t ���FW<k�?U�-��G�q
Nr��*�e�D�m��JB-��v2+�������}e�B�E���N��p����Y��.p8��������3=
 �M�U�%�Z)UR��c�T���x�I����J�i�t��n�7h��=���G����.������U2�E�J`�X�F��0Q��-��'������������w���4�U.�N�p�sm�;��1��r�T������:c��[�������h��sT`~�����4����)���������b8��2k�I��!I���.)}J-�>�k�'����b����������c`��)�������� ���u�?��L}���@�\R~�:�+UW���
�J�����-\��s����=?E;�����K3��M�W�q!k�����3l�hR����'y
E"FxU,�����}<�<�Wi���D���f[���
�&��X��K�Y�-���W\�b�r]�}!��WkE���U�*����5����9����
E�X��mE'�"��m�)_!����2��-m����(����J�t�.���}�Zsl�!�d���aE����	��D�����U/�B���w�����'��h���~{���5�9��E���ze��t[@a����
s����T����#Qdw�P���P������(�8���L�Vw�97�n��@h^��A������!"#���������RU��[H�{���������Z�Ag����yE�����\����=bAt�(��5q���Y��?�������wZ��?��]��Cr^���Y_`�A�9�qXB����wKq�EQ�J����Xr���T��'����l�a�< �C���vj���eB�A�w@F������V:��>t�����iRl�i/:OR�B��[���e(����T���#N��,�^��h�����(�q�QSw��km������:��O\Q������h���ZVYT��C���@<��*[f'd�������P�o]UBm�[���/�ZDK�X����mS��'���O) �/���W��WM)
g6���9"�j]7�{���N|��]���eb��v���N�:
�>�
�0�[
C��t���&���pI��`��REGr��0R��X��Z�����B�s�������DPR�����=��&���>��~clc����2����<�e<
��R�UV��4��h�|��D2u�]bH
kj�i�jL��8���Y�>�?!�2Spy�����������F���,�;���hy�\x8����Y����+��q�^Sn-;�sYj��Y����`j�esL���nu1S'a���R�<�BE^�������	
�����kV���O�bn������d�W;w81a�|	��JI�{�>�����J��ZO
�N�8t�0h5�2ao����;v�	���;�����+�Cn
��l-����������i�.�)B�8g\X����x�����Y�	�0��$���t���m(����
h4%��(I�e��g���<{���p��9��C&�P����9f�j�W��u��1����j���$:��]o85a<�&mw��D]����B
�I`�&��(@s2�b�l���w����F�����iY$*L�t�:I;:X�Y�&Qb[J�n�$8�g�<E�Q���]������'�[KSM���-��[H��;���'D�=u���n����-S4�6
�V����W��:!+���8�	c��L$�/R�#4`�B+1w5��@��F�Y��J��I/f*-�G�=�q����������e�
u��a�Q��F�1��W�jR
�7�\[�F���b�v����n�z�������v]j��m�������-�h�*��"9������P��^��T
����$!h��NgHutfNx�F�fd���H-m�D@�)�t�YZ:[j��Gv�����:�s`�rc�j���/��g&
N*@�6���o��F>�{�3>���'mviO:2��� '<=[�]5�Fn�oY��Z�����B>TznM������X��U��huj���8X�,�l��A|���0�X���kq4-b�o��3n�N��X�V,�>41���p���:�	����"`���c���E��ue$�r�%-���I��Y��a��B9[��	2�xb������~:�Xv�����_�E��(h��?���yS4�X��}��8�	��Y!k��o�`����&*�b0u������L�{��D�Fu��l�W���w�X~/�,����;r����1��j�Z�0����m�
��[}a���y��
��s�_i���fb��9�&������S�F���lf�4/��
��:K{9�M�r<���2D������Q�����[8X������`��R%��4�i1�/X���%TF&M��p"H0(�U$���k����&Z�>�h��l�����l*g\�2>Yr�8
m3��T>Z,1}	��YE�,��S�A��W6)L.
7��o�������R�������c6�]����	3�KI����M�2���S�����
u>�s�����.;+���LA����$�I�"s����,�����mH&
Mw,���7�x�T�U�P�v����G���<E7PY���p4������M��q�*��U�O�F��3�v*�z����CL���H\�zl@s���=�����U���E�����$�S�&�b.�GXH���U�P��UAy6b������vPl
\�:XM},Z��2	����a�d~�&lQ�f^X�v�s�A�����&�����D��5�Gv�%��-D5�Y
���v},z��m�������P�5��r9��I�"����L�4���������������8=gRi�	.�4Y����x-<:J�!E�xd
�v�?cW��.�������X�V��>*E�jk��vHV��GR���zQe��)�g�i�q{��g��h$��y��w��,��m!��!y�q�;_$�EZ!5�U26
�N���of�%����o#�$S��	���(�a�6b�������}E��`�2�$O����l�y~$V��#,W96���>y��.�Y��=o�<�"��}<�Vw
W�z�����U8aJ`���I{X��a�a��H��	��|EG�'ES=@lU���d_�T��Y��w�$�n����co��($S9���m���\6���
u[�
�/M��>�:G������\Z��"�R^�����9�a��g]8��o��R�J�zcAY��l'������H�	e����`�3-C�0��K*���~�v�4���E��J�*��!n��������M��6N3��*����G�4H�����o���V[�m���_�������o��������������*�M��?�x�����R��z��ZR2N����7J�I���.����(����E�N_�DZg��LG���������� ����Kwn�`��<(�\C�[�dP���6��2������U���m�;��m3%���j��<�����>z�������2�`}i��L�J��6UT&c��%��?d��
��(u+�E�������dJ=2u��"&��wi������5UO�u �.�&4���,��qN9n��w�^���"W�hb�*,
#cNx>,�#iuo�<��Y9����b�R�����e�g�p�
��s�2������P� ���\f;eN����x6�Z]���&�����|Y#n����OI�1V�m�O�DV*�
����;<>��m�1|����&9��=��*����:�;��74l�/�r���i��.�Ly�<���o�iZO��3�M��!���NIS~v�% ���<yh>��o}\S���7)v��D*~�����c�K�a������a��0#������P=��]�U&�{/[��^M�G�7F��5�W�Fk��x�/=z�"h�v���yw�j�c4�:����P[f��E�
�l��"_�������R��i��T���d�?�Hw&) ���s����)�l:��lO�dOt�������z�������A���/H�>��kV;�_y�����A�G�G�P�c���UDQ�2j�w���jTYB��hM2�r�N��lT�JS9+��(�q��wL�4�]�3��ljcf�c���� *�L��������D��Ew���EH���${ ���69���B��G�.D&m�<�bC$�?
&���6Dd�{�C��'4��+	eW-�Z�s����d8��V�����|�k�u|\�H�nC�]�G�����/������?c��s"��N{]�C'�&�w��[(�������T������X��M��qc�0�2�E�mw���!����1���t�z�sd�c���3R��b+���%����{P�+��tU2�T��b��mC�c���X�������.�~�<��LF<5Q���/��V�������q�
+���+g��\��w{�Y��^�5����+�c�]g�fG�Z���\��a���M.�b�jy���h����#�������
�����"�p<�W&�-K�9�*�8�LG����2��A�6��=�#*��p`+V���� c���fP���5�����|�����a�3q�����;m�Ck�Mb��X��/h���Ry��������O���1�[�#!,Jd�=�����_~
 1��"Cb��SE�>���T����JiT�]uI�#]=��C�B�8���.����g����\�^�9z�PGo�S���a�*�-\'�v���:k���Pr����������w���m��b<�t]Q��+����������-G�!f��/��a�!O������*���&*.Cq���lq
+���/O)��	�[�r�:f6������oVE���
�h��F�����Z�|����v�j���m���D��l��p�`I�)����D%i�K*�)�<}�51�`p�)�5���Xpe�������2������
���s�OS*�U�-�7m�7���H�	&�!���������f�
��/��@��`�-6>g�w�8�^����%�<>�X5�e����5�b.�G��������������"�C���:���]��MU���4�ai�������4 %)IH��d����5�����t.�(���Hy��)����t�(���j<��s�=0ph����$��)o>����d��Y����|�n���7��]�_�d�t�*��i��Pd2��-�jxJ)U�/���'h �������n���Rv&���$I�5������y �+��,p�K������K@�G�|#Y��h�w�]]z@�3����f�|�=����#����1��������������8�:_���%�No?���cc��<����ZH[�s��~
�I��l��� ���c�b�mS�n����9s_S�1��b������I�t�qu���>A���P��j��O����������8����E�<��l�����&]i���� +z��tw���A������*���c�m���%8N
��f1�H�EF��r�S��j,
�)����;q�=	�t������l�V�R/��e��&���e��
���q���#�)K��{������)b9���y�7��k7��� ��#�}��F��7��AN
Fn���NE-��@�� ����1�W���d�Q��Xvk����D��[@g��~c��!G*��u�����K��*��^|Z��kM���� z���Q����
�t��AD^)��6��3o������V�������*|��G+��L;:�d�7�`�&=��q�4	1�C-8\�@��(E�t�
�7�'2@�5>+��rc�CU��M���i�$K�`1S-<��Hp-%@��[���X�a(���R:D��\Qm6��������d9
tg�%#Wc���t�/����f:.W���j���t���r�����tO��[w��O/�O�g���:;>��]^�-;��u���n|�H���2����e����������
��Hru��xn��ujZPW���Sd�U������W���������2_a��"1|�:�Y>P�\���M��(�w,:]S	&>un�;M���*n��f��[�e��R�����]w2@��9��7���r&�����V����|�[Dn�L{i��B*��m0�4kUR��`�|P��'��VG���}M�cji�1�^n�)7���l�0]Mi+�D<��������Vk,z��������������R�`Rt
y]��G�T�����J��lXsC3�����__���A�A�fM?�6T�]�x���J���]	����Jvt��a�3�87�A���������&v���D��d��6�#zs�J{(�����������!������
��s��Mo�tK�������i3�������L,#���x�[�6�����	����,�� ���1��8�{�0,e��Bn[�L� w
�����}�.�v�X�:d��`���|�bq�E�A����������2I�Pi��s�* f�gh�S� |�2t�\���3�{SCs9�C���7+y��� �Q3�AAn��>^���z�F��V��?q9��,�mZ*1��z%k�%gy�Q��D��R���iO�[F���������U�"-g�d�^�M�bf�"�{f�EY�4[)j�h�:7@�����������m����*�
���&-�����'�$�:��X�vgr��(!&	 -kf�����7l%;��?3!Et7����k/�����5y!��v�x���euwP���xM���V=�mAl�`���>n�fR��E	A��}��B�f�K	`�tm�nv�K�,�1�(&+�l+���s�M|�1�}�yYZ�R�
/����N��I����+W���
(���������T�����&�F��[�j�Vc7��Z�d4T��{50��y?
Kz�U��+;��Ru��z�����]��!H�(��V8����8h�}|�Z]%T���;4���!bz�H���6I�S,k�6R�����h6i�
��{}��y���G�"g�v2�b��2�^����?����������A���9k{��<����B���lE�4s���lz����6�������T_U��"�4�Y�?H���&�3�0���pt"�����>;�������S�*�{�,�] O+y���<B�>�M���{�wa��i"�J(�'��H����}���Kp"WQ�R�C4�[�^����PN^�a����8��J*�s���;�f!����}��������4������	f�����D���[(�~�����>GH���#�T��f�c���y�U���E�(��������"�pj?#ovn�?JB��5���M�f�6UE�����������LS%�K�8"^@��,L/Kg*�@�#R��T�f�,����i�~�d��P�����a����A��W|w�FRg>�N����������f���]��3���K7DZF�)F�6���P��2x����8��v��k��c
�� �"`��_���+�\�Rq�Z�g��U6��Y��j�[�X�*vuR��"Or0�%e�Vc��cl_�]�a�H�~N���+���"�n��	}Z3���wD<�w?�!��K���*IWE~P��g�{���b�n�w��<�<�&��[�6�����>]��z<���9�\h*�<7`�������VG��/�������k�rt�y(qOa�����~��WU�D]1J��j��*�~�j
��jg���o�B^T�^|r��O�_���q���AP��ysD�>x)�����k����s�}���c'�A�����06B������%��;�<Y�Q����$�Q-���e��gr������/L���<�m�Zq�\�v���G�N��	n�.�Z�_���kN��VM�f=���	�R`����r7�Tn�;���j�d��>S�I;��S�H���������w�sY}Kf�n��Av�~^���/X^i[�Ji���@����P��a���C�����+�ML�3��(gQ�_aD�8��)�.���G�.K������e��H�<����2{���u�b�
Z2�6�z3�oN�]����F��h�ju���Z��iv��5��D�i�?p�mL=�Y���N�1I0��'��������an�6V�3fJWB�|x-eY�D�-���O�n"U��tS<"���,�?�����4an���N�����sb�rA+�E�Vy��^1�>� �+������z�p��vdW���bK��v&��;,,E��������P\�H�3��F��";�.?��e���w[}o��N���Z#o0��qY[�-G+~@P*���-~���9���*/��w>���A��E��`��o������)�_�/dv;�-G��+K)�Vl�����R�0�-�zV��8���S�5�
�>k�����f�9�h�@��z�.�X�U�9��`][���c�Q�;"oy~4��f�XJ�`4��������-����w����hW4��.��v/D�a��N�?��V�`�����)(%�'��hRK4����p�qix9.��@�*|K�5�(#�[A��c'��#W�3�p���-���q3�\�X�����;!�u�/�)��,:m{�����
�f�5���� ��p�r����Q�K���|��el��W�J�T[�HM/�"U�JB�������.�:��v��kEI%�eL�!���ii��c���4����R
@���a����%q��Y(Hq��Mtg�k@]�)#�S=nu���=C�W�Cc{��g�������������L��*y��.IQ_
IdFk���������Q{2x���+Hn�m�bB���|��^4�^��c!w-/}I(O��@������?�_�v����(*/[.�]009�-��,@YI~�,n��J�K�������~���������=��dK�xY���v�?@�?}H#��
�"��7/�TG���������>nAL��T�������N$�~sD%���]��~|��<J������w��^?/�V�Vl��K�2��sTqzIF�>��	cUJ�V��������g�R��d��d0��G�Q��h��^��z�GiT�n��4S����W��t���o��"�He�Y�}P������'@�����KW��*^^��4]�&��&vp�K,<��s���{,�B��t4������F�GP�E\��,��`""�>��I�%�8��8v�P6c�i~0�R����������%V�p��=,X!9d}�n�OO��}���YzXg��\^�uV5���h�i�(�>�O�����H�(���N"���+����
�V�m���)����S��c�+NE�g�%��P�j���J�����!�2	[-%m1�R1�w��~�
�	���`��V����=jz���lv��IH�r�i��)��d�`
@���^@�������8C_�|,9R�x�����F�������
4����M�	������zPz��)Xz�>�~m))�����i������c���3��eLv����o1*xu��2����7|L'Fp� 9����V�;���I3d�%�+zNQ(\�K��q�3vrT�Y&�	6'{���W&c������O�'�a5H��X��j���U=�5�(�A�'��?%���������H�3�@9gk*Lrp���5���<�}��5�����#XVY�[�����jj���������-�b���h�m�F��h	��^_H���H��H���Dg�a}@���D�I��O�d1�TUR��N�@v���'���N3��C����'�p=f��!�
H������p!�M��q��B���1X���W�����WZs4�9������zr�����{�w:���y��_:�dfy���?�G�i������;�x���m�9��H����4R�j#x�H%#J7�_c��7���(�*�[�������a�I��_�_
�k3��b���M�v//��9�P�\����t6��Fc:h�g�n{4*��)�h�S�I0k�������nV���_cN?�2����+�X�
:o��gSN�.��%�u!�8�s��?����������Q%���d]-N�Y�� �3��^�����d�f�����E���	����n��n��6$�� ����1aU�U��{U�c��b$\��m��.J��$�L ��[@X�-Q���U���?��y��M�[u�6b��4Gv�Z�
WIAU"���W^tuIv���ku��I��
g~�9�u��]c��3h4��Q\mq���CB<&�
�������}}|q��������l�8Mm>��~�c�����D�a.Gqq#a��oDZ]�<�f�����g��n�=G�\�K���X#�N0�0=�v�3�.�cz��s�� ���b�V��*�������@J�A��e�������B�&���7���t��Dn5
���OWT/�&.Q���;�S�o�����������������n�yd��>��O���`_���rP>�����C��*��bHQ�������yP��pU�*,��hL��������S3�����lb?�y��P5�5wZ�
n��2k>�#N�8&�W������u4�{��B�����>��1��d3#z����^�S��@�Mc������~}��2�v���p����=��:�~*q�$���{f/������<I�V���^��2V��\B���<T�Q��9]��&
��!'7k�5����wq�v�X�**]�p���)��'x`�tA9R8�8h	@m�JD��
��@���� ��_�����W������.�aa����fJ� �g���;*�[���_bPKmP*����t�L,o	4o���Z\n��r�U����1���a;w����3l��f�N�qi$�aW��hXk������')�1~j�S��=������h������'B������3�o���9o�5#�@�K�^��i}��E��j��y��
bBN�BL�m�\�I5^by{l�R`���	����9)����{��G���|����������������Wg���.�_TS�`S�G. Q��q,���uR��/c<<�c�(��U���tv�l�T�u����^��f���]�j'���QEUMI�������@Q�h-������`i���xq���t�]�Wk31�B�<�62��^Z_����	p1_��f%DJ���f���cY�"f�hA}��Q�g	6��
����X��jX7H��k��X
�,&����R���~	|�W���t9y����W|&j��2��o�xh������������������~������[s����U�}�>���%�q�Jg?����V��G/-.������2�Qx
1V���SB��r���m�Gl�.^���o���Z�`Z���ba���O��O��O���u����b}���qi�z���Z�������?Y�[�,��3~��]I���e����Jg�$w�����!�=m�^��OU�M��d>/����q�6[�voW�����������2�������#W��|�N��0r&����&����FLWM������S\�$��%���N��2�k����� �)\���G��T�9��t?��_�^�M���n�v�(E'T��"HH@\�&/���B\r�$Xx�x������|o��,��?��=���7]�����#dv�}[OSri8���0X���
�im��+����L�m�=`��E�@Jj�r�����)������U[~H���
+(k@i0�����HG�����-�k~z1n������B�l����������k�n�:�f�������
�[��5��z����d���0����e���:�c��������W2	F��%��V�����~W��?9���C��b?�Y�S��'����3A1�A�E��D��y�A�����3��5r���/pr�?��n�\��	3�.����`������2�e�����,���|��}D��5���G�f��_E.���,�3s�f�����F���F�f�w�g�V��o�#�����j�[3|A���%���X����T�Z�S�y���~����i�t�b/���smu@;
�0+v��=T�_V�dj���I�Y�Q�R�/��Y����c���r^s���v{~%h��p��cN��w��d�#�h g	�g'��{;U���b��U,�g`�x�5�LA�Y���+��_�<U2�#�l�T:����������sq-�DT\:y�ac�Ua�m`�i}�(Us�Hr�GS�j�&0eM�2e��5��pAE����wPP�I�g��2D�j��*h��TPI!GJh�:J�)*P&���;��D&^��Q<�r�p�2��� r��'���[1�H\�au�"���8�Q��K?^7*0T�������%�3�~{�������AnV����|.>[F�Qq[`L]�R�,��P�=#�
��Yu�X!x�DjzL��:��|���n7�b���c���J��G���W�8���J����i ��5%F�B�G��Q��*�3��r�R���.����+����D
��`,ly��^�`�u�.�x*c���������f��G����{�f���S���fC��}@��M��D��Z'����?[��i�_�_*�&�cM�'������R�Nt�7��
p��XL[��Jd�8;���l���T2����0Hz�jj�@4���%�GP�=6�+e��b^�H������[�ik��
_|u��h8�� �S�H����0�����
#����T,l�E�xz�/����o��>����#n�3 �D$�fe�-�3�$�iF�>'��T��'n�������	x/?���/��Cd��_�O�:4�=��R�m�O<
��1�)�{	1@i[�����,���Q2�Gz���1-�S�e����p�%�IB����*���s�DU��?��s�����T��X��?*��d�4;�6$I����X����RR��q��L�2�w0o��%S��\6����O:�'��tL8>�'�KR1%�&D 	h�7��JZ���('<Q/���]��b�l�4��C�3�u��F�;�:��?������py�5���.&����:�r�!%UlY:����7 �C�.��.���o�}�����v���{���y�Or�}��C�s�����k������^���ZB�T�Z�w���nh^����$�b����#�<����p�A���W���q���<4|�!������>�:�2sS5u([�1����-�)��t0a����T�co�^������_�71�j^�� �J�$������c�H%"�F�I��5����5����Q�]��<g��sl4���>L�eY�����
�V�V���lx��e|OM(M'T�K�����F�Kft����R�[T��X��5wJ��2��Z���&��J�qtq����o/N���~8=�=zw1>?�?�*�boD!���- Z\���6]�pY��$��B��d����]���%��x���^gR�GUc��[��y�XC;��E	�6��9���?K,c�/�5��RE�"������C��S�$jV�j�$��I���B*�TZ�$�d�,Y"���Z9�@����	>#\�}��W=i_���R2��7�&��t8���Had0eBC+��2����6����+�p�V���z6&��Q���`~��8���4��a^�M�$*�0��Y0r���d3��r�n�W)[��N:��#�!�F���/F�g�������P	aH|�UM�UB/�"�c���Wn<^��1/b(KefLH��[(v��4M����!�{��M��o�f��gi2X��}'�D���1%_`+{i����J��Q�*kJ�(F�A5��g��LG4�!�4�	���e�/�P�ikbl�	;����h�)o�;�����d+G��E�� \#p��(ro���|u�v��m.�7U�@�{������ �*�������5f^"G��=��8����n#�`���\��>�?�(��8q���m�l�g7��ybu$.
��,0}7��QXjP��!��$C
�
���=wE�i���G�AO�q�$�)�����5�[��������,�S������t1���f0��0)���y�g��
��c�p��_6��$j����������Y?=��/�
A���<���`T��B�V��j�#:��@D���`-�u��j`�>6���;5���d���.;�N��=������ow���,y��$��������=!B�^!������������+�\���T|��LF	�=!Q����^�I��2�����R���&h���4;}���j���xC�������]�bh�0%^N����+9L������r�����������F<V��SX��/�_<�i'z��]wy]���}�cnD[��{�����}��8�k��F9p6o?��(I*��
����Gd�d])4��5��}��[��d5�]�$R}D����B=�yknjQ]��J�m�����T�p#���#qe����
��2ogUjU���}�[�/}������Ke�(/���+hm��!O~.�5�g�h�i<�eAx�����5k����y�jWsf+Lx���u�Bn!{�����L�V����[�(�|c�^�U�G�Hg���B���S��b��������kT������j`9�U53��W(�`�>Y�0���xQ��:T��hV�*4o�sq���l��e����;���K @k����������'��n4���3s��Y�����a4����jBn�~?�;��K��F+?Z��e2���Mp���s;K'������e��8�l�_<��d
N*]����2UN.=7������e��O�����R�*yM���v�A7�}������;=�8:1�����f���Wl�����j\,�*K��(�M��[�`"X��^�c���{S�_��>)�P5���������2:��-�*�9�v>��;�����7i4��iw��>�z����D3�l)�7}`lK�?��U8�Y���*��C���\>�S!�B������Qc����5���:J�C;�VP�_6��=�	�!,�l�D�P�{`�lW��d	���^�x�������7g'@A�c�v��0������?{��G �Kq��
���{��1|.y�������u2�v������G����zp���Wz��4�iO�1��C�wqv�:�9m`��H"���a�e������>V������XjBl����n���j2������~~�{�Zai;@M\�}��Y�������50 VW#�DA_?!� ��U���;�AU�#�rT�o�����=B�F��x�]�f���Qe�������X��8��z����%5��a8:R�](KvyAZ�k(��.��@3���Q�kNW��YX��P�P�����v!�HVD����t�������p4�N�FO���cR
�f�j]�MSO3a���1��

���8�k.A��P�����]G��U��I������n��n\T��������wjew��e��x��__�0�6���H�_6�`�Y�����ARO������`9�o<�����+�x��k�d�
��������j4fn���;�A?�e-M#`q;�k���)B��`��k�U���6�]�N��R�*��������/|w�]^��������5������N�F$��)�V���AHEM�iL1��9z�j��S�����@-�T�'V����=�M�F�%�_��x��X�k��>�`�<��`V�I�ZU�g5&������!v��h�=|�1�Z�N�U :�&z���u���;�tG���5����f���%���s�'��R�>4p?�m��p$�o��+wE�C2���:T������b�y��q����q�xb��__��8�; ��mq,6����7�Y��,��`�)�����)��8�9&4_�x�Xgal�f	g����0�L��*P���y���u�Nk2������=P���M08�	i������[E�Xt�]{�?�����1���/q�/� v��(�'U��r3�+���F�{�<NK������Ub5����a����	K?_�v*�/�?A�'4����`�t��������b�
O4\	f�K4��z5�[���P�l��${3�OG���o4F���=lv���!c��IO�����������c�U��T^|�<�g���pY��_p��	��4�A�=�]X��V[aL��4&<owh�=C8��	T�����q�E'|�c����o:�3g��'���D�#�����n\}�8;��~D���d:�]sB G��#��������������P��c�{��m/c��Kn[���Q#��Zt���{�v���#�TKm��Fa1uR�9f�R_;�}f��a�/P��ls�����u����+,u�2�HQ��R���2����^������z�mc�y�`:�x��5�j?���h�
-��wt[�������W~�f���/w�����I��x�m������������-+���
����u���Rk[a�<l�&���#w������NC.�����Y�[E!�����q���:����Z����`*�"2�[	p�����n�J������V���"�N���a��5��e�Y�:[��a�Fz$X�~���9�:��
��(S0�^d�V���@}���EHF���S��2���4j�=�?���_�Z���(`(�E/���c0Q��q/��cH���t��e�h�=�LGo0k4��7�g��?������z3�`^�f��������^A���L�n�^��	���W��>�G��&$;/u:�[4:'���#^W
�?����M�Ot������B�����Uf�����`��1o�t4������m7S�;['$�^�m��������
{P���?��Q<��X�A�;���������y�]!F�w������J����h���z�X�$�~���QZJ���l�T�\��/�l�a�#�C+�\,���8�$(T�(�S��Y�6��^�� �����h�e���n���w��O���o��)�1��e�M�mK��!y���
�0����j�wl�
\���`	a�H��5���9�E������^��5|�*N8BU�P�����eI����).���YNd���_��*J�xa�������8����^��8\:�M��q;f�Y$�@m��n���K�)7��X>(u�p�b1�	`�nr���F���|n�������3���4d�>����.���&:�nt�Cb8�����z�L�^PsR��9JRE#qs�D���B��|�0(m����w�Y���O�I�
���YYm�d������l������M:t��x�/lg4�%���A<�z�.<e��L-8 ��u�1�d!���F��<�%s�4���4/�P����4��$|�����L��������Q�$��)������0��P��8�L7O���y�������8�1x����l�#��M�G:C�?��|~�Av)�Y�����,�}`�\����(^�g���f��O2a��5�b������o�u(%/���?�!Os�f�,�txrp���S�����Q��A���;��*����w��	��*����JU`��28;��\�0��B�dl�9afQMd���dd�����2o��
�d!�u@^��T�}64�����]����*|S�\w��g>|�Q7N�!�}�N�����s_�c�b�~��vQ�~��3��H�?r
�t��O�a�oS�ui+�hj�����j�`T�����RF	�����e�~[
w
y�b���6��5����_i~������%~�R��J�G�z���v�a���f��4_i�'��H���IC�B�����7�UP��6
����lz�y{�f�3�Hz����2�^h�;3����UZ�*:8d��(i����Np�h��?���c5w�Lc�2M�o��j��S�oo�l�T��G��e�,�D���8�m���]X:�2��u�@�~��"Z.A(EV���������V�
�`0���7���v�5k�Z�aV����
[�%U�RM��Fj��yT4�vnf��f	��(�������{���]	'S+s���O��t"���b|vq�
}���9Pf��3q�p:ui���H��
n9��v��%L�2�s+pI)�x?�(!�N!~�\#L^�"�@f�6 [�m��5�C��0��h��!��3������v��\N�O��s&u���M]w0M����u�n��MJ�:5R�Sm)z�Z"����"��L������j$�|�F��3��@B9
`�u����o������r]tY,���\xd��|�[9����2xS5#�G�9d���l0������0������b��@����+�N����_���P������b	���6��u/~�f��%�7��A�� �B�k�Z]�����:�������������VB�m1�h��N�L4��d�������1"%��c�)�2@�?�7�=3�)���{����O+�����UL��5����w{`_3N������:X�'�����)�p���S��/����
�XL��u��M{���7h��*�������C�������qJ�������2��W�xs}2���cV��dV�'�Zs��}��^�O&g��7�(M\������h
��V���m�+w���"���a�3���2��]�Ek�B���M�[���fr�DG��F�� f%[Z���4�z�������V�7�����/A������H��Q 5}�����l 2��;�X��=5�d���aT4���~~Q�|����&���^�Ky����y�T�g�o��2}~ �����D������`JK�=�H����<dI��)������l�(��������-�2���f�f�������������pP�����xM*������~�
��q;�Xfk�����x�	�^GX����+�v�[������E�
�y�����i����}c��L�I9()!a��z6���R������Y�}�b�;�4/��������5��^I�����q�������3pu_��O�]�m��_��.]��Ll��2�|e\w���tk�MB�p*w�����
@��w����o/���9}~���0%�|�������3B�7F�$l���O>	,	9T��>?j~�m@^�HL���w��w�w���g �v�:(}Q{a��
yz���\h��4�����4���A#&.X�W8�r`�; ��`7
�b���#�2�D������2���3�N�i��6���t�~�?K3�9�h�:��'4�}�>ObA$d��\Y{�p�����N��������;�r~�\��{p�k����>�;�(�c���������_���]��z��8������k�
�T�%�t���v��i���������g�N���l���6V��&8��f���c�<�xZ������:m~}-���up*����]�'>
!���3�������f���o���iuZ�N����;N�#����i���9SYl�QX��;�<��8���,��7�u;�~w����A�3�&=������l�������r^	jw�����i6���v����a9W�B��WJ��5��@�~�#{��M�p�t_�
���,
jN��m.�H4n���	1���Ue����=�8~��{?9
��T� '��]�]�8-������������"�T�F2 ��J/
v�u2���H2��at�����	����
�y�����f�K��H�(������i��Ubg�S�Smo���L���YP�2�.k����w=�D1�h���+�SK�jf�Xo/��$b*�k����>��Kl�5'���8�}�;�QeRnB���]`��M���}�.�%b�&F;q�CF6����?���L��<���bEW����?	��&����Z�W���C���e����������Z<i������W����?���*����?y����}�������������s'�}�p�y��+���}�L������E�T9�����a�90��s��x�y����G�����A�]^B$G����P��rP�9�����O?����	O��?)�J������7[�I�1�z�;s�I;#{�x���y-1��U��N|��X�{��Y���4��2�\^�S$S!�0�B��/7��^�1���65<�������o`_@��r�X�/t����~8�$�X��vTY("!4��W�%$e�����F�����w�)�T5,8ypt	�xU��J�X��__������U0�E����@�+~D����bP���������"�O��@zk���$$-���R"U�[q"�� 7���)�����������Qj�{�)!�r�>r��Q�Tw�{�{u�������+�Wl$h(�\�f����3�����������c�����L�#�!�#H>b\�s�d��F;�����Q���^��0b=�O��K1��}��C~r�.��}�Z5����"��0�91^>��W��@l�^�4z�fN��A�t�>��1`&���$������`�&����
������_w
9��%��3�<��j$+H�C��2��/�v`"x�puC����(���Oa�L����1�p}�E����
G��sLJ)�9�BQ��"���
,��:;���#A�������r2�?���2(f�c(����X�L�^���V����a�+f�]Jn[�F�2VO'T��r���@I��v4k�.��+����e��.���R�'�D�E%a�Xx�3�A�h[i�s`�v�F	en�'L9��T��0|��k���~��;ysv"=a�Ou4���]�N�����~1g#s�R�<�<k�r����B��!�A���z��e������v�'|s~e�g��U��-��K��84K�
�]�x����ts���	z�*3��e7�t��2JE?�G���"���q��76�f��*9��F���
��s�&�V����GaX\�)�D��>���������X����e����b�8A�h�_oq����`�k����n��3�}�zA>jh�*@��C�`
Dj�D|5�H)@������AO�B���� ���H�n��Fzk��;C0*��m@��|����
U���8�(
��SH5.�D)�\���uX�P1�M���m���_B0��c�E4g�p���>��<Uh:�u#c�����������D>��&^�P�Y:����}7�\�%��&�������y.X��5O����i��+3�h���!��*�\�ld�� ��d+���&��G&�,x�4moi��8�C[wh��J���g�"�G��a����l���1��RCYA��|����X�$�;�q�_����1����Q�\�RX	�T��6����+
M
�cG���L�yX����`����P���e��bqC��({oH5�z�{	
���t��^}�$'����Vd}d>X���Y���~���]X����q"��~�����?�q�en����q����%�e��#]��Nf��ou':G�r{4�|��y�zq�e)�
Ud���]��T�q|#AC3`�3�A���!aw�X
DYU�4�=)i��?��u��S�{�Sn�@�B��2cY=�u5�g���t�x��@`��0Iw������r����Nz��0Q4o}���]�;%($�^��IW�j�e~A:(������+)H�����u�sK��W�"�	����7c��i4�C�?lN�^{����Z�J9gH��/�F�,�9�BzIt�4��	V[���4��Y���'2�H�e6U�f(D�L�^���\��@�H�$���c����r�t��bRN�^����h2�
K*H���2Uv�lIY�)�Q���9��8(��
8����'�)_�w��@Ip����L�����g��<cu����W�����=���{������_t?��'*`��#���_)��N���1+e\G��W���O���<Y�����������;���W�������[�C��Nk�����m�&�Yk6�tn�mf�����G�������j��;�Zy�_��lt��	�I���w�y�~��#��7x��N�W�0@xa��8��dur�t�u�r?�zz~�H�H �_����B#�+�,�K�Z��X-�����p���3�7SA��a�E������9T�#�023�>��U�a�:��$��@2�+�E-��{/�2�	�S�A������������UsN�y�r��Y�Q�yN ������qu�a8���������'���v�����O��w�����w��������u���?�*6&����>�on]��u2�z1UH�mF��V���}����^e�����-Al�q'�T��uZI��vY�2k���G�������?m7�V�1��~���2�Oe�v�����I�o&^��D���+
#����!�o_z9�_�lR���75.��������>�'���n E#�_�����T�rue�B��R�@�/��~��b1)���uF��@�(M����/1���C@#�?(�V
�f�K�{�������������4��5���'���-N����8 �y8����h����N���#nz��C_sP)���f
�5<�_
�B��;w��+�'�^�jq!�q�*�r������4��BY���c�����_�P��W#�_W\'KS��k��8��V���8W+(��b������U���=C��X�
q���$9���<_��G�n��/^vL�����O�b�'�{��
zM�R���w^����1�"��/�[��L����m
rQ�
���k�'��`�W��=t<��q�b5�Qb������[;&&5�cpc$\���p����8������P�	6Z��SOI��;�N23���c]��g.Qaz�xb�>�	�
�-�$

FPc�#:m�x���3����/�K��#��!>�!(x���Q=�1v��(^�^��n�9����$
�8q��d��������_�[<����������)�N�G�����9n��{�T4p����=<�`Dr�%7-��_��z�����sY�F�f�������A@"7��-F�W���HF�X@��!��$�����
���)r}�3�����s�>���|��b�G1NS��:D�:���{���@L'����@�o���XF_�Y�4�6kH��]A��� b�K`eq�K�G����q��� {s��;XsV�(��8]8��P8������]�aW����6`���.��z4*�N����TCt�C���l�y���A�J�2^�_��IDGOC�kD����;� bov�ab�AbS|��0��XnAw������YC:V��Nq$���v�=d�c�`�M�.��1;
.Y���<�6��/dx���:�G�/��L_�TBt���A�� �����O��XOH��+!�a�Qy,���\��$i��6$n��*RU�V��h�����h2��?/���n:'@l���?���j�H]!k�B�]+��o�b�Q�}���;�Y���������gg�G�|�6.�;R��p�`��/���G�7���2��{�s6K�i���5b��p#�L,v3�~�����P��_�"*�*B���8���LIqS��L����t����gZ`#$^��$i&�jwt��91�c�Q��������T�<�5
!I,Q.%����c�F���S��H�X�7(+����x>`��2F��Lp(.�
�|������wo^Qrf�y������]�4�h�oa@	���^�[��&�B���W�o�}���:����9�Q��u
�E���bK���F�A���D�6
41bO��b�m�f�K�f�i�fNW��@������$���[�
�(p��bK�k�|�����8����Wy}�v��,�K!-��`LABg��
�6�FjZ����;����l�w�Ms7#�v���(!�n�m���+=3��6
��B���)���~H.
~R���h0}�bN�@���R 6�DFy{��^�A�(2���QK�.E`�����P�(Y���Q��5}8A��*�\cF���*�|2�!_�D26x�����d~#Q(a-��lH���_�����
a���Hp�S�Z��%J#�g��`3�k�F@�>�/2bg��N�����^���
3���`�����������x�q�����?fH�>V�������!.I��MM�"K�.f�tD��	 6����[c�|�
��G�g���@�7l�����;h����ir��
���{+|���{��Xu\���b:�c�xo<�������/MNB�,/v��Y��������q[$����p�C��4���A��������i���'N���
�	��yx�&d�o�	���W~�Fy�IN�2S]�U7��2���j`��%�jOL�}�Guf�HHX+1���"��QOL�W` �O>��0m��w����b��Ro��	6�^zJ���1��Y4H��CkS��MQ!�V���������Rn���d�1����,Nh����z��55����7j0
;��j�]�s�,V��k��$�������F�`�!���P�]�P��*�v�#)9+���V������RM�
�,����&�.9Y9S�M�LNzU}A���3&kP��nN���L�������jR���
#�}�$�RiIb�j-(����g���U���S���I�
d�^+�Wh�����������o����&3nM���j����^�q��Q�
TiUF���|&������f�A�KS�f�������o__$f�w��������TO�9���qv����=�
1����W�RYv���
��&�V(���*�[�����8:���&����O�Jg�%��������j�T�;��<�J���F�������W��#�-�oc���������@���.x�K�.�o����  k�C�p"?�.������
�Xi��:)�����'�?(���^�`�����C��0��D`���� ��z#|�|��5�z6c�D+�������jN�c
y[X��g5s�b���f!���<u�}7�>�2[�"��A�}��[���i�o^����;�l�G��g^9�T��F�oSiB4�������)�2~���s��y�*���Qm�`n�U:uP�E�n�~W���a����{�#��R��]0�pX���P~C��1 ��?�xQR�'�\^i����D����&H��&�!S�����L�����8�����X�g,1�lYE�0�
���������%�Z�-�2�D��8�%��[3���D����W4w1sJ��S{:[���0��#	52����`l+�j�6�y�n�{N2�`��D����w�l���JO��0�nT�y�ks^��><�9/E��|���f���|s�7�	��o�^L�2x���X_���&Hn�l@4���_�A���P!��ttP\����J�$|��b�oq�
�S(i(p�M�����>�����K��2��U�z�!M��"��xHZ�j�2����	����Q^�I#���,^{������<	�j�7?I�d�'V�f�w�R�������	�R�L�]�����m�JL�Fo3���Wqj����3O�'�����9���.������8��~����}b�bvC2�<�����)�,n!��&��&�Tj8��"a�/G�t�
��'s:���q��9�r�0z4�TzZ��ip�l��y�*�
�i�9��\
�n$c��Y��"�A�YB��U��W\�	C�g&H5I+�������r���[�}{�l"�Xz�(+��P���@�}��Q��?�S������ :��
M"AN�%u(�����2
e#�������>�8gKdOb���Qn���S")L���?c�5�����h�h��_~5�#�k���2������.H�
M��-i@�]�w�n]��N�-n+pc��?����k�^��h�s��������I��=�]��=\���x���1�4y�n���h
�8���$��dx-_pr�����,=
1��/{�T�z���]4
@���b[���`�]9/N�
�027vYW!\W9���cs���4Y`g'6a���M,�y�A^�#��t��%j:��i_���>6�����~j�b[��w`[�@1����)���J�����0S<����P��d�!��`g_��L�Cr[�y7����J0$9�
da����(���
�|kX�����������`$�C&=������`������M1���s�(p{Y�=�����_��X���;��[ ���4��i��@�1��,���V
B�F�u���[!�"PGc�++�*���W���U����\�N�O
��;q1��:�NO��;��J����u@3?�����c������:���� \b:�����1����"S�����b%��&�**�y��}u��AV
�����9�o���D���Z6/i ��]���/�=�m�*AI�O�^U�Ag��R�T��b*����7�M2Y2�X������o��y���j��K�Lx�C�;0��Tgpj"��M=i0{(�	�C�V������R�D��c[�7r���b�� �X9����C��~��<���,�f����=����TD������u~�Sr��l�yI����8���
�yU�C�����*�Y��~7F%Brhd�(���dpHUa+�������gO_o�@E�G��v���p6&��O������#���zu���9��v��5�R����"t�8��b�D)0d��y��I�-9#�*���s�	^P�H1X�����%��	���W*)���lv����\�
��=�^��Zs��c��Ih�H�i�Or�EF+�H��~�1���HI�������*�~mSF}&�*�m|����o���;u���/�����T
�e��"EN���
���My�$�d�l�[
���g3M292#':*�S3���04��/p���o�PSq�Rk�^c�L��25����"��w��q��r8@�Q&R5�K�f��e�B�<3������d�#4�|��*�g5�������3y�}�{_�7������S�&\�����0�[��/�Ts��F ���jYaDJS��>y������n���P3RK��e�'�ry�U8�z����(DG��j-u��Td/����2�F�_�w3��]��]����T�����*��_(��s��������7�������Vw�P�P����M�����E�G�� �����+����b�kT�Am���@1������7)��s��2~�N��/J�,�����/��Sy���C�|F���B�X��a��)���N�1�	��ypw27"���j������0�D�J�Z�RKX��T����B�|"����ot(s�U@���/�< �����p�s���G��U?�7����*{+k���jq�$Q��oH�Z�9")��9��r1&����t-��45���	@�7��L�gAO��h���W)T8H�r2����Qt�����������`��������K������{�
8�!�eVi����,X"^q&�5i�9G���[�B"&nZ�Jc�"'4:�_����y�!����Du�_��'���ex-��K�v$�s���`2-)9U5ivv;7[�������6�Q��<�`�m�i*�������o������5&���D�L�^�'�&�h��.&?	���8dVWS�"���O�J~����V`-M>=��9���*��H�wg��p�dy�ie�#:n���,c��OI�K	�iz�b���+m"� ����)C��d]��Y��t%z���L��^�u0S
PU�9�������������;@G�R}"�8r[W���p�R%���$t�u�D �.��q�g�jJ��i�u8�0��c��1Dm-'�7��c�8K(�+�w^��M����=�-H��T�r����i(�1��l/�����(�	��P��py-��gVf�g�]�Me5{rgr_@�s����9���}s��?[�
���{���/��1P��|���:�<����vI6!�P
|
�n��J�%��J)�l�d.�������365���{k|��=�����6��r�EHC��G$�� ���-��#=�oH0�i�����
r�M%���miF�s����Vpq�;�����I��#�f�)C�JV*� G_�}��_H�&����
f���3`��H�F���>x���meV�2�n��:K��@�T5���%rNB��.�����#&�
Wt������rq�c�X4}�a��<��5���S�&���^�wS��'���;L�9�C�qI��r6#�I����u���\��6�%�DNxm�O��Kut0e�L����%�	�{<��R ���&^'E��^!"��E��$$�T��	h��/s������>iVNy8d)"3�!10h/�@y��V�)��@D���d�5�����>q���bd�t��Mc�����m��6���^�8�)�V^Irh���c�����H_H	c�nb�����������H��~0wc�%����h�Kf��s���BR1�����$XGnt�H�C�����!?��<������I���y-+!��q�T��D��)����(��D)
��Q)9�KV�\3k�
:���B-Z�b�h�N�����S��8��B�AZ�f53�=�1�����\��Ps����s`��+������F��k����/=3��v�W�9I����{�H$S1C�	'�T�7��`[r?d?���)�s�SAD���\{�F��cbI�B0�����/�_��(��*��L&��i�-���0�y�����U������o�t�q���]��D�)\���1��8�����.<Jy���$w����\�����L��KS�E�S��9T:�H��='c>Z,t����cm/�n2Y^i�2��D�
��>�g���`K�|��H|:e[����
����F!���@��_�	���3EK�_ecM*<r/�jb���*IX
���>�EL��t����
��fr�8-�r�U���JY.Z���q
��|S�����Q�gi]�FtYe��.Y�p�jdT2�5����V�Z��,b�;T�����%G�B8��J���rWk�6+O+��B��f(�(��Y�p O4��������F������'	���Q���`���f���-��/��![3����-OI�5�MRK{J++���EC��e�$��_x�Z���/��n0�!MH+
���Q�|��dy��gtMv���r7&���gj.�E���6�hf����5FU+�$+4aIqb}�3��!2Yr,D�^g����)
Zp��/�$���&��#�n�uwG�|�*@(s;.�h,Q�"9*	z���'����$	������G�b���p��!�Q��	���F"�A'��;�*����>�9|A��Tv'���y��� �VY�����2��������3�w�~n{����d�[�S�a.�,V����A%0�+?f�������/l�I�!�2�.�	R��vs���`�g�x�����+rO�/�����^��U��&{���q*�,�PF�`�s�(��~EwS��]��G���a�#�
K�����_B5`��������������l� U%���{�3T6�<v~�!�����XUt��t����L���Y�]����8q�����O�*&�������p�QM�����
�:����2Y�x���\s�y�������X?����NH�9<i��mK���������	9~K��N��
u%� �v�5��j.����%���X\��63"g�&$;#4
������*�@����%xii!O8��**i��)��)6����!�?�l�hy����~G��A2�e����#�8j�������O�}�����b���7��^�ws�L&�������[����X/������bd��,�l����$>����f�z�B`��q������l��r�\2jK"X�A��� P��Q!����Md8"�r��'K���=P��s����K�b�	���W�`Y�s����H[�Rn������ V�0f������V���V=�|y��8�Y���+��j���u���T�v���|�i!�C�����0F7fa&,f��$�hK�w}9���	�8S�gp�4�'�RL���}��'9�<:����SH���8X��4K�[R)���Y,��2�rA8�bz��F��u�����
9
@�;���[�b,-�����\.���Y�9���x�F�%h�*E&S����
o�%���E�,aK���9���<�-u�:���%2E� ��:��1k.U��Y`��j���)Uf}`������T~�HEB��R�yf��R��Pqb^ K9�]�9�|���<KL�<r�r����GFV>���vd�Pt�w>����2':��|_VO�q�4w �����e�����k�`m9eK�r��*�,a���,���	�F_�&3�����D�95f�d��y�"!��j�5�?�fpQZ�S m��w�Sz��Y�?��P`U"4���H�`�Y�������>2;%P�WK�;g�I]4]�V��p���`�sc��w
�z���������dL.��C\t0��1�2<����<�c������S���rp`pD��#5���F���������O\4I�2
����������F�/b�S�
�B�
I�4�P�-r���-��*��n�R&�gf8'�'�
����*!��m�'�*�6	.ef�q���xi���O�8��j��R4���u=�����x]�e�G�.cp���Ol|���%#=�3�${�/,e�����N����!��A����3���\m�gJ]�-�#��3uf�x��]Zt��OFZ�#U|@Q(�-C�Dz��WP��
*{H�
k�L�J�]�F�����hb
��k&cW2R�?}��l�9J ����a��������K�K;��:(�4�z�}a���&�3X�}�����&�o� �8��B�=�?�o�^�W��W�����-.���dE�BI>;�>���74�S ��>F[�"$��
�$�=��"��X!�Z?�S�e���zU��!�RR\E'a�f�76bKq��������a����')S��D[�f<�.p��X�L�6I���J�"�,�������d�P�c�e1}���9�0�}�0����vv�S�g�����K���g>5����B���s�
?�Uk������z	����_�
�[���X�g_��d
�M�{���%��Zo���fab�5�#4�S0�e��N���&B�bL]�q�1u���3b� ������if�X���>G��s��f��cy���XW:)C~�y�]Rj4�4�,�R0���r*d�%6}�����km����Gw�Y<�f�,+�i�P��mc�d�;������p�����'�-P�������o]��VE�Y1A��:s�1j��5��E-; Y��
uT ���0z�H�kL�����Y�df6�7 �+f_��
��kJ�f�1?Q'�@���>�?���W1����C}.���pi�x$��.\@��U�vR�Q�^"��L�?�r�K������I�Rm��q����0)/|2���c���.����t�	�����g�~�w���jL�3�����#P�	m��+B�^)�(/����pB_H�x�	�2Q�������E �#�QOH{���[�B3�B�����P���7���*OS�����.��.Z�6$����,�1�u�yxc���h�Dbv!�l��	���V��"gm��Jf�,�@:UE*b�)��+5/g�6)���2T*���
	�����_�ag�=r�+@EG�*K�u�\�(�b5�qT�5�e��4���D?����^�q�V����(4w��dkL
z��- �=�t�������{��'��dB���h��2[�]:9�2N��{�����[���e��">H	���5�#��Y�!�r�
�Ht��|,��V������a�d�94��&)�ed���6\h��S�@$���
$
�? g��c��BQh�_�mM�F���!���U\��o�����w���Y��&�M<�n��3����<3~��8��Z�P����w*�UJ%GL p��.4p�	���`>1�xs�90	E����?�\R��]p��y�)��E��g���Y��[���_���GN1���h��b9�X���n������+#�,�#�M�U-��tM%�5R�����*W��RP�Z6����P���~H��<�	���rYG�9Wq�PT���[����	&RZ��\����/,��Q�����0�D� �I��{6����`+=d� 0i��8]�,�)U	Z����~q�\5��c�����7F�������� ��.Qq�v�D�����{*���f�U��q9/�O��mI� �d9�c���\�Qp��f�[&������S��!t�R���I�����/�I~FI��I�T[z#(��{�v��c���U���n�E�XEJ��T�T�D)q�0�r� ���([�k�)�d����%����;[
j�B6��+'{��e/:������8��`�$I��5���x>j��uU�5�,f
�h'���y�^fy(���
���4���T$��Zv�K�'^g�%2�g��O:4g����k[�t��U����\��
�QA@R����������2�|��i����W����L�c<T>M��|?���[�`Et.�ll��}��c���{��@�d*k��2.Q%�G�:��s|�}��J����*��*�[��?��`�+Xeh�py^�T��V�W5Z$�n0�5�uHF��P���h���\�TN���T���� ��/b�`}��*���C��$y��1�}l�e�w�-��;+w��j��
4�%�����	��y��g��
�)�S�VQg��Nd��L��(u��}6a���r�f���@���'YI�����pE������~8zy��uE:J��SUi@ny-a���d�{�Ti�:U�\r�.���+��	(%���9���i��$zqM���
�I~.J��,m���������7�&�����a��2�j�|�:S������\�hm��y�L;;s"@�~.���G�H+�k&���P��L*g�a�V-�z�������-
�4~�rZ�<�����1����`6s��K8��������?��C/6���r&�Z��5{{�)I[�f����t�M�����n���Z��~�^w=���r��J�����z��tZ�V��|�5������,�����=VZ��>&u��7Kw!NX�1��.�
#�����,W�V��h�����AM������?/���p���Q����fu}������5����#f��]
�(��
�r,�������uv"H�����#P7���mw�� �_��@�jA_�?�w���OxNsr�m��<�rW�L�X�W�/���Y��J!�$����
�-RQq��h��zj
�v�Y2B�u��"��B�V��W/ ���dx��bAI>�4��mB'�*� ��<�<�w�?�J���K����	��@�Z:D���uX��3��@@!�1�+��I�C(#v(�j�$��l�E&�x�#��hJ�[�:e�D9x���H�Gz%�n�NH\V�2�r��
��#R�{{����V#%7V��j$��6������a
 ]��Z�BY���h��|&��}�"�c�
1�C���v/qFrG��y�BU��]��k��BSt����)�%�������2��w|��%���b�JN�4�$jO���n��u@9nAg���!p�C��7�6���PKd�$i2���I�+��������^�n20���D�������.NX�uo�Pj������zX��0UDG����
�2��J��1�[��?'�$*K6��qe��N�7����?&��9'�]si�
CB��k�B*�D�C��<�E����<i�|��$N���X��~�kV����X=� xS�C1��%���as�_����7�u3��u���X(���0�v# ����>���Q�Me�D>`L���$Ac�����J����9��+����td�<oJ��D���EB��]�\�0��`�����ND� ��B��L�R��u(F����t�&z�B��SUc��O��c��������$��R�����U8�8V�1��������YYZLi��DR
;����;�������9��:�2��5*5����[�1!9����fF��=
��^���� _s�8<TN��B�Av"����\����u�h�8�(��{������M!XjG�T}0�:5g�#	�1 (������3(����D��!UQg_�I,CB�(��Z `��VlJe�M���]�I�`p��n;��0�d��rT&�^�o���:�M�sA��zTnTc*�=[z���%��dU�2vT��0�t�&R����%�+zX�4G$d�rer
(����{A�iu��_k������e�������s)5tR]�v�R������(�x	�z�J����Z���Z[Tn����
Lw�����8�_�P:s��A
U�;��F��h������>�?�$���F^]�d���������x�l6��3�{��[��"��b[tS_�u�8�xG��|�=�l���f���z]��iu;�~���8�N�7h����-�CLe�YFa���lV��������{�M{2��F^{���;�a���F�~�=��^��L�a�+~r^	N��_9���l>��9m���0�#���7v^(����W�����h3����qo<r�EA�i�����Jf�5z��=��&]���A������G�/��a�'Ga#�2!Ua�C��HlD�w��}�y�����e�C�����
���S�� ��;=~�����\)���i�;Rk�UOup�?������4�$���~E��Y$@����geIv��e�%yzz���PHu��)������|U �{gvW��[PY����xG�3q���e�x9HGj#s��2�3���uy�z��$C�2$��FV�3R=�t����z?�d�^�I2�e��T�zrue�
h�3��I��K��*�T*:H�
��3��	{e9l0���8/�|�1�U%~�o#4��*{P���`�5�����e� ��N����A�i�q���B50$�S�r4W�<u1|�2��E
_����F`�S��@x�I�u�8����l6�^nnN�zRO������i2�j�e�c�j������?O�?_�&�>���_>m���|3�:?����~/�|<��a��7��:@`�ZC;o�&��7)/���$pF�w��k��U�n������l�z�W�|+��>���k6�I6���������[u���"k��M�%t��~G���%���|d�<��H�5U��.��k��7���q���-h�G��
��+�^k�x�e��J5j0�?�*ky������������^���������D���V�P�m�
]V��2x.k	FO�MuWk��V+R��D�hr&@�[�M��D2��Z�u-Z�Df������
��s�4��x�*"�[���7��h�J�� ��	]b:�'��C9������T.��imG�(!�z���-������R�����&�H�P�������xw<�Q��N�g�5�A�t��%U;'�s�k50>*P����Y���.�_gj�}��RkZ���/� ���������'t#��l��<6*� R�)f 	���h@{Vqk����E[���>��I����\&�cq�����(��L�����Z�2���i7��%�7��M2����$N;m���C�(���aAh��>"��VM����RI�:������.%:�<��vLx�{�fck+�����������;K	Ob!�	�E��������7�����
>�P��1,v�a6�'�nC����\��x!���l�U�r��'B��� �������w�N��1D���������T*5w���$1�d����eY�>�����$b���B_D&6<���)��j�ct0|��E@,H��B�����GQ���H��@r���G��,�=�A��#��3�h�;,U��-S&��#�:����z�#�Fv�RF)^�\�^]AT�ZS����-���u�9V�b��%��Ev��,��8����v����e�������2��"v���d�@�������8�8��!`�*�!�]�
�6>s�G��B
bEo��f��-0j��R
-0*���v�R�@=�o!Y�x�����?������W����������W����7�T
����#�F��g}w��v��K�'�����O���?�p�����{E���<z��^]]�.�_8MpY�p��]�T������/�qTa�a�4y������U�K��4����
��������!m���V@�0R,���p~q���=�R����pDo�/J�D��;5"B�[�vW��X��������S�v�o����=���K3� ��!>a$���
Tkr�t����Uq�'�������g�����)�e����xaUB���EL�z�X��-��u��f�����4�+s��#�� �8����;3��
����S%��$_&������x�i>���	��SP�R%6bz��
��VC��#Q��������i�>H;�B��#�a���+o�iw����uJ9!�4����T���	�x��R��|��,�0j'��*��q����r�����'��,�lM�G;kq~cGQ�'�8Q��L�5A���������)�3������w���OG��N�\����S7Qy�>/N&2e� �C�j�=�m2��
;��T��B�����g�n;�f��r��C��4��I������8\3�h�m�69&Y��;a��W��5FZh�6��F���M55FAQQj���)4qQ�s-�4Q]�*b����fyU��i�D3q`!X��!���V!+">�|���_��o��� ,����&���AB)����`L0�� E�����=���t�.� �������3�`W9k�]�v>��4,��4���J�
!���e	�����'�������D#%�:�,�x�~'�����u2�Jfd���T��m�����e���������v��#k���j����03g�	�x/11��	���5��p.j�c
F��/KP��8�����K^=�v�(f7z����?��C���0�C�������R53Yn���[l ):��K��	X�r��y�^�}��]�{����-[�7Q���*��������5a!�U7Sx����3Q=��c�����T�4Q��(�m����`
��Y��Rs�t�����=N��*^�A���{�Z8@/RN��h���.|�:�����Z�������U*J��$������H�)ch3{��� �u��Q�b0%V�`(�K����\H����W��h���e��T�4t����B.CC�R��t����A1����LKKI V����sgi���pJ1�+��$�H����7F7J�q�.�}A)i���.x'�"B���7�D�J�N�)���;�S&��(���C��?U�t�./%�� �3x���+ 1�HP4�r����S�H�����\�-�9R�C�/�.j}��Z��n!�`����i��l[&�� ����;�����)����j��T@�wN���
�{V�K�����|J�	)��al�2�8��N4a�u��2m��yi���%K_"���eO/��.�r`}-�g>���<�q�Bz��$�����$Kk�Z�����pO����D�2��d��z�m2�6O�u���Kz=��#��yW���"�^��|�����U�_j��d�'�&�+:���!�9��"u�#�S����4'i�3��w�L����R��p�'�?�o:�'\s��B������FC	�Q��)�pn���cP]�RQ�{*h}$_�E����_. �D!$����*.�bj���%D���s<��m�'���p��	��$:�A�P�;���@$t.��ip��m�Uo"R�v��
�vHM�@�^����^���������3��)������C��xO��x^3�5
A���@<:<Q'�LYbhJq���iPr��-tA�=$MAL��`���T���<��f��
/P� �f�Y���7oN/)�P��KR#��ee{w�Q7E	t�(= J��`�N{x��i�d��a�Ov;�d�`CO�o��
���j�p�[���cfz�L���L�������@O{�g�=k��o������i�f;���bu�����'x�T�u[]�,�+��9K�2�Z�����������b*l������D6.�d���;2,����}4c6����f�s%��O��|���d����#Vt��fp��)6R���av�����]�h�E���bd�!����t����������gV#w�:�3�vw�
���Nfs�U;�n�[YR?�8}�>>;1Rnn��t��#�C4����^�1��&2�r�����JB�},I��y�r�IAQ����c�4��r�1o��v�>4Q�-z���G����R%ti6�.�����-������Ay�?�o�DKg�s>�\��x�+|u@^0x�
k0��+��&��'���7I���_TQ8������{gyv��o��n������{{�������8F����"���Q�����DI��L)/�5�e�X)���<<�N�{�s�&y���#m�~�n�=T����Gj�%*���:��sT0��0��L�����yR��r���.,uQ��c�	����8��K$�<�L"�"��#��gx%�>b���G���?���������.�����]j��d�5�+^m���i|6�W��G��'�
=�����-�`H��&!��YO�cz`}��>XP6�DB$��X[��F��_��=4rP����g���F��|��tVh����Uc{�����[[;�v�iu���\��4������_�3Z!<�lNn`m�o|y���m���4zO�/Iw������yVr�9��'J�S�y�9�#�����#|.�P�h����C�t�}���}Ef5�7����GOoE��������#m�1�p����I���=�����Q4L@w�fCQ����q�"�����Rl^ �0���P�Y*��G�V����80���������)%�a����K�^�J�=�O�� 1�kb�Q���<���i��I��f5=4	Dx���`����zf"R��G�wP�����B2M�8V���6&2$��I(&�o�_�|�f��YB1�����������5���a5�8�2�Q�0|�/�x��N���5���jv�H�"���;��Y�5+�t�l
?U�V�~�*��T7m��RV�;���� ��J(��R#���V�����"�Z���������X�����e�z��=uf�`�D?pk/���L0����
���*��Y��zH����!�R��Gm�%L���#��qjj��UT�������^���#)>�+N�%9->�<9p|WC�,�@��n���V��l�6�����Gq�2N��j{�����U�8�i� E��/I�XE~jof�@z��y�<w�&�fO;Z�&�o�#�l3�<i��G��u��wb��t;��vca����w���j�DA��2HF7��������	�{E����08���B�%���j�����,����.rN�Sv����u����6*8~1��
O+��w{1��`#��L����z)1�v-�l�oO�L-�`<~���1�IVz�=T����lZ�P�K��
_��cuL<����8��W�����d`�0�M��Sl|~���z�T�p�9�0A����CT���3Z�. 
�������F�V���\:H�37����:
QH�3n0���s�BK����h{���-�������5>�����"���������6������n���6���O�A�
��!�@�#���������__E��o���j+�y�GE3����;���N/��N�)`���\�B�=���M�����E�_�k�����.g����	a�2�=����)�q�C������}�
4O���{�v��jI�asN���������U����fG�)d�^�Z����hz�f�0�_�����Q����"^�&����Y~�6B3�F
]3���k�e
tHJ��m p�^@��oO�wH��0�u�Y����YS����]~��l�uPm��3���S���K:�/���W�l<��r��)�y+d���;D��T����
��`j�Xm(Qh�uq��:AA[����5�_��,HE|FLJA�jb�������)j��k�~+���t
�������i�����o������50m��,�������!���G`�S��A�g�x���Fb�e��Al�6T-���z�zIB�D����Wz�oN��LQ�C�>������&�S���3��|S��.gp��!��mv
��a�V�dv:�\gwW$���$N��t.�����D�����c���S'����J�oxWn��WD0�T���e[�O"W���_W>�I�|%����E�}/	wf=����71Y��eGL�@I�}�3�RZ��
�*>���>A$�@�SR!�^���/.�{-}�(��������{kd�hg�|4�Y�/K�chE���U�
�h������<��X�pn<�nnxd3�Y|e��fk�L��F����{��:��Bc@I5'%!f����e������Z�l���qz���'�D��H#=&3��=�u��h���}�p�����P�*�[� �Y���Vu�����j�$���fdIKIPd:��Lt8����3*@ V�XqP�P�8k�d
|?��$,��L���a<�C�vvtIG���.�r����t�����ij��R#j)_�\y��a���#]�>�^*�Hxp+K$����@��=@�(����5$���<��W<�e �/Fk���,b�"��#7.F8�����@a�������I��+��&�o�{�<����j�E����n��t�w��3��zD�x�����Lfb	�H�!���f�F��q\��V�$��2�#�N q��(3�j�o������'`.1o�4��~7�a�38�������	��R���%��'?��1D�pE!���-66���2L�X=5Lr�;H�����!83�-�[��1�)�L������D����[m�<���)E8`|<T�2z��*4R���<�Zi��:WD�x.�uJSk�*���BWXRI���F������)y*�(�`���B��M���X+
��f��+/D�����.��.�FA����e� C���(?V�%R8�@�Z��"Y���UP�#7���a^��0`t�c7z�S�Vo6'f� =�GVfL^�n�JS�����S����@�h����z��Q�"�$WwH(�����Ak��{XpY�S���Mz�bU��
������!_P �r ,|CGQ����q&M��x�E��!Y?\����+��:����S�I"�D3�� ��B���j�WV������=�}{9���5;��.�N��p�}�E��Md��!�������3��s�aP��.���>��|�,u�����i�����f�^�,����&/H���p��h��yA@{��*?7�����y������#[5����_�H+/���9����L(zBj�q(���Fumlh�)���DgD�H�]����
�x��Ho�Orf6�m�����>�BC����![c��55��% G�B95���
��Mi#�������dv������mN��0<�l}HQ��Kgk��)��������q�k�k����W�v9�-���&:���}g�����/'"e�q��*ge�52�jEg!�<��o�9{��2AL0�	�)����wT>OG&��W]_�
���Z#~
�F:u�D	/$�j^)-�&��aZ)�l&���wf�IM�i��Nj��<�L�
^�^����|6���M�w|����suQ�a!O�{���\�w:��$��4�\�}�H�;� ��7�T1�j�d8`�
��2N#w:K�?�</��,�`g�;����3��0D�����������L$��P�Z����#�B�a�s�&'N���_��`.��YO
�p��>�e��r:�i������!+�����T��'�j�����'vq]H�I	:E�Z�N+`�"Q�JJ"�$�������\�Y|�Q���=�~��I�z����c��f�pL��=_������E/���0�"�q&��H�Y,�s�4��G���r���b��Hb�d�����Gl�v�@5F���/"������X��)K��Z��iWA! ��~���Vq}�=J�f���hf��Q�v:)�A�duq�HK?�]�A����+������%]��1'H[(���@>��ky)6�A@����Ba�CX��-���'����=�S���k��-���&3��,K�Jx �T�����{�%�2�31U�Q?�I��
��>�JR�T���V���6w���^�`�d	XYu�@���-P
~�����%0��[>���YQ�P�`�1�I3��\m�D��HO]�?>��(�Gx���t�q��d?mK�P�<mx��l"[-���4������ ��v@�'��G���Z�>��r����U�M�cE!�B]R%ykT������N�-
jM�P����XrF��`��==)�n��b�b�}`�X.K'�ui���1�D0�8�ao\7���k�e	��Q���bL�-2a.��f�O�V��OtR|���p1
Y���W��@�;$�.�����H�l�d�j\U�?���x��'J]|���E���xO�Xc�6<�&�B��3t�x�*��Aj��x����DM���T����������=H����h��}��HT)��E��j��%�Z�xaDP��;��������&��3���O0����zS�X�*|�����B��_2���-�$,� ��e�%���X���������g���s

����LS ���AM=�����b�F�\6`������X����G�c �����d�����;I��U	�k�2J��%S��P��_\�F�}����y������8�q:E��yvOg�'�J`�����Y��z�Z��=�&�Fn�Px�P7��E�p���S�vL1*,k30}%��?�1���JukRr@�>t�����>YS��e�o���?Si<�!S�>����'���N�����������5A/G,8�y�1�h����J������? _���k����u�<�
�D)�e��(b w��WJ ���
x���Y��}��i�O���d�������}����Oe{O��hT�t"y������}���5������?��
ov�����{����f��;w���7#��
	$rG�p�i�D�Q���3c�����#CQ��h��Q�N\�C��YvI]�MzQY�'��ESt�C�C'2px��S.��$R&��Y+����Jl1��T�1�\m�yUh�&3���:�9'�����!�9����j#�����^c.)��F�f=x��N`>Pk��k������N����+,q�Y��S�m�'���no�uP�w��Vko�q���+d��6�,��X���W;S�l����x(�=Z[�����2K
n�J.��������j��m��i�5h�;,��xE0�A{B$�*X���N�V��$'*|%}h�S��]}�9�����N[��V��w�E�s�)��=�U�cB�R��n�V4*}�X
�(a�\�\���0�W�Ye������[��
����hV�z�������
.�3�C���OK�,�1��^s���������~k���jm5[qP���P������4v";��G�)c�=<�e�:R����8�c����������������1�[�&�s#��IniC�$�k��:Z5;������u����e����������������"�_1���1#@����j��~�RT������]P�"@PrM����`�������>�w#BJ����G��P�J�@ \��S�Y��9�d�Cp4����I�i��1��)2��NS1H���nR���1��;S�Jm��'�1m;�QJ����1��=�!
�k�����,`��9�����-�h�5���b�����Fu[���d��i�C�Q����L�6Uzo���4� ���������d���`H��C(������x@W+x����~A���~���?�H�n���?�c�j�Z��������-�a�t����DS��]��R1M��Fc�v�������@����j��M���HS�a�z�^�}4�jE�����o�E��fk���5���[���������j�4�-j��[��5��2�����T�~�s�L#���o�yBz��I����$���w������n�7�v�����z��$:W�U2��{Q���?j�=\0/��[E���2��~����g�d�*z��+R���b�����iZ����h~� )����f����U�����S�e�����v6w�6Z������� ��~����PM��9��������${�]�G���`<����\~�.�{�>�-�EQF[�7��z��q4s�O-&���Ms�����JN��]�L>���:���_FG#�i��L����I�u��}x��f�����D��n�$�n�&����_f?��&��������$���n���_~�������7����?��������T���~����d-��I����q/ycv�n��_���hc����V`�z���W�B��zc���W��[�Y�����pk@y'����TwK�����T��y��Q|o�*�a��O��Zq���v��a^2��l'�nk�^�?�w�{q�S�D<-�C<���@-�rc��%5(�#�u7A5����z�_��������$�}O���oV�u5SirXT���;�	G��}�J_F�-���[�J2
X���KlS0;;���>�������
d���}_���c�_t�z��QZ�c���� �o&p�6�zr���������@��1����&�
��Cl�?w������]4-�%y.����
�;[��^����lo';����
��!-�	���U���"��1����E�$�m�,����9f�H���^��Z����X�|!]�/��|vO�Nu���|����Hp�N�G
�"4���fq
O����	���F�9:0�XWI?�������H��H��r�$+\d�:���zX�d
+��:z���REf����I�B���F�jP�+P`��;-��J3�sb������&��,�V
� u1zd��3�d�T���� �,P��4���3���~2%�>�\�kD&�����Y#7��)�A<#@X VB7A�6��Q��1�.hL�X��h��D���#�����p(s\:��}�n����u�$9;������Q�}���d8�:".����tp�dCd}%s���-3r�����x�/z+`��*�3v��2����Y�'�a
a,m�xFq���T �FI&��q%YZ���P��tbX���i�;HE����5�#0l$����C�(3����'c%Ea�G�+Q,\���7�����V��F/����D����kU������w�[r�?I>'���o���d	����n��0!�8�s�!V���T�Q���Sm�C��Fu���!!'�X��Se����%Y�KhS[��M��w�w�x�z�BE��BFp��P�hqo������c"���Av��:5��J��?�uz����������Z�m���)3�~2E���)%��74��Pv������b�� ��9�x���aV5����"H���0�"8����!|�h������/�~�w�}FQS��d��Whp��]%��mT�K��G$��"d���2�d�o��2��a��*�6���\$J������d��x2�j�D����1� z�����*A4NA��IzxX��{�����lxc e�9�2��I���M�(bw����)�=������//��^t_�=��T���B@'G�G�����pZ�Ad���<AU�$A����4P��m�/�Mf��a�Ps4K��|5��y�aru�J�+�t6�\���8���SX��Y������H/���X���"���:L��w*��L����fd����A`W>35�
Y�s�tH����+�rmRp���x�[�@�@'L"�u��IE����C�#�G�����)���5�B=��a`a������	��P���]'��+%eC��v3 �
+���N�I��������v�/�;���zGh�������.�d���Y�f�N�����O�)�,Q"*S��F�M_�h���'�!�S�uO>�'n�%)��g^�&����)���Nl6��pW����.(���b@��G7��9�M�W5���4
�33�o�
����V8<�o
���7R�+B�.k��3U��A���X�����XBt�|�������c�y4�]L�A��^�vCp�J�R���ll�\���0������!JV��Z����� ��5�������������d��`�6�G���r$�6�u����*�����s�W�,�xh�����p��^@�.��f��Ms�)!�=�d�{�������fAS��#��s�N!�u���&��_���3�6R���.�@��
.�;��/a�)�L��.�M����'�{|�����p�����Fa���~���Bg�$yT�V��x� t�#EBp��3N����%��L5q����m�F�~1'����z��p���,Q�����)Q#�@�]1`���if���#���$���0	HF��pS���Tq%�t����4B�^dv���S]��x�����s��/x���K��|�K��X(c���p(_��.�ttc���~O��=W��\�������*:�Fg��N��(7��Es����� K���qu��b��C9�D���������d���z�6qG+��O�d&2���Kn �2�
'��b�kJ
����H�I��t�'8��Oo����<���k�6FL}s��N���
�������'�y
S�����,�d�a�����Nf�e��C�1����3Bh���,�!$����c
��C`�� P���v�0O�G�'a�������5�G��z�����]�/�d1-#���r���d��0��'������,��$\�i�D�d��@��z�����BN-��D_�D��h:���B�e�_Ke<��, �_�.Y�D��u]�!"�O��0a��Q��PE�J�R��P���D}�,���c�x�C�j.�(�Y�:�]E�G���	��@{�x��?�tc1�HI���gc�HCoNA�O��C%}�C���
��������s���2��^��X�l���x�A�U�C^f��<����!n-�/�K���vv��S�p��$�{s+�h���p5��>Q?��=&%�9�}9����Z/��F�1�
R�j�����$� �\N�zgNO�����q����NBKf?Ypn���X�����c���������QQW|�[v��#�����G������%�0Z�So 
���(���v]p�-������H���#h%�}#d���]x����ODK�q"y����cz�����ST]�7#NeL��R�����pg���x"��k��pwW�1G:]�_�u������z���}��2�	<�D��a���p�"W2���$�^94��4���P���p���\v�V1�UQ�3�Yb1�^���x��.F��&�y:���vc���cFy�&��#�q�}���$]Y��x���u@����� Q*D�Sa��3BT�
$[�>a����R�b~=���E3����7��4l,��aB�8����(K!�,V���b�
������LuX���!��x��&���%�Io<A;�m2MDGV�R�[�>���;l�z2�
s�����={��x���$ZY[���$��uzJ@�!�:�()���i3����Gg
�<������O�V�(���L	���3~�r�Q:T���D��� `z�Tj��^�r2�:�j��*K�!���a�
v��-�I���vT�sYUL������{e�d���eF�	������^,��������rw����+��5�zZEp^E!��#���Q9����)����L�E���n���
���I��)r������(B�����@<W�K�)�:�h�C�6�	���ws���v��I{�r���#��w%��?%x���s�@�C��Z��JHf��8�R.��/�b[g���b7	�����2�P1��)��iM�C}���Wr<���{ml�wH�F��S����TOz��s�h����8h�y��V�L�b����4�S��s�ZM3�~�5�����Cb�h��@��2	����&I�N�
T��a������F��j!�2]%3�h�)�3�;!?c��--09��3����w�W��{���u���/�W\q��`i�<b~���W��,j#J��)���2D����j�O�H��5
}S�P\'k�
x��Du�W�4���}��[k�6���O����v�]B��K��C��#]���CYE[��L�{�[j�A�k.N���4��?��^x�8��	*���
����%*a�X	�JjiKg�>s5B���	�WJ�>��g���3~-x���p��o�tfd�|�jT[�����Vu���E��j)�m�'L�b���L�Y�	V�U��{��*�R(�������]�11���x��9D�`�G&m��99�����4�p�U�G�X�[�q�NT[Im�@i�����[�
���w�a�{Sr!����x����.*��������d2�-�-���G�#b���8�Ey/��.y#�����r^�������:��:G�8ii���`_8Z�}���)�t����r���������k|�*�JyT������2���Da�BV������v9JA"N$�=��3�X����d��FS6[�CQ-���z9�����r5[��u�������;<����x�{�)!TX~�@���J�	e����D}��!e�'�`��C�Q�-���zl+E�t�+����!��&/P��*�Y�����*�[�������0�����`{:|���i�`	5!�3���2����4;�wm���Y�+��A[b�L+�����~
����2N1$�,Tgy���
����<S�k��z���@-�
)��O�c�c����U��z�"!I�Z����Wz��:�4&��av[���q]����Cz��IDCR�c]q|MGks�����E�����������X))*����S��FFF�����L PR3!qI�a�XA�?K�7+���������/��U�N����}b

x��aE�u�8�����z�i��`����~B��U����s7����K��������]�[��/�)2���g�
~�_v�0i���Rr�@\=jr���A��������e��$���w-�F��M��'�7���Z�zB6Y<�����B�;���!�l)�@�W�	���B�r�a�
���Ov�^���6N��Ra�����%�%T:�%C��y��8'|�yY�����G��Ddn��,-P�%<b�V
�M�Bf���	�������Tl�7
yk�p�	��G5@��{�������QYAQC�����L2��y���H�KQe;�]���"����@�D���	#��OF�:�<����|���NM-�@?�`�����*��}�������e�661��mCp4���Q����	%��w��T�7��bv�9_�%I�3���{�d��l��a��D�#�0��c��y��5�*�i�/z�Z����g��x��Y2��C�a��3������8����b�
��Dy{�C�=���Fq��Ar���q��5]���d��6�L�7�NI���}^�r�2�#�Bqb��(J������g)�f
���/B��^
���k�Pe�Q�mB�p�&1L�FW+Z�&���������b���#���*�E]�6�5���=���l�&�q^�t��8g����	ui8,��>��"�������t�#,���r��Yg�78�%����*Xo�:�@H��7sZ��<VJB_�+���sJXPO����.���V��!�z��(]I2�3�k�O�=WFcX�J�c���]�r��c������8��g�(��(v
d�	G�KX�g��wd�+������+�����vD�T�9��c�Pk`�����R��i�1���&��,��]���4@5�K��F�.���
,mqr�����j�L�����<b�I��{�<�Y��
�����K��fc��@�B���kD^f����0�n���EK�pr�I��:���������9��:�k�{^����[1�����z�c�z
�����V+p�V�ZSQ)�&�QZq���xw�
o���-@5�:Z����p!E����~�G���0�+.}��"w�\Q��0��%/@k�8��u���������Wm�D�����v>���Fr����M���������~�S�G,	[}���N cRds/�mhHc{4�@[�h�Wu�Z��q\�C2B�n�6w �F��G���������p�<�GU`k�O����7\��:�������;��L#A�
�~�w:�4)x��_�����ZEz����e.`Rv�H@�(���""A	h7�I����G����b�a�W������6!zK�f����x��*3���S��M#�t�V+H\�����xq�.��W+H�F�����R���o!<�'�`��5q�f�5�%�a9GJ��zgF��T����v��C�S)Qs>���)���&(������/;�_��7����7�����EK���--���b������Z��-�#������'N
���|��}��T�	������[b�G)�pC�U9�^��v]����8�x����=�f^=��w�����x8I����G����UAg$v��J�SC	�����z�9����A�)�������
����N�w��y����O��� w��*'�Gb�����nw>�C�������RX
N�5�<�� 9f���6 �8����b��fC��:x$�h�E�pC��!Th;QY��$@
�x�7<c�-�(��R60��T�H>�4��ddj�
��pE���(b(���px��4X�\�bP����-�������\�^�G|N<=���2�7r,�=	���X��`���
��`���5�t8�L+�:�P#+D���V��M=��G��N��9UDZ&'=�Q�j�J�w}����W��O��0��]���M��lW���7�;���\
0�g<�[��W�����;��>lo
�&�t}����w��Y�o�?(kA�[����km��{��Y����9�"�"C�!�Bd9%����0fL��������5���/d^a��);AO7#EQz����$�*A"{����9��N�U��=���������<����A>f��1��8 �����f��NMJZO�m���"�)�Z�����?��=�I����F�V�A��wcr�'�����>�G�40���:����s��m�Vw��5?���?����J�r��A�N�3���l\YNStC�=�R��#��<Mn�3e����Q��,0
�$�c�Z� ���������`���<�eg
�Nt,����4\����\P���}:��������?dK��I�p��Jk�����vh������� �S8�hSA���|�	�OR&����S|����}-���d�s������	�s��Z"��*����-�s��T����7:(}�R���v����N��z�������6/{g���U
�r��YA`�3i���s#Q[��S_����q��[V&��c��E{���%�j(a6CB��xD��bJP���/$Iw�����S�s��:��_<|����G����D�-J�c2����jd�pVH��9��{0qE��`�y]P,NA:������R�B(��Q�^?���Ua����[����rx����2���4d,�w0.O������]���=�:5�BE���y���h���_��t�U���%�[���u�CZ�F�j6v���uv��3f���'g���U���]Y_�������~�^\�2�z.DRy���?]\��Q�<�)�s8�E�����Z���!>XJ�|��)��;&.{��)Q5�q���]���rt�����4 M������Uss6f���L}D���P�����C���-$m����K���d��z�42k������'��E`~�<��R�h|�O��+����AV�?�����M��S���m^T����J���s�;�t�q�d���P����
G�j�A���o�%�Mgo��K������������
�-@��#"k����A�Ef6	Y��"�����keuWS���_6���kM���O��p��R1�!o�����^�ry������#�S�?'�>D����s�$P�0�����mi|�/�j��������k���&��l��,w�W��*k�q��[|2������%t%y e��"�6o���N��8�@�,o�\��'��j���]U��tM����|(�v���y����Q��:������l"�+�g�8;��v�Cq��c]_1�kz����^d���Sk�sm��;�L�����1�YT�F�t�p�p��4��7��V�>��f*������g6�����#���\���`����L�}sr�s������C������v�!oa��#o`p����`n*b*��xB�Qg@�8���M�T�Fi��c,������`�!7�v��W��t2�"�rEq��5m@���)�
������w�Wh	*AP�P���@�|�@hX�+B�s2�
�aWq'��9�S�_i�E�l6��n�f~����raao�}�{��������E�����\?�h���b8U|�l�Pa�bWg�B�����p�uJ�����F82�p������0�2��E��Dn�����S"�������@��n�����e�X��x���9�}x8� ���������)���C*&���@��;���`)a,tt8pgV��K���M��r�-��B���\0}NJb�!����e�W���?"��l�]���b�%�V����8f�Q���`��E���N%�{��;D�mY�
�of���t���3��"��{,�6C�=��"��]���%�Vy7!������F�n��*u�1����Ty9��'|����#��x����B�qH�M/~7�!p�����k�u�B�����>q��J��FX����9[���-K��,�
@ �q�����d�t��q�����������^�J������������'�����5��8�\�wPdbX�����C���`���������/J2�+P�
�2Vl����f��'\b_7����B������I�B"�#w�M�v����A��b��������W����P�)FnW�hrPA��b��LF�i&�T����
���cz+���[�p�Zk����p����Q�!�O�)Qdi��B���Dlb�/��9S;��9j�t_�M�98Dc�xF���M>�P�9�a<�%P&���))r����_���:�>e�O�0��^�����Q\����!�Xv�l�KDv��~�"{��C�F��o�~uKe��[
�Z&�@��a!�*�L��r����2<����;����*�M�,60���*���Z��;7�+_t.�F��.\�c���'�ADL��<47�c��z�S���I����I�tDj����-�0��r�^*���Q�
��5������9.?z�>y+<����O7�����p������3�G-����1~k�����9�(t�V@�EuK��~t��xdOAH����]�F���/1�Z7�m�V:!��K�h��Q�g��N����S���������#���
:�[�,�C��HW}�����H�a(T�'\J�C�����T*��~V\��Y�i���]B/�-(��Q��N{�h-�'�YDOr�y5�}����q w��2�-t��<��Z�4pf��"���"�p��)����^��:��-��G��V���J�<2Fh-�j�"p��*�k�*��_.#��h<�XM_m������W��.���|�]c���b���5w��.�����e��I������k������	����_/ix5�����|j"_�� ��z����w[���H;'��,UID��n��-b��V�L��:���ymy���4����(��f�u����Q�v%�6�iw��%���|��M�D��*�F����^�%jm�$�q��=������~k/�tw�tw{{�V��������j�B,xsk�m�� <�s�F�4������K<��^N������ �e<E�z�
�2�e]��zY�����m����B�2�I3���qU^G�3�yj�So2hL��=��_\��*�f����W�Fm��RT�\
�H�E#�i�����^-|��!k�0���Ou��>)Mi���i�>�s��H6%�&)e����������^��m�o(L�uzI��i%yL+�d���
����G�_�-��^��l+�mk�P�k������Y��Z^���������G2������T���W�*`�o�M�,}p�'����7^���Q�0��"��h�Uon�[kk���y�<}�>O��������y�<}�>O��������y�<}�>O��������y�<}�>O��������y�<}�>O��������y�<}�+?�A�Fr(
#15Robert Haas
robertmhaas@gmail.com
In reply to: Thomas Munro (#14)
Re: POC: Sharing record typmods between backends

On Fri, Aug 11, 2017 at 4:39 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

OK. Now it's ds_hash_table.{c,h}, where "ds" stands for "dynamic
shared". Better? If we were to do other data structures in DSA
memory they could follow that style: ds_red_black_tree.c, ds_vector.c,
ds_deque.c etc and their identifier prefix would be drbt_, dv_, dd_
etc.

Do you want to see a separate patch to rename dsa.c? Got a better
name? You could have spoken up earlier :-) It does sound like a bit
like the thing from crypto or perhaps a scary secret government
department.

I doubt that we really want to have accessor functions with names like
dynamic_shared_hash_table_insert or ds_hash_table_insert. Long names
are fine, even desirable, for APIs that aren't too widely used,
because they're relatively self-documenting, but a 30-character
function name gets annoying in a hurry if you have to call it very
often, and this is intended to be reusable for other things that want
a dynamic shared memory hash table. I think we should (a) pick some
reasonably short prefix for all the function names, like dht or dsht
or ds_hash, but not ds_hash_table or dynamic_shared_hash_table and (b)
also use that prefix as the name for the .c and .h files.

Right now, we've got a situation where the most widely-used hash table
implementation uses dynahash.c for the code, hsearch.h for the
interface, and "hash" as the prefix for the names, and that's really
hard to remember. I think having a consistent naming scheme
throughout would be a lot better.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#16Andres Freund
andres@anarazel.de
In reply to: Robert Haas (#15)
Re: POC: Sharing record typmods between backends

On 2017-08-11 11:14:44 -0400, Robert Haas wrote:

On Fri, Aug 11, 2017 at 4:39 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

OK. Now it's ds_hash_table.{c,h}, where "ds" stands for "dynamic
shared". Better? If we were to do other data structures in DSA
memory they could follow that style: ds_red_black_tree.c, ds_vector.c,
ds_deque.c etc and their identifier prefix would be drbt_, dv_, dd_
etc.

Do you want to see a separate patch to rename dsa.c? Got a better
name? You could have spoken up earlier :-) It does sound like a bit
like the thing from crypto or perhaps a scary secret government
department.

I, and I bet a lot of other people, kind of missed dsa being merged for
a while...

I doubt that we really want to have accessor functions with names like
dynamic_shared_hash_table_insert or ds_hash_table_insert. Long names
are fine, even desirable, for APIs that aren't too widely used,
because they're relatively self-documenting, but a 30-character
function name gets annoying in a hurry if you have to call it very
often, and this is intended to be reusable for other things that want
a dynamic shared memory hash table. I think we should (a) pick some
reasonably short prefix for all the function names, like dht or dsht
or ds_hash, but not ds_hash_table or dynamic_shared_hash_table and (b)
also use that prefix as the name for the .c and .h files.

Yea, I agree with this. Something dsmhash_{insert,...}... seems like
it'd kinda work without being too ambiguous like dht imo is, while still
being reasonably short.

Right now, we've got a situation where the most widely-used hash table
implementation uses dynahash.c for the code, hsearch.h for the
interface, and "hash" as the prefix for the names, and that's really
hard to remember. I think having a consistent naming scheme
throughout would be a lot better.

Yea, that situation still occasionally confuses me, a good 10 years
after starting to look at pg... There's even a a dynahash.h, except
it's useless. And dynahash.c doesn't even include hsearch.h directly
(included via shmem.h)! Personally I'd actually in favor of moving
hsearch.h stuff into dynahash.h and leave hsearch as a wrapper.

- Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#17Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#14)
Re: POC: Sharing record typmods between backends

Hi,

On 2017-08-11 20:39:13 +1200, Thomas Munro wrote:

Please find attached a new patch series. I apologise in advance for
0001 and note that the patchset now weighs in at ~75kB compressed.
Here are my in-line replies to your two reviews:

Replying to a few points here, then I'll do a pass through your your
submission...

On Tue, Jul 25, 2017 at 10:09 PM, Andres Freund <andres@anarazel.de> wrote:

It does concern me that we're growing yet another somewhat different
hashtable implementation. Yet I don't quite see how we could avoid
it. dynahash relies on proper pointers, simplehash doesn't do locking
(and shouldn't) and also relies on pointers, although to a much lesser
degree. All the open coded tables aren't a good match either. So I
don't quite see an alternative, but I'd love one.

Yeah, I agree. To deal with data structures with different pointer
types, locking policy, inlined hash/eq functions etc, perhaps there is
a way we could eventually do 'policy based design' using the kind of
macro trickery you started where we generate N different hash table
variations but only have to maintain source code for one chaining hash
table implementation? Or perl scripts that effectively behave as a
cfront^H^H^H nevermind. I'm not planning to investigate that for this
cycle.

Whaaa, what have I done!!!! But more seriously, I'm doubtful it's worth
going there.

+ * level.  However, when a resize operation begins, all partition locks must
+ * be acquired simultaneously for a brief period.  This is only expected to
+ * happen a small number of times until a stable size is found, since growth is
+ * geometric.

I'm a bit doubtful that we need partitioning at this point, and that it
doesn't actually *degrade* performance for your typmod case.

Yeah, partitioning not needed for this case, but this is supposed to
be more generally useful. I thought about making the number of
partitions a construction parameter, but it doesn't really hurt does
it?

Well, using multiple locks and such certainly isn't free. An exclusively
owned cacheline mutex is nearly an order of magnitude faster than one
that's currently shared, not to speak of modified. Also it does increase
the size overhead, which might end up happening for a few other cases.

+ * Resizing is done incrementally so that no individual insert operation pays
+ * for the potentially large cost of splitting all buckets.

I'm not sure this is a reasonable tradeoff for the use-case suggested so
far, it doesn't exactly make things simpler. We're not going to grow
much.

Yeah, designed to be more generally useful. Are you saying you would
prefer to see the DHT patch split into an initial submission that does
the simplest thing possible, so that the unlucky guy who causes the
hash table to grow has to do all the work of moving buckets to a
bigger hash table? Then we could move the more complicated
incremental growth stuff to a later patch.

Well, most of the potential usecases for dsmhash I've heard about so
far, don't actually benefit much from incremental growth. In nearly all
the implementations I've seen incremental move ends up requiring more
total cycles than doing it at once, and for parallelism type usecases
the stall isn't really an issue. So yes, I think this is something
worth considering. If we were to actually use DHT for shared caches or
such, this'd be different, but that seems darned far off.

This is complicated, and in the category that I would normally want a
stack of heavy unit tests for. If you don't feel like making
decisions about this now, perhaps iteration (and incremental resize?)
could be removed, leaving only the most primitive get/put hash table
facilities -- just enough for this purpose? Then a later patch could
add them back, with a set of really convincing unit tests...

I'm inclined to go for that, yes.

+/*
+ * Detach from a hash table.  This frees backend-local resources associated
+ * with the hash table, but the hash table will continue to exist until it is
+ * either explicitly destroyed (by a backend that is still attached to it), or
+ * the area that backs it is returned to the operating system.
+ */
+void
+dht_detach(dht_hash_table *hash_table)
+{
+       /* The hash table may have been destroyed.  Just free local memory. */
+       pfree(hash_table);
+}

Somewhat inclined to add debugging refcount - seems like bugs around
that might be annoying to find. Maybe also add an assert ensuring that
no locks are held?

Added assert that not locks are held.

In an earlier version I had reference counts. Then I realised that it
wasn't really helping anything. The state of being 'attached' to a
dht_hash_table isn't really the same as holding a heavyweight resource
like a DSM segment or a file which is backed by kernel resources.
'Attaching' is just something you have to do to get a backend-local
palloc()-ated object required to interact with the hash table, and
since it's just a bit of memory there is no strict requirement to
detach from it, if you're happy to let MemoryContext do that for you.
To put it in GC terms, there is no important finalizer here. Here I
am making the same distinction that we make between stuff managed by
resowner.c (files etc) and stuff managed by MemoryContext (memory); in
the former case it's an elog()-gable offence not to close things
explicitly in non-error paths, but in the latter you're free to do
that, or pfree earlier. If in future we create more things that can
live in DSA memory, I'd like them to be similarly free-and-easy. Make
sense?

I don't quite follow. You're saying that just because there could be
local bugs (which'd easily be found via the mcxt/aset debugging stuff
and/or valgrind) making sure about the shared resources still being
there isn't useful? I don't quite find that convincing...

+/*
+ * Look up an entry, given a key.  Returns a pointer to an entry if one can be
+ * found with the given key.  Returns NULL if the key is not found.  If a
+ * non-NULL value is returned, the entry is locked and must be released by
+ * calling dht_release.  If an error is raised before dht_release is called,
+ * the lock will be released automatically, but the caller must take care to
+ * ensure that the entry is not left corrupted.  The lock mode is either
+ * shared or exclusive depending on 'exclusive'.

This API seems a bit fragile.

Do you mean "... the caller must take care to ensure that the entry is
not left corrupted"?

Yes.

This is the same as anything protected by LWLocks including shared
buffers. If you error out, locks are released and you had better not
have left things in a bad state. I guess this comment is really just
about what C++ people call "basic exception safety".

Kind. Although it's not impossible to make this bit less error
prone. E.g. by zeroing the entry before returning.

Now that I think about it, it's possibly also worthwhile to note that
any iterators and such are invalid after errors (given that ->locked is
going to be wrong)?

diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 9fd7b4e019b..97c0125a4ba 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -337,17 +337,75 @@ DecrTupleDescRefCount(TupleDesc tupdesc)
{
Assert(tupdesc->tdrefcount > 0);
-       ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc);
+       if (CurrentResourceOwner != NULL)
+               ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc);
if (--tupdesc->tdrefcount == 0)
FreeTupleDesc(tupdesc);
}

What's this about? CurrentResourceOwner should always be valid here, no?
If so, why did that change? I don't think it's good to detach this from
the resowner infrastructure...

The reason is that I install a detach hook
shared_record_typmod_registry_detach() in worker processes to clear
out their typmod registry. It runs at a time when there is no
CurrentResourceOwner. It's a theoretical concern only today, because
workers are not reused. If a workers lingered in a waiting room and
then attached to a new session DSM from a different leader, then it
needs to remember nothing of the previous leader's typmods.

Hm. I'm not sure I like adhoc code like this. Wouldn't it be better to
have a 'per worker' rather than 'per transaction" resowner for things
like this? Otherwise we end up with various datastructures to keep track
of things.

/*
- * Magic numbers for parallel state sharing.  Higher-level code should use
- * smaller values, leaving these very large ones for use by this module.
+ * Magic numbers for per-context parallel state sharing.  Higher-level code
+ * should use smaller values, leaving these very large ones for use by this
+ * module.
*/
#define PARALLEL_KEY_FIXED                                     UINT64CONST(0xFFFFFFFFFFFF0001)
#define PARALLEL_KEY_ERROR_QUEUE                       UINT64CONST(0xFFFFFFFFFFFF0002)
@@ -63,6 +74,16 @@
#define PARALLEL_KEY_ACTIVE_SNAPSHOT           UINT64CONST(0xFFFFFFFFFFFF0007)
#define PARALLEL_KEY_TRANSACTION_STATE         UINT64CONST(0xFFFFFFFFFFFF0008)
#define PARALLEL_KEY_ENTRYPOINT                                UINT64CONST(0xFFFFFFFFFFFF0009)
+#define PARALLEL_KEY_SESSION_DSM                       UINT64CONST(0xFFFFFFFFFFFF000A)
+
+/* Magic number for per-session DSM TOC. */
+#define PARALLEL_SESSION_MAGIC                         0xabb0fbc9
+
+/*
+ * Magic numbers for parallel state sharing in the per-session DSM area.
+ */
+#define PARALLEL_KEY_SESSION_DSA                       UINT64CONST(0xFFFFFFFFFFFF0001)
+#define PARALLEL_KEY_RECORD_TYPMOD_REGISTRY    UINT64CONST(0xFFFFFFFFFFFF0002)

Not this patch's fault, but this infrastructure really isn't great. We
should really replace it with a shmem.h style infrastructure, using a
dht hashtable as backing...

Well, I am trying to use the established programming style. We
already have a per-query DSM with a TOC indexed by magic numbers (and
executor node IDs). I add a per-session DSM with a TOC indexed by a
different set of magic numbers. We could always come up with
something better and fix it in both places later?

I guess so. I know Robert is a bit tired of me harping about this, but I
really don't think this is great...

+/*
+ * A flattened/serialized representation of a TupleDesc for use in shared
+ * memory.  Can be converted to and from regular TupleDesc format.  Doesn't
+ * support constraints and doesn't store the actual type OID, because this is
+ * only for use with RECORD types as created by CreateTupleDesc().  These are
+ * arranged into a linked list, in the hash table entry corresponding to the
+ * OIDs of the first 16 attributes, so we'd expect to get more than one entry
+ * in the list when named and other properties differ.
+ */
+typedef struct SerializedTupleDesc
+{
+       dsa_pointer next;                       /* next with the same same attribute OIDs */
+       int                     natts;                  /* number of attributes in the tuple */
+       int32           typmod;                 /* typmod for tuple type */
+       bool            hasoid;                 /* tuple has oid attribute in its header */
+
+       /*
+        * The attributes follow.  We only ever access the first
+        * ATTRIBUTE_FIXED_PART_SIZE bytes of each element, like the code in
+        * tupdesc.c.
+        */
+       FormData_pg_attribute attributes[FLEXIBLE_ARRAY_MEMBER];
+} SerializedTupleDesc;

Not a fan of a separate tupledesc representation, that's just going to
lead to divergence over time. I think we should rather change the normal
tupledesc representation to be compatible with this, and 'just' have a
wrapper struct for the parallel case (with next and such).

OK. I killed this. Instead I flattened tupleDesc to make it usable
directly in shared memory as long as there are no constraints. There
is still a small wrapper SharedTupleDesc, but that's just to bolt a
'next' pointer to them so we can chain together TupleDescs with the
same OIDs.

Yep, that makes sense.

+/*
+ * An entry in SharedRecordTypmodRegistry's attribute index.  The key is the
+ * first REC_HASH_KEYS attribute OIDs.  That means that collisions are
+ * possible, but that's OK because SerializedTupleDesc objects are arranged
+ * into a list.
+ */
+/* Parameters for SharedRecordTypmodRegistry's attributes hash table. */
+const static dht_parameters srtr_atts_index_params = {
+       sizeof(Oid) * REC_HASH_KEYS,
+       sizeof(SRTRAttsIndexEntry),
+       memcmp,
+       tag_hash,
+       LWTRANCHE_SHARED_RECORD_ATTS_INDEX
+};
+
+/* Parameters for SharedRecordTypmodRegistry's typmod hash table. */
+const static dht_parameters srtr_typmod_index_params = {
+       sizeof(uint32),
+       sizeof(SRTRTypmodIndexEntry),
+       memcmp,
+       tag_hash,
+       LWTRANCHE_SHARED_RECORD_TYPMOD_INDEX
+};
+

I'm very much not a fan of this representation. I know you copied the
logic, but I think it's a bad idea. I think the key should just be a
dsa_pointer, and then we can have a proper tag_hash that hashes the
whole thing, and a proper comparator too. Just have

/*
* Combine two hash values, resulting in another hash value, with decent bit
* mixing.
*
* Similar to boost's hash_combine().
*/
static inline uint32
hash_combine(uint32 a, uint32 b)
{
a ^= b + 0x9e3779b9 + (a << 6) + (a >> 2);
return a;
}

and then hash everything.

Hmm. I'm not sure I understand. I know what hash_combine is for but
what do you mean when you say they key should just be a dsa_pointer?

What's wrong with providing the key size, whole entry size, compare
function and hash function like this?

Well, right now the key is "sizeof(Oid) * REC_HASH_KEYS" which imo is
fairly ugly. Both because it wastes space for narrow cases, and because
it leads to conflicts for wide ones. By having a dsa_pointer as a key
and custom hash/compare functions there's no need for that, you can just
compute the hash based on all keys, and compare based on all keys.

+/*
+ * Make sure that RecordCacheArray is large enough to store 'typmod'.
+ */
+static void
+ensure_record_cache_typmod_slot_exists(int32 typmod)
+{
+       if (RecordCacheArray == NULL)
+       {
+               RecordCacheArray = (TupleDesc *)
+                       MemoryContextAllocZero(CacheMemoryContext, 64 * sizeof(TupleDesc));
+               RecordCacheArrayLen = 64;
+       }
+
+       if (typmod >= RecordCacheArrayLen)
+       {
+               int32           newlen = RecordCacheArrayLen * 2;
+
+               while (typmod >= newlen)
+                       newlen *= 2;
+
+               RecordCacheArray = (TupleDesc *) repalloc(RecordCacheArray,
+                                                                                                 newlen * sizeof(TupleDesc));
+               memset(RecordCacheArray + RecordCacheArrayLen, 0,
+                          (newlen - RecordCacheArrayLen) * sizeof(TupleDesc *));
+               RecordCacheArrayLen = newlen;
+       }
+}

Do we really want to keep this? Could just have an equivalent dynahash
for the non-parallel case?

Hmm. Well the plain old array makes a lot of sense in the
non-parallel case, since we allocate typmods starting from zero. What
don't you like about it? The reason for using an array for
backend-local lookup (aside from "that's how it is already") is that
it's actually the best data structure for the job; the reason for
using a hash table in the shared case is that it gives you locking and
coordinates growth for free. (For the OID index it has to be a hash
table in both cases.)

Well, that reason kinda vanished after the parallelism introduction, no?
There's no guarantee at all anymore that it's gapless - it's perfectly
possible that each worker ends up with a distinct set of ids.

Greetings,

Andres Freund

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#18Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#17)
Re: POC: Sharing record typmods between backends

Thanks for your feedback. Here are two parts that jumped out at me.
I'll address the other parts in a separate email.

On Sat, Aug 12, 2017 at 1:55 PM, Andres Freund <andres@anarazel.de> wrote:

This is complicated, and in the category that I would normally want a
stack of heavy unit tests for. If you don't feel like making
decisions about this now, perhaps iteration (and incremental resize?)
could be removed, leaving only the most primitive get/put hash table
facilities -- just enough for this purpose? Then a later patch could
add them back, with a set of really convincing unit tests...

I'm inclined to go for that, yes.

I will make it so.

+/*
+ * An entry in SharedRecordTypmodRegistry's attribute index.  The key is the
+ * first REC_HASH_KEYS attribute OIDs.  That means that collisions are
+ * possible, but that's OK because SerializedTupleDesc objects are arranged
+ * into a list.
+ */
+/* Parameters for SharedRecordTypmodRegistry's attributes hash table. */
+const static dht_parameters srtr_atts_index_params = {
+       sizeof(Oid) * REC_HASH_KEYS,
+       sizeof(SRTRAttsIndexEntry),
+       memcmp,
+       tag_hash,
+       LWTRANCHE_SHARED_RECORD_ATTS_INDEX
+};
+
+/* Parameters for SharedRecordTypmodRegistry's typmod hash table. */
+const static dht_parameters srtr_typmod_index_params = {
+       sizeof(uint32),
+       sizeof(SRTRTypmodIndexEntry),
+       memcmp,
+       tag_hash,
+       LWTRANCHE_SHARED_RECORD_TYPMOD_INDEX
+};
+

I'm very much not a fan of this representation. I know you copied the
logic, but I think it's a bad idea. I think the key should just be a
dsa_pointer, and then we can have a proper tag_hash that hashes the
whole thing, and a proper comparator too. Just have

/*
* Combine two hash values, resulting in another hash value, with decent bit
* mixing.
*
* Similar to boost's hash_combine().
*/
static inline uint32
hash_combine(uint32 a, uint32 b)
{
a ^= b + 0x9e3779b9 + (a << 6) + (a >> 2);
return a;
}

and then hash everything.

Hmm. I'm not sure I understand. I know what hash_combine is for but
what do you mean when you say they key should just be a dsa_pointer?

What's wrong with providing the key size, whole entry size, compare
function and hash function like this?

Well, right now the key is "sizeof(Oid) * REC_HASH_KEYS" which imo is
fairly ugly. Both because it wastes space for narrow cases, and because
it leads to conflicts for wide ones. By having a dsa_pointer as a key
and custom hash/compare functions there's no need for that, you can just
compute the hash based on all keys, and compare based on all keys.

Ah, that. Yeah, it is ugly, both in the pre-existing code and in my
patch. Stepping back from this a bit more, the true key here not an
array of Oid at all (whether fixed sized or variable). It's actually
a whole TupleDesc, because this is really a TupleDesc intern pool:
given a TupleDesc, please give me the canonical TupleDesc equal to
this one. You might call it a hash set rather than a hash table
(key->value associative).

Ideally, we'd get rid of the ugly REC_HASH_KEYS-sized key and the ugly
extra conflict chain, and tupdesc.c would have a hashTupleDesc()
function that is compatible with equalTupleDescs(). Then the hash
table entry would simply be a TupleDesc (that is, a pointer).

There is an extra complication when we use DSA memory though: If you
have a hash table (set) full of dsa_pointer to struct tupleDesc but
want to be able to search it given a TupleDesc (= backend local
pointer) then you have to do some extra work. I think that work is:
the hash table entries should be a small struct that has a union {
dsa_pointer, TupleDesc } and a discriminator field to say which it is,
and the hash + eq functions should be wrappers that follow dsa_pointer
if needed and then forward to hashTupleDesc() (a function that does
hash_combine() over the Oids) and equalTupleDescs().

(That complication might not exist if tupleDesc were fixed size and
could be directly in the hash table entry, but in the process of
flattening it (= holding the attributes in it) I made it variable
size, so we have to use a pointer to it in the hash table since both
DynaHash and DHT work with fixed size entries).

Thoughts?

--
Thomas Munro
http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#19Robert Haas
robertmhaas@gmail.com
In reply to: Andres Freund (#17)
Re: POC: Sharing record typmods between backends

On Fri, Aug 11, 2017 at 9:55 PM, Andres Freund <andres@anarazel.de> wrote:

Well, most of the potential usecases for dsmhash I've heard about so
far, don't actually benefit much from incremental growth. In nearly all
the implementations I've seen incremental move ends up requiring more
total cycles than doing it at once, and for parallelism type usecases
the stall isn't really an issue. So yes, I think this is something
worth considering. If we were to actually use DHT for shared caches or
such, this'd be different, but that seems darned far off.

I think it'd be pretty interesting to look at replacing parts of the
stats collector machinery with something DHT-based.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#20Andres Freund
andres@anarazel.de
In reply to: Robert Haas (#19)
Re: POC: Sharing record typmods between backends

On 2017-08-12 22:52:57 -0400, Robert Haas wrote:

On Fri, Aug 11, 2017 at 9:55 PM, Andres Freund <andres@anarazel.de> wrote:

Well, most of the potential usecases for dsmhash I've heard about so
far, don't actually benefit much from incremental growth. In nearly all
the implementations I've seen incremental move ends up requiring more
total cycles than doing it at once, and for parallelism type usecases
the stall isn't really an issue. So yes, I think this is something
worth considering. If we were to actually use DHT for shared caches or
such, this'd be different, but that seems darned far off.

I think it'd be pretty interesting to look at replacing parts of the
stats collector machinery with something DHT-based.

That seems to involve a lot more than this though, given that currently
the stats collector data doesn't entirely have to be in memory. I've
seen sites with a lot of databases with quite some per-database stats
data. Don't think we can just require that to be in memory :(

- Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#21Robert Haas
robertmhaas@gmail.com
In reply to: Andres Freund (#20)
Re: POC: Sharing record typmods between backends

On Sat, Aug 12, 2017 at 11:30 PM, Andres Freund <andres@anarazel.de> wrote:

That seems to involve a lot more than this though, given that currently
the stats collector data doesn't entirely have to be in memory. I've
seen sites with a lot of databases with quite some per-database stats
data. Don't think we can just require that to be in memory :(

Hmm. I'm not sure it wouldn't end up being *less* memory. Don't we
end up caching 1 copy of it per backend, at least for the database to
which that backend is connected? Accessing a shared copy would avoid
that sort of thing.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#22Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#21)
Re: POC: Sharing record typmods between backends

Robert Haas <robertmhaas@gmail.com> writes:

On Sat, Aug 12, 2017 at 11:30 PM, Andres Freund <andres@anarazel.de> wrote:

That seems to involve a lot more than this though, given that currently
the stats collector data doesn't entirely have to be in memory. I've
seen sites with a lot of databases with quite some per-database stats
data. Don't think we can just require that to be in memory :(

Hmm. I'm not sure it wouldn't end up being *less* memory. Don't we
end up caching 1 copy of it per backend, at least for the database to
which that backend is connected? Accessing a shared copy would avoid
that sort of thing.

Yeah ... the collector itself has got all that in memory anyway.
We do need to think about synchronization issues if we make that
memory globally available, but I find it hard to see how that would
lead to more memory consumption overall than what happens now.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#23Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#14)
Re: POC: Sharing record typmods between backends

Hi,

On 2017-08-11 20:39:13 +1200, Thomas Munro wrote:

Please find attached a new patch series.

Review for 0001:

I think you made a few long lines even longer, like:

@@ -1106,11 +1106,11 @@ pltcl_trigger_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,
 		Tcl_ListObjAppendElement(NULL, tcl_trigtup, Tcl_NewObj());
 		for (i = 0; i < tupdesc->natts; i++)
 		{
-			if (tupdesc->attrs[i]->attisdropped)
+			if (TupleDescAttr(tupdesc, i)->attisdropped)
 				Tcl_ListObjAppendElement(NULL, tcl_trigtup, Tcl_NewObj());
 			else
 				Tcl_ListObjAppendElement(NULL, tcl_trigtup,
-										 Tcl_NewStringObj(utf_e2u(NameStr(tupdesc->attrs[i]->attname)), -1));
+										 Tcl_NewStringObj(utf_e2u(NameStr(TupleDescAttr(tupdesc, i)->attname)), -1));

as it's not particularly pretty to access tupdesc->attrs[i] repeatedly,
it'd be good if you instead had a local variable for the individual
attribute.

Similar:
if (OidIsValid(get_base_element_type(TupleDescAttr(tupdesc, i)->atttypid)))
sv = plperl_ref_from_pg_array(attr, TupleDescAttr(tupdesc, i)->atttypid);
else if ((funcid = get_transform_fromsql(TupleDescAttr(tupdesc, i)->atttypid, current_call_data->prodesc->lang_oid, current_call_data->prodesc->trftypes)))
sv = (SV *) DatumGetPointer(OidFunctionCall1(funcid, attr));

@@ -150,7 +148,7 @@ ValuesNext(ValuesScanState *node)
 			 */
 			values[resind] = MakeExpandedObjectReadOnly(values[resind],
 														isnull[resind],
-														att[resind]->attlen);
+														TupleDescAttr(slot->tts_tupleDescriptor, resind)->attlen);
@@ -158,9 +158,9 @@ convert_tuples_by_position(TupleDesc indesc,
 			 * must agree.
 			 */
 			if (attrMap[i] == 0 &&
-				indesc->attrs[i]->attisdropped &&
-				indesc->attrs[i]->attlen == outdesc->attrs[i]->attlen &&
-				indesc->attrs[i]->attalign == outdesc->attrs[i]->attalign)
+				TupleDescAttr(indesc, i)->attisdropped &&
+				TupleDescAttr(indesc, i)->attlen == TupleDescAttr(outdesc, i)->attlen &&
+				TupleDescAttr(indesc, i)->attalign == TupleDescAttr(outdesc, i)->attalign)
 				continue;

I think you get the drift, there's more....

Otherwise this seems fairly boring...

Review for 0002:

@@ -71,17 +71,17 @@ typedef struct tupleConstr
 typedef struct tupleDesc
 {
 	int			natts;			/* number of attributes in the tuple */
-	Form_pg_attribute *attrs;
-	/* attrs[N] is a pointer to the description of Attribute Number N+1 */
 	TupleConstr *constr;		/* constraints, or NULL if none */
 	Oid			tdtypeid;		/* composite type ID for tuple type */
 	int32		tdtypmod;		/* typmod for tuple type */
 	bool		tdhasoid;		/* tuple has oid attribute in its header */
 	int			tdrefcount;		/* reference count, or -1 if not counting */
+	/* attrs[N] is the description of Attribute Number N+1 */
+	FormData_pg_attribute attrs[FLEXIBLE_ARRAY_MEMBER];
 }		   *TupleDesc;

sorry if I'm beating on my hobby horse, but if we're re-ordering anyway,
can you move TupleConstr to the second-to-last? That a) seems more
consistent but b) (hobby horse, sorry) avoids unnecessary alignment
padding.

@@ -734,13 +708,13 @@ BuildDescForRelation(List *schema)
 		/* Override TupleDescInitEntry's settings as requested */
 		TupleDescInitEntryCollation(desc, attnum, attcollation);
 		if (entry->storage)
-			desc->attrs[attnum - 1]->attstorage = entry->storage;
+			desc->attrs[attnum - 1].attstorage = entry->storage;
 		/* Fill in additional stuff not handled by TupleDescInitEntry */
-		desc->attrs[attnum - 1]->attnotnull = entry->is_not_null;
+		desc->attrs[attnum - 1].attnotnull = entry->is_not_null;
 		has_not_null |= entry->is_not_null;
-		desc->attrs[attnum - 1]->attislocal = entry->is_local;
-		desc->attrs[attnum - 1]->attinhcount = entry->inhcount;
+		desc->attrs[attnum - 1].attislocal = entry->is_local;
+		desc->attrs[attnum - 1].attinhcount = entry->inhcount;

This'd be a lot more readable if wed just stored desc->attrs[attnum - 1]
in a local variable. Also think it'd be good if it just used
TupleDescAttr() for that. Probably as part of previous commit.

@@ -366,8 +340,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)

 	for (i = 0; i < tupdesc1->natts; i++)
 	{
-		Form_pg_attribute attr1 = tupdesc1->attrs[i];
-		Form_pg_attribute attr2 = tupdesc2->attrs[i];
+		Form_pg_attribute attr1 = &tupdesc1->attrs[i];
+		Form_pg_attribute attr2 = &tupdesc2->attrs[i];

I'd convert all these as part of the previous commit.

@@ -99,12 +72,9 @@ CreateTemplateTupleDesc(int natts, bool hasoid)

 /*
  * CreateTupleDesc
- *		This function allocates a new TupleDesc pointing to a given
+ *		This function allocates a new TupleDesc by copying a given
  *		Form_pg_attribute array.
  *
- * Note: if the TupleDesc is ever freed, the Form_pg_attribute array
- * will not be freed thereby.
- *

I'm leaning towards no, but you could argue that we should just change
that remark to be about constr?

@@ -51,39 +51,12 @@ CreateTemplateTupleDesc(int natts, bool hasoid)

 	/*
 	 * Allocate enough memory for the tuple descriptor, including the
-	 * attribute rows, and set up the attribute row pointers.
-	 *
-	 * Note: we assume that sizeof(struct tupleDesc) is a multiple of the
-	 * struct pointer alignment requirement, and hence we don't need to insert
-	 * alignment padding between the struct and the array of attribute row
-	 * pointers.
-	 *
-	 * Note: Only the fixed part of pg_attribute rows is included in tuple
-	 * descriptors, so we only need ATTRIBUTE_FIXED_PART_SIZE space per attr.
-	 * That might need alignment padding, however.
+	 * attribute rows.
 	 */
-	attroffset = sizeof(struct tupleDesc) + natts * sizeof(Form_pg_attribute);
-	attroffset = MAXALIGN(attroffset);
-	stg = palloc(attroffset + natts * MAXALIGN(ATTRIBUTE_FIXED_PART_SIZE));
+	attroffset = offsetof(struct tupleDesc, attrs);
+	stg = palloc0(attroffset + natts * sizeof(FormData_pg_attribute));
 	desc = (TupleDesc) stg;

note that attroffset isn't used anymore after this...

We have two mildly different places allocating a tupledesc struct for a
number of elements. Seems sense to put them into an AllocTupleDesc()?

Review of 0003:

I'm not doing a too detailed review, given I think there's some changes
in the pipeline.

@@ -0,0 +1,124 @@
+/*-------------------------------------------------------------------------
+ *
+ * ds_hash_table.h
+ *	  Concurrent hash tables backed by dynamic shared memory areas.

...

+/*
+ * The opaque type representing a hash table.  While the struct is defined
+ * before, client code should consider it to be be an opaque and deal only in
+ * pointers to it.
+ */
+struct dht_hash_table;
+typedef struct dht_hash_table dht_hash_table;

"defined before"?

+/*
+ * The opaque type used for iterator state.  While the struct is actually
+ * defined below so it can be used on the stack, client code should deal only
+ * in pointers to it rather than accessing its members.
+ */
+struct dht_iterator;
+typedef struct dht_iterator dht_iterator;

s/used/allocated/?

+
+/*
+ * The set of parameters needed to create or attach to a hash table.  The
+ * members tranche_id and tranche_name do not need to be initialized when
+ * attaching to an existing hash table.  The functions do need to be supplied
+ * even when attaching because we can't safely share function pointers between
+ * backends in general.
+ */
+typedef struct
+{
+	size_t key_size;			/* Size of the key (initial bytes of entry) */
+	size_t entry_size;			/* Total size of entry */
+	dht_compare_function compare_function;	/* Compare function */
+	dht_hash_function hash_function;	/* Hash function */
+	int tranche_id;				/* The tranche ID to use for locks. */
+} dht_parameters;

Wonder if it'd make sense to say that key/entry sizes to be only
minimums? That means we could increase them to be the proper aligned
size?

/*
* Unlock an entry which was locked by dht_find or dht_find_or_insert.
*/
void
dht_release(dht_hash_table *hash_table, void *entry)

/*
* Release the most recently obtained lock. This can optionally be called in
* between calls to dht_iterate_next to allow other processes to access the
* same partition of the hash table.
*/
void
dht_iterate_release_lock(dht_iterator *iterator)

I'd add lock to the first too.

FWIW, I'd be perfectly fine with abbreviating iterate/iterator to "iter"
or something. We already have that elsewhere and it's pretty clear.

+/*
+ * Print out debugging information about the internal state of the hash table.
+ * No locks must be held by the caller.
+ */
+void

Should specify where the information is printed.

+	 * scan.  (We don't actually expect them to have more than 1 item unless
+	 * the hash function is of low quality.)

That, uh, seems like an hasty remark. Even with a good hash function
you're going to get collisions occasionally.

Review of 0004:

Ignoring aspects related to REC_HASH_KEYS and related discussion, since
we're already discussing that in another email.

+static int32
+find_or_allocate_shared_record_typmod(TupleDesc tupdesc)
+{
+	/*
+	 * While we still hold the atts_index entry locked, add this to
+	 * typmod_index.  That's important because we don't want anyone to be able
+	 * to find a typmod via the former that can't yet be looked up in the
+	 * latter.
+	 */
+	PG_TRY();
+	{
+		typmod_index_entry =
+			dht_find_or_insert(CurrentSharedRecordTypmodRegistry.typmod_index,
+							   &typmod, &found);
+		if (found)
+			elog(ERROR, "cannot create duplicate shared record typmod");
+	}
+	PG_CATCH();
+	{
+		/*
+		 * If we failed to allocate or elog()ed, we have to be careful not to
+		 * leak the shared memory.  Note that we might have created a new
+		 * atts_index entry above, but we haven't put anything in it yet.
+		 */
+		dsa_free(CurrentSharedRecordTypmodRegistry.area, shared_dp);
+		PG_RE_THROW();
+	}

Not entirely related, but I do wonder if we don't need abetter solution
to this. Something like dsa pointers that register appropriate memory
context callbacks to get deleted in case of errors?

Codewise this seems ok aside from the already discussed issues.

But architecturally I'm still not sure I quite like the a bit ad-hoc
manner session state is defined here. I think we much more should go
towards a PGPROC like PGSESSION array, that PGPROCs reference. That'd
also be preallocated in "normal" shmem. From there things like the
handle for a dht typmod table could be referenced. I think we should
slowly go towards a world where session state isn't in a lot of file
local static variables. I don't know if this is the right moment to
start doing so, but I think it's quite soon.

Reviewing 0005:

Yay!

- Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#24Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#23)
1 attachment(s)
Re: POC: Sharing record typmods between backends

On Mon, Aug 14, 2017 at 12:32 PM, Andres Freund <andres@anarazel.de> wrote:

Review for 0001:

I think you made a few long lines even longer, like:

@@ -1106,11 +1106,11 @@ pltcl_trigger_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,
Tcl_ListObjAppendElement(NULL, tcl_trigtup, Tcl_NewObj());
for (i = 0; i < tupdesc->natts; i++)
{
-                       if (tupdesc->attrs[i]->attisdropped)
+                       if (TupleDescAttr(tupdesc, i)->attisdropped)
Tcl_ListObjAppendElement(NULL, tcl_trigtup, Tcl_NewObj());
else
Tcl_ListObjAppendElement(NULL, tcl_trigtup,
-                                                                                Tcl_NewStringObj(utf_e2u(NameStr(tupdesc->attrs[i]->attname)), -1));
+                                                                                Tcl_NewStringObj(utf_e2u(NameStr(TupleDescAttr(tupdesc, i)->attname)), -1));

as it's not particularly pretty to access tupdesc->attrs[i] repeatedly,
it'd be good if you instead had a local variable for the individual
attribute.

Done.

Similar:
if (OidIsValid(get_base_element_type(TupleDescAttr(tupdesc, i)->atttypid)))
sv = plperl_ref_from_pg_array(attr, TupleDescAttr(tupdesc, i)->atttypid);
else if ((funcid = get_transform_fromsql(TupleDescAttr(tupdesc, i)->atttypid, current_call_data->prodesc->lang_oid, current_call_data->prodesc->trftypes)))
sv = (SV *) DatumGetPointer(OidFunctionCall1(funcid, attr));

Done.

@@ -150,7 +148,7 @@ ValuesNext(ValuesScanState *node)
*/
values[resind] = MakeExpandedObjectReadOnly(values[resind],
isnull[resind],
-                                                                                                               att[resind]->attlen);
+                                                                                                               TupleDescAttr(slot->tts_tupleDescriptor, resind)->attlen);
@@ -158,9 +158,9 @@ convert_tuples_by_position(TupleDesc indesc,
* must agree.
*/
if (attrMap[i] == 0 &&
-                               indesc->attrs[i]->attisdropped &&
-                               indesc->attrs[i]->attlen == outdesc->attrs[i]->attlen &&
-                               indesc->attrs[i]->attalign == outdesc->attrs[i]->attalign)
+                               TupleDescAttr(indesc, i)->attisdropped &&
+                               TupleDescAttr(indesc, i)->attlen == TupleDescAttr(outdesc, i)->attlen &&
+                               TupleDescAttr(indesc, i)->attalign == TupleDescAttr(outdesc, i)->attalign)
continue;

Done.

I think you get the drift, there's more..../

Done in some more places too.

Review for 0002:

@@ -71,17 +71,17 @@ typedef struct tupleConstr
typedef struct tupleDesc
{
int                     natts;                  /* number of attributes in the tuple */
-       Form_pg_attribute *attrs;
-       /* attrs[N] is a pointer to the description of Attribute Number N+1 */
TupleConstr *constr;            /* constraints, or NULL if none */
Oid                     tdtypeid;               /* composite type ID for tuple type */
int32           tdtypmod;               /* typmod for tuple type */
bool            tdhasoid;               /* tuple has oid attribute in its header */
int                     tdrefcount;             /* reference count, or -1 if not counting */
+       /* attrs[N] is the description of Attribute Number N+1 */
+       FormData_pg_attribute attrs[FLEXIBLE_ARRAY_MEMBER];
}                 *TupleDesc;

sorry if I'm beating on my hobby horse, but if we're re-ordering anyway,
can you move TupleConstr to the second-to-last? That a) seems more
consistent but b) (hobby horse, sorry) avoids unnecessary alignment
padding.

Done.

@@ -734,13 +708,13 @@ BuildDescForRelation(List *schema)
/* Override TupleDescInitEntry's settings as requested */
TupleDescInitEntryCollation(desc, attnum, attcollation);
if (entry->storage)
-                       desc->attrs[attnum - 1]->attstorage = entry->storage;
+                       desc->attrs[attnum - 1].attstorage = entry->storage;
/* Fill in additional stuff not handled by TupleDescInitEntry */
-               desc->attrs[attnum - 1]->attnotnull = entry->is_not_null;
+               desc->attrs[attnum - 1].attnotnull = entry->is_not_null;
has_not_null |= entry->is_not_null;
-               desc->attrs[attnum - 1]->attislocal = entry->is_local;
-               desc->attrs[attnum - 1]->attinhcount = entry->inhcount;
+               desc->attrs[attnum - 1].attislocal = entry->is_local;
+               desc->attrs[attnum - 1].attinhcount = entry->inhcount;

This'd be a lot more readable if wed just stored desc->attrs[attnum - 1]
in a local variable. Also think it'd be good if it just used
TupleDescAttr() for that. Probably as part of previous commit.

Done.

@@ -366,8 +340,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)

for (i = 0; i < tupdesc1->natts; i++)
{
-               Form_pg_attribute attr1 = tupdesc1->attrs[i];
-               Form_pg_attribute attr2 = tupdesc2->attrs[i];
+               Form_pg_attribute attr1 = &tupdesc1->attrs[i];
+               Form_pg_attribute attr2 = &tupdesc2->attrs[i];

I'd convert all these as part of the previous commit.

Done.

@@ -99,12 +72,9 @@ CreateTemplateTupleDesc(int natts, bool hasoid)

/*
* CreateTupleDesc
- *             This function allocates a new TupleDesc pointing to a given
+ *             This function allocates a new TupleDesc by copying a given
*             Form_pg_attribute array.
*
- * Note: if the TupleDesc is ever freed, the Form_pg_attribute array
- * will not be freed thereby.
- *

I'm leaning towards no, but you could argue that we should just change
that remark to be about constr?

I don't see why.

@@ -51,39 +51,12 @@ CreateTemplateTupleDesc(int natts, bool hasoid)

/*
* Allocate enough memory for the tuple descriptor, including the
-        * attribute rows, and set up the attribute row pointers.
-        *
-        * Note: we assume that sizeof(struct tupleDesc) is a multiple of the
-        * struct pointer alignment requirement, and hence we don't need to insert
-        * alignment padding between the struct and the array of attribute row
-        * pointers.
-        *
-        * Note: Only the fixed part of pg_attribute rows is included in tuple
-        * descriptors, so we only need ATTRIBUTE_FIXED_PART_SIZE space per attr.
-        * That might need alignment padding, however.
+        * attribute rows.
*/
-       attroffset = sizeof(struct tupleDesc) + natts * sizeof(Form_pg_attribute);
-       attroffset = MAXALIGN(attroffset);
-       stg = palloc(attroffset + natts * MAXALIGN(ATTRIBUTE_FIXED_PART_SIZE));
+       attroffset = offsetof(struct tupleDesc, attrs);
+       stg = palloc0(attroffset + natts * sizeof(FormData_pg_attribute));
desc = (TupleDesc) stg;

note that attroffset isn't used anymore after this...

Tidied.

We have two mildly different places allocating a tupledesc struct for a
number of elements. Seems sense to put them into an AllocTupleDesc()?

Yeah. CreateTemplateTupleDesc() is already suitable. I changed
CreateTupleDesc() to call that instead of duplicating the allocation
code.

Review of 0003:

I'm not doing a too detailed review, given I think there's some changes
in the pipeline.

Yep. In the new patch set the hash table formerly known as DHT is now
in patch 0004 and I made the following changes based on your feedback:

1. Renamed it to "dshash". The files are named dshash.{c,h}, and the
prefix on identifiers is dshash_. You suggested dsmhash, but the "m"
didn't seem to make much sense. I considered dsahash, but dshash
seemed better. Thoughts?

2. Ripped out the incremental resizing and iterator support for now,
as discussed. I want to post patches to add those features when we
have a use case but I can see that it's no slam dunk so I want to keep
that stuff out of the dependency graph for parallel hash.

3. Added support for hash and compare functions with an extra
argument for user data, a bit like qsort_arg_comparator. This is
necessary for functions that need to be able to dereference a
dsa_pointer stored in the entry, since they need the dsa_area. (I
would normally call such an argument 'user_data' or 'context' or
something but 'arg' seemed to be establish by qsort_arg.)

@@ -0,0 +1,124 @@
+/*-------------------------------------------------------------------------
+ *
+ * ds_hash_table.h
+ *       Concurrent hash tables backed by dynamic shared memory areas.

...

+/*
+ * The opaque type representing a hash table.  While the struct is defined
+ * before, client code should consider it to be be an opaque and deal only in
+ * pointers to it.
+ */
+struct dht_hash_table;
+typedef struct dht_hash_table dht_hash_table;

"defined before"?

Bleugh. Fixed.

+/*
+ * The opaque type used for iterator state.  While the struct is actually
+ * defined below so it can be used on the stack, client code should deal only
+ * in pointers to it rather than accessing its members.
+ */
+struct dht_iterator;
+typedef struct dht_iterator dht_iterator;

s/used/allocated/?

Removed for now, see above.

+
+/*
+ * The set of parameters needed to create or attach to a hash table.  The
+ * members tranche_id and tranche_name do not need to be initialized when
+ * attaching to an existing hash table.  The functions do need to be supplied
+ * even when attaching because we can't safely share function pointers between
+ * backends in general.
+ */
+typedef struct
+{
+       size_t key_size;                        /* Size of the key (initial bytes of entry) */
+       size_t entry_size;                      /* Total size of entry */
+       dht_compare_function compare_function;  /* Compare function */
+       dht_hash_function hash_function;        /* Hash function */
+       int tranche_id;                         /* The tranche ID to use for locks. */
+} dht_parameters;

Wonder if it'd make sense to say that key/entry sizes to be only
minimums? That means we could increase them to be the proper aligned
size?

I don't understand. You mean explicitly saying that there are
overheads? Doesn't that go without saying?

/*
* Unlock an entry which was locked by dht_find or dht_find_or_insert.
*/
void
dht_release(dht_hash_table *hash_table, void *entry)

/*
* Release the most recently obtained lock. This can optionally be called in
* between calls to dht_iterate_next to allow other processes to access the
* same partition of the hash table.
*/
void
dht_iterate_release_lock(dht_iterator *iterator)

I'd add lock to the first too.

Done.

FWIW, I'd be perfectly fine with abbreviating iterate/iterator to "iter"
or something. We already have that elsewhere and it's pretty clear.

Will return to this question in a future submission that adds iterators.

+/*
+ * Print out debugging information about the internal state of the hash table.
+ * No locks must be held by the caller.
+ */
+void

Should specify where the information is printed.

Done.

+        * scan.  (We don't actually expect them to have more than 1 item unless
+        * the hash function is of low quality.)

That, uh, seems like an hasty remark. Even with a good hash function
you're going to get collisions occasionally.

Review of 0004:

Ignoring aspects related to REC_HASH_KEYS and related discussion, since
we're already discussing that in another email.

This version includes new refactoring patches 0003, 0004 to get rid of
REC_HASH_KEYS by teaching the hash table how to use a TupleDesc as a
key directly. Then the shared version does approximately the same
thing, with a couple of extra hoops to jump thought. Thoughts?

+static int32
+find_or_allocate_shared_record_typmod(TupleDesc tupdesc)
+{
+       /*
+        * While we still hold the atts_index entry locked, add this to
+        * typmod_index.  That's important because we don't want anyone to be able
+        * to find a typmod via the former that can't yet be looked up in the
+        * latter.
+        */
+       PG_TRY();
+       {
+               typmod_index_entry =
+                       dht_find_or_insert(CurrentSharedRecordTypmodRegistry.typmod_index,
+                                                          &typmod, &found);
+               if (found)
+                       elog(ERROR, "cannot create duplicate shared record typmod");
+       }
+       PG_CATCH();
+       {
+               /*
+                * If we failed to allocate or elog()ed, we have to be careful not to
+                * leak the shared memory.  Note that we might have created a new
+                * atts_index entry above, but we haven't put anything in it yet.
+                */
+               dsa_free(CurrentSharedRecordTypmodRegistry.area, shared_dp);
+               PG_RE_THROW();
+       }

Not entirely related, but I do wonder if we don't need abetter solution
to this. Something like dsa pointers that register appropriate memory
context callbacks to get deleted in case of errors?

Huh, scope guards. I have had some ideas about some kind of
destructor mechanism that might replace what we're doing with DSM
detach hooks in various places and also work in containers like hash
tables (ie entries could have destructors), but doing it with the
stack is another level...

Codewise this seems ok aside from the already discussed issues.

But architecturally I'm still not sure I quite like the a bit ad-hoc
manner session state is defined here. I think we much more should go
towards a PGPROC like PGSESSION array, that PGPROCs reference. That'd
also be preallocated in "normal" shmem. From there things like the
handle for a dht typmod table could be referenced. I think we should
slowly go towards a world where session state isn't in a lot of file
local static variables. I don't know if this is the right moment to
start doing so, but I think it's quite soon.

No argument from me about that general idea. All our global state is
an obstacle for testability, multi-threading, new CPU scheduling
architectures etc. I had been trying to avoid getting too adventurous
here, but here goes nothing... In this version there is an honest
Session struct. There is still a single global variable --
CurrentSession -- which would I guess could be a candidate to become a
thread-local variable from the future (or alternatively an argument to
every function that needs session access). Is this better? Haven't
tested this much yet but seems like better code layout to me.

Reviewing 0005:

Yay!

That's the kind of review I like! Thanks.

I will also post some testing code soon.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

shared-record-typmods-v5.patchset.tgzapplication/x-gzip; name=shared-record-typmods-v5.patchset.tgzDownload
����Y��{���0|��?���CY��%���Hr��r%%iO��?XJ�I�@�j����ew��A�i���95(`w������k|�F�?��F�ar�^�~|�~�Z��w���}��v{48t�����U��tz�~�?�:#������r����m���`*��*
+�A������}�7�/������������f����C7I��pu���Wx�n�=n�p�C��f����0��;}�������N�������������(\:�������O��x����Qg6�������G�n���W��X;�H�_��;]��8���M�tc�5���2��Z���DD�(��?ky����7_8?��t���5�v�_�&_���A~p���$���/o����q:OG?:L�NJ�/�����IB�"��"�������"����h=xpu����,�C��p<;X9�H���M��yA��8r`�� q�7qn�������hqn'���h9��*�B��k|��t`��,N"�K��/7X��+���O
f�D���8	a��w�o�M�����l`�1����w�;�x@wz<�$���
`'�1����[�_�}�����2����$��M,�pn�d���:����%�����G�b�|������������o��:���P<��?���w=�����������wg���>�uM�L>�������E�z'/-�)���������<b:�o�����3t��7fy�a����@��[���V�����p�����������=#��,|1���=	"����qa���Z=�?2��=�u�4	�h��X��&?���-�%�U��=�T�����UPC��E�L}��|X��_�hi�8���\��X�O]�q�t+���U=���[lp�c���VK�Cn�5z��\������p�+��5�����j�`����`���1����UK{�����������w;r����n@@�c$������A���7��S��h�.�
����6��W�9��Z$%lg[�M,h����e]����m��_�=JQ�F����W�'��\�Cno�TA8^�����Y�0Lp�^��rs.���&�"��9��N���*��������|�xm�^��$�C��]�����n)y8����r���3\��������t�H���m��&@{K��k1)��K7A���K{a�;��n���[�t/�!t��	�~�� �
�F������H���=��<��[:�<%[]�gS(�����-�������g����k�������e$����[2�m���G��K{��]���C��<��(�!lj��s�����9U}(�|
���
�O-���p�\/���1[z�K�-� p��]�w�~�x��R�+��"���&?o���(Q���@������s=�5	����)���d����R�s�*Xhy�K{�6;����t����IVw��$��NXQ��X�N��.S/���^,AX=KM5��E��q�^G�K�7*�+%S���0�
�`�Qo���� ��R�"L��x1��zf?����{��/��f>�P����a��\4��k�\_��d�I������Q�u=�`���Z=�)`Q >������<�)����s�����m�Y�N)���'�P��H���8�S������[��
��Nw�����p�x��(8n8�{G�5	r���G|X�)h*_�����#�(�%%/�����z��o���W����/�Yq�v����.����ly5r��
dR��BE�:����Mz�e=TKd{���T������k���"R�*�Y�4�����O�O����k��|O�,J{�>%����9#�����M�g$��"	��P��,��k�D��QO����C�������c��P	K���V�|�{;�����X���3��`/	���}��;�X�s�����s���:H�i��}V�����q���:��7i�f�I�=���vg5��Q�_>����7��;��������?���c
5�/l\���f���N�'�wn��6���^���p�E��'�)5z�.���J�O��sg��3����Fw������Y����`������^0wr*�S�#�
b?
�k��?8��{F�)�m�A���3Q�H0��g��-�#:{{{�`������R�p�����7���V���������y��}�1��q;�D����M��i��U2�b��w).�Yg�O���;j�>Qw�y�(I"���_`�L�Ns�D1�����&�])�w�8����\���
Z��^pI�.�$�<r���8�~-�R����[���!Ac���������^�*^d
����GM�����X�~��Y��D��9�n��f9�&���i�4%},���x�"�y'�`V�a� `i����Q�u#:�u�|{���|?!��'-_�d�=2X�������w��C�L��p�Ig��c�E����i�B��������G�`"[{h2�R�}( �INp�Y���������9�8E�Z��3I�	-�~��.�OM�A�	�bC��E��>�e<2Z`���T���	M��O�����?�� ���G.���*v�p�ym�O	������X� *\����F88X�'9����->�y9�����[�7��~�;w[�I_�b���f�g��q��[QD\��F��E�&�$=MQ�g��}��������p^����F���X8�}R|���\q&��P@��#�\��0����Z���B��������������������U'������7�|*�_xj�O�S��
�Fx��1H�^��Y�r��)�������u���y2slr���	2@u�i�
���p���CnJ���:g�(�AjA�W^��z�N��u������o����u����A�}�}���_>��+D1��!��a��^��})30Y��h�x_��
\(g%S�^�=�& �Zn��uf�no�+fJ��<��-����Zo��RI��G�r�>���_O_~������������}S���S��6�9#
 tL7��O��u�p��;����L"����K���#�Rk�>2f�'x����_*~�R���c��G���XOg�+�Xn�V��.
��.rl
e��v�|[|��!c>�v[��d��:�V����:�h��d.�J+�����z1�&�6�E������I����i��M�k��f�k��:\o�{���7�w���0�����	�����l(C�U�'�<8oMY/��a5n���N�	5h;m��z�^apS?�E�O�����V8����O��f��)0`���n��A\��8Xy{-�����q��,���UjGwX8����R$.�V�M�Rz=�4+���@�x�i����WV+$��5��a[���e���flk�6?-#�_������*�w�=�A=����R�/�2LY(�l[)���>�0�}w�j��d������)�f���E��V��^�x���J�I�x�V^��Ez�R���t�.�e�`%��E0��%�UwD#:|��[r*��y�L�L��_�	����9��oN��N�9�$m���1bv���4���$6���GD����A���L��sX4��$����2u��
V��������MH���!��^�P���_�80�>i�z�A����YI�%,��*\8�6x�\��0�Z�c!�D�35k�����|�(�-��[�5�?�89�:j�������t`��$���}x|���! h����5�h�*���ne�i�Z1����1o��:���w��Vw����y�kog��H�l1mC�C����v|4������E�t�Z}-�W�o�A�!���9/�;�V�'�*��G�*}�L@&M�)0�3��o�hX �
���������NP@�P)��KzIA�D\�(���a���-"�mi����|����v�����>���N�+H�P�7"���![���-����SQk�H��)��hG����Q�CGc��Q�b-�����T:��sF�~.��!����5�;-_������mE���Hj`}�����X
���R����5@?$��HM>5�4�)9SB�����vHwFJ��?I��`O2
�J-�AQ�v3��J��5�@�6S>��K����4�)��i,`H9B���*��d���$��G#������f��v���?��i2�U��LK����y_q+���.����4��JD����0ijh��$�k�5l5��&Q|���/��kI�a�.x����V{�=���5�4�vF}�b�����u���s�`b�'�$��,��t���fcM�.�U:Z�4�����rp�������X�Z���������,C�6���+�)�=b*�(��������>��	@���'�'���{a@�����V�mo��K��"<���(}�@��,9������0��aJ(�p�,G���S�} i]���,��,o),��,g"��m=�G^��j{N^�g~n�B����0+}$��|4iD���\��������\��� 6��=��U��5_�d���rb�-E�u�Q�������)�or��mu�2�?���)��Kg���96��J�{���Rf�eH>+��W��`��^��*����8����>��z��3������4/Z������������so���K�i��qg4�
;�V������d��l��#��S��6�~��&����8���Kfw%)r��b�M�v:�\8��w���B�
�����s��F�F�8��e��6�X\bF�z�����c<��p��t����#"���:�������B����x�/�������W~{:}}v��*������u�Q�"����l������]�^����'C��b��#��/��xrDj�`f��7W�Rhw3���������^�:�����v���C�R�*�W2�Mg����&.��&Jnd��w/��0p�m�cJ��g0��`*���t.A\/�$R}�?q#6"����8��u�-���(lc��W/���J���������.�"�'N�Y�	��Tt{_8�$��mF����"�#�g%��������y�`5��o
�t�n�����i�v�<�c���7���[_��/t�8W7�}k_d���q��P��0�?���xNV�<}c�~#r#>�J��9�"!%b�����hY���p�K����S��s����,8����|���xu�g��j�T�7���^K}ZG��>d����$��=���4m������hM1��I�i�|R/��n���~����m�����3�	�f���e�����yEd{�X�{����WD��+"�s���������a�9�A�t=Da�Q�c���}�����o���|i#}���`����qr�T�%w�������[8j*��5,_���D�n;��1/�_H��#%;�����%P�bTv4��W![(���6m�*0oJ$E��'=y�F�c���.���$���,�b
�N�:�����������x�ju'����
'����wF;��3�I��4�@��kL*3��S�4A��T>%N����F:�������;q���t�a�0��5&�n���E��*����$��wS����&J�U��4P%jh�a�G�kw�-#��z��q����y�4���@���j:�D���\V��W�xV���?����R�=������l4���%�W9b�&+���[�����_SNc6���f���~n�wM�	Q>���S���l�{{!@`���b��F��X}��H}��($�B2��'V��jW�H����/!��9s,)����f�]<V����On0G{���%�lK��431k�.�����A��t����-C1QliD�w#2�n�#X1� Y����3L��?��N�����d�T|K.����E�n:_�������vz&�4Yd�b�n0����k���y�gw���f>��t5�	!7Z�1�����-���bKR@��F��,����Y�!5����T��*���
^������i��Yg�\<�������*�D)����V���4%UG9_J!d
'���w\XU��e�������?04\�����U
GZ�g��7A!�����%�cicM4���fg�"�q�+��p�c������`���?hw��{���~J���k�pb���=����U�f��W��|���/�	�H��a������U�{;����.�%<�����`RI��P��#�;�w�����������f_�����N����=o�n����'��{����t;c����t�1�@7B��)E2��5��{�2�P��=�2�c�.��K ~������4u��#��:b`���f��Y����\\�[��q{��6�w���,3�
p4�@Ed�B�Y��\�yqz�������g'
9EHS�_	�#�B��j���GX�e��� (�c�$�hr����u0P��7��
�KX
J��`���)�������6�'3��XDg�sm:�jof��B1�p$s���w��A"�����T��	�������r)�������1}c�q�����������nj�����
o�\��y���!����te�-�C���k�������������MF��x���#�`^�-E���?i�������h_����C
_2P��e��O�_��������_%����!����?��g���w[���������v�Vj��J5%i����������h��K&�E�1��U�
�/�-�XWe2�%���j<��P��;?%0���7�g��Er-����}��	�a��S!���$����mw����J)!���f���F,E~A[NX��j��m�RaL��w����F!s�p.���5�7�7E���?W�){>?J��VZV{!����qi4&*`��R"��c���h�;�����7��y��0���#��(��vR������������G_�:E^`t-����|~|�f�������7�W���������|������\�%�H#UT�������}1w��<�]^�^��~sz�������:'%������U�;w%�]r^X���/��4�IJ��%be��x#��
CDc�� ���w�s�q\!�<�(/�V����'��B�u��lk7�gw�,��\l2���������/T���dM���^(�"�������n������=���1���	Y�����u��=]���-
6&���.:��J�)��bN�����*���J�V�����k�/���o(p�y"������������51b*��d����t�I>�<�������0��y����$�*I��#�'�[qzJg�����S&y���������������B����������`WAt��rP��������;� 
�&Q}F<�7a�hj��FB�Rp.G�	4�
��u0�/�c���!��p8�<�H�G~�����
�;j���n5�����c���j���r0tl��>��?��\g��������������sO���FE-��dt�b'�2�j�Q�1��{Z@,�{�MS�����(��v�H�E$0������PCTaN����AG�������[�;��t'$��B�)|��F����G�)�@��6�CnF�\I��/�)����������I��N"dh�5���>D���yo��e&_�/������S�������O��K��0u���R��I���]�#����w��}J��ow�Z���!��Z������l����)�W�YX=B�n�}��Y,���Y����%O��J\U/a���B����R�U����	-���V7���X���XY����DXM�_���5���yU�"�uU���"�3�tfn����id_o�P�#�o�X4�'#hV���b^D'E�`�a����u�&������4��J;��S�����^	�`��~�m�W�G�B��{pd�����~��*`������d/�������7(i�srl����H��M�n)��V6B�Hr�(�$X���SA;C��.9��J�N�v�������*q��T�������PZ���E�}n������M_����e/���R%���o��=b	�C�&�9�����L�:d�������p(�c������$/IN~���w���J�e�6��%y�2T��q3��m}l�U����9���/|�����c�)��A����/���l4#��L{�C��?�G�7%���?i�R�?��'z
����=S��:$Z��0\;�C��^&qNI9���#�K�������'��9�fk���+}b���0jN0qE.5��_������1�,����N�2��J��'$9SP����di�:��:iy���v��my����I�|���C����G`0�8��>��4�iVen�1Lt:�^J��E�*
i���9�������j-��8I�� R�������^V
�Rc\k�5h1S��3�"*�:�cv�Fv������Q��:�_��3�K�n��E	 ���&���J7�Q�DKM�����yb���l���}��9_d~���Vm�B�/��/��D%;*7����6��":��V�G�X��m���-��TFX(�'��+��J�B��e[n�Y�o#�f�c?�=&��a��Ju_���X�W�����7n����N�~����Mc	���, ���	������Nt�H�0Ai�=�33c��<a��d�1�����l5�v��l�b,������T�e����6�i|�B7L%�h�����	���������3�	���P2�������P	����X���o��uTi�X�Dz+��E��&�%�PJZ�Z2L���=�����T���D�m3�u;m jC.��#w$m�����:z��
�B��0��+A�
 �*aA3�Em	5qX��� ?� �I��q�2����uc���2�)}��5n�K�nY�@+�9i����$���L�t����}<Y���!��:�8fC��������o��g<u�������c
��!�j�q'v�����4��l6�$���}�����jz����O�o�������,������(��r���f%c��f�3��l3���j���lO,��5��I�UzwT��c.O}������#PR��_���u����m�6��nG���p��p��I��#�����5���5���������w����M�w���6vy�M����N\�7aYH�e�Y�mX�
�hX���r�����y��i^��d^��&�y��T������������	�22�������M
�SC]%����v��R*����'���\�[�YOL��p���V�cn���y����*��>�Rx�f�H%���>��b��G8tD��#�`�P�VT��fw��t��Kj����vXn<��?�������X>��1��34EQJ{�jl�����zs,K��y��6���A�%�)1t��Au�wY6��:k����[]������f����b�����Xk�pS�'�yD�#�	���\)X�b5�xl�\�.�[�����1�������Ti*�]-�7��3t�p���R�HE|�X9��9���:
�����������������9�h��B�<V�[l��z��?�d��'c)~��.�S��v���j�<tN�}��r��lK�%���H�U����9���IX����[�[��3.���e�in���
�����H���xwz�6��6k���
�Oz�����d����''u�K3���.�`��U`�x������t��%�U�G�A������?Z�W����7��-%{H~R��Jy��`\#��aA7�$���6f��������?'3S���������.����d��k�J
�8�������%gG+"�o����Pz��������?S�R��d�g/@:"�|��7	&�jd8��;����S������8������F�h���$�j��u�0+��>���erl�t��
�H��}����p��2��T[��.hh�B�p���(]�t���/f�������}.��i�U����p�;X����<8�X��7SdS������bR��sIb����).��Ry�r�8{�����&����#uh#���N������~b);�:����!}/x������������^��#�rs�gh�q��sL��o1s6��8Cu���\��dp�G�\��*�B��=o�(�-'����2�~o4������;��Q�FV��!��:���[���}���L�Ogw�u��:e)8mh��:a�"�Z���5��&s����������'#�>��y�n��{}y��l���>_��4�&���������^92�)=�z}����y~�I)F#${�q��n�^�2����oV�X�Dk��U�Wn
Y��
�)4�`}�����$\�v"���/����3h?V~��	�n�����{� �Y�v�aU�B
(���*���kwM�F��]Xh9O:m)���m������e��'�,Y��	��q�o�[��'V�Y�0,��j�k�^GBZXt��U������3����66�e�W7����+{����� GV����c����w���[�
(oXKF�aU�VU�*�d_L=���7�:�=.
��r�.������_�e�k�n�H=SL�_DA0f�%�1m����k7Vx�beI�`�7"�i/R�u{�3��I.Cg�����`�:��V�32x$��\7$����RsT�`�S�~*�V*��������������?���x�#���yD��y\%.��G�M���q��;��������y��:P�a,�o��ZM���������N�/��tz2}{tq5��H�d�@6���
)3�Q��[T�F�\Ve"��,��6o�]��,p���������`*��%���-�+9`�jS[R%0,�*���E2{nQw?���������&w$w���J����U��}�H�
��;���#^<�9��}�{K���
Enq�H�����}��������<������G�&]+?t��R<;,���]N�'#��I�K$��M�KZ2%Z����P�-�
m����	�����z�eS`�����4:T�:�ra�r���������x����y������T��EbF-��_
=T��mknx�vI��p(�a�����PqUZL��xb~��7�sw�H���-�!cQ���D�5I1fK�7�(�����`�K0���m�G������������
;o��Fm�d?o����8��t
�M��~�������zNtLI��C7����k�)s���SA��
;v
�����8$C<��
��*��z������D$�PN�~G���|i�p��I�A�Em��x��@����	�����kc��YL37u�T2#�~~�R�\�i�V,�v���$�\�9:���pQ��R�Z���&X�-��t��N������9�6������f�����s+7�����!(��~8$=�p8.��cU)j'�l�5�pH���5����2��zf��>���������/~�*U5��!���tu��1)tO�;OB���c���4Py��E��z�J^�j�����-���e��[`�9�#���� /4���#��;� �4;��_�s(`;��d����pS���2
�r���4��g�BX~2����2Y���p%*���U�����x�6,�_`i�Z�g�����
����vTLy+��/��������]8q�����W��k���OzaWM2(����N9�A:����B�70�lQ�-�	�E���;�����/�����c}D��V��L�������o�Y�+�1XM���?�w�1���Vk2sg��l��y[�`��������+���S�v��'�<o����n���?�;`��������/<��uj:��"9�Zz\����e���1h����F�!X%�*_�F�I�A���T
�eJ�#X�3M��d����@�B�B����`6��fc�b<����x��_�kUp+&
�c�t������X����6�V_�#N(��6u�2�P��]SI2�`�%B��r[�]���S~�
�h��|��&c�����3"VU=�����)i�(��n�"{��Q5N�~�6�g,�$�M,��7Q
����co>����d<�O����j3�`�(A6��1?&���"H����P?��h�Ue�n�$!!*�����]T�����s��\u��N(R�]�s���0��Dl���mQ���xBQl$���C-r�]�v�C�����]1�SYj;��<���0E0#-���fd�_�����n�z���8t�*fR�B���,�7����f=�;S�Y5����j�@�\�������7����o1h�x�����s���<�����vrCM����\S��)�`�2��&,"���qzqq~�_�9�}����ON���99�������Fah����<�'��U�;L�.�n�x�Q�M��t��,~d�wP��p�����f:�1by�NQ��x���_�\Q}�p���V��MA��b�B[%��_����	�C���7���;G b���������z��x8����bk��d���
�J�g���g>������r�R�5Rd��5,c��<wx�w^������o�����RlN�'�DW��T�XF+��(�RF_�d����tZ���P��oN..����bt���L��=������z��Ez���L�gX�;,�f5Ks 2=�Q��r�s66��Hq��!�?�����&�cw�����3�Z#����h�n%a{�R�����zm)��9w��0���>�R��|���O�O���K���|�� �m8F��}E2�~�	��lj������Cd����]�"
�6m�-	.�F����)���9����
�~���N�R����}	G��B^3��i��@M���2B������J���;%[N�	G`$3���&�YK&����3����<b>�U�1��������$�TV�,>��s��tD��Tt*��l#�K=���tQ%g1��
r0���3o2��Z�������H+����XM	�����0uf�6S_,��Zj��'e;��'�C�*�x������a/K{����]�`U�e�(��Mf�{�~�	�:�evz�`������Pu�0���N����&*NQo~�����P��@��.�>�v ����Q p
�k�HC��5U���@��*o�i`QJ����n��8���Q&[��aGH��A9!��9��74� X!�a4%��r ��-!�Lj�=6�0�|�����d���G��K����f�-�8'�b������U���#��m��#�a�t0T���Z���#j�ry��4!�y�5���L�S��.-�ag���|Oe��4Q��	��t6�E���rE�����������H8�|�� J����wY��Yg���Lt�
�Z\����{�|�����OW�o�^M����]~��(���Q&d���p����
�}L����@��J�i�����t?�.��K��|Z�W��j���P	�T�<6<b�.v��::{���,�&j`<�V"_[8�;��?��!:��p��tI��Y���r#���!'*/�O�dd��A�`��&O����]���Un�;���v���Q�
��/S)�2��KaV���[���20 ������a���7g�p1��S�7��[R�C����t|��������I��zF���G�T��C@���3����� �mbQnPF�.�������������/�Yp}-�-E��T�'i8���]e�D,���A�g�����r�V�i�2��8OC���������j
}���h���y��3
��������td5v�g2nx����_u�Tq�����+�?l��'�r����H~��'����`������=�����6'C��,��!�Qs�|"VE��
������g���^t���5�T4�b-U�������{|�!����>&�'/�����1&���>u��g�q����B����� 4MB�gT]j2�0�GWe����yy_�B.H	H���N=��T��2�[;]�<�3)%��`�3Rue���|;��Ib�[�:$�k�#a��7I���
�//���.dj+�4B4a��|.������������a;0�_w�o���k2����2�K��������;�z^���v�����i�p��Xjs�7e~���#�=<����.B��D��:�v��C�yT�����{�	�����%���*��w�0Z���!b*���8�-��C�#���m��<0��t�1����~�+�S��c��b0��M���W��h��V-��g �R��< ���u�+G>�m$��z����y�n��j
����u*Z�G�Nq;Vg�������!�~ �J#5��hu��W�X���tk���'�&8!WK(
V�,�r�
m7�IO)��&S`�m�eD:��6���@mb��Bj�Q��G��4^����#O��&�����cQ�"�n��~��������$��������>�����5��0��ZUC�^1�S=@d�
����fm�n��������C��'�*�9I��	�7T�C��� �$�H�1��&`7�#�R��S�N��#����)�#X	1��|�`��_��'+q���`_��Jx����G�H�gE����F��\��1Eu����;}��,`�P?qDkvG3���C��2����0���a����p��S�c��C���7�&�yG������Lf�q���g��d��u��"����P����.Q��B���.����	�\�s�(�ty'b�_���Ba8)v�Nia�9���N�B-~�����l���nu�mO�{����������^���:��A�Xp�^^;��sb�:�hj:����`�X�S��;��N#&�~2R1������fJ>�'�.%"�:��j��S����th��l����2A�,z�5j�p�W��~�/������S���������/����9T�!�p� ���� ���L-�u��V��M����d�����U�|��a�Ti=<�v������D�-dT�)<��v�lJ��2�c��r�k�J{S��XW9��F���� �f�cy���U�g�Z�I�N'#r���8��7�6�*n����g|���TkS}�G��"+���Q$�$���N�������`LZ
��}b���e�\���[���:gA:��lH-����4��m^`�`|~���W�xq"���$����O�'��X���}�@��/M������yd�:��
a������2��b~����:��
�;�����R�D��T&�inYI-�Y�S)�v�`&\2�xHo�����{N��q��V�����8q���f��x��!{��D�2���Y�~6��
B7l8���4�3���o�]��"IU�	d��I~5��(���z����D�Q���-�?W��7(U���ck��|�X�2el/�=�S�(w.T�t�0��xL~�pZ��`���G�k��Fi��L`�E�k�8�������l^���l�f��*Gz��b;F�9aw_���t���a����	���6L�8������x.$i2�P����AE1o=���5uZp����p�~�,.t��t�W����g7���(D�R�{E��)%8���p(+�u&H� ~SR(|��o��5x�s�t�7���$���f���v)���kLH�2e�[�&����c��)������S?��'��F���U�U���S���)"�Udo�m|5L��,��t�i*�����5���;���x��"���f[7J�;�\\�W�
�v9
�b���Y�5�:����]�}�4�*�1�F������HP��)��HF��l��IO�1����`s��Q��|q� �b�m�%���������f����f�s7����.���88�g�M����]1�
��0�4�R� �T������C���f�$A������Y�������yg�����f~�+��0&�G�z��33�5v�C�Gu�XP����H,�����^60��-L�����6�/��~*�{��F
��R�M�56�OW������
^�	q�@���7���eQ��]	�v4Cz�:�������L�-��)*�k</��[���Bj�v�������$���-?�P���dt&\az��(o,s:�3�v���M�������`d
b�u�r����7Kr�2��I=�uV.xwPw�U�9�=j�d���P�������q����U�M�����'�����4\.]�q��@_w���7P;�l�wG�Y�o��3����;��+v��P;i�Y�y{����2�nY�C
W�vS?� 5'!��(���:q��.1��;���V�H�F~03x�
��O��x��J���	#���h)j����r	�����ZP��%pYl{;}x�Ma�p���������O����#����~
�H����������"_�V$0���N#q(w��B�,�J)�B���!7�����QP��<��[-C��Xf�����Q�Q���
*uqJ��/U�I)��]F���<O���y��^w2��T�"��S�KnA��i��+f����R��u�X�(R���M����G��N�Egzq�������i�m����������_�f���h^��8����?�xL|�}gK��X�y����������	�����zs~k`��{���Z��9������%��7G}����_�������. �L��4��?�S�oA��V���������;�K={v��*��_Q����u�����U����S	�rY:N��jz*�o���<N���.���X���8O�
���0�>U�G=��U ;��s�l����W��X�
O����3F��������;
�b����t����V#��B�SoJ��������=���9��d��Y��@�z]�U��D�|��*0�W�s�o�������9_�$�)z�N���Lg��,_-���?z��S2[���6�6m�p+\����2'/�9��Y����*p��Y��>�w�7�����5_,��_�?�0�?����O0�f���iF��<p�)I����gSg}����m�������W4YfR	����|M��|��E�U
�!��'CM�7��Q�,��4���N5|��\������:��S��C�X���������	����5X���J������Q�O�\+�|��\���i�Zk�m��x?����M?��>�.>@5@&������!��P*Kl�|u�tU���3E�_>w2��|r�|.�� kb>�U\0p�����u�C�v6����v�-]��{��fo ����D���]���p��;���^���p����r�H��2�wv����uF;M'�F�N�����E��|p��������D��),$���y"<�%��9[>�G�2��
dc�����r������Re�nhIT&�d0����;n��dO��n�|���S�`����������]"�D6�S�����
-��NzG���)��������)���IB3\vA�#Z	*Y4�Y�]F�y��b9�]Zf��
�/|/��K��#<k��������)��_]��A���������f�i���r��]��l'����7�?���O=8��@�A^0��}������?a��H���&��O�J�������%X�;���,n �_�xb&��R�U�<l��v����\��i2������1�����-2��=������4����m�L�M��>��b��Xl�����yT��at!\�%}tQ��(`�����E9��h��C��2U���oV$S�B,Z!��J��
�J��=������L��v��J_��'=��l�-:C���~�O1��JDH�JA"�(5g9�D�(�VSBM����j{V�O��X�E�)43U���F�L�^7��/(��O+L�mj76o��A��;a����X�g���
q<���"K�g��a���Yhg������Dl=�(��z\29�U�h$3�*��b+vz�C
���a�������PR�g��[�0I�rn>aBAUh4�x��?�VK��N��u��:�t�*#e�������Z0�,�_�b
Yr!<�k��j��C������A�p`�2�����`���t��H��p�z;U�q4q����r������5	UueU���q���Y�Y/>��a����7�+�jh\���_A FI!�n{���?i�Z�a�u]�;�Wd0/��D�f������/*���9��DJ�-(�(\�g~Q�&��zHU�Y�6M�A�[k/���x��8��b��O��P��M��+���B�0;�����Z)f0\�ZT�Db���Y�F�8�K�1�	����
;v��:���;_��������G��M�T���bSL���qw
�����CV�h�CW�w��Z���@���.�@���?�;�����d8������X[�P++mD�<9fm�B�"1�D|3��M�X�o��5�����S��Z��
oA�k����0��
~��G���Y�����j<&��l���s���y�O�=��1d��@D:�4kj���u��Q:`��k:�=�S�i�G��-j
Z���#m�P=��F'���������t�*{���Q�������0O�3p����9�z;0������;��`<�a6�����#����U���8�)��X�4�	�����I��L�X����_������/QTX'Q��>�/���fff���KU��;W�������"_��
��rN����)�8�{{���&�g�C#���2s���I������q����2�-��2'�7�	�l�4�g�����N�CH���ojXd��[u:���0�W�%I��b�A�(�I�
+�{#� ��p��������U[z$L�I&w�5�����n�o�l��
c�q�}�����k����&�����n���B������������	��Xh���8������F��>"d��z������f�����C���y��������(�C�f���&S"��x���~W���������(
�h��H��Y����Uu~v�F��s1�������'���y���/d��"N�����)uVu�������q=Occt�@������!]i<2�E�3_��Z��cU�k4��_:��oA�{���B Z�L 2�.�5P�;�`{e����Jy�N�?;udWC���I�3%n���U���Z�y�#���+S1],+�h�n�Vos�,���|)1��f|:�s:\�J�j��}J����b�/����~��uxFI�P��i�g<"`oGM(�"�����/��O8����^*^��v'V�����*k�Y����JV�u����E�D^��������D)X�����_]}v���Y3����p<Q��Ax'�b��	e�5�f��q+#�T��glY$��c�
U3����A�r�+������$�yKS
V��s����Wo�W���W�N�T���Z�%�7��������'Q�Bai^4���;��\�����>�"�^}+��,4e+������p���DK�<������e�!����z.~����C-�;��	��"u{zN-��N�����6��� p���)Q�7'O�0eX�����
yl���287�k:�����{�G\�?��a�&7�,���]�.$S�^�I�b��}��EQ����(S�u�d*\��G��H�V����`�p�^�r�[���y��:� ���Qn���G|�P��k�*M3j�T�V�������~��c��a���D�ES�B~v��)�Li��hjN1��1���GW�]�������W�'�k�L�]��q��2�9Irx�Yt:]U������VIx�?��4y7�����'�$�+���#��[J���TSK�s�'=F��%
�g�+Lj��������-�W��w8K<���/0��y���|�&�$���U��K�Ot��2)S=g�X����Ws&�������0W�9y����D��)o��JQawfD�f������*�b �%a�kK��:��D�/J�	[6�����K���)�CcOn�CZ�M�%�'�X�[h��[��S� `%��k��-,|�C(�hs��XX�>�����$#�$Q{v/��$��}��+�hu�� ��h��d�l@hW�c���tJ	���f-#���/������b��
�U����O*Kq\$Fs%�N_�B��M^��"��3�+� 3�����p����������d�-7��ihG+.Uj�45:,�LdcQ���Vy��Y`�l��,�}Z��<~ri2�a�xv?��pq�����W�sz|5E��/���c�P�g�����n���U���}r��F`�~�����������S���>a�A+�p�����H��8����p�{��~R���`R�=�&KW�Oe�0�fQd�7v�����X�8gp�O,��:�����1�����V�[��i�7���?�����3�O�:�3�XU����M�z���$����7,��a��k�Y�-���&wTN�i7p�������,�w�hg�.�XV[���y�N�-8w�s��S6��^�4 �:����}1��M#�P��f���:$�l�,X��K>�����]���b,�J�}��|Bn�T�S�s����PX���������U��!�bM�)c�@)�+�'�����/� $�����C{#J<G*����e	�h'e���02cU*��w�x/:����F�+���KW�_Y�W���c8�<�4K=������;^^W�6JK�(%�> �	C���F�!"��V�����h��[���K�Q�'5������e���G�D��#g(�S��w�=����� ��Or�eq���b��[�o����������+�a"R0;��{g�99�q>|����J�P���I��~`�M�Z(G��d�O��l�j�����f^�[�'�h�<A��"�K���d���B�o�=#���OK�{����/�t��{7R^$i�Z"�&�A����o�C�YlRFf���!�o'��F��FR%L��K���	�_����q6�3ad��p�S�������l��Q�djd�?��-^�
��t�r!�^A4�}$���R!��"�.y�Q�o�#�(��V��o��O�Qy�r���b�����:V��
��P�B��i&�^����1���&�?Th����M$���-�_��/�!�H�M�Z�s,����2HL�;�"q6kG��#C�w�zME0�tI�w7���T�QY��RW��c_�JX�GrVi>t��<��c�q��Jcvt����<��-��7n�;�5����'q�'mVR5Y�`�	I�P�te��������$��G"�^�0/��3���H�5��	����TI�D����r�T��*�8
����=��Zv2�I��g���?4��"�!�3�N��pY���Y'#�]�p��1��o�=fz@&�j��K��:T����[u����g�T,_���� �Aw>�vz�Vk��~���{�������U2�E�J`�X�F��0Q��5��'�@��`M��vbV<��w���T�U.�N�p�sm�;��S��e�������M�Qu�-�w�����X��W�9*P?w��gmwR��v�NQIHh��1lt����$\��$f����>��b�5��`�	���i��$��u��1ur��XgdJ�p�l<���4���c��������{�	.)?A����`c��@�Zl`r��a���o��O�������f����@��������5f����6a4)m�r��<�"�<�*�n�eK��}4t����n�����QH�����
�&��h��K�Y���kQ�����g���x2^��U/�bV���~�q�����y���W(��*tn�(
8��t�oN��+�t�����@h9hs����x;��.*
��]T�������xC��;|^�m�[��&t�a�g�zW-�p�������#o<��Z���O���?�[�X[�J����z�m�yl�2`w6�a�����A�G���������Do!��+��Q�qN��(���sn�������t��J"rsT8���t�;<S���_�����B�pG�;��g�Vk���G��^�&��X[P.[q��� �N�����M]��,F��f�L����;-D��y����%�c�X�gNe��e����R�hQT�R��(1Kny����������1���=tx�j��g*ip_&��+pd��<w��Cgj�C'8��$�I�Y������B��[���e(����T���N��,�^���������Y�z��0�Z���4�A��@�{�o�(�kv�TR���v-�,��G���@<��*[f'd������p��7��G	�qn9Dd|��"Z��z�U�l�J��<q��l�<\���p.O_�_5Q��4�I<���z����u�����;�!!F�W�%���S7��uG��u�|Za����S��-U�+\���
�� ����4Ea��/�"���=�]o!����	�)< ��!����3�W8��@n�����8�#�]�q����st�j���O+����4��K������t�y���W���@�!SG�%��aM�7�Q�	�Gy�e�S��c�(3����Il'?��f��7"_tgY���D�+x8����I��Rg�Z�+��B��)���Zv����
�y�Viyy��=��1u�V����&�q�^@�J���
y!6�!���NP��/�\�rU�|�s���},�'������	��K�VJB���� �LeV��G
�zR�tz����A���	{������O��l�i��n��Lh�E4������+�#����������&����I�xF����Y�	�0��$���t���m(����
�4%��(I����g���<{���p��9��G&�P����9f�j�W��u�����e5iPI��X���7�[�������7uO�<�R�FH�6�E�E������j��6�C�.v�0��t�PVOK#Q�`J���I������R7��R�vC%�	�8��q("��?e�JM��od=�H��h�j���naM�B`���YO���j�Gn����}��EE����q!N������D~"V�l��X%�Hx_�dG��Z�Vb:�jjo),��q�J�,q���9�^�TZ&���{2��s��������e�������=D�I42h�����P�R��y���6�g���;��v�����������R�m��v��&�o�E�VQ�����5hf���������}�Rp>�Z��0<\�p��v:C��3�p�k5��0#s��&��u�$�e��t��������?�ur��N��0�_���LX*@�6���/��F>�k�s�Et��6;��'��XO����^�YO\��Z+��\���n}���|���5�%���bV,�R}�@�S�������dQen�+��q��/��X��i�|c��q�t�p��em���Cs�i���������K ��>�.�\T8_WFR*]:��1I�1r��EX����P���p�C!����`�(������<�?%����y�d7
�E'�w�6q�M0V�h_F��������5��wI����eD1��so��u@&����p"f��\^
����f��;],?�N�����������1��j�Z�0�)Y��bd������5�.�Rk�8V��4�����-9r�M&W�//����������i^�-�5vu��r�3���������}�g#��G�^��
o�`Dg6����	�WHa�Tr�����|����-�22i*��A��A(�"!_]S��7���QE%�fc��n�jS�8��������Qh�9`���b��KxL�*�j���Z�t���HarAh��1\|����f�Z}�I��?f���Uj\TN�_J����nr�w������o������3����-v�Y��T�3��Rh���&i��-�	Y�/���mH&
��;
}���c�.e�����������u9O�
�	��G�����)���������{�P�$m���8�ow���I{��������5��4��Z�'�@���"��\ ����A�5#��}��DrB������Jc�1T��jUP��X$ f���[W�VS�V5��L�l����C�4��	[T�������r�����vpd{���(�d�������dT���&?����f�Y�7��X^~�����n�&�Cn!G�1IVD���>;�n,�)7���Z+\��y��L*
��rN����=���#�P��)����#k(������t�~&�(��l�J��<��R����������H�r3[/�4uV#��,:m1nO������|8���N��%{�-d�6$�6s��$"��A+$���J�F!�	q������0�����bD�dJ�3A_��Vl#�9���	X�W��+cI����f���#)���L�1�Bp�Ca���Y�g��l��E���6��(r����hu�p%��!��]�*X��f�
����uYf�.E�Ue���+�;�>)���b�x��+U�M%;��{~W�A������+���WQH�r�-L!�����lH�k;���'�_�"%}
	t�TKO;��]Z��"�R^N���������Y%�~����XPV{4��*a&{�/RoB���vlw�eH�tIe�9Y��QO��y�(]�pA%85��7sr�!��y�����p�
)�7��t$wp��d�c�4���Ve�?����n�s���w�Hw����i�?S�Q^D�&b��e
�L`���Q���z��ZR2N����7J�I���.����(����E�N_�Djg��LG���������� ����Kwn�`��C����G2(MWo`�����/-�HF�B��o���(�l�)�4��S�������v��@9���-�$����m9��H+m��L�(`��1W�D���7��f�����E�G
o�W�)���it
TP��$_��}���c[k����@b]l-Lh��	Y0����r�2r��zq���\Q��!Bx`Qs����`�I�{;�9����)X�v���J����-�<��sU�-�����������rQd�]�2�)u�����������~w4���v���5�Q�jk��)�?���
�i��J���c;G�����M�"�����;�$�?���ZZ%��^'`�x���m��S�:�"����%��)������m4M��1s���#<d�T�)�c��.���R���'�������5�>I}�b��J����;�8���DV�=_-�v�3RH����3���Xeb���E�J��tx�}c$�[�}�h��*��A����r�y~�����;h���?�y��`RQ�-3L�"����m�/p����t��y���8'��@DD� ��/��I
p����a��8e�Mg���TM�Z�{����{J���_�>$�2C����/H�>��kV;�_y�����A�&����P�c���UDQ�2j��;���E5�,��e�&�R9�s�KJ6���T���;�~��D��S%��q���k�Mmc�2co�tQYg� ��E.,&$2�,�K
���.
@z�F] �1�����%���?�0�}!2i���"��Q0Qf����
�������$�]�|kQ�uH�7��x��Z����{�A����q�#Ul0�
�w��.v��� �G>+����O�:���8�ue��`�m�o�`v:6�cRe�oW��c�6y���9(����m��aT���z�6�PZ::{���#;Uvf���[A�u-�u���:]�����������Kwm+
����2D�n>~-�2�/�Gb����F#������Y�
�\�C�8�����#,]p`aZu������T�no<����A�s�7r�|��1��F�Z���\��a����\r�t�����Qysu)G�s���R�%��}E�x,�LB76*Z�&�srU�q ����	�3e�u�@mB�1z�
FT�i��Z�jM�A���o����=k@�����a�������Yja3+�w������&��VS_�%y��������S�Vu���Ho5���(�9���t�R�)��8b}�!,������}�PVL@�U��UJ���U���=����:�.d�s������jzv9=�������W
ex��5�Uo�:	���EO��X�?�����
�6�F���3��n4�a���r_1d��W4��H���t9�1Sv|�v
+�y��7L������~�`��2�(����������)�`�5�(��q���=�cT���(^���������?w�V����~w��Wm�C��
������-�naR,)=%2Y���$-I�6����`#�&&��r[3��W��MX�.s\�[��\�a��Rx��iJE�����A��z����T�0�aR���a����~J`a��0���A��}�	�
V�b�s|7����;�Yb��Co�UcZ�N?Z��+��~$�LH+���j(�(�1�	z�31��5Z�T��JUv��<��+6���)IIB��%SE
d���'F���sQD	��@�K�L1`����+E��`8�W�A�0�����{�@K|5�&	O�x�1e�$���������v�5��������$�V��LS��"�q�n�T�SJ���|9�?A�)�T�G�p�����31��&I�~�DX�P;�Tbe�@�.p�_�@Q�Ax	�����o$�X����K�u��U"��L����g���pg�.f�����6=�n���)kF�}��i��w�����dj������<���}k{�����U��{b�I������l��-W���Mr��(!&	� %+m����.W���&�QS��:;;;3;�������&?�7)o�3����5LV1��G�M>�m���u*�����:G���dL�GF14��y�������F��>C��dS�!k�����
�����w���m�����Y�4�mN�KsMLi�nm�V�V9�n���>��3��
[�k�u��bK��-AGt[y	�����b7�������E@��{���p�|fNN�]g���Y�E����/�%��2���v��E-D!K��5���	�������^4��sb�,B�����?�o@��u��a����c�����rzc�.���j(}���9 ����,����n�'�hyH�D�-6�#h�1���#�����j���s��*"�N|�i.sSmz�����W[���s�����>�t������$�G]w�t��W��l��
�a���)��\LH��[M6{�t����jx6X�
�1,Nu >&J�F#�%����h������Y�1��J��e3*aZ��$e&3������8�,�x!�aa2�EW(����p�0Z�QuV���'�5�6�
Tg�%%Wi�����.���Xf.��oT0a�i�
�6���B����F�n<�������xp������������E�D�V���
�u��O�l�K8~���/�[���;�T�#�mPyE�g������nM3���y�X�BklD}M��`!�W�+6���e��Yb��<�?P��#�M�\��;&[\R
&�u.���E�_���fh�[Le���[���:N��v�c���;�L�jk
��R������	����dc�X(�m���5+��i���fE�IZ�m"x��.�&�kj��c�W��+�6=l3]�Q)eD�k9���t��Z��5��N}������!�����,�n��bY8�T�M��?�/8���dl����_�����A���v&n���3��UmDm��t&h��*J���7:�O~n��\�u���%�3��	���Y�w+��G:������P�3j'������!M�����
���Uo�~tM����q#�i3�)�����L34#������6j������GW�%��Y�L3�Z�,��b-�E`����Z.<�B	� ��-���sr�����y�����]��`��D��G�C�j.�������
m86]�bF�z�f=e	�7m������v�������	���]��*8�C�Fqf������l�r	%���P��v_�t�}I�mj*1����������1/(��R��R	���iG;�����G�1MAY��NAX�D���|�N��%�����%Ui2S�|�-gQ�s|��0�W��s����l���< ��@�����AY���M0�>�Q;wq�c�r��di&u)��U��?D��H��J����s3�v�E\|�'l#�l\V�E�����0�P4�ny�fi�6��pS*���)\C3in������e�|�� h�>
h�P�K�b�����r�����������8�D �y��U�e�d��4,��X���K]�lT������i7Z	���`Q�?
��p�>�.p���M@��[1����~��0v���i���������bYS�Q�]�[o7���m�j^�s;^��w"�g��F����G3���������}���}G���P�CtGs��)yD���E�'���p�5��)W�Y���.�~WU���Q����-Du!�/���m�X���%�
%��Y`�4������W�|����`��K�?Cv�-��Fz�gr�Dc�l�u0�����a���
+1����"yJ�n��z����\{�+�l��l=ys~|v���r�����K���Rp�����x�)���n�8������7r���;b��2�8K��sWqDX;Au��&�&�92@*����1QeX��^#��z����������9���*X�P?��H�"���[�[�CEH,4�U~�����������Yps������2�*E]J�#��:*1�J��1$%XY��53����X2M����\�_�
V6;��0h4u��.]H]N��C�����!�K6��v��mxxE��9���D�87E_��P�)�^��=��>\x�����RG�a��D9���	�&�d0�8j��o��D*����d��N��Z
���X_��N��\�I6���/�uox����G{�*r��#ou���(
��M�3fA0��>E�%��0������O�� �L�E����P��|�����I�y��cM`���m���#>�v4����p�K�9�\hj�<Vx���
�
���H��?u������	���$������"i������k<�����dV5�u�z�s��w��3�8���������${�\`�E�/s�),)��xD0�C���-P::����>��������xB��&�N6G�"�W,��?�ez���6*�q���c�� ������L�9����:��m��J�������=}_��kbBK�i-E_�S�+V���M�f=U���	�R`��� fnS��w��UP�	}fT'm�6HY#���XoV��|�<G��|M��s4�V���s����UtW�C�Il6��Kl6R�[i��;e��,23�
�$d8���VpU.��F��S�Pw?.���a)�(���z�l����h2�?_� ���N��r��W6Z���wZ��m78��6��ZQi��I���TpBObVJQl+X�$����XrR�����w0+�3�+�@'����r�%	th��|�y3���=����Ar���?���aN�0'��q+�$���}P�r��A���y���M��Y��BH��x�#z�`l�rdg���bK�I�L�����)�������#=80��Dt���EvrM~�+G�����k�@��m��v�n��/kM������
�[xmK�qT@��d��+~v��M�	�N�Kr�zuz�����cT�����v�[N
/G��*����%�7%�Z�]-8{V�~�RO�h�Z�FB_l��|0�P2�+��x@=ug�&��9Y�`^1�8s>�	���
�D�`�z��j<���A
A�v�������Z���������m�8�����E��3������K=�Rs!;#TJo���a%Q��7�����a�8c��>��-A��c�?��
[����
�����w�l4����q5�X��������[!�M�x�!���*I������N�����-{��u��~�SAa����.���T��`�O&-�8�{���M�E����+�����Fee��G�9�����]c$B(*!�	E ���6�M3��b�F��_���R��_!c�a�.��v�BA�S_�m�9����#�XE����O�������J�g,'B����`:�_�8Y�`�I��1S��?nx��/Y�_
IdFiu���;��9�Z����n������k��Pf�`�'�u����������7$�]Fz�����3���F*(�����Lj�}f���>I�v��)��%^��i�q�U���f��r�W��%�����.�������?�%��
cb��WF�#����f��"7?���T����W����~�G%_���v!��{����(�������7'o^U+�[l��K&�2��sTq���}���U)�[��Nb!^:3w��JeS�"�n��w��v����
�m����5�W������vU�R)%�<y�F%`-t����CUmH��{2x���[��<���*5�i�5.W�E�X_Q��(<���qO�B���#�M��Y���QD-h�G0K�7���8`��7�+	#�:��� �4���8Pvk�u���%���$��%�Pr�����O�?=����\��S�|��g��M� |]��6�A�'<b��]�_�:D�tt�n_QL�Y��
��+#�p��n���t���*NE�g�%��P�*'��J:�����[e�l-m	�r2����L�u
�B(c0��+u!�=<DqC6���YH�r�i��!C?�8���x���������)��W&_+��u�n���zN��t���v�q��2�P��I!��������������S��$�s,�A)���PO��-��*D��7{��(cZ��A�����U��8��2T\�����/JN�>��G���rx��c������GTI��@�����*Y:�����=�(�W&c���������S��H���I�|�����`w����l+o��� �:[�#�c1�0��`_�L�LnM�A��j9�s�G���Q�b������U����V$�QEO�h �_�8i����b14L��-{���}����; �6
<��-�#�(�X����F&:���~x���0����"�;_��h�����uM�������g��P���eI~�
8\�q�3�=���#,2u>�SX����t��z�����n�F�fe�����m�t������~������$3��=����n��o�j5����^��o�����sU��Q�oO>Q#�W���@*^���3�@�y�W�G������{�$e��OB��1�2�����&O��syY����*NI�����^���F]�3n��"���&�V<U��P�&Jf�^��������~De0	�����vu�t��G���ua�8���?��S�X�e����%���T^-	�Y�����"���~�L5��N5��P���E�����z�P�rN��_���RLU�����h���^Y����`	�����Fo|���0��!	��\���q
+��4j�J�8����,R�ir����Z���f�VR��%@�()�J!�������"�-g���Qgh�j��������fd�h�h�h��W6~6Y���U��������7�'�og/����"j������
�ON1�Q�%���- �r��A0�H���x6)-lKv}��;�S��$sW��:�T:����h�
��ywQ�;�����F'�����.�%���M�7,r0�����w�yJ�n���}c��O��`����-���2�o&.���X;�S�k��]���;�+,����N���NO�`���*A
����G���`_f��(�mO��N1yGY�C�����c>�3�	��*V�� �J��d�rlf���)KOf`V���q�Y�����6�g�h�9���	F^���+�7��.�iM���7y��D�'7����Lo���n57C7�P;����Y����zG9�������l����������
�$���d36�������������m�
���f��&��<��Q��X-��H������^�;�M���3��������.��61��^�����.(G
��}�7Pw�����x���d��'�b���3
=�uA���F�~�Sr3$F� �g���Z:�9[�G��� [/P*��Z�t�L,����X��-���V�����i���G��a�n�0�F��.������J��9.)�=}���Q�������=���`�I�9��?��������J��#<�$Js��+�p���&V�D���&`M�
�*h�yK�
0 +D]!���\�)5bykW���*&�NJ3rR�X�rD������������������������'��._�S�`#�GL1P���a�O���ur���B-<�c)(E����A���R^W-RQ��D�8�
�5b�����)��#��>!�4�����/[(z����x�POa�!��*���y�4��.�y�3h��_T+������D��i��"�[�p3��v�m��s������{�F;&��#+b3�N`�b~��;AhC�
������!b1���X�e�K���6�5�+�+�����=Q���Y?�k���p��<�~;x{vz8x~���i9���,�	�h�n)������A_�0�kY����Xi��YiI�YP�C&��{�5��6�:	F�����Ek7�r�]<�?�_����&s062��0yK������R��xVp^,o��n�b��~�u�#'�,wdY��18}{�!��}����J�[gs�f�!�!�}����~���8���q���C��a��U�+���(����/qqe,�^����ZK�F�FK���E

q�,|t�.�lC<�A
J�&�$��%P��>
�f�#������S0}�[�sBB�x�9�9�*~�Ix����Z��i�FtB�:�(��t���S�"o�M��SW���K&X�c����;�Mb���?{;;���~5T��V���V'���pj4�����-�E-���p����z��f>^x��\Z�����)�U��-V�������O;-
�����.i�SG��a���V��4��{�����������M����V�m�wA����12������o)`��3v��K�z���~��q�jYVH��>�����E��;-6��;�fG��R$������a*;�����?8��"���\����N���M�]�C�F����5r���/Hp����O�\���0����Nw�~�~�d6e��C��bLC@�����4?���P��,�W�~��Ge����c��a���j5���7��~{�md�o�Vh�o�c��~�b������Yh��+Os��Ur=��@������s<2��sr�$
��75b�:�:�����F�=��_V�dh��I�Y�V�R�O��YB���$1��b�����
h��`�^|N���os0���4P�L������MF�s+�,�'d0y���La��?��+@�/^k�Ra�T(��B�����#��$L�+.����%c���$j��m������V,#�y���V#9�9j�)��O�Xb�?�vPE�M��7�~��=��H�c,+�'T��
����k�
T����=��W�3��plIVf�{<-�j ���)�V�@�&WyX���H�|�x���������F>���D ����L���w��a�� 7�������m����|�Jg���;F��3R�lM(C<�L�����2q��*S��N���� �Wc���^�� �J������B�� #�����=��d���Lw��R5���DS�ml��o	�������=\�{�9�/�t���[���j7������5c����M�-5�l����6=f��C�x����n�@Q�����7������E����d�I��[@A����U�u�r|����H�iKw�E���#����������5��|�#��0�6_@�{�Z��w?�b���\��>N'�Do���f�n��^���<��x�~��n�H�N"q	�n�'�=�B���)�0p��e0������:Y	�O}��>����<n�# ��H�6! wp����!!M3J�%���*�=8m��|�G�O�s����
�G�"��
��e�(���Y���P�]x����Ps��
��m��-���K�?��R1<��$M��h��
�(K�.���Q����O����?� &Tq�?��g=��Sv:���:<�|�5�"$i6�4dI����0���JR��I��L�2��0��������f����:�'�t�8>�7hKR2
%�D`	h���J���KQLx�^da�.�K1M�:��[�v���;��h�4�pV7���9����TA�������r�_�JR��������w�"<��R�P::~��EN�Cg����=�|W�����}Y��q�����\���X_��U����Sw5��B�����Yk�d<w�1T�����Y
�
 X�����o;s1��B�tS�i	������9]f.jD6���	��S�����Q�L�����R�Sm�^�������8�!���9�`�
D���1�
��qTH"v��a���{�Z�����^���p��c� ;Lw�a���z0��cdj��"����&��h_��T�q(�a:1�]"%}}�9b�/�
u���RAlQ�*a�R)����*�,�6����������g�.��O�=>�=8����������z�~�g�.H��G5�CQ��IN:����o|x�I�u��#�uF�����q��N������<�9��*�\kX�Gdi�0v���[��Je�Z��4��j��
;�A�f�HM�\=	84\H��JC�������R��{0��%h�W�9�fD��������b�|�������ns���'�u��!��W�X0�y�i�u����s��'k�v��gR�lel��GN����V
�-�7dS3I��?�|k����z����r�.�R���:E�-
��c�N4��w�i�{�'f�Sc��j���z�	���C�WN8��
/0��2#&���5���D�M����0����of�~�s������y�Y�jA����r��e���R��
�D��J�5���5�f�8:g��H�#�Z�����r��/	���iF��\1(�s��F]�������S�,�1^S��x���w��*Mo�V��[]Nn�r�TwRI�5�- AU�%!8����y�|Uy�5�J�A
?�F�(��5���	�<��t���
�l�m�a�����U@��$)tbU(������4�J�@z��-�Gj)�������hzPM��55�6zj�G� b��k�bwzN����A���������O����lQtq��e0�aR�~����@R"����u�UL���af�*�V2B���t��P(H�Kg�����c��\%�
S�0�h!���q��r9����NfT��q�����6S��z���:v�����UC���Xs��<����M��E������^������������+�\���P|�,��������[�����D�W�D�b���O��#1���y@��*��)��c�k�&6o�Ea��������YW�[�|%��w���_�����	�V,)�>��y�'j��;����FKi^���Dg�� ^�qc��>�17�-�A�Z��5���,��Q��
��[,�(�$��|��]�.$���(�W�AM�"���"}F��[F�d5�]G$r~D�E\��B5�xkbjQ�H��0#Zo�5a�i�D��H�/jI2��nt��]W���S�j�X���V���_��7�m�2r��S�z%�m
� O]�\KL4���p#<G�e <�gK^�6k�O�c�<�
��U���^k���b!���@��hHI&l-���b���KT�9�)\/�3��i�5��W�X�����l�uN�'��^[�5�s���J�9R	U#f�l�Hx�P}*!aN�K���GB�uX<*����8D��p6ZY���ZK����	�7��@������J(����7lz�F���G������^@�_@94�������������1���[,Jy����djCEj������1;Q�Eq2-hk�.#X��
�`�;��
A��!���PMN�#��'<r�����Ue�	Ry��o��tWr�3�v�g�����/_wn:��~%wyz���.�r�M�/$������s0���S���,�{��_���MJ���HJe��O/�����9[:^D6��ku��m���Z�������
�h,o;'�g�����|[�`r�
���v,�W�h �etp��F �b�������c��#�%���:S�M[����?z�?�<��m��;�#�i�X��m���DNCo�;~u��������8��f#&U��Ewn�z�1�����
b���[��\�pO������*��g�]
�������9}spvv�]����w��e��@�	���X��/�z���[���c�����(=w��j��%�p��?���g��!6��on���j]���6~%���yq�[�������9�E������u?��`��]�m��?pPF�:���z���`9Jd�7[^��}AB�v����j�����T\�O�J����V��c�zY���jF�#����&�|/�5���Y�,��QNn��^J��5���G�@B���9�B���������Q��Apz�^����F�uC��<�!U�O6���b�4�6��zhrz��P��c(���U;UW��cv qCW��Fv��z� ��[<>�����������Q�����J��,��?�������oTFmy���a�;/���W��$��D�{i�g�����Y|�����[:�%{%�PXF9xv��u��]���z��4��N�Z�Z����������������v���k�O�	/��tN��J��n5�����3_z����	���7.[��k�Ti'�GF��'��)���v���:5�CS�v*V�B9��)M��mf����*���Uv�0��
���@���2�M�dIy�������+���p+��e�w�V��YN�	>�z�3��2D�iD�W,��2UZ��|������l"�����Vo���j#��s���d���n%w��{�'��T�M�4x>���	��@1����kg��C2_����Tl��'*Okb����u�����>q���t�����A��;�=`[l��}}���q��(h�P�)����.���Yw�&<^�xd��� n�Y�%��B���=�_���z��v�C�������u����"����!{�:�U�{���SE�Y2�]�o<
V��c�	�������D�l8���g�����/�{?����J�4,���9����7��j���!c;{��3/_��J�W�?~�G�$�>.=`"�]y����f�
K��fMR4��z>����B}hN6�B���7�F}��vj���y���e��2��V6�%���=�S�9Uz=����VU�6���X�*�*\���/�xF�$n�����zx�]����V���,h��LX�nQx��pH�����^�����++V��M�E��%"p�[�5��hM�|��B�qu}q+��//��FK��iiv�
�e��w�%Jy�]��5R�����v�6��Z���%�~�\�f�&��=��0���h���N���m&!G�N.�{�����Adp#Y�eU������aU�7���/P��ukS�;�Ce](��+�d5R��2LI��Kh����
N���O\�DT���<H��B���q�R\lh����S3��6q4�����_h�[����[-�����Gc`�#&����yGi�����	���.�(���s��E�G�(M���!6����������x�N��&�>��?r&�����O��� XWj��=D����ST��
�$�z��E�������P:�e�d�7���������Rq����Q�E~����~��X7�-j���XM�_R���;T�R�~�EL��}M�+�]�x��M7�yQ2-=Q�$����eB5�P�A�~�uI��}���K@	�9�@�	�6�Z��v����u��Z���]o�n{��lnCI�6�E`�����?���@Sys�&N���������/a����lQ�G��J�r�-�YBzbAw��H��h���h��NTT���e�jf����d�����`���,���k����j�j#�5��k���M�M*Z6)B��kc�=�DGx9�S�KF����Jo_�/������7�g=��3�i����S�8��*s*e$	�$/�(c��1Y�@��i�T��\^��/���a$�!}�,��u	JA����6b�b�5�a-PBP��+�F�G	����V/k�P����S�K���l�,�.�vK�)���v�6�|���%�c�RXh�c�3t�&�8�n8j~�z��c5������U�5Q��D���z&*��KF4Qs1�9>�;~V���F�	zk�R��f�<��5��6
�h�"~�Q���@��bd�0u�����}�\��S-��D�h���a�[	aJq��0��*f���U<{���NC@U ;c�(����;.a����!�hH4��4�� �rHa!t+�U��R']��u��/,l��Gk����!����u������X����9��6�E������^�<�6���6\z����QWl
��>����
���
25�����`��X�<mk��������:���8Q�y(�.�,9��3����\O������C�Q� �kS�^��6�����b,5�ybt���Z���*<��&H�c��q8�^`4�6i�8�H�g��ip���R���'�����]�e�����{/��,�=�����R�����XKw��%QJ���zY���-��h�.����=���3�P�i�R�{���Nt]��I�.�[�6:����(s��D�r�^(�B	��ugs�g26\�1��BDd���dd��6�eYe���4A/�y�������J����p�!�d|TQO�l�lD�e�m�����g��t��W�b��K�X8�T�1AA�"��:O���=��;�$���+�`O�?����0AD0`Q�B�7�f]��Q��|����U�a�9�H�**\Uw�B���4E�5@`�������?n�W���d�?Ro����o��z�qZ�Z�e{���;�WZ��I*?�%(6R��M��}u���o��s�T�!&H�XooO�?a4���	�Rf��:���?�V8�f�L�<�46����]p�t�=`���z�����U@�Gr��H��~;;'��2�>��0��$R��������Y8��b��u�@�~����
��DV������������d������u�Q�]�5�����^��C~{�-Q�s��8{b/B.�3w�"vnl��j�q���
�Voo�|���)C��UQ��n&�bQ�i���bprq�����j�8��9�!8]�����h-����8�%���U@"��4���Q�N&�IB��B�d�$��
"�@���u@������\����<�����C���4���d��� ����B������;��j�V��r�V�v��:�R��e)��J}��S����L��pv�x4��#��|�,.��~u�O,�6���������p�I����������hw���n0r�Z���r���T���Qg�1nU���f���v�P�F���!9��%tL>=9���N<q_xK�0�j��zkg�,�$�5����rF`\��Kq�#>�]&
��Z�S�_d����\�D��zRD�F�~z)1T>��>|��;��YM�R��D|�e	p�G��_~�>Es���x��b�B��m���O��|����0�O��y������
+���cc���Y�xg�9G��O�/G��O�%�E[�XLK��i[d��v�����2;�W�,�2�����-�S*��v��#�)e:��+w��|���W�+W�/`���l_)q{�M�HG-����%�5�^�b�������v�s>.L��2s
�Kf��Sn��[��V`�����/��d^�������dK+����\��-��X�T�Z����}^���e�\�}WR.��6�L�'/���a�> w_�PZ�������%{��#w��6Z�?m�?a �G�	�}���7�\^���Q
��]N���_H�g���.���@z����?���R�������F����eW��������m�[���1�H�?�u�Q�ow2�e�zLH6_�T�?�K��3��`�*���3s'�"��_�rd�,�5�w�`.�5j'�(��1�p�H/l�^�����n�hI���S2�O��a��EC�]�j�����j9x��F��)�Y��r}�������D���<FS76!�����i��Dd\�4�,�i vs���s$x���*�-�K����2��u��E|S�_������u���3{Q�����g�������G���.���8y�8��2��������5�)��FA7�����5F�nOL}��������q��H��qc�����m���g�@,K������Y4�?���s��y��o�o�I�|�h70\>���Q��\�� 2y���/�n���2���0�����Q����o��lBe��@&R,8��r�].0��|jV<��
��mG#o�t[�Z�m6�������,yN3k�S��u����?��S0�[fl�	P���tYk������
5�[�lrk���hVD)����D	d-��0�j�3��82W24�t$��������(�A�D�W���s�r��*Z�U��F�n�����_������*����vm��h�����F�R�a,��x��fUsu�V�U4������U�7}�i7Y����������g7����[�6��n��7���?V���
��e��-�"@-����=O�����?�a���z�s\`�z�Q������U�]�[��=��{m�����@W���2]����������#��*�uy������_5��<b�~����(�>��]�=��/��e����Z�v[��:j4������Y��=�8|i5��?Z-��T,+�$N	�FL�f!�hh}t�����I
��Q��
���6tU�������z�`qk9��G�|v���e���zP 4^-(��LX�.��{��~��e�kA
����G#�
�F�	�`��h���.�>�����%AU���VC���!�(��R��pG/�/���3
B���8�
+#1��jI�N)�j1���.L����B�J���E{v��k�������B��x���{���U����Z.����}�fy��y��t�����O���w����<�<��������������������w�����o�=��;����'�K���]��)a���4���R#��V�>���q{���������oX��!Dsv��vO1�\i�\��]X�������`�ynRJtj����iy
��<�{�F��e����t<������?�
Ix]w��k�k��Z��.P��#,�o���P���L��0W�����dW�8����n=��X��`��Xo�.��>Wa���AT1Ab:�"-�b�l��;�A�V����D�Fu����	CZ����^0.%)m���tJ)&r����.�����2��V���w��W����6 qat�Y��s�p8����������+l+Fcc����g|�d
����0L�Jt����
�5�Nv��7-
��D��f������tsIa�#�`���1UOA�b]7������E������q��������Ik�����N^�)EO��$��9���;�]3?[��E��"(oS�� ���E�����p*>vQ	��*�E��g�)�(q	#���B�%il�9����T�w"	�R����905����������������.l/���<�H���h�����lU��a����
��	��Nq��He{�>�w���+�����T�ZAE>i,z	�����D�%D|xw�_{���m�I�o�	U����E$i5|�C���7����!����fNS���?�PK�1�
;�#xM�]�Z�������M	�rB�?����F�\m��N��w�9BK������v��.}��Y�J����p{L!|�#���!~�W���x:,.zBQ����I�m3����+$?9���W�c&2�1'��[^=^*77��szvtzr�xy�G�yhpE_�W*�*�a�Ef(�tT�*��%{yro�/��8UZ��N#�&�id���|Z��
��YQ�f����v<2��6,#�[��<�JY<����Za�5����~�?�"8c��]f�Pv�-��Z�����32^���2���^m��v?s�c#�q�����v���8N����k�
6�D=������D2�������t�B������!�},b���^}�#JH��;���s�9�M��������E4L[��Lj�;�1(b3E*���3v�W�G�ZK-1��'�^��)+��[E�*2�<[�*b)��<��4�p��l_y��T��|JX���Ci/�R[,��Nvfln����oO��:Pr����������*+���}���cho~�~N�������H����n��RpU�!C�MU1�UX�����F�|�����������������_����mw<��;�f���:�z�7����^���#�W{�������jV�����v�Q��E�L�����pb���Z�|&l��`��.Id�H��@cA���px!f��D���-
muL']�+�7�P�<j.������}}�������iT��g�/���?~S���{����_�~���Z���z]7�z�I&��V�9�Y�����p�l-~A����7k��Y8?{�����4��(
�F������hj�w�o��Ik)��lLi�����jX�~ei����!���_�������?����w��}�%3��t/�N�+(Th`�P���As����
�]��p�R����B9��	�@��u��K\9�����`�L�#��'��4~/�j���#K�g<C�
S���8yu>xZ��z���Q�7�9��Ds���$���%*Q����n��Kj�wX4�s�O�������Y*�d�h\����	fE���^�,?������p"���c��'������^�����������S��O�Z
�D�s��/2o�'�)�&(��_j������ycs'b	�]
�����Hc���k���u�i;�v�����5�����=��jz�~��
�������F{���l�v�1n�*��Q�����9�[x�+l�"l|`*Q#
�����v�]��*��VRWhr�o��_�8Z��[�=�N���P��/�NO�B�P�;��-%l5���0h`,�>	�)9��16��G�.u� �`4(�cs��`����|��+��
�������ok�]�����8�GK>�V!N!\��Q2s�vn��>���Q,�����\�o���^|su����3g��c�s��{��}�|�������vk�}����_O��iFg����m����Y����[O34�t��+t�L������f����7.�k5�1r�N�k6�_���c.�B�"����=F����a����J����o�f��X���t&�%�6�`��E(l%�%��������t�p���k����$�9Z�E��K�4x������f����
43�Px2:�h�����Z�J����>�l}�jl;��QF�8_�!��C�.a���I�c0�|�"���QF�.f�b��7�H1$i2S( c�)��- ��z�M��o���(g�\�-��9����h����_
��_�.�{���^�F�@]c�|a1�|t5��7���q���r��;;��SU��b�V������
:�:���d��KI<<�A��4��e^�9E��r'		s����iu��:����^�ixks"������*������GO���%t�h O!6�g�)'����sf,��P�C���2��X(����+����?�R����a�����7�hd�BQ�=���	��'a@����S�~�x�R��e)%n�h�W��ZZ���O�B���������V�#�$6Hd3��Q�he2D-a���DH��:�Mz�q�}�V�P�|}�����!i&�d��e�.jZ%��g��������w�q��%���w���c���|������P����~|l�z^���?{��
\���K��:��A#�sYQ����;��I�v��HJ�[qy�j��lI1/����g�C��K���c��}:[�F�_i�3<G%H_4��I�x��z	p]�f���Gi9%w�_8�&a����O�!�K�0��Pg������(�X���>�!g3D�a��srx����L����������V6h<��5��'N��9�������`r�s:{z��t+@���C�_�����	 �!�$IOG(x�����F���r+�y�#����������`�xt$���[\_
�����lO�>�v��\L�B��1�n�H���>�~�}0qG��f����tj5*M{�9���`/^����q��.��e�><�X������b�j0Y|��$��#��S�)^�N��9�����c)"
$�"j��IO�.�����.'�O�gi������=~��"��^�����qb��W��AN���^�>;O�����'�j�_L�><}���L�� ��'o3����)���|�B��Lbg+&����������_�52CGNl�F��������e|	V�F1K1re8��m�Q���b(�I6$
E5��do���=J�����ro��&Yd�"h��E����)%�0����D����DVY��������?F�7�gYi�0��������@PC�T=�8*��9|�x7L�a��=\��^���'�� 3��W����B��Y�;*��c���x�9�S�T��5��N��M��-M�P��#N��4P6�y5�FtR,z�&�|����^v����F��6���>�����=e_��
U�H@����K����Z������EP� !L�n������T��m�����B?$���VC4���LM�K@��#o�.	���V3n�p�}�UiwF��e�:�����	rK%i$���Ys4���x�nl
�i�IU�r�����f�
��BgL��^;h?���������o��[�����^����o��*P����B�8D���B/��7>Det>6�x�2���3lgO����<����������$�R*��E�Xb���8�;*Z�Q����;����������n7�����o�G�Ui@hg��W�
��z���&����������6��f��m��@N���u������{�n}���N���o����v��w�_jG���tO����kL��g���(od����a��������w�����������n��:p��$�6[B
�%zV:��s��a���vx%�^�K�O�������+��Mu'vD��t~_{�?��]���!��w�]L	�7�h�Lg>����R9JbY��������M�h<0���|,��������� f/N=����dFZ��3�y�.�@�W���N�e�����������Y%igw�O����
���u1��K�	7�
o��������w��'��_�_�n�W��N��}>x�r�~:_~sx��Iz��'�'�p�5��d��o8�[�^F7dC(��u��������F{h�g�{�*�����;�������
s_	9n4���Qw�Z����������k$}�{�
j�g��H��^R�:��������j����r	��O�,R�^UO�K��C}�����C��?s����t:�M6 �<��g1	��%��Q���g�<�A��v�bw������p��<�RHTP�����z������7��Ql���FH���j�����>���������t����z�\����P�ulZb�?}Y����v����9�U�A�vAS�������8�Y������(��t3�-�%����"T�"�M��dfE�G����y�)����LI������d��G(���$�/�F���!��H}�J�!�#��G�;�y�=<A��?�@}3�=�n|�v��|^�������	��������aU1�
&qL�YV�k��\Bv"��=FM
�
�7Fu#�t{�p�-�gx\�.�Z����S������ p�	*��|��o�<��!F/��l��Z���:�F��n��KOP�I}Ap�-�Bq��a�l�!jZ�)�-=5!�;^;3����d���1FO���Z�98��sJ_����98Z{��m�#F����2��p�5�}�(8�pKZ�KP	��(���+����C^yJn�@=�Y��e�b��9G��~�S�`a�H��Y��pFW�21/h�L��R;����4�%-��`�3��L}���\�<?9<��
�a�$Uj}V"*&�Yg�&�3�B�P�;��|��frC�,���
���@���Wt�{���TyS��?9x��(��Z<P�_8��Y����-W�Q��,���Rbc�s�}d{sL\!�I&K����:��|!����F��Xd�!
����R��Qml6/]!�$���s-�\��`4?j�F����;?i,jc h�I$�;�g��}7x~v���������/�����i���'p�*�ZC?�����oQCo��<{�����?��f9W��Byb�NH>���7�W�s	54��P��E
&��&�Zh��>2J5�Z�QS|Z"�F��(���+�9&b��AY�����*��������]S�*�����B��o���\��tc�1���{�Z-L��5��/N���n{<�8�����W�����&<��YfK$��;jl�1����&���#�1K�O��t��c����5��Md�3+�O�S	e�j������&�J���HC7�us\WZ"��
K����X��H@��Mi��`;h���������PY|����J��J��[$2���
Vcg�D<��.�%�C�BO�hj'�e#j�v,P�}���?#�p�gT���$z�V�@�/wDk1����b��!��xv��Vp�S�`����|<6�Yi3=��Gk���3w���,�����(Q;_�P��`j����l��`0�#��hD����+��2O���+o"�5'�6
��tm�%8JDW�F-�������Q�e� �.J.s`c�m�*�@xn�;�`9�C�	�r����[)�
��#O��S��w�h��RB���E6�����
ceqRF8�L�[L��������G�);����|)3~��"�sC��V?��x��p_���;8�����#��d��x1hP|6q(#�����0�����:c����b�����@�IH��@U�S��������>?���n[�l��_cL��cL�9=�y�
�Km6QB^��I�E&2~�frI�*����,���0eb�����r�iP� ��4�[������xl4��/V/<��8p�z�\��U�"k���	����I8�J�ctp�e6�fW�����������N.�K���\�3�����h�a�Y��0�,��H&�������/�oM�56B����N��qQ����eNG
���Q[�Is���n#T08�1����Cml���8�C0��d���wo.#���a�=��d�i4dk�X+�e�����U@qu5���'�(f	�7���{�Aa�~$�����u��������r��$x��*%������������r12�������������qG{F����o�Z.�y�$���=>�m�O��Tu���K���Z�Z=L�z9���,��Lp�^{���+��5xI�G@��mR�a��9:�V/�����"�����=�Ys�(B��!.���z-4��o��#��5b�f������/,���Gv�*�&�Y�l����a���~������D�7Y��1@65���G��X����.��p��Cc����s�N���6�D�oNY�b�R�q25���F����A�x�0��t[�&^��I�����j:�-Zw,�{����$�X�N��a,�m��G�Lj��N��#��1����Ao9^B�b�YV�`��Mj�2f�!XE�[���$��������"�q�Q�(}�9l�����E���R��2s����'h��uz�u�,�eE��E�����1�Y
�
�|?Tv�x�jh���G��L��!�����WP�0�r l��i�	y.*WS��T��D�NjoJt-?���QG7��t���3J�#L��3���lh_g�U�D�ZE�R��\��K��
w#:��"m���D�>UlC�o�0�b�C�!/~Z�~��W�,.�i���`�"��M@9��z��#���B��������n����������+YF�.N����i.�L��6�8 ����{�q�5P�=[��g-�����X�+,7u�{c��$��8���(�*�[�q'��"�jg�7�Da�`�z�"��s0�ePVL���U��vxB4�q��^�vH��5���|��%�G���M#��2�U��(� _Z<�]A�N�Sii��JV���<~U��I�S��������9��y�HlRs|�M��-�R�]�bgD�� ��(t|��q��|������]��r��p�zC���51�vf��T��^�(n����w�h�DH�3�t����!�AuBn*~�c��Rsg#W�/zR �����=aT����A16�����mgoG����N)^�e�`�8&:0
7h��[��s��rI:
V\*�4��1���5L.��z�u��F�5�����<�F��X���5�CXa����u���������fn����v'����Y��r�)<(b��<�����#RKF�Y���$L`
��Z-F��`��dV��3Q;l���� ������x��5��|���>=�h��z��F,QhAuW��
�[f�'�,�2Ff�F=�T���[tt����0���m��.�?Mf]O�l�z�I�,��rP"�0q�@?x�M���x���X�)&�`�/LEj����{�����>|'M���3�!6��9d� ����j]nd9����b��4�|?%�/�	!3��q�fiZ`-+�*���NL@>2��{�0W�,5��%uF�18iD�8 5���z����$l���NDa�PY�!�D������=��!��L=>v�Lh6J��be09CU���������<7W��P:
IO|~��*�t����s/�X^�@3����tI��8��O%�W��h�����j�G���-&��)�)��8e9W�yFI�E;oo2\�/��^�A9�m�-f�����~�
�^�z4��Vs_):UeLS$h)���H�����&�a���@t���m]f_d����/�>I8E]t�]���.���E��#
E&N�YR�"���nx�}:��U$�(vd�r)�,������0���W��C1�;��;7oJem��@��%�d�7����Ge�j��*�����
���u/������:����f�����Zl�/M|��nE38�����{uV��pGlc�x2�@_b��Y�bO��T6�hz��7^�]�j�T��7��a"=�����~�3#Ln�v��~����������<��dt���~�|
��b���H���&������b����E��929�
��C�����!9q�ps��M-(�����'|��dz�����`u2���z���=Bh�K8I���(.��?�K@�'�-o��+_�x5h�C.���`�y��0D�����JU*�)�L:D�fh�W����������|sl=��0������f�*e"'��*	+��+��A�C
iEv�s��HB+�O����f�*��t2U�Z(�j��Ah��#@X?����<�|�|�Xq����r��w�=�L�<�r���'u����'d�$�3��p��!�Y�'`�%�b=�7��+M����&V���K���O>��=�b����2��uuW���N�v2�Z��:�p4rZ���� �q,��Xga�k�/�n>T!�6&��C��%x��(+wYL_ct������M���	4�K����H52"�&���fE�w��fQ���S#�}��#|C*���� �&EB�'
6\=��{��q�!����\Z�c��'Z���?Y6�Hs�N^����!�yj�k�6���q9C	��7t�����\����7@)bj�������[[zH?�5JPo��)�s���:v��6��y�)�X�@���	g��������W8��TUm��w�������#�b�X�z�:�
��������J��tdt0��U��oPZPP��JY�.Gr��+4�`��	���/�v�q&�FJe�Y�J��s0$�f(�l��������%�B�8Pf�t2�=��9��0����W���������G��c��K�1���|_��BH�h���~��tq�����^~l��R"XJ���fY��zVk3NK�^�c���ipm���0.�c����(.I;%�N#���"q`�n"�\F]�N�
V���|��q��L�Ml6=3�82��X��na@�������%��+��E>��s�k���om����U�Y��������G���V�m,!��d�w�����p�`#���w�#���^��T�
81HU�i���$�O��[qA"�_SQ���w�{aD�,S�i,"L��,X��s�Q�����]�2<�#�V4��/�\do����*���[$}���O_z���k�d�:p�S���bP
����b6����]m�����n��T���j�SN��t�d��`=�����"��W��;�]\0��
�$*\�n4��4Z����o��������d(g���3��$�}��S�:�K�%�K���d��l����7���M�����S���������Y�f~Y����-��,C����v��x��0.�*�v�g,�"|�������xx.��	�,��
`V���D/F�;/(��+p�s��r�!�8��fP�-h��8��{��+l}�+3Vd�F����B��"�2�F��JR��C��	cl"d[?����p
~24H��3��<#(H��O��$��D���S��X���mH�9��`��  �k\�Zd-�I���T�4�M��-=6&�~��	���X���rr"����M����B-���DZd��!���M���Q�%K��)��$Ci����w���.�>4y�'
��uA��'Nq��C+'2M���%IE��s^n�T):$�=�7`=���3����pBg�f�(�q�����PIN�2A�dc�a�����Tq�Dh��JJ�������Y���z�3��b)*	��@E��M�Q�l��,��(bn�����2�
�,����]�.���%����[�Vvc��2�l�r#�j��NC�=����9�E/�����l���N�lx�:����i�������/Vu��wE]BqS��nD�S��c���e-�UT�BgAf&wQ��#>�\�JB$b3E�N��O�>�s����gP�/v�4�4�'�4��S<�������2#G��l�%�U�o�z�=���'f�>{,����~
�
�SB#��w!r���)v_E�eg����������G�'�&8�XY������y�8�������p�@���Dm���d�����j�l��8���?�P���Hp�����L�[���s�5������o
�y�>�2�2��:Hs,��IE�L(j���rb}
�Zg�$����q��d�?4&@.L"���,���3Ydv=:�!����
�S&KKf�V3:�tP��$��W����[�f��������Y4Q����d#�7p�����af����h��t��0W7Z���t��^�����i��l��D����E��p0�>9��'P}����$�
�/�E)!<�>�
���qKsyO���)&�j�Ms�q��K�/7$�;��Z'*��
��L1�^��HI���6��#D��6���eW�Y�~F�-)�Ik@k���3�t����k�����Ud�����c�`�$06��������s\����$�����O��So:���2��l�������
���:�%��m�Ct2H6�k�sr�'�J,P��,?�YQ,�5�9V�s��d0�O��J�H%g���v���Ai�����}A
m�A��x���g�������0�����N�xI��H#��A��@�?/&I9��'��W�v�(�q6Dc�c����b��1A2l\r������%Yv!��(3|�9�w1�^)E�1H
E�L�i,�P\�^s|�@LF�7W���7S+6��X6�y�nX�8���{����F�
#��BJA�����~qk�������������e����8�^�ku[N���qn�����{���'����U�'���-UW��}�T]��3�j"
����=/�Y��Y�t��`�|��,�s/(#&'����*����H��%��Y��?�/�A�S�;���b�������4���XJ4&������)�E�����M1i[�OIR;r��K�&�<�^B��L���}84Bb�kU%��<��In��
~��k�R��w�MKO�`���N&��Y���s��������r�P<6I�!���PdCf�@�e�xF��s�Oj�C;��o9�|Y�(*Z��VQ0;�4z��$��CB�o�&�H=���h�,�!.	q��7#4k0�x����7G������h��g��|��*��q��
Wf�Sw��8�TjT�j3	�V���OF�P�Y��3�n����N�1�2a�8���R_�0;�bt�
U����*$������^�H�"����������>��u����[�����.�.�z��"�/��(w���{�F�h���n*=�������4L�R.5�%N]��)�F�b(I��q��B`K-Z���k��	��vP��h%u�9����J*���v�G$�}�TNV���V.��
��j�O�8&H��3���Y��\2�4�L�Q��/���D�t�X��'�=��cD�����
�)�����gT2�t�<YJ��<7��y��#9���,�k��`���R�8d����Q.+��E��H{�U�S� ~
�}N<$�)�$��!�X��h�g�!��=���qM�SU8F��:���z��b�� �D�,a��>10T�P�D�����ce�	 }#
�r����0�(�����H6��[�<D��;�!���s|�Gg����n�N;jE������N��Fn���1:����cI����Uk�Q����������X�b��[9
��z����~X��z��������yU�*UX��	�.�[�ru��N�?��6}���^��f������j���F�c���n��?V���
��a(��lj����Y�e�����s<�F#����n�i����i���f��un���G����������son�]�^D�Y
X�]l���d��J�/K�U#���C��q`���|�{ TY�V������B��Uo=���[��
-���(�#����/��~�GKc#�q=c���hil���X�QO�*��E�����������#���C�r6iLB�Xx����#/qV5e=��J~�O��_8	�@�QD�!���o	�!�D�*�`����������	�8v�xLj�`���Rc!��#Gut���KT%��(� �SL����6��<r�|r0��7�>�<����8�D�3E�`\��!��^�����Y�"��U��]��8��5�%=:?����6���R������+��E�w��k�������u0�;2��	�����V4�G��r9���Iy[��������7�>������yxyx������o�;�����������.g��G�tz�3|w����Oj��/lT���Vd�c����i0�G{6�������
k���,TmD��~��3��j�s4��P5�'�Bg���j@y8��0��s��O���}�B��Tz�X�K;lV������[vl*Y
�J�-V0����mi�$*(P�+X+Q\����(�L����%����]gr`����� �fO,nPz.��+V�/�(>��w��9kpi�A!���~�5���Zml7����l�mu���Q������x�c7+l�~.��`��'���EY�������M��)�����j�vI�,���u���������_�H���hu-����������������^��:�8w�~8@�T���!Q@Wp��b�����i��)82k����AD#�D������=���z���}kw�����~E��:6H��G����uc[�$���f�h�^���������zu5 ����+��t����]���}���
�x�V�?�!��J�q/�f���I�^L������G�����Nlkt�)1�Lk��O6��}�|���:�I�{J�O�0�G��!�b�{�a����r�gG��O������K��u����nc��Zj�O7�[n����h�E�SA��*����2�@���(��,���,#m����/L���HD s������HS��i���Z1�u�U��@3�#$��,����������g��qR�l��?=���
���o�_-%Fi�l�p�������Xi����6�{(�����W���VA��/�������$3t�~����T}{}vyu��rX��:Z��T�-O=�������;�a�B�K���	�(�_�?tQ��������:	D�Q�6(����������Sv
�@z���$��m:��\M�W���@|�rS��=
�����m�
�55��x$B��6�X���:�P������va��@*��Nf��,���-e��*�ll�H����0������PN�g�-�x��{��28��'A�Nqts��2���	��2t
4��v��?}�������[%^�%��z����Xu��TDg�'���.�c
����u"�q������BT�[:I6OG`0��_G�1�<7���{����6�vD/��#�������0A��K���#-�.�^u�#�&������&�e��4qqvLH���J�r��;^S��d���Cq����ioi��0$~l�'����s$���tp\��E��o� 3,�s�Bn��,�R92^�MJ5���l���T�i���'?
���\l����%��h`W)�7��w�v�7���Upni�!�V���2����?|��������� =��I������%�2�:����U�`�\����@���+������]�����[|;��yr/�aX�����xl,l��������2�K0OS k:*E}	#����4`�^����hg��a���~��8~����Z�
`�G�9DK�38������,r8PJ2����M0_�C�QDr^DL��\~�"�����/�A��;StJ�G�r��y�Yq}���a��XB�]i�N��n�2K���3Ls�x��,��)�K���}9y������)������:YZ����St��w
�2�&���V��������}���SD� �&C_#-].a�t�7�\���b.@����F���J����V�d���vM��w���
��d8�.�/.�/���z>a��M��<���6����Bz])��J�<�L>/���d2�F"�BY��I�2�V3o����+�
��z��P+kA�i&
tm�ZZA� 0T���=�zR�����M1oT�T�{�A���>j`����*
���C���u�*�B�s�6�|e��{	+��|�S�+�j��*�cb#�E�h�E�6W
Y$���MY$�A��l�sPu�+�8Cf��+p�2G.>�����d.52:mh7
��\��&Y�'Gk{��l����7��~PK���fD����7f�e
w�����V/��5���a���k���V��X��c��R���h�9�EK��O��b��
�'�a4+�<��lD_�B"^��;������O���7K��0��u��Q���l[�7�L�H�t����T�9�_��p3zC��([�%�fF��'x���BU�m��Z~����&���A��A��u+�N�Z�j� ��
����D��
�Rs�8>W�V�cb4�TjT+�����@�9�����}i�G�Kl����]��!��Y��xLr��>�����U��#nm7v����v�Q�2E+{3=��dp#��?6"9���3I�]m�J��:�'�W��u2����;m��Oa��r�%��G�}�j�i@8��l��J
H���D'�<��vLx���fc�`;���������w��F��� ��@[$>D{v����Q�2�������v������y���zO���_�C�r�Q{F���y���{�W@���lOG������	/f������v���v���o����v�����~o������ �&8&!u��I�o��V��'�����+u�(��
��N����i
cu���_s)��X�G��Fei��!�{�(�m�c�������x�Wgn��e��V�����`�*�{����'�z|ru��v������7�W+�
tO�K	�W�W��
g	�1>��wW���e0t`��U0�+�������_C���G�*F^,����.���}X�V��Kn����RY��������>\�%�Y����b</a�����5�L31���^bR���/�:S���,��KD����d��z�����i]�8�R�n��c��yW�E^D��A��|��fPA9�K����#?���6��T%�����V����e#
.`�&�S���m�����t�:>E�R�i���t�H8��6��v�\�L_b��t�0!?SM9&M]!&<
����R�&���!��:}���BKB2�������_u���V�y;*�[ 8o�����y��4L����& �	���+�K�H�\h/��YV�z����mkg������]�p��_��g���>������}O'���P���d`i����@1� ��SG��O��I7����r�)��BG`d��
b�`����41
G�F����H$�_5��T($�\�G�����^�/(s����;��>���"����{�MQRgt��_D	V�����d:8�*6��'�"#F�b7��n�!��
u��o�Z8���js�!3�Lf�\_&����B���M��u��X�$���o�����<p���,�&|�gI5���	��e}�9c��X<�����;�	����,��f��p��&%�q�'se�P��an��l	�}��]�����|1�@a���x'���]���9&��=����,�:'K����0��������o�g3+"���xz[^�3��;O]�9U%����8T����LF�I/�/3�8xtr���y����"P���	S�h�|s��"K����H�C�2m�*:#E&���`2\�w,W4O�/�sJ���^�5������iZn�C=?��M��*�K��@ti�����R�(�?�e�Q~�&�U3�s{�wA[�-��m,x���x�=�X1���#���3]�G`��N������z�*)�����5��mX�v�����ns������^c�H��~��--�r����9D},�G�@��dF���(�@���E/��10�`�j�-\�?�~�/�����4[H�C�De ���&��*��K2�{�V���aE=o
�J]T���%���4[��H�y2�%D�1,�3��jfO�(J.u� ���G�k�9^�M������!|���dI_5���RuO8b�=~��H�'h�2��!��WC^��W/�~2���T�}+�#`Yu>"��� ��F#>��a?�&q+������d'��k����
���yAC<������i�Y�
1W�(���/�h'������;�%�}T��F�5�=�xz����z��9�����`kwO���@����� 7{�O���1�%�-
}.x3���9/B��HQ����0��i�NN��ss���I���p���"J�,Ms��w8�b���ZM��8�l,iT�MPp���,&_(S)�fSR���@]c��d��b�QV8����m"W�����ja���l1D\���c��(���0�Rw�%1�Kq���%d|�����z�S�;�^�K���bS@K[��1~������]����]�6�0d8f��(�u8k!�_��`:�#]>���
7���������aB��Dv�6��c��ZRJH���CVY�oV�����!��Ey��;=Y�0��I��3�UMn2+/��|�3Dl�@���h��y.1^�`�H�cs�cw���yt�s+�2��aCJ���9�5�m.��>&��0K~�!�Y@U��gp����GVZ`v�H� �����u�Xw������f(K@�*#�E`
ey�J-5�bb�,��b��y�B+1u�A�t�Rd��$/�?
��n��q�k07��Imi�=����2Y�\��t��8>@�s_�.|v�*S�I�B��V��
����
Vl~nB��3���[�^�������M�����f|n�~��A�����Vr��)��y�c{�H2j�m|J!���N[�u�Z�)@�6�N���i�OV�7���U�~5�#������y��k)�t�����U+��FkS�=YoK��l��������,�T��pb9����r�2Y�i���1V��l6��;C����1D��H�X���x1�8_j���tP'���thU3�A��e��a���|y����YD��-"�\g�|��-(�������m��k �~�~�6-��]�b��4��.�Z�d�G~����DEjR	�G��'J������V�>B9(�W�T���~�p�;��;�J2��u'mJ9Jb��j[�p��=0������#T�o�3Z��!�.f`���Y���H:j[�5Ls���|p��	x��y����nl	:�6��V��Z}��I>C��tB�5���R��A9Z��0y�Ah�[[�y��,4���6�iyX�4�;��d
�y�,���;�P��������E����N^�=~}�Q����������������������7��p�E��|:��3Z�g��h\g��D��]���y�.ge�'z`��4/c@�$Z��!�U|���F{;Fuj�7�V�}�EgR�&�?����j���l�e��!{3bFL*�����al���W-
T4���[_�PE�hM=��y~��B3�F
]����k�e
tHJ��m p�����db��&�pD����Q+�4���l��)���d~��l���6w���7��H�WtP��O���x�R��6x1T�����xr�dRB�~K��)VP�U��5I4I[��S�d��)>��_#�F���7~E\�Lr�F3>�A~)%3�:q��V*���\S�����v�����������}rwq����C�(�.?�f�p ���g��N�O�Z��r,�[�s���6$�#����v�$�NO��9��������>�S��O%��NP�'a\��4��t80*��?�E��4�0���02h��=C��T��������[�F^ �����`��P�)
��4����\������\tX�h:�����WN�[�]�z��v��/����7q@���~���������9��M���aB��T@V���������o�Wq�����)�r��v"�CQ?U�!��}������0����N��4/�,+Isi��
�����7�j�r�Dn�)���r���q�#���+��5[�-0*5w8����*��*�%��,)����m�.E
Y�B9����8E��qz��Mt*�HR�O���z�A���]6��x�.�����m�G����
�V/�CVd{����%�����v`Mbu|���T�E#������PO�����[��+�zs��r��F��J����Bf>�0�5<��������#r�<��9{��o���Y���b
,a�l*�r��)�Zh��".j�
�Z8��ZI���k7*.	�B��M~<]����,V$�=���Sg����Y�n�v��5"��$Y�V}����j�7�b�7���t�7��;��.D�x_�E9���m�'��g�~gss����K�F�C� g�,��.�}DC�����a��Y2�0�^R��N���kv�^�n�VA��pJ���r��C��d��b#��bk����0�����4C
#Vk���j��uK��'zl�~�7g���C����.���v�:��C����g1��F�����/���"t�1���S5>S�QXkT�%�(��%����x�0�eE>O�����5�3�lx$��%����;��I-����DDz���������1�&��:�����(S�P�Z�gS�;�g��h����37��{?��gf�9���PxdI�1��{p��*l���u	��3
Ivj	���H���%�����!���.���m��a�b�N[�1��
���t.���C�9����2&t]w�
�C.�&}���cE��e�`d����P�<�x���Omi�X�,�-�I{�I�I�4E�����'�uM�P+���)��Ww�X=.{��io���^��
�jMf8]��z���'*Y���2e����*����\�����;�F! �<
�%�]�M!`m��[�V�_����w�M)@_]�2A�K���� h��U��W�q�����b�U~jz����y��,����f8�e������*0�����|���`��*��G���WnTG�������H�����6Ky�n�l����1��e/��CS�OB��z�EX%�uH�%_��bu5���n��i����W��#�d����:���1pR�g�g��u>�~������t�>�7QEOs��@]����������1���.y��rt�����(��-|�U��k�	�=]5g��
�n������~A���`�}�Q�z��o��4�}~��w���������3I
(\��p���T0���hG�U���*��,�3��R��r��D�S�c�������d�k�I�D��U%��������u��"��H	�=��7��q�1��jM��Ds���5��@k�������s}�Aj�OG�D���v�j�+�I2����l���_�U����/�x�uEz/3d*Y
���"g��7�~)ea���O�H�u���2���%o���I`�����Pm���L2I�I��r�[\�M5���
fIvz�y��r�����JAF��Uu�B�^��JO#*��������S���jM��=��@�^�'��lTq}f}J�#�\<@Y���Jv7���&cE*��.�����t+9����b���!�����I��u���������T4[G�-M�5��hc@��>H�ux8k�u�����S��\���m��Hz���K���xx|x�H:�:3���<`�S��H��?L�J�B\�����:DZqR��vu2��V�;	~oG^0��'��
h_6l���x����V�;�v���V����@�lL���X�9�ZO�ot�5�!�}c���dx!�nR[�Zq�)�����+�8-6k4�)/�x�3&�����������X�\�">|�df;����671a���4�5�QW�����zf�����~�~��BE1�2'���E���8��%�G�������P�k��#���������4
���
���"����R�U+�A�@Al;p/s"t%���������hQj��Xp/T�,HXt�6�@��5����}��X31 :�J-���������c9u}:j{�E5\�0�n$
�����y���h��4}%��4����Q�C9Y�c��*���R�L�-��?��\]�L�/K����T+���)Z,�v��
��_i�'�W'o��iS"Q��������u����zsq��lCm�?5��\A�������w-��
f�J�h���1�&��a��7��2V��^�7KE*���6���B��*�����u����������R��}�w\D<Y~�P��!�A�d13�M����*�c�\��� �:Q��vW{��\J�q�Ds	!y�s����@g
����L��/����H�Xv5��6r��s�����9�����9��������3������r7��F,��������<gG8
��=<��	d�H8:�)
�rQ�3�gX��Ww�^��r}�NJ(;wf;��3����$�/KfU ,�#���5�z�loJ��4�����\����18d�� �0����8
#�A��j��@'�3�/o����V0��V(��yJi�W�a�@6@N����~����oC9��P�\`�1�a>��a�`08��IS�=�'_��P\�+�4�j#�jVwv!A�:����=UV��A��(j�{�4[�wN���3�J��<�I�r����j���~\��m��Lu�P�a2E
�z6YLM��f����d��y:C	���a��KQ�0�+p�o`����)�*O�k�)~t�d�`H��P`���"T�~��a��[XJ�������4�l}c�u�����8B"{,y�V"'�S�y������4B�k�3�k!<������se9/�}D�zFq*F��Cs�C���/��Il�R
�\�����,3�y$.g��<;L[�����V�R�Y�$-H�]��WYs�6�S���"�G�ohe�-,I��]U��k�4�����m��R�K<cV�0Da56�Y!������A9��TSOz4����E?�����p��ot�I��k��5����a>�d!����4�4���A�KyRy	�f&aJ�m��na	:�F(T�U2ce�����<�d>��b��]��e�hF�N����53�:�b���m�,Z���~�d��'��wQ��(���G�|W�������i��c[\�Bl�SSz�?fl��7��������^��v���?8lc[P�rM��L�*��(!��`��^R�L�{��0E�d�y{~���������U�������	�%_�&��7�?����7wU�[�s�c�����������}�9m�:{�>���*���Q�V�������V�����f�z���9���u��:(���&d{�z�X20X�M"'
� _5MPI]�zj���J�E�Hgn�>������4V��p� rL�>���qr�D	X�<�����]$����Z���Py�����#Qv�:��E������X~.���^��,�Y�$,������kTw�J�mW�
�2A�x�8/{T�{�3�u��[g2�LN��t����E�C��u��a��"_�l
1�~�%�0<?89*���:��zs��������?�����X�T��[��X�$���^�B�/IMc�z�KN�����WRm�u|��/�>H�#����.������F���v#�Z;�����^��n�v��#j�3`ncj(��x6Y�N5�<'�H���?��
������A����A���l���v��w�;�Vk�����f'z7!b5��F�G��R{�`~T���Y��2��9~����>S�������+�����W��B���nt��FK������q������q���)?F�v�h����"�F����!6rH�`���'r8��|6�/z���=�G������4vl'�k��/E���S������c ��Fv>���:61�D�IF�Z-Z��pk��j����3wu7.�/ir�N�����1x�D�f�b��8M������|����9���'������v2�h�e����4�>�����y���r/y��/���y�z�������?���������zX����3����uJ��������r/�����v����w��V`�z������l)�Y�%s�r>����A����W#E��N���[����^�v��V�(.�^���6R�����${���z]������a���kC�Wk�C�g�

��Ql�����a����oRs�����0�
�!���4Ik?���5��U
E	�Ti�p��z��V+E�����_����e[[�^H^K>�#tu�m
�`��P�>���<�h)��M:��}���1��^w�A9+w���%>h&��n4��a��98����B��Hc���3��Cl��"�P#'��?]4Y,�%y����������~������$������NyHKpB������n��������d�Y����/���S�Q��K���Z���Q���2�r��fT�VC&b�]�[�M���W���� �����(�<�%�!�V�g�h.eS*�,iv�h���B��Q�C}0=T�M�{.\6�RGP��/(���t�����Q2��2��I�A��u�Y�F%ya�(p�+��zaML�3�����ZV
��F�`�@h=��`-\���|��S���W8B���x�f�G
�L�e+���	�#,��e���
��P;������=�m���Hc������
���e��-Lt�?�o�P�k��'�v;KaN���j��	L����+D����b�����0�"q�NK6D�w�K���F�C7|����[�O�q�S������*���!���OK��
5���QL!��Bf�Q�O0�j�P
�b\AI��q$5��B�Vg2fZ�A�(denL,[\���'>,�:a`3��O'( $�*r��:�G�.�z����aBL�g�n����o/��SU����ma�)�wQ�X�rSu�iL�n�y�5���M�D�,I���u�j�V��1���kOS�W��gXT5&�������({�%����'��.��rO�H��@A�
>^
����C���M}� e�C4Hf�i�fF��o�

q�)T!��>r,��:'5H�d�4�|��d��T1�yf�"8����!|�h������/�4q��]FQ��������^��IXC�t�Q�.1�t����J�q�%��,}zjj����|R�n���,�";�Q*����R0�&o/'�f�S���S�h�����E��(	�G��)�� e��D�rJ�7��B�R~����u��Ar����W�oK�z�@��L��x��:=�:.����Yh���X"c�Po�!&	$��o����o�{�M2�2���"���y:�����-����U�]���#�q��T�1h�BW{�q����Xv�ab_�~(�I��N�s�i`xb11��3�?8�a���\�!�~���O��]	b�
���B�*��s#n]�(��B�4���99�;2�(|$���)�+V;���k���0��o�}����nx����n2L�WJ�::��i@ VN�I���xM�!�9A2?�����OOMv��D�EC+�<��\�]�xXdRc��L�t�N�����'�D��a�z2A|P4�6?��}���\E�
�o�u*���^�1����+���Nl>���pW�g����vn�3k	�ncsv�0�j:'Vi�gf�=���)�'px��.���)n�`��<r��A����3�����N/���,��R�F�~G����5��$����k7����.u�Y��&f���h�	�,��&�-�q�f��7j�+��H�E����������U2UQ�mL����8~���m� �� �Q��|&3��_-�p2T��=^��z���z�ju���6�=������d�_
�)��#S���Jv�Q�N�B�0�%��L��x#�d���G�]�N�\w�)^�xS�Y_�]:2���1������~k�-�;���QX����+f` h$�
���S�P��d�Bp��3N�<��0�z;����;��^�5�~���m����p��,Q�����)Q#�@�]1`���ci��&z�I�1�`���|������IrNL��)J#�Yfg��o�&��1{�7��=wop��G���=�����+trfj��Pvf]����J�l���0W��������Uc�Ut����'���Qn"u��fa��A�%	���6G�����}F \$���y�eP/+@V��	�qF+����t.2����'��h>�J�������p� �3����-��|"���.���_'M�\���1b����wZ&�U��{,L�n2>�k��lD%gI��.�f����t�0�3k���t�H��S��	��=zi�� ���_%�`%��C`�� P������S:@�'a�E�J����Fm�T�l=iQrP����HhE��p���_�Y.��y9�>
s�I�5��rrK��T�:�������BvH�V/S�V��]<�W>�~��$��y`>'wQ�3)S(��%��������6���7"Dd�	� &,�*
��HyB�\*tj�������9{�O�3`�Y����u�#�P���H� ���=�z�cc�8B����Z����h�<��Fzs
P���K<R�g9Dii��0���?�@E)K�y�U]��y����X�;�e���cz�����r���]iig��:%�����J5��uC���C�(
=9!%�;��r���1l�^��1w}�~�
�����$�R���������Z��G\�������O��[nf�E��h�y0��Xq�,|��D��aT����V�����g���UG�=�q,�r�9�����O��?�O8x�\o��;�AcA@�0<���Z�}��@���`@���=���h�4N$/�Yr&�=LH{�:E�Ez=��b���.E��*�ww(��'���lww9��i��u�F���Uu���,*���e,�xZ����0?y�\���G�z��p_�$[�B
��jO���([��p��sU��,��F�X�b�"�"��q4���|��Ni�1���1��~s���q�}���$]Y��x����u@����� QjO�Sa��3BT�
$[��c)�t����C[.�IKP4S�J�Zm����%Y<J�y�b��O0BR2`Pm�&���=�w�"��t��T�u�UU����47�,=X)Ko�|0��%����db�,�7{�����������-A�Fg��?w9zB����e
�Ya���yHO�jC�R8�x����@���j�9�s�O�
��
B�
�h��L��������N�P�X-��@eI2�4�s�����2I�5���.���������/,�L�t���7�����������e�� ���wv�_���P���%����T����(D�sd��7
 ���6�3R���HX�Mv`^�u���Y"E�Qs��E;����
~�>�P��p���:���n.Y^���5i�`�#.$X��/5�����:#/%��1��&����kO+!�%�\JY�D�0+s����]'������C��r��>(T&p��>S�
�����mN�^�D}����`��s���uk���OC��cpN�!���v�+�`�zf�C�TD���z��3�j� ������^}�c��FMB�v��H 	5s0I�t�T*
h�?{��h�.�0*hwX�P9v���2�K�%U���A�	�	�,li������m���=?�l_A�����W�����/�Bw���p�������f�Y�F�F�\/�Qe��koa�b�(@���k�s��J!�pd��4�Y�[�y`��d
~�b0����Mai?5��v�]@��K��#��#]����lE[��L�{�[j�A�k.N���4��?��^x�8��	*����i��R�0U�Iy)u����s`���%����WJE��g�e/�3~-x~�w�p��o�tfd�|�jT[;���vu#��E��j)�m�'�f���LY�	V�U��{��*�R��������]�11���x��9D�C�G&m��99���r��4�r&a�G�X�[�q���V[Km�Di�����[�
���w�a��3r!�����x����.*������:�d:�)�-���G�#b���8�Ey/�]�F���S���fU�7Tou��|��#p������4�6p����I#S>��W'����'�!<���C!�����T2����+1��e ?
����9���+�5S��q��D�Hz��g*���[E��d���l���Zz-�;��r�5��!���|7�-W�&&3�����'������`pPa�\1�{*q'@�b;L�9R�N�aP
6o6��o��+-���RDK�R?��@1�l��@VOK|%b2�
���n�eG�4L"���$�����:�,�&$t�[ZR�[���f�#��m�1\�������M�n����7��� ���_������S�1�Y���g#�0������:F��	i$P�t�B�*������0�d&���HH��V9{.��^��.0�
jk������jt\�i,{}_�@�^`Ml���XiH_�q����%.x�g�r�%w.:�%VJ�J��8t�
�������@t|�	Jj&$.)�!L+���I�f���1R��p��e��r�������OLA�x�>�)��N����~]O<-����O�����}��&�pa2
��R������w4E�{��fz�.��	������e�KmjvW!����&�� �����<��n^��/A�1.z��o����
~��=py����'d�����9�!D�C;��8��B�f%����,t.���0=�9��d�[�uh�o���(���m2�V2�pA��y2��G����������C�����)�������YZ��Bx���J����D��sV-��]5�5���o��.��M�<�5L
�u|��aYN�s{�4
#+(j��X�����I�<��>���b)�l��M��Y� �S���{��"a�Ct����J��{z��0��?��ag�`��|�dKHQM���i�m����C����&����c��7�;*uY�Q ��n��j�FQ\L������$i���Y/�����M_0A���{�~{20o<�&T�o8
�E�_�9w��q��?Kf�r��!L��}R�'�Yc��R,p�!�_(o�r���'�8��(N34H��Y8�������pr�����)�F�)�h:���P�^�u�W(�C�!8E�3�]�]��,�L��R�EH�{��A��v5)Z�mB�p�&1L�FW+Z�����������B��a-�H+��JkQW��k��pr�G/3<�I��&]�>����C�@�<��pX�1E�D�imi>?��/��GX|��Y��b`p�K��	���'lU�tl�w���"KX�/<V���������Ro>d�B	��)?U�T��Q����zt=�g��$����������`�*0x�1TK[��Pd�	G���&.P��8��]��h���5�A&u.a	��a����~��_�=y)_ V���+GdPK��#Q	�0��8~qa�*���%��n�l�2����9ET���*��JA��JT���_oW�':k-�w=�sM�����e��%oh��tl,X��5�86���8.]#�*��G��v���](�2�c��N�d��5���T�t���)���X]���jT&���W�X������
h�W�8I������X���^���B�pK���yyFtg��6	o��P���e�p���Qd�b�!�����$��Xq��E��0��"@^��9%/@k�8��u�������{�_m�F[�����f1�On�r����M��f������~�S�G,	[}���N��V����:��q��\�-����u��VywE�������{��js;�j6�i��t�����N���
l����[������C�#����xr'W�i$@]�����N�&�8�kZ�3Z�H��}���L�n�	(Y��P�	H@�IO��v�<BG��U���G���F�	)�[j7��F��&3���3�%O#�t�Q+H\������q�.��W+H�F�����R���o!<�'�`��5q�f�5��a9GJ��zgF��T����v��C��)Qs>���)���&�u����/;�_��7����7��{R��%�S�����e�i��~�P-�n��KL�y�y
�g
���D���>IY���Ml����e�aQ�-1��d�!y���m��FS��x����8�x����=�f^=��w�����x8I����E����UAg$v�����k��G_9I1���YD��
JO�
�?��["+�V�+:Q���w�w�|:�����?T918���!���b��(��N��SpR�����1��<XQ�Q�?���$���1D�/���.	�B�����t.R�5�����n�FQ����|�E��w�)�&#P#8o����+*��GSr<���HK.h����4��P[c�Z����

84
�����xzZ!�ez`�X�{C���g�<]'������8�L+�:�P#+E%Z������H�h��BY"<B(�v�t���"�29�!��(<P�XT��s-����ON����y�.m}�&E�+�'H�57�P�]�.�3��-��+�H�W��A��u
�k������w��Y�o�?(kA�[����km��{��Y��k
�Y^E�E��Ct��rJ���d)`��N|-+m�k�q	_�����SZ}1�������E�3��W����
�>��o�:�VQ\��o����w�"2�DZNk��Q��^���b����I�;3)i=M��Sj����j�j;�[�����'�Kg�)Z�>8�_��5�Y����4G`D�Z��ht��������z��!?���W����p�
ep]�`����\��gv;������StC�=�R��c��<K��3c����Q��,0
�$�c�Z� ���������`���<�ag
�Nt"����4\����\P�
��}:��������?dK��I�p��J�����vh������� �S8�hSA���b��OR&��V1���z<�D_�b�A'��<dh),g�\���9��<�	eK���!(������J�����b�]������=�������r�������Y�w�F��%����y@�G��L�����H����p��~g��ee�>��[Q���X��6�f3 D���G�\,�u���B�tp�\]|l�����Q?���Cx�4��-����&roQ����U#���{�B���q'���+Z�+����bq
���EN|�W2�B��B����]82�
SuEW[���_rx��o�2���4d,�w0.����������sa����s������<z�m�u����{�a���{����c����!�C#T5;���:;`m�3b����w��U���]Z_��<����~�^\�2�z.ERy���?]\��Q�<R�9��W�R��Y-������{�[����
�=�����8h��_���rt�����4 M������Uss6f���L}@���P�����C���-%m����+���d��z�42k����	�����"0�i��o
)w4>��P��C�u� �H��t�a��&��1��7�6/��W�{�KH����L:��l�l�y�7U�P��#s�������������7w�%��mq�������i��������T�vU� �"3��,onJ�Sw��������P����\�������&e@�����7��gDl�t�<��_M����1��_C"�U����(v�`CPr������z5��p�\V���a�e�s�D�sy�����f�5��8��->��IMM�\���<��lV_����f'J\�a d��.����P5����*���&gh�l>_�LS��T����KfM��E���R6�����Q��U;��8|�����?�5��
��C/3��~����������O��;S~X]��,�u��I:_8Y8o���q����zx+G�o�{C3�W�ut�3`��U���Ff.g|R�[��-�j��\�����|��t��Pe� ���y�[����\�ly� �����-(��I��@>�tiv.U�Q����zy`�0Xt�
�]��U,b�����\Q\�hC����f
i���:���Z�JT��E��>P,_/V���P�\��v�d�S��S{N��W��B��<��|�����qbn�AX�[q_��h���0��z���p7�o"�>Z2%�N�4[5RXE���Y�f90<2�j�R�<9r���?\2�0��1��L�n�:-�[�pc�,~J��]���h���d�x>�,��4�7'�����������_���3upqH�d���y��Oy\,%������#���*,qo��\�G��C�=����II,3��B���L����`��������B8�U��D����q��;*������,P���Dv��C@yb��A��-�������U��N����]��|�%�f���][3���r��q�*��&�����z��hX��P���"fRW���!/'������Q{`YO�^b�](<I����o�6�{�l���h�/�/�2�O�,����y��9��8�k���rc���9����y��r�$�+6�8Eh�D��xb���/�u����n_]^]|<�r����c5�"������c���.*�L,��xv�{�?��L�sz�����-�����
�����hF6����	���M��*��;r�!��j��H���ml�~�]z�y0�FPa����a.�f�����cg�#Tf����#��T�*��� ��Q}��,U$mx��.�����Zc��V-����fx&u���yH�SqJY�4��,�/�X$�Bax��td��9]�9B�l��3��i`m����Fr�>G	�	"cyJ
���'��'��>�_�]���0L���=2gaG����cu!��(���j�_���<�P��s����_�RYz������ �&`X�
*Ss�\m�n�
���7�N�%0�
|���
:4a�
����,��
��JE����K������X���	f� �
�������90bR�.tk.����q3�;���J��|� �}�Mn�1nz������O�Z���5D����>e�Cf-�a|}?����Q� G��m��Z����j!
��5�~Y��������n���l��j���kP��S�u���i�B]����h5{� ��Ly1���MMX��;�N�������=ta�t�w���:7�t�B@5|���94��8HI�"�)�g�UJ��VP+\�����j51]������rz��E�$7�Q���_MMrX-c�B'����5QMg���/�A]�*���r�:�����+��Jy��ia�����#c��2?��+�B����zO��0R>�'��e��U�v\��*~�0���I���@�50V_(Q.&��]s���z}��_�=�t��k8������7��J��r,��"��W��n\�k���!���
r<��@q��x����-���sBN�R�D�(�v��"�o%��D	������'/XKA������:�mF_7��`�j�P��y6�='�0y.������~u�?�H����(�o�������a�>���I��m%�
togg�V���ickkkeoX�x�zm��1�"\�d�q��������=Q���{@��#�\}$��0�`,�q������^Y����3�q����tS1����q��o��/O�yo`�E�w�Q�E�zs�������������y�<~?���������y�<~?���������y�<~?���������.
#25Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Thomas Munro (#24)
Re: POC: Sharing record typmods between backends

On Tue, Aug 15, 2017 at 5:44 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Mon, Aug 14, 2017 at 12:32 PM, Andres Freund <andres@anarazel.de> wrote:

But architecturally I'm still not sure I quite like the a bit ad-hoc
manner session state is defined here. I think we much more should go
towards a PGPROC like PGSESSION array, that PGPROCs reference. That'd
also be preallocated in "normal" shmem. From there things like the
handle for a dht typmod table could be referenced. I think we should
slowly go towards a world where session state isn't in a lot of file
local static variables. I don't know if this is the right moment to
start doing so, but I think it's quite soon.

No argument from me about that general idea. All our global state is
an obstacle for testability, multi-threading, new CPU scheduling
architectures etc. I had been trying to avoid getting too adventurous
here, but here goes nothing... In this version there is an honest
Session struct. There is still a single global variable --
CurrentSession -- which would I guess could be a candidate to become a
thread-local variable from the future (or alternatively an argument to
every function that needs session access). Is this better? Haven't
tested this much yet but seems like better code layout to me.

0006-Introduce-a-shared-memory-record-typmod-registry.patch

+/*
+ * A struct encapsulating some elements of a user's session.  For now this
+ * manages state that applies to parallel query, but it principle it could
+ * include other things that are currently global variables.
+ */
+typedef struct Session
+{
+       dsm_segment        *segment;            /* The session-scoped
DSM segment. */
+       dsa_area           *area;                       /* The
session-scoped DSA area. */
+
+       /* State managed by typcache.c. */
+       SharedRecordTypmodRegistry *typmod_registry;
+       dshash_table   *record_table;   /* Typmods indexed by tuple
descriptor */
+       dshash_table   *typmod_table;   /* Tuple descriptors indexed
by typmod */
+} Session;

Upon reflection, these members should probably be called
shared_record_table etc. Presumably later refactoring would introduce
(for example) local_record_table, which would replace the following
variable in typcache.c:

static HTAB *RecordCacheHash = NULL;

... and likewise for NextRecordTypmod and RecordCacheArray which
together embody this session's local typmod registry and ability to
make more.

The idea here is eventually to move all state that is tried to a
session into this structure, though I'm not proposing to do any more
of that than is necessary as part of *this* patchset. For now I'm
just looking for a decent place to put the minimal shared session
state, but in a way that allows us "slowly [to] go towards a world
where session state isn't in a lot of file local static variables" as
you put it.

There's a separate discussion to be had about whether things like
assign_record_type_typmod() should take a Session pointer or access
the global variable (and perhaps in future thread-local)
CurrentSession, but the path of least resistance for now is, I think,
as I have it.

On another topic, I probably need to study and test some failure paths better.

Thoughts?

--
Thomas Munro
http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#26Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#24)
Re: POC: Sharing record typmods between backends

On 2017-08-15 17:44:55 +1200, Thomas Munro wrote:

@@ -99,12 +72,9 @@ CreateTemplateTupleDesc(int natts, bool hasoid)

/*
* CreateTupleDesc
- *             This function allocates a new TupleDesc pointing to a given
+ *             This function allocates a new TupleDesc by copying a given
*             Form_pg_attribute array.
*
- * Note: if the TupleDesc is ever freed, the Form_pg_attribute array
- * will not be freed thereby.
- *

I'm leaning towards no, but you could argue that we should just change
that remark to be about constr?

I don't see why.

Because for that the freeing bit is still true, ie. it's still
separately allocated.

Review of 0003:

I'm not doing a too detailed review, given I think there's some changes
in the pipeline.

Yep. In the new patch set the hash table formerly known as DHT is now
in patch 0004 and I made the following changes based on your feedback:

1. Renamed it to "dshash". The files are named dshash.{c,h}, and the
prefix on identifiers is dshash_. You suggested dsmhash, but the "m"
didn't seem to make much sense. I considered dsahash, but dshash
seemed better. Thoughts?

WFM. Just curious, why didn't m make sense? I was referring to dynamic
shared memory hash - seems right. Whether there's an intermediary dsa
layer or not...

2. Ripped out the incremental resizing and iterator support for now,
as discussed. I want to post patches to add those features when we
have a use case but I can see that it's no slam dunk so I want to keep
that stuff out of the dependency graph for parallel hash.

Cool.

3. Added support for hash and compare functions with an extra
argument for user data, a bit like qsort_arg_comparator. This is
necessary for functions that need to be able to dereference a
dsa_pointer stored in the entry, since they need the dsa_area. (I
would normally call such an argument 'user_data' or 'context' or
something but 'arg' seemed to be establish by qsort_arg.)

Good.

+/*
+ * The set of parameters needed to create or attach to a hash table.  The
+ * members tranche_id and tranche_name do not need to be initialized when
+ * attaching to an existing hash table.  The functions do need to be supplied
+ * even when attaching because we can't safely share function pointers between
+ * backends in general.
+ */
+typedef struct
+{
+       size_t key_size;                        /* Size of the key (initial bytes of entry) */
+       size_t entry_size;                      /* Total size of entry */
+       dht_compare_function compare_function;  /* Compare function */
+       dht_hash_function hash_function;        /* Hash function */
+       int tranche_id;                         /* The tranche ID to use for locks. */
+} dht_parameters;

Wonder if it'd make sense to say that key/entry sizes to be only
minimums? That means we could increase them to be the proper aligned
size?

I don't understand. You mean explicitly saying that there are
overheads? Doesn't that go without saying?

I was thinking that we could do the MAXALIGN style calculations once
instead of repeatedly, by including them in the key and entry sizes.

Ignoring aspects related to REC_HASH_KEYS and related discussion, since
we're already discussing that in another email.

This version includes new refactoring patches 0003, 0004 to get rid of
REC_HASH_KEYS by teaching the hash table how to use a TupleDesc as a
key directly. Then the shared version does approximately the same
thing, with a couple of extra hoops to jump thought. Thoughts?

Will look.

+static int32
+find_or_allocate_shared_record_typmod(TupleDesc tupdesc)
+{
+       /*
+        * While we still hold the atts_index entry locked, add this to
+        * typmod_index.  That's important because we don't want anyone to be able
+        * to find a typmod via the former that can't yet be looked up in the
+        * latter.
+        */
+       PG_TRY();
+       {
+               typmod_index_entry =
+                       dht_find_or_insert(CurrentSharedRecordTypmodRegistry.typmod_index,
+                                                          &typmod, &found);
+               if (found)
+                       elog(ERROR, "cannot create duplicate shared record typmod");
+       }
+       PG_CATCH();
+       {
+               /*
+                * If we failed to allocate or elog()ed, we have to be careful not to
+                * leak the shared memory.  Note that we might have created a new
+                * atts_index entry above, but we haven't put anything in it yet.
+                */
+               dsa_free(CurrentSharedRecordTypmodRegistry.area, shared_dp);
+               PG_RE_THROW();
+       }

Not entirely related, but I do wonder if we don't need abetter solution
to this. Something like dsa pointers that register appropriate memory
context callbacks to get deleted in case of errors?

Huh, scope guards. I have had some ideas about some kind of
destructor mechanism that might replace what we're doing with DSM
detach hooks in various places and also work in containers like hash
tables (ie entries could have destructors), but doing it with the
stack is another level...

Not sure what you mean with 'stack'?

shared-record-typmods-v5.patchset/0004-Refactor-typcache.c-s-record-typmod-hash-table.patch

+ * hashTupleDesc
+ *		Compute a hash value for a tuple descriptor.
+ *
+ * If two tuple descriptors would be considered equal by equalTupleDescs()
+ * then their hash value will be equal according to this function.
+ */
+uint32
+hashTupleDesc(TupleDesc desc)
+{
+	uint32		s = 0;
+	int			i;
+
+	for (i = 0; i < desc->natts; ++i)
+		s = hash_combine(s, hash_uint32(TupleDescAttr(desc, i)->atttypid));
+
+	return s;
+}

Hm, is it right not to include tdtypeid, tdtypmod, tdhasoid here?
equalTupleDescs() does compare them...

+/*
+ * Hash function for the hash table of RecordCacheEntry.
+ */
+static uint32
+record_type_typmod_hash(const void *data, size_t size)
+{
+	return hashTupleDesc(((RecordCacheEntry *) data)->tupdesc);
+}
+
+/*
+ * Match function for the hash table of RecordCacheEntry.
+ */
+static int
+record_type_typmod_compare(const void *a, const void *b, size_t size)
+{
+	return equalTupleDescs(((RecordCacheEntry *) a)->tupdesc,
+						   ((RecordCacheEntry *) b)->tupdesc) ? 0 : 1;
+}

I'd rather have local vars for the casted params, but it's not
important.

 		MemSet(&ctl, 0, sizeof(ctl));
-		ctl.keysize = REC_HASH_KEYS * sizeof(Oid);
+		ctl.keysize = 0;	/* unused */
 		ctl.entrysize = sizeof(RecordCacheEntry);

Hm, keysize 0? Is that right? Wouldn't it be more correct to have both
of the same size, given dynahash includes the key size in the entry, and
the pointer really is the key?

Otherwise looks pretty good.

shared-record-typmods-v5.patchset/0006-Introduce-a-shared-memory-record-typmod-registry.patch

Hm, name & comment don't quite describe this accurately anymore.

+/*
+ * A struct encapsulating some elements of a user's session.  For now this
+ * manages state that applies to parallel query, but it principle it could
+ * include other things that are currently global variables.
+ */
+typedef struct Session
+{
+	dsm_segment	   *segment;		/* The session-scoped DSM segment. */
+	dsa_area	   *area;			/* The session-scoped DSA area. */
+
+	/* State managed by typcache.c. */
+	SharedRecordTypmodRegistry *typmod_registry;
+	dshash_table   *record_table;	/* Typmods indexed by tuple descriptor */
+	dshash_table   *typmod_table;	/* Tuple descriptors indexed by typmod */
+} Session;

Interesting. I was apparently thinking slightly differently. I'd have
thought we'd have Session struct in statically allocated shared
memory. Which'd then have dsa_handle, dshash_table_handle, ... members.

+extern void EnsureCurrentSession(void);
+extern void EnsureCurrentSession(void);

duplicated.

+/*
+ * We want to create a DSA area to store shared state that has the same extent
+ * as a session.  So far, it's only used to hold the shared record type
+ * registry.  We don't want it to have to create any DSM segments just yet in
+ * common cases, so we'll give it enough space to hold a very small
+ * SharedRecordTypmodRegistry.
+ */
+#define SESSION_DSA_SIZE					0x30000

Same "extent"? Maybe lifetime?

+
+/*
+ * Make sure that there is a CurrentSession.
+ */
+void EnsureCurrentSession(void)
+{

linebreak.

+{
+	if (CurrentSession == NULL)
+	{
+		MemoryContext old_context = MemoryContextSwitchTo(TopMemoryContext);
+
+		CurrentSession = palloc0(sizeof(Session));
+		MemoryContextSwitchTo(old_context);
+	}
+}

Isn't MemoryContextAllocZero easier?

Greetings,

Andres Freund

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#27Robert Haas
robertmhaas@gmail.com
In reply to: Andres Freund (#26)
Re: POC: Sharing record typmods between backends

On Tue, Aug 15, 2017 at 6:06 PM, Andres Freund <andres@anarazel.de> wrote:

Interesting. I was apparently thinking slightly differently. I'd have
thought we'd have Session struct in statically allocated shared
memory. Which'd then have dsa_handle, dshash_table_handle, ... members.

Sounds an awful lot like what we're already doing with PGPROC.

I am not sure that inventing a Session thing that should have 500
things in it but actually has the 3 that are relevant to this patch is
really a step forward. In fact, it sounds like something that will
just create confusion down the road.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#28Andres Freund
andres@anarazel.de
In reply to: Robert Haas (#27)
Re: POC: Sharing record typmods between backends

On 2017-08-15 20:30:16 -0400, Robert Haas wrote:

On Tue, Aug 15, 2017 at 6:06 PM, Andres Freund <andres@anarazel.de> wrote:

Interesting. I was apparently thinking slightly differently. I'd have
thought we'd have Session struct in statically allocated shared
memory. Which'd then have dsa_handle, dshash_table_handle, ... members.

Sounds an awful lot like what we're already doing with PGPROC.

Except it'd be shared between leader and workers. So no, not really.

Greetings,

Andres Freund

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#29Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#26)
Re: POC: Sharing record typmods between backends

Will respond to the actionable code review points separately with a
new patch set, but first:

On Wed, Aug 16, 2017 at 10:06 AM, Andres Freund <andres@anarazel.de> wrote:

On 2017-08-15 17:44:55 +1200, Thomas Munro wrote:

@@ -99,12 +72,9 @@ CreateTemplateTupleDesc(int natts, bool hasoid)

/*
* CreateTupleDesc
- *             This function allocates a new TupleDesc pointing to a given
+ *             This function allocates a new TupleDesc by copying a given
*             Form_pg_attribute array.
*
- * Note: if the TupleDesc is ever freed, the Form_pg_attribute array
- * will not be freed thereby.
- *

I'm leaning towards no, but you could argue that we should just change
that remark to be about constr?

I don't see why.

Because for that the freeing bit is still true, ie. it's still
separately allocated.

It's true of struct tupleDesc in general but not true of objects
returned by this function in respect of the arguments to the function.
In master, that comment is a useful warning that the object will hold
onto but never free the attrs array you pass in. The same doesn't
apply to constr so I don't think we need to say anything.

Review of 0003:

I'm not doing a too detailed review, given I think there's some changes
in the pipeline.

Yep. In the new patch set the hash table formerly known as DHT is now
in patch 0004 and I made the following changes based on your feedback:

1. Renamed it to "dshash". The files are named dshash.{c,h}, and the
prefix on identifiers is dshash_. You suggested dsmhash, but the "m"
didn't seem to make much sense. I considered dsahash, but dshash
seemed better. Thoughts?

WFM. Just curious, why didn't m make sense? I was referring to dynamic
shared memory hash - seems right. Whether there's an intermediary dsa
layer or not...

I think of DSA as a defining characteristic that dshash exists to work
with (it's baked into dshash's API), but DSM as an implementation
detail which dshash doesn't directly depend on. Therefore I don't
like the "m".

I speculate that in future we might have build modes where DSA doesn't
use DSM anyway: it could use native pointers and maybe even a
different allocator in a build that either uses threads or
non-portable tricks to carve out a huge amount of virtual address
space so that it can map memory in at the same location in each
backend. In that universe DSA would still be providing the service of
grouping allocations together into a scope for "rip cord" cleanup
(possibly by forwarding to MemoryContext stuff) but otherwise compile
away to nearly nothing.

+static int32
+find_or_allocate_shared_record_typmod(TupleDesc tupdesc)
+{
+       /*
+        * While we still hold the atts_index entry locked, add this to
+        * typmod_index.  That's important because we don't want anyone to be able
+        * to find a typmod via the former that can't yet be looked up in the
+        * latter.
+        */
+       PG_TRY();
+       {
+               typmod_index_entry =
+                       dht_find_or_insert(CurrentSharedRecordTypmodRegistry.typmod_index,
+                                                          &typmod, &found);
+               if (found)
+                       elog(ERROR, "cannot create duplicate shared record typmod");
+       }
+       PG_CATCH();
+       {
+               /*
+                * If we failed to allocate or elog()ed, we have to be careful not to
+                * leak the shared memory.  Note that we might have created a new
+                * atts_index entry above, but we haven't put anything in it yet.
+                */
+               dsa_free(CurrentSharedRecordTypmodRegistry.area, shared_dp);
+               PG_RE_THROW();
+       }

Not entirely related, but I do wonder if we don't need abetter solution
to this. Something like dsa pointers that register appropriate memory
context callbacks to get deleted in case of errors?

Huh, scope guards. I have had some ideas about some kind of
destructor mechanism that might replace what we're doing with DSM
detach hooks in various places and also work in containers like hash
tables (ie entries could have destructors), but doing it with the
stack is another level...

Not sure what you mean with 'stack'?

I probably read too much into your words. I was imagining something
conceptually like the following, since the "appropriate memory
context" in the code above is actually a stack frame:

dsa_pointer p = ...;

ON_ERROR_SCOPE_EXIT(dsa_free, area, p); /* yeah, I know, no variadic macros */

elog(ERROR, "boo"); /* this causes p to be freed */

The point being that if the caller of this function catches the error
then ITS on-error-cleanup stack mustn't run, but this one's must.
Hence requirement for awareness of stack. I'm not sure it's actually
any easier to use this than the existing try/catch macros and I'm not
proposing it, but I thought perhaps you were.

+/*
+ * A struct encapsulating some elements of a user's session.  For now this
+ * manages state that applies to parallel query, but it principle it could
+ * include other things that are currently global variables.
+ */
+typedef struct Session
+{
+       dsm_segment        *segment;            /* The session-scoped DSM segment. */
+       dsa_area           *area;                       /* The session-scoped DSA area. */
+
+       /* State managed by typcache.c. */
+       SharedRecordTypmodRegistry *typmod_registry;
+       dshash_table   *record_table;   /* Typmods indexed by tuple descriptor */
+       dshash_table   *typmod_table;   /* Tuple descriptors indexed by typmod */
+} Session;

Interesting. I was apparently thinking slightly differently. I'd have
thought we'd have Session struct in statically allocated shared
memory. Which'd then have dsa_handle, dshash_table_handle, ... members.

A session needs (1) some backend-private state to hold the addresses
of stuff in this process's memory map and (2) some shared state worth
pointing to. My patch introduces both of those things, but doesn't
need to make the shared state part 'discoverable'. Workers get their
hands on it by receiving it explicitly from a leader.

I think that you're right about a cluster-wide PGSESSION array being
useful if we divorce session from processes completely and write some
kind of scheduler. But for the purposes of this patch set we don't
seem to need to any decisions about that. Leaders passing DSM handles
over to workers seems to be enough for now. Does this make sense?

--
Thomas Munro
http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#30Robert Haas
robertmhaas@gmail.com
In reply to: Andres Freund (#28)
Re: POC: Sharing record typmods between backends

On Tue, Aug 15, 2017 at 8:34 PM, Andres Freund <andres@anarazel.de> wrote:

On 2017-08-15 20:30:16 -0400, Robert Haas wrote:

On Tue, Aug 15, 2017 at 6:06 PM, Andres Freund <andres@anarazel.de> wrote:

Interesting. I was apparently thinking slightly differently. I'd have
thought we'd have Session struct in statically allocated shared
memory. Which'd then have dsa_handle, dshash_table_handle, ... members.

Sounds an awful lot like what we're already doing with PGPROC.

Except it'd be shared between leader and workers. So no, not really.

There's precedent for using it that way, though - cf. group locking.
And in practice you're going to need an array of the same length as
the procarray. It's maybe not quite the same thing, but it smells
pretty similar.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#31Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#24)
Re: POC: Sharing record typmods between backends

Hi,

Pushing 0001, 0002 now.

- rebased after conflicts
- fixed a significant number of too long lines
- removed a number of now superflous linebreaks

I think it'd be a good idea to backpatch the addition of
TupleDescAttr(tupledesc, n) to make future backpatching easier. What do
others think?

Thomas, prepare yourself for some hate from extension and fork authors /
maintainers ;)

Regards,

Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#32Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#31)
1 attachment(s)
Re: POC: Sharing record typmods between backends

On Mon, Aug 21, 2017 at 6:17 AM, Andres Freund <andres@anarazel.de> wrote:

Pushing 0001, 0002 now.

- rebased after conflicts
- fixed a significant number of too long lines
- removed a number of now superflous linebreaks

Thanks! Please find attached a rebased version of the rest of the patch set.

Thomas, prepare yourself for some hate from extension and fork authors /
maintainers ;)

/me hides

The attached version also fixes a couple of small details you
complained about last week:

On Wed, Aug 16, 2017 at 10:06 AM, Andres Freund <andres@anarazel.de> wrote:

+       size_t key_size;                        /* Size of the key (initial bytes of entry) */
+       size_t entry_size;                      /* Total size of entry */

Wonder if it'd make sense to say that key/entry sizes to be only
minimums? That means we could increase them to be the proper aligned
size?

I don't understand. You mean explicitly saying that there are
overheads? Doesn't that go without saying?

I was thinking that we could do the MAXALIGN style calculations once
instead of repeatedly, by including them in the key and entry sizes.

I must be missing something -- where do we do it repeatedly? The only
place we use MAXALIGN is a compile-type constant expression (see
expansion of macros ENTRY_FROM_ITEM and ITEM_FROM_ENTRY, and also in
one place AXALIGN(sizeof(dshash_table_item))).

shared-record-typmods-v5.patchset/0004-Refactor-typcache.c-s-record-typmod-hash-table.patch

+ * hashTupleDesc
+ *             Compute a hash value for a tuple descriptor.
+ *
+ * If two tuple descriptors would be considered equal by equalTupleDescs()
+ * then their hash value will be equal according to this function.
+ */
+uint32
+hashTupleDesc(TupleDesc desc)
+{
+       uint32          s = 0;
+       int                     i;
+
+       for (i = 0; i < desc->natts; ++i)
+               s = hash_combine(s, hash_uint32(TupleDescAttr(desc, i)->atttypid));
+
+       return s;
+}

Hm, is it right not to include tdtypeid, tdtypmod, tdhasoid here?
equalTupleDescs() does compare them...

OK, now adding natts (just for consistency), tdtypeid and tdhasoid to
be exactly like equalTupleDescs(). Note that tdtypmod is deliberately
*not* included.

+       return hashTupleDesc(((RecordCacheEntry *) data)->tupdesc);
...
+       return equalTupleDescs(((RecordCacheEntry *) a)->tupdesc,
+                                                  ((RecordCacheEntry *) b)-

I'd rather have local vars for the casted params, but it's not
important.

Done.

MemSet(&ctl, 0, sizeof(ctl));
-               ctl.keysize = REC_HASH_KEYS * sizeof(Oid);
+               ctl.keysize = 0;        /* unused */
ctl.entrysize = sizeof(RecordCacheEntry);

Hm, keysize 0? Is that right? Wouldn't it be more correct to have both
of the same size, given dynahash includes the key size in the entry, and
the pointer really is the key?

Done.

shared-record-typmods-v5.patchset/0006-Introduce-a-shared-memory-record-typmod-registry.patch

Hm, name & comment don't quite describe this accurately anymore.

Updated commit message.

+extern void EnsureCurrentSession(void);
+extern void EnsureCurrentSession(void);

duplicated.

Fixed.

+/*
+ * We want to create a DSA area to store shared state that has the same extent
+ * as a session.  So far, it's only used to hold the shared record type
+ * registry.  We don't want it to have to create any DSM segments just yet in
+ * common cases, so we'll give it enough space to hold a very small
+ * SharedRecordTypmodRegistry.
+ */
+#define SESSION_DSA_SIZE                                       0x30000

Same "extent"? Maybe lifetime?

Done.

+
+/*
+ * Make sure that there is a CurrentSession.
+ */
+void EnsureCurrentSession(void)
+{

linebreak.

Fixed.

+{
+       if (CurrentSession == NULL)
+       {
+               MemoryContext old_context = MemoryContextSwitchTo(TopMemoryContext);
+
+               CurrentSession = palloc0(sizeof(Session));
+               MemoryContextSwitchTo(old_context);
+       }
+}

Isn't MemoryContextAllocZero easier?

Done.

I also stopped saying "const TupleDesc" in a few places, which was a
thinko (I wanted pointer to const tupldeDesc, not const pointer to
tupleDesc...), and made sure that the shmem TupleDescs always have
tdtypmod actually set.

So as I understand it the remaining issues (aside from any
undiscovered bugs...) are:

1. Do we like "Session", "CurrentSession" etc? Robert seems to be
suggesting that this is likely to get in the way when we try to tackle
this area more thoroughly. Andres is suggesting that this is a good
time to take steps in this direction.

2. Andres didn't like what I did to DecrTupleDescRefCount, namely
allowing to run when there is no ResourceOwner. I now see that this
is probably an indication of a different problem; even if there were a
worker ResourceOwner as he suggested (or perhaps a session-scoped one,
which a worker would reset before being reused), it wouldn't be the
one that was active when the TupleDesc was created. I think I have
failed to understand the contracts here and will think/read about it
some more.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

shared-record-typmods-v6.patchset.tgzapplication/x-gzip; name=shared-record-typmods-v6.patchset.tgzDownload
#33Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#31)
Re: POC: Sharing record typmods between backends

On Mon, Aug 21, 2017 at 6:17 AM, Andres Freund <andres@anarazel.de> wrote:

I think it'd be a good idea to backpatch the addition of
TupleDescAttr(tupledesc, n) to make future backpatching easier. What do
others think?

+1

That would also provide a way for extension developers to be able to
write code that compiles against PG11 and also earlier releases
without having to do ugly conditional macros stuff.

--
Thomas Munro
http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#34Michael Paquier
michael.paquier@gmail.com
In reply to: Thomas Munro (#33)
Re: POC: Sharing record typmods between backends

On Mon, Aug 21, 2017 at 10:18 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Mon, Aug 21, 2017 at 6:17 AM, Andres Freund <andres@anarazel.de> wrote:

I think it'd be a good idea to backpatch the addition of
TupleDescAttr(tupledesc, n) to make future backpatching easier. What do
others think?

+1

That would also provide a way for extension developers to be able to
write code that compiles against PG11 and also earlier releases
without having to do ugly conditional macros stuff.

Updating only tupdesc.h is harmless, so no real objection to your argument.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#35Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#32)
Re: POC: Sharing record typmods between backends

On 2017-08-21 11:02:52 +1200, Thomas Munro wrote:

On Mon, Aug 21, 2017 at 6:17 AM, Andres Freund <andres@anarazel.de> wrote:

Pushing 0001, 0002 now.

- rebased after conflicts
- fixed a significant number of too long lines
- removed a number of now superflous linebreaks

Thanks! Please find attached a rebased version of the rest of the patch set.

Pushed 0001, 0002. Looking at later patches.

Greetings,

Andres Freund

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#36Andres Freund
andres@anarazel.de
In reply to: Andres Freund (#35)
Re: POC: Sharing record typmods between backends

On 2017-08-22 16:41:23 -0700, Andres Freund wrote:

On 2017-08-21 11:02:52 +1200, Thomas Munro wrote:

On Mon, Aug 21, 2017 at 6:17 AM, Andres Freund <andres@anarazel.de> wrote:

Pushing 0001, 0002 now.

- rebased after conflicts
- fixed a significant number of too long lines
- removed a number of now superflous linebreaks

Thanks! Please find attached a rebased version of the rest of the patch set.

Pushed 0001, 0002. Looking at later patches.

Committing 0003. This'll probably need further adjustment, but I think
it's good to make progress here.

Changes:
- pgindent'ed after adding the necessary typedefs to typedefs.list
- replaced INT64CONST w UINT64CONST
- moved count assertion in delete_item to before decrementing - as count
is unsigned, it'd just wrap around on underflow not triggering the assertion.
- documented and asserted resize is called without partition lock held
- removed reference to iterator in dshash_find comments
- removed stray references to dshash_release (rather than dshash_release_lock)
- reworded dshash_find_or_insert reference to dshash_find to also
mention error handling.

Notes for possible followup commits of the dshash API:
- nontrivial portions of dsahash are essentially critical sections lest
dynamic shared memory is leaked. Should we, short term, introduce
actual critical section markers to make that more obvious? Should we,
longer term, make this more failsafe / easier to use, by
extending/emulating memory contexts for dsa memory?
- I'm very unconvinced of supporting both {compare,hash}_arg_function
and the non-arg version. Why not solely support the _arg_ version, but
add the size argument? On all relevant platforms that should still be
register arg callable, and the branch isn't free either.
- might be worthwhile to try to reduce duplication between
delete_item_from_bucket, delete_key_from_bucket, delete_item
dshash_delete_key.

For later commits in the series:
- Afaict the whole shared tupledesc stuff, as tqueue.c before, is
entirely untested. This baffles me. See also [1]https://coverage.postgresql.org/src/backend/executor/tqueue.c.gcov.html. I can force the code
to be reached with force_parallel_mode=regress/1, but this absolutely
really totally needs to be reached by the default tests. Robert?
- gcc wants static before const (0004).
- Afaict GetSessionDsmHandle() uses the current rather than
TopMemoryContext. Try running the regression tests under
force_parallel_mode - crashes immediately for me without fixing that.
- SharedRecordTypmodRegistryInit() is called from GetSessionDsmHandle()
which calls EnsureCurrentSession(), but
SharedRecordTypmodRegistryInit() does so again - sprinkling those
around liberally seems like it could hide bugs.

Regards,

Andres

[1]: https://coverage.postgresql.org/src/backend/executor/tqueue.c.gcov.html

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#37Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#36)
1 attachment(s)
Re: POC: Sharing record typmods between backends

On Wed, Aug 23, 2017 at 5:46 PM, Andres Freund <andres@anarazel.de> wrote:

Committing 0003. This'll probably need further adjustment, but I think
it's good to make progress here.

Thanks!

Changes:
- pgindent'ed after adding the necessary typedefs to typedefs.list
- replaced INT64CONST w UINT64CONST
- moved count assertion in delete_item to before decrementing - as count
is unsigned, it'd just wrap around on underflow not triggering the assertion.
- documented and asserted resize is called without partition lock held
- removed reference to iterator in dshash_find comments
- removed stray references to dshash_release (rather than dshash_release_lock)
- reworded dshash_find_or_insert reference to dshash_find to also
mention error handling.

Doh. Thanks.

Notes for possible followup commits of the dshash API:
- nontrivial portions of dsahash are essentially critical sections lest
dynamic shared memory is leaked. Should we, short term, introduce
actual critical section markers to make that more obvious? Should we,
longer term, make this more failsafe / easier to use, by
extending/emulating memory contexts for dsa memory?

Hmm. I will look into this.

- I'm very unconvinced of supporting both {compare,hash}_arg_function
and the non-arg version. Why not solely support the _arg_ version, but
add the size argument? On all relevant platforms that should still be
register arg callable, and the branch isn't free either.

Well, the idea was that both versions were compatible with existing
functions: one with DynaHash's hash and compare functions and the
other with qsort_arg's compare function type. In the attached version
I've done as you suggested in 0001. Since I guess many users will
finish up wanting raw memory compare and hash I've provided
dshash_memcmp() and dshash_memhash(). Thoughts?

Since there is no attempt to be compatible with anything else, I was
slightly tempted to make equal functions return true for a match,
rather than the memcmp-style return value but figured it was still
better to be consistent.

- might be worthwhile to try to reduce duplication between
delete_item_from_bucket, delete_key_from_bucket, delete_item
dshash_delete_key.

Yeah. I will try this and send a separate refactoring patch.

For later commits in the series:
- Afaict the whole shared tupledesc stuff, as tqueue.c before, is
entirely untested. This baffles me. See also [1]. I can force the code
to be reached with force_parallel_mode=regress/1, but this absolutely
really totally needs to be reached by the default tests. Robert?

A fair point. 0002 is a simple patch to push some blessed records
through a TupleQueue in select_parallel.sql. It doesn't do ranges and
arrays (special cases in the tqueue.c code that 0004 rips out), but
for exercising the new shared code I believe this is enough. If you
apply just 0002 and 0004 then this test fails with a strange confused
record decoding error as expected.

- gcc wants static before const (0004).

Fixed.

- Afaict GetSessionDsmHandle() uses the current rather than
TopMemoryContext. Try running the regression tests under
force_parallel_mode - crashes immediately for me without fixing that.

Gah, right. Fixed.

- SharedRecordTypmodRegistryInit() is called from GetSessionDsmHandle()
which calls EnsureCurrentSession(), but
SharedRecordTypmodRegistryInit() does so again - sprinkling those
around liberally seems like it could hide bugs.

Yeah. Will look into this.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

shared-record-typmods-v7.patchset.tgzapplication/x-gzip; name=shared-record-typmods-v7.patchset.tgzDownload
��k�Y��{_��0����������n0��0�m���8�I�<����ZRk�[`��w���}���I��Z�PW���:u�'�	b9n�r��Fz��E��q��\��&�������j��vw����V��?�G����^�����D�������������I�0��rG+�A��d�s5���'Y��0�v�8�'�4�l�7�1Y�Gi��(��2�7e�X&kx���A��������{����������vw������/��q4������p��q�:h{��`w������d��;�����h..�B��D������
�y&�n�Y��7���?S��I�{�X��C@�as����.=?�q]t��hy
�n��������6@�\.��-G�3��wGW��D{��W�`�l��FA�(��^6�4G�J�8���T��Z0�`���^�7���Qg��/�������Q�i	G��1q�7�����,��$b~���~��6O���f�4��m��<M`�A*��8��	���y��`,��H������l1
'�4Wj4	Fg6��u8��3|Uxi~=�b�1�(Y;�a��x�4"�#��7`�.�9t&�*
G!�mX�2���lAo����Vi(o��0�a��S���R��y%��,�X�!�bG��X��8X,�[�������z��+�h>���H#Z������y����������y�;	_��1���X.���I���I�?7i�H���,�$���2��� ���;��nHZSN����{1	�ww�$��k�`�m� �&����4��H���0}����4�h���~Kl~���2]����u��J��p
���k<��
���i����:j,���h�*�q8��F�� �)����Q%���Gq��?���{�fS�;�~O���@�.�
����>~�{�����b����}E��Z%�(I�8��;�b��yT�dz7�F2�i8M�����~Wd�n
�T�_�aE�<������@����0B(X�i8��M�`:� �����4��~j�<!���X"��hPcZ �x4��[9c�K3��o}�u�h���q;�i�L��O��\���L��^����b�B�F��"E���J�-E��h�p��z$<�S6O���	��G1_N�uz��D3�<�`1���U��F��I��^�nE"h� �;b��
�5+�
���7q���Q��'�o�������#�W@�`��$��n��i�6v��S�i]�F�_��������#�}=�b�;�Y0�w@�{t
_W[;O�e0�{�?r��vN+
�&]z��v��v�p�[G	R�j�G��^���P������_���������0�|��ap�����Z�����{����1�2���]��>R��?��4�rw `�FzKFh�������by�h�Q(�U��Gh���{h�i0�y0�}��o�V�k�{]��ww�� VZ\���	9�K��"U
F�M���j��G�[�9^�9��x� �g�n���6�`e��]����ljL�?�ub�)��h���+�[|��.���zf����]�6��n�� 5_���FHb���X{���l�|�6?>d*���.�O��$����4�`-S+��S�#�x<}����$�8V�	`�=��7�t,r/iH�q��=����T�-���}��S����0���d��p	8��b�|~�]I+�m+��]]h	�w8y�DTm���|���'I�Z�M��BWu�%�!�y9M��<@��~
^�F�������</s�i���� `l�O�	>o��lze#>]�:�z��)L#F*Z�+�7� P�
�S�d%>���CzB�J�x��V�HI�����1�d���� er"n��=,}�D����dw����f�;�'�������[��������w;�[���Tu��D�l���@��c����F��/����6�.%s>��@y#�
�T�.���dCP�SH��@y���8l�R�4��"'B%�1o�
�6#U������fz�x�T�ji&�&5����Tj���Y������*��HYr���7�m�EG��.�LA��2��I���U>��A<�8���&��� Q�%�"<#�9{��-�i@@I�8%��	�(�hC]��vS��k�.�^�%���	�'���s������&�nm}i���2c����f���#3p�E��9g��s����g]m9�l�,+p��G�U�{������'s��\����h��s)Q��[4\vO�BtG��P���f�GO"I���@�������/��-���0�Xqx"�!H�!0���T�VN������A�9P���z���%�Sd�8D(�Ya�D�C}I��P��0��/�{����������z������?88��
H"c�bxK�w���a&�4����>j���j�H����[���#�V�]V/)��a|0�W��;����v1����q���s�0�DC�M���,Mx��V|���Hb�rA��t��g����#G��
�U����e	�.[Ff0��i���N���6E��>�tG�q#h$h����:�d$m��6�*5������v�{���Gw����?����Io8�w��v0������~��k������������?�������m��t���Q��b���(��T�(����u�L���
o��Y)��=O���^ �����K�$s^��������c6��x�X+��8 �?�?�����`G-��������N��(%.a:�7P�����K��S.�e���	��*���
\
�x����b���JA���a�����M�����{�~������f�w�[��O�
���aNm1
@�� !���V�sF^g�LT�����l���
m��"����L@~����s���_3�
ez'�1�'�������Sq~!.N��>:>/��=�:;+f�9`HU��j�m���_������'�������^���tq����������������8��l���W�8}y���.��.O��0��^��mq�?.���k���1���]�E]t�r�G�v��a�~���u���=������.O�������>����������k����QU���\�@!pTB�`��5��6���B�����\(����������;�g�k�����8'�F������wV���p��i ��s�L6��@N���(�"�y�k����:{+�@��������0X��l�����V�)/R*�`E�����?��my�5t��������������#w�Hm�{���A���_Cm������$vI�lECv>o�TVH6�`���g�`���#�@�!�����!I����*B��Xl����7i7��H������
$��"h���f_f|�6����n�����ou��������^���zA������t��#����d|����������t{��������V�D��j�Oa�R�[l���D6V,������
B Js,����(�7������W��|9��E�p��K ��\���D&d
����5��`tC������n�xG�h���!U����t���U�r/#�#��z��^�������>�80�P�da&u� 2n$J�>�|��5��Cls����l\�0���#�$�/��>5WN:%�QOp�M���}��Fo
�H��|78�r\�2FL���b�'[�zY�C-o���yA����A%����B&�k��7�Pp|t*���?�?6.��������"��e_>����~������z�}s�����q0|��YN���/��||A0!K�;o�&FdA��W�Gl�l�6�9������+����	Po��e=�v%��N� P3��}�x���c��?z*�����N��8����uWh<t��Z������bz���/���_������~�n�9O�L,����[Y��~� ���8����4R��d��;�����fs����}	�{�<��\>���!�<,���q-�e8�Ctlj6w���6�����QGd�z
�)Zj��?�b���������D��������zd%������\����=��J�;�������
Yq�E�!U�^#f��";A����^G^�����x`i��
1�*��xC���x���@��n{�>��	���X��`@����}�����?�#�����<XJ��:��������:���E���S+�������E|
�q�������q>ux�������xI�&N���}���%���+��~����!P|�I
p�vd��8���*x���i4*��r��r-q|���Mp�4��b��:?f�������%����g�oo�^���F�c0�&����w��wl����9
���|{5�|����������GJ/����1�i�]�����&_g�B.�����G�')�5��2�k�=��+����L��e�66A�3���|2��WB�s�W�k�����^$��/����^�H���������X�����O�t�J8���,���"k�:���o����p�3�������U�w|�����������Z+~�%���������_/�.�.~^
�SS�J.X�\����35�:z�P\F8�(��Ae\+4�K5������������:Y�
#S��aT��_���n���9�����#����������VnT^P���Y�Q�*ZxO|��-�����g,��Q��k��J�x2�)���^��������C8�G����m$K����N���3+L-�?���O��hR�|����Y�I�J5j<��s�Lq=�+DW^����;���ku<�=-)xE����:xu���������^��K����?��K:^5�>c�&w�D�x���:e���+�/����d���t�K�	��DRa~;�����)������S���)5��@�������o�D�jZB������������h�@���-��-D�	�W&�1�n.�F2���{=gR����r��dr�{��:�� VJ1�I��a��,a�V��6n�
���������F���B���kz��9���0�������/zRu��q�Y�,������U����X�`�����jt�_I���i3
jl��Vba�E��i7�Y�?x
�����UB
x�EgV��b�V����s,���%�����&oK
:�If��eh���B��� �������&��������W����<r��(��p����Psz�#��)��j���*=�
�3t���� 2��
_��	���p@�9���{[�`��^�tuq������i�6�	���"U��)��.L�v���$)�mN��6��:�e���P7H�QyZ���{�z�Z#Yv�V�L�1�*?��U������e���*�F�Q�	�}]9w^G)�m$
�Ib������	�](�'-�|�1�������e�������������o5�����pT��KN����:N�el��v��g��� c���(�e"/���_����;S�+XB#���Mo�8�f(����0X�Q�zL�?�r8'g��`��JH�]F��4���^\�_���Q��@����K��Q-O�U�>}/����9�%��(��\�P�I0Md������������[�GS����Am���*�[�n�J��������1��ZC"A4�����y9O��@�P8m�&�D�g���,*�R&������8q��+</�>D�s�s���*��)2����$�t�Mc�/X��
���w�������V!]�h	�9�/^J�E@�B����C�X7J#�6v������PNP���O�,"��s~�F��B����+�	��I����rd�'Zt:T���W��d�z��W�(�QAT�:��*5�i����^�?��w���fs���A�w����8d��3�����������2�f�����,�i\��}"�$�{���Z����W�p��
���2V4L�m��7Q2W��,�G���nG���U���gJe���o-'8F�����5����*,T�c�������<� $�f���`9U6u<����������*8k6N�\�?�[Fj�Z`�<�o��S�(��58*���m
���eL�E��I:���c��k\b�#m���9T�D8�qtD�PIT���O�P�]u�=�t�z�=����8����f0rr�#.D��W�>��Ko���mm](-���\�/��Z��>�^���%���Zy��/�CP�F�40�&�zK{������N�A�����I���Vw�+{�fs<���p�jk	O�R�S�����|�B��&�
����������QZ�X���/�vg�q��P�O�
{��$<�>D�/�I��d,f��Z��p�����|m�v)��T����[�v[�����H��(�6�3#�e�*������9V`�<#!`�pL?��qy��[��
���M8BHF�/dK(7#P#`��S�,�i����G�Vb3�/!�������F�����NO62��O������O���pz�>�y��#��_���tp��������5����:��#rC\^]���_:�� 0�;�w����cd�`XE��u0�j���E�Q������O��rP�w��%6S�$�p�;-�Y�Z�t[RU��mO��)}���P�VZ���	���|���d�hc��[��
���B�*t��A����n�A����^��f������.>������K�
����G��~����%Ri���o���<�=���=!�CU�~%�������H((^�"���5Ygrw�4�r^6��P2J�:W�;	iZ8_J����9EZ�
q�z� ��
!����q��#r7�\rN�2��{Az�C~d�8��n��-���KT�d,�g�J��"~�t+����+��`��h����P���'$���C�L%�l�k����%Bdt.�������$�������T�����]x��_x-�Y��s��g��&�x��L"J�5�x�N�}�D��J<8*uz���!�{r*&���D3��Q�},7�~��K��q��"L�� ��8����D#�%��c��2�����xq
�����Us9w6���0�p&�v��n�-8�1i	%��������Mf@���b�8�E@Pd��F�
=��k�|1��&��N�&���|l�������L2�:Z�� �������r����n�w��]��d��|�y��Qn��c��x�
�4y�Z��V��{��5���Pzh�X��qR$�_�a*\VJ�
���D..d�\�4���a!p�A_�����]8o�p^=	��L��x��w����vI��g�It���f��p9���dq~`4�O��5G�������O�M�M���V-�����',50��P�@��H�nP���W%�
�Q8^eN����������k�>A��c�x=����e�>Nh4Oa�&b�Yeg���QzC�o���u�����E��^��6YiO34-��E�zd�����.�v�����eg#lA���k8��
m]��Qs>��}��x�+�6��P�1�8{@�f���=j�_�e:C���>���:����`\�~+���o��n���=�rB�!����	���������Un��pc{������d���������%��"���=<������Qn���x=>�]C�
��}"�E��*�S��)�[��
�g������.j���e�����%��<Z��I8%l��9�@���"��)U��c�5�j`��V��? 1[��A-�p�T=�6����� ���P�a���������W�� �����Q�����@�u�3
2(=�%At�
�c�?h�{�{�v���Mv�&�{�nR~�K���yIC:�����s�i���H�`V��I��������|��R>D�����&(�?^\r�r���ENk�w�r��������([f4���dg,'���(?��h���%3S7	��"������;�i0W
���>9�?����>�IY����=�D/M���NPk�d>e��c�l�[���LPx��<�l���XS��5���~��'��*���S�p"/-��]������e�����J���n"�oAD� 3��4�n��$P�,�/a�����i!)��q��g��5+c|	r9��Klu�����.\R������c���(#����,�f�h�r����TdM�nt�O���Pj����C������K������������#0�\=@��+��~����#���_�-H�uwt�g��M�nsQ;Q	���������)>�Vkj��$�p��
��s�I�M�����y��gn�z^�1n�&�E��&v����u�Yv7��Y'����D
8�a\gQpj�����3B*/��mb�I����1��%+��.c��W)^E�������,'O}Tl�����J,�e��W�^)u�W����n��q�S	0o��4o�=���w�t�z[�
}��;���8u�������s��*�`���)�U�
x9�B�O?`��j�3xu���U/�E{l��v����1k|h7��E"3HJC2mk�Uz-�G+W��o
\��=��`�,���v�N;�u�Y�)B�6�*�m"�������"�����x&\����L��5���b��%���g�B[�_.V8��o�f��zspoVm
���J����G%��mA�:NY� ����uI���6q2-�`��e��I���S���M*jM��b|V����8�:����[�=h���� �t�<��h��,������}j�%k��(_W
W�+=��/�S�s����Q�vg��Z���I4!��vg�����}:�0#I����1H��)}#�R���i��q]�����CJ�.�i���2�bU��)``��y�^8K�L<EV����h�������3Z�!��v��;�
#d3�O�$�oKJ�A�K��)d09��\���%�k������E�`���H2�bc��Z��i����
�*W�2�*Ph����W��$���*����r����dy��V���r:,�h��d�G�[�=���7�G�L"~}.��_�>zy9�s���������'�w��N/��N/U��������%y��l&A�����M����"G��x~
�i��G@���-�MF�?��������;��v�`�,��kaE�g����c�f�������b�)����s+����B/;G�~*���Hsh~�g�`<}n��[,��u9�-�gOh�ek�9�i~���fT-�KUAn.SA���.����S����?ur�Z%�2�q���8��R�z��l����w�����t������ �w����\��)�;��8�����|l|�0G1E	\H�I.Z��)���p�BN)N�'0���*,��w���v��M���&��;3\*(c�O��:]H�*��X�������O�o_��?��Wxi��L[t]�%��G�)�F+1�p���r����nM<��'`U�\���a���)�I����c>�g����9�����V���IR
��/

]��
�8��+��A�����y���������bF1i�����+^��/)��{�� Ea~\�	���\��.�C�P k�V�g�����l�.g3���OV�)�W����f����^���
����P*THi�2�4��P��Y��K����Z����y*��	�n������q�
d(��z@V��4��0K���N��Z�6������,b�O����ON��5'���y���9���L����}g�I�
��vQHi���Rr9��I��A��
*�a��m�n<Y�s� 7���E�T
4M�]8���NtD�.Vm�dg���Y�����yD�����l�����������sV����;�$���*����;��,��6"��"t�zj������<y�Z������e�e^�Z�����z��;��0���e�J�v5���v�������\	S�<\�# �	��������F��B��A��J�Z��
"�:�O7��+D9�1/H@�?5]r�
V,���@��P-
����+s
+:sdYY(��k��������&�����0W�>��� ��M��	�1f�5��"��'z�'�~$��O���N��C����?���C�,�? 'G�*c9�h��lP|'K�5��L�n� q�:�U0��}�o����,�F\�����V�fc�|r,
���}���t��tg��7c����n�6�<������x��^���z8-v��U2?r�a	-�,�T�m�e�Nv�U�������!��sf�R�������:~8�1��I��7��F+���Q�� ���6A�nE*u���#�h������e���x((����F��IY�g��p�	+��k�@�[7�g�j-����'~?�E������ ���R�������2�WN~���.ta3�r����C���m)riQ#5V&���z�:�����y5��6�D��n�[Rw����v���(=?^E�Y��@>wk��,�
����6����cMV�Uia� r�J%���|umA���4D�v�����=��I��W4%����	�O:P�b��*/�OI'Y]����r�l���*��+��t��&����m��_9�$�����fJ��Vj�'>Z��2
��X��e&N�Q�/����/�gM���Xw�j�$n;�$�+nJ����Z6�VMI9���
a��V����
A��q~3Xv;����M��W��a^��L��U��**x�K��Y�R�������NC1�xU��hj��?s�:����
�2]��~Zq��w�P���[��$�������i��'E��-r��Kz9W���E�TQ)u7u��:���u��y�9�_-����)\�7wBc��vM�������}������Ja�lu�IW���C��9=jAZ��Y@��u^�`N������x�;��������y��+
T����Jc�������3t���n���uS��9k��x�;��6�x��M�/�����n1���i|�M���7-����,*���T���*�<��5(k��"��r)�q�����bJ����94�TY�=��xZ�
S���xkn;��b�������GL	���1�*;"���"��X����aA�k���Ws��y�:��P+�z�%����w�X���"g�Q�#�
���M�`�W	Mj����M���r�����h�K������������d�����u)�K� _ws��?��6K��Io�Dg�d�b��=�U�,�H�6�er���4 H��t��r��
�*�~4
���"C�p��:�[� ��3?����kz�
��=I'I������c���#��y������~>���9�
�W���u�S�|��5���D~_C�������WK�*��W�`W�h���%"���X$���@�s~�T��
x���X���x��eM��M|8M�w,��)ylV
/�5:;�?���Wf(��v���S�M��K�M\n���C���T���~������z��������9���q;h����9$~>QK������;��|-6:V{���M^	�L�����&9G�S���n���y��/���������u�0�m�mO��u��L��#W��1��X�I�����.����Y�~X�Y��y���8��2h���Im��0�QTXj.d�>=��E����I�te*��v>�p>�C|}`��tLS`��Tj�R>�Ni7����#dE.�&T��q��V���o�)�)�Y�^I���G��
:Q�Y�����QV�'�($
�%!9�C|���/�?F���b��@��jKs�������)V8a�2|��*z>f�Ygm�SN*?e����������
Z�D�>#�bE�����j;���7��z�(�u}M���t�j�$�k�8p D�������X�w/W?�)���b��SVl+�V�L:l����
u��x��GW����"����������������OU���������<<\1�p��R=�������bs�!b�A���PR�U9��r������<���"B�`��T�����
�+�����f=��3U������.W�*\~��Vi��T������������<��*M�����0��K�P��sTy�scd(�	!���'N�kNj�?���j�~<i��m�%�����9�����s:���?Os����+n\�&.��C�rd��P|.G)�\��=�0����,��Jr�2���Upr���*L�����;R���0Er�w7P�E�bnJ����$�*�
�����f�\{s����8�
����+pO�K��������ze��"B��:(���,���er���<�+9���(8��e�Y^�Hy���P;��\*{X��
����aU�������������vk��8{��X����X���(+��)�zKl��=���C��Z�
h��*@B��G�"YN��O	����2�!{L�Q�D�����w\������%�O��D� 
�2�������F_u�X�=��1����/��
�KY�2NspBoP�����[���75�?:�r�(H��	*2�[H�*�xh��sYW�3��m����:��JI�1/�n����8�D�N�r|����C�� ��d-���cNz������Q�*?�����!GZ��.��Oi��^����VCd���G�riw2����U&����E�VH3�*���e+sY������nq��^s�6��k����^��}h:�|KK��`�U��4/�i����V�v��:?�M��J�� j��?+�orwg���0�e�T)^���0���p�0�{���P6���n���{���|b�R@���mB	�)�<��Y�XFQGQ�
Q����bw��PW_�L5��/�s5�L�F��"�nW��jIMlSCA�4�����
	GiE�R�j�,J�M�����(}�d�,���-���o�k���pB=�^[����B�b[6	f���u��~����O��9�����-(�m�&\���5k:��Zv���8=��8{6N(�N������7Go�^�^��'�.NO���R��6i����>�f�]�i��3�z_�8�����<����8{{zR�r+pX����e���>V[=jw���8��N�9l���hw�~�V���:���z\�����3��K�:��g���@a�6����m�$������1�!�1AD�8�H`*�9���*����(N��
Qex�lgr[����]��@q�:�F	����aR�1*G0��SQD,�����L��\2�\<2�XHj&�����`���z��\&DJ��
�<��c�)7�&�2�$����6�u�vQU&�M��r��<��g|�h��>�������1&p�+�f���T*���}�m����T+�����p�����k\����
������F������FJ%w����M-"a��w�g���Z���hw���Nwo�'Z�Ng��;��G,�]�`(��<�V��f�����(�����T�&��^������m�}��O��a�����^kw�3�o"����'Z�g����V�3`��$&�
.�����j���^�)�P ��&���+��Ixe	���+����{���N��n�@�\.���3��wh����_c���l��*�Dcc��;��/���r�U"A���S1��:e����i4��t��~9��{��
9�q�k����g��H���U�j�d}���s����������y�;8�?�?Gst�/b���+'a2Z�d�L���"y����e�2�������y������Er}|�����H�t���?���_>t�w��w���q�h��n�i�����Cn�"���&{�<��C	����t���7���W`�f���W�nwe�i�<eU�����G��D��u���G�9��]�m����!�U�V)��P�P��m�(��=)���~�	��p�wp������+�#�a�k����N�%7���sU�sA
)]K��~$�F���~!�����	��T-)��b�s,<l���V��1���S�����d{�����=��?bW��dn�
��	fi�i20b,+aL�����c�\A�_��{����v�TC]��7�w���hwz��;
�{�!�����,SR�f��Aj��G��F^����X�K��B���7Mv���^���
{=���tW�b�CZ��
UH�wv��?�?�<d�@#:o;�I���c�Qex+���Qq�k=V����KQ���}�K�)�
�p!:4��Gd='�4��A�y� U���2#*#���ar�igU�	�d4�Z������������
7�&�b�7z����k�N��$��I�TG2��
rw si����(��r���zQ�J���Nyk��Fm?B�QHr�/ZO�OH*[��S0s0�T-�����/������ Cv���kj����u�	�d�3r���)/`G"�����&=p����cB��7�K{�uD�1�}��EC!o.~��W�.qN
�M�7}vt��,! ��$n���(8��GS���%jw��XzC��F�:
):�z�F3������x$�O ��`����7v��3����P(A��o(��������'�S��4Ap(��.�VP'�KT�7�)���N4W��$��Q����qJ8l�J�\<:�Qu�H��Y_D$$��������\����6��]�Lk���>8�76�������tv���8����*�!�o��ip|1����TN��mg�Y��"�B\[���i!�����q�"�|��������I�y8�+��y�{������4�OLm��SI����1J#����W�
ax���yj���t�&2�lP71g��174�l�X0X�����o���l`�"{���C^'��#��Nq:D��c��,2����[��:����8T<���0��]p�p����QwZ5k<8��_g�oT�KL��I:SE�%�Bg|���bSYB^t�)�Z#���Xh�E1����sI/b,���������L�
��(@.f�h��\�tR��,�s��|�GH�&Ky���I��@m�{����*�������8��d�U��I4���J@'GWG[[O�O��i9,�1�X�JI����[�1����p#��j�������^]��d9�	���T	we�����(�ct����T��b4Y����������D�t'e�Vw*����K��R�l��� �0;s����U���:�CSV���b3
�:7��`�Y�0_��*���J ��������&���E��s�j-�!�����c��I��)����r�����tt\��� 6�6e���yM�!��0��

��,�w���@�,�S��#
)^�c����Gt��!'b9����\!9m�����QD�4�7?��P�Sv�d��'j��p6H3���{��_��X-S��������q�
g	�i�c�v�1�n;gV�h�W�_07d�S����~�X�!�)#n�h3�\o��Q
����pdB�9�����6�
�f�i��yi1
F21���������S��k����/>c 9H�qaD�&���k�40������7G�GG?������)e�������S��BQ�:n��|]��(v�\��!�S���*Cd�r��@�����M�O�&������c���l��aa���T9

��bP���PzD�&3��
��(�ai�L�Cz���@�/p2S��/�th7Mu�������������������GU8n2����c`���QcZyv��5P��s%�I�0�$���%�����!�;zv�W������c�U@�W��Gi9O$R<gK%Ni5
D��L2��b�[�F�(`�>�&a�o�)�9{��3�u�������$q3����)��{H7}Y�{����A�n��ex#u�U1M�=��g<�A�C�������rD��ag���'����� ����M�vF�'���-����W��d$I ���cnOr��l-y�^���Pq�fY2026@��V�}��T���{��.�t���),C�)�p`��3��v/'G��.���X��Y���A��b����w^&�U�iXA~&Y"��0��W���$��P�f�m��P>E��Yi����S#���6e��d�1K{����� �P������]!8���JJ����H�c��^��'<j�J�e�Y����w
�/j���c(`
�P�dY.��w5A�>,����`A��E,}e�i���0-[*��H��z����k��Y�s�3�GH�D��)G�y/�*�1��}W[A�?e�8T���������[}
H��
UKy��T�!� 7:��{T���c�~�;C�X��E)y���&�A�eH$�,����6p��!�S�s��O7�[0��)\s�F{�JE�x�1���Y-����d�Y����z�|��dq/�s��
�-��d:��tG��&�e�����pkm9^*�����.�:e�����j2�O5������(��8f%���j��1��lX/��s�/_G�Z!qr�z%�G�d�C���;�z2@_q��>���?�tR�d���V5��g@��j0��Xs�|QE��*����A,�o��{x��,����~F��8�w����:��
����/�S5�T�\o��;BQc�@�a�L���h��2�~����.=I+{��i�R�8-yi���O������J�]��s������������p��a*j"������.�Q�c�l��x��]u_z|������O9]��ZOk_?,�'0@�+z���Z�+��r�j�������lZN"
e���g�:f1Tu����);��i�|(��%v���ih0��	���0������h�����L��I���$���p���{��i7�OE)���0��:WCrU��9��	'jT��b��#��\����V���S��$�I�����n��� ���
�,���6F���as��.Z�pN�XZ[�Iu?�d�����:z�Rn	������F"������l��D������SXP�X�r��l2HT����<T����'��JaO��yH�����T�x�����R.���C�Q�?[*P7�(� 42� i/�/�QF/L$�X`�`Xh��H9��)�iQ���z�hk@�����.%�'b@~
��{��dZM����q;I�{�5��],��D�����V�U�e8���mnYS��V��("�92o�5 ��[�.��R��sHZ�m�^�,���YbE�:����0,�,�B �+�&�b�e2��Ot������byi;s����%@U��-J�����X�d��d
�9'��R>�V�iI�Y�6����o���2&7�P��k�Y�)��_�l/g@�Q��5F�Lw��� F�����16i���&�"���
��o��	O�4�|=G�L���y����	&�g�s��"����#���V�aN�_f�)G�{H��
=��KL�����H���rfH���p����g�������U�B;��w)S���nrg0���H�,�����z�m{�������F��}5�<����*���aiq�������B+Q�P��r��V���_w������0�wtvQ���#���q~X��C�$$C�+��Kn������4��?m��v_�D�9P��t<"��Hy�P����?S�=A�-z���3�}Is������M/j�T�>�	j��������r�8U�y�kS3iSd�U}J��9�	�A��#����$���X126�|��x���p��������wz�b{�[�S��H�������@Y����\��!�9�]eO�boV�J��5�CA����	����������q�C���i|l�vM��3JI*�:�������2CvKm\p��56R[�P�x�<c����aK�w�e��1��D(,�!�o2���Ze,G�h0�����nIu��f;�-��7@'<��y��7V��9>v@.�A������pX��G2�YKS<F�����������rTN�'��O��@��o~J�o��w��i+��Y%&P\���:QX7�"+���a�`�#��#Nty��g*�1�D���R���T��P�e���^/gTZ7�|k�-�D��z�D�tG'<-^g=�@F%��k %�u�	�� �6��9�%��(���82��Jk�cW)b�Z�����C2�j��P�w+��b2�
�r�n�eO��L"�n���=��%u:=8BM��Y���2^�
�Z����kWx,.�PU��T�U��-��/5��BJQW~
�����*�R�f�:+c�A��=���7�B����7b�i�pHQ�����6����n�T�I�v���\�WxM���%��Q��mY]���2��d��(�l�MZ4d>U����<Q����]��O�[v�ew.>�[J)�U�?i�NS%�S�\��$e5���"`�Z1���<1<F���w|���Z������sOLIIw�>l(� ������vSO<#��T);T?E���0�e���lW����Gm)g�	F�����������QlV�BN����
~�_v�0�����v�@|=ir�C0��g���m����%X8�#�Z��#����O(�o��zj����d��?]��*���z�<u��D[
3POkbAzw����p����	�	�g���cC�6M�IRa![C=���k�4�u����Kt)�3�������~>|rM9'DV��6�$�?�<��
e���f�����U���kW�5��B��%��B��h����F����;*�iun��bdEE
_+T<�7�$�<o�>�!s`)��s'��Y� ��iLa%��N�0�Bt����Z��z��0B� ?OiScG00O�92�#�@S�_��O�\����
oSKk��=�������J��X+
�w�
�x�2�-���Q��ac�l@�WIJV��K_(A�#"��"��)������P]�E��?����cp���
?P���%;{}������l7���<(^D]8c���\�&C�?Q�^���@O.FqzG�8��"�y��8}�]lM�;g�[.�G���"���Xq_�u�W8�C�Ch @��������P3@/�?	��jQ��[m��V4a�������+���FT��%��A4a��_������y-������&����e�Ek���I_�r+�|J����[�����(Y�0B�hKK��s��
���/)��k6\N,�]y�yfj�:U2=[��t�V�N|��+����������Q:U���T�7u����T�S;�����L��N��|��� ������C�����D��p���i�
8�����9�z�2���������H�+�����.�+}��+Od���U���u�EZg�%T��:�w�%�@�z7i6l�-�������a\8tn���
M'�?����G&k/X�z�#����c�(��a��~�x��sk�2��4�l^�[j�Rq���Y�T{B�/�m���B'3<9����Y{�����rN����<��*�����z��A2|��s�c��(������bm�f]*���]�k��������mR�a+���$�"��\LF�$��
a?����������]��G9W� yu0RN^@��P�4Q��Bc���^��+��][���2Gws}�UM�.,�"��������~�Se�
8>���Sk�PU�2�{�u�BSeqhrm	�W��o�����K����l�����w����n��x��)��%�f�~T�<�G����a3Vn��z�g�����{��l#��.*o�t�ir�������1c����c�G�\��������p>
`����h7�I����rd�
T�b��}�]���m�q���l�6:u(����:�L0�N��FI����J�������%������kU��������_Ex��l�i����[l���(��9R7x�w������?����F�|�D���c����-�ya��~��+M��W�����_�R��k�5)o���)oIK���eqio�~������%��<�����1�0[��W�'9��2a3`r�PYk\dKl�(�#n��Gu�m�NFS�<I���q�A��9{���0z?��u�s���$U��*�����U!g$�j��z'�P�O�r:���3g1Q�6(3�St��L�:��he�f�]�z3x�������OP�;�C]%�@G�4�N	R0-gX���i��{X�0D'�n�uJ�9f`����dh���,�ch�;��!�~�'�X4oE�N�"�TH����v�XtC6����m
����H>�i
�������i%��[[��-�-��pxh�4Z�|��4����-������\5
x4
��*�1���B������L�tP���+����<x�5���^a%���i�_'j�Jq�R6��>�~�s2���!�W�A8%�R���VUT<P�X�
{W5>s/^�?>>����kwi��7IT\W��0Wj;���t)�(�h���w�5�W���>\o
�'��}�������s��P5�,�����W��.�{��Y��k�9^Ez�,���jZ��S��>x@����8�����-q��.���78��J�/����(�X+�pF�XqEJ�NUc����u'�)�����q���7�fa���F����j
�* ��Uwc�&��mJ�����)�U�Sz�5��m�,���xO:���R��}�Zv}�7 g�<��2GPD$,�r6��4(��bOM�k�L�������]d���]�>o�����p�W��Z���������4Fz{r��)����9�����J�zb�GRS>�BA6���X�M�����������0}�<�Q����X�B���ZO������5�����D�G�������mI:~�0\9�RE�4)��849Y�}�P� �S8���IF7��J?��l�0�*>��x�������d�s����p�	�s)�����)�3�8[�m8�T����/tP�"������uN@ ���/��i�q�������b���;��5�u

}��9Ah��i��"�F[N����
��y�X�4�,�e��l��U]����!���"�c1'���+A�Nw������S%2���M���x�0Cs��+�3���[�=����B���94�=8!E~��S���+��'�(�����a���>��_��;�|>	u��/��C��(�q���z�So�a�
S�V���K��0.�\�����}���=m:s������=z��h���_[�i�U���#�;��4
O-�����X�����1��=9{C[�?��K���?��{�_8/�C��z�DVy�9+�t��n����NQ��b^�*q
� i��}����t/�V�/E�~��e��~�T�}h��.�T�a9:�PU�wZ M�����f���y����x�~F����U����3����il��^���M6{l�� ��.�@_�q��Z�7�S�E!���g�-�|�PsS4�)��M:�y��6��[��om^V��n����ts�{�t�������C���"4*2���|\�|�/i�t���{�:s;�!�u�8{{Rw��5P�t���F*&�*0���LB�7�J�S����du��p���6���m���M������b����6?#���������Z��YD�C����/"�u�����
P�b��A����{��}iV�����J9�6����\#-��<����ts
�Z�E;��-�����6G.c	]�<�z��"�.o���(qu��"����V�_i
��Gwu�pgjr�Y���]b�z^�Q����fC��E����l"�+	�WFq�8������u}����]�`�<�*�x��/|���}n��j�����-?��4�:��n1�����.3�K����mF=:����7����a���+�:���
pg�+��s#��3=)�-���i�oN.~N����k�Y��*1��'������*j�������K�����>�������(1��m���M��e�XU�W
�kW;|����NFB��+��U�a�,�Fn��6���O������%h����2������B��8_-�{��B�����;y��I9E������hA5��|�����sb~�*�������=�z��4��������~�~���)���\���U3�*��g�C���Q#��6!(��'G7�C!��.�Z��1�z���e��
Dn��{d�'�}���|y���DS���U�>B�#���������r�'X��J�����
35������y�$l��?U��`)�X��p���<��cX�'�.�r����B��?�`����d�|�SQ,�)��A4}q0�g��������pl�X���+��U3��T��9Ts{U�|V�"������"��������ko�L�?�i���u���`Y�%�fhwO��ej'�+�rY�FU��M([��j���F�n��������7��r��Od�*��,����o)�]Sx�c��o�6��{A�>�h��%a��}B���Ut���.������p���[[���Y�G�P>��4�3f9d�p������|�����:��R/O�.�.�_yJ{V�)5=P�\���K��1%���@6@��tv�{�?��Nypr�����+�k���B�L)6��Y��:"�-qV7A���C�������I�B"T5F���v�V�A7���Jh�?���.?��z�J�(*3����#��\�+�����p}�EI��o��:f&��X3u�V��hM�fdL8p���KyX�S�J9�4�Z��l,��X^ejG:�$����K�M6G�d��R��TX��s�5�����!�L�CV(�L?��{|����������&h�K�g�FW��Y��B*;Q��%�jm^���-��#�F�0��<����2�n)i��
�
Z*Ss�|m�i������N�KPh��
�20����~�N�Y���������WV�[s����C��""%\!Z5�c�n�$�����&����L��#��|Z�na�y'���R�w��B�H���`�Mi�f�yp��������#�al���|��O��9g��q3�}��[����6�o-��[z�C��
�~U��������i�MG���jL��o�M��1Smt�����"]��M�40{e �i���2��H������w4tk8%�}W����1�-�U?8{���&�Q0a��K�;4��9H�JE'������}����[�����-(���f�k�=�T��'�������Y�����gS����el:��`y���&������U2�oS%Q���\�	${�'��,� ����<Y������J)�GFA��lB
_.Z5���
���O�X�<���c������2�U���e������o��`��PD.&��]s�32�������^�|�9�]�Q������0��P�dl�7��^�F�qq�%���/�U!��{� ��J��q;v��qN��Y�:%���{e��w]P���,j�k+#/8K�������:m��*�p2��5�x�I����rGg��aY�FW?�����(���^0���{��d<:�ag���
���U����*���k{�b�������(���W#����_��a�%,��B�32�r�2��=p����.�L[���[��5Lr8�41|�njv�����3����������`��hY�8

Q�4��f�R����������������������������������������������������������?���~�
#38Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Thomas Munro (#37)
1 attachment(s)
Re: POC: Sharing record typmods between backends

On Wed, Aug 23, 2017 at 11:58 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Wed, Aug 23, 2017 at 5:46 PM, Andres Freund <andres@anarazel.de> wrote:

- Afaict GetSessionDsmHandle() uses the current rather than
TopMemoryContext. Try running the regression tests under
force_parallel_mode - crashes immediately for me without fixing that.

Gah, right. Fixed.

That version missed an early return case where dsm_create failed.
Here's a version that restores the caller's memory context in that
case too.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

shared-record-typmods-v8.patchset.tgzapplication/x-gzip; name=shared-record-typmods-v8.patchset.tgzDownload
#39Robert Haas
robertmhaas@gmail.com
In reply to: Andres Freund (#36)
Re: POC: Sharing record typmods between backends

On Wed, Aug 23, 2017 at 1:46 AM, Andres Freund <andres@anarazel.de> wrote:

For later commits in the series:
- Afaict the whole shared tupledesc stuff, as tqueue.c before, is
entirely untested. This baffles me. See also [1]. I can force the code
to be reached with force_parallel_mode=regress/1, but this absolutely
really totally needs to be reached by the default tests. Robert?

force_parallel_mode=regress is a good way of testing this because it
keeps the leader from doing the work, which would likely dodge any
bugs that happened to exist. If you want to test something in the
regular regression tests, using force_parallel_mode=on is probably a
good way to do it.

Also note that there are 3 buildfarm members that test with
force_parallel_mode=regress on a regular basis, so it's not like there
is no automated coverage of this area.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#40Andres Freund
andres@anarazel.de
In reply to: Robert Haas (#39)
Re: POC: Sharing record typmods between backends

On 2017-08-23 09:45:38 -0400, Robert Haas wrote:

On Wed, Aug 23, 2017 at 1:46 AM, Andres Freund <andres@anarazel.de> wrote:

For later commits in the series:
- Afaict the whole shared tupledesc stuff, as tqueue.c before, is
entirely untested. This baffles me. See also [1]. I can force the code
to be reached with force_parallel_mode=regress/1, but this absolutely
really totally needs to be reached by the default tests. Robert?

force_parallel_mode=regress is a good way of testing this because it
keeps the leader from doing the work, which would likely dodge any
bugs that happened to exist. If you want to test something in the
regular regression tests, using force_parallel_mode=on is probably a
good way to do it.

Also note that there are 3 buildfarm members that test with
force_parallel_mode=regress on a regular basis, so it's not like there
is no automated coverage of this area.

I don't think that's sufficient. make, and especially check-world,
should have a decent coverage of the code locally. Without having to
know about options like force_parallel_mode=regress. As e.g. evidenced
by the fact that Thomas's latest version crashed if you ran the tests
that way. If there's a few lines that aren't covered by the plain
tests, and more than a few node + parallelism combinations, I'm not
bothered much. But this is (soon hopefully was) a fairly complicated
piece of infrastructure - that should be exercised. If necessary that
can just be a BEGIN; SET LOCAL force_parallel_mode=on; query with
blessed descs;COMMIT or whatnot - it's not like we need something hugely
complicated here.

Greetings,

Andres Freund

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#41Robert Haas
robertmhaas@gmail.com
In reply to: Andres Freund (#40)
Re: POC: Sharing record typmods between backends

On Wed, Aug 23, 2017 at 12:42 PM, Andres Freund <andres@anarazel.de> wrote:

I don't think that's sufficient. make, and especially check-world,
should have a decent coverage of the code locally. Without having to
know about options like force_parallel_mode=regress. As e.g. evidenced
by the fact that Thomas's latest version crashed if you ran the tests
that way. If there's a few lines that aren't covered by the plain
tests, and more than a few node + parallelism combinations, I'm not
bothered much. But this is (soon hopefully was) a fairly complicated
piece of infrastructure - that should be exercised. If necessary that
can just be a BEGIN; SET LOCAL force_parallel_mode=on; query with
blessed descs;COMMIT or whatnot - it's not like we need something hugely
complicated here.

Yeah, we've been bitten before by changes that seemed OK when run
without force_parallel_mode but misbehaved with that option, so it
would be nice to improve things. Now, I'm not totally convinced that
just adding a test around blessed tupledescs is really going to help
very much - that option exercises a lot of code, and this is only one
relatively small bit of it. But I'm certainly not objecting to the
idea.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#42Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Thomas Munro (#37)
1 attachment(s)
Re: POC: Sharing record typmods between backends

On Wed, Aug 23, 2017 at 11:58 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Wed, Aug 23, 2017 at 5:46 PM, Andres Freund <andres@anarazel.de> wrote:

Notes for possible followup commits of the dshash API:
- nontrivial portions of dsahash are essentially critical sections lest
dynamic shared memory is leaked. Should we, short term, introduce
actual critical section markers to make that more obvious? Should we,
longer term, make this more failsafe / easier to use, by
extending/emulating memory contexts for dsa memory?

Hmm. I will look into this.

Yeah, dshash_create() leaks the control object if the later allocation
of the initial hash table array raises an error. I think that should
be fixed -- please see 0001 in the new patch set attached.

The other two places where shared memory is allocated are resize() and
insert_into_bucket(), and both of those seem exception-safe to me: if
dsa_allocate() elogs then nothing is changed, and the code after that
point is no-throw. Am I missing something?

- SharedRecordTypmodRegistryInit() is called from GetSessionDsmHandle()
which calls EnsureCurrentSession(), but
SharedRecordTypmodRegistryInit() does so again - sprinkling those
around liberally seems like it could hide bugs.

Yeah. Will look into this.

One idea is to run InitializeSession() in InitPostgres() instead, so
that CurrentSession is initialized at startup, but initially empty.
See attached. (I realised that that terminology is a bit like a large
volume called FRENCH CUISINE which turns out to have just one recipe
for an omelette in it, but you have to start somewhere...) Better
ideas?

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

shared-record-typmods-v9.patchset.tgzapplication/x-gzip; name=shared-record-typmods-v9.patchset.tgzDownload
#43Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#32)
Re: POC: Sharing record typmods between backends

On 2017-08-21 11:02:52 +1200, Thomas Munro wrote:

2. Andres didn't like what I did to DecrTupleDescRefCount, namely
allowing to run when there is no ResourceOwner. I now see that this
is probably an indication of a different problem; even if there were a
worker ResourceOwner as he suggested (or perhaps a session-scoped one,
which a worker would reset before being reused), it wouldn't be the
one that was active when the TupleDesc was created. I think I have
failed to understand the contracts here and will think/read about it
some more.

Maybe I'm missing something, but isn't the issue here that using
DecrTupleDescRefCount() simply is wrong, because we're not actually
necessarily tracking the TupleDesc via the resowner mechanism?

If you look at the code, in the case it's a previously unknown tupledesc
it's registered with:

entDesc = CreateTupleDescCopy(tupDesc);
...
/* mark it as a reference-counted tupdesc */
entDesc->tdrefcount = 1;
...
RecordCacheArray[newtypmod] = entDesc;
...

Note that there's no PinTupleDesc(), IncrTupleDescRefCount() or
ResourceOwnerRememberTupleDesc() managing the reference from the
array. Nor was there one before.

We have other code managing TupleDesc lifetimes similarly, and look at
how they're freeing it:
/* Delete tupdesc if we have it */
if (typentry->tupDesc != NULL)
{
/*
* Release our refcount, and free the tupdesc if none remain.
* (Can't use DecrTupleDescRefCount because this reference is not
* logged in current resource owner.)
*/
Assert(typentry->tupDesc->tdrefcount > 0);
if (--typentry->tupDesc->tdrefcount == 0)
FreeTupleDesc(typentry->tupDesc);
typentry->tupDesc = NULL;
}

This also made me think about how we're managing the lookup from the
shared array:

/*
* Our local array can now point directly to the TupleDesc
* in shared memory.
*/
RecordCacheArray[typmod] = tupdesc;

Uhm. Isn't that highly highly problematic? E.g. tdrefcount manipulations
which are done by all lookups (cf. lookup_rowtype_tupdesc()) would in
that case manipulate shared memory in a concurrency unsafe manner.

Greetings,

Andres Freund

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#44Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#43)
1 attachment(s)
Re: POC: Sharing record typmods between backends

Thanks for the review and commits so far. Here's a rebased, debugged
and pgindented version of the remaining patches. I ran pgindent with
--list-of-typedefs="SharedRecordTableKey,SharedRecordTableEntry,SharedTypmodTableEntry,SharedRecordTypmodRegistry,Session"
to fix some weirdness around these new typenames.

While rebasing the 0002 patch (removal of tqueue.c's remapping logic),
I modified the interface of the newly added
ExecParallelCreateReaders() function from commit 51daa7bd because it
no longer has any reason to take a TupleDesc.

On Fri, Aug 25, 2017 at 1:46 PM, Andres Freund <andres@anarazel.de> wrote:

On 2017-08-21 11:02:52 +1200, Thomas Munro wrote:

2. Andres didn't like what I did to DecrTupleDescRefCount, namely
allowing to run when there is no ResourceOwner. I now see that this
is probably an indication of a different problem; even if there were a
worker ResourceOwner as he suggested (or perhaps a session-scoped one,
which a worker would reset before being reused), it wouldn't be the
one that was active when the TupleDesc was created. I think I have
failed to understand the contracts here and will think/read about it
some more.

Maybe I'm missing something, but isn't the issue here that using
DecrTupleDescRefCount() simply is wrong, because we're not actually
necessarily tracking the TupleDesc via the resowner mechanism?

Yeah. Thanks.

If you look at the code, in the case it's a previously unknown tupledesc
it's registered with:

entDesc = CreateTupleDescCopy(tupDesc);
...
/* mark it as a reference-counted tupdesc */
entDesc->tdrefcount = 1;
...
RecordCacheArray[newtypmod] = entDesc;
...

Note that there's no PinTupleDesc(), IncrTupleDescRefCount() or
ResourceOwnerRememberTupleDesc() managing the reference from the
array. Nor was there one before.

We have other code managing TupleDesc lifetimes similarly, and look at
how they're freeing it:
/* Delete tupdesc if we have it */
if (typentry->tupDesc != NULL)
{
/*
* Release our refcount, and free the tupdesc if none remain.
* (Can't use DecrTupleDescRefCount because this reference is not
* logged in current resource owner.)
*/
Assert(typentry->tupDesc->tdrefcount > 0);
if (--typentry->tupDesc->tdrefcount == 0)
FreeTupleDesc(typentry->tupDesc);
typentry->tupDesc = NULL;
}

Right. I have changed shared_record_typmod_registry_worker_detach()
to be more like that, with an explanation.

This also made me think about how we're managing the lookup from the
shared array:

/*
* Our local array can now point directly to the TupleDesc
* in shared memory.
*/
RecordCacheArray[typmod] = tupdesc;

Uhm. Isn't that highly highly problematic? E.g. tdrefcount manipulations
which are done by all lookups (cf. lookup_rowtype_tupdesc()) would in
that case manipulate shared memory in a concurrency unsafe manner.

No. See this change, in that and similar code paths:

-       IncrTupleDescRefCount(tupDesc);
+       PinTupleDesc(tupDesc);

The difference between IncrTupleDescRefCount() and PinTupleDesc() is
that the latter recognises non-refcounted tuple descriptors
(tdrefcount == -1) and does nothing. Shared tuple descriptors are not
reference counted (see TupleDescCopy() which initialises
dst->tdrefcount to -1). It was for foolish symmetry that I was trying
to use ReleaseTupleDesc() in shared_record_typmod_registry_detach()
before, since it also knows about non-refcounted tuple descriptors,
but that's not appropriate: it calls DecrTupleDescRefCount() which
assumes that we're using resource owners. We're not.

To summarise the object lifetime management situation created by this
patch: shared TupleDesc objects accumulate in per-session DSM memory
until eventually the session ends and the DSM memory goes away. A bit
like CacheMemoryContext: there is no retail cleanup of shared
TupleDesc objects. BUT: the DSM detach callback is used to clear out
backend-local pointers to that stuff (and any non-shared reference
counted TupleDesc objects that might be found), in anticipation of
being able to reuse a worker process one day (which will involve
attaching to a new session, so we mustn't retain any traces of the
previous session in our local state). Maybe I'm trying to be a little
too clairvoyant there...

I improved the cleanup code: now it frees RecordCacheArray and
RecordCacheHash and reinstalls NULL pointers. Also it deals with
errors in GetSessionDsmHandle() better.

I renamed the members of Session to include a "shared_" prefix, which
seems a bit clearer.

I refactored it so that it never makes needless local copies of
TupleDesc objects (previously assign_record_type_typmod() would create
an extra local copy and cache that, which was wasteful). That
actually makes much of the discussion above moot: on detach, a worker
should now ONLY find shared non-refcounted TupleDesc objects in the
local caches, so the FreeTupleDesc() case is unreachable...

The leader on the other hand can finish up with a mixture of local and
shared TupleDesc objects in its cache, if it had some before it ran a
parallel query. Its detach hook doesn't try to free those so it
doesn't matter.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

shared-record-typmods-v10.patchset.tgzapplication/x-gzip; name=shared-record-typmods-v10.patchset.tgzDownload
����Y��y��(��O��}WE��U<�	MR�n$Q!�8Nn~�
�9G�D3��g��6Rv�����"0=�[uu���Fs=l�� ���l����v�9���m����|�O��:��S��>�����������^���i�T��j��~�����5~i�a(��t�,i�F�%�e���&?������'�a#]�f�<k��yc�@�xe��&�v4I��Y� hi�ww+����k���C�V�������g,����5O&��?����:<��]}�w�:��G�#}4�����������]2UWz�`�Z����u��P���$J�;\J�}j���VO3=���T��A2�v�,�4���u��S'���pw_�_�v�v o^-������'���U{���� ���Q12*AFE���f�<��X�%�Oz�677���N� ������7��<?��<�xs���PEFM�����6�T<������T@����N*�������0��0O�dp��J]'v���Vx�p������������������&z���i@����|����f��3��t�o�4��t3xg�(G�q�0�:����p1�j������]�S����4���\��
�qw�a��2G�<��|1��������>a����a����,�El$#�)�$���/8�j��H�6A����r��8���{l�i��Y�Pd�>����=�:���z�]J��"X�8SS�����<��%��p;�z��p���|�2��a�a��G���<I�7�K@����8�Gc��WC�2@���$������avi|3�EXC���h1����0�����I�L���M�Y�{�"��`z�j7�������4tj��!��d�x��K�9�wpm|w�B�L�s8
��z1n���`A�~�n�l�����%iv3o�tg�szr�g���?e�o\���������Y���}����~�������7����?������N�x����@96���J��A�^�xg}�]�I��P���������=k�=� ��~���~ /���*o���iMv����9\��������3���w�w�d����DYd�������Df����-"�?��dxx�h����]3�x:/����,���?f~�/��_P�y�v9x%r����<z�as`+�y��tS
n�����Q�
T$���������{�n�������0�T�q� �Y��k4���C���;G�����v�9jw[�}����<��Z}n�Z���o��n�~��,����x<�s�R5�;���J�V�?5��ud@6�xm�Mu���������f�nu4�K
���p1��'�'x������dF�_�@�>�{���{�������1+�T�	�����:�6'���GP�m�m�9�w��
���xFOBj6����38&mHx�3��w���x����p�[u8�:R���vs{�y�����V��?�:�O��A4Kxg�8R8p��}�C��P�&��y|s���`���:<�KD_�����/�3�c`��������~����?��w6���A�'|���y�ds��'�O��y�W�0����d�{���=����#�4����]t{��t�W��8m���p��Z]�_]��x�{w����
�i����Qp�Pi~���d/����[8�v�l@l2��GM�����D#'Y�Q�*Q�hLl�,����[�6-mY@FL�a�r�L�e<�������4�<�=%`�:Cv�qx�9�4~��x�n���B�)3��R��E�3r��$����h��4�k�U/�(
j�6�w��O��]����;����"�#�Z���o���w�#�������������W���O�������f��w���wg���7W��?.���\���Z�5ly��f,53�<�1�/���o�{Kt����m����1dC�����n�sZ�a)$���V5�F�w��
�xG��)�*?e'c8����v���'�H0����-�o����}42V<B�S��h�;�7b�:$9���u��������4��"3�G$���eY�+���#����6����[���4���8���d<t�qG���@����+x:M�p�"��r��=#
�29�
���C��>y������N��9�%��z��a%A`�p�H2.�\�3�r���d1��2�2L'=^�����A��t����1F�{O>��J*���{L��W���9�A�`w{����g�������3`a��GK|[cZ`s"��*~��j�<�������<B����AL�k�d�� �BO���X{E����}�.�[g��dX]�I(�r$�\=Mz���P�t�[��0��2ogd����2��.P���S��=Ce^�@����$�H�u�S���8�^����@=G����?����+���"���]��Ca��<�`�U;�m7�]L?
�����Uq�y�e'V��"6�����9����Sn)�eH0-S�!��w���
xZ!6���{u"v���'��=����W�������w�����MDxz�!7��r���R&`NB��"�/�@��d|��g�~��
Q�����\�p�N��[����yc0�d�A9�k���NSo7��Y�vL@@�0�go��<y������������pp������!X����+H�����B���C�lL�V�\=��K�7Q����F4�����%��c����I2��H��������)rJ����j�f|1��=����.��XT�D�H�p�:�����'�>3� �@;���uX�9(�`�	\��������rn#��[|�S�����9UC��^[��{�Z_
;tq7E�%$<�8��������d���s���{�����p�`F5s.���
�QK�NbV��Q<&M;��sroU��K%V�&����K!�Y�����X�;�d���x����_��a��y��!�>2��yj	�l!���+,�����j����1��������������"SM��r�����v~yyqYWO�b<	'����mB��Tz/��K�zN+o�q�|Z�*��Q4Nu�����������/b�;�%V�������$n��5����e��y�k��+�6�'Ql�{��b��c�V^����n�3�����c	�Z4������B��58�
����\�\rj�Q���Y�������i6�1�I�)�+��FS�"K���dl�2���8�����1���������v���H���j���U�xW�FYB���\Q�3s���Ny��
�,o���O�&���vy�-`a��p3=M"���W�V����%�Qtq���`�B�+ ��k�q�e��Jl0������~w�}�l��� :�F������Uf���t�{d��:��1oM�H0���O������-94�;l����i�(|e�P�OQ��8;'�q�9����U�=�y>Wx�(#���/��1��������iV��R���
��,��0�(�h&�|���IG1)�P���X�>���t���������0��>b���0�����m�d|B����U���m��%N(�h{�L���|
���%F�~1����C�8�c:��(�F���j���P����G V�w�R�-��3���Q�`���f����n�}����@�<2%m�������mS�[���@�Y8���h�+��S4�*�+�{�r��L���������8��Aa�E�^�!j�Q�*}�N���f����`�`���C�����`�������������o��{p08���R���#\W��@������C|FL/\��Ldp�~�������.�[
,Y����F#�s_x�����fg1��
��o�X�lc����03J������%�z�����o���D��������`t�	��Kd�`d����78�]�?�p���z������
aK|u}r}��a�,�����]�RG��2���p�V�8�b�W�����A����]"�3t%�[�v�Q����W��j��{>��T3q��ll����Ki�M�	���K�%a1�Y�U=�Rc@b�tCH]���_�*o�C�,p�
&�t[�R���^g���i��s�]|�Y�^;�/�7h�:���i�;�G,mI��M������U-|N�m�N���c1�\[��������,3�9i�J�N�b�D]�y�bhP�S�=
$�i�t�Q0�'1�Fz��8�[����AAUB��#�G}G���JD`Wg ���X�+�S`�'v� ����O�<��(.Q�����*Bl���l+�H�6*�D�e�,R�	���|�C�OH�1���h�k��0o�[:>W��������l=Xx������mgw�������]Gx��_x-�Y������ghs��{����b�;���q�,P����]�J����I�=�4�M� G��([�
���tE�"M��F#At��9���B/tj�W7��h���&Y����^��%����5{yw6[���0���m�?����3:'-*���N��yK�dz4�ru��'�WE���n�!�^c��u��v:4���z����^������$w���0z0��z�{NZ�e�O��$\P�q��{�����������������,A���V���Z�Y�Qp2V���D��)U�]�|���)���y)C��X������s5���c��Hx�=D$<�g�Ll��n�wQ<��E�b�Pw�N1���U�����7�Q�"5Io��.�:G�?��8"u �l���T��gs���P6���IY�7^�p�����j^�6H�$i���d��_�d�O�p�� ����w�]�N��9}�����<z���oaZN`A�_e2C=2M0Tu"'��p��H�v��$x:��V�����B�zv�
N��4���b�:+�`������G���x����n�]��{����-�������&��
c�t��9�O�!ho�5�y3B^���P$_� u��`���l�M��p�k����N*�:���������G��W���a'jF{��VcIZ������j�[9�����n�.�����lu���?��F���[��MA��<<���{gE�����j�R���5T���9$b_���!9U��o���+.56wXU��
�]l��������;���t�d6�L�{���I4��?����@�����������>d9=
���x�2�$�H�����wo��������ASj%d1�Z��W��; �O������oMk�<��h�$���C^��_������vG���^�����������z��9�hH����(���iQuc�W4)W��f�����ki1�gs��u����-;�����g7����gkh2J�=4,l��x�[����L&8���h��=���`v�����8����(����1o�9���8���i,Q�8�}���aR�:�8��HVj���K�ZP���� ��d��S(��
k���%^���tiKf��:���4�6"�u?Am��W�K��=2�(r�<g�{<���?�!��q����O�R��%�G�!�@Nt��.��v��&�q��|���:��L������v|�)�a}��Ih�{pvN�?��t��|���k���s��X2�,�����R�/��	F�����2h�4k�7Q���53�]�Swp�SD;�3J�TGs���!R��.��[��%o��}�9���B��Dew�1@�Lh��1�Z��M#e�b23�-��+��	CXn�?
>���'��A��A
P'�;-���L�����g�<#����7&����l��d��
>�M*����k�)�$�s�?��~��:Ka�#��n�W(�nkx���,M	�r).�[0�?�de�B�*���#��i��6��g�9��ek������k�Y9��f������	
��YU���X����d}����f:fu�D(:����s@�����V���Q]���4�e����Ic"�������P�~�6�,��\x��������K^�,{��/z`;��sl
���oy�� ���l�-�������UW�-���Cc�������v�Y����N
�)�T���6�~�Z�������S���{�0�}-��p�sH����`�|,S?/v�D.'��	7g��Y�5�3�6���Q�3I�3����XJ=_5������#2s@�y67��-�&%P�d����,��g��a�8�%���1�N�2!��:Z1�:�����[���A�i�����B:A���,��z|Y\@��_�����!��):e�9����&d��������}:��B���������(e��D]��.$�V��zP�����h��������$����M=g��{�^x���z�,9��G����,������Vn<0��A}�Hz��������\x`�	��8�..��X
r8���y����a��+�q�A���{�3bT���1&��x�:`V�{��s��82X��,���S]�����GG�DQ��'}�P
7�>""6�9'�2f�M��)��%(�.��
L)8�\�AL��b�QDso���Ug�*�i��c�����f��t1��:�������x�<��&V����0VAX8<u�oi�������]��z{��U(�����z�O�^�>\^�����������|����tv%���^�K�i���(��ao6��f�N��A�G�o~(�x��y���-
��������c��v	?0��.c�T��An�����`y���,�$��4����O�r:N���	�sZ���"�S��:F\B��C*�+zzz+�4c�)�G���|_o���
��H��o_V��q� ��=h�`?W��o��
2�����	��/m�U��0-.Ca}s~J�Ukw7P��>m����Z���5��(]���$R�
N��/�����yrG�Q�m���4+�w�s�n����8�=���M9���������u=:\�A�*[<�+T��ir�N1[�����
����')G)>@�Os���j��]kl�8� �����{�y�k�2v�	�5�Q��|c���l�9�0%��������=������������]|��O���;��m��i����x3�$N���z*�r������(B�	�`//�!vIjhmX�Ct�?�C&\�2tz���A�������:�`�Z��$��\�j���Q����`��p(�OB�(��y��|��R�����E��2���28���#@��H���������9��A�����9�d"��P�5�������Z"������+J`��e���v(��"��!B�efl�#�[��5���^��sua�ED�(� s��a�1�Q\^��9y�b�>2��<8vm�k�V�E<a��7A���(e��C��7�����b�/����_���w�Cr�]������G����T8�/�����h�l������.�����V�3��!C�`�W]�r�46�Ls�R�N1j���!�zZ%�@�i�1�Yw�4�����Y��	y�>���i8���b@Y���zA�"��"���5J1#�'{*J�LT���Z�7�f�����>4��q�?ll��va�p��8sdV�D{mv#e��w�]���j�;��G���z���9O<���E!����J������!�V��)���(��:����k�$�.��U��,D���J`�b���0�e�t
����Q�Ns�i��6��B��"j��g�
/p���L;������,������n�B3�V���.�V��Q��l���Z��QQ������)��)aE:�A1��K$G�	��0��
��~�%;]���5B����k�"��f	�=M)�x:����"o*;v�r��� C���I��<Bx�L��KRt>3��3�7�����t+/�L@��7�Bo�I�S<�c ��@����u���\h�3d���9q��1�����R����f�����t�E�$��l�u�`�W�3L�)y�L��)s���Z��K�a�J��$��}5��x��&���^����f�sT^BZ��x� ��r��D���%vZw9I�6��{X�gDj6���(R�������f�r%�)O���$\yR���G���].���]�d1�(#���0	I�sdEB�e�l�B�)��r5�5�r��%�
��!����S_^�;8�g_	����4tP�yy�~0��F	���5�������|���^�W�m�ksmICk�����$F*v�C�N���6i����C2O���i����LZ����y.�����s�����qn��=z��l��?6�-q;t�F�L������u@+�0\���9�Bq�-���S���lX/*L��lR�B��3�S&]�� #�aD�JKA��MIHl����iuL�ul�q�Q����v�91����U��Z��F�1���n��4Gg��-���zg��L����j-qe�!��CH�T���Br��I�a }�=db�m�?c[��cT
us&,�, �r��i�������7���4tFd�E��(��|�U�9�h���%mh�����GV���)%tb;\ ;������CDJ�a��*��V�j���[.Se���@�l����!�#��I�d��qi��}+��-�������^������a����6�$���nZ�E_��)��*B�-+P�J�7���E���?OQ����$���=Q&������.J3wz���`]UGc��OC�D�Oh��z(�������[��$.�w���	�#L~X��0���P�4JU1�N�QT���9%���%�9�)����!�)���Z��k����Lb�W�Zlr�R��o�����.3o�ylP���&����H��e� o����`F�y) �F��tJ~iu6��I>jE���'S�*=S�
��D����S[�3d6�`��Q8�� +�|g3w�nk%�Z��ln�u������(�w�I�*UZ!�������i������{���U:��ba.�b�����c�NC���J�a����j$��:z�_X�w��?Mr��e�v�������X������E���������**��
�"�G!i�2�"LR�~�E��)��(i�P!��"�uz�w�BZ
I�E���/?3���~0�X��5�r�-�
�[4#M�oF�1
pvt����~�H\1�X���2IuZ�����#^��bH�I��wdz?���)�L�/N��.�Lk��7��ZN�[��
"���/�v~��(L���%�Vf��Gr��p%Y��-���
+S���4�2mQ�Kt0~�G)��X���L�B����|��'��r����f��*�����a��@�x��u>��n����`R����K(7s�m�y�����e��?�����%��a���;���E�b�6)�>��n�m��/�d1��(���{��[h�*���:�B���0��IF�?�F#d���#?� �g�=����E��u���i�lh��v�\�M.�\����x�������3��V0��Y��������S�Q��]`{I�����.J��	3�6g������p����)��<���%����*|�Y$	����D�wU���!3���������.�r����hwS���P��%|t�
#��C���8���e������W�@!]=-��m������(�Im�a�\R<p9���w�N}/_/oD��dd�c�Jq��T�XL)�k8����Q���q�J�t��!,�r-�V��v�!Yf�TYH;�8\���0G��4����6�\�v+����)�f���r�#�	�iJy�g
<G`������i;��Hb���*���6;�������2�I�~TXR�����v��F[:�T3F)5�	)���Z���[�����������+�3��&l�\��0�k������3�|���\��������b��2~\���__^�P����?sk��:dU��}�o{���`��xJnM�r��	<b�E�����q��\��c��_G�W#^	�����#��������' �z������G��D�G��pD�����[
<�S?�"�a7(��h��5�����z�B�U�����%����{��q�\J�Xi��"a�Q�F��c�Apy��!�v�8[OR<�3C��O������bo����DE��<�8���bkT��k�~�������-)G&+d��9����XQ��^���<�5���u����0)[��[��`z$g��`�Q�%�J�����u�z������f���8�K��]|W�������2��`��}�:�EF�������c*@����WKB�~����\�W|������S�
mm%�	z]O���uN�fk�����&�9@��U��a�R�7��a��Vy<�k.X������ �u�L����J�VlL�(Xps)��7�s�� Vg�n��^#�H)b����6��2��45LD�����h.5��}Dt���rK�q��>�����r�4���\>QQ�r���a���qy�4:��gNY7��R�\��+���m��b��E:a`��lN�*�0�z�J����Y:K�)U�����0��)���|���������d���e�T1vzk��
�������M1��zC���J|[��y���j�%kw��!�2f��?�01&�p����[�xj�t��M
��6�?��Mb6�<��*�sl�K������S���B��s���O�����WL�7�pA�e�����q��l4*�[6���F�����c�w6#8��_x4�K��SN��)��	�\r�o\�$���}��\T	5+l�\��+�R:
fn�|�#T�,g.,��y�����4�Hw�x#�Q�(;�F�d����~kx�E�f�5j;G�h�*OvZU��|;�O����d�W��~`��>����-R�e=:\��s��Q}���by�����7�c�s�O6�'
��2��.��/���R�K(-qXyD���sy�Um�����*(�Z+79���|������"�Tr,��q���$���%V��`���&�������;�=��z��Z���M�$
�2n��O�y�=�����!��P��.�cu+;�]o�[����]DZ�?��
Zs�a�_�`o�p���J`s�6Z����`KuZ���d���[�=���aa�dF����d1s�.��4�Y'��gv4�l�&��kWe���:�;��<�7']��U��C��G�AU���� K���z�9hv(�=rG
}��BbW���T���CD%�����o��h����121q�uE���I$\��uwZ>q
��~��S��ppl/���<��k4�U�B��r0�������P��h�>�8�T&L6X�/sz,*�N�%�de�����X�&��Dr�(RT
�>���"Fen���3�����e��%����������"�C"��r7D�n�<��	��� �B�����9*#�a�������A����������n�����"�R	����&T���4�o�E#�������	��x��ne}z��
]�K����\�%�Q��dT�3��vu���n�mj�����S�6u^@�(�j���R3}k+��U�-)O�C��d����'�~�O�Dj���m�3������|����u���v�38<5����n����v5���������f&
MFw.���Z;o�)��1���^������������������7�'�K��@*KN\� _�hU�!�s�|�����u����e�������g��`@a�0��z����?�������:���^��m���Q��������V����
�R�D���(BF�����.(-Z��W�^�%��tjk���4:5�d�ax,�1e%<j?
�g����{��"6��.�(���d��*�c�I�!&eu���%�D�/e=��y�>�n�r������fm���)+����	�C�3������k�GN�i/Q�����6;v�����E J���E	�x��aKT�������12|e�y�]�'H�����i������o~����#N�|Pb�h|n��3����j�:�K q�u���z��f �5��M<h�1��Qahd�e�G�.�������n����������.��4hu;�n�7���X�*n`(��t�,i�F�%�Y����M~^�����{8����������u����Qg�vvukw��=8P��f�}�Z����=�D0/@��+U�p)�d��I�[4��A��k��
��&���B��]��`L^t�/�Gp����Y^�?@�0����E1.z���EE�(Y;.nn��_hY���p10j�(ge
,z��Y�<���s�;X�Q#K�.��7���j.�F�Kj�\��4Q�$e����������0��,�<\��K�9�wpp���N��V�^��b:�<�������6�f��RS����������M^�����q5KoN����g�����O�����7{?�L����ppr8��O~,��yC�a��m�T�F<(ah�$a2���u�����&C�=�����Fw�7���
����~�.����j���d�tz�
�|��%c.�!#��G`��'5o��Ra]u�y
�����8��"�-����Ym8����:�Dp�;�������f�`woou�{���*xE�IUKd*�[��6p��/�(��!���q4��/��=����o��s�J�g�Y���P�2���,�������P�����T��(?��2]����fC�����J�I���el�����n5q44"[CS%5��t��q��8ZL�,�����k���Js���� n/{6��(������42G���q��+y.y�j��s�R�_[�c�^���G@��TGfW�K���?e�o��=^MIh���s��w�X�e�s����fF��Q��%�'�>3��h0|������V�0y�������ZN\/j���A���#P]�� ��F��M}$L.�����^Mk2^q��6���������=2d�Z���n���	�-�2A;\���v�r&p������;�R
�}*��|�g��FF�`�w��F�|v-K{V7�^ �/a
����f�d����<������%z�����Lr�aM��p(���m�������T��!(g��� 
����pw���w������������A�g@����1+G0g7��!����E������g����6�����`��*&JE{�����5��BZ�V�
�w��v�l"�_���<Vdr����2pOQh���9�2;VRAF�wF\�5�?am/r��D���E9�$O�pX���w����������.�;1.�4�&�E����3M�i�d
���:���q���Z�Ay��-������Zd�����Sd>)7�2|V?��_�D���f��0Y`���~�r\/*3���';����G����m�z��]G�[I�X�r ��-e?�`@����y�E� JA�S
qG��Y#v)�<${So��T%����T�(�QlQC���@DO4"��_�#j�,Q����+�-
��q�c���1�I�Z�x"o�g�d���@Z���I�'|���-��XM�9z8���i�����MlzR�S�����J~���i<QB�4���K��������=jSJ����G��������������>(����V��I��m��C+�~�RH���p��GaV��e�I)��^n�	f�N��6��0*��E�A�d!���)�/K�p�u�r���e���R��h�����rH�uu�o�[���4���Xdq��1�i���Wo��U���np���8��9m�Cb��}'7;�����z�G�.�7�fq�[t�J3�aX�2�p��h8�������P�d���b�,y:������1�����v�Qx���yn���t�FzNUn�\������0-�����X�
u1�
L���4�|������>�S��KY��5$p��"��	�������\2���>0�p��V�-�I}Qgh�������I:SE��/��8��)��n#)Zj$��F�4lx�2j�,a2��s��gp�����Y��]73�*�W��_����XJ������F��A�I�ha��IjPM��b\^>~x{��	�{��f�����������\\��D#��/tvr}���l��C;���b0d�/�1 �\k"(D/Q�M*��I������Zz�5�Y<�����.�[���2U�]���
P`E�1:E�q��@*{B1�I��Q�5�~&(��&��J�>����R�VIHi��� �2�`�
2�sBnb|H�0�A0o�Ueb`�[�V������B��^����W8iH�Q#(�p��<2�,k�t8X�7~?�������(?	gp��q9�+�7�O�_b�jqS7y9���2k+�(
��mA������ �~F�8!������F8��9�����Q<��L��<$3�I���������rN���^��d0�p6�r)�X�����z8�,�5�xW��m��n�%������F8�`����sf5��|af0sCv?M$*���5}3�2�F��3X��;Z����Nl*BNX�=E�L��������l
tjJ�k7��!��$�D�I��vd���������3�����6���f�vy���C������:k�[~��������
��6�u�&������-���xd�t��?^q��Q��c��c[������)!f��'+��4P�M�OJ��<U�Z���p
Q����
�
�or#}.�{D��+�dM��yQ�	�M�� 7��+��K�n��C�D�]�V����d��}[9�pT��&a!7p�~�8����D(yl1�|s�����+Ev���8-f��E<��8zx;�+��W�-�j�QEQ_A^��6��4�pH��-D�2j��+L2P�W����#"$�Y�)�r���xS�����2�u,�jF����d.+y~�s�t��t������
m_�tO.��!�XLI������pq�]�|z��������};���c��*:����%bx;+��Z��-�����ya�$I ���c~�R����Z��Y��.+B�@N�D$�Q�}������{��	�Q6������o�
B80����7�/FGr�(.���X��Y�e�A��1����;/�j��,9�D��a"�c�$�E_H3�5F�>E�����1<�1���)`3F�\�j���|��[��\�����]!8���o)��-$����w�����	���4��z����]���,�	�0bE�� �J������$#�����Bz���,�9����]Y'm���l)
(b��T[�35��
�i��f��g����D��)u�'}�jR��}�����wq,��2������[ZJH�*�Hy��T�!��0:��{R���c��;C�X�\BQ*^g:�u0v	E"
����m8�L��I>������bXF��V�9KX#��yyE���M@���QZ�k���<���N�s~������9Vu���cNr�a�u�#^f��rL�c�#�Z�_���w����+�N�����8�}�W}�������|�x_bQf*�<6`�0�MM��j]���8y]����3���H��;z�@_s��>O�OVtR�d��%�V��n	�7 �iL�8V>_���sq���Jbe����{��X���#���8Vw����:��
��
�/M��U\o��;BQc�@�0<t��������_�yp�+O���F�g$/c��0{d�H�������o�������+��[Gs��.�l�Dp�7������I�c����Q=G��z(=>O�x$���t��rJ�n)|�������Up��cI�r��/�Th�r�,�����$�b���g�yf1�������]�.��T���)k7��!��}���e`N�C���g�����+�(���+z�\�d����SU���(���0��:�@�U��)���G2*�p5����i��f.N���Q���*��$�&�!I^H4�-2I�J	(0��A���)%������]��$Gs4��*r��~���N��	����J��5=�T\�T�Q���Il�s��o�����	P��H.+�%d��Q*�/l�����H���y
�/
{r��C&97�����>�V���$��|J
��gK���C�F&$�+$�"�������>�T��}�2*a.A0��f�hk@����"�bj/"�T(8��K�'3j:T^����I6��T��`�b9�&Jh��O�-�.���|[��M��*���v���������5�R�����IU�r���0���
X�,����YbE����E��#�X7A��Dx6��O[��ws���v�I+�KUdv�o��J,�g�Hq���`�F;L����8-�2�2���K;�~Q�1����&���C�(j�����
P}T�$x��>����+9��S[������H��'p��.�.���m�S����o�����8��w��+�`�z�p�1�$)"Lf���	�k5N�����>����+o�0�Q�th,���rB�<�pD���� �=����^8t8�����Q	�+���}��^8��1U���,bi���	=��=y������3?�||{��z���+�������Vq�����"jJ_n���2���oa�c�8��C?��)�����x��KY.��:���pI�_��8r�M������3�i�.���%���bP��I�E�e���D[H�L%�i�`��^7�\���y:6�h:���NI�l��S�+�����*�
�=�g�>K��"
�!W*������\�%#c��Yo�����K�FF���V��{��0����	�����Q�](���v(��
�-8�]eO�roVQ�r6�P�^s3�w�xL��"��K7N�c���hr|�/T��t&5�Yfa�ni�~6��Zj�%J�@�g��.�����������]H���7�$�Fe��z��7����&����Fh���
(Wz��EH�Xuc����hF�y��
��>��O����}$��4���h_8Y���Y#S^*���������<���C%�����T2mQ1������~y'
��Pf�^�,�����a"NLy��g�)�
D�����r���-��@�\x�^���7
9�ni�w�m�Z7��Y�������}��:J3���5�2��&��s;���E}��!Wa�p��N��L+�����W�X��V���* ����,PY*t�d�WFLFU�B����H��I3;�����sI�^�PS&t������fC�4?��t���5Ek���L���lY�}�U>�R���5*.��-8%��H����P�=�������N�	k$H��t�C���+���1|�������LE�$��`�
~�7��J�E����e��eK�^?T1P�X��h�
|*�k_�i*���e\��O������\|�7D)iT�?�Nqw�gF�%������@@��f"��B������y9�x��7�7���2��X���o;������9�����:W������z�Y��b����~�h�Ke������9�.m�_@E�U��3�3=�������R���a�c��s2��a^�C�\��P��`Fx��>2�-t�����`�]���b���7����
�SO�V=�l<����T���w��S�<N��0�|K�H����&�Zn	=�9��l�[�ul(o���$��5��[=��pAo2=�Z W:{3�)�����F�_�\S.����`�iAB�K�#n`�����jO3V=��]]�l�����.��5�&S1J3lu|���Q�y�s{�T�#+*j��X����a&Y��y���	��KQ;��~�f���1����;U���������5��aa��A~�2��=��>�����MA~���=w�����MM�Av�Ko�{R��� �,:~�}[�(.e]���Q��aC��G��HJ���/�`��{D��d��x�M�.�h�����9w��I��:����!��a��q������u����j����By{�C�=���Aq��Cr���q��;��'wz��n1��w��'���(��b��;q9�W8���Ch X���.k!�I�j�E��_�����:T{���&�M@��!4��o2������Jc%���q	#iM*B�k)Fza-u^�&�6�5���=�,�|�f�qQ����`��!N���x_|�F3��a����V��c��J����(��k�_�J]�"�����:�o�x���5��_�2�J�>�z}z�vcc����a��S���ZN��vj�����I�2��������=A�=��Ub`��b������T�#�?MZ��q�rv1��9���@��p�\�_�r��������+�c���x��T�%���b�Hk���?=eu5a	�b c��4����j�����0.���
�Dq�TP�����>~���'61(/X�z�#�����IN-�V-���|����em[Y��JP���%q��WY��=!������b)�	��v�L��;���p�	9����p
���U�}p�����Ar|�����!�#��*�Zk��i�um�(���tQ*�
�������n��
[�0�&Y����(p1E�.&�~�;�P�	_���W�.����+sL�������5�8�5m�������w�[ow�x��o�<�2}����6eR6���;���{�]������y�]u�`%�BC��(���v�t��J�j�@�uF��l���-�aXlK`~���n^������
;fc4���x�;�������~�r�?���Rl��������{8�@�A��=.!��A�,��x�rW������Q��A4��������.����r}@�r��N�y5��b�)�r����?W��@9xs����>rb��z�~t;R��Q��5��=����g4���|�����Z�Q����!�"3*���Ur�o��e�?���y����.>�m����,�*'I���^��0��������C����^��"<bo��QB

����v����+\I�MNZ\�	EZ\����2I�`��v{��K����=�����s@j�C�9U����q��%3_�� ��'���8�dD�K�:�S�o<p+eg�D�F�_m:`ka���s��9$�~���A��s�����8a'�X2��~��3����S����N�)��S�A01D=k[6����w�w���p����x���K�t1���� E��b��)9��M����ct_oP�[��`����G�1I?����x3�ZtT�=C�e�H������*������	�B}������U+�����1~�))�����#$}��R�@�C9�����c+/������:� %�V�����Hh�?�0P,Q�O���������������s�b���9&���%��}Lx/
J��_c����e'����v �a+��G��uJ��>���sJ�r?����@*�`�i����������z�������WW'�05W�v�����l@�?�o	�{��T y������~�i�o;�r��@�!���C�������_B;���%~g����dS����9
P�5������}��<���i�w��T[�������8�1pC��������z��bO7S�(C����`��G�?�-!/�9����������]8�����������35��M�!�1����
��w����t��Ni,C����*h��o�eix����t����T�C@k�����Q�(�_��,�b2�W�4(����6���'�8�����mL����!?���W�r���\��kqH0W�sr9����4��d��wO)��\��3q���IEL�.(>�$���]A�#�s(S��
OJ������q��S3�,��u><h-�;��x���_����L�_P�[[�Ld�e�
��M��$��uu�����!�� �m86q�I���'JL�9n�0���B�a�;�!���}gx)<7�i��31r�������9b�P
����+]���s��$X��=(�$���z��+���m^!*��3�a\�$�0wz��+�:���i�/s&���(��lc��R���?�yl�oE�X�c���0�e�C��sD*�bN]��W�.$�������"2��m<��y`0Ks�T,�=���[�}��������;4�=x�Fa��S���8Z�/�(�G�T�a���>���d�=�b>��:������cU��E\ul������[uX~�dB����f�'\	��������~����xO[��\���7
���6�;����{�����{������s�CV����6v2��
v��7f���go����#|��>������3��{q���s)�l{��x��v{�,�t�����������[�F~HV��������S�=�1��{?����Y~��
t�����#�D���:����s6f�w��Y}D�����������}[J�6�n����&�=�[e��R�O�/��/��f0��`�@���5}� t[N�+��OG��0t��k�/�^U�n������9v�#�����P����4�kMU>.lY������y^rn�wH����?��q�h`�
Dd+�]_d����m������z��a*�}�m�M���F�m�/�L������2����������kt�uR+]�<"�(�kt�?/���X�-��#*U\B��	6�+�����7��]��v�~!+�����f���l�,w:W��+)�������,������%�e2D�e�����Enl~
��������eh(����MX����Y6g1_��R��T#����fK��E�����"�������C;��:�?����Y 4�����^f/=��O�t��U\��d6����p�#������|�hf�rS���;����#�u\�2��7,SQ|�_��=���B�{n��r�'����eZ��S��3> Y�w�1��|�R[*�z��<�-,�|�
�.@�<\���M�E-(��D��,�'�-��b�M�l��.;�J_>,-G��.w���K����kZTW6����p[�����3��&��{�$K����f�9�T��K��U�F(.CA����;y��I�J��C�KY��f�zv����X���V�V�W�'^o5G7�^L�ay7�o��=Z2%SK�kAY�jX�Y��,qp�9022�j�R�yrp�<H�,2���E@��n4�d��Z�e 
+W�8 �?h��O.������6������4�7���c�d�O�6����_���3
������{�;$l���?�qq��a,l�8rg^�y�SX��.�r�9"�\��?�0���$2C1�,���^#��<��Q�n{�"�!�*VB��E��qUG�*�!`U�^B���(�w��Dy��a�/��A���9W�t���!�E(C�c������:B9�����\��Q�gt������zC�hT��PM��"���}f�CQN�����f��r��
a�
��!�/K/~1�!t�����M�u�"���O�rq��� ����x3��-s���eg�&�	@(q���2l<Fq��X���x�����:�����?�����xz(�Y�'jz ��FU��cL��>)�\,���tv�{�?��N�wv����{_�7J6�(|�h�(6��9��&!�-q^7A���C�������L�B"��#w�O����`��`K����si�+���,��e��C=�����j����1�)W���c i��5v�\��$�k�����Z��lF����`� ���?[A�"O�\�����^K�Kw�#���
���4tpH��$cH��&�{��M�$�R��<f�B��S����>\\��>�O�a�����Q8?���o�:��RA��+<���k�����O:Rm�������,s��������M�a��U�2��*���v���_w����V�o��`����&d��KL������/\~|�=��N���p��`)��������iA�]cL*���DHGLu������N,���R���`�@_����,1�������������#�al}���<�SF>d��9�7�G��B��������[+<
��P�@e�k
�_VA���y���6��#{�B
�;&�*4
�&�t�T[���uz��HWn�JS4
�^�D;-S>\��2SUfs�����d���jn��:��E��{o���p�u  �qI�&4@3i������5p����<��V��+�En*�E�������@4U���x@V���pEM|_�55!(`X���N��ykMK�L�!_&��6U��7v��m �+!���E5���2g�U���R�����e���\-���F}�������y�$3���WA�qe!����4�'g:_a��X{��BL����#r������Y�b::����c����;N� @�$��
�vX�k��0..��s�2�e�*�xBo��Q�n[�c_��Z�����nRT�����n����%@vQs^[9y�[
F��oW�����9�G#�h�`���t>�a�P��<y���h��������j3��OJ;���3����M��w�A4j�0a����f��X������5{��e��C�
�(���)������c�4��<V��&�5O?��i
]e��|>�?�k"�w.O4���j��5����]14��*'n���j)�{�b�-���Z�g:��{@��X9����c�@���qz��wO1Y�b���/ul�V��jg%p5v�����A� ��w����pp4��N����ki	F�6T�{q~��r�����-\�8�y���n���^fie����#'�&�����&��r�m�N�?K7y<{pV)���9�o�<��!U9��H4�f���6;��������_~������_����	�l�h
#45Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#44)
Re: POC: Sharing record typmods between backends

Hi,

On 2017-09-04 18:14:39 +1200, Thomas Munro wrote:

Thanks for the review and commits so far. Here's a rebased, debugged
and pgindented version of the remaining patches.

I've pushed this with minor modifications:
- added typedefs to typedefs.list
- re-pgindented, there were some missing reindents in headers
- added a very brief intro into session.c, moved some content repeated
in various places to the header - some of them were bound to become
out-of-date due to future uses of the facility.
- moved NULL setting in detach hook directly after the respective
resource deallocation, for the not really probable case of it being
reinvoked due to an error in a later dealloc function

Two remarks:
- I'm not sure I like the order in which things are added to the typemod
hashes, I wonder if some more careful organization could get rid of
the races. Doesn't seem critical, but would be a bit nicer.

- I'm not yet quite happy with the Session facility. I think it'd be
nicer if we'd a cleaner split between the shared memory notion of a
session and the local memory version of it. The shared memory version
would live in a ~max_connections sized array, referenced from
PGPROC. In a lot of cases it'd completely obsolete the need for a
shm_toc, because you could just store handles etc in there. The local
memory version then would just store local pointers etc into that.

But I think we can get there incrementally.

It's very nice to push commits that have stats like
6 files changed, 27 insertions(+), 1110 deletions(-)
even if it essentially has been paid forward by a lot of previous work
;)

Thanks for the work on this!

Regards,

Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#46Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#45)
Re: POC: Sharing record typmods between backends

On Fri, Sep 15, 2017 at 3:03 PM, Andres Freund <andres@anarazel.de> wrote:

On 2017-09-04 18:14:39 +1200, Thomas Munro wrote:

Thanks for the review and commits so far. Here's a rebased, debugged
and pgindented version of the remaining patches.

I've pushed this with minor modifications:

Thank you!

- added typedefs to typedefs.list

Should I do this manually with future patches?

- re-pgindented, there were some missing reindents in headers
- added a very brief intro into session.c, moved some content repeated
in various places to the header - some of them were bound to become
out-of-date due to future uses of the facility.
- moved NULL setting in detach hook directly after the respective
resource deallocation, for the not really probable case of it being
reinvoked due to an error in a later dealloc function

Two remarks:
- I'm not sure I like the order in which things are added to the typemod
hashes, I wonder if some more careful organization could get rid of
the races. Doesn't seem critical, but would be a bit nicer.

I will have a think about whether I can improve that. In an earlier
version I did things in a different order and had different problems.
The main hazard to worry about here is that you can't let any typmod
number escape into shmem where it might be read by others (for example
a concurrent session that wants a typmod for a TupleDesc that happens
to match) until the typmod number is resolvable back to a TupleDesc
(meaning you can look it up in shared_typmod_table). Not
wasting/leaking memory in various failure cases is a secondary (but
obviously important) concern.

- I'm not yet quite happy with the Session facility. I think it'd be
nicer if we'd a cleaner split between the shared memory notion of a
session and the local memory version of it. The shared memory version
would live in a ~max_connections sized array, referenced from
PGPROC. In a lot of cases it'd completely obsolete the need for a
shm_toc, because you could just store handles etc in there. The local
memory version then would just store local pointers etc into that.

But I think we can get there incrementally.

+1 to all of the above. I fully expect this to get changed around quite a lot.

I'll keep an eye out for problem reports.

--
Thomas Munro
http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#47Tom Lane
tgl@sss.pgh.pa.us
In reply to: Thomas Munro (#46)
Re: POC: Sharing record typmods between backends

Thomas Munro <thomas.munro@enterprisedb.com> writes:

On Fri, Sep 15, 2017 at 3:03 PM, Andres Freund <andres@anarazel.de> wrote:

- added typedefs to typedefs.list

Should I do this manually with future patches?

FWIW, I'm not on board with that. I think the version of typedefs.list
in the tree should reflect the last official pgindent run. There's also
a problem that it only works well if *every* committer faithfully updates
typedefs.list, which isn't going to happen.

For local pgindent'ing, I pull down

https://buildfarm.postgresql.org/cgi-bin/typedefs.pl

and then add any typedefs created by the patch I'm working on to that.
But I don't put the result into the commit. Maybe we need a bit better
documentation and/or tool support for using an unofficial typedef list.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#48Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#47)
Re: POC: Sharing record typmods between backends

On 2017-09-14 23:29:05 -0400, Tom Lane wrote:

Thomas Munro <thomas.munro@enterprisedb.com> writes:

On Fri, Sep 15, 2017 at 3:03 PM, Andres Freund <andres@anarazel.de> wrote:

- added typedefs to typedefs.list

Should I do this manually with future patches?

I think there's sort of a circuit split on that one. Robert and I do
regularly, most others don't.

FWIW, I'm not on board with that. I think the version of typedefs.list
in the tree should reflect the last official pgindent run.

Why? I see pretty much no upside to that. You can't reindent anyway, due
to unindented changes. You can get the used typedefs.list trivially from
git.

There's also a problem that it only works well if *every* committer
faithfully updates typedefs.list, which isn't going to happen.

For local pgindent'ing, I pull down

https://buildfarm.postgresql.org/cgi-bin/typedefs.pl

and then add any typedefs created by the patch I'm working on to that.
But I don't put the result into the commit. Maybe we need a bit better
documentation and/or tool support for using an unofficial typedef list.

That's a mighty manual process - I want to be able to reindent files,
especially new ones where it's still reasonably possible, without having
to download files, then move changes out of the way, so I can rebase,
...

Greetings,

Andres Freund

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#49Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#48)
Re: POC: Sharing record typmods between backends

Andres Freund <andres@anarazel.de> writes:

On 2017-09-14 23:29:05 -0400, Tom Lane wrote:

FWIW, I'm not on board with that. I think the version of typedefs.list
in the tree should reflect the last official pgindent run.

Why? I see pretty much no upside to that. You can't reindent anyway, due
to unindented changes. You can get the used typedefs.list trivially from
git.

Perhaps, but the real problem is still this:

There's also a problem that it only works well if *every* committer
faithfully updates typedefs.list, which isn't going to happen.

We can't even get everybody to pgindent patches before commit, let alone
update typedefs.list. So sooner or later your process is going to need
to involve getting a current list from the buildfarm.

For local pgindent'ing, I pull down
https://buildfarm.postgresql.org/cgi-bin/typedefs.pl

That's a mighty manual process - I want to be able to reindent files,
especially new ones where it's still reasonably possible, without having
to download files, then move changes out of the way, so I can rebase,

Well, that just shows you don't know how to use it. You can tell pgindent
to use an out-of-tree copy of typedefs.list. I have the curl fetch and
using the out-of-tree list all nicely scripted ;-)

There might be something to be said for removing the typedefs list from
git altogether, and adjusting the standard wrapper script to pull it from
the buildfarm into a .gitignore'd location if there's not a copy there
already.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#50Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#49)
Re: POC: Sharing record typmods between backends

On 2017-09-15 15:39:49 -0400, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

On 2017-09-14 23:29:05 -0400, Tom Lane wrote:

FWIW, I'm not on board with that. I think the version of typedefs.list
in the tree should reflect the last official pgindent run.

Why? I see pretty much no upside to that. You can't reindent anyway, due
to unindented changes. You can get the used typedefs.list trivially from
git.

Perhaps, but the real problem is still this:

There's also a problem that it only works well if *every* committer
faithfully updates typedefs.list, which isn't going to happen.

We can't even get everybody to pgindent patches before commit, let alone
update typedefs.list.

Well, that's partially because right now it's really painful to do so,
and we've not tried to push people to do so. You essentially have to:
1) Pull down a new typedefs.list (how many people know where from?)
2) Add new typedefs that have been added in the commit-to-be
3) Run pgindent only on the changed files, because there's bound to be
thousands of unrelated reindents
4) Revert reindents in changed files that are unrelated to the commit.

1) is undocumented 2) is painful (add option to automatically
generate?), 3) is painful (add commandline tool?) 4) is painful. So
it's not particularly surprising that many don't bother.

For local pgindent'ing, I pull down
https://buildfarm.postgresql.org/cgi-bin/typedefs.pl

That's a mighty manual process - I want to be able to reindent files,
especially new ones where it's still reasonably possible, without having
to download files, then move changes out of the way, so I can rebase,

Well, that just shows you don't know how to use it. You can tell pgindent
to use an out-of-tree copy of typedefs.list. I have the curl fetch and
using the out-of-tree list all nicely scripted ;-)

Not sure how that invalidates my statement. If you have to script it
locally, and still have to add typedefs manually, that's still plenty
stuff every committer (and better even, ever contributor!) has to learn.

There might be something to be said for removing the typedefs list
from git altogether, and adjusting the standard wrapper script to pull
it from the buildfarm into a .gitignore'd location if there's not a
copy there already.

I wonder if we could add a command that pulls down an up2date list *and*
regenerates a list for the local tree with the local settings. And then
runs pgindent with the combined list - in most cases that'd result in a
properly indented tree. The number of commits with platform specific
changes that the author/committer doesn't compile/run isn't that high.

Greetings,

Andres Freund

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers