Inlining comparators as a performance optimisation

Started by Peter Geogheganover 14 years ago101 messages
#1Peter Geoghegan
peter@2ndquadrant.com
1 attachment(s)

Recent discussions on the threads "Double sorting split patch" and
"CUDA sorting" raised the possibility that there could be significant
performance optimisation "low-hanging fruit" picked by having the
executor treat integers and floats as a special case during sorting,
avoiding going to the trouble of calling a comparator using the
built-in SQL function machinery, and taking advantage of inlining of
the comparator, which has been shown to have a considerable
performance advantage (at least compared to a general purpose c stdlib
qsort(), that takes a function pointer as its comparator, much like
tuplesort).

I've hacked together a sloppy POC implementation in a hurry
(basically, some code is shifted around) , which is attached - I felt
that it would be useful to determine if the community feels that this
is a worth-while undertaking in advance of a business trip that I'm
leaving on tomorrow lasting until Friday, during which I will be
mostly unavailable. The patch breaks the Postgres sorting executor
node (at least when it quicksorts) for any type other than int4. I
apologise for how rough the patch is, but the code itself isn't
important right now - the idea is. I anticipate that the value
state->datumType or something similar will be set in a near future
revision, so that tuplesort_performsort will know which type-specific
optimisation it can use for the type, while falling back on the
existing generic qsort_arg + qsort_arg_comparator, and sorting won't
actually be broken.

I've been doing some preliminary testing using the dell store 2 sample
database. I increase work_mem to '50MB', to ensure that a quicksort
will be performed for sorting (otherwise, I'm using the
postgresql.conf that initdb gave me). The query is:

explain analyze select * from orderlines order by prod_id;

Once the cache has been warmed, explain analyze very consistently
reports a runtime of 123ms for this query on master/HEAD, which varies
+/- 1 ms, with a few outliers of maybe +/- 2ms. However, when I apply
this patch, that goes down to 107ms +/- 1ms at -O0. I think that
that's a pretty good start. Funnily enough, the difference/advantage
vanishes at -O2 (I'm guessing that the higher optimisation level of
GCC 4.5 hyper-corrects away the inlining, but I don't have time to
check that right now).

I imagine the version that I actually submit for patch review will
have a macro-based infrastructure for inlining the sorting of various
built-in types, initially integers and floats. It will most likely
have some other optimisations - I haven't even used a profiler yet.

This performance patch differs from most in that it's difficult in
principle to imagine a performance regression occurring.

Thoughts?

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

Attachments:

inline_comp.patchtext/x-patch; charset=US-ASCII; name=inline_comp.patchDownload
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 3505236..c5ac708 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -1224,6 +1224,285 @@ puttuple_common(Tuplesortstate *state, SortTuple *tuple)
 	}
 }
 
+inline int
+compare_int4( Datum first, Datum second)
+{
+	int32		a_is = DatumGetInt32(first);
+	int32		b_is = DatumGetInt32(second);
+
+	if (a_is > b_is)
+		return 1;
+	else if (a_is == b_is)
+		return 0;
+	else
+		return -1;
+}
+
+
+/*
+ * Inline-able copy of FunctionCall2Coll() to save some cycles in sorting.
+ */
+static inline Datum
+myFunctionCall2Coll(FmgrInfo *flinfo, Oid collation, Datum arg1, Datum arg2)
+{
+	FunctionCallInfoData fcinfo;
+	Datum		result;
+
+	InitFunctionCallInfoData(fcinfo, flinfo, 2, collation, NULL, NULL);
+
+	fcinfo.arg[0] = arg1;
+	fcinfo.arg[1] = arg2;
+	fcinfo.argnull[0] = false;
+	fcinfo.argnull[1] = false;
+
+	result = FunctionCallInvoke(&fcinfo);
+
+	/* Check for null result, since caller is clearly not expecting one */
+	if (fcinfo.isnull)
+		elog(ERROR, "function %u returned NULL", fcinfo.flinfo->fn_oid);
+
+	return result;
+}
+/*
+ * Apply a sort function (by now converted to fmgr lookup form)
+ * and return a 3-way comparison result.  This takes care of handling
+ * reverse-sort and NULLs-ordering properly.  We assume that DESC and
+ * NULLS_FIRST options are encoded in sk_flags the same way btree does it.
+ */
+static inline int32
+inlineApplySortFunction(FmgrInfo *sortFunction, int sk_flags, Oid collation,
+						Datum datum1, bool isNull1,
+						Datum datum2, bool isNull2)
+{
+	int32		compare;
+
+	if (isNull1)
+	{
+		if (isNull2)
+			compare = 0;		/* NULL "=" NULL */
+		else if (sk_flags & SK_BT_NULLS_FIRST)
+			compare = -1;		/* NULL "<" NOT_NULL */
+		else
+			compare = 1;		/* NULL ">" NOT_NULL */
+	}
+	else if (isNull2)
+	{
+		if (sk_flags & SK_BT_NULLS_FIRST)
+			compare = 1;		/* NOT_NULL ">" NULL */
+		else
+			compare = -1;		/* NOT_NULL "<" NULL */
+	}
+	else
+	{
+		compare = compare_int4(datum1, datum2);
+
+		if (sk_flags & SK_BT_DESC)
+			compare = -compare;
+	}
+
+	return compare;
+}
+
+
+
+inline int
+comparetup_heap_int4(const SortTuple *a, const SortTuple *b, Tuplesortstate *state)
+{
+	ScanKey		scanKey = state->scanKeys;
+	HeapTupleData ltup;
+	HeapTupleData rtup;
+	TupleDesc	tupDesc;
+	int			nkey;
+	int32		compare;
+
+	/* Allow interrupting long sorts */
+	CHECK_FOR_INTERRUPTS();
+	/* Compare the leading sort key */
+	compare = inlineApplySortFunction(&scanKey->sk_func, scanKey->sk_flags,
+									  scanKey->sk_collation,
+									  a->datum1, a->isnull1,
+									  b->datum1, b->isnull1);
+	if (compare != 0)
+		return compare;
+
+
+	/* Compare additional sort keys */
+	ltup.t_len = ((MinimalTuple) a->tuple)->t_len + MINIMAL_TUPLE_OFFSET;
+	ltup.t_data = (HeapTupleHeader) ((char *) a->tuple - MINIMAL_TUPLE_OFFSET);
+	rtup.t_len = ((MinimalTuple) b->tuple)->t_len + MINIMAL_TUPLE_OFFSET;
+	rtup.t_data = (HeapTupleHeader) ((char *) b->tuple - MINIMAL_TUPLE_OFFSET);
+	tupDesc = state->tupDesc;
+	scanKey++;
+
+	for (nkey = 1; nkey < state->nKeys; nkey++, scanKey++)
+	{
+		AttrNumber	attno = scanKey->sk_attno;
+		Datum		datum1,
+					datum2;
+		bool		isnull1,
+					isnull2;
+
+		datum1 = heap_getattr(&ltup, attno, tupDesc, &isnull1);
+		datum2 = heap_getattr(&rtup, attno, tupDesc, &isnull2);
+
+		compare = compare_int4(datum1, datum2);
+		return compare;
+	}
+
+	return 0;
+}
+
+
+inline static char *med3(char *a, char *b, char *c,
+	 void *arg);
+inline static void swapfunc(char *, char *, size_t, int);
+
+/*
+ * Qsort routine based on J. L. Bentley and M. D. McIlroy,
+ * "Engineering a sort function",
+ * Software--Practice and Experience 23 (1993) 1249-1265.
+ * We have modified their original by adding a check for already-sorted input,
+ * which seems to be a win per discussions on pgsql-hackers around 2006-03-21.
+ */
+#define swapcode(TYPE, parmi, parmj, n) \
+do {		\
+	size_t i = (n) / sizeof (TYPE);			\
+	TYPE *pi = (TYPE *)(void *)(parmi);			\
+	TYPE *pj = (TYPE *)(void *)(parmj);			\
+	do {						\
+		TYPE	t = *pi;			\
+		*pi++ = *pj;				\
+		*pj++ = t;				\
+		} while (--i > 0);				\
+} while (0)
+
+#define SWAPINIT(a, es) swaptype = ((char *)(a) - (char *)0) % sizeof(long) || \
+	(es) % sizeof(long) ? 2 : (es) == sizeof(long)? 0 : 1;
+
+inline static void
+swapfunc(char *a, char *b, size_t n, int swaptype)
+{
+	if (swaptype <= 1)
+		swapcode(long, a, b, n);
+	else
+		swapcode(char, a, b, n);
+}
+
+#define swap(a, b)						\
+	if (swaptype == 0) {					\
+		long t = *(long *)(void *)(a);			\
+		*(long *)(void *)(a) = *(long *)(void *)(b);	\
+		*(long *)(void *)(b) = t;			\
+	} else							\
+		swapfunc(a, b, es, swaptype)
+
+#define vecswap(a, b, n) if ((n) > 0) swapfunc((a), (b), (size_t)(n), swaptype)
+
+inline static char *
+med3(char *a, char *b, char *c, void *arg)
+{
+	return comparetup_heap_int4(a, b, arg) < 0 ?
+		(comparetup_heap_int4(b, c, arg) < 0 ? b : (comparetup_heap_int4(a, c, arg) < 0 ? c : a))
+		: (comparetup_heap_int4(b, c, arg) > 0 ? b : (comparetup_heap_int4(a, c, arg) < 0 ? a : c));
+}
+
+
+
+void
+qsort_arg_int4(void *a, size_t n, size_t es, void *arg)
+{
+	char	   *pa,
+			   *pb,
+			   *pc,
+			   *pd,
+			   *pl,
+			   *pm,
+			   *pn;
+	int			d,
+				r,
+				swaptype,
+				presorted;
+
+loop:SWAPINIT(a, es);
+	if (n < 7)
+	{
+		for (pm = (char *) a + es; pm < (char *) a + n * es; pm += es)
+			for (pl = pm; pl > (char *) a && comparetup_heap_int4(pl - es, pl, arg) > 0;
+				 pl -= es)
+				swap(pl, pl - es);
+		return;
+	}
+	presorted = 1;
+	for (pm = (char *) a + es; pm < (char *) a + n * es; pm += es)
+	{
+		if (comparetup_heap_int4(pm - es, pm, arg) > 0)
+		{
+			presorted = 0;
+			break;
+		}
+	}
+	if (presorted)
+		return;
+	pm = (char *) a + (n / 2) * es;
+	if (n > 7)
+	{
+		pl = (char *) a;
+		pn = (char *) a + (n - 1) * es;
+		if (n > 40)
+		{
+			d = (n / 8) * es;
+			pl = med3(pl, pl + d, pl + 2 * d, arg);
+			pm = med3(pm - d, pm, pm + d, arg);
+			pn = med3(pn - 2 * d, pn - d, pn, arg);
+		}
+		pm = med3(pl, pm, pn, arg);
+	}
+	swap(a, pm);
+	pa = pb = (char *) a + es;
+	pc = pd = (char *) a + (n - 1) * es;
+	for (;;)
+	{
+		while (pb <= pc && (r = comparetup_heap_int4(pb, a, arg)) <= 0)
+		{
+			if (r == 0)
+			{
+				swap(pa, pb);
+				pa += es;
+			}
+			pb += es;
+		}
+		while (pb <= pc && (r = comparetup_heap_int4(pc, a, arg)) >= 0)
+		{
+			if (r == 0)
+			{
+				swap(pc, pd);
+				pd -= es;
+			}
+			pc -= es;
+
+		}
+		if (pb > pc)
+			break;
+		swap(pb, pc);
+		pb += es;
+		pc -= es;
+	}
+	pn = (char *) a + n * es;
+	r = Min(pa - (char *) a, pb - pa);
+	vecswap(a, pb - r, r);
+	r = Min(pd - pc, pn - pd - es);
+	vecswap(pb, pn - r, r);
+	if ((r = pb - pa) > es)
+		qsort_arg_int4(a, r / es, es, arg);
+	if ((r = pd - pc) > es)
+	{
+		/* Iterate rather than recurse to save stack space */
+		a = pn - r;
+		n = r / es;
+		goto loop;
+	}
+}
+
 /*
  * All tuples have been provided; finish the sort.
  */
@@ -1247,11 +1526,14 @@ tuplesort_performsort(Tuplesortstate *state)
 			 * amount of memory.  Just qsort 'em and we're done.
 			 */
 			if (state->memtupcount > 1)
-				qsort_arg((void *) state->memtuples,
+			{
+				/* For this POC patch, pretend we only ever sort int4 */
+						qsort_arg_int4((void *) state->memtuples,
 						  state->memtupcount,
 						  sizeof(SortTuple),
-						  (qsort_arg_comparator) state->comparetup,
 						  (void *) state);
+
+			}
 			state->current = 0;
 			state->eof_reached = false;
 			state->markpos_offset = 0;
@@ -2628,72 +2910,6 @@ SelectSortFunction(Oid sortOperator,
 }
 
 /*
- * Inline-able copy of FunctionCall2Coll() to save some cycles in sorting.
- */
-static inline Datum
-myFunctionCall2Coll(FmgrInfo *flinfo, Oid collation, Datum arg1, Datum arg2)
-{
-	FunctionCallInfoData fcinfo;
-	Datum		result;
-
-	InitFunctionCallInfoData(fcinfo, flinfo, 2, collation, NULL, NULL);
-
-	fcinfo.arg[0] = arg1;
-	fcinfo.arg[1] = arg2;
-	fcinfo.argnull[0] = false;
-	fcinfo.argnull[1] = false;
-
-	result = FunctionCallInvoke(&fcinfo);
-
-	/* Check for null result, since caller is clearly not expecting one */
-	if (fcinfo.isnull)
-		elog(ERROR, "function %u returned NULL", fcinfo.flinfo->fn_oid);
-
-	return result;
-}
-
-/*
- * Apply a sort function (by now converted to fmgr lookup form)
- * and return a 3-way comparison result.  This takes care of handling
- * reverse-sort and NULLs-ordering properly.  We assume that DESC and
- * NULLS_FIRST options are encoded in sk_flags the same way btree does it.
- */
-static inline int32
-inlineApplySortFunction(FmgrInfo *sortFunction, int sk_flags, Oid collation,
-						Datum datum1, bool isNull1,
-						Datum datum2, bool isNull2)
-{
-	int32		compare;
-
-	if (isNull1)
-	{
-		if (isNull2)
-			compare = 0;		/* NULL "=" NULL */
-		else if (sk_flags & SK_BT_NULLS_FIRST)
-			compare = -1;		/* NULL "<" NOT_NULL */
-		else
-			compare = 1;		/* NULL ">" NOT_NULL */
-	}
-	else if (isNull2)
-	{
-		if (sk_flags & SK_BT_NULLS_FIRST)
-			compare = 1;		/* NOT_NULL ">" NULL */
-		else
-			compare = -1;		/* NOT_NULL "<" NULL */
-	}
-	else
-	{
-		compare = DatumGetInt32(myFunctionCall2Coll(sortFunction, collation,
-													datum1, datum2));
-
-		if (sk_flags & SK_BT_DESC)
-			compare = -compare;
-	}
-
-	return compare;
-}
-
-/*
  * Non-inline ApplySortFunction() --- this is needed only to conform to
  * C99's brain-dead notions about how to implement inline functions...
  */
#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Peter Geoghegan (#1)
Re: Inlining comparators as a performance optimisation

Peter Geoghegan <peter@2ndquadrant.com> writes:

Once the cache has been warmed, explain analyze very consistently
reports a runtime of 123ms for this query on master/HEAD, which varies
+/- 1 ms, with a few outliers of maybe +/- 2ms. However, when I apply
this patch, that goes down to 107ms +/- 1ms at -O0. I think that
that's a pretty good start. Funnily enough, the difference/advantage
vanishes at -O2 (I'm guessing that the higher optimisation level of
GCC 4.5 hyper-corrects away the inlining, but I don't have time to
check that right now).

Considering that -O2 is our standard optimization level, that
observation seems to translate to "this patch will be useless in
practice". I think you had better investigate that aspect in some
detail before spending more effort.

This performance patch differs from most in that it's difficult in
principle to imagine a performance regression occurring.

Really? N copies of the same code could lead to performance loss just
due to code bloat (ie, less of a query's inner loops fitting in CPU
cache). Not to mention the clear regression in maintainability. So
I'm disinclined to consider this sort of change without a significantly
bigger win than you're suggesting above (no, I don't even consider the
-O0 number attractive, let alone what you're finding at -O2).

regards, tom lane

#3Amit Kapila
amit.kapila@huawei.com
In reply to: Peter Geoghegan (#1)
Re: Inlining comparators as a performance optimisation

Recent discussions on the threads "Double sorting split patch" and

"CUDA sorting" raised the possibility that there could be significant

performance optimisation "low-hanging fruit" picked by having the

executor treat integers and floats as a special case during sorting,

avoiding going to the trouble of calling a comparator using the

built-in SQL function machinery

Why only for integers and floats why not for char/varchar?

But I believe this can make code less maintainable as similar things can be
done at other places to avoid SQL function machinery.

Once the cache has been warmed, explain analyze very consistently

reports a runtime of 123ms for this query on master/HEAD, which varies

+/- 1 ms, with a few outliers of maybe +/- 2ms. However, when I apply

this patch, that goes down to 107ms +/- 1ms at -O0.

Time 123ms which is without your change is with which optimization -O2 or
O0?

****************************************************************************
***********

This e-mail and attachments contain confidential information from HUAWEI,
which is intended only for the person or entity whose address is listed
above. Any use of the information contained herein in any way (including,
but not limited to, total or partial disclosure, reproduction, or
dissemination) by persons other than the intended recipient's) is
prohibited. If you receive this e-mail in error, please notify the sender by
phone or email immediately and delete it!

-----Original Message-----
From: pgsql-hackers-owner@postgresql.org
[mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Peter Geoghegan
Sent: Tuesday, September 20, 2011 7:26 AM
To: PG Hackers
Subject: [HACKERS] Inlining comparators as a performance optimisation

Recent discussions on the threads "Double sorting split patch" and

"CUDA sorting" raised the possibility that there could be significant

performance optimisation "low-hanging fruit" picked by having the

executor treat integers and floats as a special case during sorting,

avoiding going to the trouble of calling a comparator using the

built-in SQL function machinery, and taking advantage of inlining of

the comparator, which has been shown to have a considerable

performance advantage (at least compared to a general purpose c stdlib

qsort(), that takes a function pointer as its comparator, much like

tuplesort).

I've hacked together a sloppy POC implementation in a hurry

(basically, some code is shifted around) , which is attached - I felt

that it would be useful to determine if the community feels that this

is a worth-while undertaking in advance of a business trip that I'm

leaving on tomorrow lasting until Friday, during which I will be

mostly unavailable. The patch breaks the Postgres sorting executor

node (at least when it quicksorts) for any type other than int4. I

apologise for how rough the patch is, but the code itself isn't

important right now - the idea is. I anticipate that the value

state->datumType or something similar will be set in a near future

revision, so that tuplesort_performsort will know which type-specific

optimisation it can use for the type, while falling back on the

existing generic qsort_arg + qsort_arg_comparator, and sorting won't

actually be broken.

I've been doing some preliminary testing using the dell store 2 sample

database. I increase work_mem to '50MB', to ensure that a quicksort

will be performed for sorting (otherwise, I'm using the

postgresql.conf that initdb gave me). The query is:

explain analyze select * from orderlines order by prod_id;

Once the cache has been warmed, explain analyze very consistently

reports a runtime of 123ms for this query on master/HEAD, which varies

+/- 1 ms, with a few outliers of maybe +/- 2ms. However, when I apply

this patch, that goes down to 107ms +/- 1ms at -O0. I think that

that's a pretty good start. Funnily enough, the difference/advantage

vanishes at -O2 (I'm guessing that the higher optimisation level of

GCC 4.5 hyper-corrects away the inlining, but I don't have time to

check that right now).

I imagine the version that I actually submit for patch review will

have a macro-based infrastructure for inlining the sorting of various

built-in types, initially integers and floats. It will most likely

have some other optimisations - I haven't even used a profiler yet.

This performance patch differs from most in that it's difficult in

principle to imagine a performance regression occurring.

Thoughts?

--

Peter Geoghegan http://www.2ndQuadrant.com/

PostgreSQL Development, 24x7 Support, Training and Services

#4Peter Geoghegan
peter@2ndquadrant.com
In reply to: Tom Lane (#2)
1 attachment(s)
Re: Inlining comparators as a performance optimisation

On 20 September 2011 03:51, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Considering that -O2 is our standard optimization level, that
observation seems to translate to "this patch will be useless in
practice".  I think you had better investigate that aspect in some
detail before spending more effort.

I don't think that the fact that that happens is at all significant at
this early stage, and it never even occurred to me that you'd think
that it might be. I was simply disclosing a quirk of this POC patch.
The workaround is probably to use a macro instead. For the benefit of
those that didn't follow the other threads, the macro-based qsort
implementation, which I found to perform significantly better than
regular qsort(), runs like this on my laptop when I built at 02 with
GCC 4.6 just now:

C stdlib quick-sort time elapsed: 2.092451 seconds
Inline quick-sort time elapsed: 1.587651 seconds

Does *that* look attractive to you? I've attached source code of the
program that produced these figures, which has been ported to C from
C++.

When I #define LARGE_SIZE 100000000, here's what I see:

[peter@peter inline_compar_test]$ ./a.out
C stdlib quick-sort time elapsed: 23.659411 seconds
Inline quick-sort time elapsed: 18.470611 seconds

Here, sorting with the function pointer/stdlib version takes about
1.28 times as long. In the prior test (with the smaller LARGE_SIZE),
it took about 1.32 times as long. Fairly predictable, linear, and not
to be sniffed at.

The variance I'm seeing across runs is low - a couple of hundredths of
a second at most. This is a Fedora 15 " Intel(R) Core(TM) i5-2540M CPU
@ 2.60GHz" machine. I'm not sure right now why the inline quick-sort
is less of a win than on my old Fedora 14 desktop (where it was 3.24
Vs 2.01), but it's still a significant win. Perhaps others can build
this simple program and tell me what they come up with.

This performance patch differs from most in that it's difficult in
principle to imagine a performance regression occurring.

Really?  N copies of the same code could lead to performance loss just
due to code bloat (ie, less of a query's inner loops fitting in CPU
cache).

I did consider that. Of course inlining has an overhead, and I'll be
testing that each instance of inlining has a net benefit. I just meant
that many other performance patches have an obvious worst case, and I
think that it is policy to focus on that case, but I can't think of
one here. Does anyone else have any ideas?

Not to mention the clear regression in maintainability.  So
I'm disinclined to consider this sort of change without a significantly
bigger win than you're suggesting above

Sure, there'll be some sort of regression in maintainability - I think
that HOT had a clear regression in maintainability too. The important
questions are obviously "how big is the loss of maintainability?", and
"is it worth it?". We'll know more when this work is actually shaped
into a proper patch. Perhaps I should have waited until I had
something along those lines before making an announcement, but I
wanted community input as early as possible. I think that there's
plenty of tweaking that can be done to get additional performance
improvements - all I've done so far is demonstrate that those
improvements are real and worth thinking about, in the fastest
possible way, partly because you expressed skepticism of the benefits
of inlining comparators to Greg Stark in an earlier thread.

Performance and maintainability are often somewhat in tension, but we
cannot ignore performance. If this work can bring us an improvement in
performance approaching the isolated macro Vs qsort() function pointer
benchmark, that's a *big* win. Sorting integers and floats is very
common and important.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

Attachments:

inline_compar_test.tar.gzapplication/x-gzip; name=inline_compar_test.tar.gzDownload
#5Noname
karavelov@mail.bg
In reply to: Peter Geoghegan (#4)
Re: Inlining comparators as a performance optimisation

----- Цитат от Peter Geoghegan (peter@2ndquadrant.com), на 21.09.2011 в 02:53 -----

On 20 September 2011 03:51, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Considering that -O2 is our standard optimization level, that
observation seems to translate to "this patch will be useless in
practice".  I think you had better investigate that aspect in some
detail before spending more effort.

I don't think that the fact that that happens is at all significant at
this early stage, and it never even occurred to me that you'd think
that it might be. I was simply disclosing a quirk of this POC patch.
The workaround is probably to use a macro instead. For the benefit of
those that didn't follow the other threads, the macro-based qsort
implementation, which I found to perform significantly better than
regular qsort(), runs like this on my laptop when I built at 02 with
GCC 4.6 just now:

C stdlib quick-sort time elapsed: 2.092451 seconds
Inline quick-sort time elapsed: 1.587651 seconds

Does *that* look attractive to you? I've attached source code of the
program that produced these figures, which has been ported to C from
C++.

When I #define LARGE_SIZE 100000000, here's what I see:

[peter@peter inline_compar_test]$ ./a.out
C stdlib quick-sort time elapsed: 23.659411 seconds
Inline quick-sort time elapsed: 18.470611 seconds

Here, sorting with the function pointer/stdlib version takes about
1.28 times as long. In the prior test (with the smaller LARGE_SIZE),
it took about 1.32 times as long. Fairly predictable, linear, and not
to be sniffed at.

The variance I'm seeing across runs is low - a couple of hundredths of
a second at most. This is a Fedora 15 " Intel(R) Core(TM) i5-2540M CPU
@ 2.60GHz" machine. I'm not sure right now why the inline quick-sort
is less of a win than on my old Fedora 14 desktop (where it was 3.24
Vs 2.01), but it's still a significant win. Perhaps others can build
this simple program and tell me what they come up with.

Run it here.

Intel(R) Core(TM)2 Duo CPU E8200 @ 2.66GHz
gcc version 4.6.1 (Debian 4.6.1-10)

g++ -O2 qsort-inline-benchmark.c
./a.out
C stdlib quick-sort time elapsed: 1.942686 seconds
Inline quick-sort time elapsed: 1.126508 seconds

With #define LARGE_SIZE 100000000

C stdlib quick-sort time elapsed: 22.158207 seconds
Inline quick-sort time elapsed: 12.861018 seconds

with g++ -O0
C stdlib quick-sort time elapsed: 2.736360 seconds
Inline quick-sort time elapsed: 2.045619 seconds

On server hardware:
Intel(R) Xeon(R) CPU E5405 @ 2.00GHz
gcc version 4.4.5 (Debian 4.4.5-8)

/a.out
C stdlib quick-sort time elapsed: 2.610150 seconds
Inline quick-sort time elapsed: 1.494198 seconds

All -O2 version show 42% speedup with inlined qsort.
-O0 showed 25% speedup.

Best regards

--
Luben Karavelov

#6Dan McGee
dan@archlinux.org
In reply to: Peter Geoghegan (#4)
[PATCH] POC: inline int4 comparison in tuplesort

This attempts to be as simple as it gets while reducing function call
depth, and should be viewed as a proof of concept. It is also untested
as of now, but will try to do that and report back.

I'm hoping I followed the rabbit hole correctly and are correctly
comparing the right pointers to each other in order to short circuit the
case where we are using the int4 comparison operator.

Peter, if you want to compare stock vs. your patch vs. this patch, we might
be able to get some sort of read on where the maintainablity vs. performance
curve lies. Note that this version should still allow sorting of anything,
and simply shifts gears for int4 tuples...

---
src/backend/utils/sort/tuplesort.c | 23 +++++++++++++++++++++--
1 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 3505236..ddd5ced 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2652,6 +2652,22 @@ myFunctionCall2Coll(FmgrInfo *flinfo, Oid collation, Datum arg1, Datum arg2)
 	return result;
 }
+static inline
+int int4cmp(Datum first, Datum second)
+{
+	int32		a = DatumGetInt32(first);
+	int32		b = DatumGetInt32(second);
+
+	if (a > b)
+		return 1;
+	else if (a == b)
+		return 0;
+	else
+		return -1;
+}
+
+extern Datum btint4cmp(PG_FUNCTION_ARGS);
+
 /*
  * Apply a sort function (by now converted to fmgr lookup form)
  * and return a 3-way comparison result.  This takes care of handling
@@ -2683,8 +2699,11 @@ inlineApplySortFunction(FmgrInfo *sortFunction, int sk_flags, Oid collation,
 	}
 	else
 	{
-		compare = DatumGetInt32(myFunctionCall2Coll(sortFunction, collation,
-													datum1, datum2));
+		if (sortFunction->fn_addr == btint4cmp)
+			compare = int4cmp(datum1, datum2);
+		else
+			compare = DatumGetInt32(myFunctionCall2Coll(sortFunction, collation,
+														datum1, datum2));

if (sk_flags & SK_BT_DESC)
compare = -compare;
--
1.7.6.3

#7Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Peter Geoghegan (#4)
Re: Inlining comparators as a performance optimisation

On 21.09.2011 02:53, Peter Geoghegan wrote:

C stdlib quick-sort time elapsed: 2.092451 seconds
Inline quick-sort time elapsed: 1.587651 seconds

Does *that* look attractive to you?

Not really, to be honest. That's a 25% speedup in pure qsorting speed.
How much of a gain in a real query do you expect to get from that, in
the best case? There's so many other sources of overhead that I'm afraid
this will be lost in the noise. If you find a query that spends, say,
50% of its time in qsort(), you will only get a 12.5% speedup on that
query. And even 50% is really pushing it - I challenge you to find a
query that spends any significant amount of time qsorting integers.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

#8Simon Riggs
simon@2ndQuadrant.com
In reply to: Tom Lane (#2)
Re: Inlining comparators as a performance optimisation

On Tue, Sep 20, 2011 at 3:51 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

This performance patch differs from most in that it's difficult in
principle to imagine a performance regression occurring.

Really?  N copies of the same code could lead to performance loss just
due to code bloat (ie, less of a query's inner loops fitting in CPU
cache).  Not to mention the clear regression in maintainability.  So
I'm disinclined to consider this sort of change without a significantly
bigger win than you're suggesting above (no, I don't even consider the
-O0 number attractive, let alone what you're finding at -O2).

More copies of the code are somewhat annoying, but its only 100 lines
of code in one module and we can easily have specific tests for each.
The extra code size is minor in comparison to the reams of code we add
elsewhere.

It's a surprisingly good win for such a common use case. Well done, Peter.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

#9Simon Riggs
simon@2ndQuadrant.com
In reply to: Heikki Linnakangas (#7)
Re: Inlining comparators as a performance optimisation

On Wed, Sep 21, 2011 at 7:51 AM, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:

On 21.09.2011 02:53, Peter Geoghegan wrote:

C stdlib quick-sort time elapsed: 2.092451 seconds
Inline quick-sort time elapsed: 1.587651 seconds

Does *that* look attractive to you?

Not really, to be honest. That's a 25% speedup in pure qsorting speed. How
much of a gain in a real query do you expect to get from that, in the best
case? There's so many other sources of overhead that I'm afraid this will be
lost in the noise. If you find a query that spends, say, 50% of its time in
qsort(), you will only get a 12.5% speedup on that query. And even 50% is
really pushing it - I challenge you to find a query that spends any
significant amount of time qsorting integers.

How about almost every primary index creation?

Don't really see a reason for the negativity here. If you use that
argument no performance gain is worth it because all workloads are
mixed.

This is a marvellous win, a huge gain from a small, isolated and
easily tested change. By far the smallest amount of additional code to
sorting we will have added and yet one of the best gains.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

#10Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Simon Riggs (#9)
Re: Inlining comparators as a performance optimisation

On 21.09.2011 10:01, Simon Riggs wrote:

On Wed, Sep 21, 2011 at 7:51 AM, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:

On 21.09.2011 02:53, Peter Geoghegan wrote:

C stdlib quick-sort time elapsed: 2.092451 seconds
Inline quick-sort time elapsed: 1.587651 seconds

Does *that* look attractive to you?

Not really, to be honest. That's a 25% speedup in pure qsorting speed. How
much of a gain in a real query do you expect to get from that, in the best
case? There's so many other sources of overhead that I'm afraid this will be
lost in the noise. If you find a query that spends, say, 50% of its time in
qsort(), you will only get a 12.5% speedup on that query. And even 50% is
really pushing it - I challenge you to find a query that spends any
significant amount of time qsorting integers.

How about almost every primary index creation?

Nope. Swamped by everything else.

Also note that as soon as the sort grows big enough to not fit in
maintenance_work_mem, you switch to the external sort algorithm, which
doesn't use qsort. Perhaps you could do similar inlining in the heap
sort & merge passes done in the external sort, but it's unlikely to be
as big a win there.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

#11Peter Geoghegan
peter@2ndquadrant.com
In reply to: Noname (#5)
Re: Inlining comparators as a performance optimisation

On 21 September 2011 01:48, <karavelov@mail.bg> wrote:

All -O2 version show 42% speedup with inlined qsort.
-O0 showed 25% speedup.

Thanks. Looks like the figures I posted last night were fairly
conservative. Does anyone else care to report results?

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

#12Greg Stark
stark@mit.edu
In reply to: Heikki Linnakangas (#10)
Re: Inlining comparators as a performance optimisation

On Wed, Sep 21, 2011 at 8:08 AM, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:

How about almost every primary index creation?

Nope. Swamped by everything else.

Really? I think it's pretty common for shops to be able to dedicate
large amounts of RAM to building initial indexes on data loads or
reindex operations. Enough that they can cache the entire table for
the short time they're doing the index builds even if they're quite
large. Witness the recent pleas to allow maintenance_work_mem on the
order of tens of gigabytes. And it's also pretty common that shops can
dedicate very large I/O bandwidth, in many cases enough to saturate
the memory bandwidth, for doing these kinds of batch operations when
they get large enough to need to do an external sort.

There's still overhead of reading the pages, the tuples, finding the
sort keys in the tuple, etc. But I think the actual qsort or heap
operations in tapesort are pretty big portions of the work.

This is pretty easy to measure. Just run oprofile or gprof and see
what percentage of time for a big index build is spent in qsort.

--
greg

#13Robert Haas
robertmhaas@gmail.com
In reply to: Greg Stark (#12)
Re: Inlining comparators as a performance optimisation

On Wed, Sep 21, 2011 at 8:47 AM, Greg Stark <stark@mit.edu> wrote:

On Wed, Sep 21, 2011 at 8:08 AM, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:

How about almost every primary index creation?

Nope. Swamped by everything else.

Really? I think it's pretty common for shops to be able to dedicate
large amounts of RAM to building initial indexes on data loads or
reindex operations. Enough that they can cache the entire table for
the short time they're doing the index builds even if they're quite
large. Witness the recent pleas to allow maintenance_work_mem on the
order of tens of gigabytes. And it's also pretty common that shops can
dedicate very large I/O bandwidth, in many cases enough to saturate
the memory bandwidth, for doing these kinds of batch operations when
they get large enough to need to do an external sort.

There's still overhead of reading the pages, the tuples, finding the
sort keys in the tuple, etc. But I think the actual qsort or heap
operations in tapesort are pretty big portions of the work.

This is pretty easy to measure. Just run oprofile or gprof and see
what percentage of time for a big index build is spent in qsort.

+1 for some actual measurements.

I don't think anyone on this thread is saying that if we can get big
performance gains from doing this we still shouldn't do it. But at
this point it's unclear that we can get a consistent speedup that
isn't heavily dependent on the choice of compiler flags (to say
nothing of compiler and OS), and even if we can, it's not clear that
it will still be noticeable when you measure the run time of an entire
query rather than just the speed of qsort(). Like Tom and Heikki, I'm
a bit skeptical: it wouldn't surprise me to find out that qsort() is
5% of the runtime any realistic test case and the average qsort()
speedup based on tests on a couple different platforms is 10% and so
on average we're looking at a 0.5% improvement, in which case it might
be more trouble than it's worth, especially if it turns out that there
are OS/platform combinations where the inlined version is (for some
crazy reason) slower. I've seen performance differences of up to 3%
from minor code rearrangements that don't seem like they should matter
at all, just because code and data shifts around across cache-line
boundaries and the new arrangement is slightly better or worse than
the old one. So if the performance improvement turns out to be very
small, then validating that it actually IS an improvement in general
is likely to be kind of a pain in the ass.

On the other hand, the performance improvement might turn out to be
large. Maybe there's a test case where, as Heikki suggests, 50% of
the time is spent in qsort(). If we can reliably make that 25%
faster, I wouldn't dismiss that out of hand; I think that would be
pretty good, assuming it didn't require massive amounts of spaghetti
code to make it work. I don't see that that would be any more
marginal than the sorts of things we've optimized in, say, commit
4fc115b2e981f8c63165ca86a23215380a3fda66, or commit
f4d242ef94730c447d87b9840a40b0ec3371fe0f.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#14Peter Geoghegan
peter@2ndquadrant.com
In reply to: Heikki Linnakangas (#7)
Re: Inlining comparators as a performance optimisation

On 21 September 2011 07:51, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:

On 21.09.2011 02:53, Peter Geoghegan wrote:

C stdlib quick-sort time elapsed: 2.092451 seconds
Inline quick-sort time elapsed: 1.587651 seconds

Does *that* look attractive to you?

Not really, to be honest. That's a 25% speedup in pure qsorting speed. How
much of a gain in a real query do you expect to get from that, in the best
case? There's so many other sources of overhead that I'm afraid this will be
lost in the noise.

I'm surprised that you're dismissive of this. After all, we have in
the past indulged in micro-optimisation of qsort, or so it would seem
from this comment:

* We have modified their original by adding a check for already-sorted input,
* which seems to be a win per discussions on pgsql-hackers around 2006-03-21.

"Makes affected queries radically faster" (In the best case, a speedup
somewhat greater than 12.5%) is an unreasonably high standard for a
performance optimisation of the executor in general (such a high
standard might be sensible if it was due to a particular
maintainability downside, but you didn't mention one). Even still, I
think that the 12.5% figure is pretty pessimistic - I've already sped
up the dell store query by almost that much, and that's with a patch
that was, due to circumstances, cobbled together.

Not only are we benefiting from the effects of inlining, we're also
benefiting from the removal of unnecessary indirection. As Tom said,
"In concrete terms, there would be no reason to have tuplesort.c's
myFunctionCall2Coll, and maybe not inlineApplySortFunction either, if
the datatype-specific comparison functions had APIs that were closer
to what sorting wants rather than following the general
SQL-callable-function API." He was just referring to the benefits of
removing indirection here, so ISTM that this is really two performance
optimisations rolled into one - it's conceivable that the total
performance improvement will even exceed the isolated inlining
comparator benchmark.

As I've said, I believe this patch can be committed without
compromising the maintainability of the tuplesort code to an extent
that is not clearly worth it, through the use of a clean, macro-based
abstraction. Concerns about bloated binaries are probably not well
founded, because what I'm proposing is to a certain extent emulating
C++ templates, while using a very common pattern used with C++
templates. In the C++ world, algorithms are often generalised as
templates, so that they can be used equally well with any datatype
(that supports the interface of the template), while availing of
compiler optimisations per template instantiation (instance of using a
given type with a given template). I actually got the idea for this
patch in part from a book that I read years ago that described the
fact that counter-intuitively, std::sort() consistently outperforms
qsort(), because the comparator is often inlined, and the compiler can
generally avail of optimisations from knowing the comparator at
compile-time.

On 21 September 2011 13:47, Greg Stark <stark@mit.edu> wrote:

This is pretty easy to measure. Just run oprofile or gprof and see
what percentage of time for a big index build is spent in qsort.

I'll do so soon. I intend to get to this on Friday evening, and
perhaps have a proper patch to show next week.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

#15Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Peter Geoghegan (#14)
Re: Inlining comparators as a performance optimisation

On 21.09.2011 17:20, Peter Geoghegan wrote:

Even still, I
think that the 12.5% figure is pretty pessimistic - I've already sped
up the dell store query by almost that much, and that's with a patch
that was, due to circumstances, cobbled together.

I'm not against making things faster, it's just that I haven't seen
solid evidence yet that this will help. Just provide a best-case test
case for this that shows a huge improvement, and I'll shut up. If the
improvement is only modest, then let's discuss how big it is and whether
it's worth the code ugliness this causes.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

#16Tom Lane
tgl@sss.pgh.pa.us
In reply to: Heikki Linnakangas (#15)
Re: Inlining comparators as a performance optimisation

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:

On 21.09.2011 17:20, Peter Geoghegan wrote:

Even still, I
think that the 12.5% figure is pretty pessimistic - I've already sped
up the dell store query by almost that much, and that's with a patch
that was, due to circumstances, cobbled together.

I'm not against making things faster, it's just that I haven't seen
solid evidence yet that this will help. Just provide a best-case test
case for this that shows a huge improvement, and I'll shut up. If the
improvement is only modest, then let's discuss how big it is and whether
it's worth the code ugliness this causes.

The other question that I'm going to be asking is whether it's not
possible to get most of the same improvement with a much smaller code
footprint. I continue to suspect that getting rid of the SQL function
impedance-match layer (myFunctionCall2Coll etc) would provide most of
whatever gain is to be had here, without nearly as large a cost in code
size and maintainability, and with the extra benefit that the speedup
would also be available to non-core datatypes.

regards, tom lane

#17Andrew Dunstan
andrew@dunslane.net
In reply to: Tom Lane (#16)
Re: Inlining comparators as a performance optimisation

On 09/21/2011 10:50 AM, Tom Lane wrote:

The other question that I'm going to be asking is whether it's not
possible to get most of the same improvement with a much smaller code
footprint. I continue to suspect that getting rid of the SQL function
impedance-match layer (myFunctionCall2Coll etc) would provide most of
whatever gain is to be had here, without nearly as large a cost in code
size and maintainability, and with the extra benefit that the speedup
would also be available to non-core datatypes.

Can we get a patch so we can do benchmarks on this?

cheers

andrew

#18Tom Lane
tgl@sss.pgh.pa.us
In reply to: Simon Riggs (#9)
Re: Inlining comparators as a performance optimisation

Simon Riggs <simon@2ndQuadrant.com> writes:

This is a marvellous win, a huge gain from a small, isolated and
easily tested change. By far the smallest amount of additional code to
sorting we will have added and yet one of the best gains.

I think you forgot your cheerleader uniform. A patch along these lines
is not going to be small, isolated, easily maintained, nor beneficial
for any but a small number of predetermined datatypes.

regards, tom lane

#19Peter Geoghegan
peter@2ndquadrant.com
In reply to: Tom Lane (#16)
Re: Inlining comparators as a performance optimisation

On 21 September 2011 15:50, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:

I'm not against making things faster, it's just that I haven't seen
solid evidence yet that this will help. Just provide a best-case test
case for this that shows a huge improvement, and I'll shut up. If the
improvement is only modest, then let's discuss how big it is and whether
it's worth the code ugliness this causes.

Fair enough.

The other question that I'm going to be asking is whether it's not
possible to get most of the same improvement with a much smaller code
footprint.

That's a reasonable question, and I hope to be able to come up with a
good answer.

I continue to suspect that getting rid of the SQL function
impedance-match layer (myFunctionCall2Coll etc) would provide most of
whatever gain is to be had here, without nearly as large a cost in code
size and maintainability, and with the extra benefit that the speedup
would also be available to non-core datatypes.

I'm fairly surprised that your view on that is mostly or entirely
unchanged, even after I've demonstrated a considerable performance
advantage from a macro-based qsort implementation over my OS vendor's
c std lib qsort(), using an isolated test-case, that does not have
anything to do with that impedance mismatch. I'm not sure why you
doubt that the same thing is happening within tuplesort.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

#20Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andrew Dunstan (#17)
Re: Inlining comparators as a performance optimisation

Andrew Dunstan <andrew@dunslane.net> writes:

On 09/21/2011 10:50 AM, Tom Lane wrote:

The other question that I'm going to be asking is whether it's not
possible to get most of the same improvement with a much smaller code
footprint. I continue to suspect that getting rid of the SQL function
impedance-match layer (myFunctionCall2Coll etc) would provide most of
whatever gain is to be had here, without nearly as large a cost in code
size and maintainability, and with the extra benefit that the speedup
would also be available to non-core datatypes.

Can we get a patch so we can do benchmarks on this?

Well, we'd have to negotiate what the API ought to be. What I'm
envisioning is that datatypes could provide alternate comparison
functions that are designed to be qsort-callable rather than
SQL-callable. As such, they could not have entries in pg_proc, so
it seems like there's no ready way to represent them in the catalogs.

The idea that I was toying with was to allow the regular SQL-callable
comparison function to somehow return a function pointer to the
alternate comparison function, so that the first comparison in a given
sort run would be done the traditional way but then we'd notice the
provided function pointer and start using that. It would not be too
hard to pass back the pointer using FunctionCallInfoData.context, say.
The downside is adding cycles to unoptimized cases to uselessly check
for a returned function pointer that's not there. Perhaps it could be
hacked so that we only add cycles to the very first call, but I've not
looked closely at the code to see what would be involved.

Has anyone got a better idea for getting hold of the alternate function?

regards, tom lane

#21Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Tom Lane (#20)
Re: Inlining comparators as a performance optimisation

On 21.09.2011 18:46, Tom Lane wrote:

Andrew Dunstan<andrew@dunslane.net> writes:

On 09/21/2011 10:50 AM, Tom Lane wrote:

The other question that I'm going to be asking is whether it's not
possible to get most of the same improvement with a much smaller code
footprint. I continue to suspect that getting rid of the SQL function
impedance-match layer (myFunctionCall2Coll etc) would provide most of
whatever gain is to be had here, without nearly as large a cost in code
size and maintainability, and with the extra benefit that the speedup
would also be available to non-core datatypes.

Can we get a patch so we can do benchmarks on this?

Well, we'd have to negotiate what the API ought to be. What I'm
envisioning is that datatypes could provide alternate comparison
functions that are designed to be qsort-callable rather than
SQL-callable. As such, they could not have entries in pg_proc, so
it seems like there's no ready way to represent them in the catalogs.

The idea that I was toying with was to allow the regular SQL-callable
comparison function to somehow return a function pointer to the
alternate comparison function, so that the first comparison in a given
sort run would be done the traditional way but then we'd notice the
provided function pointer and start using that. It would not be too
hard to pass back the pointer using FunctionCallInfoData.context, say.
The downside is adding cycles to unoptimized cases to uselessly check
for a returned function pointer that's not there. Perhaps it could be
hacked so that we only add cycles to the very first call, but I've not
looked closely at the code to see what would be involved.

You could have a new function with a pg_proc entry, that just returns a
function pointer to the qsort-callback.

Or maybe the interface should be an even more radical replacement of
qsort, not just the comparison function. Instead of calling qsort,
tuplesort.c would call the new datatype-specific sort-function (which
would be in pg_proc). The implementation could use an inlined version of
qsort, like Peter is suggesting, or it could do something completely
different, like a radix sort or a GPU-assisted sort or whatever.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

#22Greg Stark
stark@mit.edu
In reply to: Tom Lane (#20)
Re: Inlining comparators as a performance optimisation

On Wed, Sep 21, 2011 at 4:46 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

 As such, they could not have entries in pg_proc, so
it seems like there's no ready way to represent them in the catalogs.

Why couldn't they be in pg_proc with a bunch of opaque arguments like
the GIST opclass support functions?

I'm a bit puzzled what the arguments would look like. They would still
need to know the collation, nulls first/last flags, etc.
And calling it would still not be inlinable. So they would have to
check those flags on each invocation instead of having a piece of
straightline code that hard codes the behaviour with the right
behaviour inline. ISTM the hope for a speedup from the inlining
mostly came from the idea that the compiler might be able to hoist
this logic outside the loop (and I suppose implement n specialized
loops depending on the behaviour needed).

--
greg

#23Tom Lane
tgl@sss.pgh.pa.us
In reply to: Heikki Linnakangas (#21)
Re: Inlining comparators as a performance optimisation

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:

On 21.09.2011 18:46, Tom Lane wrote:

The idea that I was toying with was to allow the regular SQL-callable
comparison function to somehow return a function pointer to the
alternate comparison function,

You could have a new function with a pg_proc entry, that just returns a
function pointer to the qsort-callback.

Yeah, possibly. That would be a much more invasive change, but cleaner
in some sense. I'm not really prepared to do all the legwork involved
in that just to get to a performance-testable patch though.

Or maybe the interface should be an even more radical replacement of
qsort, not just the comparison function. Instead of calling qsort,
tuplesort.c would call the new datatype-specific sort-function (which
would be in pg_proc). The implementation could use an inlined version of
qsort, like Peter is suggesting, or it could do something completely
different, like a radix sort or a GPU-assisted sort or whatever.

No. In the first place, that only helps for in-memory sorts. In the
second, it would absolutely destroy our ability to change the behavior
of sorting ever again. Considering that we've added ASC/DESC, NULLS
FIRST/LAST, and collation support over the years, are you really
prepared to bet that the sort code will never need any more feature
upgrades? (This concern is in fact the source of my beef with the whole
inlining proposal to begin with, but allowing the inlining to occur into
third-party code that we don't control at all would be a hundred times
worse.)

regards, tom lane

#24Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Tom Lane (#20)
Re: Inlining comparators as a performance optimisation

On 21.09.2011 18:46, Tom Lane wrote:

Well, we'd have to negotiate what the API ought to be. What I'm
envisioning is that datatypes could provide alternate comparison
functions that are designed to be qsort-callable rather than
SQL-callable. As such, they could not have entries in pg_proc, so
it seems like there's no ready way to represent them in the catalogs.

Quite aside from this qsort-thing, it would be nice to have versions of
all simple functions that could be called without the FunctionCall
overhead. So instead of:

FunctionCall2(&flinfo_for_int4pl, 1, 2)

you could do simply

int4pl_fastpath(1,2)

I'm not sure how big an effect this would have, but it seems like it
could shave some cycles across the system.

We could have an extended version of the PG_FUNCTION_INFO_V1 macro that
would let you register the fastpath function:

PG_FUNCTION_INFO_V1(int4pl, int4pl_fastpath);

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

#25Tom Lane
tgl@sss.pgh.pa.us
In reply to: Greg Stark (#22)
Re: Inlining comparators as a performance optimisation

Greg Stark <stark@mit.edu> writes:

On Wed, Sep 21, 2011 at 4:46 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

�As such, they could not have entries in pg_proc, so
it seems like there's no ready way to represent them in the catalogs.

Why couldn't they be in pg_proc with a bunch of opaque arguments like
the GIST opclass support functions?

That does not mean the same thing at all. Everything in pg_proc is
meant to be called through the V0 or V1 function call info protocols.

I'm a bit puzzled what the arguments would look like. They would still
need to know the collation, nulls first/last flags, etc.

No, I wasn't thinking that we should do that. The datatype comparison
functions should have the exact same semantics they do now, just a
lower-overhead call mechanism. If you try to push stuff like NULLS
FIRST/LAST into the per-datatype code, then you are up against a problem
when you want to add a new flag: you have to touch lots of code not all
of which you even control.

And calling it would still not be inlinable. So they would have to
check those flags on each invocation instead of having a piece of
straightline code that hard codes the behaviour with the right
behaviour inline. ISTM the hope for a speedup from the inlining
mostly came from the idea that the compiler might be able to hoist
this logic outside the loop (and I suppose implement n specialized
loops depending on the behaviour needed).

None of that stuff is inlinable or constant-foldable today, nor would it
be with the patch that Peter was proposing AFAICS, because none of the
flags will ever be compile time constant values.

regards, tom lane

#26Tom Lane
tgl@sss.pgh.pa.us
In reply to: Heikki Linnakangas (#24)
Re: Inlining comparators as a performance optimisation

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:

On 21.09.2011 18:46, Tom Lane wrote:

Well, we'd have to negotiate what the API ought to be. What I'm
envisioning is that datatypes could provide alternate comparison
functions that are designed to be qsort-callable rather than
SQL-callable. As such, they could not have entries in pg_proc, so
it seems like there's no ready way to represent them in the catalogs.

Quite aside from this qsort-thing, it would be nice to have versions of
all simple functions that could be called without the FunctionCall
overhead.

Hmm, that's an interesting idea. I think probably the important aspects
are (1) known number of arguments and (2) no null argument or result
values are allowed. Not sure what we'd do with collations though.

We could have an extended version of the PG_FUNCTION_INFO_V1 macro that
would let you register the fastpath function:
PG_FUNCTION_INFO_V1(int4pl, int4pl_fastpath);

We don't use PG_FUNCTION_INFO_V1 for built-in functions ...

regards, tom lane

#27Greg Stark
stark@mit.edu
In reply to: Tom Lane (#25)
Re: Inlining comparators as a performance optimisation

On Wed, Sep 21, 2011 at 5:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

None of that stuff is inlinable or constant-foldable today, nor would it
be with the patch that Peter was proposing AFAICS, because none of the
flags will ever be compile time constant values.

I was referring to transformations like this which I believe compilers
are already capable of doing:

v = ...;
while (...)
if (v) {
if (a < b) ... else ....;
} else {
if (a > b) ... else ...;
}

turning it into code that looks like:

if (v) {
while (....)
if (a<b) ... else ...;
} else {
while (....)
if (a>b) ... else ...;
}

which may not look like much -- especially with branch prediction --
but then it's much more likely to be able to unroll the loop and do
clever instruction scheduling and so on than if there's an extra
branch in the middle of the loop. But if there's a function call to an
external function called through a function pointer in the middle of
the loop then the whole endeavour would be for naught.

--
greg

#28Simon Riggs
simon@2ndQuadrant.com
In reply to: Tom Lane (#18)
Re: Inlining comparators as a performance optimisation

On Wed, Sep 21, 2011 at 4:22 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Simon Riggs <simon@2ndQuadrant.com> writes:

This is a marvellous win, a huge gain from a small, isolated and
easily tested change. By far the smallest amount of additional code to
sorting we will have added and yet one of the best gains.

I think you forgot your cheerleader uniform.

LOL. I'm happy whoever and whenever we get large wins like that.

Go Postgres!

A patch along these lines
is not going to be small, isolated, easily maintained, nor beneficial
for any but a small number of predetermined datatypes.

That was the starting premise.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

#29Peter Geoghegan
peter@2ndquadrant.com
In reply to: Tom Lane (#25)
2 attachment(s)
Re: Inlining comparators as a performance optimisation

I've produced something much neater than my first patch, attached,
although I still consider this to be at the POC stage, not least since
I'm not exactly sure how I should be selecting the right
specialisation in tuplesort_performsort() (a hack is used right now
that does a direct pg_proc OID comparison), and also because I haven't
implemented anything other than qsort for heap tuples yet (a minor
detail, I suppose). I'm pleased to say that a much clearer picture of
what's really going on here has emerged.

Statistics: Total runtime according to explain analyze for query
"select * from orderlines order by prod_id" (dellstore2 sample db), at
GCC 4.5's -02 optimisation level, after warming the cache, on my
desktop:

Without the patch:

~82ms

With the patch, but with the "inline" keyword commented out for all
new functions/meta-functions:

~60ms

with the patch, unmodified:

~52ms

Recent experience suggests that using a macro rather than an inline
function would perhaps have been a mistake, as it would prevent us
from benefiting from the compiler's smarts in surmising just where we
should inline. As I've pointed out, individual call sites are inlined,
not individual functions.

Tom wanted to know if I could prove the benefit in inlining, as
against just resolving the impedance mismatch without bothering with
inlining (and, indeed, further optimisations that the compiler can
avail of as a result of knowing the comparator at compile-time). These
figures support my contention that these optimisations have an
important role to play. However, there's no question that resolving
the impedance mismatch is where the biggest benefit is seen,
particularly relative to maintainability. As Tom anticipated, the
question of whether or not we do the inlining is less than entirely
straightforward, as it's less of a win, and it has a higher price in
ugliness. I myself am very much of the opinion that it's still worth
it. As I've said, sorting integers and floats is very common and very
important, and this approach will likely yield benefits beyond
increasing the speed of executor sort nodes, as described below.

I accept that a more sophisticated benchmark is required here, but
I've been a little busy actually writing the patch. Any ideas anyone?
Independent benchmarks are welcome. If someone could suggest a worst
case, that would be particularly useful, as I cannot think of one - I
believe that each instance of inlining a call site more-or-less either
is or is not a net benefit here, regardless of the number of
comparisons performed, which is why the improvement is so predictable
across the size of the set of integers sorted for by qsort in
isolation (which is the big reason why that ~8ms decrease due to
inlining turns out to be not too shabby - it's a saving per
comparison, and they can add it pretty quickly). So, for example, when
I build the patch on my laptop (GCC 4.6), with an 48MB table (the same
orderlines table query, but I doubled up the data a few times
beforehand), we see an undiminished, proportional (to the prior,
isolated cost of qsorting, I think) decrease in runtime:

Without patch:

~615ms

With patch:

~415ms

One key insight that this work brings is that resolving the impedance
mismatch - making comparisons as inexpensive as possible (including
using inlining) - is the best possible strategy in improving sort
performance today, vastly more efficacious than, for example, tweaking
the sorting algorithm. If anyone can find some more ways of shaving
cycles there, it's one place where it really matters.

Changes are isolated to the extent that if you decide that you don't
like this one day, you can simply remove the calls to qsort_arg
specialisations, and once again switch back to just using what the
patch makes a fallback, qsort_arg itself. I know that we'd never
really get into that situation, but the fact that we could do that
serves to illustrate that these changes are fairly isolated.

Incidentally, if you find writing code that is heavily dependent on
macros to be a chore, I can highly recommend Clang 2.9 .

But what of the maintenance burden of mostly duplicating qsort_arg.c,
to produce a "template" version? Well, take a look at the *entire*
history for that file:

[peter@localhost port]$ git log qsort_arg.c
commit 9f2e211386931f7aee48ffbc2fcaef1632d8329f
Author: Magnus Hagander <magnus@hagander.net>
Date: Mon Sep 20 22:08:53 2010 +0200

Remove cvs keywords from all files.

commit b9954fbb4ef25fb1ea173d26017d4d128dd15be5
Author: Neil Conway <neilc@samurai.com>
Date: Sun Mar 18 05:36:50 2007 +0000

Code cleanup for function prototypes: change two K&R-style prototypes
to ANSI-style, and change "()" -> "(void)". Patch from Stefan Huehner.

commit b38900c7677657a815e75781b776fb1e41054df3
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Thu Oct 12 15:04:55 2006 +0000

Use Min() instead of min() in qsort, for consistency and to avoid
redefined-macro warnings on some platforms. Per gripe from Hiroshi Saito.

commit f99a569a2ee3763b4ae174e81250c95ca0fdcbb6
Author: Bruce Momjian <bruce@momjian.us>
Date: Wed Oct 4 00:30:14 2006 +0000

pgindent run for 8.2.

commit 6edd2b4a91bda90b7f0290203bf5c88a8a8504db
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Tue Oct 3 22:18:23 2006 +0000

Switch over to using our own qsort() all the time, as has been proposed
repeatedly. ***SNIP MESSAGE***

I think that it is fair to say that the maintenance burden imposed by
this change is well worth it. Only one of these changes after the
initial commit is not mechanical, and that is still pretty trivial.

I've attached gprof flat profile output from unmodified PostgreSQL (at
master) when we create an index on a pgbench table:

pgbench -i -s 1500 index_test

That puts the number of rows in the pgbench_accounts table at 150
million, while the table is 19 GB in size. That’s a reasonable size,
and should usefully demonstrate the likely improvement we’ll see in
the performance of creating an index.

I increased maintenance_work_mem to 756MB, plus used a fairly
straightforward postgresql.conf, plus some other, more generic values
for other GUCs. The query is:

create INDEX on pgbench_accounts(abalance); -- abalance is of type integer

Here is the top of the flat profile, by far the most important part:

Flat profile:

Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls s/call s/call name
43.56 79.29 79.29 6972025063 0.00 0.00 comparetup_index_btree
22.39 120.04 40.75 150000000 0.00 0.00 tuplesort_heap_siftup
4.36 127.97 7.93 2677057771 0.00 0.00 btint4cmp
2.88 133.21 5.24 314766350 0.00 0.00 LWLockRelease
1.81 136.50 3.29 314612660 0.00 0.00 LWLockAcquire
1.65 139.50 3.00 450905247 0.00 0.00 AllocSetAlloc
1.43 142.10 2.60 300000001 0.00 0.00 LogicalTapeWrite
1.17 144.23 2.13 comparetup_cluster
*** SNIP ***

This looks pretty top-heavy to me, and more time is spent in
comparetup_index_btree than any other function by some margin,
suggesting that it will be worthwhile to pursue similar optimisations
to make index creation faster in a later revision of this patch, or as
another patch. I didn't take care to ensure that I'd be caching the
entire table in memory, which probably would have been more useful
here.

Thoughts?

On the subject of highly ambitious optimisations to sorting, one
possibility I consider much more practicable than GPU-accelerated
sorting is simple threading; quicksort can be parallelised very
effectively, due to its divide-and-conquer nature. If we could agree
on a threading abstraction more sophisticated than forking, it's
something I'd be interested in looking at. To do so would obviously
entail lots of discussion about how that relates to whatever way we
eventually decide on implementing parallel query, and that's obviously
a difficult discussion.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

Attachments:

createindex.out.gzapplication/x-gzip; name=createindex.out.gzDownload
�xNcreateindex.out��Is�8������5��27[�]������c.&	��f�i�)Y���=���$;#�+Ke|����~We�uj���b�����,����x���7���������X����v�,��,����3�-�K��;k��7}V���+�/�b�-�YyVU�G���f_������B�����L_�$rm7�C��W��=��7�S���|J��`?�}�2��r���!���o����	l��h��X��}z��S��w�X����H�r�x�F�DQ�P�}_��������<o��?�\��?
C/ ����M��+�X��3Y�.v$�
#<\"��������s�
HH2A�������G�uU5�-��?9��������������%��Cs(a��e'�w[�b2N$8�����Y[�������Y+@����''dGb�AzjB]��i�������� [l��\!��.	�n�R�%;��Eq���+�����������U�����"C<OC����&1�@�=�^�����>�� A`�E_YV�����E����m=ja���
�pgK�g?K��cm�a��� ��G����a���J��A���VD����Pi����H�0�q�y��O��\VEV��
���@��=X)0.�2r��}���!n!��|��!H��
�6�����
!*�;#��������W�:��B��9qBox*�bg{��=_�+��,��1��$����"�'���'N�|X6>�7�D�6�j�%�y��@�a�.5����?���78�<��>5��)lL�~
A��H�'����sM����9|�����/���6�)���P�!�NV�v&oNOK����,�7r�!�h��/�����vM!�|]!��f�;8��F�c����}����}��oEhp4*�P��KWac��>�X����c����Yu��$��;,�Lx�O�k���w\8C�?N�JW#<�%B�x�YZd}��?bFb�����A7��K��Gvl�����A����H��n�$�t���i)�p������������	!����b1Jq� ���J�M�L�L_&T�L(Cw{��}VWhN�l93or��s�!�rz�xwW���,?�v��r#�p������x����6SW�,�.���eV��d�����Ue�>��b��������]�g�����~{�r1G'�0�
f��R����u�_��i�#gu����$(��G�n\o��������z4_�\)��p����L�"d�7Qo����>����fK32FAp�m�i����!�b��z.�mie~]���	S����J�;� m��3�r�����r��]���m�lI���-���tHpl�	�3:�&g1C�G�-g��dE�������h����u5FrkF�?���w�VfL_���c�J�Iv��pG��d�3�I]
q2)�����H7��$(Dz,�
�P;�^2G�Mw�Y����,X{&�*�.����.�Z@v����a{��/�:��z��$PR!��I��we�^�9��]
% ���\���@���)W$����/2�C��B$��'X���
z�Z�B[��X.����L�0�m�����L'������S�T+�({(����M.0&�B��!����M��pc�[�������}��L�k�^�7���
5�������a/W�o��o!y\��a�T3���PI�:0�+���4W�:�bu��mBL�n�x���+;ee���G��S���6��PX�#c��dr��0Uz��I�>��
qS\5��{��@^�}�������`�����������>�j���JZ���9��X��1����m��"s�������n� Z^��;��%L��Mu�7��k���p�B�H���Oe=������s����r?�uu��%K��s�,B�	�zf��=@��
�c9j�P4Gs�&�O�q����Q@�v�����d1��D9|�4N�����6��m�������kF0c�k��LQ����hnd��x���j���5��%��IYSR�]���%��3�pF���.��s��9�I��48�6��o�`��h|���
jaUO^��j�$f�o�3��f����q���!7
�M>,�{����xd�~�R�'�|i-����#���^/h0_�
i&p}>��
���v�P�Z����$@��X�������4�``Bf��6;��8�OkBdJ`���)�larAY��P�1�,X���z2S��j��=U2��'� �,u���
�Dsns��������E�8�Q�|0���7����A�E;��i�o6�;Zjx����������6~f�'P89�����+�oh>?��xz0��%���HPj�H1�9����upy_Y���y�d�2rb��C�+p��(��+�?
W������*���O�K���u��F�O�fo	�Z�f��5aUjrC��GO���ai?4���47�K����]�3E�����6=������
W�0"cG�c=8Z�E!�Y�fe}��!^��4Go���?C(�c�H������( �����o��4e��j�J�5�#E�
s�4O5fg�p�]U*��2��p�Zu�i%?�����5�[C����Qy���'��u�����+�)���;�p�/�^p���|c�����U�yW�������`Rh��\�-��;+��03�X�������l���W�����tZ)!��O��|R�����m�OcJ�F8��2B!.���p�
�Q���"I]5�� ��-m��5}67e�����\�����Z�<@����5�";�YkFT��A�>L	q:f��D��|Y�I���,T�����A���c�-��\�l�7��
�:����t����;b�X���m��*�v*���0K�����u8����&�JL.]��8#�`Q����*�=��s��z�L�^5��*9���������3�����Msi�VRa���q�j�9���sY���!x�G��5�q�������$����)�%F�	�Ce�I��M�����V�)RV5�&]�V�DM�%2Jlr�oy����^�}�� ������C~k��M�m��R����|�����X��g���N�z������K�{���F�����t��w�c�3���J��]��|ju,�v����O�����n,����>�y@�O��{L���I����:��'^Gp��f��i�M���V���`��u.^�H���v�ga�uHV�p1�������������CV��c���2�\B��Q�|��3�D@g�$B�`�uH��D����_gvJ�o0������\vP���M����";]�?���p�=�t������DS��h�Co�<3�fy�4`Y��<�Y������R/���Mz7c�nfF�P�f���R���]5��nY^D�z��U�>dm�tf&��T���9�Y/���*oj���:n��O��k�K~PE�);�{��d�h`�A��6��������Y�����6�w�����i����>=UC���9XEC}����������N�kw��]�2�}���vM=�Ta���q��c���Q�Y��*�r�]r����XI�.}0#��ME�`���=��M��/��&�#q�B�"�T�NG��V.�U�t����&��x?���f�'~�����H��
F�p-9dl��M��m�������������0]��6Z_��[�}:��c���u&*����/��7$�T�y/�g]��2�j�����4�$oj,��3Z<����]5%�I
/��[�7��FS
B��j���S[k��r�������?h�u
L"S���b��z	i�w��	twx��<~@D�2���;�h3���>���r��\R~�
���0.-1G�o�G��P������C����u]�����B���uP#Kk�&j��
�Rl�q]y��:��o��1NB�����v�@��8a4�-�Y]�y���j�^
p8�)����?��Q.n�
n2vl��4���M���SS?��.>�/dm��B��&����E��
�Xx���]5����h�e�s^A!��[���������$t�L�3�H��[0�A��e����&
l��������C���Y��Kh�g����Mk�}9ex�����!7��p�5�7�����h$';?.@�����z��'���.�r�������������I�-�2��S����d�#�9e���i7�������G^�x���.+��J/
&�U�s[�.�DyR��Ug�s�m t���G�NU��k�dg�S8�������c(f5L%7��Es�X9i�������<��C��86Z��iS&B��ll)��M�O�or��D�<�k7����y�rYwm�+):���-��O���Q�.��M/f���8\1�O�
�:�����~;�,������������@<��?O�7�LOd)2;�Pk��*��bq�D?�������G�7(W�h����n�12�O,&������4�094u\���&]0����9� E�X�M.�'�Ba���c�!����-=�����jM�K���L�dM�^���]��d��_�_N���������M�K�f��>�����������g�a�.P��������NH_m�	4'L���2���M4�W�Z�H]���KZ�wn�b&C�e�p��4;�r'��su���SE
/^�Mm^:�U_Z�[r�&^��#�>���8Du���E��Aj�c��,X��w�^��sc*���Yw�����^H�w{v��n���r��������p�����J�%�C#�����Ij������V��t�s+�XqGd����ck<h���V��Y���o�5z��L�C��}���������6�[Rd�[��m
��7����V�*����4�^�����74����)��_�����FE0�0����r�B�����:�8�))^�������;~1�a(8��xQ��b�����~�O�����A���M_��
TE/_����0` �7�kM�p�5<��.��Q:���v<}����<+
Q��z��H[|��u�����-�x,o�����Mg���S�;� �mGFg�6��7������`:�twc��?h��@���4^��D���"�}}���
FL��~��#�o%��/�`�h~������Y��@�m���k%6�c����F���t���k��r0K�R.�bC��4�wy���8�3�����jSx6=C3�[q�,��O��������[��6N���sS���`)� r��u������?��+R����g��m���xJ��1eZ+gE����3��������_}���'e�����phE�W��X�����"��w{d������6�Sk��a8�0�Z�T��~BZ4;v���cc|DW����ep�m)����E�����0�^mc���$��F�\;1B/o7��^�p�5����/�����DZcy���6����z���7e��O�[�8���Y�_�aT��WH���������[;�}���k+�_��Q���v<x�����*p�m���V�:���
]h0�����W#�DJ0��c�e�7xSp�M������'��8�x�%b���2��ct��|���w����4���C�~��
�-�k �����
���6������W
sa��-�!������<�{�T �3;�5�����������W���i2�
��+&G�V@�Y#�������Y>�l1������V��d0~lS51
,?B��1a:��U�W�����0��A��K��\��5���5t��2�s{N-���
�}S�K����g�==��MSiO�-?T���.f�G(
E�Q�3�%Tf�7�n����&(����L:4��1*
X���Q������������Vp�%N�,�x��ob��E�b��pd�*����i�1�f�$6����M��Q���!^<���>�1����d���06Xt\$+ �+��-�>����p���$�>����m�d��d)�o�1��oy���"
�,����B>O��f���G'���U���<:mr�/����2����7�� 5����M��%�����=n#[�(��l���/�5S)������vu��r�v��6�qA0%��m��I����~b������w��\����->
�����,�|w(�S��D�#B.�nz��D�T`�Wn!9l�'�G�9�Q#�T�,�\V�� \�dI>
C��9����8�����J�&�������/�6:
=6�^�~5������A�b �6MU�����OJ
��u�Gd���oH��)���.x���3��sW�+��-h.����j�yg��/�A2n	$6��j�f��>~z����������5J��q�!+���E9
�2�`G����S�	��6O�����j7���0��l3����"c�fM��=:��4��y4�?�3��%�����.|D�C5Y�����j�W+{�����j�����0o���J1.;����W��������n�*����U3�q��!rq�i�����@j�h�T'��������N.��Lf�]���p�����3�fL��p�V����T!e���T��Q�������A.�$���'��G����y~���5�)~5�;�L�P����B$�W*�V���E/�h�u(�8�T��EV�����>=C����#��v��>���-#R�\�fK�
_2�f�]x��I���s$��7��x�����#` 2����{f�;5e���K����S&��?c�i�a�vL+�n���X�9/"N����\�1�:�]g����2�z>�PJ��$����uq�����8��H���.9����u���t�_.�2�	�"�������5�e�By�V���t
��P���RO�]G��`t�'��������	@���U\���)�`�P�?��j�z��H���9���M�����(#���b���I��g2�ik�ZeJ�pB0T��<��(�{�tW����LH����m�&��F��[�I%���5�~��{�����b6���o�.�'�FFpP@��1�=�����1����G��L��%�'�L�\��G��K:����5~b;#�]�DW#����&�E�(�n];g�qo�%���n���~����HI�q��}[���r� �>�3b�2����9	�����n@��g���+��;�U��C�,��qK��$������X�>��=)��5S�cQ.pj��$!��u�����T�w�m���t���q�E�����G���T�G]*�������t��k�r!/�;�������11�r�1i��f+l�"��$�
�������$Is�S?�����!;�-!0G�"%��=�78���j���	�[����{�1���j����=��_�eJMct����������Tl��8�cv��c���G���(��7	�q�	#�o0c�q��)��A�K�}BI���px93��0�Q���I�s��Np�;fgg������9?���H`ifS��`RF��e������6�l����#e�4�h�$��cB���;��;&2J�w�P:
f�pT>�QF�M�~� �=�����������@1��"�1;���	,�d2���hc���v"�B��?
f����`���*��lS���3�N{���(�3z���
z�#�����gS������L�k�A���PQ1��Z�?
b�����3.8���`8loG��F���{:=OS����y$����`����1o{���;jK<n6����@#(`�������?�~:}�ixr����1
0V�[g�7&�����������.��z����t�����6�O��:��n�p� 5n>�w���ufFC�X�2�.��k����r���w���,�:;F����d�Ra�����q5��+%T@�)L����9
�n�]V'���:#����P����+��������#�����
���*w�[��?��\+���x��AI�y��]��UyS��0�����DR���:���c+LR&��$�y��������|lv�f�m$�m�,�sN���dS+!�mm�YGG���#���*�2P��o3���N������AG-	������BL b;�i$���u�"4�C2��N��F�%�?h��f�:�i��m���4v�'��6�����>�����7t����Y�>?��-��1�k7��)�$�gO��c6O~��=S�����
e�k�j��c<��O����!LS��������7���#x�b��.�����j�O�8D�j%�#3
���t�����h�����MM!5A�~�ysVtY������_f$��	��9v~w�<fLl��U��X��ur������=-�RS�F#����"�X7�Q�'��b�H��E#$���x�JB��s�
��?�
67����1�������w���v�p0����K�+�8��Q�%P�@�9��o9�7/�5x�o���`���$�M??����b��(S�2������{���W(j=�n���5'i���v���t}�cN����,����{H,��CyW��m.�j������f,�����A��F1���I}S�c��@%��J��������R��3.���I��_x�a��h<���H�Cf��\2�-��Q�F ��
�G��V#0�����pEW�F�N�A����C�R���o^�<�$v���7��%�wM6]�0��.�f80�dp�l6����W�����8"���a���X�t�L��;��S
@���;b{�1n�!���������
%�Ng�]�[}eN��X�s����`�����������vC��mBa����!��|,Dg%+�d��

�_��f^�t��y��2�fX"�k5
2��1�J�x� l{.SZ��:�N�+s���yS2�F�>�y���,�c�2�B��<>&c_-/�e�������Y��RJ�"�
iA���1���m��D�EY\��U��X'����`��l���2�F0SP
�M������o�M�q�Q�#X������B>�1����
�������mF`L�����(����i�(�-c�\Ppt@V
o��9�uS�a>��8�D���M�-s+g,	b�`��%����{j.�VJ����
�U����Pp�b�B����\�3��f�	/2��N�%J�8�� S�����M�XV� ���t�7m�v���%ARR�F���DJ�`����*��%����_<0�B���#p����3���ch�T�#�(��0�;m�U���:�u���%l��?Ax�������	�]���� �{R�\��OZbr���IYBc(m�m�^���v[v:���>����>�p����3X0O@ab������Kp���Z3�-�2��B+�:���S�P�`��J���2��MHJ��p��N#�w[9�k�?�=�<���s����X^����y���(>�Q�������&��e?>�QA	E�����mW����g�$$P�"��0�)�J:f#��n=TV��L,�f4��Y�&�vH�����]i�#���1�Tl��@`�>=�l��0�l����f��q��B�hF��
��w�@^k6���3��WJ&��|�]������G@x�.��f��I1@��$uv�[�!��<c�<�z��.��y~U��lK0\:~m_�x��4W%0;8��t���F8�������������c��������Q�')$&^�)y�h���`N��f�CN�tF����D�K��9�#h�H��6"�TL4�"V���2�����0������]t�����;.Y�m�*K^_iN5sG
~����ty�A��L�������qv�Kp�NC�5������l?`������T^��)W��� ���F@�nT���F�e�tU�%�]��'����0f ���
��F�/�J_���Jt(�l`H�N3�!�z���z��n���57sQ�)�{#8xelO��a����_�����`�������,�J�\�_A�,���\�?����d���C�%����B��C�$�2�����}�-���"o3��!K��e�x���bKMJ���Y(=1�#(J������Y:��hF�vn^��������,v���P�Ey>�]^��6����%&7y[n��b)bL;��;VL_�uG�I(���&��M�[�E�+y>����"T3����!�b$@����>����Jx	RT�Y~r�p`��z�U8���m[w9E������e�(�J��8V�Nz�8�������d��*YAK�=	/a�I�|3��!���&��nWe;R��(Yt"���r3F�g-���T!�^����<���V2�H�����MG��O��s��z��O�c$�+�8���6E;�cm-��S��6��HI�������D�tKuA[H��
v��'�f��B������w���OzL2�>Fzj�1P�{8���f��`O�mWW&���������C�J�g�
=Rz@�A��`N���#8��e l)
5������#��\:[$��Yv��>X$�q�v6����s������J0c�t�X��[I����������4�;��?�:�*[Q�)P�F���{z|&�o_�;����%o��<�GQ-N���P8�����Ol�B�X#���	���R���x��_��1�E3�_1I�j��>���t��$W���5B��(�-dF�/)��@U�lx�{b�0C~ ��1���LB�����KBT�7j�;��7w#o�B���L���/�I�1�a�G�L8$��������J�����4p�����3]�L������P"����t�<!�UB���0j}*��$��]�	�Z��z|��7�"���s^e�>V'J���B]�A�I�h(�493 NT�J�C�^��2K}^�|3��b�{C��7?������@�:�2�B������^�&z����a�-�E(�!��(���^���4�u�eN��i �nCY�e����v{a��Q�S����U���q2��9M����@�Q(�3��7���)X��("���[hf����'�Y�4���
���c���}�9(O���4�h���4��H���2�z������)�G�mD,�l�X�p�������]s_I��lpK�����5g�]�:��<�6_2Z�����v�Uv����dC�{��_��|z�[�t����B�6�����'�:�V_>���Aa<�`	��
�@B���}�+i�%�4�2
13���!���V��	#`��
�i�h��f����{J=()�����J������#m8F�_{��M���(�X���f�����i�F0 ��5�C���DB���������"�"M���X��
$	���bTC�oF	��1@`�9pP���05�y�N@��TvK���5�8�������^�h
�M�����>�{���3Z�J�Q������U0������@��4�:�����1�y��]����#��Sb�$��M-3Y;v�+j�{K4E%��fT�����=~�41��@"e	�B[�k�h4�-�^�oYO�I���+(^;3�=��`y�����n�_�^hG��$E��8x���������"{c�I,aXkn=�]/D5'ozo��h�u���%5Y���S '��EX�I���(���c�o��T�Y�T��r1�]�s��cE��f�b5��;Bv�����p��`�������[Y�W5E�N"@��4�8�H�O�����GF��M��^�M8�U�+��q��m��[~���9od�����R�V9��b�)��l0����HXGw6'8!���� ��#�5i��f`3D*!���,���p�������&jg�h��-�<�I�D��!B[����
�����@�GQ���/���9v v<9��S��6L<Z���q���6/���� ��bu6��D�B����LNBj�<�&�SMI�tpG
0tz�m�����G��~*AVP�������TXf���'��h�����j*f�Q*�F0�4i�0`����>�E��@G�+�S��
�����Pp�9��6zwx`�
�/>f�,�f
$�>���yd3[VM�[Z}����$�%4���_���X�C�0��Ja�Y"r��#g�6�R�;f,�����`�B��F��fn{�>nh'3�[m��b/G)�5�6���(IF���X���d�p��O�-�l�b{�$Z�/
���$�sq��3������`�*��Z�g�
1wG��1�����+*����CVM��9��S(�i��q$��xK���"���L��'�np���cp���+�����6������o��@p��;%���NyB� X�������m:2rK�t��H�?���eE}b��	�g��_��Ff���59����0?������F�O=g&����8hTf�*��Ql�	�[H��#2[����mZm����j�u0@�����I
���U�-/O5�����-��o����>[�`4u�c�9���$$��������yH��@s��Je�-�i��(}�?g� �!����{,�|��������i`3���<�`��4o���]��~k	�9xL��Gp����M��o!
�t��H��H����>�M��:LuQx8��Ui��.M�5>�����x����
5�������Ul(.#Hr�>o�:��R'���*���!�-1�|�0Kx�����_��#Z�&hy�=��z�#��w
k�
*��(�P���;P%��a|����f��ug��?QT#o�e}<���"�T�i�a
o�}�Y��;#�X�"�f���f�=^e�o�����#�� !j���^*%h�F��d��B�myh�o�����#��+��3����|���_�;*\�`��n!�wy�n-fUW+���#y���(-w&P�B`�"��_�U@���a���8D��<H�q���F�\�����?��7h
�������7f\�sNGD�5��i�a�H���QZ�s��(���B=>��� {v��E")�{}0���B �:Hh��Y�9����T��-�������(��]lP���AIg�9�V����i
��#�'����H���-��������2<����(����z���%f9>K��a��N�6����������6���h��\
s��5{�#�nq�I����L�^��-a1P�1#ZF���n1�lcSV[M�R"#h�zm�9���Uv�Q��#(8�s��;�E�G,t'��������E����n4o:��m��P;h`8�CJ��4��W@�l34U��:�D����d�B���o�����a���)�;f�����%��F�zGIB��k��� 1��,fL�(�I����K8�X�k{yJ���B��1�yU��6���)����������3���:��M�y�}���s��p�)��������4���e���2uzcm������z�f�H��7�����t�y}�������f�@�?jF�$S�=��ti��f8>"����M��Z�Y��P�@hl������:Z�304B����<��UL�=�X,��"%��G�0�����3���T��1bZx��-�����*��W����Z�������]�=U��� ��6oBm�9�_���$�J�IMi�i����	;4����G�����U<|I�3 ��`q8�-J����q��#�u���W��a�=�BfF����UU�_.MY�EA�����0��D|�����nYQ����8h�
��[A��#�)7�1�?������.h�x�s���������V����8G����rP�l~������������$:{8���|f�
U�/��+p�X�v �e�=;)��l3DwL��S�-���	P��'������y���$/�%�wvEo^abg�i�R *�0�����x*.=>cF�-��=d��\B}4�����>��H=	�p�x_��'���Uf��2S���9iuV�PlG�6�U!�����M��jycl�n{�|Sn:#d?��*��`���W��z�nH,���&K)Wy�km�YNRFIEK'nY>���8�8������q����A�����%������)�i�f��mp =��e�L���1�`�x@��*��n�nh:����@3c���>m�I�fhp@i�3�4lt�K�s��3��oD�����7����MVl���^Q�2Q���f���]7}�@[�<m�mR@M;�5��4��cK����4�@A�tN�18����	���4A���>fh��i*[KH��1!��N�:����e��e4��6/L~T�P��u0���+F4�&:]G���B	�N���%������]7���M��l��h�B��M^��(�g����p�������#������������LPd3$x�n�����
M�2V52� �S���#l��v�o���i`��[0W,q�5���	{�������j���G�'���;6%�}�b�-��3�~d���"��}	�
R}�1��k���*2E�;v�VHk���D�GU�2�iB#4��{�ntJ���������r�P8���d[!�mIk�9��d�z3�ZOg�����fc=���P��:������zG�?���-�;�/�G��YX;�hy�}M^��T� }	��`�t*��XE�m��%��c����)���H�3��9f�1A>��6>} D�t<Q"�#(
T�&��a�	xq��F$f(������sj;+L/f���{QLc3��b SI�a����z�MC E�o%a�V&����#������PW��,�'
E��������/�l3��/���N{"_�d;�7>~W*�����a��9�0�������5�G���H�G�����9l0)�5�d��D$o:�P����W��{�lP5x���������M)�G���7��g����<���������]��u���/��6��5p�,r�ZR�M��'%�!wf�l��]�,1@��B��lO�:�(�o�ioF��Hj0��*Q�#Fo��������/����������h����N��2�U�����g9��]}�7-��$6>�LB\��x����I�I�$��I��x�l@���>�uZ���C%�7#�5���Dr����+3D-������v4z��Z��W�o����wB����m^��z|y���1�k����b���[�����\<Owm5?��,
�� �0^J��]@D�G(��;5=�a�s�B�!��i����Iihse����AI�3�R�lR����Lk}jF��	^��f,?3v���%�J����eO�h�%Js	���1��M^1
Qn���Z����n�B(�D�#���f��f��7��oNM�G�9�����l�x����?�ye�`VJ<���-�|�Eh�*F�2���u�y0�deNK���������Q>��8NG�A_ Aa�@�;�3Q���mO^LN���,��%"��X�1���1�!lK��3�q���>�Q�`�c�1&	oQ=�����Ej�� �"����(��3?|�$Z�:3Oi��e������e�T�a����N(�-��m~��K���b�%g�l^���#X�cpxf	�H#uv���.4f�����:2?�u��ZQ�K��5�`�>��*k���4%c]�/ �%�H�,�A���j�}����]��ZK�MkT>����s@~}d����Q�{4:�8m
��p���%$�?��EF�+r�BH6C�K��f����{�=pPyt��H�~������{��>��b�D
���W�XciT��+������s��PjC/����!I�n`�����1	R$DS2�G������H�����5�3�<����6��i���\!�>+�^��fB{g+�����(�fHI
�P(�����p�1��U�.��3#~�y5	?�:�������N:9S����,����0@b�"�fh'���� =�8��\�����=�uz��(�XL�K!Hj
�V�<B_���(��W�b�����{^r��e���B�%6�!,����!o�VUYq�Q[�
�����}V��l�x�q#��E~�2����#�L��mH�"f�=�m���vD�a��dQ�b�;JW>3�Z�_�J�-��J�9����*�o�=9�c��3$��}!�T3!�
����L&��-��h��$8�����q�	'��v>,qF
cK8t��������e���ll�0����$1�m��%���bb.��:�I)��$���0vi�����iCX\5��?�t���DS`�4c$A�f�1{��P���Y0G#��bkJ)O���w����h>b#Y���b��=K����8��cb�U�dbML����b�g)Uf@t�������%c�&j��~#���}*������~���
���<)�n!���T=a��5��cf���-���&V��\v�vL�%���=��=JW�KY��
�����B���O����_(tlf����z�2�Z�p|�LZ	=~O��,�n+���yn���=�P��=h�������a�}O����*l�=E�E��� a���S����F��!e�����X*��4���E$�4#�:�KF�[w
�(l�`%��/�!�9���,mXX��B��o9�� D���@����7����&�;z���B�S�of
4��y*�0m�����@b~f�����B&����S0��W����WJj���_�M^_L�~BOtBc(�3�s��=�����y���GR��(�y�?���y�-=>��������������o�X#x�F
�8�?P:B�q��r�4�o�M�%��F,����L7��6	�y�����f�Y��Ss�=%�������K�7L��g�B�?��Y�-�v���k�����!��2}��NA�
��p|��B5�_
����e�0�����+L��$�2�i��zd+�Y�h+�g7CY`)���P�>��3yu�J��^���*m��3��!����m,=����-)��|G!0�8���m���B 3D��������I7��u��3,Rxx�a�!q����0!���#�	�W�r�)s��a�c��<w� ?�]C�{�Vo1o�[,����/������I�SX����-o��������
dBCd����S�$��|�����k"��bSLd����%���,7M1c��_���8�����A����1��>��'o0�sdf�D��$f��$�4�0Sd)�����TSmY#�����e�tCR���c��E��5b�����K���a���=e�V��(
���������|��C:���>-Y����f�j�N-���$s!�#J�O�`I����F�4�p��c;;L3����1�W3���%z[��{#8U�nI�U���r��g����6w�3�Gp"�Cd��S`F��9��~��(����e����l���z/������!`�E����tQ���q,E��:��C���$�3@���MY��SvY"�)�����Vu�����P����@b(3?/Z�`�|�=�R�O�@�o����KR�H}�`�jgY�z+��E�M�:���&����p�oI	*�)�e��'�VR�V#�QL���v	2�4%�v�F�c�(�\�9WCj�G��CB�q�;L8*���
���%~3��8�`��z����,��%����HI�$0�N�Io��r��p(a��Mn��O%D���Tk�4b3*lZ(�fd�������5J~�������@%5"��V�Av�F5C(~�@i:��O�-���m�K)�5���Y���R�M2��X����Dq1�Azo�BNpo��xJ�/YAd�0#��CZlo���Uw���~����&�w����w5������k�ZK+�&z��S�g��7��v�||q���e����^���������q�R.����h������E��=��o�%��P?G�p�E10��5�9��YlS�`sw%dn}���.�h�n�(t�8s�3����?�/"�m����0}����6W�I��B{ %�L����`�%����kv����0���kc����Xo9��MC3j����I��hRF�Vu��^07�����~�N��=����������J����%�(o���C�m���Z-`�����]��`�7:R1lnK�\Z���������?��`�C��X��
d
��� ��I�y���U����T�d��� -�p|sk��{���d�\��
>�����~�������c��ML+�G����IEGE&E�c�H���k�yq4��?�MsW��i�h���#�j��4O�9�BK�`.�-Q���%9�D�����]_:�]��x�l���U��j�O%��y)���s�v����(I���7T�&#&�,����*�n"$��
��7[s��aR��Y�"�s���a��2��a��
rX����<B����@��.���i���@��~U�|�d��AZ��
�>�Mb���!��_R�;?9��t/lt����JJ���i(W�)9l9y�=P���w���s������LK�-�$��q���
dfewJ�C��b����z0-'g|����X+!�Q>Ul���]�}�2>��=S��gM;��^9�D�-� #�����Te�7�g���if�	"�%�N�?A���pH�J���N�r�=]g���Y�<���<3��cD�m�'���8�1+*�m"�~f
+�Cz�9z��=r��@LijH,�������`
�C��F[w�>]H�h���cc�E����s���}8�E��wLKNtCd'��� ���i� �5�c�j]�A���\�O�"�r��bj��8&g����n��b?I@��#5��h��z�}�|�V�������(v�|�gkNG��������[�a���f�6kJ�@�l�&���I��"�w�Nu�u���X�&g�8��mj��+�@)Um�d�N�.��{[}�a�-
H���a9���K�eR&���-��^1U��c�����J##�(]��{~����C�5�~��W�{��)?�g�i��g��Wd�<=#����9i1�L��}�\9jF������2��'�c�3��O�+�F�tLp7Oe�T��~�����)^/�95����eR,��
2K� ���e�lZ�v�D�q
��o�PI~�z)������S0�43sPj�����������(���@ �)������p�"|���%�}�S��2x6���~K���pr���a��vA������X�{�J��������6��P;�Oe	q�f��X����^�j�~[��+1���.��^�eP�`{��������%������y�d�H�G����X)(7�m>C��h�t�OG9y��������b�����!��Pl��z2������/�?���h}n���$*�n��Z���u k��Ev�!pk0�P��������y��z�3XU��d�Y ��R�O�[��8&_L�=��)Mb%�9���f������4?����������1�(��Y�s^��h;��.����RinDx�� ���oi��(as3��j;������D����nf�
n��lOk-7����a3�e�~K�d���xT�!�����9�4���d�d%���������� ��e������B��ld�����\s;[1@<������(���J���n��}"Vn���f�l!��{�D#xs@���Te�;�����j
3F�,��`S�\�K�����������B!~�y���s�r vq�-O��?�����!���;�?H=��0>f��#���
#�]�nL^���yk�Gh��7�E.T����`�3����$��F&�����P�g`�W�VZ���`o�t
9:RH<�Pp�f�7�e����
F 0��������L�{�f�3�~��O�|�� ��E ��n����I8
�����tHs��"����'�v�����0AA:���[^��)f�D�j�'�.��AB���M��<�a���B*�L &���������q#|v��3Z��Z7������,z�g(�� ��~T7���f���*����3�V���
z_��g��z�e�8�T�>z�!��}����{C��<���������t��[�Z�=p���Q�@3v'3����}�h��Ff6�.�{sn��?:w��g��j����Wf\9�l_���@�.:������Bo?c>@����*�(��c���O�Q�a.�>�
<\sA~o����0��=`��[j���% �~cfYn-��j��i;�_�u3&bc7[[�������G�?=��	Rc������rk)����3}r�\��"��f���~^��fAu:Z'9�G�q��H>q�N���PP��r?�nh�d	���X{�[��o�#;�����L�A���9������MY����b��>���I|�������:���b�[���Am
.��\���}W�'�`�0���<&��6'�����d
	[Q��9Ds/�6�>�jSp��G�H�i����9��#/]|T�C��>Z�������?����?��I�O���fu>���c��zsS��������N���^��=b����	�%
'�3�����%F����|��a�L�����S����n������X�S������s��=�3�3T�E%���T0Ik%R�1�L��
O4�y-6��H,�����������b�{,��{V��5M������	M<t�3���#R�����4��M��Z*	@�6���w�M��Ul���+���{}_�%v/����3%������f%�+`w�6`���E��#S�Hl�����>����2[x���s�2�.�s(��3��p}��rm�}9���I}Hg��c����a_�y�-b���O?���@��}�)��"g�<���������������5��^�D��g |����d��Q��`���&E	�2g@�G�>������(h�0��9�IKf��n�?�m���2k�0�~�-6�f�l���,^y{�����u�4� o�&��<��e�7A-�O�!�m��VH�l=�=V�c+3��?���!�r>/�/��3{~�0�%�K���
z�6�����������`�U(�rG�S�e�H�m�CRg�>�)��$e�8��+�C^�>�a�t���Z����|��q*��t3'H���[�����sL�d�1���P��pZ1���9D�R�,(�fv:�@�����)G/3���PXd��eL���<k�����������&�=5���v�l�;m�o3��dnN{($lk��Sj#�Af�8Dg�%�8�T�Q��
ks�va�3� ����#���������`����[V��V8�2ZA�E��>�r|)�W�s�9.=6_�8��l��c�
 pj��E<���KU����*�O�HE?o���"�����m0J�dY����}������4Qt�`���R�x�aa�1s��,��8�Hn-q��7b
SKC�'WE�2��3����8������?gm�����Gs���YoD��jcY��(�F`�
�p�k�%�3���������>`�C�r�oin+jwY���IG}<C�����`���;jT�l��p����;@���b�~���e�����[��/�������e�=*KRZ;�����O�*@����3���|~�/�yyJ�TC��Y&��j������=��Lo2���)gf��JAK���aS��kfD�KZE�55b�e.��l���[3f!���N�y��V��d�$��^Iy������x������E^���;��ATX�`�vH�<>����?����-hG��E]YG�����u�M�1��C�&���In�`����^�HK��6;|�*�-l��������l;c�2{o%�����O��q�}]I	D�h8�#=��<c6���n���>vh���y�x+,2�i�au
Tgh{�Tg������Q`y�\&�u� 4�a�)��B@��c�bc��������0����J|a�/���!O�����[V�������MD���Z��'/ei��-���Y2C8K3'qf��W&���A��Y��|�T�1g.�����s��:����j^�������$�cUn����C���y�P�?c�������j�>����}�,�Q�>��x��v��0���y���.p_<[>�r������pjh]JG��&K�������� �M�f���]�fxFq �0��H}��5Pyf�w��6��}��.����������:��,\k���A�z�>g���S��IfOn3{�J�R~�t+[���:�5K�����[U��F;��N��bT:���"���m��,����Ft86���V����7P��\V�����%�f�HX�QV
wl�x�Sc_�����%�,�<j9�}���L�����w>h{�k�oW��}��y�����al����g�/M�����T�����_����c���+�K�WO�����V�����mN�c��U�a��C�#����U�o��)�m}�nP�����?]]��2@�bG��v]aGs���S�~'o���g�������o�/J?�g�����O�������{
�>��^�{Qx�t��M���������@����m��W����~���M<��d_��l��i����]�/�2c����Tx��Y�����op��a����}y}{��Q��������M�����7����c�W��}88��<�
\E���[�������Xry]�����=��(�����Va?����tv����0w|0�M	<��	FR���������U������=;�c�V���o)�0W�]x�)��2nf��������o�Z�yx����������	>�g^^���oc�k��6������t�rU�P����7� V���V�����������a���k�������W�\�[x���m���"��
�k���?�#$�Yy���o����k��e&��@��G�G{;���b�~�HB��\;�z����������2>?���
ja~; �a���@��@m���y��M��2qU3�8�})4;�>������~��'��b|�>GT,ui6lP��^<4���>��Rs����wuC*�+�,x��:��cnW�com�����N���=����;�J��lL�;���f�f0�6�����^�r��iw�9��n����J�V�	�������h�p������H�?�����'�*�^���b��^�A�����@�N�{����.��,��6i�H.M��HK�my`M�����$K��W���^zf�jrxM��2M����:��B�&�FG�]Ktc����q���qD+*�� Vd�$}Ak���2M
p,������fi�*2�5f�fh�7qP6����� ����bCs�;R���W��O�/�v��>�hs���"��7���$��'D�p��zG�N�{e�S	n�O���a�1u0���S���4��uT�R�l��a�H�D������z��v`�?�O��X�a��@�Z����o	��|v�����^��(��U�
���W�O6�N��V�u���@Q�T@��J����|�B+������^�7�m�~�7>M��@L��D����;���Z�~���%9v�/
�����,���`<��� ��\��������H��

���[D����x��X
&*�r���]k����@>]�jur�M�pY�����0u%I�Kx)��fq�	L&��?�m�|�,QXB��O;��_I��3��MY�"�gS��b���w���!��J�x2;��kO����Bn����z�:�>G������T�<�s�	��(e��+$�3��������2N�y�k���`"��K�W������� �)PoH��@�9��K�Jr6�a��gQ�K�e���;q��n<�ds{�/aO�/i�q���R��b>4�_i0l~��ig8���:��u^�|�X�����uH������"�����w>��v��!���T��vU�ph�)��A4�D}���lLk�/����������%Z�l�i���]�@.�MB��gf!�x�����!�f��M��x�Q_��#)�0C�j������Pv3��������3HC��,L�OU%�����+��3����g4�l`�%��������
y"���]�^��(��o����y�0��q�
� ���^��D�m^�V�I�tx����7Olog[I�a�o�+��:�`���Ba��"�
�h���ai�K��M������	1J�m���BrE���&���g
�FzCmh][�����Pb(T���@���W��j�3:��ac:t��9�`�[
�f�f��E��	if����p��_*���6wF7�������@�LwNND��l
NM�Q�Z5����rLo�.�H:Nf�>cZu&��D}KI�k�l�?f�,��" ��Q�����1��4r�[�-��$I~��y����@�onh��`�Q+(�R��������<a�:��TW�!���*Z�I����NU*\b!���T�H2�Z�
U��	��jn���` ��S]8�!5v+����c��Je��]���`10��-mY��NByb��������8�4��g_8����1��iwR�yl���(S�I.������1x�	3���{���l�����?��8}���������|�(?/���>S��2�����9��cQm}����-s�Vl���l�����(F2������r�%}����q&X1������]����O���Y�:'�f�D�R�$o9�S�$m������p�;�����aq��� +�m�x�K~��42Z��%�����n�{Bv+Y����*0m+B�����69��l��;��B[�3#����d����YO|7MSA�)=��~��;��C���=��3f��//Y�FB����7@ �� �n���l�����|�J_�})� �c�7l�\�����
�!7���Q�*��
��~p�n�h��I���>o����A��r����O�t*�������%�Yh�u��uQ��*�f��z���$!
G���l��&�T�L����\KZ�<1�['��2������T��_@�����;p�8V��!���};�������L��']6��b��A&>�G,��x
4���Mv���
���Uo�����*l+H�s����L�+w�c]&�1:$h��gC�{�~=R`#%VG��L�����DC���r@���\���?1�A�wHL�S�H�M$�K
�)K�}�o�S8������1
�5-��u�������>�>/N���FC���;�� �S�K)A��/����}���gVV�:�P�}��T��>��:��y�����*R��N�o%���u�0;N\�I�OmD�4�e����?c<R�����/���
���w�]H3]����z�i�t
Lr�	�g�V��s���
*�z����!L��5]Y���=<Q����H�������Vf+.v����U�4�7�c6vs8AdS� .�[�0��
vq�rKgM����+}�����{��D�����@Y}��T�IE���d�}v3�����c)���I�z,yY�@�5�L���N,2S'������i"\Juq��L*x�C
�Dw�V��ZGX���2f��Kg�_8Jj��c�	�j{������ {�}� qvE����/7���i��C'���0*\Tf������y��� 3:���KO�����k�Z����*A���_M�l���U�5�u��q��
0�%�s���`���~�K�1L1|����>'~�5{h�������M
��Q� �����QV���1����`FI��s� K�K6�w��2h�T��I��m����d�e������)@��g�����we���+"�,�^�PaV����c��ur���V��V������;83�c�g���I����C������w�\w���)�\�V���za%X��.��bqPd�l�Dt�Rx c���
��I��2a�C���u���]{�1m-�/��O�jc��no�g��1�	�<e��
����c��a �(/�>_��e�G`��:�R��1�=���#�x.��z�$o���z����*^���N�"���@����o�v�����v#%UY6�1�����&�~		��#�5����-N�q�������|I��?p��+�#$��XJ��Ibo��?��q$���/�CV���X:]�%g�������wb{�-� ��v�P8��h��|�����6���jMv�K�	gM{��6ju����[�QdojD������R?���_�9!�tc�&�x��G���q���H&�q��m06����xj�Nk]g�#��
f���3 �Y����O6�29�2�
�������f�8�0�>���8R�C�*��\��"���b�p�yv�i��������MTWp���b�s�^�2#�_�l0^��9;�0=�^~��{����u�5��l�}'Z��k������t8����P���� ��Rq�{����9;Oa7��uG�JVB-�>�)tb;���������w�Sz�����@i9b����1�	������3E��G}�m��?���n�V������c��d%�X���-0.���/�<;$�0�hEL��E+�Q��K�Z�/�0�WH9!���*�&�����\�I8r�������YcHq/�'����9�j����c[��
�M:FW^���~����H��L��	�[���'�����h3�O+���p�N���G���/�.��U��p�Qx� ���i��t�8?G��eF�����C��	����p�"\�5�(���9b�]i0S3��.b\��xE� �Bf�����`�Y^4�
z�����bF.O�9���3O��J�F�	���c�v���*�����/�>���U�l���N����W`Fw,��Br��d�������%��]�����(���b��z�Lxu��L�;%�/����r�2D%-g�pG�8rE
������@��A����r�Cv&��N���#uN�Y��
j��8��3n�>6Hsr	,�r����q�&�`�=�U��$�����kG�����QhE�����f%��
F��q�p�}Y~9���#u������c��FAG�?)���pV��_����}��'P�&C����&��'���;��w��(�j�}8�I^����������9=��W,�V1�C��t��<5����S���N��R�����=�����!�"�;�+U�����:kJ��[�!�a|�^k����F�BwK���C"\�J�|2@�E���t)z?SUPxs�Q���F{sJn�����[q]l28�K���2s�t�5�&4��v
hU6T,\�����3We��4$���R�/��x	U�tbo'�D��4vEg��1:|Z����
N���FZ���h{3�M���3�
8?�>��~,�0U' nnU�e3�AC(tm
5����q�:�C�Z4'���my�p2��}8
�4j���rx������c���<������N����|�&kR>�����Fi���SY�U�3n����`�������	#�YO�7��.��!Y��4�Yc�'���������!e�]�lR6Y
A��8������FuN�Q����������6M�����?5�^'�uH�|��n����i�G��hjJ��fK�����������^3El�Z+S�*�M�D������o����c^�s�P���������o����� S�s�k �nU���1���3LiQv�b����{5��X��u��8���j��E|�Ax�[��y<��_$5!#9�u�s8/���x����C�Q�����E*W+�����0�x�������������I��P������~��T�!b�c��}|��������6�"�����+t��'6�=P��u��-�&�Z=��GN(R�x�������+r;(���d���9U��X�'
�> �; �����y	.Or�����Nw�������r�`�,���aa�*,��� ��{J��/�Bt��t�`)����q�4b�����}�{�?W@44t��_��OT@.���"@��5�
�
����l��@�36yZ[���n���5�m�p�HgELoYkhp[n�6���r��
kv'0b2S��^�c������<��`3
�4�+ �#���?����4�M���f�l���ID������6�p#�b�����i5��$�|'0�"�[����A������hN,-r��	M��r�*0�{��_
��������Z�K�������u�Z�s
:��u:�)�L���V��|g|��@����{\��wc�&FHJ���Q;�`��
JN��s(����v����+���e���K���k~>V9�#�%��f	������@]gW�)�'��*��YmXA��t��)��x�{50q0���F��u���10��-�:�./��-�x�0����Y�I��"��	���k-�\BY2��He��{7nW�E��
�J����!�9i��)4����n95I���C3_��GC�pv�d�NERc��`/��0����X��8f����#oAE�]��Z1.K����#v�4w1pMnl�0��/��������Vr�V�iz���L8?�c�}]�Qt�]7�����
NC�lYsv��Dfp�5��'�<+�r5D���s�
xD����o�����a��;qG�5� k��<�������NZ�0��M`}�2��]V?��?���9�����
?����A�\K�����J@���-,���M���.���4[��d���!l�w:x8o�*}��"�-�l	}CC�C3������j��g��l�f	��].��(�9������gr����y��h*p���}��Q^�l�A�y�u�����DJ��<�
kI,���:�d	����g�<��6M��Y��>�,��	-a��W`�k��������������V��?���c���Z���G?�gZ#,���.�=�^�O�Ud���� �&�������kH���^���B��2�{-�]^��2���@A��/��=����Q���\����h��D:Du�����9�=�2���AOT�Np	��X7�������cvH�/e����_��n3R���t���9������,�RV�#kd�NL�
\2o��tG\�������i��=������S������;��Z��$0=bS���H�����4p���=�_������J����0P���
���4���7d��T�Z�)�Jx��0�_����
�?���T���Nr�������$�����3����:�	_?#a�3�a�����T��^e�Z��M�-�����e�6��K�%���X�������|�^����*�����!�4���Dq����7��M�T8�q�}]�#2��4�e��������npP����5�����NEA��b�k�'v��J9�#C��_�v7���%{����>a7k}z����:�f'��TP�Yc�n&Rp�l��nx�G�+�����[P�����{��@{gN�������_<�����T�1���:}���-�K�q�v�s�=:���lU�����w�O=1����0���Tm����KZ%���Pn�?R�X����L@!��{?�I�KC<0���0����5t�Nxx�L�,��]ktH�1���|~�AW`8v'o��<���?��@n[�_���W��H���d�R|b��C���<�C��c�����k�F�2Y�s@�5e�����h�px��Vo)����E��&�������]��5�C��:�������������u$�2�fE4�|
tw��T|a�j����e����P��m%������ghr��kY�WDMW�D.����h:V�t��	���H���A�����U  mp�>y�FwV#��)$h�%6��Z�!���:@L���,xGPx#�Y���j��P�����b����������Rh��� ����]��@L0+Hu%��K;�:�.+p�&)�HI�=�
�2���^RA�]U�����[2v��m�=��-�������8���e������H�&�
o5I�\&�n�������7Y�w+V#�)G�J��
4���Qn��
���� u0�+�&�`��/��`Y��:�x���#X����H�a����>-���o#��C>��@g�$��C�O�S'����;��
���r/��hp��Qe��&KHV��]��������DG|�L�=�.��5��i~������FQ�zl;��T�%b���P�bS2������)�����9�����)e�����ht�:N�����\����r<}�h�����w�>o6mG4�b��c��8I�7�1m��&{���vu�}�����	;�x�i7�{�h<��~"��-�f��bO"n�\+�t����[K0(���8���;�y+�����K3��i#��c�YL��ru�8�c��p�4�z�NnpC�]D��%�:e��#�F�`m"��K��<~���h�&�s����w�1{��D�0�"�z�@ ��/<}
!G���l(�����^���%i�(	��k����)���Z�T��]V�6������n��`C�mMs#������a|��%��@�C���*�*��x1����>����M�������@�r����t@:������8����i�]OX����.[$���D�}������t_z�My�.�.����E0��m�KL�hk}jv+0����.�P\��-���u���I�}O6�#$�xh]�aPO<�pH�|�����[�������&9��qi%���V��0Y��qD�������[�`�Im^O�]Z��>����-��v��"���'�����������
���v������p�Q���4i;������[b�i�`��Z4��\�N���}��[��Y�$���@T�aLp&eo:���%���A�k���w��W�|Biw`�bx�i��!�^����U�����
�i2k���
n�ns�M�^�C���=��Z^ "(K��u�Y������������&E$w�?X���/nqe�.��i���
�%���u��b�z
�t<�>IX��>i��2k�+,f��M��n���-�J�XQ�X)�Q��{M���<���2������E?���^!��D_<pF�r-��7/�������8��nO(%��������?%�n���������^lRf3u���a*�����s��2�f��������rl�BQ��N�'U��)*���3�WkPLs��8�X8�;��R��<e=�(������CJz�G����5x��F/������Y8 1��\J�!
��hfb���|_'t���F�f��
�*����*�308@�v��U����u�FJ�4��`)Cw�i`� -��n��������@��C��U��F�$�![���F�z�r*Q[H�F�P=�}S�ao�s�������%[�_VB���}?��]w��A��b�=�3��.;�eC�N�
u�M0��Ph:�Qi���{���j���.��b������DZd���7�������N_Q��b���Tc���C9��Z(y�����`6-��3���Uy�Vl�@��QJq��F��3H����"_�z]:�����qZ��A*�����?������l���'����E.~��:��������q�K*a$�:&����9�VT%$�^R"!	��|�'3.D�pLr�a�-���-�S�M�x���#�?��C
��*�$<p��H@�]�$"��h�<�M���Wf����Y#�����1ynK�8���fT��R�;r�<���uJ�1�2e` �Wc�����$�
�$@��.�|��U������u����1;{�s'�����m���Wl��3��j���,��	4��A�������[(%AYJ����C�*i�u!-�8[�8I�����-�J1G$7���JI�K���1X*��;"�o������xS�����,��;���V�>v|.����U�����@?���t^���i���e@�{Vl�]�#���� �����<Z���p���3�$iL�5�u)�o:E��m16<@���}��%7>�.��fo��o)�Y��ES�����eo�=�KR&�^�������c�9=@��K�]�8O�=G5Q����P4	�F]8*���RQ�8���,3�%#�M^RC&7�v�T�#d,%xd�/������2�W\���GO7tuA������������o��� \��P��%�kby��ws<�_������Qo�����d�Q]���<d �uHnW�-�U~�F"�pBD$pVD<_��iQ������h�dk���UafY��}�H��7��?'�0U�)3�h�<�&���Q�}�D����l�A�z��'��d�}���������A���#��u�,�:H{	���W�c���#je�B;X+�C�u�I.��k.���?��L�#9|���gd�Y��c��d���������.�=���*���h�����}M^�	(wL9Kx�m�-o^Ab�����U{���r�b��5s��=����*���y��
`���\���|S�+$���N�1z1��e�=�s����/(BGu�'���Z1�zrE�h��p6��W���:�{��r���]�\�D%KGS���c{����*�s�N�@+}�[���.��P�id&q7/I}�69�J���G8i�v�4zg�5D��M^�.H�V�&����b���^y����]q"����@����-�������nC���K�u{���C������L�+7����B�aZq���g��>�V�<0���6K���X}����t����||�������/�/���qS7(������M�<^#�_N
$n�]J{��������<>���s���:���� G�S�-P�2�`���i�%kx������+>6(����<f�}��i�C�������}m�\�X���O����@�s�����A���7�M����,����u����������=���[9����������v��\�m��3��>���0���#`]��,q���i�M���XVoQ���O����pd;@�_���;�S�2	������,1�Mk��J�]]>`�m�JUpcZ�������������%���K�
;q0����MK�@K� 2� af2:�B"-��_�X_�4m�_��.�
l��_+��S�5j���r�+n6Z�e}�<>&/m
u$g�����,�=����)��c^+�vX�	�vp��Ze�=�m����3��}S��l��N�,�~��\��Z>5@���d����_W���R��:�ErS�ij���m^��-N��� H�yO�.���}�|�cA?�g8�*r|h
�����jjsM��/@�!��CxU ��v4`�%D���Qg��i���;�l9�8������`Z�A�`(�#��L��p�Z���t�8���L#,v�v]tD)'e�9+]�A�E�N?$1�:����j��w�xH������<�\3�J�H$����+=�E�yM!pH�����*�t�{8!&M����cI���h ��gI����9@��m�"��r����ro�$iOv
W����>6�&<ZHN��J[�����Z��7"�i[�bmU1�����85uv���E�L<XTU����}��Z��@�6�������U�V����B-D�k���-!���{�=�<J�]
+:3*�����v)]���j1	�T:I��'�����rz�h�k9����(�p���3�D�x�
���i�����+� >����$!�p����B
[���2�C�9����O��l$��Z���}w��9`Ds��� g���Z��1/�������}�P���w���R=���~fJ�o���8���
3�Y0��	I����-cj���4�)��|���-)�Dt�����������$m`�_��i��V�)lk?p����OfK�V\��m7�F�r�U��0���:�m������#���2y����k�anK[vQ���<8��f@
�B�ObN�O������V.o�y^��H�h�!�W��|���+ �dD�m�it����P���-�������)E��hy����S�-���S3(��u�sU��_8l��@�������0����c��������6�� �:^���?��^4�e�Wo�p����5����>)P����mV��@�?����S�b�]1��{�2�8M��05���j�K����O��;���frO��@����
�4��DQ�,|����i���7����f���C�����q�pU`Z�������H(�'~�=P=t+�H'����!����ARY�������	l��D(�7Z��5Z�D;��B;jO��r�H�6�"#�N��V������*B�	��,�;������'S��`r��m)��~�I��L���0�P��f�w�a�1��_�;��BS�+3��42p�l���T�y�J�\~��'���w��V��pLRr�Vt�s�u��88;���>��Q+�U�m��l�	�`�9�
N�����8���r�jXsZ��X����`�
����2.`���a��
f�2$j��������)~h���&k�	�1gV��`���C(�*7an�2� �@�<���q�e����JhO�_�g�m��"*�h-���-�4�{(����L�@u\�%_�(<�	�K�Ea�	�X�"�z�f�:"[k��l��4c+�� �s�#/��|��ED<__qS��FE��v
>�g(��uh���[�vKK�-g���6����	>�������\���/�#cP������Xm9	8�(��%����]�� ��\@�	��������A�&�^PZo�3�E��t�#����b�K��V=�vj�v�Z���cv%�&A�����B��"o��G���I���C�P�R��#��A�[�-��o�[	�8@������g=k���I�OL�N�G|k��v�!��w���{����w`;�9o�!���=p���	�5O�K��3���7��sr.�2��1��4I=	����#?@�Z�?`��'�kh�t����o������]����cU����m9������	�x�����@�M�c�@�=�e�I����e��{F�)6$s���X�5U������R!���(�p�;�C���M]c[�]d��A����.T4�����@%.0�f�>���Q��-�`��U����L��E�-�}����]l}*�EkS2��a:J���3�!S�������������\���Pq^���j�nn��|7����{*���2�������=���t��#���p}p���B�)
���b��������u�����[1���.'?��>m�1��~�����zE�]G���B�:
{^d �h�������vW����G���s<��AD��5e����c����N�l+�?�`���;cQJ�:�]�]|��������� %��\�����?N*��Q�����S|�.��F�5��������DS��d$�~Y���I�[��&IN��]�%�$��^�o^������DF��>��(�u��<���
0��|j�����4��[����c�!1v����X?p�ED��K_����OM��=�q��������r������L���O
l>y�)�A�4����{M��;0�!���M�����(��� ��8F4)��"���o0�R���~����������V���q��'��p8�c �DT65yuf��
�H��
	�����9��q=A��;kmBAc�YRw�A�0%U\��b
��b�g��9/�[������k��=����W�f�Q�	���]�'�M��8S6�����cg����}A���K�+@E�8}V��JE����>Wl>��o1=�'V��R�v�q�_/��~"p�;����}��R�7EI�����?{�v��c��	�v��k�n�@8�B����*0���o����CF���(�����8sV~���NA(4���q:��8�����������ji��-���C[��M�@K�k��6�R-m:��Z�t��Z��(�Z�th�����n��6��������ni�#,���[��M��TK�p��6�<���f �g����hE��9�h�q�kE��,��FH3Z���mn+��w�z�.Js�!��Ag���P������E[��2g��6�:�li3�������y-m4�3���&��[���dnK�i��6�y2���v�X��Q��-m:�Y-mdM��!M?��������4�
it�9
i���\C�^.��F��K���grv��������C\���&����(&��m_�a�n����k��8Nf6���"�d��a�N2�"�dz�yn'i5��$���3;�t@�:�H�L�N2CO�b�d�V�u'Y��$#)}3����k=���,��f��,��X�+�@����2�gue��,�����e/�j����+�>�0�i}")�>�����O	�\���m}����Z�H��9�O��Y�>� m}����>�X��������D������7�}�i~��^����D���W"������2a�#��P�9�d�aD
8-�aD�W��0��.�a�C��a���#��X����������o�#��3�����A4a9�;��
���!,�D28�����t�=�u���y�A����;H?��D����D�p���Hinw�s��A���� �`Z�;H������Y�A��a~w�k�� �'xvw���q�� C��� ���D�����:��tQD�����"M=r���w����"�����\���CE���cD�i�!���h���6�������h��C��SM=��IYk�����z���{���������������9>c7�`��-/�5VJ��=��{�<OIhj��B�pu?����V�\���������(Yh�d��{$�J��2[����9#;�]kX�+h"����X�8_��6K`zq`��'��K�`r�.�#d��U�$�V_�FS�<����r��b#��?f������fS����m��sy����!p?�Y�p�`j/-�EEZ��VE\�<9i^���PB�/�����.��J-����8/��W0/������.i�Yi�O\�:�0�p�f
���
9]`���X4;RDu�y�ed����1u��I<1�%��q�X[4?"������+����@
<"�3��I���E�h]�O&
���`��G��F��H�#h�pW|��rD�V��qAf�Y�D�}4/���|fz�a?y��������[������%mu�,/�������H��R+��m�K��q��o�w[&���,��	��Zt��G�%6�Q@�G�	�|O!��g1��=�>]�
QH
t����I���{��(�Yh�	���v�����I�I�\Z�,t���/���j�9�b2-

X���!������?�]Gk�Dqo�tEIJ��e|�2���>��������j|�� ��i+���6�McV����>du��Q������i]m�N/�HVe(�5}��Xn80�Z�q-����A�G��d�Ss����yW���{����<j97F�C����[����R�Z��:���"*� [=MxN8*~�XC��:p��h���q�Z��b^��>>��p�~�^���Y��"���!��b��(ar��MYaT/ ���p`���`n�9��WZ�@A|��C������\*��k��Q1j�����:�	������w���@H�K�F2x��~�@"��i�_x.
�J��@dQ@KEs5���Eskb}�'����W���!*���Y�-��>�l7�Y���������.0\�����w@K�����
�3�D6������(�m��)K��T*���pT���:$:�����lcA]��b��
�����.��jx%��4���
'���"c&��f��6�6k��Py�@�\\���5��M��E��� ��fM(z]���bD����=�E&�h+�C���EF����A�*s!jV1�6WhS������� ��t����[��]W��8g��p*��~�:V�t�����R^�G�:��v�EE�����Ac�����Y@�H��+6%7�

^U�<�TL�2�����6��Ynv����3;����{-F1���.�"�.Q��`�+�U8��T�c�+�+�����nS�9miu<�����8���.��eY����H����^P+��@�=bu0�o�b�_n��+&�l��04��k���i_���BF���)v����X���-S����8�w��l$��������A�`W�6��i����h�mO�L�D��-\�8���������)y�7�����i�����f�����,�x`����IS[�K�tj�i�=A+ZpW�1��K��m�E�t�\z�Uy���!�K����gg[�h������$��u����wu={������(�.�i��{��	�������F�6���m% !�B�A,�n?�G���(���p��wp�qj��f���4C������NM4�$W�f�b�;~-w�����I�S�@� W�N���-���-�����J,����z�BKI�n�vWz���i��%i$	��x����G]��<�k�S��vY�X�H!
)��.�n����e� ��4���_���D
�;���XN���}!th����~�������[nf)l��&z�^p����./,B�	Z�k�U�{��&KF�~�"
��B"�m�����$�: �@Awd^����C�h�zb�r1�iu��~o3�yb��r����^b&�!�������#�-�l>�����yM�q������nA����8$&g �s����=A�M���c%�p�f�v���-��$��d5���>i�v)��	����Ft"]V�s�����7��p"}�������X��9���%�CUB����G*GgxD4��ku�+�����I��;PEq�xw�S�#9lIas��>op�t���h�@�"��i���� "(K5�CUQ��R���4�~����O��@��2�
��q�F6�������G�A��)����w�I�"�I�l��~1X�*?��F$��u����{����v;��(�t�hMP���C��/@[�B�@�uV����v�Q���;�N���R��X�k�>M�lZ�Q������}��n
�9���:A����SW
�8�����R��[�j�bl9�m�6�I���q�`P����c������_
�AEp0����b���@>���g��,���~o|)I�n
�[jS~��!fH	�������M/s)	-Q8����@3�,$Tp��x�x��M��|�q�
h���������q��}���;K��X�k�b�m�t+����G��'�<�X��u:�i�(���mJ��j?�r��������\�W��{�*�����/�>�������6�������������*P�`�!�,���|���?���3�}�T��5_��s<�@Ru�Q�QyM��*��������0w��_����}(!#NA7_����`e�r�����*��Y����Q�Dk�8�F�O�vuC�f>A�+_��"!�x\�$����V���s��M����`����h4"gW�	�8%���D��$�����a�;���}�F-��g+4O5���h�Pi�i���2T��JE��
U�%z����B8?��[��y.�d�n���2��(�Bd�&X"����%�jUD��0��z�<#�����W�&��70�)��H6��=?���V���R�GK�T8���1��R��$O�|2.�]�_
,�9���}���4�>�V>Ro�e��5�������Y_Rpi�P�@��|3�����t)WvA*��v��hu��v�/���j/7�67���d����G��I{��U�XA7�7�(0me=����vcM�.E���4�}�DS�H�bG&�����1l*��:
�\�����3i��45���-O�������>�"�<���f	WFuH$��E2,-��dF�<@��1"�d�h�E ~o������4����]���t��o���BZi�&�4�XG�(��^�R�x{h�� K����iv�rH�;	�����;P��e5������F�m��'+��V�5M
�����L�z��7���S)5h����>��������i������"��+�`LK�������W��U�|�1��W�z�g�8Lk�T���q*�N��0����
.�HSZ"��,2Q|���%l����#�6�h3JGQ�-*M��J�}�����������q���"�Av{��xz�oLgE,�Q�B	�����H�Rd�Z��A���C�~r��Pl��Ae�E�tIF�N-��{p�|���<�A�5�p�T�������B�<���K�&�('P
���d�_"+�TP�.��V�����
�gIWl�&����4|{���=>&zPCE�5��:����7��K8�2j"��I]�t�--6��U�N�1��F�i��o��T��r��0A������x���)�,Bv03L�V��^SU���XO�]m��dI�����z���hs��r"b���G'0�T������C�=��Pj@����U�)���c��m����]�ywQ�h^i]u��/2��]g*E��;z��]�?���vL��P�+�����l�U��!?�3�M�2�^�6��+����h^��i\�����������13�����s�����	S�k/Xu�����n��Nw���p�`Dw"A������<p��<�gx��9�����(��(�MH�Lta�<}�T�>X$����*��X�_��|��C�u�CzZ\a���j�S�,A�+:�\����gM���pJx��kUZd&3w\u��T$$Z'){��c�^}}+*MT�����{Bn�����,����'?��>�E�Yy���
<��������~{86�T��dj/��ZXd���;kI��xja�B��[�V%��#�H���&�@#HI�k)��� h_<A3������D���N��%��@��Y�FNd<���$�?=d��*����An����y���9mDJi�Z��o���Y��YUB�X�����G��}!����N~x-GKt80)i�~l�\)��	y���6d!&i[V���(������^-W��v���W�����H�������O1���W�w7�>��.98���p�@������XG�6�QN��T ��)BmU�X�p�H����#�`�zU51�_��?f���mR���S���+��(�R��f^�GC$�B�
Y�c>�����a��T�/�����{�{S-�b/	��;D=��#����Q��!���0=@�~2�3���vF����.��-g�#�����b{P��!��cL���t9c���-�L�t!�M�X�R����.u�����i
�\��-�/O)��T,?�5)�H���~�WnrN�����\��CC��>V�*�(��J�<�S+�f�m���~K�t��Gl�*PH�!pN/�ET�adV��R��;�)�
�b�����s5��4n����W�.��i!b�f�v���5�F�<��l�����0F�p�j�0�����.4����0Z�Z��G$���I�@zV'n ��s��N:*�p(���H'G0;�Z�>lq��L�0����3oPgc�5��)}�FG�e)���b�W�+B�-�(��`��Q�;���^IcM��k��$C��D���K��9�k9?��6�k����;O���G1�ra��!�x�U�P�5C
$�g]��Tu��F<��~����!�7�0�1���uH�_T���)��K�������*��r
F�T_f���V������.7����A�o��+<qd�d����y������!����jS' L5/dP��*@�����mZ��q������V>6:������f��"��Zu��4"UqHL_U���G��WP�A��j��P�tSf���A6��x���-�����%��)$f����0��uW"�	HdW:��hM���v�1l���7W��Oz�����K�A�+P��J���x�D�o�|�n/���=6.@1��HB0(�f#X�Zg��PA�2������,��z&=����}�V��s5�|�_���!6��>�v�����W�W��)��`@T�0-�+'��M�i�<�"��L��Sx'��lP���g��4�1_��H���D������
���9S��h?�G�'R��*���,�NZ��U�7����o%]iK��-�yt��u������Z�9f�y���'Rq�
�j���V��E�9g3�3�
��G4c�*0���]gNRv��v	b�Q������7���e~I3����8��oN]��lH�yZ*��m
Q�
Tz(3���~��� kqG4'���C^�T�S�w�\���!X6������i��S%���z{�L\�������1k����OU��B����G�����H����x	V&�6���*0�OP������i=0+���W!'i�c`������t>*�v3��A[8���G+����Wh0�,��Jm�\)��F8[��@�2-A�}�<.��[����c
<oKt��~8�q�^Y^�X�DB������,�������lA��s�����V�-W�hwj�J`(���O��S��n�X�h���S�-����I�����=���2��T��
�q&�c0�~)���xR��cP���Hp���n��H�V�p3���!=���?E�����C%�,���Q���4t�\�2�����X��;���Sl������PQ������h���x0�G<7	��T���C��E�H���N�HB'��?�]��Gj"�z+���b-5��Hnw�S�
�����g��?
��sIn70.�d�]��'
W�;�M�3���$R�@��T"����R<�qo��;�<�*@�DCIA�Bq
8���W��i6D�b��[�K8^
�r��#
�R����z�:�3�
�>���V����I���YwFh�<k�6�
������y
}�
��{1@=DF����a7������	�SN��,�.���M��c����V%G�}5���J����rd~���=)���8�hj�����Q��pQ�����n��+&g������I�G��#���o�`(�Y�W����/���TH���x*tn���2����"x#	�87��M�����eK�Lj�P�=��GA�[���%����6+2���.Z�l���{u����P�/[�_�v �3c�L`T���g}���s�b��UjG��]���>����>���c�=�s�h��z��V�z����[�e
�8��{_�_�#�9EX^=>���i���up-�<?z��g0�X^}6_T�F������k=+������(
(��������k;����v
��u��)�:���[����8�����@JKqQ��U�s^�}�G�YmH��.��5�Z
�J�UPb��+�X.Us7�a��k���c��y=�;�#vh��!;�W�|���_�ky2n�����5t����JB�~s�`v�
�%�i�����l�0����?%$0����������#;���^v��MT�[6��t�+�4�����p�t�%���v�����m ��j��>?�+�UrW�l3��+��C���{vWT�b�$�#�e���EY�,��
�6R��v����u��'OY�rj�qd��/��D�h��J�nc�����!�<;b�c��\~]�HD���}��_���D��LwB2��8	��8-`�4�i��@1-*���]��{x-�ql�\~���kW{m��%'�����K��d��(~/��r�1����qw�+���;1��)�wP�qt���[x���W��9l��0��-95�;i�=�a�\HV3[7��qdn`����u�zW�C�!����5�q�m��3�my��C�#�i�8�����x�[O��P��^���(8-]��Sm�H��lu���>�p���'
��n�>��8���M2�l���.�TG����p0_!�n!���0�%O��|Hw��]�e��cYN,'0r�������V�Cu�0~�fHy��Y'�}�N��F��X��in���^4����L2�8o���YB�>��>3����K��,B(�,N-���E�0�������G��6����]P,r���\8��.%�\nv<n����r����S�����eb{�jM��U�����D�8?)]3��yB:?�s�Yt�/..fR��m��\F��-�����x�^}V�|����S.���zhc���(��
������+d,���P���!o�!�0uLI�s�{�)��
���P�#\�t�z��4�8�"��$]Dp�IJ=?r~*����"x�i@��,U���a�
�,��=��3���'�U�x�@�k^~>�l�m��i����`~��9{�\�a�$������������\7<a���Q`�������p�����f����`���Ic���9�����xf�nz��mK��]��k�����{9��+����g�n��pW�;���<��+��p0k����x��g�1���	J:z`#����h�l�FD[6���Pcb@��P\\8����%M4��+�@`^X�B<���*�@��!��QiFT��'���: � [����$����[84��)i��'���hA�4���s��l�5g����^��M&���������u�*�h��7k"���zm����5cb������s��X��i�S��������l����D�VX��^/"�w#d�2����O�l�s�^��`&
��6������OU���vN��h!����D��a:����qt���b_���@�.v���_�w��hW�g�����c�n��.
�v�����D�K�d��������5Q���FC�8��x0i���M�����C��C`�pVD~��]C�8.I[[^�� ������7�)%Z�����6�wS[�!U^A-����Cz/S[(M���)��
��k
�r�H�5.���d��=uf���4�V��^o�C�j
[g���/V=���)O���'��:�'GA���G��cj�������	|eN�h�g�uZ���(�X��yi�@�5Q_d�\�zH�Jgr�����y3V��R�G��g:Z��6��EQ$�i������~�o����	�K�����HB�`$�[������k- 3�p�;�:R �-�l��94K;=�Y��R����>��3z����H������^/|��o`�?T�6�0��=Oo����|�*m�Y{��p���t5p���~�A�V���q@��
b������1�uUVT�������c��M����a��*�@k�'TC�P��/ch�EG�H���s��<u�t�^��7������h�<� ����������-�"�d������lSV[D����i�^�hD"Y7�%jL~��0&E5�����#-$�%�"H�p�����M����V��H�f��
O�73����4�-�q�F��z�����Hc^�2���#�H.���gQ+��1���_��|;I�af� ��R'j�����%�,^t:$}h����<3X�
�eBy�Gn'��mv�6�TI27�
�������,�'��^�v�n��d?������XJ$w�Ci����ye:c����5�`�(���0���zZ������lh
�	������cV�d�����u��<�N]1���(xj�	?v�z�))�7������Ku~�3��{
�cSm�A�HH`dN_X��+�2����&"R����M(Sx���`
5t3�q�1��cb2����^�P���;���!X�aed�.dX���y��Z�*����U�	�D��Q�j���q�Xa��H����	�'�#����v�/dR��HS��|w���w5�]���������Q���m>�:��i�<�&Gr�M	�QO��
�.M�6D�/��j)�����
/lN��L\�N?���cr���&�b��������1k�J�����y��E�\K[�_O�?������I��}�H���/�$@��e
�
V*-7]Z��y�:@������Y�Gl^�.�A-$���2��r�4l�����y�-����'l_=����)�[�So ~H�/%r8��H���6t=����)_����a��:��8t��C�u��:��xP.���D�K���_�����3��������
��P����_����V���6�?�M#�'$��6�����/�t�,��[�����|�x��QYD�d@��Y��W'�z.��A�B�f�9x{#r�u��>}/^@��_im$OvG`6��	&yTD���QXt�V�<7L3�#��Ny|���*�/;�qW
�
Q��^#������(������o�`l�y�#;�2��<C�<��
I�Az����5�7�#����4���KG�	i�h�K�)obb�v~�.*���k���<��������(��+�n7�����[	���T,��U�.�&��(_$�x��$�P�w;&�s�����u��AqM���d4B�������um��2Q��*%e��r��sEt������b����J���s;� iWe�E=��\IWA�C.)�p��������=����V���v�h~�QS����>?��vlf�����vh[.�$��$y<�H��,R���^$-�ok:�:�Eq����4�E"��9�����+�>W�2��g�l#�nU;�IW����}_���h�6)c���\u��|<O
�AF�S�����Fs/����E�Q�;�2?�%EJ��.F�-A<@�S�Op�/H��%�s�k��B
�h����u���pd�@�A����NIh%eQAX���C)P<K;��d(09�����'6������#�R`��B/����|�5Q���*�+�9��U�])�a���i���N����r�����jN}F�w�J�C[]G�~��D��}2#)!��u-�G5���u�����V���2��}`>�dCfx��At�5���_>�b�U��r6t7=&G����F��4�7$m�;�H2Rd��FB�lm�fjS�s�����(�@��������������-�����`���V���������^5��,����C#v9��Vz!BUb�%f�:���y�{H��<3s��]�����_��
�����f;� ���3�����w�����F��H06S�.Q.KN�n���3������gT��r�l�.���C��d����u�~���{<��t�ot�(Z���#���&�s��@�^�G��I�^��X(/��6@����U���B���z?Jjj�����]��z�?�<S^
�h������1OA�dNZyB�w4P2���jyyF4i��0"�s��T���e_��55��!����n��2�j�W���i�*%���2&_�b�C�Z�������dX[��
�'��Zq�Pq��]R�$A\�����I���`�j����[D]�g]��)le�J]���%�"�(3��e��9������7g�����H|�N�+8c���'�� ����Js��~�w�'���(w	�����?��Vw�=Wb����
�%�.��ux��F"���`�pwiyZ^`D$2P�=�{L`��P_�VF0sf(dtkg����7���Lj�<���C��|�_���5��!���,���C)'1u�-�/����#����8C�6�����z���������Z�_D���]*�-'�q����eb�/�����8���1�Z,0�����D�Q����i�S*�C�|�����cH�[��&��|qiKE����c�������������@K#]���u$f�V��k��d���D����C���7�K���}�!���Cka$A`���g"5�eZ!��L��8�Eg��Z�y����S���������Y��������3J����M���CV0P6OY��n�S��<�/�����&G���v�X��JW�m�5��0wW���������[�(�?2��W��t���X�?D��U�*����
5���5d���*�K��+�=������0W����L��$0Kv�q�����Z&�9�����m�Z�.����k!��/z��s����.�J�7we����S7�z�G�d��;
`^�!�1��3�UD�/D�M���M%S��a����cf����k�0���c+R �}b"��}3@CS<�����v����]w�o���I�"c�O>W>����+v��j�.��{���u'#�Z$�]/� � �`��SF��;@d�l�j�:������pMK��Y��b#�-������^68�&����{f�l3��\���O� �y��p��x�g<�7q������&�����dS��� �
M���=)�X�uu��#��P�����z:�	T!�����!
��|l�Zv���=�(d��=�0k�Rk�������	��'��n���5��Yu�[�[,���Q�<-N�PV�3@A�)��Z�@�>U	�U��%r��V
������~_�D���:�\��c�G��l�-U3���,t������� ����D���������^���L0���uD���E������	����jl�sWi}�^})���%�8$��'�3F�50������:�k�C���� g���q�+6�>���
2w~8�u��� )���Y�����!]��U��|?w#d����U�K�b,������94*�
]3������Z��c�=3(�n��2�O��.�GEZ�!j@$��/�&�_x��J�����XFi������_]��I��^��z�����T I���;2��dU6([������.��a���A��iu_�de`���{N~�f=2�w}�?J� ���:�X���?/i@��zD{�����p���+|�|�����!��C>)�Q��)\8�S�Z��a�9f{zx��#�Yl�!>J��==W�Cj���/g���[�WH���U����������&����u�XL;l�	��Wb�|�}�rW!'O�������x�Xh�g"����i���C����GC����r����x��6S���
�T��t!	�g4i2��o�p�2�o���l5G�@���Ga��e�-9��G�����Y����<=��o�f����/y�=$K&H{|)�K�RN@��o����i�0��F��o��&�>U�n�U��=���������ty����I���A����������w^�we��U��f��4��*�i�1W�����X�?a�%�����C����!i)*���������5o��M��z&�-��6��4b�!PZ�_��I��!��hj�Z�rHkW7��$!U��~�)����!��U�D�^����o��_�Qn�j*� �d����F�=�m�!=d�h�����Py�>,�):�@����I-u���vp��R/8�*;�� 6�l��/�y�Az���ej�z#�2�~	�}+TST��lH|.6������X8A�W���aM�#��X`N{�c�g�3�	����Ye��g��~R}y�����9��l��-�7i>��!Qc��/��
6w������Q&m��G���<W����=�?a�z<f|�JL�U�Ws�K����'���:��hWz\�*U�#�����`��l�
<�h��"+H9���kf���@��/;�B�y�k9�a�@�u���R|z ��`/�W=���m���|����^��F��1�If@��^�l~v���kk�Ox��2U���z�%��^Y���(��o4�J{���+V����3�lr��r� ]�p>��BI2��lh�!�--��8��4�N�TL�����d����<��V���[�#����s>������g�<��j��rU�a�_��M��OlO�����F�~�*�/�����.��5rKC}{���6�r������9���L{�d�5���p��8�}�c��eoZ��>�������[���N��t�Ar^����	�C�}�L�+)lS�~���\[�c�UA�����Y���:�A��@�U���Mc'���C���T���a
q��3��~���1�NL�%i�����
!���]���|�����	&
j$A"����X��#�1��������4s�x6��!J_��m�)!�n+���9�������!a��������.�FO����{j�?��KV�S��Qd�����������]wE�9�.I@#����@�g������*�@G���6"
��>m]����-�oy�����a��Gn��mxM�r�fL!�wj{|���)R������(#�]R�v�T��[���6M�8���������i���k����=�����%�a�^R��Y�d�w�+TC�6
�����SU�]�����C$WxH�l�uO�������D5���O��~�um�`V���1.-���		��c�0��>�H-��MP��������RQ����}�Qk_	�{�5�D�:w��i_O��]�H�I�%��"��d�~{^(Cqf�%�E�$
��z��@��nM/Q���O|���zD��:F/�����1�e;T�d��2?e&z����|�v����B���1KyCmgE�~U����,A�������K*��~���I�P�
�CN����m������%poz�d+J�]�L�;�����w���`n��3�Z����eW����gGm�<��1��<I��O�D�q0'���e[�6X�Rx�����.��'T����l$k@nr�&�}~������l����/,LW�b�W�<��D[�8X���Y%KxZ� X�l���
LB�ld��+������w1a�i��z����A�{�����b&��w�L��U��Vn�]�0�{{Gb9��C�H�����?n����y����7����!���#w5`i�wR��GK����e4�/$2���Z�#6�6�o�k�Z��m����!:���J����4/e���&��r�����Hj�C1$T�	:��\U��	�����Z���W�9
��_�������a?D��nOU�n��b����l�]Y���h��W@9���Z��y��`;&&$_�
 �0���<l�(.��FB�[�������:�0����Lo
���#��f�����udnK�B�]��>f��0P���o����Nx�����Vp����b���&����� �Y�0��s	����M�8���0��#��W9�n���Q�-l�<��Uw�K����������C4��fF�����
����quD2��*����o�m[��d4�$x�*���_�0��x�$q�]�����������WW��+4l94�4
�x�?���Y�������������J����&���Y9��8����(^��&D2�1��8��M���[^�M�p�
�Fr�	X�iXu����R�T�@X��S��
b=���(�#�����JAy�]������R[�����0,}N���9�����G��
Guq�SW{5l��dG.�#R�T�^�Dg�����+a���������������tk�Tx�wD���k:����gv������D��:^ fc@H.�~��R�����E�9����r��p���(�_e�2�k�s�h�7:�_!����#S� #_�Ui����r
�
<~M�[�L�C;a����`��L���8d!W��t�	��AC�~�0p�Dr��������`���2She���$�h���7������������R���7N�xd�{��0��R���hAq��|�������>��tJ�]B�E�}�r{_�
�����C���EJ��vn�+�HT�8t����H�I	1D�����r�+�#'3)�i6�,��"yegH!;3S�H���
��#��z��\o��:\c�j�7C�����	X�nTX+N*����zN������(w�Mk-�J\�h}}����(�]v��<�I�y����;b,��%���Ms��hYkp���`����F�R�E�Tx�NLL�!��R�2������Pk��w>	&b��EhD��,����m�%��H`�	WK����8Z�e��e�5D����tW���F�S���A|2	���<����\8n����!&��l����A�w���~4�s���\��7��0H��]��K0��6�	a�`V!��H���j�S+�/��u��u�ja@�-�x�"��c1����Y�
�-p&��_\.
ey���%�z�y��J������"@h�gv] ?_c�^�������kW_!WT�4�1�ki�������&X
A.7������h�'��G�w�.hf5`�l)�RS����4v����Q�p�5�f���Q�`"J0�
%q���Uu:6�Fr	.,���yO��{�6m6/�c�f�b��>���z*���J�3{� }�l���A����3�'�[������[�O��V���
&����)A�o�����g��c,H�E��	���������q�D�
�0���2a��o@���"	����n^�k�������/��I���^^�n�1�36��Sp�`[��Y�?*�zPP���lV*������{���9�{m���R`��1��%A��}�#&����;/~���V���L0y���H��6Mq��a����GJ�fZ����Py�X[��b�-xL�a�51����gzs}h��m$f�E?1��E�a_�K5�@����@���5�������1�X�V��U:r�$3�h��p��w��J���ik�u�AL���,��m��!z(���6^��-��M�5��%��U�Zc���;!���	���X�}8�|����(&��"luQ���������]G�/����3!�$f:+HI�D�/�Y�L��Qb�oTz\����+4����
���+|�
�C=	CzXz>���Y��K$QN�zB���T3�F<���������i�5$^�s�d������A��l3��h�umH �yVh�����r�
��{�����U�Y,s���%��D<=a#Z	����?��\�:�O�.���L��tW
�h�.2R�Ag�<����X�k+u���tb�9�����AZ�gm�e1�;\�9�R�86�BH���3
!=�������|ac���4Gb��i:#�n����h�2�R���4�z/�w�l����
!�j���p�Z_�j+���:���z��(=k������j��#�B��������k2K�l=2��ye�F���m
;�v�D��Z��1���nPQD��Q�I���k
���ELLN�*�hy^=��mv�t�]��]��"���.lgo��n�K��Ez��Z���@�����z�l���)�������D��v�i����fLQ�~�o������!f���q�w� 3[����IO��U��j�){�����8�<���dEj�����6.�o�?�"������)>��[8��htVzr���DZw�5��1M����;W����~�����r��b"�O��Yj�>A������A�?�(4�03���z
���qt�=Q�8Wz�zj��S�]O
��B���#����'���X��-3m�;F��	�d�w���{��D��9���V�oSt_�w�]l(��po?r;:�� Zrd�AR��[g��mJ�~�F�@0?DB�h"��?�����z�W��7ar|y��
t����sh�Z~����k�!����;�gO_���bl��Tg����������}�+nkhT�k���'���Y�|=��0���%�F���+�����d��>G4��J��[��v����+�Z:���Uh�DL�Q����y���m����
T0,����"88D�x^�*A}*�����T�S>pz�����d�A�gD��p������q0�mh{����T��S���5�-�6�����y!��R75V��A];�Ug��+�u���+��H����%s�����n��<���eB��	=u���8.���V�Uc�XR�E�mO�6e��6���L������M~IZ��
Y����"b���I����R���H�d.�R������	ir��&��#p�:�)�8�h���@]�������QW�7�����>=����j~,���v����z���P���$�wLn� ��/w��|J��*�`��IY�z��bH�L��>���2�}�P\l�iO&��4��U5�rN�m��'d�kdP'�3H%�-���9�
- Df���;�Q�/Ai`�����	����]�g��]dr�?�$��^���H���?��I#'kG��l���F:gkE����?�e�
n���_�M�d�����U4��B��k��A�U�C�X�����Du����q<�����*J����������I�B�2������SY�3��
�����5����hsQ�>�1�/�XL�!~�����\��'D������:,`�gh�<)1��I�<1M��]?��g����V��D�����c����!�O�f��N��#��b�b(�X����2���*���Bnq���7X����V��&���H1K�Ch+u�0+��o��D��i�@A�$����S����oJg4|�[w�������U��������qC~K���c�(���]G�'��m~8d������In��@d��;)'��u�.��1�z�:{�������[(��3�na�'�Q��Q��HS��\���l[h��j�y�$����]��&��=x����kZ���,���_�iA�nh��P����^H����M"zIEk�G��(X��@��a����1B:c�$�d�$���Z&[4F5���l:�@��-doL��������l�$E�Q�c��z}��/�9*�u�l��K�t�z���!7�����N$!�VK
vu�P�:���8����~��B���<��U�)������~w��D�&(g���?����z@Ev��`6
�Pw"�����xf�i���
^jSV�k��D?+	a�m�A<1orp��y+#.��������M�v��*|����QZ� ��(7eJl-��cx���$Vf�[���u)����N�*�5�(Z��
��
S��y�aM[-�-F�~�c��G����hb-��&6���[�� HSs�t}��{��m�|�/�S�	�Z��r�����dPh	pj�"���t��oL 6I/7e���j������ h�
d�e�Qxk��������`�@+T���9�;e������>�4g�K�c[L���6����(����<_�j��[��N��%T6a�[�NR�K�Q��n
-q��$pT'A����M$�x�@���^����F	�����
�����[x/��&��#Qx������j����{K?��:�����vrV#�Cyy���|�I�\�&��'&�]�NsF�x�;��������3Rr�v:n�B������c�Q]}%H#�=�M��CQ�w����Ax�Q�H�Iw����i/�)���9L$���>�nv)�]"6�gDw�w�k(��z=�Nr�d��n������g��0����%g$��������������K]�a��E4WMh_*X�)B,��a}���[�Y�~j��e;``�(�*����d���;���=����.�b�u�k��w5dN}���]Y��8��*cv�:�/Mh����Z��g���J���f�B����y��yZ,G�x���
����I#Et���.����}I��/�,����%VL�����i�]��C�v�z����Ox!�)��v{Sl+a�in_`����>�:)��]��0&�Q/Q��&�0�7y.��LzH���m(����h�������������&�����c�G���?&���S%�s|?��?�X�Di��x+w�Bk�@��w_��
�R���!��BU���*�q�02�>��x�\Se���z�
�Z7�>��V�\�g�T�;���>=%i��_��f@R�������@��.m�G}�	��i8����e$x��y�'���o�Y���"qs�6���Q��B"kB0���h�2TPxH�M��L%����0���m�������H�O�m�:r	O�V\��7�x��g�)��FN��^�I2dz����]n'�j�\us-���IU��	�a����K�NM
mK���4���4D����o�~�#�C:&�J���V�`5fyd�
48���d�-$�J�;������_��u�����Ob�2�DR������3}�r���DW}� 	�.��+l�+���^f+i;L�[/o����Rko�|W1��Qo�m��t����5E��N�������5?����|.�����C�=�C�_?���DPN�Ec������+m���D�H�_7A����H)�(O�g����:Kx�A�<�b0��R�=���Q�� ����\g��^e���i�d�v�Q���*�G�8�P�Bp�V��!)<W)���&������N�0�������)��]%v^Ds�h����`T��D�]��
|�j�1���n+�]����~d>������/ST/6�R�[,y���4^,G��Y��T�t��+o����i0�M^Z��}4���q@������sB8�*�c): 6]TZU�*�3��]Cl�k���b��R��
�U%���Cv��iL�O6-�=8Z�L��9��q������
�]'2,�El~C�b�nL�.�7�1����F��
X�5�1#y��IR%m�Y{km!��{,��(�	��}m���|�p�S "m[����)Bi��5���S���7n����R+K��������jG�SF��za��M��<��U{�;�/��}�5��N^��v)E������rD��I��Z��N����K�r�z��������5���B�o�� -��:�F�E!�@�,�2���n�2��z
pK�yf���-;�n�[�3:��bh�$}��{?^@��������v.^`����������'`v<pd����8��<B��l�������)q���s��q�����J�*�����4��I�?��I����P����4��W��� �$���4�w����Y������g�����:��A���
�A�]����������-�0���i��y�vW`���q	��I��5��������B��}��)k={S��?���M��5|�?��fb�c����>5��[� �]�m����u���p�+f&����x'����}p�F�e<4�v��T~��:\~�CO�be��be�t*\��@�^4f:�1�����iF~��X"����}M�G���Zd	zc���C��%N�	��pP_��t^E��2��U4P$y������3�1���a
�3���AH���M-E{43s��L���o���'t�d�����-��$��P#���@
L�]
�G���A?ZZ8�����I[��B�� cC��a����z�-�7��Y���-�u���He�Z��U9���2��l���<��$��[�
N+������
��R��������93�9;/��b��G4_,��z��90z��}x�������>��-��S�V�	&c���i�=t1�=6���0I�<�t��u�!>;�a��?���$
(�'�?��q��D���?"�w+_$��~���`<�]K��#�J��jU���z%����*�X?����=���So�v������
^{��~J����G�f<�r���?��8k�V��	�OF���
�}�J������^�du�C��/���������%y;�}��S	�����t$���/W�UE|H��l���6
��PL�v��;;���)67	;9/�[!�rp�
]�oQ��z�CH��fY�ri[�Cc�fJ��+�1�8��S-T��K-���!���v�_R�����
�I��H��/w�!����T�;.5�5�QH�q
��]}��V�<b�7�Jn��^�����a�$t)������:���x �{��1q�X|����8���x(�G��z��T��RHK�GL�TP6��aX�Od+�-ng���|�`a�	�O����'��v�(�+3����C^4�q���$�C{^{����.���~��V8z��6�v�RX|�%6�SEc�6e��7���k����L=@�:��r���+��Q5�QJ�)�y��\5N�B���@�����0� ��K6���v�(�S�?O�e�I�������~s7KL2��c$��'�Y$0�N����:�$G�CL�S4�������Mf`���i�l�{�V/��
I���G+r�U'�&��QL�+��(��C��������+8��b�H����}y��J���.NL���R>�&G�<���y��)_@�D@
h�l[�m���/K��`�sdU��S\���������%N��+�X���`����P>^|C��mi�C��az�T55*�d?J�������$�#{�����J�m���a��b�`��5�*W���y-RXf����������<�
7��.��2c=�@��T��k1m^�| u"=��e��t�D*`��
fG�84�P�Z�HX��*��Lh�:&�u ����s�vd�P*z���P���H\�<m
Hb�[_�>��x�(���f�5�@x�
���1t_@�gAp�jD�k���
��\����$L1/���T���W����7y�;pd�Ia3;<���X��U*C�')�Mu��4VF$���'
�#�1���IRi�}^K�W���s�)D�v�;�PED$�%hSs������Oh���:���,����N���1�""rZ:|Z)���� 0l��GN��aA�0����x9�}�-��a�4Q��#,u��|���
����>>&:�J>kb�|b�V�V�����5�Y�����X�l��\���T��AX��246�Y4lP&�x�{<-�\.A.7���7'w���k'�}i��nF* J�j�C�@�m'&�{S;;���e>~\��Q�#����2�����8��UjL��0�6���@��.k����!�@�R������R#�MVi<��|�d~|[�x��v<�
�s�P�$6bYU:�?��O��l����s�{������$���������H�]���#{���S��%�K���N�{�2`&��u��!��%Q��_��w0����#���F&b1��*��G&�\.o�+G������U��w��N�t��<c4��H����]�n�[����s�t�^%���E���.����p���Z������!��2 3�8��]�A��������^�s&��nvwn��,`=�0j�����
�Z$\s!1��s,N�����}��o���Hy�L1[����R^�����OU��e�/�./��gELc�C�d?
COx6x9nOu@��!��W�k`��+���4���
�@�3B�Zb/'y>fuy�6��w&��9�iT,0�u����m�o����:lO��S%��W�5k>�=�������V�����{��'*Tq�7����c��a�aJ�(��l���1�t�N@d�d*���^g����GH��
����4��G����2��f�������Imj�x��~��A}Wy=_�����e(h������V�s6��]���8��m��ey����4�m��g��o�j�+�Z������KC�iF=rS-��������j\��c�BG:C���������F$��%O��I���9���8R��W����G�>�}�l�%K8��#3T�6)w��:�^��a�i��IHK�$�[�`_�?�����gl��������b�4�;��W�|1��h�y�u �W���2��!R>��r����M�*[���/%�������G1��F�����-�@0�@�ur���GM��qh������i^���y�n�o�co�Jxr�>o�R%�z%��&���m������u��e�'�����!w`l�aP����tq=���`B��w�U��a8��`�Y>1���Y �N���nA��r���BIy�@�K����M�#aw��E�%yWE�s>hs%=~�i?���2��,��J�dBD^��"�-A��M���;�,�$���vye`U������y<=��bOG�z-A��~�a���P��(�1�J�����	�	e
o���w"�Q�r����'�u�,V���EV7H���d$�������<�~��h}I�d�u1�5`c�`��`����=������n�5@�MDD�A�%���O���C��o��WjdB���8&7����<�el����@��m�&+xi
�E�
DA66�3�V5�����P�#Qf��������
~�g���m��M��.{��1Dl�T=�h$l_�Q�~�)/o��|W�` &�����|J� Ek���>H&>R8�1��Fd�F�O����RGc����uo)���2�K����?s��xu-3������0��J�t~�)�����g�������l���<)/��d/�+7�o� �B����������h����t\f=�])���T��(E�!��/��GO^.�9�U�4�������xGz:k=ai���H�3��c�dact��="���6���F��������a���~bU���2�JK���3v�0:���V^�����PiEA��VI�11��}��3o^0��-"�E��b,��U�
�x��E9�Z^s��jt�����ao�&e���E<3�yd 8� ���W�44�������R���p�1�s8�w����.{N���?��y��E��*P�����b�;���9�� r�Zs��	����*�O'rN��Dp-	��2�J�8a����!�2j����$�c_b[��]��U#���t�}������Gm�ya�f^��#(R#�L�������\G��v,W��R���K7��`�]2���F�����Ef^��\�5��7�v��������� ���3r��>�J���g��&�����fW?�p�]��G�C����E[h�����~�C;swYiF.���Z��(�����42���_���D)[�\���vgEf7�(�y�'����5�D�P��4�6e�ii9#b��a���,{��<pP�����]�/BS��A��OM_��7�L�L��B�����o���rE����.N�,�����]���C��Cvx�*���L%�IK��fR���,��s�JL��0��*�����I�Oh�?hF����(�n�����������u��/�0�uO��^�mY�5�Cu��y^5�W�Ik2p5������;1?�3>s����? I���+���%(
��k��@��lZ��(P���i���!%��@���]�����>��(	��9���k�����SQ��
)�]�����x�
6��V���
!I������8����6��F�`=��>(�Q(�o5C���{�lI���2$
Ov]�~��z�O��#Q��{��������WH%���6�PL��v�mZ:��v�U�Qd;�4:R�5Q5�����0��T��nW_n�>�vJ��D��L�c�"d0�u��\4`}(�=��x�O$WK;m'W����Zb�y,-c'�4
�$s������G'_C����	�	�M���"-�n$q�����t�I{Ax-S��	��S��M�c����Bbb��#Y<}0��<�n���4����a��,��I��'��������+���HB�N�L�9'���_	�]4��PG6?�?Z_��������By>�d/���fI�<@��{���hG�P����J���	{������K�i���}S�U������fo��{'9�V�4�j���,�R���^���0�#b���V����,�>l��!���;G!QWw.�M+x�j8��W�������t�tzf�(<[P��.\9�[�
p{$w�~���x��'5 .�.T
��Y��y![�0�A�aD&��!��C�vA��=x��m��x|�/X�uw�B��1��c'h�V>C�*���b��F�w'��*u���~
s ��������I���X���>�rj*��n��D��L���W�����-A�3@{s:����#��z*�������{1����U�dm''���"��W�Ur��W�8u�e�~;�P�������T%L�|�"gP�@��g��Z����TN8T;�]�'�!T�cm)G�w��(��D�:���:X��u��fF��
d��p�.���K��������-���4��
_�s�����H�V��^y�#���@�����]9CbP��0��3���J�����t�i4qXS��U�c�AvLK�5��t��~*�D
���v}�u��]Pn|�H��8�?x�G�q|��(RB;�2��c�~�`�?��|��#������Z��M6��������>;�x�B�mm�=h��4%��-w���J�����<AC�x�;�oH@X�|	�]"RwC�:qW�)Pq��9�T�����6�D�����}(���
1MG�fC�p����v[$�H��5,s�E�%X��$��76����H��� M�F?.=����O��:A;-�ZI�
PA@	;OM�$�m��s�4�
��c�X�7�b��R4�7�s^d�����D!����S�RV�?yi�*j�K��"k���|c�$V^���!qG�5����q^�<{hZU�+�D#���<��R����j�b�2�\T{���������$�������|�i�q\W���zz�t�5����t�a��,��'"��G�W�m���{Y!�zD
y�����Y��[lMr/�w���: ���*��b��������~A�`��%���T�n� �����6mR���h>/X�*^-&E�&$00��Y pV�M����CR�H�hNe�j{�?"�
��e;mm�[��
��o%l�}�I��a���Tm"��|��k�g@#
���,�[L7\��4$��7������q!�����0�l�bw�J���C�k�n��g������p�T�����U�9�K�����
Z?B
	����vy���6#�LN��|aZ���]���S�'&��i+@X!����j� ��J��/�j�<GE��A%��@��W<"n�`�6u����|�d]6Q'LZJ}Y���b`z�F?*4�
lF��p(�"�O�k��@	2PhU�����	�pp��!�����u�Gd*jS{9e1-\�@��n"yN��v�LG�!�]5����F]��<�X�F��o ��i�Fo�*yO��������P�����y��(UX�lPR�"����[[�@j���#�h�����.�KO�����%!-M�5*�5���i)��v���d_A��c�.�s8����`���%@Y��\��
F>��ii`V@�	��CZ*��_��u:�-�'�{bE�lf�-�!�����^/���/y��W/�6����
(6H��=5�l{4��j�l[�!?'H��������=.�B;(��p��B���d��O�4_j�nw�������GKwU�P���<�=�U�G+��|���|M�yp�|��?����)E�:_��n��kBZ��z�1��j������d����-�M	qIbo���Vaf�%����4/q��]��&��AZ���O��S��x�I��Z\�x�<�b������YM����{��x@��;nh8dle!��~�HG��V��@��i���V(/I/���6�9���������F�	�k�������SUXR�8��kJ��L@�!	B;qL�t�q�o�jw&���6k��L����n^|
�"q���F�+���$��Uh��on�D���`#�Ra
'��J�|�xH"�[
_����
S������������������d�$�g�&�W����%��(@�R�(����5kZ��=�'�5:G�,^!�C���;���\�����=��]�O�If�Z�>8���.d�9Y��c�`S�="�!�zN1����~��n�m[����mQ�<�_��v�G^sm��Dm=o��o��������\,v��;�(�E����������-���VH���.m<�@�yzD>�5�����f�Th4�@j#����S� V1[&j4���K2��y��>��H�@m���3{���!�N�\/��t?�LP�r�O����C������T�-��1��~�Vp�Y�MA�Z|����A��r���P"�����4�Eb�*���0������Cf��m�0�q�f8�IW
���gP� �P�
�Es���}��5@�����#����$D&��}��S��
?��i�������!�*�����Q����Bbf���$����W����O�k���Y���*�+e����H�XbE��R�r�ev@d�Sa������w��> ��:���:=����&�c���bTH���ls��>pC�Vv����D�*sH��3���0���nGg�vd�|�H�,tL�78/��!�
�������S	�m�Y���9�U7��F	����)�,y}�C~���q�fD~�Jx��h�rz�+|�H�5 ��q��a��J��7J�����I�%�6T����}z���.Q�@zo��
�!�6�F�tO�i�$U���NQ��8$����z����y�qD�vL7_2$���P�pB������<�����;�/%��Yq�m<�^�j���|/J���b�D�3	���@�9�64��EM�p$��|��@�������]*�~������#[�o���Sv8��E �Z���7x����b�%���rW ��"6g�k����V�0:n�����f�/�xr����P���2>;��n �$���k��?�#�GYy����,�z��0�&�����	�wL��w�S��v��+�Q	?9�q����3ev��>��0�	v���#$M5P�
O�eJ��{�K���4��-���}��(
BJ�n'��m��
�
��aT������|R`���`�eSb��+�
��J:���%��]�O�:��\����D�
�'>�k{�wk����%�.-���������_>����K%���V��w?=)�h�n<����{���=�-Aki 5�n!����V(���W���*�Y�eCR8���k��6������:�yv����8�y��|.�!�^]�O%�F�����[v�X���W��/��7_����eKKSvw��`p���H�I�s=�_�U�
E"I%F������:WV���Fl�9����i�F28.��)d�5w1O�"�h���i�67&3�|�����;!�v:��d]7��l0(-x�����B���$XMA�-9w�`���\�+�)�p�6v��q�E7l���[�
�0l��&���GH��{d�x@��6e<8�_jR
_'��|}S������v�N�	���Zb�S���
�4��������dM���@g4 E� �����"�-�?}N���Xc�9E�S�sr-G�G�s^�J�.Q������VM�e���#4�6x9�zQ?������YV�30u%+�^�Id��p
��^���k��-�o�a�	\.����"�^���_��1��T�8\J1����x���#f{��.`�U�5�y?&m���� �M,|{=�@Fq�DMxH���l�B�l K����:�r��Q����^���HH���o�2^��5�/�Zl��I|X@���������K����8G�@����������~;�M<�Ium��q<.�����f������BQ�T����p7��sk���Yh���zT/�A�����i��&�����}J`��eJ�F]V%���jA���+�S: ��a�ub�����=��	����gt�m���8"�%������#"����R=��_�W�S>F�*���Bw$3���S��!��+B�"0�"�,�n
����%��+0*������^��-T�#[�����SdLO0c+����D��7����'���R-�?�)����b���?����������4������a�fI��j %�6�~{kAN����EO/���l�����#���E`CV@GvX1���km����Y�QnUA�R`�O<���#�Ax�lQ�^�O;�l����������!������<!R`����X����,�o��<��laK�0A��-a�l�l�I�j�T:R7DW�$�~I(�.��n���^�&����{A$`s�y{y��8�#�G������Q��)j�"������C��������m��=3 �B[���gJ�I����BK�����A�e@��>:AQ��v�7�m���z�\���mJu%�hE�W�Jre�o&���5L4�@�Z�b��-���Xt��e�)z@��m�Qj�g�r]*���]"8���sH-
{y\��Q-Gv��4���4��PN�q��PF��6�B�$X��mS*�
iO(�@D� ����rz�u��i���Wd�
�!
J6%v�nG��	x	Ti��B�|��"NzX����|t��	��f�&�^2�j�����s-���f�d�M|����s���
��U�����E�������8��qIj�s�-��R�5�>t�5�%�h�-l�0�?u�\G�cU?�(�om�����8?��v��T�U������E$��h�(���F��=u����K�t��O��>�X �&��+�5M����3�4��X+�:�����pc�EP=�.#�N���)d�K�=�(Y�^3�
�(�<���wx��VO����`�������3��k��.����������3�yRK'n����+]�����Q�(i�An������8��^�����a6���V�[(���d�|���=���gM6(�p�/J���o�ao,'&f�.�Em��sr�����BY6��Vl�9�q�	�w<�y�������v~H��a@��!]i0@��=Yh
��y(�RB)�f�}����u����#����R�	5����g�k��� ����@����*��������!�' ���~�����b���Gp+�2KD�L3,��5�I���#1
������! ���-���_V��Dh���}�Q��r���X���j^��[�^F�t�0��j���\������|�����a��:���'��o���bZ���~�!|��p���(�y����`LB�=�j���]VKS��-���o7Gy�o�����=7��`?�#���e�sLC)�|Tt)���[����d��$%)�
i���������
��.�,�W�`���c�PI�������`x�n�n�����K4=Ss$�yT\r����0��In"��})�����th��������Ha.d#B9�^3�uan�;�#L����r�d$�b�er1���-�*>jm��j8a��!�;Fl��Y������u�J��S��t�&�M��}FMl��,o������E����r�}�O!�7L�7`�����
K�Qd�4�	��.o&]!���|������MY�%�����W���`��t���%tTT�����<�;�1�<r;��f��@����\�b����K��/��"��s�
A�P@4B�}�D�.���O����F��f�
�^1�D�0������>W|�Bk�A�11J�:����n���I�9	c��^G`�����Qc/�m�.<�����Y"z(��|�j� K�r����E�B�dY�&[==�.�����O3199��92I!�f"b$���8N����@
�fwN����n(T�z'C���.C��#9���E�-#��9����!9	����{�x�� 8�8u0���\����s���_Z@��h�5����R�<��3b��I6���8Jg"�1O�=�Fp�+�<z��Y�u�XlUjz�
��$����
����c�HYZ'7��&�\$���m����������!Ib�T
����}������U���J&u:�W��Bz[��Y+�����?������A����M�<P���W�����E������e�mS^'�<�]���6]��&�,��;Wa�V4���b��6�OF8��d��Y�I^��y���9�V�B�<���VQ�����
K�k�����]��do��o����D�)d���Z�����6�F_�^��4]�u�;�@�O�P�uQ5!��7�K���=G<���n��J����W�{������|K*V�3��Tha��b��><o(]2�`��9�[��%��&��3���� �Rz1I���1�_)P��{k��J6%_��6#��5�  �E��C%\�2�N�e|AA��k`vAc4�X�������JL,^y�� K#|d1y�/����M����5
	V|�!�mi�f"c�`������:.��r�J��0�y�)�W@o)G�i+����xkM(�h�.�^��Z��\��$�0!&��U�2�������9�^��"�)������.j�mC��@�::�,0p}���l
u�4l��R��D���ysP�Ie
8���6!O�NHY������|��oJy�lV��M�X�rD���{����?j4��y�	 ������FL<d7�c�f$�v���2���*�=��v+Q)���J*W�#\��H/gP�^��1R�����I�\���jzhQ23�#.�V�����7�����J6�9����A����xM��@����r����0��{*���y������7���B�����-s�WW�1��IS����@����p�����_/���{��"��?��o���S�J�k�0���]ip�_l������p�����/,��j*g�����T3Wq`��+�-��D�b�w�v����dU�������
�����A$J�\:�a����~/�����t�t4����'c�bE�^�#3?_����J�E�^j �p���1�4��%�����(^n����_���P3W�����,S@���d��D�_Ks�G�@j�������=Csg'
���m��X�Wh����:�i�;���L�N������G?v9��+n�;�����J/�^?�w�P-�QWw}[/�N?e��
x����)9���� ���	;3\0��2�}�A�+b�9;�
��&��l���������CJ�/`��e^j�A���H�����U���LR, y�������y8)3��~?��h!C[kv��;f��r��%�u�����6���d�kYmA��c�����En`�@�E��]G� iW��_�F�nR8K����������������R������������������Yil�������7_����H��)K�A>
����R�{��{q��������>)����;*���_��>�������|y�0���������MB%D�C���3��An��E�lu����s8-����0j��.��vD�+���n���m49<B�����t�`��`E�M����X\���	�SjK��Pvq�O�������������%���aV���1���'Jgr�:0P�����F����,-�2�����Xm�z�EB�uv.@��jZ��t���|=1����������i�v< �<���������u�M�5�S���������.qF�W�2l�Q/�z��pDn����Fd��o�=l�?�`���F6�x�����r�2�?/g������;��0��kT���|j�R�]H�de�[���suQ�������i :w&�#�I�!�83��W���h. I)���
���5|�����Y�A5a����Jwu>s�}rhI��z������tU�Od���QO$�.�e�������{���b��h�e�:)�)������m��
� ����`�z3�1�,z6�� l�3p�n��c�R�y���i�{a���`���g,���aM�1,*�Y������ �``�S��2���r����Lf���.QXh�<���A}���h��n����M�?b"}t�;�����>.������s������-�/���U�W��]6���*z ����C9/�$�[��� 8�j�,@(���r��"j���8�U�G�	"�Z�]����/�nO���a�&{�������U�]��Z��]�4����r,��OJ���:��3�y���{��3�l[��W?�^�'T���7��|VZA��
���$	M��_C"qU�<"+����1�?��D�0��+(w�bl!/��`�����X���b��)��#GZ�i��������[5K"f^�Q�
���"���x��<��k�w(f���T�$��|�����B'=��%8�B2r~���v����/��T/�5Plq��f^V;��,  �b����n�D~]<m�B�`ok��t������t�\���4e������n@����S]{ic�1"{�XP��g�*<M�I&?��b��o[��}���M{X�5�Ia�<>C�r�"���5�mNB���t�mt�aJ�T>���i�����1�1`�v5`�����_r������+�y���g�=�!�P���QG�g�����J&�����f�������}g�Q�����Q��q<��n*�Y��S�eT1iEu0��C:�:��A�M���}	3��|/W�YbQ9�p�;\��gc�����\�Z�W����bPza2]#�����-�%������5�F���:=�b����8��`�'��rZ[$6����2�f��b�]7��t��2���r0Ey\h�v:*���)�ms�NY�5O���z �^QO(�J�yO�KX]P�xDN���(��a�����c��%�����P1��9~g���(+�1��D^�b�{���N��q��!���Mxr'.$h�A.��%<�.��L/�Np�����
a$<���w�������}#���NXB<�{pl�/�0�O.E���rp]������Hy��[���$��Z��0��s�%�!!{
~.�;�Nx������N�+u��]K��^���OB�����9�/�,],���� ��0sR7i]��X=�iR����h���<����#F�a�cXg������
�
1�O!�d�@"���u|�"r����"rmf���Y5���"]���z����R�������Y����m9@-M�:Wb��������@|5P���IX�|H�t\6��*��
0h���Alk����V���5y����n`������YY�b)��9���o
�u�������5&k��&����3<��hf?f�o�D�Q�;�F����=��l|z�������}��7�0�'���s�%���6�V�V�HX:�i�F��86^dQ�X��Y��n}��
>b�[f=�h}�,������\�}�[)�� ���6�p�ssP�q�q�C��E��Q��)�/����sS8m��Q��q	Y"} Z���B��88) g�s>d"]	���,
�p���b[���
c#������&,�7�,'��R\�{�e��Z�Y��Z�v���(5em�<��GZ[�"��)i�K���K�h�'?O�����`�&^��8�AN+�5���o���_��DV(��&�(��:�����m#�������,&X
��_"�^��u,z������R��J3Mv~B&L��k(���0)f#�z�����[���������J���i�J���-48�(�����<�C#`�[�X��11b���A�b5���������o��������UyU~����	4���T��a�;����+�C2qh�� �l�|ti��H�"gf����+y�(<^���6UtwzD�t��"�s�L�Tjz�Ky1�z�Z���n��6F}���3��M������F2��	���v��V�7B��C;?,va�Cf�|�!��l�ZZ�2��L���J������B��;+�?��iV�K t�2��VEb`c���(�hp�1A|H,����q,�P����+i����b��/V�R`p���ws�m�K��`�������x�g��:���w��1l���T�k}�<;E�`�u�t1���L�Zf�\h �
��xdz��V��cvs~���~M?��
8�����?����u�7�v�$�� ��]{�$�;flx��O�u�8��k�\hT;h���lB�8�H&��^�Ke5* �3�/�_�S-_�w���b&�5��)3?��jIowh����y��n�����X��_u�4=�Wh-+���fX���<(�B]�`V���!/M�B��8���$0h���g������u	fao1J�����������R�Q���r����U��'��-|p����BV���i,
9c�7�T�S�[��?���B��w�W*��(�0?�Rw�;�7�K[����xi��O�W%�Q��G��3b����\�X�b�]�W��Vr�I4����G3��f�������k�K����6u������-4g+�902s�.V��H��9�U��s�v� 5:��,tr�1�~�$��XY�v�b(������X�q ��{�����0�Cfs�q@K�k��R�X�� �Q���*����[��Q�� Kc��;���F�e�E��������e�G���|�!�qWR���2M�d���Qb����*�W�9�*6�M��}�x+�T
���_� y���|*�<@�	�7�I�6�u��G��0�X�@��`S����

0?��:<0���Y�����!3������S�+v�S�vU'�k�n1�,�
��P�'.�NH������c�w�
�r��f���������\p�0c�j�F��,u���*3���i���[j�L�]T��@N\
�RE�1����Z�Ro�v
���F0�b�d���2K,�JYTVAO����bz\3q��X�Y����AqC��]1�t�=�<C���_�[Z�QB��� R�J�K��������	�\��*�� ��X������!G'�.�Ur�q~C�C�x����!����x�{h	�fH���eF�K ��0d2�
��6���y@"dZ,���r����5�/������I(�d���o��#%���|W���,g
b�od������b|��[���������CD��9���F���x��h"L��o�'�L�nRo���X�&B2�����~�`QS�q�
����D�tC^����H�D�z��!��+9�����@�Mt��	��l�F��>�r9�e�6(��j+
X�["�
��V����N8
u�Vl���z�����[s��[Z[��S��-��)�Q7@�_@�e������(`s�8#�s���)��w
j+���Cf�4�G��!�T�G&�w �d��k����<�y��|P@�+�q�<�Y�.2�]�C7�t�i/�6�,���<�6�z����R��A������E����2S�H����JF�pL6�dE}8J��������~LBKI�cRuN$��9�4k���}Za���=-n���$��i�
��,���^�kp�'m���=��
vv:T���\����4�~��$%T�
d�7"/W�@�5��Xnd���<�;���o�P�y�%�zX��T�&�p���c_�"~�O
��������:l�q������
�HmHm������8X�8"��mil�nq���^���E�n��`ATP>�7�0���i�V���A�j"�<8�=���w/��(����c+Su��bW3w��X
3Le��T���s�����<��\�+`5�R�����JSv�u�V���U���V%0�j�O�����7��i��I��a�������CA/$����	���������}��"C��Wn=�1�#�@�C�sU�Dy6��
��U�[�Z���F�a�[��[%"��|���#$�������^"������+�Ai���h��z�����d�-�C��$o�JQS���*���������6+Z7��i8h�\9���|�&\��B�2��1���)��XV��f(�q�#�)������Z3��~��L�B���9s�S�#�����P���h����e"��G^\Jf��$s���������<�$��-<X����:��KM����b���F���r��~�p:��=�6��&�N$�NM�Uo���x�T��*��XR�3Q��5�/Q�8�YY��l\2��ZH.�c}�*�h������'�����7Lo ��`O�����nr]a����>���N�4�+q������[l���=d1��4@[��P�5e���L��!�*��>������Kq
P������J�k�����|P����%T�%�	�����Ru��P����Zwa�������hO�!���"iut+����g��K��r	��U�K#f��<��n���A	S�D"E��kV�������
��u�R����8����
X����d����^t�?��iZ��������h9p%��H]&��Y�"%69��s�������a�2+�������`*ZY���Y�Ah������~�m�5�i5�#6��������\�o�[/��wb�b��i�wg�r>?c'A=_9�rZ�O���*����\w������O�"X������d�;�F`��-l�ix��E��*�����04���y�_.�5Q"~a���~��MT4/���#x�,H?q�@;��!��=��q?��1y�b��X�'�����%�xU6������7��	���C�y��M��2�ve������
	�5�H�C��D0��C`J�S
�Abt�<G#��u7����eG\��%��z?W������}��	Tw���h��$c�$�V�������h7L4����Z��?x;�8�Z���k*��'��a��f��L���(�K�^)�L������7:f���orX
�
���X6}�����t���6f
��x��e�P��]e��<Se���1�i��R�C���U��~�|���������mhv��(�iN�<�x�������5>|�0^���S���7[�<e���m�����G�k�-;�JE��F9������/L�]"m�j��F��������K��?���q���M�$_sU�W�nx;Yh�G���o�qB�8��dI;�.V�r��3)�Y���XBB6h��Gu�,b���#��J9���fU���$�U��9�!������R(�kb���x�SA�(O�v���*D!�Zyj�x�����CcK�&�ya%Td�/����5���l������E���.P��f��W|�0�r�n��!���=,��!�eX�X�dw��6d��nH�#���#���B�w�%�4'}��I����p9��PhQ��E���Tn�.��z�����J�'�=a.���fY;��M�������c��pQ�\����yR$��yjyO�H���l>V��*!��wRB�l�� ��v�(,�YW�c�
�jV@���	�|��d��s5m99�xF^��I*�O�E�B�?Wo +�x��&Z{��
��WM�d���<�?�T��!������>�9��5��W��� ��c+����+*����L���f�9��&x��J���I�
�g�f���V����),�]3�u�G�X�/����iA�v��f��>h�+�k��p�����;��������Fzk2:���C��,:@�m�������AQIL(
y#P�1k7�j��,��	�5#.���?����F%%�|p�����6G�Ey��q�]N��ql���G�s�&�cA����U'��^������>�T�������������[I��Q5����)/�4"�{��
��y����Nq��y|�k(72�=��g��:|<����`��b���<�L.�s�x�\�q�[��J5)����T��
���_��������	.'i����7�'�P��m;�	�������8�.����\�It�c�!���6�V$�rK�ms�{�0���2+b~Q������}8�h�=2��m�-(��x�K(�k<+]�\.�r�)g�G�M���E�!"���F�yy+����|��zS�N���OtU�G�����.��NM��l.gS�geqX���}���ny�:���F��uen�A+x��F�X��k�n��1�H���:/P��2�mf���x<��6BZ{���Z���*�� ����\���V�_�������Z6��PA\�2�oW9O����������
�S�m��V���N��.�e
:�����������4Wck�(�%�*�br��6�1d@��*����E�N�9��d�A�����L>3[���i���C�!�4��G����r �L!Z���	��l��(��-�C����Lv��Gj����C���!��y�z�Fs=����z�����y�dL=VsM����j���%<mU�_f���r�eYA�>�l����Q��~���Pd�h�X���'S\<%��r7h�;x��G���]����_}���4������K�r��(��n�R��
c�"�s�f�	���$���=J(~=��YA|2���������_�Z*B���##�E��m0� %XJ`��������l�����
:}����G��{�j��<����������t������@����c�&���������;���A���'���V��O��h��<~�o������'�644�[g�����K~�x�U�0 �m�������$�����}���3=���Z�#o���r}���D�rI�~�Ia�{p�As<)������:+UUM&����[dX�����aZu�b��E��P2���x����"�����;ly�?�.��[c���)�<x��"h�vUx��OQ�����
H�a�[
a<��e�iJxo�!��',��8���0��j!v{��������[ �b����)���z��iub��l�b#�uF����C-�~������H!���e�Jp���
�t����B�4^l~o&3�!�{�`^����yQ+u"�y$i�}���:Ss4�"�@���pH�}���!�2IZd4��Yn�� �4Q�nv��O�8�c��KC��[��`�i�1rd��
BxUA�(gB_��A`���$@v	�N���~���8J��e����D�����L���<�r��D�S�p�����;�������\���nCx�������w�V��=L"�^�H�)�@^�%x�>����h?23�1���LM��C�����M�k\��%/j�FW��4�S{�u6������c&�n��LE<�aX�g�.���������r /(q&K�j(���<���j�K�����k�x�X,w��M�K�,
��������\��������Oa$5BsMo.A�U�2s
���U�D��8���6a&���]	�v��n������w�5�o|����x�1������p������ya��@�m�+�'����@���4�'8-!�@ �vx9����S=�BV>)6����3���p��a8��_���D\mP T�KUX�M"�cT�M$�6�4����gc�!o��{*;�P>��W��!�H][�~PN��@]�rn^�W�H�`��,��^U5	�����E(��"k�u���E�R��+��jPdS�������`Z���aH�>������_h�z�����^����c�w��p�.�w$��������ZA����	fL����o��P�td�b���naL�����a|�bW�|�Z���|��2�eWz@/T93��u��������N.���c_���\������gjm�f3����j�#������	Gf��`�9��_�]�Q�S�����
�B�[�v�����uC(��9\Z�I�B�<��h6%|�:R��	�s�:��1�f|���9�m���~�X:�~(�G1�M{@����<�j�C)w���\0�]�7�_@-	�@���X�{� 5w��:�D�}��hMj�.��h��y�����,a�5���,rc[s��;w�{����?������5�T��)����W�����y�q>��/GZ$?�����8E�8�K�_^�,)�%V/�kz�V�	��`�����!��\d�p�%�4=(�|���o�E�A%id}>��E�'r���2'��_u�0��v�������		���]A���q@z���S�.wK�"��NF�&�	Qv�>]��v�F���"��]��j�4��F��m�p)��D��cT$����"�S�@v�b����D�"�Fs�L�h,&�>W�m�J���������d�8�c�%�'�%�e����
��EWr��'7q������fl9]��(������M]|x��>m;��b^�A�D	��gJZV�~;�a�]���q`�����|��?�j�����\�)'�$���'�Ri���M�r���������$^�y��K�����*N�o7��>��i�����Q��+95/�-�L��3���e�d���7/!h6i�Y6[j���n��g��L�K��5�l�s�[$�zd��9�L�����knC�w�:E����k���&!�r�q�h _@����U�����������lJw�����3����x��
���o�1$�4$�~g�0��U�kE����A }
�����`iz�V�n*sZ�v�H����c�����t�P't�t���4�gB���ew��
������j`9�����\db�������x�^�m,�����2�Y����L�u��d���6�V���|��
�To�WE�?���:��e����	�'jiR������H��n� ��:Z6_	��>�z��8O�;�6��Y��v�\�w)S���������Y�Mm���ep�"��1�����,UH�Na�
R���)�;h �-IU��%+J�7����UZ.���Ns�*V���D1O��J���i_�]���.74��:�yC��,~|1��1��h�&��YGg��G:�#mMU�gY����Vr=>+��Zn7���xu&�YP	���%u�Gl~*cM���~Q3K{�C�xbc����p�*4Vb���Y�KS�q�����������M��W���|]�l+�{��|,��W0�d��}�D_��w9Ro�jo"�8�c'��2�y9/p���� �l��2�j'��kl�+vy>���f�h~�����9��BP)���.��	�.[��*)Z�b,��0���6���hB����pv
3�
�(E����!�4I���/o�G�@!�\sf=��A��"���X�� eOYA
�%�r�,V"y����H_����d�gQ$�;�3/OOO@�����j�X)U�������&�m����E>
`2���������gs��F��r�X�S�"���'QIY��7���"�\j�-�Hj�.�*��|V`��X��q=���g��^x���<��:$�"� C��P��\�0w�0��#��;�p+^����v��k�������873W���g���qS��I8�s�b�5�`
��U+8��%�o`Y���{�r�b�6�-��w���7����A.�]!��<:��Q�����[$#~=?���5iO�_~F��6�b��}(�#���T�>���9������h��������y��j���i��^��Ec��"f�E�W�G�0/�p�]b��@$�L�}�������G.���(6�&���v|���%�L	/(����T/B��D�_�8��.��O�M�%��*u��6���pP���1�E(�^��/�����ik��$;��g����������cU~�N����#b2�Sj4`
��8�-'��X"�8Kuw���/����>�K'��S(�������j*�Lh����#�`���3���=�io����`�������L�|sDZV�v����y�0���y�a�tC��De����$��9�$��<@s�:������z�$CIR��N��%�,�@u��b`����	Z��@5�S��.��wi]����Pbe*�"���*sL=q���y�i
��#���wgi�����nfg�����^� -��T�vA�G	�_��u������}�.���j�*j#?K����&Q�������(�M�`6�y���
A�vN��������<�����o�.�.������^\=��)u[#%G��&C*����!+�.�!�6�H'Aow�Y��j]/vF��'�K=4�������
���#mf�BC���::�T1���n�m�#~�^�z~��<	��94��.��/���IC{F6L���Ws���f,��H�h�v[���}Q,��:�3V*��r�%��(nE^ZQx?�����[�[X���Y�)��)�Hu���1a�����<9��f�"�q�X}D���������g�i2T�!�U�I��q
��4�����6�R�U}���p�����U�W�w����Q%o���*�'_m��fp�[&����+����b��F8gA^���m�N�otq2+���E�6��(�����t���12��P>�;M���y����b���f	��Uk�F�_�\�n0������M�:���x�0��
��_�g�)��C��Z�>E-�	T=|��n1�Ip����\$����8�8B��`���8���&���M7���4�u��R/i&�[�X����@�H$�s$������64�Yt"�\�a:�����PB]�&���f�=�=e9���-Kx+���0j�KX�=.�;�4��W��AD���Fnwd����	�)b������]�:��b��p��H��lr��E	��{i�-��-Jy�����>TKF���F�+
��Eu�H��+�N���f_�c���Hf�|�?p���b���9t@�GE�� ���#��@�
��CC���VqCLc����g��X�@�H!��h�����5�%������Br��	���K[��2�6�z[�� uE�H��$��3}�^��&����
����A�yM���|��n�/��5iYg����b.j�Q;��;d&��(\��i@�3�/9m�:>�\�$=>���Pp��uR�=��i�P��zS��~Y�B���������Y�?8&$-~�����������-[���~�*J���NI�B�$m��R��}���E��`����X"����Q"�������(C^,��^1��c������h,���n	�v�.��@B�K�V���t�Z`?�X���{|��8r<��1H�c�L=f&���u��Z��O	I�������$��KCB�S4����j��UXc`p���m������=�[�I%��Z~��r#����_03��j%�'On�_p�����V���1�:����[��{�w����H
=��%���C��(���?An�� �FU>m��j�J���Qlu��we����5h3��C���R��u3W����1��*
����n���c]|�,�+����g1������Q���@�.�~��uq:~���t���r5���g��9/�H�GK�G9���L���zt�0���E���|�G�_A0�l8f7e��UP��$��b�<3����r�����t�cE���"	
�7|f���].����ihF0cF3]�b����Y.��z~e�.��>>�:��|���q�]�	�n)@b��E*O;��#�8:�����@F���n��&w�b�k��S�:8�q�r8<`�!��-u
����C��������V���
��@���N�U��@Y�v�"��~y:�]�M5�D�S������`]=��U�G<N.���O���V��(� M:N��~ �&r������y5C�,�u��H�
��w��B�P@�r&u�i��R^1���������T��C����W������M�Kx�^�TF��B/��`��`����A�r�h�Zos�Oz!������*X�����j�;^�&Z��;%7c���$�n���VK�>/� �[>~�sa�Dc6a�j��U1�����"
F�<��p4%�0�����F�Jk�%���`�%��3;��}�|r���,Y�V��G��^B��0�H���o��F&*��#���qy��;1�>����!���y<<��g x�-z(��c'^����P�R�X/�xfAbu�����fX��^L��+�;;Z�^e�9B���n�?�P,V����������S� ���]�0'�s��i�5]�I�E���|
J"�$�(/��65��&�Xn=��(�9����o9��+>
a��d���r�)��K�IO:V�p.�Aw+�tcj��>����)���,��A�qt�	���<�-��ro�o�F�
����=�����Ba����H�)	@p�p�1���djm�	Jq�UFK4��������������#(��D��H*�D����A�+K]g�E����@�Kl34�����xxM�%HN��6���gL���,����k��V�%�= �Q��2���dF������j-�]	��Z,���&�O��2
�F�X�6�����!<	��M�[��w`���.��?eD�'r� �}P*�{[f����i���`:�F�i�e5���|b@
���{�"
��x&�� ��+�	���K��Of��Q.J�+�W�	�E00e(W��������[^�E�{4��L�k��������r�
PlM�JZe���xB�//�of������ydQPhH�tSu:�ic�#y���_,E�.������a�S�0I�����\6�+�,��� 	I�.
��l�vr$�uX����5=-���L+3�R�����<��f��K
;�|oC%2_)!n�j��:@B��OI�m�W��l�N �O�9��V��Ox#>t��i�J`8�i5v�?'��������\^��F�$�+d�����Ul�. �F�Q�U����������G!�|��@���'��G���7�����nU��"���;X��2k
p�>af��]�-�/L�iB��d�Q��y�d8�0�>�u���K	����/�����I2��o���}{��2�����>��}���r��2�l�h� �(dt��x�{"2�q��5xy�J�����2}����s25�	bfLb������c��h��_��Aoy3�L�"��X�&%8�]���;8Mx�[�40�:N��@�@���p�w@4�v�r��@k��CL�[0=�3%�8��$*,�Py0����*��������(#kE�������J�W	��Qx"������B�N���B�t�:��+ZBr�9Z�K`�
����=�1�b���?�:������������IC&,f�O9`f������p$?���e{�Q��K�8iu�
�vJ�:~���)������F$T^��IR$w]��*�e+cU���'���;P�� _��oP�To���7�u
��,�!���7_������>#���.g�����/���_�
2h99�x`IS0^9�t����)��C���X������J���?b�$n�a��l������"�x�>/���Bv���/�������T����@G��A9���Fy H������Z_�:�w�W�����������/6k�q�ij�fo����O�������H�����{.i����R�����@/�CF����	�\	j��f�=d��U������u/��Xx�Sk�0��_�
��o��z�����%D:����k���N���x�
��S�d�^�#���:�>P�����4��-��
�E�T1��o%�w������?a0GR�*���Kk�(O������\�^�g@%6���'��gz����A�=���P6��G6�[^#���i���&�����z/Wm��S?�8&�-�hf�>/�c�Z�5B���.�Wv�@�e@W"j�T����z��]��nYRc��c����8��c�WI�Rt��2o:X�k*��S^��g3�4 v]��R�T�.�����x��IPz�h��B�t�0Z`$�`�Q3������G��|�>�n]�Nc5����,kq�y�))r`���+��������?7������8�]�B��1�&if2���h)R#�n�����	�c�l�_f]h�@����!�	x�:`��:�c!%��wRU�
�+�C>D��A�6(y�~>`k�/�
MLmoum����$�K���
�pXn��
���'"���]H���ZW�l��M�7��<f�,�S��>�>X.�U���$N�;io����%%��xpa��uG��U�>-��dE�r��(�l���CC��jA��l����\y���(V�#rB,���- ��U� ������@�|�X��-v�\��&��t����vFF�#���+�������uk$�U]�����/�}�~����3�#�G=��A�l9�vHNx��cFI�)M�`*�����h�)@%`�r�z :!���^���9���:��05p���1f��+LE�������,�A�����6�8�Z��b�� e�1
 �E�����������<}�����Yl�5��U�������~[�K.�?���ua'���z�������n���5n]�-��k`��]���D���~<��2a�j*�Nne��N�����DoY�0�,o���}:Y6f�cU���b0�������;�BSMY��b6����g�?0^�l�P���c�}+���r����P<�33�-�����.��/���N~(���%z%8fe���i�m���o���c�K]�Lq\J�������K���b�=�^��w���8��w�M������
x���px�9��U�+JW��4��xB�j��hx�|"����<��cyq�8��I�
��=a���M�KU�04�pf��	��U2�NFP�u�~�}��lt����*g`�����T��(-M��{2RpC8��k�r2�����
By��F,aT!�����cF
x�,0�9l�>%G�:,�|��I�4�Q?��_�'���F�8��iL�l�-��T����
�/��O�F�0�P[D���l�z���3��-�N'e��iM� Dp��]�N�]y�4�v;�v�c�Lg�T]H�JN�>'`�N�����$xz���_N,0��t4���W�r6
������7�g9��(��;��dp��=�_n�Q��L� �8EQ�kw�\��#,���a*"��0���@�~!��
�H<���>�����������xA�������"�}�S�i'mb��'�������0��qb����sH��v��@������O�9�'�
/��A�3�R��?�m�����2k���,��'o��0���%>���b�S|��sS���9)N�����&��%"��0I�ija=V���U0���r�B���5���T�����(O;�=	����MG0yYX�Kx�E�L	�������B���	����fm�B�����z"��k�9�*��-P�b�8�|�X��
���wE[Q��I=��@��g���j�`���sb���#�CW��X1:+AG��3]rBn��oA�4�l�-�J��C�-����	�l�V�������eJ��1�E��`�_��yv�iU���K�'3�35F/����)q'���<�zu��z��B
�$�r!��/��[�f5��@�S#����n���O]��u��ka��'����K���l���v(��i���
��[6��/Xh_�b/
���=�H���~Y�����������\)}�����q��[4��!A6p��i5����$�h^�O��F�tI���VA{�Y��
4���QFn���9��,��*�b��s#w!�c�k1�T�����G�����I�u�<I�������},QG�8�KwyQ�m�����&]"����mQ>U��<��K�Ql����9fN�'���z;GT� M�j����Nr	]�
\�x%~��L�3����4��!�f��e�E���K.kLq�_��(�
����`l*���������w��}"���
����@���e���h�t�u%B0�x@�1; E��.'-5W��X���wiE���}�]�`0��q9�Q"��N�-f�=l�c�eC�'�-���~�`@��d�h�� j��(9v���}�(��?�cE�&�OX�W�fZ�������p�[cG$���;��0|��*krBPf(d&BN���4��b� ���B�c0s�"��j �VD�@�_6��H�Hh�����B��N�b�����}a:�����#���vx�Y��Fn�+�R�����hZ_�u�E���t�E)^�b ����A�����������x���wB%�`^�Ku��;C��@��6<�g� ��"�H:�����V����=�%/��r��U����[���+n�	~{���XV�J=���4���ee5��
���1�V�F�@1~
�DJb���0��U*m;Q��k@�\��N,I������+�����j�|R>���\���WIbb��,�~��+vh�L����U��9B��|Od����g)���]�
�P1	����?���Ls2$��K�nL�C�)�m�p��7������m�X$h�o��#^,F�Om���������<'���GL���G�UB��`�)�>7�s�A�b$<��h$��dWG�`��P�?+�0�c���+V��b�]<��_��l�����\_�ds�R��HS!y��0���5�nr��S�?��s$�MX����sR *Gk	����E	�� ��x���><��AcB���zm(j!����G����?��5I��
t����_V0�l�r<m.0�u�����	����,J��#�uqP3D�1R���n��i|r�
�*I���J�3�"v����'�������{���`K.�L����Uy�4F��_�;U�`����������O�t���������
<("0a�.����G<�x����*����\R�?��^����w��H��`��i/`�K���u��%\O��B��T�����^-[y@�!����#������6�*����}Um7���������������x���=�l����kX�'�q�4$����T!{x���7�J�_�A
J�e�Z�4����^_K[H�����m��=/�d�@�r3���}M@�4��u���iz�i��WyF(Vx|��b|�.Ok�y�%���������;	�|��2���D�j��}X��%�'��p�j�d�(��^[������[����Q����f}V��s�$���]�E�;�CQ�\��Wu�.�t`�S�,JI	
�WD�1W�B�G��&C��p�\:t����d	���:��m�2)� V����� R<U�k��<Cb�
j�Gt���
��NB��e��W���CF�����iHV&�	��:To<�o:�i����lJ�aIs���||�X�,�[�De�&S���l�����]�Of���qo�>�� O)6���Zv��,��<!��e�����4���z��rb���w�RJ16������(�J��hI���!�/���J���S�xG�n#�fg�l9��u�Y���El����'}Uu���\9���u����t@�<�����	��
N=P��B�o�Z��'��\�����uc��7����d�[ F����2��`���Aj�L���y'~NS�i��fH��QA�f��L�����D�����#&��t�k;*U8�g0i9�X%NS���7�(D�Ox��HQ5�����F��Xb���F����e�i���4���H�p{���3��h*�����)�����~�g���r��t�`3k~��M#�
/��E6�~jF��Q��7o(�3&�B��������o�e�
��#��t)���S��&��U'��
��SM4���F���P�������q���o���/� �fa���<��`����%�,�l�;����u���n���
��
�?e�1���`<_M��p��&����'_D����7_%�x&W���nJ�[X�S]B�F�|�=cY�"6@WH�x<U&����L����C*6�-��BCf�O!?A���M/�LZQ���UY*�z�oy�5�2K��8������T���lDa���&��?&������r�S�b�Nd���Y�����9z�A'�����[e��P0��/��3yW�����)�{Uj�����<��yw:�@)�m~�]p����Yr�c�Y|��������Z��o��,��3o����p2��4��`�V���<���-u��R��?+���������!�)i`�g*AJ]WS����Wiz�S/���MK2����!	��o�/ES���8��F#��K�������X�&V�q���L:��b�[�����X�����x�c@?�q�	��R��ny�}�D�f*s�
��Zn�c����0f�`.���<������x$�"�y�]�:�h�e�T��r:�afT�����E�jN2��E0&����U)���H
�4F��g�W3^�<YEw�aw ���%Q�����;vD�4�R��3{d�%)W]���a�HS�{����c�7�V�Q��)��7�i5�	�<���H��`[,����p����X�(��bN�S��2��YM>(�J�$��V�\��A�Ik
@�h��N(�$��,tu~�ki�[I�G�j�i��� ����;k�y5L������-�M���<��=�����������+�Ih�L�>CI^^:m6
;�A�r5)!;N"�ND�����)����(��E���L�B��k��k}
�v����y�[_~�(m6��<~-�����Xm"$��%�5U�!Vf5�7hb"�������2��g�#Q��r{����8jAL
����^+`�(n�������[e�Z}X��	�t����e�PC�
iO�'W������T�*��W���s�G�B��p9�>V��LI�E��iAU�|p[�i�K��U�3B���	|j�DDw��J
�ao�@?Hpuh����V��
Ay9�	��	������>xg�B%x�;~���Nw�6���.����g�B����5L����QDE�q(���i�D&���N{0K]Y�<�R�s�[�r}���W����A�pC/>}�r|�r���eks��yV�f.���T����4����(��X�^}3��'g��k�8)U��2�$�^`��@����~�&3�c����>j�?�9zP���
�c7l�=�v���w�"����)�B���Z@u��������
Sv=��Ay���[
D	&�	���������+��$4�H��xZ W6� ��V��j����{����Y<�Q*��h��K��m�	�,�~)�_����5�jr��=S��R�!��7���h
���@��?m
,����O���!��i�����GA5>Mcb�G���
?M�h��Q�L$Lz������������G�G�
��ddT�P������������<����'�`d	�S��0���B6Df@%�-�a��B��V]�>�+coQ��gJZIs���H���F ��f��Jks|�P�\����P��i"�c���?^�$ ��@UNKs���=@��,��2S0c
�}��7������4��Y0�bc��y�?��[�Ioi�x�(�%��`w����r����{9�w�x�^�����Y���B���1)� �z��d�f"�R8����[�W��;Te���,2)C�/������������@��T�\���7��=A�Kt�~��AnF�,�~\�6���G%�u*��	�0s��"?h�J��/�i�b']e]�,���Pwc��C��V�
e�u6X�����Y���s��8���h����yq(�chj���9����_?I��j���79Q�A����TA�������i�l�����O����j��t.|Dv�����]oV���?�R
^�:��H����`�l��`4��E�%�6��]$V���xA�<��������33:tm���p!����������D�y`> �t�4���rc���A�Y���~K��!-�s��]��tB�'VG�Iw�`�����N��\��2KDkY�k��������.������ �a����-Up ��rK�`�??�Xz�Zar/�V���,	�td��7��������}��W��w<�D��`�~"�(�"��r7���xb�����yOD���A�6���6���k��e�7o2�����cR����_j>�����v�����E6�).@�,F[���,�9�hzU���Xn��X%<��)��M�����;��k���G������#��3�T�����'@�4i��F���N��_6�#�������zM%2�U�J��S�+P���o�a���rE���!�i�"V6h��L`B:�0h:e0a9� 5���
S���3N* ��p7����f�h��V�Ar������o�r��������B
���}-��}�r�lJ�V�A����.�����|f�r	��N���KA��E3��d/3��_�^d�h�?Sp2�+�s�w����S<k�T�q?��H����F��A)��C���B�Z5=��	���������b�X2��rs�]6�[`q'���9�xzz�H���.TI�iy�*Y�	X�����=�7�[m���o
r����H8+"�a������"d��,TO����=���P�,�~j���a����d���	8j��|a-�$�@���z<Q=��F
���x0�P?��&�-�4	g�������Ry����X�T�%7?���p�\��n�-�T{���t�������~������[�.�H������UA��%�w���+�
u>
���������d������
�2
a	�^D-�0�U��F����G'��u4��td��O{UU����W��
�����X~A��=�Y�h�H�~��i����OooJX(�I���&�� D������y���P=i�]��z�����������3��3���J�#�#"�LXV��uP�
����s}X��E�D��a��4	��=x���o�>���<}�#x�sQc��["���A��(�(�Q5��[�N$x��=��J����_������w=d�_
y�����7\1I���Cj2�uy(������r���6+�Y��p;a�N<��C`:�|�O>1I��	I���+$�Ex�,��A���7�\,
���wE��(Q���By�v^�U�N�k�m9��j���jMS��(�/f<��p��c�'�c�����Yn��8
P��
����xJ�.�����Y�dQ�Z���4U���;f�p��o��)��?m����''KG~nxg����r ��uRJW���g��(�K6r!��R���~���%���$
���ttXV����D�K�G�Fd��I�csoE&��d�V{�]1�������k���yHbI���&�2���;�@>�-�&���[�j#�'�A(pb�De����\� }�����N\x h������B�$(/mrNx�Z>S��Wc{���"�x>����6�����wy��������{zM�P������;�����p=������Rn�4���".��m�aRr<�^bA\LVl�+����@C�}P������Po��NY����c���/U�,��\��m�:jr���a|�M;���E4e��5�#��\��1����6WW;�.��������i�������8w��w�s(�}�����T��}�e&3�~��U�O�mL?A��O��~-}�h):	*�.���(�0oVBau��t5��/��I����4�3�X�6
������N�p���o2����N��I�S�'�e����L����E%F@�����8�w�eb�zu�,�5�$�"�B��&9Z�����)�u%X:��������$4�������
�r�6�!Kk��&���\�]�")�L�����e�DQ�����R���0�� �+�++q#��_�>��!��9���N�b�C�Y�gB��@}8����JKm�|[�b!p���w637T������������J��x�Z@� ����/A��Sqk����a�	��P���R��,�ui��`�����'�I���;�&�����`�mm�d 
E�n���r�����nd\�=�S���Be������E�������Z����g��i�^/�������"�(��|���E�Rn|���Ym�6���Y�����5�!
��nM�{��E}X����������Y��[C�x�w�A��&����].�����)��p�6���N�^;{s����O���"�0'�]��i�T��&$�T�@V[Z�-���i�Av���H^f:W�����!i$��nDw=%��C���V�S��K#��D�`�G*�(���I��>�{�����	��:w]e�a������B�
����!�x<==A�:�9dPb�2�!�%�r=�6���?_���eX�q���'b1SV�+���CQ#��z���n��w�# p�u�#�O�i�������2UVC!��%&���>���Q�$��C�@�.�6G�����ox\����*���b���9�B��t�(��~��yM`��~���'U�������r@@�<V#��lPN��V��#��<}�-���������kt��?�q��;��������
]��p�v�eW$E�������8$a%S��<��I�K �.2�@J�!���z)���1�
��4���Mi`��jK�����=����Z_(LJb���+� ytn��4���)t�W��0vHm��;��K�F�Z���	��n)�����-/�X���,13�%8_�������@*v���@V�Pq�$�b��F)j�?��M�~��pG"��u�)�5�P��E�:������ ���0�*,z���X)�W�Ei���>��9�x���������������C�D����Q�����U��z������c�$��I5A�_��e���|*�\n�K�f�S��:����P��}bC�G�Hy��hg�z/7O\5,���-_�dNT,�����|Q�{0f��S��#j����+��:N������<j�v����VV7�\������	�I��� BV����{Lh�F�b���^v����O�m�=�����RN��	���b�<x��k����~������`C#��]r�������,�C���RV@��y:��#8J�N�L�K�"������5��G�S��Y����6��#v�����r��q�T���'����rY����@��}���sHa�L:��w����7)���r�8K0u�q��=v�������Pr{@�R6%�g���N���/CBa z�P���hr|��!w_b(o��)�Ie	�Ed����K���jPv�z����5���*2y������iQ�F��S�F����Cb����`mZ�U���-U�$n�@z�MB!9��z���`���KCiR���s)OX'�D8A��Ea������#����{A�
�Yn0���-�J��uF;���dl��r_���"�����+\�;|2�t������W�oy�o��������=$34 ��!�3j����j�14�Xn|�������G��-'8	;&��������"�
����
��������eFl'Y��_��	�����/L*�������lVD�����i�M
	s����7;��0gqnIh@z�&��Y8�L�U?g�D�A(\���%8�r&�^��d� abn�S��\�LL�m8����(�m"���L�v�d
"og|�I+�;1�������o���'	f��#���UoZV�y�E=-�c$�d�D�.;�m p��+������f,.O�(	fz]|(+�9���cQ� ��+�@����$������V	���
<:|���(���_�M�NOn�D���mxQ�����Y�Y8�&�<���i�lO���x�����F����"�*`��+>:;��L�-2��,D�A��94�F�B`�n�@�5�W��{
������;dF�%FyT�%�S~��=��G������^��]D6��#����a� s��;������Q6��l ���v�(�j��J�����"����(�0Qx���d��iF^
��_HC�|nu�XLi�@�>;Pm�
#�+�t�.:z-E�����w���1z�k�7�����7*�����3
��g7�mA�u�����k�GS�i+���0ogC���9�8
%_Y��d&����EW� ��x��UtN��T�3��:R�gd��W $ ���r;Tk���2��2�����l��j�$��[�m���_�3
�f���=�_>��_
�����HP�D���2����������|.;���b������!
��P���,WTH���xqh
�U����r�����x�<��u<�Pl
��A*�%�N�AAa�KuK�2�*]�/�h�L2D����<V���h���L|��B��;��`��&���(a��,�s����So��@>*H�r�8�v����FX�A�m����UNN���N(����#\,�&�����l��?Q���������c6���R7+W��c#�C�*��>��I�o�&�7�v�;!L����`�����b52��!=�wXObcT�un�rT����v���x��1O�B�"q��6���/�jY'�$���:���aJ�E�p���Ss����R���[���f��O�|�QWS����K���9;m�����n��������v"�_�@�l���0��X�(������1�h�-�9$4G�P��3�
����@���o�����K�E���4C��@%�u_�P9����or�=`��0��A[�<��t��(���&	|�f���h�D���9�����W��_�P�K�����u�IRx�O�s���2)���RV%(�7���Yv>���C�c���8q�����i�w�\O������zEw�5���&M)�!�E���<�h%�P�a6�2�~j�7�a�1V��%�VWiI>"V�rm�����8*4$�kgP���/���l@����Q�z�9~W� ��77s	�P Z(AN����y",��A���Z���J�J��c�b�mj�N��@�u<�0Q��o���i�������j�'%�����,��?[���/���#f~��k,m�/�V�z�[Pf������J��a��`���xk�QW�����~8)��4���3f~p�S��uM
�X���H�C�����-�����<������A
W���iV�_���=��SF��D�&?@����F�
)���EkXq�W��yiv6����z1�!d2/�d�����\v+�0�b>�L�����M�|��m��L
O^��VP��Q�8��\���zU�����{��$rI))�~=o�V���[U�0T�s�����N.�!�Ec�_O����n�
��?g\H)�g����rS.�aR�K���x���f����w����D!���A/���N���MQr����X�+p�$���y�2�7��u�T�u�RMI�L�H_c0�n���������l'���qt��#QF��0`��?��v�g,*X#q���������vWX��_c6g��`��K�@47�}�e����!����@
>�qO���;UwW������g����s��+vr�`G�X� ^H��\b�h��(?�Hp,�$2`����@RO�b�,?(�d!I��U_�c�-�������C��T���{�&
�j��	���p&t���Xc��%�"����u�K�^�t����00��Md�IwM����O��P�
@�J�I�)�c�f����(vd���${�
�eq8������<6o75W���M���d���m���xN�xJ�&�E�%1K���Y��C���w�%[ �r:�����qT��B���F����bR������x�%���4����.qX��*��&@�q���qY�efg0����[��Y������7������jq|X���v�]�<��������b�����;�������6��P��Xj��Y��\��y&������%���}X9m�����/�f�������?���fk7��KYc4=4�T�SB�Z���y��HxAY�l�j���&z�Y���>T����A���������{��k6�;�*��2�*�x��&���V�8 ����(���?���Xk�������$��-D��9<|^�������^����1���\��	L�R��0������7~�m�o����1��f����`<� �p-��+=D��JV-i���+�6K]��6D*����AZ������CgF\/2���]��+	y�=y��8gLB��!�91P����|u��E��Unl6A_*�wPr���Y��f`>��IZ�Iv��Z��M�{K��Cm=l��8�#�k�zw�+���������y�t7*iW]Ouy@�<x�rUPCR��F����O@E��\�
���m6�{d!�X>��
��A����3+'\��6��Y���J�#H�O��V�:S�3t���E��)9��"5+�,�2P��2b�M����,�[����s)�v�����P�o E�wy"YDM�����9���+�`���CR�����.�`�?�p������&})���v~VL��*d`A���(�+���\��x6P�1�P[�f��=?�Ws���V��G�ru�6�����'�����`%��#�A%�eY��<��(�����(���[!H0��Y���8���*<�{p��#~�&r��},|R���������E�����v���y����i�Zl���jk��������L���#�8���N��HS����v6�< `���>����7��6�n���F��o�V�Ln�������]��������L�������#Wj���V�c�+��+�	#��
��]s������\��0W����+�lk��#�7����!�x�9@�����6�q�4�:H�F��D��8D��$	�9qs�\6K�q��^��~����7�����K��b��"�V������E�On@��n�
<��o��>Z�rl�����?5������9���f>���,�H��cSCv<���nQc��N���6=���4d�"�x���~�k8��!�L�����-���[�P��H	*��@a��7��Bn��/u��#�c/8q���y����1jD��r<�F�`��vu�,�����/�&$�Hp�Z6��a~�I8��dg>c5����r��~����A]����P{8�Lq�{�1q��AS�?���i�1R��{r������G7o�|�����;��Q���#x�����H���=��F�(�s�E�G�����<,�E������'�;��Sd8&�����(}E��Aq8RG���U\Xu���1��K~�����l<\�����F����9%/8��O!�LM��0`�Pn�M�`+|�<v��&T���[�,��5���vh�?q��S#QN���i��!?^�Q���aq������F��_aJ%B�w��HE��V��fZ��y�����Q�C��A��\�w�����b
L)|�0��m��G�}b�&����N��N���L�$����G������������ �����F�n#.R���x��0y��a������)Ml�O�*T>|��7�9Q����:(ji;�v%��a?3i�L8|�+�g�b��\WE�u���`���=��m�����Wm!��|��X�[	����$��B�����OLM@J��z�l���O3^���x�����AfQ����V9��}2�%.����RT`���X�2y��pZ���������P�C'��oL)c��j��<m����*�+|y7���IR��t[U{eD�$���Y��T��A����D4�$�o�� f;{�^vT�O����&����o,anv�� ��{��V�����uV��`�Z��.�����+����fE@2���h6�]r����)�r���C�# ���* ���,$p��/"��wVM��c�33���Ts�����WrNx�MP]�N�s�����-{~8���x��=�Y�N���X�5�<	�����E�"t!���6K�x65|�~yip�Y���A�4�0���N'q�\�0�(A������}&�~�n�lP��[�f���Fy���>�P��*jS0�-	��d`�3�����j�����4����� p����>1�Q^R��6�,��C��Y�]�8�+Y�5gj,��
'�E��C�O��B�P>4x��������"�:	g.L�D�%���,$<��i�/M<5Tsq�
#���P���;i�)�=u�;)Lx�I��8I+���,���T G'���)r����3;yH��jh�q�pz�K�^!������y7���/��"I����B@�����E��vfd^���1�m��Rk�[���n��NZ�<�b>I����;��h3�,jb�B#)�����V\3!O
���H����E�PY�{yf�S&���(\��4A��JK�����(����9��FR���+�
y����fS{�^j����QqF6ey�0��d��^����^qy�y�,��8����� mZ��K=�����}(�%�>��?��Yb&���q�u�U>��%��b��7(r5*�����~)kC����|�]�-�,f%#Y�y�A�ag��xk�d���\��s��<��H���Ck�$�B�v���(wWO
1��c(
c`�g���6��'4HxXio�b`���h<F��q��]���(�Oy��b>�Y���Q��|�Q��
$mP3OA��C��������4���1qFYe��o�:f!�T3�VX�8��M�t�"lg3���P��rS�U��3�����H���<�Y� A���8&�,GL������p����dM��qf�??�n��\�$������7���MQ���f����Odg�-tjB$�fb�]m���2�
�(����I�Jx�2��Ga8��i,f�4I��Q�����-�(^}�4�Fk���3���L��(�UIz�t?�c���������;��X���m�n�,� �g�����!H2�@2��{�\��
��;>�^�v�Y����S(�:�;d��SZ��:{�
��&��!���\bn��P�*T��@��@�~{bVf�g^*�I2R�!�G+>@oR��P���EQ*��9D$p�������fa�U������"�J��h�|Ex���K�����N%X>-a�Z
Xx����a��z�d0R^��s�y����j�
a��`/�]����|p0e��������iv������Q)��Hv�,!����O���X��6N�gj���,��C�+�����"Nf
�s�"�[��i/_��Y,��f�Kp8tKo��n��1����S|'�O�IC���j���������L���n4��HLb�^�*�'�)7 ]����=n���qQ.U�&�
�`��pB �V{�H��n����~���	��j=F1[����k����-�� 3��Rv�0m ��)&g�XteSR"�
b+��[P�W#�3%@����Z�c8&���]����PQ��J"
��e5�����#j���l��E��Sy��Kr%&�b�P��Y�D�Kc]��m���0:P�R���V���G��v�<���I�%a��o�����������i��������V�4>�S�L���A�c��k.�`$��c�>|���\�{����o����Tb��c/�����V-m�!���}�vx���c^s����mA��t���E1�N�c�*�J4 ��m�~
f�f�.V�3?w� ��A���kp~?�S���Kl��R�(�������|xf���"1�C�Y����/���l�������P����?H3�\
�$���`/����?7���f+_�-��7���-�v3H��F�PF��U�A�|������~��[����}�v&��|�T�P�5B��� Z����l�sj�H&��<�A�[3��z�l��x��@��s���!o���4�2&
.�@�?@���0/J`�s����x����;V��BE��Qkr�8+�:*\�o21���}]�=?����t#C��w������o�W�n��a�S37t�?�A>
�:��G��)�x��x��}�����4�%�� ���T�A$7H�����m��Xym������`�:�"�d�� ��� ��R��G��D����F�Y9z"��9�fsh�"dr�B��J��-�5����Wss�ra�)#����q������`��+gzW��>�e2efp|
c��*�z��"G�^J�t0B�R� xc^q��:����j��ua�	:<��1�=����z,�����4����
r��6�I (J
��
��z�`
@��N��Y\�X�N�{���}�������l��w� >���R��Z��W�!!�0�7bfC���g����JW�Q'��? �p����d�)�E���'�����W�:$f.&�������T�ony�%s�L�����8��:Zn��T���@6C��3,�h����y���O�+QH]+�K(]��$�'�DkIg�"�
��/��8��13�E����&N@�����6��8��2��"U�G�L�o1���t9�k�@Qz�2�Oy���lQ X�Q��%
Y�e�Km_�J'`OF/+�������b�N�6�mt��JQ��c+����l��9��-����A`�FhMdr���n� K[-�L|��G�~��ru�n��@m�Y�%�;�9x=,�K#eF�Cb6�(1�������*��|�OIru-�R|������r9�G����6PS�1�tE[1Sn���\L>�U���Z
aI���
|
�_�b�W�����}�WUg������h���5�sd��n�WM�q�h�s�,@�2u��t����F@]�
u�L�6(��0GT����C~+$���E��7#����+bCu���I�,���G��AwVas��.F����8�_�2����)������s1����
�����?h������B^<6�8M��z�?��
Qe#&<7ql�k���Z� � X��a�Gb���r������+�
�fX��l��C��=-�	�E���~�O@��[��n�4�F�����+��iNX��![�8�;�%��������
9��SO$|�q��gIM��G4`.Y�@��!��drtg���G��	W����	o�}>	�����dJ������o��G�L�q�+���*�!�����=f��iB�qH�A7���1=��Ty[-���"O�yF���A��c�#�����>�ys�X-TdZ�<0$���Av�o��)���6t]9��i�s�-�
)�V�	��[=�� R���J���I��k0>P��Mt����,�bi�`�zP.&I�X��F�n�w��u������)�3��9��k(d��J{U3�#����;y��R<��,8�Z�����;W����D��&����0�^n�:q�NGj�E=Yy�+�Y�e���!c�{�8z���@�O�c�q��jm)K+b�,f%�d_w��h�=��[��b�
�������������6��Z�����@�P����^Ls�wX�_�)��5���Js5
F��C�jNxy��QU����^}.��{�~P��d��G�P0*W���An��i�d�e�J��x
�g����W�T: #�+\b��$ax2��g��U��y��}�N����@�r"L���y�d������q����a9p����!n�M�9"q���-���z���Zn��j�y"������K����$�
<�:�sD4�����g��F��r��2��(�Q���n��J�yL���!����DLU��;@P����N=����3�eC=k�_�p�g|�r����w��A�QLk���&Ik���P���'�	?�4@�!.����jT��`R~�Yl�x-za��=J,�y�l�����R������f|9��v�����J��x����!�}�63��2KD=�wI���q5l�$�8��'�qT����]����	�"�t�O���LMq�h�3�n��xz��A���f���n�	��Cu��\�z��J��M��X���Oh
�V9^b��!�r�C�m�h}�����#e�,��Y@�q�T����y<����C�G�WC�G�����Y��$���71R`��+��:>�
�&�_��hq���|�0����2%.�M���l���cM����O[U����*� ��j+����0��������P`�;��?2����!��'������D��� .m�n�6����V�9`4�3fN�	c@Vu;R����U(<�]T@��N�-t�_�J
#����z'��9�����i��}�S���(	��Z��R���d����"���
��X�|�b!�!�9�nV�����CV@XO��������R���h��k����?c�U���%i��O9/�:�Q��)T�-�����<�;��[����>/�q�������-���Z��-��;���r��Y��'���
{������? ��^]�L�@a�<@jy��E	;�����.=���U<�2R�$��$���H�&
�
���������I��re��BtF8zh��%���f�o���Ff<�����1Y-�{����H���8T������w���K�f&.V`'����Z��K���-�fR&�W�g���@�-pg�
����V*!_]�E�4�"W��@�a������3������;����Ptf^/�P�5�j�z�P��b����ztU&j�A�&r1l����T&������Bkr�8Z6�������o����sW
���	��p�U�+�J
S;��0���'Y�4t,?VU�<�� 1�$��<��y�:��gS3�<����R�j���3�`��V�j���w"7��s{����Pr����������9+�8�yr�������/a�L����
���Q3�E���Y��v�
uG`��@A����D�N�z+�����w�H
r����1�����qq<�����F��b@+�p��_c�2�%|��\��xns�=����b!�_��������bE�<A��YC"g���g9d��m���Z��X�cA���G9Z����l6a�/��{� .oY�w��sp-�m����k�l�8Lr��j������������f,vG����Ir�����d�cb��/���w��*i���"�$|7y�8l����E=(���9��nj�7��3gS���G�����b�_Q����T�/y�8r���94����G�T% I�d*pA����AS�^:�D8G�s�A��sCE���u��s2PV@�M ����r��jc�����f�m���
�RcVa5:g<���'�i���{�ng����\����Z2���m��\�������j�����+]kpu��U�����"�����	�)[�R��*p������3���bU0/����,��E�1Q"#����k���/�H��$k��!�0�W�@Z���!�����t�5���7�?`(�����|J|���5�p�_��<��6�����&`�p������~������T���$�'>��C���"&	4�E��C��I^���/�G�	v^e�S�=���B��:�_�����i�D��SQJ����t��F�Ez����w���$q��(J���&och�U����tfeki2��6B�������v}�
V��KB+��mX��]js��U:�Pz@/7��b��#s�YU���G����j2� �]v9��{,��l��qH��2fx��#���\x���Dd���:������qQ�1�������p{u|������^sk�9��|?�~=?nv���k�	b0S�����!���������:/&���N
�fhh����T�
�I�J���<0��&���A�)���)M�w8=�\	&Q<���>x����a�.J����d'f,�zPJ�[���n��s�<��w�B��E�U&�U�����9�q�O�����i�j���4��kq�� 6y���i��j�����k����`������}�QW\�n��3��I��,	����w�L�%��>�`����OP�h�mY61�����MZ'�����6���w25V�Kxl��A�$��� ����NE����u�_�����W�>�<2e"NP��O�=aV���Y�BE�^U��I�����V��U�\���ve�L�P�6����:�y�di��#-K�SQ�Q�������L�����6r��r��<�=5p��<j��� KP���^X�`�}[@=���I/;�(lqSi7f�KH/�����[6��5�A����&��1r�u�W�v:����=���8|�z��V���9t�1
B���]����Q�Pg^p�p��s�1L�qEw����c������a��(��PX��*��p�zO��X�k�_�0q�����<FpA��g��^�K���]|�Y0%rrX��
te��E�
x�	$V�t�t�y���Fb�P��#�IxR|#O��Fx$�I���(��x;4
�Gy��)C���
��e�m���.���/p����r}����4R](������%]���}��d)����f&
/j�����������
�����F�9��	R�!�� n7(���mE5�����e���1�SH���7_1��)�����;��Iz���1�^7�Z)�6cdu�^�����{_�����*��x�0�r����g�@���Htv��K>��;:�������$��4 M������8�fy�M���v��cf��P�G,(���2/��H�����]�.�c���:���"]����S�XE&�%���B^����"�Y>}p�B��P��w�q��gw8*%1J�0u���[�v�c�)�h(�R>C����}�)��d0�)�9L&W���:"��H�
H�O���C
NG��47���hS�s�S���:��ocb������$6��$����$T������3+�C�;\�;NC?D���"
4k��uu����,3��59aCE
�2���"M�jG�-�5�f�Z8�ny{�����lR��n �-�\*�����*�9�g��A��Y!��U�,f���hKd�u���89&��]�]�Lr7��pFB�P!�T�;�P�lo������$�r�
�p�)\��	��
�)f����r��B5���2
Ld	B�������`�}[l�t[>�c��)7�1
��\��]uq<�[Y�t� ����*QCA��.T����\9��u�qBR��v�Yj
���]�"W���4�y�&��J�������3��g'�/L���{E/*��
7N'V��g�T]^�����KiG�zU e������+!Q��rGni*bM�����0�mQd�8���l�E(����X+c�
M��H�APxJ��o��q����� %|����[��3���c����i���������3��[�e�����
�[�5�g�}(W���	�a������+Q��b����c7���o�u);G����$@�$^�8�m�n�p����D�����d��Y�%��� -���f-�����k�B^�L���3�^S�\�{iv����G>3�/Uar�T�����#T�6K���57��,����nVOp������@�A��4��4o�,����3��$D�"�Ql�����Ug>����kY���H�����q��2��)�4�P��<o$��0�����3���J!-WKYi#s���Z���+��t4g\DN�����k�M�L���j|��! �p^3M��������b���p���8�83g����t&e����y!���W��������bi~N��*��g�F�PL:YOs�����gC}�j���>�������`n�W7jU '�H~���	S�^������BN���+��Uv�k�Q�:U���*�#��B�M��9��+V���y��e)33���] �b��z����,�w
�Y����<��pi������7��j�ms^�����`A�b�)$���������)�VA����X�H�QF��\�A'W���o�D�)���M���)���rL����;
37>N;% �@�+/p���-FF]�3fM�i���S�f�7"���`w��f3���_�? �igV��@������RZn(b6��/3c�L��P�;�*d��~����b��T>�	Of�"��X���"�#�w@F��e���
4��������rr>T�y"�&G���D�6
�>�����~g<+�,��@E�<�L�W.�'m���)w��a�*T�����OX	�b�t%���V��`$�9��<�hX_��P6:�\�	~�=�3`U>n����a��P���Z�m��
���<������	�_�=��f�)�6,p3���������e��K��<	k���KKx�%L����ga���O�F2�}��XC�*h+�K�������
#rd���]��^f�n"�
;��j2K-Y'�SE�����Z[��}X��lP#Cr���R�h���x>������-����z<L�m-^��$�Vx/��G&(V0;�K�G�����f4��W9���,H��;m�dx�,3���r��a����
2>E��!kS�����M����9c�|��@Fq�`x�~��'�y�0k�M��70k@��.�(�%���K�)����
���@U��Ce�����~N���<s3fSJ����nS��}�p]�����&�;�Z�3���h
����j���\�<B9�fa%-,pb7k�����F�p��ao���@�f2��YE��@U������
������=��m�0B�e:���~*H#��46$��x�B�&�g�l�-��+�X�Z0������<�#�N�?��h$��bj�%���t�BY��{!��|�^ka��#�:��Yj ������r����|�,�����O������O����T��~�KLtq�|�m1qi^���yA�Du�Z�jY$���'C��k�s�?5��80H��N�GN����`���
S�m�?m�P�|��������;���3�q���O��'y�C
����EG�0���UM����4���*b��*F?���%����������w�������O���.,�*��2���N�
.?Vh+�������������,�����V�5$���A�%��8s,�F���U�:M	���_���1�����e����<�y��40{��{ d�n����U�$j
�L<_���9aV�N�{����ME�Lb7��#K��(Z5Qg�������Z/�5Z�_�"�EOx7Y`�M�cd�������C�|#?������mo��ek�Idh'����K��p�DE	�n�HxQ# ;)����3y��3�f���z�		�	]�?i{����H$�x7Tr#�t�����Z���������z���"���Ch���f�1i������?�SB���U���iv�~������U,sctE������)�"�,��9����.����(�F3��p�'
/[7�D��ybgz�����m���g
%�]U�C�v���S
~\,���g��j]/v���1E�Rl��j�$L���������1/-�l�hkG�pV��>����N���)�
��#T�4�dV�pz��q������s��9s,��m^�x����|�8$�F�>J���h��0|<���	�^��X�
���8��7���Iq����R
�����Rg$,F^��3<�v���$n�#��1^�(���e?R/�����8�0����X?�R���1������r�|.9aE
I6��Y�e5id1�1j��f�;D�oUu6��?)��U����~]<m�nh�cf)����`b��M��V�P3�(��;;~��5a1�r.5PF��A���un^E�[�}���l3�Q���4��hya.Y_w�0W�r�/���X�S	6s$M�F���B���l�F�o'�7(}w�x�|�Y��.���y�.���*���di�:g</!=����	}+;�*WUQI���!
�E��;F��3��8������'s���d��.`u���35"P-h��1��jk����;���2��>V��{�F�=����t�_���P�r	m��*���hu�HN�����d�{��7�T�k����jh����g�)u�O)�C$y�7�j����O+QP���b������������	��p�,pM,p�:�u�aX���W�����t1��~~U�I��n�j��U�0�+���%��?�U�� �Ca&��-���0�%��(�A		�uqT�����=����/�l�G|+0����������[�g�R�d��#�]�Lt�*?std�'M�m^��9����E��sD`r����	C%H�L�i���}Q�p���5@���"Xz6_,����������#?��V��D�9'i��{jk`�`����L��>� ��"�~ �
����
A:��x�B���eE����?���������(2Z~IsuL�:_���$��*(0POX�[mTEt�����'�+����U��1��i����xP�S������k�IX6�����+H���!:����1��kM.�5�[���$�����Ly��	1	.-�;�0$j|}m��^��1�B���+�m���l�K���"�jF3O�BuMI�g���+��,dv����1�pF5�4��@G3�i'����#P��O��f����E�}��i����*����W��f�@��!89��5��1!��<-j	���&&�L�K���B��{��3��NSr��Qnm������*���=&S��];O�F�21I�>]a�WS*�R����>�Y��M�K���%�K.�����T�}�������`�����-o'����yO��ZN����j����Xwy���.B���? V~%�=�����B�.��Y�����
S���v�������K����M	*p�vP�N`Vmjv���}I]��X��1g���#CC�b��<8���,���.���0����3�{!��7����<�x��\1r���jI��,�L2��oI����C�x��p4���9�a��I�~�����[�f��X�=,�=�=���}����Fup��I�Z�C���g)hH��Bz��� ��{X��}8�K�����>e��x����<B�N���rY��@J����$S���y�21D��Z~��S��N�e���D���p�C�&�=�#�H�7/�}�K��FK�H���D���@My�Cc���.�j]v7R��a�?�U�����)1E�p|��B���JW-�(��% �����/�
p8��BIx��b��|����v�T��\;�$�.~+�!N&Rj]JA�G���$�
`SX���x	���x��q�a�`�QH�Tp����a���Q0%d���m���1�R�;�0�'�����������VT�)$e~�g��2��&>�����I���W	��~�Y��/�T��on��]f���x�������Y�d56�n$�(T�L���r�P���U
�,���c$��$��R��v�?=����&��eS�'Y#d�5�05}�!:����Zi����M�A�*a��p
N�v,�S�����������a�]K09FO���2� 	��6j����������cmY�v��MY��0����!����k��#�|�ILQ%��Y�'f~A�_B���gw���K�%'.�����U�����$��S^'�&���,��D����q:�Rw�eU�	��r�s������zS����g:�a�r������ ����yY���X-���A���'F����p[!���c��x���b�RD�~������C�b�2����f��e�
�Lf�F��3&�������U��j���{)��5���[���4����@����P�����y�q3*J��h
�D��o]Ap�s�D��<�����A�P��q�
������%%X&3�k�s�-�m����o�
�)6��� ��[HG������n�����W�H"q�[��x������3��r�7���������I���uk�#[6��������M~h���}���6�/�������H�$tL�f��~;e�*%MYv�H�f@��&o�Xq��Cyp���w&�;��&:�4�J_iasj��-�]B3��@�C����
6���P�uq��b4���5IB�Ag��O�k;�������v�7���9f�`6�-�(��[��I+���<<�C��B�|Yh����=~�&������_�b�vd��������7wG�g"����}�4v�d�,��-�r�jU*���M>��&rk��_>���G���?6 ��Y��
����(���,��!���g����OI4��KP��O��%<�J!|�p���v{���[1%*yI�Y0�R���%Jf�,"+����}=�qj�=8?G�yX��Hx� �4'�����8�3yH��7dC�e�
G������+F��
/��0�Ya�Yp��w��B�@2��
�����25��q�h nA �d�h
��#���1����& 3�\g^}��a��vd������pF��j������ro����#N��\����Ob��L=��d�;���k^�B2�r#���.��'�).�)�:��U3
�D�:��r�)_��� �.8e���H��6�P�S�,�}+���
�8��C����.��
�u��Q(�T���+�N!.����{�\V��i�,x��`�]{��ad�'����XU���i��E	�l��(I��v���
�;F�����	E�����0	�M����N]&�������>������kI��->�v� (�;�O�G���.u�1����J��7�o<��$��+EP��<�Z�'������7/V_��z������c�#���3��x|��@��RX)@�y8=���	�L�J��4&&r������"������ ��~5������X~�EI����WK�L�a�'�u�U}TE��@<��U>����B:�bU��=��m�S��Y�LsR8s��ue`��������`�q�G�
�/7��M��Y�<�Bx����wf&Y����y
�pt�)j4�O��U6���^�����+O���H]��F���sC��tyS����z]=4n��������t7���8���F��� ����h���v�
�,&�!��'y��U��U����z[�����|�
sY�"�-@����\aJ����^Vs88~����/���j](&����C���
�_��e�w�����n�$��KdS\���P����#2���e�	�3^��u��������M��)V*<(&���M������e��t�+��=&S��e�}��(@3%H�
�!y�]8��sK���V�g�,�L����X)�e�kgBG	[F[9
!���Je��.#���]�Z>#���Z�9iA�� �\S��ZxA����x�f�#=�i��y�MfW�k�p^\+�z�.���)����??�%��49��j[�0F���3f��0P��?X~^��)���xJc
�����a�pJ&dRmV�s������	Q�G������bS����
Z���� �����[�9��!�\��	�����W{]

���=���^R���A�z��z�8�B�wF�>��?a�����XZ�PG�d�p."�"��Uuo���0:x������7w�y(%D6��:�:�C����hu>���rj�������4�@���W���i��#�����2�ny�U�"ak.]FD(���L��u��;�nPd�2���+K^�Q?����o�J����@jJ�4H����6A�����ZC$���(��H�l�#y�Q��A���������+B��G��vk�W��5���~E�	�"?�_�!��X�@_J���_�j���_IY�L4	���L���LFV������85�{{<|(�r� ����C1t2|��I��"�/�o�5p�\���$B�^��|V���4�����As�f[�^L�P�5��{{M����0�����U��'|�{8���@�*�J1T�����#�cU�#����Z�I�J9F�����&H|c&���m2y�����u%U0��'z�H����aQ>cV4�C!�eO�
�r����J�i�� ���`GN��x�O��s����I0T����E��Z��!�\.���*3��E�?(@�����\
Jdu8���z$X~e=_��������������I8�+�Mtl��m�wZ�25�$1�o
���s�oL�Q�\��u�4�P]>X�s��i�Q����E/n[��)cS�Q��Z�����$��gCl��tw�fp��b�9�+�8R����K���Ho�}��������%b��-Hg��^IC�O	Uc�\OQ����	U���Ho`y����S��^��t��������M���$�ey���
��� QW�~�������4>UaN����x�g���qu�Xc����S���P3��Kp�>.�����!�y^p�}x> �6���~"�Z"f��V���}o��m���`�����2�����]b��g;F���@Y�Mc���]{�]]����no��
�D0s������E�*�'N�2�|*S�rH�~�)7���R9�V�>+�SX�!!��v���t)r-�������v)��Cu@wX��N�m.<�4�9����i*��;w�C�M����=���-X1���
:�����3������5�
D������
Z=wC&flg|�di&|��|S����I�6,�_�P(�L������(�<;��w8�L��5f{�����Fe��^�����c��lt�B~���Y������8�&
&�)a�r�v��!oWE�8�&��RO]H��vZP�5����{S����~�*��B2���XH��[��f����C�o���L(
,
���q������<����5�LW%�
Gq�$�}���{���H�i>�9�.��x��h�t���6�o�����T}��4�;��ho���T���9��^�]n���l��A��@!�B�[t�C��j�y��A���3f�������s��{(�F�&/7.H�u�g�\t�� �@k:D��!��_
&e�����A�(�����f7�y�/Y��Y��Y�;,6�t�i����'�,�D8�)�<	T�����+�U#��@�#�1�����z����tR��	~���q���S�B?�F�����1��G�8��K�}`Y��u}�C^3|�/�x�_l������o�����W���^��?��X��)L��)���f�`�B ���D[TV��i
�xI4��g��A���j�4Fu$Z����A^k�A$�*`�_df�T/T��N���i�����V�	:��!S=a��n�S<�d����L�v^���Ll����n��+��
?h��!���S,��E
��]A�*��L����x2���}��pY��w���zl>5�B���r)V�O���P,��E��w�-Q�
Xc��fU���Y�h�t�F�(Cb���"����	��O���b}H�J("���[��(�[K?�veZ���9^d����4A�Xy��7{LNd��p����\���X�JpXqY������.�|"�O��x���@t����BIC|�D�8�^��X�[b�2V�����)-�"�X�y�4�_aP���VB���4	}2�R�]�k1�	�J�8����r]W��<�������m���w���_c2"?������/��/#��S�]v����xQ�M-6V��H���r�\bL�(z��~����"����_+/�o ^ve.�v������R��B�J�US�C��B"d�'�T��6�c�3���������s����Q�Xl����onqv������D��O"w�C�~�����1A�����k}�qz<�������|�,�����;*]�0U�M���?�:�u�����Fc!��\5q:h���j��-���}��:�_�c�~
�^��},v{�Ep�PW������D�-�[�sp�tdf	����JC��S�,3����q������*�>��������8c���M	�*��xL)�|L����<�����K��������|���J<R0m\��}���g4|��c�PL<��5�L.}&E�����`�c��[�.�Q=���:f��g�;�s��1���V��p&e��&VQ�����x�/��3�Z=(�����K�x�x�2�T�����}S�813c�����E}�������JR����K�j�����j�-NKn��Y�t^\��|�������1�\}k��3�%���
�!�$1�������_'to��h~`��z����7#��jd��{|��c��=�UM��i\�����w��<���:l�=�������{��nYX5��[�
X�!��x5q���]�C�V�|��G��{y�c8	�{dOx��,�O����)��,�����2��E����H��@��4
3�y`<8����TNhr��buZ�}�/��!������Od�/;f5��r/&�O
�����P^�Md�/b��M��|��r-�W.�N���}��/^�10�����4DI�tl�1G
[����]l�"y�5w'��n�Xl��P��l�lY3N����Fp���<K}�3iP��"!��T�#'�q�]���p���]�������d�|��V�|[U_N{e��2��1��lT��*yi
	����p�1���%&u@5�i��l����d�e��g~2�2?iE�-�}����7�U1v�;l�d���L����D|���]Nw�`��{6�u�|\	5�/��v�`Ob���k��X�%���OT���mL��[xP��.9��C�X���DWm~����
��}�,��������-��+�V3����@��0��X>��U��&��e*0������AX&��!]l[���������	��s�����V���
@�'N> ����p�� ����'
N�������=�FT *���0`T���XWx��f��P�{&���� V����/
�9 �P-��	
�-�g�!����kxJ!.�.����>�g����Bk-��z���	�_O��9li'��;�RK��|
�qy��������J�������@�d�@���8:b����O�������I�"����G[�-��������j,�	32�}��}��gf)\p��?)�;c�7��,/�%#e��|���M$�)�x�y�}]��f�"��r��YO���!��c3�l��>���i�\qJ�us|�^7�0%�����IUH ����O�EL>1e�h"�H7pz
l���1�5��:�F����<�V���2KD��(8����q�����Tai��u>: ��M����9>�
wi�*��`���;�k�(p�DL1�1u���P(J����0#���c���?.����/�V(��^���rN�����7��5��9q�l�����nS-��<�<�SP.}z<j����.r����p�B�l�8�w~��H��mV�h���ot^�n���G��-�\:h��#���l��1���0��IT����$�W�t
]�����//{8������E8�>�/_����_ki��
�+x�������;���{���u���k������#��Gx��Tj/�x	����;�S�G�K�KG�h�����1��<��(�Fig����z�%9e�R�_���E6�zsH{�%d�@!���F�u���?��cU+�G��<�����A;a9hy0?b�mu1i.��|�7q4C�c>DV�a��EL��,��
���(���gf4��(�4�99�@:�;�Jz �cQbZ�	���������Mx���,a�����/�/`�Gh�D�9�l��F��/m6���
1�P���)�(�	�e�@�����`5�ZL\�JX^{(�S�
���s6����+f���D�V��L���v�k�u�}�����R9)h���2��G5�Z�u�Zy>�{^����p���[6���zHu4�5+�C����>�D��?�D_x0��1�7�
�I	���S���l����%�	�4�*��$�O�����(��yx
������U��A��\t��2�w53�y���%z�{q8��*�P;���/�iS�`�m�X`e� ���\4�X*O��g��y��g+W�������I�����}^b���qs�-��@�m�-�
���g$��0u��vM��=����LBmI��=��M��������	�Y|�fgK��<��\4���f�@EB7+��
��*�����%5�B�L�����l���e���|��3�%���}��*o^L83����~���+�������XE7��c����hS��I�����
TV��C
�R�n���p�_���f��[��cC�E��j��r�rPs�[��T	u!�BA���hLh,���C�Ef8�	���t�`����H@�3�����E5��'��w�&'p�r�'���h�B���9�z���]S��>�gH6�~<C�c�����	�K7��<���D���9e��!El:�����t�'���s�r����,�W,�mxt�A*������E��m���BUW���J��h&����`���1c2&���Ki��
#����ZV�i���J2Z���G�� ~8��X�;%7�+���`�H�9�*�q����5g��|�Cg����*���gw{S�ka���v)us����F��1���D}n���w�T���J�I7g�y�(^#�$ny��J�4��kQ�qL=���f�F(V!�J�J��Y1!m1bf"BXp	j6�T�����1����C0���{u�S�b=X.���3=<�Pb�����(93��=�*d����\j6O���	+��-��o�|��?��r�+��L��Lg��w�*{!��&����be�	a���iY��\��l��k��q9
���M��g�����(RZ%�����E�&>���j}��(T��7eH�L[*0`4��_[y��s������TYM���H�*�T�[/�%G���S��r�I;u�)����W2�����.R-���HP�\�4=@N
������E����:)�QO1��N�(��d�����E�
@Prjqt�pk.'H�P�2�`��=������Y�F���5� �?�D/D����6��'�~��|����J!���]x���E����~(����&D���~j�y��$�l������j�@L�8w������Z�������W�M�d������-&��
1$taC�����m;q�����D	��N^�����_�cwF�����A�eP>m�����B���W��l9<wl4�6��X��T��6V?"X;,����~7;��a�
���A
��@��
E&��p��4��E>���)Z�GG)��>��Jz_u�_G(G/I>A]!��
#s�r��wE�.�v���?b���}BaI����F�}�r	�<����{�`��|S�O��q��j'=��r���[�x�j`C����\�����	�S�����QJ��$��Y1��q1�A���|$}�v�����:�9~�*�D�3���(������>��*=i�^���*P��bSW��FQ�o��[U��=�P�[�.a+F
���������f[h�*�6W�mam�?0�����Mg3^�����oQ�[���������p����bUW3���O5��U���	�_�kq|���0S�����>���=�����UQA��~�l`9G�T+_v`=z�U/����'���|K�B>��a\�7m�&��h�FD�Q�d�������"���5
.{?)�J]^�v������.���#"�
�8���&�����^:]q�(�%��D�+�A��v] �A"���}�$��w���Z(�gE��@�0K�������]dh@e�|��,��y�[�)!���k�d���\�M�&Rd ]�!�R�V�!�����j�8���~9lA�V��@t�0f���e��r#����Pq���A��X-G%�4��N�
-�{��(#���}�+�����&��Z�O��m�����)\~�������ENiVW~����~c9��d��	#u�����W����������r��~��F��#������y�����7cJ����
.��-p�$�����j�1A!Hx�A]|�7�%��d)K���f��������=&�?V��t�~y�:��:-�A�O�6u������x��$���X������Q��Iy������&��eh�]��m���C;D��Rv���%n#E1�da�U��]k|���I/����S'�x��������g��z(���@�,C�a(hUl��)fLJ��@� ��yI%���[s�- ��H���$���{��D���k���&����%� ��5��K/�x�����������b��[�sXpM�_��mZ(�Bn(�����z��
L��a��@LJ�1s��ql�t��JA�0�G<u�8������
�U�'&���!�M�	"-g,�v�!���K�]Z��������=L�������Jq����@��]#�w��<]�����������|J��aJ�=�@p�H�Z����(?���%S���?�pz����^�������!��q�x�Gj��UO&�7bL�r����U�g�� �{1
�k���������9Q��O:���%]l;��f+��b	$Y|��Z<3D�n��ry���P2O���H������~���[�if�5�q(
�/Hp$���Y�h"��6����<����$����`�����
	��0
�6]�����z"��#�#��" /Q	'��i�t�8��q���B�dv�RwS��)��d���gqs�����>TVM�B����y�����
�������a.�yfC�C�_�=�d)��������*�s�c!��;������B�,y����X�??6����j��'�6�|+�����C��	����%�� �hq<����D~�E/r�0��!�Z�-L0�;
�t_�=��B�8Z~�K����"�!�$Q|�iT���51�@�\���[���=�R%�"`�1�G�Fj���Rs�tB��J�x�����H�:�����
�����a�FYh��������'���#3��!�0M���������d�n�^��"�h�>�#<���I����c��]�N���<0�9�Uv5�N��J�T"X��Tn�J�.�$��z���\�+��Y�x�)h�T
^����Q!����_v!������r5x��ZY�F�T�c6�0
�`��D�g5���
�4����^Du��^+�j����+`�.X�e���J�d}��}tfD���������m|��;����Z����$�|���
�Ew��e
Y.b�h��f����������f����%2D����k� av����4���K�����5��DD����-2�4rl���z�h���C�����
�
'�=w<�y/HU-��^4�A�E�ypV����lx���\����j��?��R�^b
2��a�)gR_��+��Fcw^C�$1p��#
K84�b������D��rE�q����.z,)��m�����r��X�7��q�@�_,a>
�C��/��6u���G9�P�(y�h�]Ipd�O{����d�S�TP���E��4	�k��_���T��7����/���V[��������SQ<�s	B]=��jIQbq^<�B���$�Ou��l�/�5�@w4�����������e�"��2X�WN���
|3��/�IM��'0R�rk�y�Eq��W��+�����-eH4D����G�m������Hx��P)�Cyp)�� u�$Jc1o�K��������Az��t2uW(R��������rM���A��>iKn���&���0pD����K�zOf;<����&��W����I'�!����RM���{-�u���}��,�]��4;�����w�^r7o����J�9�uU����N�C�EN��ZG�	���x�@�K�n��!�.mR����CfP:pb������f{��5f�������_r��'��B�f*�9p?m��u��/j�d���`�0���p��=4���iU��5&��l8`0��c��=p<0�-$F��f��VX�����K��5X"�w�^��W
P����"th�����6���a�!A����O 81Z�4�x2�C(e]�2�����J��R�������B���M!�P<�����������U�zY��&m��2�I������R�VZ���1_��dN�0��6�x0��L���v�z���J��\RgF��zlw���Z2:93�Gr&���V�Ob6���@���XV&��UY8i���x(5���)����F�=�����/�6�#�0?�uM
e�����_�����~-������K"����;v����[%��{�����t�,1��3���k�������LJ���..T0M���g]�<��\�D��d�9�Z��8��#���&W&=�eI������g��g����?�����9�~>���w	!����a$�Q������K�)M��g��d0;0-���|(=����&��_���'^��ySf?V
]e?��JU����V�/U��@ ��K�KAA��\}��38o��8�����k���yl$�����!����H�+�wx�6i��.���Vy.T��`�E��6����o� ��/���~���3�Z�;�������P�6GJ-^"#4�����g&�Z�����(�d�y�C=�a�������M��{���Ay������XN�Ao�^o�by�8L���r�:(�1S�2��z�����c{��X��2-�]������$V����0���o��n*�0d����	��C���C��N���������z��1��U_`�\�|�h�y*,�+
qoM��
�&�c�����]Jn�)Q�F�U�v�����OD���H�wn�D��Bz�"3���n����V� �J�:`]5=�f#w�j��V���q�\���W3��0�PZ�vV��j(����x�e��M�����Z7!7�vS
!!3������*E&fy����0[�����R^R��Q���Z��(o`�{Au�O�K�Ol�18���/��W����d���M��>��|[|-@d���Li.0d�����5xQ�[$/�:���H����{��(3�cx,bQ��0m���j����}�*H���D��������+�H�� ��1���b2u{?YE��8�L�t�v(c��&v�����E���o���D�S�n���$�n��
f�Ln�X����h�����I��P��������X�XA�^��?p%(/�C��4HL}�-p�/���|[�V�Uf~���Xj6N%���>�J�S�S�$���*������I���1i!�\��M��DI#sW5���q�� ����<��|�x4�g�����
���?������9&���*N�o7�����JQ`���t��+c�z�8��x{|��h�����"o~���gEY!1��)�����iY7�=�u�:��J
Py>���2FyG��JS����U�-3^�V������{�Y�6�����6�GnOt�i>
���$a��/o��L�f��%��Y�F�MK�����D���� ��J��$.��&�� ���������Z���{^��`����v�)����Q���q�z$%QMF��nvl�b`�[B����gJS.��rM����;��|S������A���A�=a���|(Q������~W�REXd,��O��A�E��\NR��r(���Q
��A!��<��%��rE�Q
&
p�� ��42s��)�>���n�b��U��VH�n�����
�P��^l�I��g"_��]��������Pj��r[b'u���R�tz	����=��pO�����F�c�q�����<��L"��c�O��z�����fE�FTt4c�6�iN�)T�@w0c��������������n���-�$�)�����x<�I�@�����B�z��1'6�[��
����iB=?#D=Ci<1�0�:�E�k�����qd������QI�2m����+�v����$v��� �����D�r��0Z�hZ��U�Q���iJ�|b�
�G@Gd�Zj���!�?�?�$��g����u8���C��P~O`�9A�g�����0���"��Xu��O��\��>����]W�$�DI��'��NKp@�����"
�<�;,���._
p���4U��$	����F<(��S�Z��Y���P��QW�`i������o�J�RwZ��hnH�yb-7��l�-�AXCcu�����MfH�������4�����7�8�~"	��3P~I�����w��\��jM��h�S��������Z�?h�K �
��B^�}�;�40�B����#-D:�
5'i^�;��}
T�GT��W����txF��yql�gu�W�6���6"��Q�s��?��������U�b0ut���),5�=hD���������{R��#����������?�m���Lv��������-$5#Jst����W����
R�'�(�xf9�O��@��H��~k�o'�S0��D��[�v��V=-@wkz��JA�}A�n�2�$�O�n��U�6�m�.��-|�������' ��Lt��\@kE����Ez�Yj@>�U��������,��Vq3|=��p%�.�EI�t�������K3����>ft��Yq��^��	��4^a��d�`@2Q�w���s���}�l�C�j1)
�2���4�z���|�
aG^%������:��Sq
���W�2��<�V�B�D������7IS`�p%�(��R����"4�e(���q�����;��}����m���
�}�C�1�)��nJ
���+$M�*�)k�%�xZ�4���V4��~E2e�������l30m�>����N��i!��w�W�|<b2�G�7���&���i<��$O��{�s���J�H��P��ly��X������Z��Z*�\��EDg��2z���W�]Ha�S>D��!a<
��D
��5o��{�H�'y�G���N&����}�~{��������c�K��"�>�
y��,D4�U���`a����v�����?�O��[m��|�]���f<�"	8>;X��}�X�6�ZS����������z+��!�������l�y�	�%!���&"������W5*��<�$��`�/U� 3��@^�4.�����
D�5���C��_.��$P��r	6�C~�E�C��+QN�4�������I��=
��f����|�zZA���XC=�ab���V�#�fH3<0u:�������e����A������l��CQ|��v��g����K���@�0��������#��m�,=p�0�<����d�����|k7���>z�gH��%[���9;�XW��Ow'G�^,�
�0��I	�c-���(l�����p�}������
����K#�ixT��]|���l`\s!7��D�&PI�j�y�c=;���,�:V�#l,F~�"�p4�j��T�����.E�U���4���K�8��b���[��j��I^f��'��&� �-VG�]e��	1
H�������Z���v�sy��Q�.}���X�O�b��o����@�R����@l���(Mm�A
UF�D	��@�H�4��sj���S�;��(��!
�^g#�"=P�?�?��"d]>����g�9$���\���ps�(�v���i����X��mS�K|��[4M���P�L��eJ����b�GF
�����7LZ�{��H����C���.�9���?�k6�������CMs6�s������*��a�w��3�JTO��d}�m�h���ls���d�����������I�)f�H��Mq�k���z&�3�ol�@�<�����f�������������L����a�X�//(}�`
��)_6.��a���[���	4(:���x�94����/�R90�<wud����u��@����z7AGd������,�I��^�^j���F��X�	V�3����1��?t������^w ����)���:���dl��g���g�5��
������6��!�Q�{L�����~��*O�����
3�X�A������������T����k�0Z3�`�'�\j�jU�����qG xR�j��!/R��pdZ��W�������m�X���rs��@�4&��>l�+�kB,�e��&`c��,������K��o{ut��o�x�a�Y~9���]iQoi�'�pb�>v.����K�����^�i�q�l���I��%n��G�����~��f1��x��g?���Z���+�$��p�0����K+�5S�(�q(�7��7u�)�����.�U"K.�r�m�`&�!�k .��A����u�pv���z�j�A�?��c��X�S�N���e,��(19��$��fP����V�g�����i��g��1���)�4��id!!��J^^�9����Pd�����U�-w��#�M{��w":<�o���?|C������O���=����D/`�2
�p����a�<��	2}��Vs%��	�<f��75�������
�q�b�<�]��wT�d�#;���`,dCn�\�<�k�w}l�U7���>����(��h�8��b�V���P�%�>%{�y��~Y��4�p�!�mk
��t/b���`&8�CE(���z�<�(K�o��J'�51��\*�������G���]�M
q8f�G�4����g��/V_����64v�������^�o+9A4����a�S�e���f����FHB��8��D{�(-fC�0��n��s���k���u��+H�y��m������B�tE;��D3HX����P�
���V�"��C#�::��+y��:�2cR���=�0.��*�e�����6�3��K�50�U	������@�%�
phOz�F���R��BC"�a���n�D�����^�[�������W�_�������3�<<N�6���,��I�T�9Y���y~�<	w�U?������,��r�|��x�	���y��jMUL������r�H�-�*��������=>�m{$O�D��A��c���n���� �$	���G{f�CX�*��w��H������?�u�}o,�r>D�*������������gf~t�����+����p�XTg'��[�?v�A!��\E���CQ����D�<_](����H�Y����2�iV2`�.������o�bU{�}����aO�R�u�cp�0���_����/��O=�	�0-�@��W6�A=�+���b�y�q����j�E��b6��w��.�8
�z��?�y�J�Z&���V��r7����3�>����tU��~,�D����<D��y���mRgG5��-�{��������%�����v���1YL�4��0�2}d9"�������
����a�]����<I2��E���M5���=�F/��������G�f���d��	,�����f't�?�+��V4C����\���=.n�Y��Op`.Iw=C����P��	5�f4�t�Lp��R����������},�.��By��9�������'p�{*Y�?f�:�c������|��we`F��h^af+	���@|��=n�E�_X`�/�Ca$%r��..:�z��I�."�Im��,��z�C�@+���r���1
g�6g�����Q���0������@z�����o�c#�u��4���,=�������o��R���Q���V�]���$��,cVy�W>!����P������uP�.^�����i�4-	��r���9�����Z��C������R�}R@��&�?���8D�d�jZ����_�u����)��r��}��@�A?{
�����x��|�%Vg}Nzz��o|
`�'��/������Q�{Z�������$��M�D��M����xZ����U����H���u��*`��6�����v��"�h��-����.?q���&�����Lg��0��j�������=Y������Xd+\5G$�����!�UK� �I�����;�>>=��_��Z�1+ ��5�i��U���w��������a����%��mT�	�������������]�!UR�:�����x��?���D(
�7�Vp�R��j(v��^�fx]�����g
y����	+�W���D������1>��zK���,�#��P��7,n��	�p=����DU������~��3K��k��j���j�����3��<��!�����E�T����q[�)W�P����4T#�������V�	�����W���������W��r��5��H/���\�o���9��\{<����w���&�}R���������1[��������W����(�`����d�%�; �9K"��|��~|���$�C�g�w�	�s!z�������G�\n�V��w#]����f�)O�ID�.~3r7]M�^a�7��.��o��N�f� ����R��m��c�����P��DX#��0��fX���d�B�T��<,�<������E��Z���1S���[�k<�t�E�+)�V��"�?Z�10
��s�������"
H�������c��o��Wa��i+����C�?���1��rMN���m�c�8�r�c�����w��~?l��q�{���nIb�b��
��E9����_��HM�W���v��
t����������;xc
�54Hg:
�y�YG�w�\#��]��u�a�2\(����@�����N������_����<�CQc<r}1������n�0j�$���i}������T�����W�HgBk���2V*�L�kh^,��0��qnKE;`��q �	�
�A�'�$�=��y�����-m#k.)�tl7l���?�<y n,sz���jR`�4��}	D�z���hV�i����q�&��J����d2�
����*aTW��[�r"�d��6�XC7��(Ba��5c�5������AS
��V��E�Z�D��;��fZ�T����nOA��d�B���(�~�K� ��o�8��������m,�f?���-Y������;�
'_���}m�x}�+�?'{q ��`��M]�8y�s,>��4���x�h�Q���,@h�r���.�4+��r\��`~
���\�,��}z��g!� �
�e��^w����:"��(l����8F.�i��,���&�mHrwht�;Bn�N��:04���\e�LA'R�V�zo�I���R���deL���#���Q��w���CT��k���g��������7�V�����Ee8��Kr��c*9
����Y���9�t�~V��EB����Z�D^d���ZW��M�8�5`jGX�	m��KL�p"������d���
�I�T�������b;_��i�X��9��_���(�-O��i�M=�@�m��h�����/�+s<�#��0:&����'�t��:�IM`3�;�tM�?� ���@���o��9���5I��B�b�R�b����n�H|��X��m��\�B����M|
uH�%���i�GP�6_W�*y�Y�y�X�B
;�z!g�@�W��kGwH�����,3=��I<��/
t�����h�p������g�r��e�_|K�+��KI�A:�
�
��'J��7��AF�kXy�(��{]�^&,��PSMx�w��H���1���m�I,g�h�!��x����B�"d�+{���=N``���+g�A�x���p����r|8����d��P3�d�M���D�����|o�m���k�U��APy� ���Xz���i��9NZ���?�i�	!��l���;��]�t��������r�Z�hF�-zK/vk�������%�s%�:�,��l������w��
�z�����m�O��$��P?�;T�o(5A&��0�lL���Q>3s�H_hE�L�SApR]M-����j�������U����0�j�����I��G��e�<s����+� F}���	��l^,l��`�W�l��QO��B���lcT&��4���2{e���;?	{�P���`x�t:��nC�P�d�_�u���o�QP�u)�� �y^Sx(��M$�����Ye�D���{od��(`��stE�}a�2E��9_��?n��r���xo�*��n�]Sh�jW��
�1��O"8vJ�l����
���s�I}��F�d�P�������e�8Lf;���U�+�a)X,�0���$�O���SF]/����_ �Jk9��aOo������P���Qv����J���/�k��Cz�7�"{�-��O
��.�r�� �V�ehEV�A{5�-3��*����,,?������&_��iL
�j��F����|���=�7
iqPN���N���0��M&0�@���@Dz*W�#
yY���~!h�:�?S��xX����Z+T�a�����Z3����+�C�5��V�(>����~���!�!������5����:��:V�zC�B��R(�����Y��h5	�W-��Ajkf���h&�a��"�����4����
��CHZ�����������m�?�v�����!R�(=T:[��t�	-u�Bt~���i�xs��o���k��28�[V�5\���oa�� �D	����K��P����J��o@���C���G��r��s�
���d��N�����������B"�����D���TD-aG�k@��f_�,�^�2�/T�4��x��G2-�:8���r��X���U2)iV��M\������l�t~�W�c�*��H�I���c��/)q�|�����f���[���E_aR$���,0c[���1�������6�0,�.�gjD���B�u��B�,
�
,Kv����}=e�Q�7SZGd	�]�)���Yb[�/�Uh Uo@�����/��1����=LKz�\Y��5W���&�/c�T~��T,����Y�5e%�)l[Y�7�:�g��^Wt��ss�!Q����u���
#jv����!M�=�SI���%�M���!�E��oX[t���o�����+v�b�D�s����}W�-�(}�k�7`��h��wo+�~W�"�q.wP��m�%�<Q�j����DZj8{(��m��7����_J��|[�6�P�� ���C�Hb�E=��'����o�����Hk0��i��DN��aYs��N&5�(�=!��B��g��E�P(��.=�xw(���Q����F��xT�:��d�48;Y�������?�M3��8@D��9�^���@��uz�xT�)v�=)VO�h$�e6�e� l� �'@m���]���.��6�G���U�m�X�}$`����k+h��M�k��?��,Z�����&`(���F��w�&`xT"`�N�q��/�OF�T�z���`i�����q_$�����������;���jfjN����ce4�}�~��P�7��M�{��_r���e�HZ��4!���[ z�@��'�����B+���dEs��k�X�;2�k�\���7�����/���|��9��]$~����������)�r�Xk���|�1*���f��`8�;����
t��3I��1�~A4�����)A,Rnoj���tp�:�-Bl�cP�����t�x��z���lS��]�~9>�'�Q��D'Q��1
�����g�����w��1;�\����o������n`��:�����i
��*�l�x���j-!|���
��PX����E�*+�����b�M7
"�v����{T@����0	�� $}�!Y�3)���6q���u�BbO��C�PmI1�T����i��U�HRu�$qi�t}���xqQ�2>M$��e�To���I�W&ZO���m���t�>�
�R�Q5q��{v�ug�`~����}VWtA����
;�j%&-��:7j�����l[��05;
^=(a�R���h����<^�0P��z��/�%?n���@��Vs�n���f��f4D�M��S�8�����"��[
����m�������8��F��������;�1U.<^�p����CQ��k����^�E.����n����K����5QG!d�Z��nJ{����y^7F�6G��bl���4{���[�i����m2\��
��!\-�
�w�H����9n���Nuc&���v�#+��3����m���w����6���"�: ��\/���,�E�m���.E��%�e�8����o����qW,J�sk.J&���l�=j [v��7;eleY���$�Sn���dn����,�=Dd_E�H_����1�yP����f	)T�����X�B#�H��'�8���B,f�V&�(�C��������P����$��SvF��snc���E�O�jV}v���-��i�h����h�P����7�S_��tQ��2Z�D4$�!�����.��9\M�mD�i���y�V{�3�b��I��x�e�$'Q�"���Z>��8���e��v�;�MD��}���5���\M��Vr��$$����b�
q��g�1�AwH�c22�;/Uz]�9A����w�-'p�OC���v��L�~�<�O�����61(�T>a�>,���rz��C�������Z"A�b��=_��b��>f)�H�3}T0 ��2��?f�*^S���U�y�q�5�o��N������S�s��Q�X���"�bS�����t\��V����z�r_,)��;~� �������������g��i�������3���d���a���pk�1=P�P��<=f����}��V�>���b�V�]�W �]�����l���?.�[]k
�M�h�_uB`�������f%#r���E�P]sc���/
eY������j�<M��'!B{�FF�QnH��+�eyP��q�P�~��*��c������j�ud0}�^��������a�
q���K�m�Y������}����+��z|��z�M�-�/P�b����w�8xPT�����J��4A�+>4�p$��G!F91�-&(a^���L��2�K7�q0`��-4Sj�,�YMx^a{�k�%���Xn��!�4k.��T�Xk��P49�U�0��R�r�\X�C?A�� ���	��[�(�`��Le��g��@�L 2T�������~hb}����2@	iH�
	0`�	���f�wc1	S`qz����*� �uqC���RA�76��@g���~4��Y.�[pTm�Z4y��H���A6��;;�����Nq�+����������8��
�_d�7���(��wx�������K�	 c���
�
�c:�V3�$������A������8M.�4����u4�����:�hq���P�Y��h~�������e�,T�w��B1����V�!��r%Z������>���>
 ���*N���@K���(P�C��l�	
�������S8B�5W��^p?�>��--�?��:�����-�/�q�����$3���d�3!>�d�����s�W����l�Y�a���Y�4���G�����$_�	P��n�+���)r���W�����MT����1�7�usDB�p���9�<�g��E�����/��-�<aN����ZS�����@���a�N�B���=~���=
q.,>��@�a����nu��e�j������r"����_A�W��]�bE���
��~�SS�!���@�/,�,���
"���k�6�c���_�ve%vn���;�?�29-������(kYm$�*�W}��z],�=/����6C�x`��W������u��Z���R��4�p��z)�����7��q�V����0C�l@t���=F��}M����5�Dcp�'���(4U��L�b�p��b
)����>=�A@;�X����C�i�c�?q�CS�[���U`��t�/a������v�H��<r���+��jum�X�5�l0(3�����bu������1&���H�����E���C����[����(�	
�{�0w����]D��e�����k���;��N-�� e4� �S0��ew�z�Q2�Y�=�>uo8��f��V�������2'���Z.����*�Pu�
���d�s�jF�Ue�kL�����e����5�Y�	��B���L�e��m�Z`L��YW����s����n������h���BVb����Z�zh������������2^�HB����*Xc���U5��eA����}��&�e��W���!�>�b�5J���Ms�gu_6v�':�����a�)��%�����H04�^�b��-L�������_���T��������d8�J�z$��n������l���+�Y*F��oX����Y���"��BZ�=��OZ�[�4��'�����]��R���Y�T�1��Zls<���q���
�3~�d����Tz�+�
0?�C���I�~X���M���Q����Q����*k�#�q^$;��
)~��0���2�vNr��w�����<_�3�J)9���9	�o���c�R��r}��7z�Gz���.w@XP�F(���1�J;�}����<��Z4���)|�����>�g���&�V��2�SY����Xs�D|[���w�O�������7����	��kLF�@w
�U���;]R�I�S"b|�� M��\�9	��*c�+�Z���z(��O�DZV��sU��O�9��Z��]��1���p�����s���j?(�KN�7���jn������)����~=u$q&� ��X�����el�npp;J��Kp~�h����X	r�'��	*-��Xn�:G��59���P�
��O"Ei�oB���c�X�M5��J}�At^k��m M��t������N	�1�(����(���j��P�_�]����t�2-�:�cE��;6��I���BN�^C����<����u�&�p�&�h�4�a����f�����s����I���-��]4����_���2��ruy�	��qi0k.S��	���	�I\�8�'������;��y�4C���sd����m�ei(@Xn"�Fa^��!%�n\ U��n8?9���7���:q�6T8�S����}X�x�u���?&0,���lC*#��+Ll�R��x|�"zfS����C6�[�v�,�Ve/!;(S���]����k��|�oA�CctC2�K}��:�pl���w�`?����*�
����q�M���u�x���m�]n�
���� �������@yi\�d����JF�+p�T����q�.�a�5������_�����gp��XyRQ$OS�K7Z��u�8�7~I����m����lH�WY��^� ��������B{}���mq�����r�b��W\��Y��r���4o�<q�A�����)��c���[������l�U��r��� �����b��F?~
�B:���R���|����<��������36zG?6�4�W�������������A"v3�,d���)����#F=<�F��#N=�'��]�9bb�[j�c�����X�Z�'��\mw�\
�i.:^����&���������;�������8�Q��p�){8��?���rG�g^�>}o��g

�K�0J���%2.�z�jU�~X��4�AZ2;��z:��M�c��f��u�����������:{q��o���/UV�k���Pc���%pf��l��/lt��Z4	�W�d��CJ5(��X�����"P+"�����^�'��Q��-H��tZVAE":��������1Y��|-�!�
�9�ge���$�b��oiV:����&������ @T;������E����<���
u
�B��J�/�����y�&�zUgG����$�2�����_�b�����f��)�
��������E��BA;��!{�����'�1��{�U
`���h���f){UV�v�,4�L�S�����1��;�/�9�<��s��\��������r��$@:���4�+��%�F�dXDV�u���i���4@b��� �o����@v�D=���9���d�5���0"�{
Fi~�����hd��,JO��h�
����^������z��1@!p?��m��\�$2$���U������}�M���X*aQ��k)/����
8�P�qi8��@nM�MsQ@Xe����s��Mh������$J
��uQ�U���O�0EXhe�?q�NBq������i�=�qRk �@4F��h{�������	N2�.����7��N��s����K��������~��T�����vn�@�@`�#���1�H&��D�"0���w����s��j���O���}�V��	�	+Z�����Z*uY��6����,�5��6�}CR�L�7X��.�m{!�H����|�(	t����2y����c����P��2��TqLF2���������'BFz>�����a9�4	<�p�3�O��/K���O(w��KOv�[&�:�����.3D��A�.���EOlH5T$���B��
^��J������R���I�<�J��J&�?DKq~�������-����s����g���Qm�OY������+��%���,
0�,bcvqq[%���V��E2�B5����%
u�����lH
���k����AR����X�zb������k���^m���c�+�5�������/�;���vG�V����}��Q�W��2�Hu�d�Tw�yi7��@��a	�`�-s������'��	��R-�����m�������`k��`S�)��%���k�6����=�4�pO~��r?��GP��eC�]_�Q����7��2���8�z�^���l�J'�����SN���a��N�Z���*���t/)����NHQ1��������n�y�����u��ly��v���Y�_�CK��}�����D!s�����lq����	�z����W���X��G)$��p0��������Y�h7�����i��b
p�F��C9��!�3L�����'E��������n1�\2����Cq%f�2$.����7W�#��������T����T]%���od�r����$G����J�t�X$Tx�����s��:HW`���c)�_e�0����nS�P�=Z���B�Z�j�E�@57��y��b�FN:��^Xk�6y
 �^cY�4�pc�|Sy
1��F���e`TQ*�|v"z��@��	��"��^Xs]�A���%S�LO��x��_�tW@R��z�v�wD��`���}��h*����5C\��m-�A�K���di�n���a����������1�o��
ug���!\&������%io���f_��f�����Kv_!���e���};6m����/���U��+��J4�[�j#��%�-�A|��c��?����+�x��K9��G����D#|���<�<:��]��R��kV���Wn*��h���������Q���U�W�����&r6!�	u������zr�Q5��X�<��T��0�m�Y��MV
�(�y��NT��	c
fhg1)����Q���Vp���;���?����@1(G�-��F#�W�����+���R�z6�!~G���][�Ny��������A%��A[��~Zn�T�M�7�K����r�����&ZE���%�n:]R�UR�7�u8���!�U���b2
!�b�����=�h�N�����S�
���'-mc�������(��KM�9x\R���`h� ]�����Q(�SY�D�:�A������9y�~��X����/<����?�L��KL�	�-O
��LJ$�p��b��$6������!��GU(�@h%�����g���%�>f1��?��6���P�W^�V(�E��[I|t��-���N�+��y�z��}z�!+�_?<%�Ml�p�[�������wY�h��FC�R�^��$^�>yG�4����o^<c*��s :�0���x�� �QV
�F�l�ueEx$�;�N��WPGWd�=�W�u��6��n���U����X^4��`E7�S�]�K�Vs��JCg �������������^3��6R#`?�9,����
%�+����"�h��H���J�*U��AKg���3Z�BA��V�&�$��Fh#k�������4�r�L�U�_'����o�*�?���%i��n����#�Xz���jYL��LN��	Y�9���I���
%]��B���l��&�1z��J��?����Oq��+g��K�����u6C�����'t�:~���QYh[��=d����G�@#����X�~N�1�z���u����
���o����s��R�����uan|����P��I k�P���F��:�
�����}�:��0�P;M�_� $�t��@�
iSn���s���O�S�������C��gR<��H�(:��]z����������F���]SU�[������0�d�W���V���V��<���Q�_�=�h�8�l�}���id��H����4�0�����dl���x�HJ�<�M��Uj	�/��V����I�'��et���=�PI�#w  ���}����`����-%��Zlx�wq��J/����2����b�XQ��r�L2���T�swU���6���m0���>��F�"��}�,0S������I��������]����N�O!�~��/%�(�S\3���X�F�-��))�]��J:Jb%h
����<O�����s���v�u�<nm-\���h�v�A�u��}����1�����n�C5�?�8
@H�����1!��_�����!\�"l�A
)�L�Y�:\���
���Tq��G�@�+���J�v�M���:�����(00���r�M��K�>k�;��H�?����clY	��5��$��1H���K*���:�v�������u:^oj�C���U�����$�e��$m
����{��?u��
w�8o�8$����WL4���b��� g��UT[����t2h.n��8P��1����V�,J�.�{������^��u�v�q�{5-]`#yaG��z9��p_4����"�]T-�r,�0��=����C�l?wo���P��a��v�������C��j}I�}����A'�|��A7�*�Od������l��d;���X�?�]��Tw���m
����4y&����)�^B(�I:-���+@�[\s��5F�������)���"��&�e�L%_o)=�q1)���!�,@D���1k"bN'�q�%���?5�~C������6=<��aO�����;>�������C�2���
�`���J�Q��>F�MY���h12�S��Vy3�e�����C�=���#Y
�������/��X%
�K��-(k�/mUP��!@o �����C`$���5�y�JSs��k�!#��}��U{���^�h����2|����D��)^��F@�^�BWjL-g�@VX=�w2(�$+���m�5�Lou���ijLVld����(��E��������R�c$:2���%�S�4�9����U�D�����/�Q����Vw*��x����I����?P�0�����&���xP��QdC����y��R�O�;�Q�^e���}�����e�����eh.jL��|m-_��t��kw]���P3h�P�2������MG��sD�*������<���.������N���Qlai�s���8&�o��Q�Lr�B1E�m`�nj�u�k�?�I��W�Fk�Y	���F8`�^<<k��m�wS�`����@�9{���3�P����A��\��EG���#������':��2���Tj��W�L�v��']������S�7�A�;ey����v�Ei�{������A�i��
sA���t�I�B�C�>Q������@�}�o����e�y�d�~xx.h[LCK��ghh�_�����Z������������
��i���%���5�36	4r���{���&��+q��7��#� �5MK��?���o\�b�{���vB��y���`������Q\hyCT���!q�=��U_n�l�d�
5&N��k����K<!�\t�W����-��_�C��	�;S e� Vajn#�����a3K��dOi5�&O����(�eu���c#��s
��t2;4 ���t��1�Dm����>�����Q��a��v8^x$��Q7'pt��?E}[������~�������%S�����_�L���a�����(}9V��yVY�3
��Rd���t#5s=�{<�
<z#_f�4B�C�L��:��K9�+f�c�������s��O���"��KL�i�������L���Ev��a�X��p~]���qkEJ��
�����!2��!�6�J��vo�-�l��4���`�*�O� tq�����Om�N��X�7�7�n�	�op*�X��=�W��:Cu���@���H	d���[��nq����7pY�3�<w�LC7N�+OW�D;�����re�~�+�:���}X��Q�MzS�]���z���>������Y���9���
��i�v�����p�at�.�=1�c�Y0�5���hC�}v�
1����pvsw��8&�`;�������Nu������ek?�LCe�<�c�*�{��/�G
��MY!?G"�������u� ��xE����,��D�B���5QwE���),"��7|��!iC;R���9�.x���|���xeG����sPMF�
����
=1��T�
�H[k��w9�uu�vC��S��q2U�8q.�C��cFVx�d(��B��H��`�^_Ej_����+])����
���I����2j������4���Bu{TU0$LV6�zFl�"t���Q1�hAg% ���$�k��Y`a�G����l���$zD�4OtIx ������H�aGS��W��m)�p�'�%�5s?-:ZO�8�<�����dH)��zU �;Eg����S����Q�&�60���r��W�O0��k|���Y�=��4�8e15����t�^�A��,*=�����]�T!����K����~;')W�(U�B��[�V���Y������
��w���6]� V���Tr�GI��:s��]���c���v8���`��	z�[e�<���!|@���hG�a8�$����z�R]������^�����Zu��/��^�_��K�!����Nj|��8��u��P�:��?N��9�w�� kz:l����jw@Ci0���
�}
cG�w����v��,���$�E�������D�Kb���0��5��LlU����2��F����q��*��6�����6- ��%�"������o�d0�p�8T]<8��L�{�E��p��Mgxx�I�����_|�d�<oV�<��0<�oD7��$��Kt@�p/u�K[���o`������r\��)�����&�V����p�Z�]�������{3��,���X����#��:�)"�x��
�5��@,�	����|��/���(|5$��2�����YO�f����8���`�����������\?�
�P:;���b�G9��?�����G���d�:���}[noAK��E�eI�1�}8�b�9<�Wr4[C���j����������C�[c�����C�F�_�O��iQ��R=w�����f����,&0��v4V�������;��v�NX��������~�/����Dq�SY+�d�8	�:sz�C���<_`�<]iTxX\����
F����������������@x)������T�h���jk�b��)�&��x�5�_��K4+kB���p4� ����hH�F�d�L�����c�8����y��2YW�6��a�M��t
���j�� ������V���r�Q-7��;���X?�
�������]����L���9;��{�����
����m��o�hn�Ue��Q(;�G�9��Q+a�.*{\�2X����Ih��bB����n��v������j)���d1��g�)��W��g��
��
Jf�Rh00�������.	Q$>R3q��_#H1B��x.����z��md����s��2����)y���Y�v��/u6��%4��$�.�M���`<%��u�?����Z?���/#�1\%����|-����Ix���6���}�u��^Z���;�8��������Zj�U��geI9$�AY^�$������?A�zz4���oW��r+��(_��Q���P���+�L�Z7��9�g�����n i�J����tu�����~���?&�$2��\�0���]R���4�\r�K[M~�h��*���c��"
��7�D��,��*�)\O�t�$�m�6h�0(�i���4[ ?(���e�~���Z-�����`���	gxx.��n�����mx�w
�������I���ixL�{A�]�U����4e0�aB����:�
��
�c�~Z��9�m��Q-��yg
��J���6�=��f=��Dz�TA�/�3J���3�[~��Cu�.��]��g"����Q�u��J����I��U�Y���0���n���&�N(b\��"Y��r�W�������c���K4��2V���S���TB�v��2a9�u��<n��!-Q���Guh�F]��p��2�������L�:��p��q�A�����=&����i�#y~�]I	�tVH��<���'|H6�1Br���O�^>~����J!��4K�!F�����_��0�����W��b	����u4����5
Z����g.#�
���z6��X��0R���`���6���x��!�=tb�1�f�[�?>�8���(��]��D�%��q�W]#*?�7�����fq,����e���(����-�([h��.0���;F�qB��{�N��n8������}�e���;��!�����x����F����:�(�^Sb0�q*>�0T�att����G�$;@U->|��j�n��o�+��<���i���u�Hx�<�1t�zc8&1����^2�������j!j5�?��?G!$�2/���+|�]�`�$E�$�\"��p��[�F���-[�^K��N�'
�X�����\n���=�`�@�L4���ga�k�h�zA�G�6����\O��>U�ZS���_k����`t~��9��5!�7rX��C�@�v 1*�JzaX�*�m�2/(��8��
i0k�n��R����l,��Y�5��P��CY���Q����B��5@���^�/�����*��0
����0�y��J��K�h��(�c��a�\�0��Ii=�<�m/���Gu��Z����4"��|]�4���lcE~O��������Vf�@)2������+B}�q���B-���s��dn������jU:h�%>�����#����;�Mv�t)w���D"z�P����
�/��4VqM�kQC���S�~&�q����C0����V���@�^��:��po���lH��0�iY����I�����&�&9�����0X�|K?��(��T���	���f&X�C��g�M}�&L ���zE��?�l��q>���?���t'5h�s���P/j�A
d5�~z�����7,T
pd,�H�m�0�7XC%��N�e�&���
�R�0/���B��px.�-�����7
\�ax�|uV�K&PO�~��PS��u��������Q��$N���u�%��l�Ht��a��u��M���>�GFd�C�?�u���d����x��4�K�P`�c�cq��~/�e�-_`:7����(�]���I�%�n�H�	�2w�m�~Q���YR�#b�/QEt���*wf}�Y
ty��7J����[����*��P��
�c�e�@����u�`���B�T�-��4A���F5�p?���F!�3��p�<����H�_�fv����[.������X������/�7{�'�M����Jy+5*��Q`�z1�y�e*&�YJ���s�?��G�2�	xt�yd(b
&9���sH�� ������y�������[�::���w�t=�{,���z>����fb�2�&�.k
o\O���b����zHS��@Y�p�1�v\�p��H�t[��:�_��2�?)N�e �����"��C�-�;aF����6�3n
�����c
?zd��@�����r
�1w�B�c�X������kU#@����}�[�t*�@�z^���v�&��E�����)�}�������it��+����Ki���(��>$��2rD�W^�w���:{���3�.8U*X����g��(s�j��0rl���b���!�zD
X����?:d���;v��c��}�/��%I���b
��iS�?������/`�|���T��!\�%�h6q���c���R3?��	���TV�c��F5/�>b0�����j�����&��N9��K�
��m:���W�,h�g��l0#$D��2�_X'�qX��PHq;QG����CI���Xn6Y�YRI/@H.{�\�d��MNS�
�f����Y�w��W���Ygq���c%i������+L��;�b�H+�s�p�)Wx!
�*�@������M,��o9&�x���nx�2��*Q 3��_/�@5�M8>�p�S��E�a��WD�<w��A8�Fs�������1^
��
����������A*y|^<j��������������rj����EP��a���u���m���-u�����(��AG����C
s�M����-t�+���~�w�,���l��G�-un�`�}���{x�&:%��dt
�<�y���CaFO����R6'-U�x	����������Q4.�����e�����;M��A������\?!����O�8������vt��n��qR�Y3y�&��WZ���PjF3��F�O�X��j��8���������`�<w���EN���2��@]���%���T���Go��Y�������O�$���v�]~��> ���#UP��ke��	,-x����x�zh� a��'(��o��^O!�g[���]����;��C�-�~T^���N����e��������Q�Z���v��*LAV	6N��Ob������S�5�f9x��C�'��$�H�x���U�$�8V�q�3�G9@���aO����*��,������99�ke���e��a�7v$dU>��f#)T�j�l��>(���PS�����i|~�O|����������g��"����RV�&6
�iXgn���Ss�3V�=��-��l����>��	�����@]s�	W0�4	~�5�WB��a\�0\�j���cpjQ2 Y�P���2���;�;_��`}"�5jk�! 3#��o�EP���������e����^&��_�^���x�����`���1����2�(�b���x�1��K7�
���U���������*qW�y~�������k��J��e9������_=e�>=��Z{x^�
}��:����7:������Q �o��B���o��U���nR���f{�v������a��}U����q��)����u��#���6��w^�,W��0:lN����nl��qH���1�.�H��E'��� @hB�#��E^��pg�4d��h���BE��7����M�RM���R	p+��_��O���$����L�����B�5@%|p,�B� ��@��ML��Xn]�FL����U}m_�u�v����E/}M�@��������+�$j������o:�-������`i�2�
���u���S���-<X�,}��aD�<��p�c�x#�[�X�����%�����qd�fgz$��D�~��{��i����-r_���uP���qW�.��qa����37���Fy��4u��nP����@\���*���� @����X��KKE��_e�a�rt�? ���gM"�$�y�eqCo������l�y��L9���y�N�w����U��������6�����+I">;������
�����E4[��J���3`��q����
�HX�h�=&{jz�X>|)��	2Z��9��+��WR�����zJ�<���%�H�B�wb��d�g�>�Q��<sl����K�h���3v�����7���=

T&t3IV?��W�m�������!jw�/�����;�����<t^�������X��c
��J[�X})��A����U9l�"���P���z>��O����[�]�%Y�P^�������mZ@�dS���~G`y��U�P��;��R��KT��]����O=�k�K7 d�iy�"�R�^�Glh���[9K�\��N���EH�D�j�2a��@���	9���`��lf�vM:~N�F/t�'��m��WW���;��+z�<�~Ov�]ba���/T�9�Z$������w�Y� YDES����z(�'���	�T�gT��"a��	���������S������q/��qia+���2�-g���mL_�dm��o��Q�}R����Oa.�T�������s�0g�u&��2Hw��V�m%�=Md�I�W�u�E|(P�������w���F��)_��(<�%e���@�@�
�!��PS���W������B��"��r��$��]�(v9
�2��������%��
:Ao�E�<��m0��]#B"�-�6#�����@�s��� Hs�G�}�%o:��k��aW�MP�I�)���^L+��������p�e���}n��+�|F���w����n��\,q���Bxc(ku;L���w��#��]����`a���zLuG4���IU^�R��P���@{5lY�a
�rlM���������4���'���h�,����Y��a<������;$_��`����"���2�Q���ad>Oz9�>���ao0gD����q����K��*o��S=w�ZpW��ZpNo0�pJ��DE}U	Be.E����<
o�gM��&������9W��(@(��zb},��Y����me�0�y_Xu�H#����Y���������"�_�%5���Ey9C��J�9�xAG(��	���"�}R���x��#�p���@H��B�=t�I_?/��F��!j:Q����,��wc�*���g�-?FD1l�9O~D,i���.�Q7�oq����r�������]+C���7	�h���O���w�Q9=k��]�����b��4Ba�!T�������������-�,�
����~����wF�����tS�Q��*�i\7���|U^!��1`����[�����g��=�g���8���~����@< G��K�s��U���?~�t���=���>�0��>���~�?yJH�aqI��v��L�E�b^��g�)3��W^��K>f�*^c%��Y�~��b(g,�����%;��|����?>��b���u��X�+����c�V\������2�cJ]�Jy@����bn���v�+O���%���.�^�8;�7�� o�,���5�
�0��XL��A`��\���p�Hg�T��g��)�|v��a=���@|��I-�3���t�~f���M��bW@��D��3z�^�}zS��0;�G����J�:��I�SRj�u[ v	�f�  ��h�C�!%�p�(����
�|�P��$���q Ln���A�v:ud���9]��W������[|�	���4~n�O������06;1���1nw�n���h�~9��2%<���:Ow��cLt�R{��i#���pn���������x]fD����PC�XJ��No��q�4��x����T�${�_w���dk>-��Ys�|��Y��-*��,K�!�28b��kW���,e�
�<1�����H3S#�6�5��(5��NDy�����u?�
���B�J�]?�������!B�`�����W�8/Q�N@e�=��4����}�tA��x�������]����<.�ro�H���h
:9��I�W1+��2�y���� �
�r��:
���
�o��0
.���|�4{�}rTX����������Xuh�.DW�Jr��=)�����v�b�U��L������2�a��i%a�:��z��?�m��6Dq���s�������{�
���J� Y���(����[2�;���[L1;�a���d�NXrP9]����e����Ar��1�����������5����`� ��O�j���������������*�0P4T��Wub��>�"5e����Rk�6�&�p��r��>��������'����x����I�W���K�
,��#���/Yo������+���0���/riM�d���F-}m	����W�!��M�}�"�X�mdc��F��Z#]t�n"V
|�J:UgJ(��i��U �X��?�3��0��5Y'�=3�[��������M���[B������Yp�-�Z���x������_�+�j��#�~�&"����o���}&�\�c�����E1����B>�����cX�H=���G����cw��E��D��f
����f��xq�b��8�
���R��p���E�,�M��
�(��J���lZ������S������ �ZN��x�I�[��]�����R@P~���U��U�I��������Y��M`�����]5���)S�����e?��Q���Z�CV����m�
��Qe�����^H!�s5��W��	j���]�,���]��\K�*�t.�
����=:����8KV�kyc�4�eR���E7�O���mB+ ����cV��������#nh�|�U��o���x���L���XA�]��9PUc6D�_i���:�E��?%P�f�����Q#!�3^�i	P�,���F�"���|{�`��
�G�ETz`�\s���{����P%�:�����]"���C���%�v>&�)�G�n������^��h������4��.���\����9�Z�w_���A��i��5�$�@��;���x�<���?|Yx�V�)�+f�.���t&�j�c�?;OX��t
�����Y��F��s�(��y7�[�`�
]�Imla��$�.��o�o��{p� 26���L��4DM�������������~���=5	��������F�\
y��$G5�?�����m�d"N����
�K�sQ�a�H]�q9u�Dhs�@����l
6��#Z��,�|�v��;�T���J�=|A.��Q�{R��K��O����ek���hL#��:�nN}���z8������ ;����w����*�pd���������������%x���%���;|����|����?#V�|�����_���0V�������RW��k�L�.�{�P�u	}����lp��
�=�%��t��3�$����8��QD�Y�EgT��������g�Z�������=������\�pD�y	&5�����P��'�_16��~�;�*��2��q3��Hy���.d��>��=�?o);~�����L�K�<�5���xq�
���
3Y���kv����a�p�����f����]��fs\�FY"�1[�����
�c&�������G4�����xG�
#���+F��������Z��,����`����ylto9>�\������UQ�o	��\�������Z�1�V�Kp�����-����B ������l���X��:V�����h��IE���E2(�rS���ma[�Bd��tQ�����J�S�V��q[�TZ2
jk>\r2����
�f5}��PH$�a�5��(��-��3z^6�U�0�� �g�����7���x*{2�lBH���M����ib�������)���B���QX� ���.������XM����\� �	�����F+������L����%T%wa8IA�����%0��y�~�V�
�%d/�W��o�2�[J��za�s�I:���X��l��B�OL8�Y�
)j4XK������f�E4E�K�%����VVp�G�����Y-���h�8��F]���mJ�j�`�?
M�o��cTPS:Mn=�nBf��d��e�o5��W��� �;�X ,�^G(���oq~�r��N_�:��V�&�X/������D�@��krc�Hwlx>�Z`uS����/���[BxQ}��b);��\��?�bc���k����W(4:�
�H��yx�{��4V$�c,�7B��������@>�������06tz�r�~RV��,L����;�����Vm���"
\�g��d������J%�+kE�m����x��������2G>�g�'�,�ud�^d@���>�v��b�,~��9�����*m�p`��dCH?�X?�I;��B��eA��u8�3I`�Z}�|?`I���[��>�{[��\�B�^_��~)����S������7��f�����x�)�n�8[i���a��g����i%�c�]���	�h	"J�{��eK���<_�����F��]���vT��B�F�]�4D�����=�\UO\%R�4[���k�H�x����jz��F�����+����������\�`YCNH�k$�$d�0�~:�^m�1�Il��~�3M��y�e>[��?���VI`r��:Xy�3��(���8�|nj�G��d���`������������p8Aj���x�/��_�o���M�_��������z��-d��c��5p.e�d�/�.-*�������6]�kP����_5���(��.�r#�y����o�������C����e�[P����<����\����]�Sd�}K�8�o[}����=�������������*K7A�'�iB`�����-�^�����v���O����?�JI����pn }d�i�[�����<��N�zW�.�N2���qx���]�;����0���"�b����X<`�d)�#���;lA� ��a4S����(��N9��H=Y����W#���XH��'������;:�d�x�m@]hFQ��E�s��B��j^{f���sDD��g��x^��F�Qw���:�T�.>&_��i�=����t<���7�����R4�?�O�O��10����%o������#8yb�t����S� ���3�����r���<ZTu����-j�U�e���~8~t�=3�l����&p��`�9��m���a����YR�%����iO�?=����K��O<q��][�t�$�..��
����������A��%8�����!i1Rxj�����wxo�{�@fp�l��b�J��s��0�q�V���'���N�xU��������wQ�CA�=l)�j
��qw��Y����n$�~����&Sd��-�����|��~�l�?�:$l���]TX=�kM
��.;e[��s8�W�>�����`����)���G��/�����#���
����O����aK���y8�z��}��R�p����AO	GT/	"��	`�S6��f<@u��*i�,�u�^��-����#���\n��zJ8���_g���K��+����N����i�-�m��{c��N����n2��L�@��W�X�}��3zp��[Q(��S<��$#��Q<4Q*�Ge���R��55v�������Rb�����9��sY����xJg�r�3``WD]�q[��\7
�{���j��h#��zR�J�D�����������TU��-�����Q�(��)!�p�&����&���X<�Y�o]x�*���X_M�����he�eo�R�)re�H�
l"`����d
�dg&LU�Sq8��?&\:EuKu������4��s ��������N�k�pT�����R�0����[�#$�4�l���lxU�,sy�acx�cY���%C�:�x��d�U�"�P��%�4�+6��S,��K_��zj�P�������������i����5�k���*u�������z6�����������Ba���z����{Y�M�1�D��_��V�zM�5
��\�4���X�����OMO������-1P�y��V��������2.�g�m����@��T�P�){�~{�P�7
'��z7Q ����
���O�m�xV��Z�
���<ja����,�-�
��@5y��]!��j�AJ���,[��m�'�;|�PtI��'D1w$�b,��:T�g�$�����������C^�x��}�^����H�RJ��Y{mX�����%#[��h�S��X�3V����`��hs.Y�23��o�n�}z��v�����qk��t��U"�m�]t�6�v�km��Q�B����O��[�����@ [����b�MMLs+�������E�B,���j��� ��_8�ES�����W�����5WS�~<;�LB&l�Vt�.W��2�8���6���!�
^�P4��\����8/���0t�=Q��5�<��{�E����i+.�m�'k�mx"C��Ms�PV����?(�D�v��}����r�b��OtZ*'����7:�,���m�U��*Y,����l���/K�z�|����sXG�R���j|tjavq���s�y v��
f�:GRPU�V�
1|Q �v������X&[ W�xNc��d8"�����)/HV^t���i�]��4�:���b�HV���gty��3�Nldu@������;m��.�h����
�f����,���x�#2&�'j-@*y|�C�FBU�
}��c������$7������0�L�)�Hh(h�o���C����N��q>����p W��2<���@��UM,�J8�-�}���|��������	sl7����?���YG�Lp���P�h�������~�4]q�&������6\0��?[�����X����=H�����)�����O�oY�����I����TP�Yo
(�����v��{:�8@g�e	w#�i0>UD{&����������o�*������G�+�U�Q�=&�k4B���Op$������{���)�B���E�x����=�P6�0�e�=�	v�sV���� ��xT `MG�8��)Z�vKX�p(�\O�s��,�����2�2ev�v��b��7f���z��?�8[q���U�eZ����@�Lq/1�2����A3�r��A]��f�c�������n^�5n�\�5���$k�\�K����ra����h����-C��E�Zn��Ny��/�m`��a��b�F]�����U_^�����������+2�-��AB�	�u��F�$��O����&�e�N{~����VQ�����X�����X��2�nY�6���/��r����������
�$����2*����:��� �������8G(!hX��<Y����u3R5�d0�e���e@\\��uU����Hh4Z`�UL�C%�X�u�=�}�~���O��vEG��2���rI�E����H�^0�kY�,�u��z��Z�K:��w�����j�~���jo���C���!��(���C_<W_�"G�&_�qow��g���K�c8��}*�2im�
CZ����K����y
��)�i�/b����se����/�\%����;2�������&��E�b������\������3�i\����ha�5b
�M"��e��+�S�3�;�����/�T�D&��0�K�i�y��~Y�W�~}WW�0���jL[�����[��#H�V��2�o��M�X���9���P#p�%��VR�C���u�'����;]��j{Z���=��@����:!""Ot�6�&(j��g
�o��K|#O�Y\a�"��wqiN�������� g������;RyV)Mh�\U��t|q���4�����p���z�Sf�y�4X����J`�v�y%0^��>��<�6���X�w�e,&�u,]�m����=�
�*x�U'��
V0�/7(����)-4C�r��-����� pZh'��aB=e6�),��W+�?&q�>�>#Y����2Y����y[,��� �Ex�NJ[�T��u����B+&S#	��|@�H84w��`���q�5��d�DW=��:�v7��@5W��c)ZNA���YE��A��k����Pf<�*��_���}T��4Y�v�=5E�Mx�{��:p)�E-0H��W-�����H����X=i�Lct�h��9�������9p#m��),�i��@��V������,^�k	�7+��	��QKp<
���'r	m�o�s3�j0	T��k]i��k�Af�������E��_��4�z*�>.I|�>�m�@��
��h.d���1���`��iB^4�HF����X��E��E5�@]���e�%������c���d�\��!-u�'q��B2M��������>	�����$��u�\V��F����3�p��	MQ��;s���WH�q�������5��1�v��
���2�`��r_�T�����n�m�J��:�������[�dm}������F,�����Q���x�}62�(00/a	��p��.*WQG�����rl�oEt�v$�!<�1 ��
����Oc�0��U����RuL+ �w�����'���I�F��T[Z�i�E���Ife�/���q(� |���o(;���dZ?/�K;i$����n�������l���=��z=������F���q�Ve��>1Q#c���j�B�"<au����L����:��=�s�Q�$.��zU�3zt"buo^���O�������d�u��W
�s ��=-���s�:�����6��7�a����_D���R���r�_���x�}���3��G-��������J�wX�J�o��y{[��>��DJ9���>*�?^�(u4�0A�T�e���f���g���sy��D���?������������)�~_�����1���'��5�8Z~�����%pF��l�]�s��&����*���P(y��Z-K�7���a(�B�sa��)�](n�Zh��f<6|��Ef7�����'ks�{.�^}R<[��<<~�<&e�V��=&q�c��}�P[�Z	��
��NV����{3m>���m,� ��3����tC�W�m����/m�l"���u���w���.��L��f={n`�C�8{Y��f&��������0W�Tv��X�+{��zL��
����9}L}�,�2��oq���a|^���r.���T!~J~"�f�o�_�>�^���s{����PF�&�?� �T�*��NRc�����5a�����d������2����������V���=)���=�gg��V�8%?�2R��{��p�i��}�8G������p�����>����!�d����������c8���Z����i����i�y��a�uFN�6|���_#��P��/9KKaW���Y�"~`�����x8>BI��FR�M�
A��B���:,��>��C�slF�]��;~|�_�#��#�'���6��Rn_���R�����t�@b���32�N����zQ��R���9����e�v~�(��#�=���l��Z�	>[PN�+��Zj*m����!��M��_YvY�o�RC�b�,b�X���?L{Wh���(�yh���*��2�����i�*aW~P���1f[�r�bA*g����c�+����{X(�E���a�l��@���pA������yw����{������oDbHLF(�����/�^�}�W&�q�V6�^l�xR������@���&W7��O���b���D�M(D�p�#i}��hPO��NH6H����:6��_�,h9�A}ZB�����E����t ��X�<��'��r[���o�	�4Q�0A��m5w��+�h��.�"����������%j�s��������P����;�l����@��>[{�����Rq����Y���<��G�v��Ms����$i�������Oo���Vzr��x$m�zV2����{Ox��x��|[TMx9vO��������%KA��h�h/�~�H����Ay��"Y��k��c�����+=j(�n@ib�<.������������b|���	f��Ew�4 ��w�@T;5��SiD����n�E.
��6����F1l�������n������M�0P�����H��	LU�<~���0����H��Y��I?p�|�t�B�]W`���M;����Uh��	-���v������s���N!%�&��v"�-��2�^K�,���Z�SL�c��C�3�G�r-@��R�0K�4�^����(�b[V�lC
���zG�/JO�s��v)�=a]+?.�������\���#�74g}��s�1���
�{+��oi 7����AuBTw�CR���9�u�����H��L���_D+ -�u�mv�1�>%�����n4�>[�^|�c8W�w��3��v�Pfa�)���8`����KNL����8�m�b��{���H�-��7F3KLV���2��O�S5�.N�9e|����-��D{���D�a`T^������d�V���m-���0�(mE�U` �x� ]�������>l_��#������y���6��]C���f��T/��L���nkb\�#��M�p�g	��cHX����e��@����+��B�;�TF
����r��+L��`�j�/7X~8�}��/�/gk���x'�bZ,���W.�9���b�Xh��& dh�a�lh,��y�����:t���~�@�
�.]IW�� a�d������(3*U��m�?�"�><��^wuE��]�p�u������u�m���4f:8�*�sK"�_3�uL��qU���-*/�Y����u�O@r�V��rwc��,[�t��^��{�3Qd���ZCs���X�g����[!�]tuX�9�$_�������r,q`e��A,��b��)K�#+��w��e�}�"�QAi�(�a[�
.�k�����#RWq��A���~��q_y��H�&�P���5�����4�x��]�P�5�������"����|�	��)�z�����n��g�-�*+���j��X�Mp���r�!@���
0�+��5��p��k}K�	�}z4�O?w����}z�/��Agi�\z{�\��"�i�-��s�A�Zo���r}�	q��b�2D��\V�i9��s=�Vc�+�z��'S�e�.U�G�
��!�;�0���]l�[wF�����+)����\�@��v5���3�����nKH��@��Q��nM]5i�V�8Wu%m�
�.{�����(z�|��Q,�5����)w
X ����������G,?:B�d�������T����q���
�1����C/�$�57w��fd��*�@��=�������OT��x��������u����.�����@��}~��������L�����7400��l���Mn66Ft1��`i��"nL�i���}�Ge+��I�<yW�{�TF��T������
RV`��\o�>�dDo����G����'
�u6�
v_���?/�)	c*�^J<z���;��Q���J3=��E�
i2�����:�������F��.pk�{��|��6ez@�����o�?z�?���J��_�i8��`azc�|Q�p���B�-L'������"�(�A��3^�E:�'��se�C2�jY�*�U/6���8I��T����j���
&��eP�������Hfy\�5^^��p�+�2��V���~a���r�f
�7�qz��YTC�u��o`]T�H�]��.�@H^ul�dObW�m%_������Hy�;x��<�)��l�{�*���(�z^�$l��$R�YR�{_F���B��6w����Q�����������L�.$<�����~����:N����EF�^��Tp��Qx#O`_��J���m|�&�g��+ ���r�I�}{9��pe�,B67P��]�4�y��m�i�����=�&B�8s���$�3+r��m.�/�=�����8�v��	���~j2��G1;��������}�����-��Cv�'����f�M������^^.Am=��\�[����X����,Z``l�i���D�D���-MXi���$%?�����0����g03"��`Gz�m��0A�e�������\u����1��Fo�z������&�E�r��&�,�ng�
� �+mr5��!<[���D���)�W�1rdM�M�����p
�3k�g.���KHjW\�h������X���e���� W;��~�a(���|�8*���Dt�3P6�(}xz����l�jJz��]���{�$���z�sh��vF�/JN�-��nC��.�{{��*���z5�9��>�_��8Kq�I�*#m�@�S��hO�m�����g����!p�����M�2�Y�,b��c�a��/Hm�7����V�(��LA�c-�u��M���u2�,����������/�]�~;����pd��6�0�V3h
�q��>��pz��N��0V���pv�c�1�����!���Y�O�e:hC�me-h�Q��M�Gg���x����������&m"��O�_U�H����z�����t�M���A���x�n�U��n�}*=uzS�	u���SSTB�;�������7�%�S��	�P�n�su�Yj�Kt�,+4�*�`�{�GJc�������a�b�_~0L��b�����!do�wU�q��x��>��i�?(+K�.A�[���h�{A�#[���Re���'xM�S�2l�����h]hcvY�~.	~^���Y�z�v�:7j6��_�r�@}��p
$���*F�������4wj#;�;E�
�Z�k��C��z	��r�a�]��kbP�P��h�����z4���3"4����~�1`�[����ja)���+l�	q��Iu��*����
��
�&����zA�����2�-;Jdj_`�1�k�3!���*PL��V �6�+��t���2-�/�x���"t�_����se������:L��+��s)�������on�t{�&
/�m�X��=7��`3i�1����%��J��Ee<d�Q���L�121�:sj������������:�iQ��qZ�h��5�8�%W�|#�sZ4?����7B����O6O�q1
e�w�JW���6��A�!�!t�
�Q�B�noh��a[���I�Y qe��>�4�V�[�H}LV%��(����pi���}a�@5���Pvmei�S���Q2H����1���([B���8J��j�������T����=d(����=6����Q�G��N0��"��X��v3zn��[0�*<0u{#j	������)NF�YY5��U������\�qg����j]�5��Z�x��
m-`�`�t�k�	Z�$r�t���X�k�W1G��"�c�-w�>���`3�W{�k�a��&J�.�&4`���d�W��pX�5[������tST`@�bp����*�������C��]��\�b�����������R��_���:>�����i�� 24�@+#,�i�M�����A�W�~e��]=�i^�b��Fm��{�s(�n.cW��s�)���lM���qd%�x�p5S�"k�c������X�����7��`��V�����r�6�>�����
����d�%�|��7�=����w���h#�4�?����e��_C[6�?�n�_�<CH)\l���8�����&K��\�/��zg�q���m5��qH�<�KP�@/Qo������z�����F��:?}��$���	#b!W>��7
�T�0�+>V��
����/��\J-i;^k0xC�x��v�H����^�G���@�<��0$m����	\Mh�R�Q��@_�Wo��A�: ���G�P��|/�?xr�4��=�������&��u&V����b����-
�0�=�d���fY���l���B���})����F	
��}����"���&��c�=�
�ZN������bV#�zc���^]��������c�v�������������10_�@q�/��j?�8��o�m_�l�����#&��13BB�F8��z�!hcr�}�Nx|�r�y`����}�;��-}��3������S�d�T�r���c��i:������H4��Yf�s ~�6>(W�A^��t�i�l��A��gO��H�v'��`#���
��Az�����U��2�	Rw�6s���0��q�)�@;(��d��,$��SH"�tU��6;��I��={.���B�@:l�@uQ��2hV�K|��MO��<���=�?~�����r���MDK6����JZ2�t=��i�����(`��]�E�p�fcXW3�1�+��`
�k���`�2��c{^���@�f�bXW7Q7��������]{�r���g|{C����9z>{���m+�X�����w�Td�-�5�P�zr��u3^G�tz���7u =d'`��F�\���cSE��}g��V��:DE�x��/eU�s$����=�� /C����fH���^��S�����2�e�
����f�����ijr��(�?��4Pl������
8H������+�D�Gp� {�x����e�pv`��U
������k����d]s�.O��Wf���U��B�3���1�J�0����]Om����U�������!�S�Gk�(�}���$Q��"���C����H��t��c-�#�v<k���Z��]�Z��h�O�)��]%#����4Vn�.�����9^��F�x�Cg��"e�P!`G>�u����+����|C�7������?+����T�P����Y���
1�.d�J�����I������s�q��LLe=���y�,�g���-^�@��6�2��BA.�|��U�x�T D����K��+��qRu�R�����8=����:�?JYThbR����k�\\��IV*���$�����g���Zn�I������n�8$�>A^v�����f��#�.�u���f./u[���3zz�����(���N_�n�Q�t0�b�i��+�	�����@��w��.K�
=�;77�>f�m��%���2�����e����9E�@V�;w�2����E�,;QPI}��U������`2�a��bK�#I���=fY�/>|�c�Y�V'v����3+2��p{_��I�q,h�Sw���<�����\g�C�:�!��ZS�%��_Cj�:6����l �Ju��>�
H}[j{+?>�����ih���W6��!k2�kY���w������d2�
�+�����|�_L_��Z������VWV<�m0��r�KT�	ky8d�_L�����7��-'QX<��
�A_~f`m���"K�������\HLK`�Y8��}�8l����*�)B���V���?IH��g���V����`����X9?��������=v4�(����pGb����9 ���}�>����I:QX�!���I��;e5,��>��';�������
$0��0y�)�i
��7���+z����6�_�|��q<$JQ��(=B��!25�hu�K����:��F$���Wd@�����Oo~K�����,�6��4����Ci�|vdJ�j�r������w���K���_P��2�YxL��<!��LD�-Y�N������8�����B;x���mo�O������,�PN��eFX��%j��������6�~N���[�������{�������G���8!1�2�V���+TYT5��%��r�}O��0�2�95�b��@|�Cm����8���r�����C��u��#|k���y�u�y��a�(�m�����)~��YP;�2;�7b���d�%w�8��z� �v��qq��J�������E�	0��g�u�[`�6I�P$�nB���,�0T���s��������l'0p��w��+�r�5��ER�|\��mBx�c�
�r,�1�U@b���.��c��1$���M\p�����V��rrZ4���4S:YV����NN�.���)��D��{����$ao���?��s���*|�Y�%y�� C��Dal��0��p�Co��#������c�>�n�p�zE��������6M���{^=@/����*����{c?�2!�����U8M4H�/n��t��p7�Y�e;���"�G@O���c��\u��9$�r����r��<��G@oL�����j@3*yL��{MvN�a��(��_�����I#v�t�^C�?}f��\�hBo<
�pE6�C
;�LP��>c�a��c��+}���#���E����4����)]�L�d<� ��Y�k��J���	����R�y�e�<;2@������|i��3+�x�K$�L9A�;��B�5��84^P��Q/��)u>���r����a4�/�._<B�-=5�D�dv�2�z�*��9�e�#�����&+���-0�6V1F����j`��m���Y���t�U��RS�=����}��}S����t��A���56ar��1�3Y[dP[^n���j9�58��i���
�HC�����&k�~�!g0
����/���M��"U�*��� �
�M��Agn((v� �`���iA�W�E�qa0�����67�K�p����?o�����d��A���v
�Ch���.����%"n��r��� ���[�sT?�� �>)����x�t�^<�A_Zb�� 	/3����Qi��3��=>�����SmE�QL���v�u����U�Z�sWa����'���C7:��-������q����<�_=5�#��6������H�.�����_���"�'��z��r��u(��7��>����L�"g�V��1�C�����-/t�D�Q��.3,y�F���~���x��7@�1ua<����#���
��?�@���o�Z���G�?5��9�v��F���� �3k���8-�C�
�&���r��a��|����oi���7��|�m���[���h����B�'c����x���`!v��B��Zh �
�������vI�}�{�8��	*�!�����rKJ�&�����-��Ev��X]���eAG�6�Vp��V�l\�o,�����y��7��!�k!o6�O�~���v`<���u�,��e8@9fB&��6
j��=�"�
���@��Md����G�����\�-S#0': �'����>�<>�S�=���W���!�-�fX��1����)���7dd��L�)\���a�z��h�2m#�������R�e�I�z�"z$9�(�'K�t
�Ph��18Y���CA$Y�����x�\����1�4/v����?�xJ]<��Y9Y����~����|&�T�����M7�5t��������6�(��)5��T�����M������a��A�00�$q�����x�D.��(R-3i\FAm+��!������c���=`L�bu'h�57��7��p�Z���R�������V}�2��([�9_�������jx�������z��8E���e���r�������������:�]��t��Q��r�Q����������",��"%)�����z�����S9*y��M�%�|�'dI���@��_r:*�GDLj�s��
$'a~�A~��O�J�����s����8^3p�Z�����q&��)�f�Qg��x��EI[��\�U�����QW1�Y-��>Y��2p��&R�?/��P'�����u��!�_�P�����<��o���~���\������E�8��[T/��/�O����<��0���8L!Et���e��{S����>b�>7��q��H��s����]�RNr�@����2��Zv��J_��m����@\�1�'.������v�����o������+s�]p�k��hX�������W���;�y���5��mN��
uan
;Z������Nt���8���N��k���f��2�i���r�p�2�]���:�k�J�R-
���u����fX������hR�3�o_sNZ~�����]!*L��H!�����M�hH'�TK�3zv����$�G�uq8;�$��D
�7i�J>�yx��||����T/�W,�-�����B�M����m<����X���2�e�U�c�SG�r�p|��p�a��kQ��e�ZDu�s�1�� �#g.b��|��S�$�$�I
��3f��%�|p�H���$��(/��U^A��������a������^FI�#/����}�*c�I�+��B�@�y6�q=DJX���2��J�Q~���#������������=?�<���8$����%=;E��Y��1��'ude����\oT�W�	P6�i�]Sf�{���,S��V��V�>=,������ ��.A����1�/��+�K�
C}�u����j���b:�("��	������D��9�BHp��L��0Y�A��
@n�:)��u��
�W��I���2�mu�
t�AzA�i�
�i��zX�"��@��� ��9���8{X�*���i�r>��Y���>d��1F���O����$%�NC���3��������x�F�;���:BP;��p{u�nQF�sZ��8,X���1�v�h�y|z"���$V0�

�`$�E�`��=]`�O�Hs%N�0�5�;$H5�=���9�-����\Q����cp5Z�v����aFKsacH�r�����UMM���N�7.�z�~c���pk�U<�x���@�9���tY�A]��n��"]B���8�f��4�x���=�l�%��!^%������K�g�����p0�����V��2k�u����P:#�	�&K�����i�-���e�c�s��2k_"�D'0�3x������yU�����/�E=�����%�M$:pWZ��j���-$��=���"��5�p��o�3�������h����e�4	.�[����PVpV	C�\JpE������zGg*�G�7��C�c�PvU������@��>�1��
(�m�s��!{V�2wU#����H�"N�u�,���XXc�t:����>jh�s�.����'�b�.��������;*5&�����bC����U�T�;:�f8����c�L�D�hU�L���v���9c�P6-U���LY(l=�������%������cG ��;�[��GH ��`���_��?��Q��Z[S?��F-�w
T�,;�HKt��A0Cg���������}Y�N�����z�j+mXIm�iy7jkV��<^��*}��N��#*b��7�����s���A�b�~Wno@��y�fb��Pb���
?���Lp��`Y��<�2D�vv���t�j�ERmo{X�vGg�J�s��?�x@��.!b{����o����g��{=IP�����,�N��$E5�z���lT[n6�
����Q��������{��V�P�4��j����{_]bi����x�m�>����T(��4!�n�S{h�<;���������y_]si��Z�^Zfe��U<�gk���8���$r��]���������$`!xH���c�z��:����1.�|z�����:�2�v7��j�x�2h�j3�!���E�J�8L��p(�~���������*��"��C�c��bU�_�_����^�?�C$i��M����\_�Y�CS�\��"���`������q�������e��nr����M��8$X�����q�^3b'i3�������k<��7��F����sL�Y��{R����C���$i�d����8
����P&;�
$�46�<�ON���>�]����G�DYy����_s)��Sk-�i7QG��f�%^P�Y�����e�B�����mO��#{4%�~|d"'���&�r
8��68
;�S�/hr(�Q������-�u�y��U�I��Xf����G���B5gk����p��v�D����E��i�l��t>�e�,=�t�������!�C��7({�M��8�$�Sw.�'�CS�N�u{W�v�����B��"X���b����]M#���X��J�~l�5����:�l[�3�w�]���
N )��l*%��i��+^��{v�S��#��*IW���'a�����N�;��q�t,����p`���S�u�W���>���	dzvd��V����������'�O%?\*�p��5T���m�e���)�����6Ec�eU�xo3����.o�T�U�xm�W�E@J��5�EV��
u���w��bo��Ig�e���j�:�}��>B_d_N[��b��OA(�,/h�u��C�\��
0�}J<&T!������SD,�����`oS�A��[`g6��o �(����j[���ON�'���D�����>D
_d���<���Gh����-���a���U�eV/��0j��D�(L���g��ZiFN��3�������o�m�_A�ZTx�v	�:i���^%���w�
�e-n�Qe)�p�+��;(�����R���}�������&S�?����u�1�C�u*a����������B�eR{
�WV�,�����,8��2������I��~y�� ���,#�=jJ�����i\����;��(��m}�s��K�c(?��	���[P������B�~$j��������k~����H������Y6����[��3�r2#�D���me������:}���mQ��
<��#��6���E�BsL�Z��J�',Q�������U#����LV�G=a@��$�sV�K,8�Q��NY���g8����������/�9�qD@���\����=/ti J|�#�j
�������2��.2���N��se��2h�)��cQ��:�)�E�Q��{�w_��I��f�J��b�#��g��z95�37���'��8&�,-��'a/���b�v�����dl~�.�������R1����"�����&`F�MQr	�`��M�R�Fy�"��6�0{X�b�yY&6j(
vE�J'Z�h�������r�,�f���_/����B�if����������2 ��:~W�QJJd{�n�2B��Xa!~�y5��]mD�3\�>�`���`�(UU��"I�M�he>O�S,$�����((��8ZAAU�9]�����]c��
�g����[l<�1�����
.:�X�k����O��My�����FY���������X��gBJW�>�����^7�����	B�����z�w��z�N/�������D|r�A{j����A�����1�^��?]��nr�l�mR�����r��� ^�r"�/����	+c���������	)�I��M����$O��@V���@�s��r#�~{�A��y"a^$�5��_H�w<�F9�����w�q���U���iv]c5/:[�u�G�����6�Fq�Y�{���Qd����[����%������d����MjC�
�VV\<�k��kv�g�S�d=�Qk����Pc��������(=��~���O��BL�/����v������5E��M��-��w���'�:�0&w�.�}�;���~}��l8��
���j��f�?nw��&�����-/�=>E5�Y��"������2Z�^
���uI�[�~_��lL�M�E
t73z��*��O�Va�M4��?��J��YY�s������r���>�?	Lc�"�s���
u�%'�T��XL��;2<��K|�;�
��f���9�
�B��o�������g��n���BpZe&�g�g�Fxl�����~���4eD.��k�<(���@�%��Q���z	eBad<�>�#��(���OI�`��2�|���r��D����y�-���������5k~���u��<nh��I���K[6�s��������+@�~	�k��N��FWPB������M�E��u��}�U��)��
�V�"L���0��'+��+S�F�]F�l�(�Z��Ht��5�[v��=��<}��[�^��xj��c���&��=�yv{ �24�i���:{��q����W��7
~������%�o�4s���	M<G�#�[Y�U\Q9��0U�@ &`D��rw����]7v{����������1k�t����iF5P� ����GE2H����a�Fy���L���ax4]�S�>�ab�%��vi��EB�?��=4GB�{������:I�K��v�;.��\ua��+
ov��8��{�m��2?�*���u�@�s�-�4�������"J��.�-�T�����r�]����xw(������A&rp�z����9�o��5��0=�e��F�h_GsTH�����X��.1���p�^1Wy�LhOD�������b��cm8Ma]�v����fSUa�sZ���i��yPR���O,e�r�4[��f�cTC*wM�����q8�ut�t�i��F���3�
����1J�;�aZv�,������a��3�����X�����l�N���c��]�h�w��m,��/�������?[��	��g+��X&��s���["������C�Bp�<S�#k������6=Qx����	 ��k�LB��6�SG�&A�8"�B9?OF4�k����4�����Mmb����)�m5��D��5'�w&
5��C��m4B����C/��m�X�	�S5Ga����b*�>)3���i���g�Y�_�;��J�W�/�;�!*���g�8�E������t��z����R���x����>*[��
A���%<�..����^�V;P����C������T{�lcc2=��Q���@ @���4�Fu/��
�����O$��a�����X��O�:mtE�=}OQ�k�GrXU00��f�1�B�j�y��6��e��\�<d`��7 ig+�7�����'K�XQk�U����\�����Zh�"K��[@��KW��D�>[E����I���J��h��%����fG���$�W��s��7G�	��������K�_��'��&�6�,�I���=�T�0"ai��g
�d�&dd��<Z�9�Yb���%\V���g�t���h)���|�.�������f(d��k�Bv���B.����d���X���2Wo�����5�������p���>�:5q��b7�u\�|���*�K���P;o��>�7E�g�������������9�.�W�5��B�7�����,����75Y/�&c�����*8��{�e����=�9���=��}��0����!��v��-r��T���|��~T�$lI3�^�����Dg\2�����v
�c���\�;c���6[��?3�44��fR
f���E��B�nJ�{=-�x]����j2�C��d����*�B��`��"PK���D�z�O
b�����ZK��He�9�$�K����i�P��|Y}N��v��Nh��H��W~�
8�T�����B�G�T��s��f�H��9�}�*��z&��?���^M�I���}�|�������������Ru���<|����&����w~g(���w�b��93Y���Cj�������@�\��a�G�Sn�+���1��4Q_��g��q����4d�T�����D��|��P��N9��.Y:���7���G���������-�
X�����'���>z��K#E��:�2I���$g��o �^|j�3zx��/�<0�n���c��d�tx,�Hlu���&���m��dH�>�����������72`�0�,������	
�{��#/K+��(�����1>��4�G	cw����#����(X���xTD�	|���1w{��d�����a���X/���+������?��������V����h�p;�"�	>!���-
az�	���8T���-�U6��J��"�`�]���:E��5��X�.2�m�-�h>�(��8�.�J���O���*Us-�����8L;�t4��.'�L�nYQ:n���*�K��RC5�EE�`5��:�7��V��P�.+���aw1Tu�m�����.A��dE�����_?��������Ne!�^��.^���<>c�8>��H	P�H�_��=�ZY�~�*+��`�kC�������^������$��Io��9�:��4R�;�?��@p�~���p�Sbp?D�#���v,��Sou��+��������:<����{x~��t����P��KD���gg���Br�3l{�n�Y���������]=�H��]���`>E9��~�A��e���qI����.��)�w}���(0�Y���;��.�*��y��<�:$`����������������l���������?��v����+��^o>@zQi�@�
:�����;dI:sua 
mr<$�(5��Z$<��<��b%���5wR�Vzy9�:�98�w1q�i������eE"����s�So�c��-�������1��\�
��\�9�Y�'XCt*��������L�>�>�L�j�����tIZ�!��UV��nF�N���*Z�E��c�Y@I;
��D]87����������mN���u�gJY�[�gI
n���_f ;�8�$A���z2u����S6���?�$m��BR�Z(U��u|@��H��T�@MY��z`���[[P�A
$`NW+���v����lwvy,�o��O�/z�c=��~�6�n2^�g�u�{��u�Ef���~X�H��7�3U`��Z�����d*^
0
�������s����A����7	��2Y����6�.k%�����\���������e5&�)�x�o�r.2X@*�^�5��e������>�z��\��e�����d���XQ����-b.+ ��]�.�����udg�u��*������@B2���Pz�V1Z�����T�:��1^R��a�7����/r���@y[���4L�fCZ�_�n.��X����u!����������y���*���4�!3����n���:�	���85����h�|����\��u���c,t�}�5z�����@���3��@�;��5��Z0�V�+<����<e{*m�_2���,�pO��v����w��\�'��Cs� C�E�g��+���s���]Qo�AsE����}^+�A�)��*?*�[t�*\[���p���t��4��jq�"P0�IZG���o�7�&�	�wX�P��4���{R<-����wl�������\��T��;uiUnV���U�8���=����B;�2�/��}umr����0��?!�;>?M���~<�<��9h8'k�p���w@�^@���?�d����	]�kbnI�s�����g�'�7��ye�����;v� ���L2%	��k�Z_��S����8���Xl���=I����a��6�����|Y���B;3Z��P����j��q�P�y��i�>C��_��w��|"�dp�j�$��T�V~Z����xU����������?��������;��(��$U���HU:�8P,8��
T������lYsB�>����S�!U�����T���%�0o�Iu���`���7Mm4��!yPG��\A��s6���	�
�-!�&c�� =�n�r\n�f�v��1�fu��5��)X�����������jY����n����[,[7>�� -����1��;� _�
���Xh��|�~��[���O�x���|�eiv��n.sbms������D�O3�1`��p�~/C��{Pv��:|�8��0�B���\�	����&�Z��&`a���M�+:��a����(Wv�s����9�=��������.|zQ>��'�6Pj�
��BqW���}n���%�V�jTM��CTH�
!���p���\�nb�17�~��1pXGb�#��6q��I�����K�!c��[}�p�u�7@���0����pO�s4P���B�ts?���Z��;n����	.� 2�-�
s����*;��Zm�k�KPo�R�D�:I45P��eHE����18*�;,3�wUl�����?���Ba�4�y`��"��7�%d/��+P�9_���3M��%���k/N'E����|�1���z�*�����O�M��h9��� C��x7�\Y_@�G5.1���rB���E�;���	�{���`����X�.�'Ywi��0 C���b�*��Q%�|I
�:���Z�N#����������4�M����w��5|��y�^�Xy8����u`��$Ri��\����J������i�m�#�]�y�@�
:��L 5��-E�
�X��w��4��)���Oj�H��[	��]��y���1�I�-��S�p���q�X�y��v'n�n�=����3�B��c�������a�][8�I����c2�B�'������_�N@M����h�]��.��	��pD����)�+��|��~0�O8CQ��jM�]��B�w���c��oKuQ`m�4�r��u�P��?���H��:��I�����t��0����I\"��	X��E����!�R\+S��e1e����p��1�o���	�
���)��+�E}>;�(����r3^�H��B[��������}5��:�IX�P$��c�5e����z�&kLN�.�4�j��y�]�����,u�77�����������rD�d��8�����Gq�
�E������������������^)�4��.!U�I\�@.�0�I
����>o�U������_n�e/p%��e&�f���[�7,�re<'�o]Y�����7�7�|�(x��UZ��Jh8�J�S�(y}v����XQ��\�j���NiyP��~�}�76W(w�H*@np��i2N��]fy\���(k�:@�QP���������y� fN�B|x����$���V����]*��T� ����?�������0XW.����v��;�x�x~t�WpH�����b5���@�0rc*�T9T��[e���0w�
����DH@4�(���Q�SB�9O^���p4��$|����|�B��������l��(����&���_��6��-����'���7`�k�W�������66Z�~�����M2�������
����v�8�����I2='_�-woue|���c������/���X����H��y����������X1�
��WcO���3�'~Y��Xam�C1<�kEb*��9T����ef�L�����	�����0Luh�{�6J�
�E�������~�a�����Q�C�l�e^����qu87.���Q|�����u�r��b�����]�~��aUP�g��p[���xg���|���?�E������?�����x�����[�*�D��rO�V�
�"0� *.01p#��F�, �-) 	��K�o�3���
+w�?��S�_e��G�$��9�~9tE����0�?O�vg���l�)G�	,�t�6��A3x���]H���e����X�8l���������C+�+O�����A���6�	�����s8�k-��vl����'����V������cFy���;����QS��0)�b4Ts�%��p+������=6
���)���"%sd�k��s���[��;0�w��@��6��$��&s_��^zFS�%��8�����<���AS}���[��@�<-j"3��|6�_�*t�%�N��a���������@�����"Y�k�7��=��'^N�5����T����F;�
�@��};�����/���_�m2M��{����x�-�c���z;Es�������P��0�Z��T�@��4�p5v3W�h��-��e����v�����CA�P�
�#kg�EV|x�\<���	
=����cbQ��:�e��{�������|0���P�Y�!7��0�=��Cc"�����W�6�y��u�b�������T�R��q�f���WX;�����5���S��v^P
y<��������R#T��m�i����!g�{\P����X#��Ht%x�����O���
���� �,�L�TA�A��� ���&�l�b���a�R����;ys
�
��
���~����&O��S��q56�� �^��1�����ll���e{T�,~��Hx/5�6���`��Wp��+��1����w��D/�	
4���Z���b9�j��p�U�9������r�~�u��4����uho�kG����
��7��zA�F������#�8L�����I3zj���o��)CP�5Xmu�o�1f��2���5��dUX��-F������r
7i`x�r�����+[FO�)56��yPh-����a����vjuBR�f�6����>���V���������=�����i�eqdu����h�n��fO����d���Rm�m��b���%5!���IS#Ff���1SglH�$�7� �CNJs}D�g��j���f1���������I�x7��s*IqA���MA-���%�o�3`�M�
c��e�,���f5C��VNd���(\?k��.n�y�4�*+�/�ur9��+��*>F�&l�p�N�.�n�"���[�,)�@k�������y�ub �UJ��m�H�~��v�.p�h��1�������O�����@>5�Rx���4��cL�����@�T:�E�2�n�-4P�:d�[���r4�t�f��T�Mb�����A����Mr�_e(�?�[��kRhq�8�cB�uT�p��[��4�����JOhF�NS���6�;eQdk�6y�{�=1���\GF��v�s h��-����9)i��n�u�sYC��"xCsYn�9��;�"��������2�;��u/�>�m}
85,#�Ul�$1�F�ho-+������z�2TW�X\f���6��<�R�#0�k���:V�ML��,c���0�c���"\��T��l������~ ����Ryq�k�;�TQ5��.Xyuh�L����$�7���1'���B��7e��a'[�����?!@@���m�0H����QPU#��z�$8�����e�!���&���N0��)����w�j����C���57_A���>~�
����y���x$2��"f�����z�h����YRo��Lc����� �Y�Q��PY�70�kLN�p(����N7�r�����T/�= J�9��h7�*X�����r0A�������z���)��y��9�g�����P�K*#�#kP�Z�������2~��shW�ls�}10xa�r�����JDZ��N����=5��"y|F9���5��@����������9;?d���;#�����X��1�x�r�e��H?c����k����;C�����?jA,�����Rx��y��4Q��w2�qZ��G�}P5Bf�@T��)�R���R���<,y��}�@xx��vOL�gs����=�*���i0��Z}=&Y\w����f�o�3����*������*o������q������
��]�qO�t����`��������������Ic���~��j�6�
�A��lJ��P�V��hL�����\f���(F6w�^�W[|"��qO�@�����*��?���0�A%9�M �xU�h0��H�Y,C��]����u�����
{���������2�Y��
>*�5g�v�[��yj��cT�/w�>��=G�[8��,V�	��)b�u#�=*����m�r�&�p�I����V�����n���g��p��4YSx)N���@�|�\�Y�
j$��L��yuB�E�����^�h;���`��a�Tl���|���$���!�� ��X<.�H��MSt	W�f�l2T7��2�0��5�d��d4���<:�r?M���D�U����a����m�=B| �$�.85������=�����5O���0\8fU���EN�2:
= �����o��*�y'��	����f��(�����L2hI��-��R��
W�(���H<S�`~��.}��S�CW�Ai	�o8M�}f�8&��Zk(���,�������y����gP��/4_�P�o�8s�������cn1-�k�WY��k��wX�og~�����&4`��@��>�/��*�J�l���O�b�~o`9c�Ju��_'o��Y�g����Z ���C�n�E5�HF;�|���My�P�bB�tj�=B�2�B=C����[���wT�G�T��Qh����	e��
0�ym )~�M��0�d��k�Q������O����/�^�b6��{L�R�
�Bo�^G�=c'��%�����g7�
o���;���d���5��~������Y�d��p��7��(�{�*3
=X!�u8��5����@���Ky�0�����[q0P����?z��nd����44�3M!zxc�8�i6B��h74%����2WH��H�=0&��k��k��Ja�
�m���[[�F�1�	S�w��@�->�OM#zd��<>��U��K!����������C����w�����o{�H�����O��3�@��c+��%��k���+��|Nw��,{��uOf����;w�7����-YO�z���|��M*���;:yt5\8�"Js����H������=���� ���u�sQ�v@����r���=9M����%!��P��q	GL�M�~k�={v;�my�4a�c���z�y�-����=${�@�%���M����	D^W���.vK���v��#6d�l��
o����D�2�#��X�:K���u^���f�o�,���,��G���8 ���NIT���,��a�����5����i��|�����-�f�������.~T[!�),������qw�U�:Z�,�k����t�u�o��)6W�|l���X ���;�
��4�L� ��_��x�h��i�����[mT���q������9����W(��D�G���cg��r�s���*=<�W��������c�K�/Gb&��_�y�T7E���kr����$�dW'4e�������G��F�H�9��r"m4�d�X���d9%���[���9w�;���������YB;zj�c4b����
}k�Xkc���>$�nI_�7��K���z���Rb�1�k&8��P�i�6o<=�K|�^g�s$�p@����`,���n�=d�e���fr��������|,Y ����`����+��?��8i�+�K9�.[�������c��-O�?�3�M��L������{y���8�%b�v��o������V�+�;�v���G;+��O�X�G����(�D�]��n;�.s�k���.l���1�G�,�{{�������S���L5 ��Wm-������R2���m�E�������Y���U�;�>��>��q^���8�F��4�
5b?e�
�m������=�~@��t�m���ue8z�X�6n��#�o������m��^��.W;#���m0�g2�\��o�;X�dS�NC@��'?>T�];; ���6 �����L�iLv�~k@�A�_p�hk�rGnA�������f��WOt��Zx����3���"�@���{�|$�@�+T���:�=��4�C���M#<������t�7���m~��O�g;��B����"2�!40�����,�D$B�_���!���|���M��������Hi���5Vc�
������?z	���(�=��7���yDD$;����G��g?`��c��0f����4�]|����@v(���q��=v���=���������>[~_��B!
{�$��'�n���N0�S�~��{5���������.�C���I��n�i,*��uP�z:����gH
-Y�-I�yE��M#q9f��x�=Q8(�Y��Jk���������	���0����Vd�H��P� .���X�PMC����bV��
��	�M��p��(< �h��7Uf%'^���
���buwL������������m�k(���~5������O�xG
D��=����o�Y���q��r�x���d�~���L���=�����������	q4�*�	�����
i����
�����	�����}v�j�n�l�:���*5KqYe
��B2�F������z7_�DV��U�r����1���Y~b�@�T�G9��C�]I�xg���T�kVbF�+�a&f.��
�X�px���.�H�w�:,�t���#��<�j;���(�����=9���SnU��5������4��ny��"�6�?��C2�6dc54	[�^H���u�6�����Gs8h@�}���.'P�d�������x�D��DJ�?����2p
����m�
��J�-���+���o�#�����L{.��5�P���a	���Zm�y���D�7����|�`n�������~�'�����SH2e�����~%�X�X%mw	)
o��c���x����n��vu��=K��su�@��L�?�!V���8����d
����~Q���cE2�1�OOl��-��\;���Z�^�9V}O$�:��2��: ���~�r�^���P�Q�FL��z�}�X��49 �����S�g@`�����\&�-p�`��4
��Y_�S��.���.�{�BwUn$�����W����w�'��r��v3z|]sX���5o��!*b�g��0�
��"K������~�����D��2�`u�x}���n��sM7�xJ�/Hl"�� ��9e�_%���&���hR	�f^�d�4����y�a����*����U��DVEl�[��|TcG^�rhX)�-�R�|76��Zj7'kDP���u�L:\�h�x5lG��T�f��0���X��#�0�K���b�\�PM�9�39���^����y���_o_���EK,\��ws�F~?'9�H
����b�S�L��^��F���b/���MG��M��k�:��$��K[�eZ{l@��!�������4�%|?��Tu��G�>e�f}�k����N`�����o�cz+[-�r*�ncTK�o�U�o���e�sb�R��n�=�c?������
10P�f*�(����������3z���,���1���'!_,kv�"Xv��
��G+1M��t���i��A]����&�;���k@�Z��4��gLa�6����m�����F�����M�r����8?,�����D����-�r�7�b��Q�<c�g��z�e"��K|�^�e�7AXG�t{�W���1��!�&b��NRj���3�|��Q`�����|�����=�cG~�U�Z>��4��?�H��JJH�
����c@H���2�>k����%����? �h��ty�?�,bc��$��	�������"6�����b��
��f���G���|F�_��8Q�z�U�i�w���a<�8[�X��-�[���.P��2�����Fb��9�m�zNU������	����
�W������*B��?t1�VK���f��fKG��? h��w�,�I��-���}�� ������X�v�/��(����K���^������2Q�~Te�9����!��7�|&����f �C���Q}�H0���88���%r����� Nk��������:B����y(�}a�+��MVu�fpE�6j^TT�a_�*��G� 6���
M����T��0���E'����E/�Qbq�T�)�u���e"u���j��������]�B:~L.�s�a,�����
���_8l�M(H�Z#EHnl���l��	�5o�R�W=���X�����F�bCr�97��Z�oj5�wjc`���f�q,��#hH�l~
cA���Dbfb�hH��O�5�`^�,
:��Y�Z����E�31���c�������h[����N>��x�f�F�b
�$��#��ru�*G�����������	��Z-N���{�B���a��3���b�j��=�o����7I��'_b�-�2@��=~�b~%F���
���rg�
'��G�l���..��>G�`l���]���MQ��H�����fFO��a�i��7�5�o������H�M�F�UG)} ��P��3Ny@�DGNm��Q��Z+?��@������t.���3�0
>�R����EI$�Gj�[oN,6YkQ$Q�����4�!�d�������o&�����l�����N���4"D���9���:���
Q���D�F�3J������������p�4b?���M2\���d��-�$��d�bSB�D��3��q�M����e��v��?f�b�D�Z���x��������'�������'�
-^��<_�mw�����4d��_
9��O����j?���N�b�n�,E�jy���B�E��`ay�Uaj�qa(�����4�u�%����I��PQa(������D"K}��x�t�)
����,�-��{}_y��W�5�kRe��2��`�����R
���_HS��3J�	G��������z�D
&����*��>���=��,��V%�D��V����n����{�x�zd�	f���~)"�E����i1���g��(��a������6t<^x���`����W�
: ������G?�r����gMF^���}�(�? ��@��]A����3����o���94�l�5�������Q�7AdVa]�,�6O�V�2�g;��E[7|���.��F���,�[o��#%n����}��������;�)��� ��������Y
N�_{��Y`,�����&K�R#�z�����h@�*����.�y��S�fd��14,��N���Y��<,s��@>94Rh��D
��� .������L��s[�����	���������N������2�5����cI�	�5���a`&V�����D0�=�&����a�/���b��-��#,
8F�/���U[�=��,_]��w����sq��9U�+:���(D��P�b�����
�k�`W��4�
�;ea4)�/+�SC
L�u����%3���[ys\)c6[�����0���dW�e�(������5{�PQ�P�l@=��4A�|5��'�?��I|��X�*��w��xyP���1��w��RK�a���LA�z}��l���t�h)dN���)V�bKH+��	��x�M��dy�lH+	�A���!��?��$cb�b���#I�?��t��4acu�6d����T&�����b�:5���@�Q�����~����Y8�����X��u�.��BY�
�F��1�U�\��r9+������K=�(.��0��_5�����&4`,�M��%���`�7�_��i��5��~��~	_�F��rj�o~u�����?n��oK_���w@���DiF�O�_A����r���j���E�w������.�'��f����O��p�[pl�����B�U���u=��KBg�"���`{/���fw��7��&�jTPG0���Y&0�� �EJ3������������*�-�D�y�vb&��g�n�PM0V�/�i�e�P��4�	���v8��bvd�
�0�$�����?0^��0�KU8�������i�xA��t{����m�Z�U)��^�����iW��O���L26H�}���}���0�z����V��=��3���������E��T
f|��^+�Q���xq<����=
nM����Bb�Ph�t�0�6\��tk�B�N�����5^�P���<��>e��o}�b)0�gB��^�;u�CJ�����i,�81��0%�"������O#��I�}���������.�����b�?��nq��,9�tV	�O�lU�W&�Vw���K?Z�:-1����2u�ba��t����x���4����p���y�b:�VH��9�A�����+���p��%k�l@Q����R���x�f�(��[�R��*���=���]v|��:���l����|��i��%Og�A��=�Q�6@a��n�[
�y������j���7�3E)L�����x��M��>��"�E����B}����=FqG�t��`��a�����.��A�C����L��b~�i�E�*�E�)4��Od-��
e-a$��I�T��G��/����R�
����ei�M%lqy���n��n����&�b��pr�Z��B���xN����
�����%���q���:���C��������.QIn=��4@P"e���,��r������FY��C�����"�=<���7�o��B&��.���-�}�j��`S����t<fz��w	�u����x�Z�o�Q�;����{��Q0 ���P~/�`���7�������$����
�����M������g�YX��0������C|��j�`���&��>�Sr�T�u�~(�������`@���_����eF����	���}���H;�
���wu�.���M==�;���H�y7	6?K{]�7��9�5
���78������
���QY2[�"Mg;�4��	�B�0j]?�G������7l��W�E��!^�����Nse�g��}��c���5�4���<,�C�J��p�p_���g��|�����cS��_�z���_3B�"�c��k��)�W�P���6�����j�����)B} |����J��h_�&w�p�(w�|�0��9X�P��b!���l���Z����,}��D�����:Mv��S������ri���,U���&.+n_l�2����O��b�m���9R`�2]��@�I�!U�,�j���l�v8��j+i)1{n��g���!���x��#q������{X�6K0AW�[O�H=|���4N�9\�a�_�a ����\gi��R���%���z">���/SG{�iD���v�>�"��B����(C�d3]n����73=��S�pe/�3��L{�/�����4���A�_��Z���?��������G\�C�
6��I���#��z`�!:����g�������-)W��8c5"Q����q��"�4R]M�C�3��_�|�/���m�$i�_��a�S����XU�Ky'�{�����N�*h���K��j�Z����s�b��3�M�1�*2�.~|P����q���rl9�Q�����7������k*|Qh������W��'����y ����mL#�w�e2��O
�3B�~F���"w.C��||������e�����b�Uw/���E�@b��d��C6�S��sq�Fp��I��pI~��	>1�m���a�6�m��!������x>�@������x��^T��N�%�B�X������jN��BZ��@d�~�o*|t�
{�F.��c(u������4����
�R%�x��������`1�f �E���@>��b�`_�=�k�1��?������_���?xI��]W�h��M���q;��xH��|X�&�R�a@����RN�����U���@�q��+ ,Q��@�~7A��w|��g�x� ���a���x��%6qb^JL�n���b�9evukl�U����o�����F��0�|��[�O��b���RM�����E(�m�wA��L��W��(�����^���"������w��0��T�cV������s��&#�m��CF��K84B�r@s�!�U���b�R�����j~N��i��'�������
�#����Y�����e2�H���q��|�y�K��&E:/A(c:SgG�d������'�=5��������4{�
�`Y�4"����E�Q/�t���j���<�$	�V��OO�.�a,�Y����i�� ������i4�t���Po�C
���o~m��x�eiv������t�hO��=�Q'L?���<u����V��*�������3���Su+gXP���KhJ��}@&�-,2_���rm����i�N�}v��h������q	#&��`@��-�^^����z�oH)!e$��h�q<3]���B����`�Z�r����\B�04�
@|�;n�t,Oc{^4�~��9M�_�c������%y��A���vX�����T���l���k����5�����]~�����j~XS<S��+��3��/�:�+��g�I �k�?��6���/�X#�S�@I��p��e�z5��;o@s���j�������l����k._��\�cT3v�������F�y@D���x'�i73���`�
����Z��&������?�����M�#�����o �s�d�v�x���]Z���039�n���m�J��.W�����DTh3������O�3�@!�zL�k��Ce���xu�,T��.�K�L��B��.%P�5������B������<"]6p��~k�I��������2�����������NZ���SN����}��%���Z8�,Y��4K��wA(kbn���^wvSE5��<}��i|_���{������B�k�
|������)�]�B��G;;2����o����0���v���b����^��z����$ROZ�i���#p�jp����O�e���7q���cR~�������uLi@a��U=�?b1�4���'
h�y���l.�u������M�)�=�`�
���3���*`@,t�:�c����������"3t��9gCt��GIu�t��dGH3�7���[1��������}�N(�O��]0 ����T�3zN���b�6���2
a���L���E�"�'��&3�>���i����~���\k����v����j1��IAEs�	���B��`��Z�?�����	�*]l�Nv�x
��@[@.��44[@�{���>]`�\����
�a�i�n����e�,��h�b���e~Vg?�<����@2��C��0������=��q��5�`�0<��lj�Ko����������	X��.���+����~���`��.*�:�e�o�
��/��*/71p��S8 �<�A�i6�c(l����>�^U�����Qw��`3�C��^z���O�������7�NW%�$��h�>��O0�������D���k�9�lD����1�P�2����$� q�fd�U�O���*0i�?z1���l���V�O��IQG��i�pR���W���������#����1B��5��E��&n���CoUh�+���������e�����Dc�!l���|����q�iJ���R���?�W��E�5�sad��?^������/`�-��/���5�qs�������*_���L�P��T���G�tJ{X���TA:�>v����S�3�@��$?>���ut�w@L��E���(��8�1���4�d.�
�z*3`*1>�{(����w��^��G+4h��,ou��m�R�/cD&w��y�=���d����X
�I��<*;|q����K83����n5|�a��'!�C�L�	&��,��o�
���kmi��f�����8���5��~�X�����mn#I�E?{��|9�����n��8���ik$-���0��@��
@C�
i8����d�[d7%������Teee�{���p�S���H-����d���Q�����,@��z�_8�U�����|xP��a�v���*�N�����%U?���5L��/������	h��&�;^�v�\,������<���x���`?�z��������*#�pK��q�d������&�v���<9kQ���?g�m��m3���eT'�����_(=��~Q�/�Q�I�~����F�A�.�V��#�C&p�\�R�h��:n����u����X��u�����?����aG
��;��g�l���!CN;q������gE1�I�~���l������@M��x�9����������
�iW�f���2s�A�������|������b=��i!��J���`�u��e�,`�Q~;���l5+f8�z��HQ
�����C��w�������^�����E���7����[�H��?>��������l�q�D����
��=����<�����K�6�7K���zvVh������af���0<�G4��������������g�p��aNw�	�J#�jv-�yE3^��rC���E�5����I_��A�����^d��
m|+��r�}���g#�-fk����?4�������&�����FY�d�,��N�|*ODG��'W5��6����f!���������v�i��^-7J)u&T�e
j,;�Kex��s��R
c��������&�[�����.����bJz�+��A����J�o	u����9�����x�Y0�gr��wa��N=����7���w��vO����I���_\N?���s�Z��!�/��A�-���"��>���xJ���������y��~[
�V�Og�y�u�������O���+������	��������Kl��v9
Q oB$���������8R�r��������+i�	_�'z����Z�fp5L�
(����6ok9mjx�����`����H�[��G�a��y�e`u����&m�0��3�G2�c�����8��q��6�X�>nlEk����qF���`G�]���0�g":��a6l������U�'�hSb��,2��S:��Hz����m|#�����X��_���������\>4>�.�[s����Fh�����
�~��t_Z�n|�i�5fd�f��g����l:��Ia4#Q�l�<����:}�#�G���.�cNE�O�
�^�����J�v�/�_�N.q����O���>e��^lR�=LF[2��&-��7��C��
��
���Aq F;/�������O��1"���{4<��4�5�He&uw��n���e/�|�����Z��k]�Z�`�������.[M���h$�m���W���M���#�_�^����&�IJ���ap��)�v�8w����K�i�����f}[ ��]e���I`��w�����|�r���>�~^.�f�09/\;f�g�-���M����X|��~�d2������>�]q��u��$����f[��7�������7��$���-4<��������q��p�~9���OK���=v�)>i�u&�<�,�P�x�V��Ze�[p�3h���k_O�R�B�������u��4m���P��@���������w������C.�����p�xe������,[s��|F����x:et��2�.��5�X�<��[2Eo|��l�N��|����V.��s5N���<t�����7L)h������O�������#W���XV�����~�7h1���c�R����/7q%�����g!E�,]�Ho�_��H3�b����g�e��������1�H�eN0����Bci{5n�E[\��E�?��oP���l!(��;���w�z��u)�o���;�0�*��%��~�ra!;�?�c�Q�!��sn��]f�v=�������aT�:�������F��qt�x+��N_yA=��5Q�FA�������h��"+�S�G��i��M�	y���@;�i���R�:��s�+�Ex�r�S��^����?A�� Ny7
e
h�Z��.]^����'����O�$��u�7y177����~�k��R����41���v�_�2TrK;��A��VV&�%>�H����
�C=���y
��~
3������
�7��Wy�X\�6���s����~r
?�-gr���o[s�/��4o{�.�1_�&�*#z��m��d����p��e�%]�{=���Yy$���N����t]fe��7M��0w�r��e���H�����4�n�����rqGquy7����������|s��m�v�4��b��x��4�*6��� 4�y�	,��Z�\;��)���?F�\�d�����O&�������Z�Y�2����"���q=^9jA��>��_��a�D��hg+�H)R�e�sR�o�d�K����bA#���E�\ c�.qB��pz��+������K-����DP8g�}v��zm�nV^��T���.64]\�n��	�6
�
��6��RZ	����<~ys��G�Sm�)En�[c�V���T�"c��)f�a�n*U
��1ZY�8���Q?�a��)�c�_����p��i�������)�R[9�X�����y��V��Z����	L�q���8���{�o�E�i��J�:�����^Hx:I�6�W�e�:��!9h���N@U��c������IO����;����]�����D����B���KjI�K-������V��u�:�A�"����=n��w5�0�"[���A8�~����r��Rk�iN����c���go?o�1b�*�G������0f-/f���r_�W�F�<���L*��6�4�&���BH��v�����e����M#l���pluQw���q�����1��������W��MSpB:"u�^���&�|�8j���A��uH�����g��|��s���l���q4\w8�mL�S���^���V�O��,��iP��}��4�T�D�:��$�4M���pi[�zb�"&����bJ>�[��yKZ>uoE�P�;e�=�M����=������M>�T�'�����h6��������:���<N�y$E�K�gW����i�������nk��������#�z�>�F=��|����#�z�>Z���>6�uV,�8���'n���mC����ai�E���/���M_�;X�~3o�&��.��#�~9���<_�v����I0�]����l���l�b�~h�&���Ql���,��(\:���t�E,/7��E�Yg���v �!��a*g�o�f����%�=�=��X-����GR�,�}��^k+zm���d���>u7��I��XdL�0��9]�m�������i�������Z��.����I�����b5C4%=���^�y���49��1��y�e��y_b�$��� I�"��vr��$�7��k����jB��;.
�4 7
�{@��X��v"�|]'2�a�����Kl�����p��^�o����������4����I���L�K��9z��d�t�c/ ��B�����~�v�F�K�W�]����c��!�����������{K�{�F_u����t��yn�������o7K�v8���m�zN�6&}�p��i O��\S�t�����Q��������+I�=H�T��4�'����5�����M\m-o���o�=����u�'m]�M���3b/���{����#���A�p�v}m�}�b�E�}��-9�?<=Dimw��m�9���LO6L�IK_��� %�y`�p�o�F����n�66�;�A �����#<���b1i�����M>e�cuW��[	i����<v�wl��r
@��5 %�g��K�1:>��x���{���e��y��+�%��4�(�Muvv>��?�'��G�E��~Hn�yC��Zcc
�T@�xu6�<n�+8y�9�����g�����2�6���/��mJ��%��_!����m����
�|@�pm�����}O�R����x����<�����o:����7���0���
���E%r�d�
`���}�!���Sr�t;
&@�PG��Rc������^���lm������C�����]^!�%���t����m���5�k���G��mU�nV��i�S�~o��Uu�t���4�)�����k����rh`D���G�Ea�IO����P!�����4���y2����������LD�m.���������P+�0K�D�Uk/���'����,U�N�-�4(��x-������%��Sm�c�FH�r9~�M�f���x>eC:nfo�:����?�{
�#���wc��F8������i��GCc�����6+���o����{\���V��f�+�B�S�q��:o&���vh�>�@f(q6��h�M���H��]C��U[m�@3�S�M�m��M��f����2�I�o�=9��T���5���i���dh�2\�(�_Jc4,:������*.���[�k��i����O#���m��1�
�I�k�:i������
��M�](|e�G`���>�y�4|�#\����e���������o����~X�VF�F�Q�������JkE��v%�-��0�M�-������F8E��T?�8V�D�?Fiq�[+��`�{j�?��/jr��H���'�pkU�,\D?���X���M�4)>�Kv�y]��3+�w��L��'qD���!�9�V=/W����__�"(����4�J
v�������2N���bc������_��H��i6P}���Z-d`��!C_����XI�w�H����'�<��O��'�eV��23���zkU��r
��}����m�"lN��|�9wgl��6�K����I���.��T	����U�[�?&I0�p����0�\XHX�Q�E^��F��i��G�U�����5+J��I��0��
�sU�}�r����Y������@W0:�d��1Yd73tY-�����H���i��)�.si�K'G�����h'N��y�����j���Fz}/��V��"��cG����ZyWr����s���N�4x�{���%�$�8/�{Q��4��{���q����09�D��E��VK�"����h#��EZ+�1E|������� M��"�vA���7Q��j��qKR�nnLG�o"G{
U�q���f�}/�<UG�k�19�h��C�����D��]u���mn��H���]���FR��*�i����������y���)������
{=t�2�m�4������R��m/�$X���g��eu7�J���\O��&M����3�u;�Q���	�e�[�,g�)��m:{L�&�$�P.:~��f��C��6�4�-��mV�<��O_"�7���"���:z6��f���K7~�BN�r\��~�	��i[@���k_�G>]�'R��z'_r0i����<�n�4���Zx�ZUx�'o�u�(���4��S���O&��e�G��Z=�'f�Ls���i�����~&���4kU��)t���U�l�`��������F�����@5�8����CV��|h����K������3L�1��v�2�
#�k��i�W@�|S���ow�]��
�	jd��n��B���Em�fjy�l955��t����|�n!� j�'j���Z�GQ�|��HD�;���ya(����s�\
e���i�V@�����I�#��m�V$�0��-�8��P�?������,�(L`% ��[y���gD$>����EV�3������ezG
.?!@�4d��,W��l��A����$=C����-Z��Y����;R����
����o~{[�g�,�Q��+�4�+��f1]�O��2�;��4���n�w+�ph�����z��j��F?����e��`*]E��]O�dkW��%�7F6�����X/M���xa��� 1�y4��'#��������p�95�M���l�������P)k��,�hF�:�I��Y~�����u$�&p�\|��dBq>��!DR�H0M�*�[����+h��=�1W�Gm��>��wk�����)�o[>�8��>��#M���\~s���i�R@%O�[k�,�<��>���'�[�-m��O�f�y��Y)]��~��#�a;\X���K���M3��}��@i��G}�H3��T��cG�9��z����u����+98��Qj�a^��aY�H�����^���>�V��SP�;���f�SfM�<�z�!{���gg��c��TXvkU�~�[��>M�E�����/�q�Kr}�����������Og���*�|�lP���q������i���*
�bz9����W�]i�]�cU���-�_%MS'Zl;n��5��Y��q
�����Y�Ao��yM���V�?�5<��O\����q��j�e���kO}Ms��Dg���#}'3�i���d���������Q��L[z��l�Iwr���?m�G?��wx����rck�k���l3�~��z�a�6����h�c����1��-?����v�����L[VS�~�Eqc5��������@I�le8i��~}EW���D\�z��qtf�xI��a��l��-;��Xo`k�y��XK���H,�<�����|���_�1�l6g��b!ej����_>/�.�x$Q���j����Zt���Ay7����.{m��2��3V'�ZgID^�'{R#�(�=<�Uf\��z�"}MZ���;��������?�Y��#�3�N6H-������#z��M�"y�@_�I�H?�}���b�"3*�"�����s�-���'6��.���z�`
'�v���(���elH���3�/�=��z'Q�;5��)q�)���<��k����XT��X���g��U�i^����Q��(�m�Y�)[�S���f_n���������t��^�KG�:�����P7j�_���x�h�/='�b���bG�����TV�3�es���[�jlZE�N~L�#�?�����n	�&m�wg�Zoa|��-iOp�_H�N�Cmm4}9o���������	������O$����GvB��iy�_�N�{�n�/[V��6�W��5���0���B�������_����Z��7>�3����i�Zm�ow���g��8�(�����V��/�r�����"[h���-���u���ko���i�X�4��ZU�.�|m����VU����'����y�g;�M���3��6V��A�N�K����o���C�z��&�\�k�s>��	G�K.���-��U�m����[nn�nw�t���uyPM�G����.v��]�{�E�T���>�{j�6��
���a0��)�I{�h��������q���W
����?�8�����~�@c�>�������1_�q�o�=��#�Uk]����WAN~�G�Hk��m���Shs������`�>�a#���X���;���;��=+�:�u
��P���V�6�f���M�����k�M���u?<���b	v�9��A�kP��*���v3[]�4����A�^��n	M����e����`���p!���8�#����g4��]<*�.LD�5
�
H�o�X ���&{��5B�S(<\����<�c�'G��^td4T�tnK�'����Q�=�o����k4z;�C��I�R�7�|M� /�U�K�������Ls�)
s=�n��l���<G���b���S�p��V+�d����G'��Y��z��<��!S�����9�o���29�h�H�G�?��_�v��<i,yY��7�g���G�.�q6�!��������c).h��������{��������&������v�&1�o<�������Z�|���H����
�l�DP�����*��d���CP�P��������l���������[k&���_���x���#���M�����l5+K��z=]|k��'��U}����?~{�����P��Y���f~��*��T�+K�l���'��qj��Q���WfW�\��_�]�������/����8����[����������*7�%g��6����y��Dl���Dm�A���D�G�s�����-"h�&���_S�j
���"�Yn�si$��4�&��y`������S=�F���@]�j0���4(&���].:���8�%�� %d�=R}�~`�X����o"/�d�T"Dh����_d�����x��6&��=���~kE�|����aGnG��Z�<��K��/��mp�-��[h]�N�����V}`�K�����vb�����9�j���O�VW��k�#�����m���;������9<@��O>�_�6gR��O[����/����v	���}��R���<$�/���xV�:�B���McGj���67� ��p�w�2��oY*��J��K���z��Rna�h����������b�v{���R�h�9(��M95����k��sC���9�n�vS���7�0������M��s����v���7��]�{���c��n�����Y���n�"�fO�Sx�Zf���Z���	������|�VZk�~+R���Lk%>q��@��y�_�V7
������o_�y����i�L�l��g�e��^n��b������]CJ�-�|�H:��������}��o	{�N4�X�F���V���I��	�4�&�KY��������P���+�Q��K5���y5�X�G�|�7<�o�-.�)�Na+0��zH�R���r��b�����S&��5�"������|���ofk��JFG�	��u�{E��L��`���;����hm���+8��f��g�;�c��N�4�&���I���r����1�nV�M]/s��ZkV�dD�TX���w���W���������of~��������U�D�X|;������������������a��Bk��M���/go�d�IS���������y:"�[3[��[2�'���l���:��+�D�OYC��v�����;�sh�j=f|a5	�N��4��Y�U7�P1��gc^.7\x���5��	vfJGoU�c�������i�o�
�A*���tk=�������-t�v���5����Zq=_����
8|"�,�h��[�x�Q�zt��Z��y�B���v�}<0��C�']�o��V��|K�������r�'c��Q�o6���0��=:Sv�7�"��Z*%��o�H���	�k #�v�����m���7"�o����]�����V/0V��iR����H�\nn�&��G[��$3Jb�|�DF�m�(��=t�*Z�����6"�w��;��g)����i�N�v,����zfS��My�l���Slj�������[mv�j��iO�0��F�Zk�_��>�����0�<�a��dk�6��i�����p��g9�g9���|��<`�4
j�;C�F�T�k::p�����h������EVA��A���������.�!&�H[D���r{Z��m9iL�S���w�N�M#��=�!��*;'oi�P�����8����v����Nr���n����c$^��w����s��&
E�4F�����2�p���.��hi��j?,��.�O^�w�d���L��n3i��6+�^����?@iVG>���\;��/~�dr�r��V�H��<�M�=P�����N���������/��l���(�/���B.>H��Z�Ch��V,�u����g{MT��`J��#��YK�a������q�����EV���/1sdE�|�u�5���(�t<��NS�m�����f���]nV�����f�<�[�4�$X/N\�&6���fQ�n���t�c.�SXt��bC��Tx��n��|u�`�M3)b�|���rS�&�����������.�l���.d��lY���	���$1�^���n�������@��������}�:��������r��Ro��opB���n?������.%O�$�;��DkE92��=t�{�wk?�"B�)7J���m���n���(�������Z�7������3xv�X����ch$'��$c0��Z?��r^�e��i����74���W��������m��y!J
��)x��>��\�����'�������P�D��
qNQk���8�p|��d����3"4�"�v�FzP�.NqW�Hk��oFex�e�����d}��u+���%.>����h����21c��tfl�OL���g�M?3_O��T��G��Z	��d�
e��t�#�0g\jN_�Iz}7���:�;���,���k�dE�F�O���lo��rZ�#~���t�0��v77�M@F'IIk�HkuF)b^�E������'�N�������D���?9����������d�

���?�&Z�n�N�m�ZJs,������ZkvG2��f6w��'��7�Z��7�\dj�')�i��S��j��~��������]��n�:B�y�w�\�M�#��G3<���~���{�!

J�h��~P��5��v����7�|�	�����9A���H��$[��]����9�+N��mP���;����v��s:��+9��I��kh��_o�w������E��
��Mww��&���/:�WL���KIG����}���k��u���&_4��W�����Wc�kZW��������5l]����@W0:Igx,��S��.�����i������&�I&Zp3�y�������������|��|'�y
��Gc��m���HEKkH�����X�V�L���B�P�q��!�/0�l�-*s���������t���|IIq�M��<G�SX��K��c%��G��I�Y��El���z�|N%Y�
��������|���7�|�,"���Qd��>��N�O���codr�)�2�iz��z%'�3��GO�*�r�����T{�A�]��->��p�H�~��d���vf���l-\�.9y�c�����g�|Vf����'17��w�g��H�RT��q��f�)���a���i�q�4�l�E��x([i�}����j��Q�8>gz��8��$��J{����L?�V�$���G��r�Zn�)n]�4�l�T>ueo�_�[����fz6���5g�F|�n��������o>�s�?:��o�����,Z��}����99X��O�%��$����no�s0����,���D�%���cY5����4����w����nW�k��S<�Hu��E�u���~��n���E�MO��W�o1��NHN�?�|�/ko�����77�t�=��f��
�09I�����I����%��V��=r'i�^o�+�v[��;4��Yn������������r�X�[��v�N�+������n����/�I��C�|bH�����e��$�vq����@!3�Y5s '��E���u�; ���~��
k���VAv����[(�r��������r����)q2T��~�����Yg����c����-;�����|:[�-?����}6�����~�a5�o�k�mv���qC������+sE��8/O�{wG��V��Hm2E�}�k�P���;�a���^���������s�
O_�kMK���)���������j���������l���Cf��Y�[UVu��(���T��a�D�M���w��5�,.,����E
m��,�\���������������a���:������L��4����l���S���ma��=�U�^g���Q-1��W�"��E�`#��ms�������y&-M�B��k$s�1��$�~���[xW[�a�Qf�E���d���~����/Y�M�e{K�n
���-��q�d��4T���H�7R�5�;���^��K[	��B[���/����c	�o���8���R��K����|Wd�$]����1�V��{O�]����s6�7&�<��k��j&i�������l�����qn�G{5��M@ON���q���6|�y�-�5�'����^�����	q�%�J_?�>���4A��F���~'��
4����n�G�\�h�"%Fo�v�6"�P���g��jy]���d��.�������L���j~���~�����h�/8��hS��oF�����"����Z��CA<�#����"�>��p��i�!q?��l(�!���Y�aw5��Iz��U=Q�ZR���yG.&���0;4�������]�z��������^W�U6��q���$i���\���:[���I������3����#_~/��}��rg����\��-��}�/�.9�R29��x$\���0��Q,����i���LiO��������}�+�����"[��i'����U1i�T���mGY{?I��J ���Zg�l�F��������=���&����� �h�Hd?�Z��x6�����z�A���QL)��w��K����P�g2L��@2��D^^
.�<"��;��GP�|�`�j5e�(�?�=���;�S����BT&���gb.�(�/����Z&��-�����u��?�>E6[��j��	��|i����}Y�v+{��f���F��M�O����Qw�#�OH!hj��N�t*�~LP�0��Yi>���
Q��F�{Y��n'r�������~2�1��p�Z�m���w7����I�t�pB.��cb���:���y���8�?�UM�B�����!@����Cp��������'M�0FdL��yG>L�oM�No����)����t'�{�YeL�^Kwj��z�*�����is��������y7
���P{�*�cR�qM�$�c"���*j���7���1�>�j�Y��uPzLo���1z�8�W4������ma �9p�{���4�w���s�]�c�c��;Zdsc
.���w�g�����#o�N����������
���1#��6�Gs2h��&��5b�P�n���=��xZ"�'E�;������
�6����g~=�6�;�����Z�1vw���=����pI��:qA�b������z��?����������k/���18t5��0����a�V��A��RV�V����������F-�:����:|�_�����[]�����k��s.�_���em;�5�=p%�������P��_�A���o��?]���f6�Q?l9�)~jVfA�u���4�Q[�fo5���R"2�Crw)��n���b��2[�$\_����nu��Y:	=\_�<�E����`Gb����;�����(�ju�i��t��Y�"��n����&��-H
GZCba���f�)�O�
�<����&aC�{}����h������{t�I���=������f�R�$-������������������r�ZV�����m���?7_����f��$\K+�h$dV��40q���9��r�I�C~A����w�����"+���:3�~���K��P����a�����:����yVv������y�WF��L��7(*��b��t������?bY�k��Y�K�
���{������Yi��-�OYa~�%����g��������A��;���Y������s�_]��[�" �)c�����Zvf��!�aZ�}�u���3����Wk��O?p�C��V��Q���`�������W8����gcR~�ev{��OF
W��7[e�L����������"fOEf���������{�wD=��m�55���a��?H�K����l���lV.���/�%Lg��|�������u����/��?p�K9�mV��vp}uG�M�Ga-��|�
6�9�sY��&����,k>[,q�@�O���w��SpB�o��nAF�h���:Y5����,��e?#����{���
������lum����b��4���j$�ggv�������~�d�^�A���,q�u
�����������E���� ����
���.;�|�/�����p������Y�����m��}m7n%��m����H��E0?f�������_���,p�Qm����W�e��*�������e��	�c)���d6�C����~�|'3���}���{����t��[g34�1�������jS�<�N�������+z�}[�~A�[�[�X{���0�����:���?����`�
����\��"*Aor]*cT4����F���?��8�R���aa�C(����}���e�8�|�r�='�Q=�����v�����@���%��{������:���-�Am�|W���E(
�?��rD|P�1���S�`�N�z5�|���w�r���T�}��Y����RV'Dx���'�
���KEoq������y�<m�=���
na�Y�<���}�2�A�+d6�6��Yv��|Y���%uy��o���_`m�T��Q����2+�e�/��r��uD����[�������m���{{��-W6�����N��'�����^��7cF��K��:��)XY�V��Jm��[�����2����������nlC[a�eh<r�(M���������h��5��U
�����(�b���U[�\��`�������Vy7+Q8@��`
���z{�ls4@&`���v�9��3#A��_%=4U������m�����X������9+K���.�����������
��.rWf7�[5 )k]�/7��!~�A�$N
+=�{A�*g!����6' ��}�XX���T�����0m�W���^k����z����)�Q��7���JGC����f�>[|�m���^���$����(��*v�J��I�f��� 
V!��Xb������3'�/z�#��P���%|W�J�w���,���K��������
��o,��f?���<c/2��B��
�8���K"��WH~�a~����.�W�^;2����0���0�`CR@��Zwv��T��A�}���y��4Jv0��n��[�����r��!N�?���A������DB�E����i�8:�pw�������p�{��|�����a�D6��]��o�``^�X��?�"}��TR8��	�K��������mu���5(=nSYN(�;��z�1,r��*�,� �:�����Q��Y���f�� �����B��4�/73<B"M����%��Y�\��c6����0P2���_<�i��|">/�"i�� =$��>|�Vs�[�Vo���p]�:iuo�TA�
�5��2=����:cHz�$����Y�FelQ����E1�?��_��R�}�]n�C���Z8�����]Q�OI"��o����szuc���Ktf:����Q��Kmk3v��1����q~��t<��s��e:��>g�k������4�b��3����;��:�
h<���`�x5��	w-�m�3s<�f�Z��Q��%�ro���q�q���Li�*�5�$W��=�TYW��Dd(aW}4�����kC��Ugg�FD��;g[]!^�{e��v��=@�R<�`������Z�V�L<lP�?�C2�`��&�� �|�v�wl�z�m�P!JL$�����w�
��z�����vF��l�>cya��1�LQ��f�(@;�Ug�5�[:rh�f��~(h�^ZG����}v���:?[.��[`���)�v�=�y�p
(�,D\��������l{�2v�n����/���ZD[d�����D|��Y {�
�_g����k������������������9^�QW{o���'\7	O�R�/mP�s\c$�epltjd�Q
�a���������g����-y���<C��4�f�%!
���&�]P������My�U��{
E3@K��w�jIU��MJ,#������������}�h�5�<�d��`�L| sb�;B�
&�`�2� ��y�V?�zw���u�A���o�O��c��%
7.�V����Bh�%'���k�r�Z�1o�Ube>�@@CB�X��kf��"��y���.7�����T�	f�����������f�r��B�$L�
��U�50���r��&s������A#�@��K��I��3���H�	�����n�|��������{�nux�z�^�6G`����AN
��-�
�H�T
D6�
���9��]���-���h���t[�6�B��:���h�Y�#�����=,��#���LDy]b|����P_,��8P+�	�E@(g��I
�'�H��.>N�k O�nS�Fk��E��x )���D��2}�!���shn&KM
l�:�����9��J�Y�()���T���Hx^�I`)���{����_��s�~����$����8��CL���L��[���a{	���9�dO�3�i
����%����eZ����?
a�&I�E(v���x���-R#a��W%\Z��g�r����U�D���l+�L�|��ro]��su3����^���x>+��������f�t�$�Fb��Kq��r����������&�:�����	�N�f��� �Z��J1���e�;:�<gh�j�0��6�t��� �_|��M��G��%}�W���O5�h��|?�&�E��h�yE�3�e��x�m��"L��FT���q��w��
GKB�<'G����o�SR��Y����3�q�J]M%s�%�������m��,�y�7pVs��1#��+�Mk�3:�*�$�z���F�`HcF/E�����B�P���5��=��zm��X/v�������W���+�������T������.p��F@��K����/H����7&�_f�B�]]��i5�����+��&���/��C�7�n������YfS��j�%t/���8�^�t,�+�� XP������*\{ ���.7�A�(A>�>NzQ��W��U�c�eS�WH����p�5��X�����>4���7p���]E���\:������S�L��R��n����f�V#����FG�ay6]�c�$�|G�.���j�2T���#����x�O~�n��5�f���lr/����o!*�\�C��.�_�Ee4.���l�tD<�����,�p|-`�7z3�d6��;��9���h�F/A�Y�6\V�~x
�<�u�����M��~8�uf�[q���/S��o�k��\�~������k�e^�j�e��q.�>Q_&��vE�������Y��$��&\n���J�/�J��B�G�y�����F\/�@���~���p�N��`	q&�u�-�����4��� �B���*���5����c�P<v�����U�}�Q���.QjO����p-#8b�,:/x������/@�%s����tK�=#�)0��+��m/�{�yk�&���cQ[�
v�B�B��;��,f��Fa��$4�4��.��RB;H�vt�Hx�iK!0i���2�%��R{$Jy5�u'�S�kB���{�;#��AaF
�Syj�l���L����_e��1?�'[������������ �rn������l��6H���(�M��J�yXHni4���)1��I
�j+;��m�Sk�C��(��Fw�
L�{<�W�A����/����q��-o.N�LC��{�����#B���Q�D�nbI��7�z��b[ zx'��'���K��!K_W'SC5�Y���`�lq��h.�U8�S~}��%���~L8���[n6��`q�2X<m�7��Wq�eI�|�Q)��T=\���'�-a�����/K4����������0Q�	6������W�6�^�:=�#�2�-8e�h��*o��x�yia	�/��=���	�1�KFu�,ZF�4L��B���YA.�Y��Y-��g�=�#���8��d�e9���6��~�����V�#���#���L,��ju��[n��qT���c�?�+�@�*WFCZ�n@���:[w���g�^��Y+�Y0C]^o��!��,�o��EL^}��]�l��t	�jiKLuB�+�*���B{�=�g���6��;\�����[A#+b��<D���t�@��^
���}��1�
;1X{v��p��=�+��0�y�P)�ws�e[�P��M�J)���bM5�����
����#Iq�����x����%
�����$�'Ba|��]��f�tU�
���1K��Gj��x%Sa��95�D�	��������]u�P�\|6��t������Ws����d�s�0xxE���u�	�hV8��&GH��7<<�mLMT�bg���������m�����L�b*u����l��H��:�e&
bsL�VS$J������f.x�b =J�s���?��I��J���D���~�iuW��d������R�5O|��+"=�~�"VV�T��8�����[Z����]h���Q���:n����"!D�7J"D-�7�Q��Nv�����^+nxcN�B4�?��{v}]d�Ji���G���x���)���f��\���.rY�����l���f"`���tY
;0�hL?$���+�&��^��	B�*������v�Bs��l���*Q���<_E�3��P����(�TM%Y�g�'��C��^{���e���w����T`-#�x2����

��%�������JN=�M,C���-�@f'�<[~Rg�82�3�PBN1JG������\d���Q��4���nHFz���ne����y�����Ae����Z�d"�[0�����_A�@��?�P����$��#(+��TA�R�����e��~j������LSwA$��OpT��Wb��gW/2���;�w���\���tX����KRG�Mcp2t��>�Xst�#�j���[g?XQ�t���.���v��O���:����K���%�m�s��D�(����^���;Q�����D�7lq��#��I�G�&"������./�lC�f�����(_F2/��Iu�R���i}�
�`e��	���F������M���Y� ���2��� {��"�������(n��|D�jB1D$������Ak�,��Hz��^�"C�l8��_Y^3H"�����|)�=�H��g�*���ye.���rwS�R�x��$PQ��������oq��l��p���g��pb�Y�.��KA��qE/��=�Z���p�,�.oN,-|����Fm|��T������1C�5���h:y�dl[1���}�}���}�}��{���s������x-��N�=�����G��UoBB�����H�3�h��*�~2T�l����qO1�g[�u�"&�3���
���+�"���3/�_fW������;�[0`�$u��9\/����EB���V�1Gt~��T��BCw�}�1
b�8�"�9��R���>S�
���X( ���XP�v-���M�p���v���$�y�������A@��j2��<l���o�H?/a�HQ����6I^�'l:l(����H7���V��>&�N���B}������0�]�U!v�&}6�3!f.h�,����y��F�)v[fHB$�
`2�,�V�Y)@$)�!�f���]szo��zn�g�����������
x<���jv�	|�!��JS���vyq�����%x��a�tc��i����lr��,��`����{Yy!���&���fB�
���PE����'f�r������K��	��q�:�'/�����O�^�%d������6�P���<�}����g�V����R�H��	��d>&�������\�GF �n�V�h�F��_�!�Z�<+>���j0�����f�JB<p���W��9�������y
Z���g,h���-��������~3[���YQ9-]0�7RZ���]kBc��X�i��NI
�4��c�x*q��Qy�w�e��(v����G��u�����=d8���{���az8���?�����|(����������gR�����<,���VJ��������%�fPG���p��A������X�O� ����z�F��V3���{`}�x&����)+�Q���>�
�"���x�>�+�U�'�Hr�}=������XRR�`��������8h�(�x������-��!�i@��7Ma���r�t��J�H����k�(&
uf�,�2�!�|�f�F�V2���Xy��[�,�:��r:�sbZ,��>_A�%����x���a�5���<���,\�x���
�]u���G�M<\)��K��N,��Z���� ����;u!C�$���P�i���gx��8�n~�-e������e�e�����h�x}�����U���p�\�#{s����Gg$��R�a�	&xl��Rs��Z���U�l��!vkj��d���7�����_S���0�Cc�EG�=�(�����+1������6���8��������}�F``�A�^?$@�<���?1Az���6�����L���"z[��svVe��p����	C
�����Q�i��O�3�`@�7�8��(�"�Y4��ZN?�VX
�Z��k���
E&6����-��}/7�h�v����u�4��*���2�X�2ct1�|�d�W�!�Y�<�G�A@��y��6.�O��������eQ�d��N"�W�����gu��{��x�p�fY�h�h*I�:��D2��F�h�e����gj8��|�\G~�d��j�d$����+"�)I������<�z�IT[��=bo�1���c������`:�|Z��5�����z%'����n���By%h��-z��h�AsH3��L��;��;YiA��~�cd�(Y��N�G�
<u�����o�T����5�v��ju�Tx��4[��x��B��%��{�aK����"��?�Ew��8�?��sc�2L�����@���i!1��hG�E��y�AR����{0���L&1�{�7��v'��tj|�����+p�X5P���j�j�
�#�r�~�H�rK��F}=�R�I����������B{��n��e�������4��k���>�$�{���
���5�g������}A��x��VA\�rn�������#�	c}X�'A��"��?yT�GscGi�o�Y��`���I#�MeK�OT�'Ip�o��ss��NY2IhW����w���(����d.�I�K����)��]�>t�y��t���x�T�5����Q����!����}�L���i��O�vjm�i��V'Q�uT��i&
&�i��@'#p?�P��+oY&:��`5�N�v5^ ��:Fu8�[l������� ��8T
1����h���p�'����I*�	���>��c;}�gE7<��[!e��������JF/�X�r���6�#���bM))V������8�������V�y��#�]P���1
���t2N���S-��g��X�"����[LmCf�TP�q�Sq��ue������k�"Q������vl�b��l�4�P�^�`D��0_hX��)��O�}&���
�q���k�D������C�R8?�~�7*�o�v�BJz���zp����IR�L��C���7����0
�x��3�'h=���qwX*d�R�*�i��$��&���^FZ,�����w����b���QRr�Q�-��YC��z�]�9#��?W|V�����(u�)a��Y�,������4�����aB{���"����>��l�
�c�II�Lb��E���z������/�b��� �U1�i�Vqo�4�K�z5we�I�n����-����3dvNH7l�,�",���o�X)v��R����29#�q��#�cv��'0���'<N�p�;[q$7%XR���m�2d���q�Jj6z!���o4�?cs��=���� g�>�R��Y~��T�$����"+�-+h5��}B����w���=��B�(�x���Bh �H:T�":}9�V�1H��6��(�i�0U�(r�H�vk�&
��C��w.�#��7G�=p�RG�pF�i�=@�V:"�s5��aw��@�i�J@��<����y���.ll���2\Ul���(������+��8g>a��rwM��%9������;�?���L�\M��WV���Znb:�L����F�!������U��8D�r��� �o����=g��RX0x�M~�U���������Xi7��%q���Q��[+�]�\
��y��:�����n +���-~���������&5�A.�K���{}H�>���2����1�R��V��)���8���]�����O��������K=P��:W:fye(���������*a{�<_W8��M�B��
r��'(��&���</+���LdS�0�(�����v5�Dj�A�^�Q_��D����ZN�*�#�h�����d�g�e�'YC�*����(Z�oE����>����z5���`L���X1���{/xq\V�*���%C�O��z������4����;�m��o�����	���"�ET(���$�����f��������(�;��G	�%]W�r�`po��d���������}�Dp.F��
���g*���^`����+	�������%���u��6 ��vw���n����@m��������j
�H>�bz�AX��1�S9|�N�4�=��aU���"Ib)y�#A���R��b��:x/{�I�2��b��"� zds|�Y��HP'�:�Qj)4]��Fut7aN��(P�����dL�H��*7w[t���/���2[R
�ri�=[��>�r`Aows8��Z���o��d�h�!�~��� [�VZ�������[#���c�T�Z�+��S�B�$����,Q���H�JP�f��c�9�N�W%��@��w��d�4��D��=)(��N�B�����r��C@LK���Q%�p�}�	��R�'���4�������S�b����q/��[?�$]� )	i����$={aH_6�v1��3'�%�3ku�}�%���$~�[��(��V<Y1)7���@��&,�zv�*m>C;V� +x+���]�O�m�{��^�#���j>d<N;9I�X�vxllA/v�b��j��*����|[hY���A\�D�N�B|n>�r��5����iW�*�+hz�6�JK74Y>���A�H��V�g�/y�����K�
�����Z��?K��y�)t��f���������'h�����F�����Q�!���$1�_	Zm��{�w,t�[J�[�gN����*dC�`cx�=e�
!�����gQ�>�����i���)d�<�S(�Z�~�X@��w�0p�\p��0��]$���N5���
������3��C�b�'@Q��,VN.�c��Z�H�&�K�����!T�$�V�L���	7Y������BJ�I
����p��@��7���}nd^�Y��M8�@��9�!�Sk������@�7j
����wcx`h���\�8���>�����kP���4+
;pu:w-:X�/��{��������}���'���j��C��B;�C��a�~�UV1w�������y�/��kl�d=��y�4�I&��uPV"P%4J.o���3���������'���v��J�||Tw�$����eiXva�96���S^�i�u�Wj��03���1�7��)�����jv�Y��D��Q�7���Gg���@&�#)�^�:��
�@08nzx$S��L�TPQfO&�j�a/H[�8�Z�e�����\�C��?�b�����w��3�a}������)�z�
����J#�]�h���Q%���_�o��RF����y�L�LoC���Y����[5*���H%���d�6��-D��s���m,u)� C��^�_B����@���#3qMdJ�w���������VH6���W�Y�A�����aH�K���!�����+��H@����4���?���0!�������	���b��'^�����7�w��K:��_��q4�O��������/��#�K-�q�(/���"���� ��xi�7p��KPLX�X������j�ug������e�>�C�[8P)��_f�p?�`�e.��*T8L��Y�!�@QQ��h�@��+-l
V_����6��:��.!�0 u%����$w��h�����F����:�Xw���L4�7�9
&�G���W��M���'�vGb������Z�'#s5�s��Ts�f�E�����)�z�z|�:��V���a���8v��0��8�C��*���n�)ewug���;���g���5�a�y���dp�3G���3]���g�h�E��;-uv�J��.���E��<��~�$4�)ivC��}�����i��UT��lS�!X6M.�g�p��y�MuP���%����+�f������zAa]n�m��$X�������tW�l��+��b�����F:xZ1���Z�r�B8�?u�fwl�j��0#���$2~��C�0#5MzQ�����:�	8�.(�{i�f�"\�F�J���8#V&����<�^�g����d�@�b��u�?y�CO��\��]�db�����C��X��0#_f��f\_�����")`�����,1�]���d����zQo������G�qg�����5cn���.a���!��Z��U������(��22�Yvt�����SNS|�/NT_��^s���~�������K�8tNh��7:��� h�����d�a�s&���:�M��m�?�����z��w���B�K�U2"&&���g�*����;�AC�NeY�S��~@�
]%��$������J�|���������'|\�R]rI����
g��X�^��q���_x(H����o<h�]���Wv���-������1*]Dn���PW)�&�,E����t��)�����p�)��LP�e����Reh&l?k?jz�����jn�+��b�"��1��Z�Ev�B�x��"[��s���~���0��8�Y2�������I�������(D�����Qk�:aWv��Vq��>��8r���L ��A����������'�y}���ns���l��>�� �j�b�Ph��<Q�l>I�L�x7X���(de��k�5����&�C�P�`L�U�:b�%�`��C�a�:K
J7��K�_��ET����iL�-o�^���(#��W�����/��������G���e���?O3[����hf�P]����?���C~�-��l����h��4�:H2�c&��|kShE�^����
��A�W�[(�WK8���l��QhM5*H^w�z\[6��5G��X��W��	(c���j��A%lV P�pari�W��
_"�W�����$�+V�4	,���e�T�l�w%��[���t�;��{
!�8����VGtN{�(��@�h'��>��fdP��Y���l�n�a{�4��������l���z�[�?���[B�g�rB�ETF^h/�W�!Q�9��y$����iK�V�K��yM���r�u\Co4b��3N�����
��52]�]�P���Z�E$��V���v���f�R�}P4�Ff5QW�4���=+V���������?c��(������d���b�����CzV��O? �v6����\�����f+B��|��������bO�Oo
����"�K4�dW�i���Da�_�@s�x�B
��1~e������,�w?`���D��@���#��G��RS�`
�uV��c���
"P4
J�""�����:Qa�����*�
�
���]i������Q����a$Hg��{�dw�3�*=�]B`��i<#n�(2V���Hdx�%��-�T%%�{����K�a�����:#�����$��S��(�-�HF�mg]K(#"��^����kDX���ck��O?�^/��E�N�����jS>�4�#U���T��~h���/������3���u?�i(h�B��%I����z����ZN���(��{�����&r��z�����}5X�x���U�#��A��c�w����t�L��l�TYC��v_��y��2���9��?��DP�wJ�g2L1�zV�D�a7d6�K�V�	 ��T? �n ���{���d��4w���5�YI�D��������Bc��0��f0��>�e���
�����u�S�
U�����len�W�`�� +`]�L[���xC6�������V���~����[�	���2�N��$���gAM'�������y5�D(�f���9@�*�9<��25���6"QZ���(kJwM�
��Hktcf�	�
�"�T�������X�u���1��������`vG���G�ci^��k�����H�^`p�~��v_�
{��)������"��(U~����.Cbd�V�T	�X�(P�ta:	�j`�L�<^q��%������ �
�2���,����J,�[�f�!8�o�|Z�F��'��:�����8>�����=���G����d�����&��]L���nS�)�&bR)�~��-`��?��a0���F8�Qp�����F�xV�c
�9�A����}�S����DO��mE���-hyU�4�<���M&Q�s�� ��u����q?�������������F7�oo4E��k���,'
g���
b�4$Y�����+QP�Q��*��K�IF���%��|;W��TQ_��N�K�YJ�S,����QHo���q0K�`?�����;��%�b�C���s
=4��c��N�k�"�9�C#�����}=3���U�+��z
G���<"%�]f[��W�2
b#��C����$7����SJ��3�X#�J�_�j���$B����<�I=�<�������R]�,����0�q9,�:/����[�tT��6/���1�M��~V���V���!R��v2��K8~LXNQw1!�<
�9�Y8��}3-���))j-r4>�9�:daw��i�,�#�Z*M�F0�-pLU��������`x�O��t1U<JA��������a�J�6
I����&!�{(zZ=�db���#����e�o����f��MjG� �}b����U0��oXj`���#��c�K����9�w9��������A���8�U�������l\��\��
�D)�JG���c�s^\���"�l_���f��^��P�@u�3�����le�{���1�;�a/\����x�-����K��N
��]+_5l=^�����3��0�TA
'C�z���!�tb��������a�)�����T4`_�t5Dp�I�5���A3�D__����R����u����c�s��+6��f-:�$EEC�(E�\�i)g����pWH�x�Z+�j%��7�O��������(�5���h��XUXe�'��^��WP�`�{�h�acY������L���������k�fH�`�z���������[��nN��]{$C;zD���^l
���[|p�����.������F�fX��m[R?}�����-`�^��F�R����W�����%�����7���n�k����!���|��u� �Jh8_�PM�����T:b�����FJb�j�|:��Z.�>����Vl/����Z���������8����VN���M�v��a6�+<���i��� '���xT����G��[����/���c����qV��s�
���k��a|D����0� l�%v�(w)-���!b��"�F���e�
��o"~�E�f��RS�xG8�E����}�����H=����&@/�h]��iGn��3���X�G����kH�K���2��GP���u��rhSA��7���B^�M��l�j�v������<�%i�~�=�,E�]z&1�U\1)���dDX^�&����{���E&Z�\"��h�Y���:I,��-�5;�������d��a
��Z�1�V��:/�=�������+v�1����{����o����;Lm&�Y��3j���S��~��x�u:O��C��w��&v��,�\/�eo��N(�����.��z6���%�����*-�G;C%��v]������Z���m�Y�m�M���E<z������U����X��	�w_0����g��Lr���� �>/�M(����z��T�y�5��3�y�R�igVr>�d�����vs3 ��_dO�����c�d�\��S�>k��d�pm��6_��0��`H�I}�^�B�n3��yp]���ZD����9��P�f�r���j�;���6����w�t���ff*�`�����h����C����,
�{�2�LgGZA��ZY~�z��v��ef4��1��D�g�F�C����X��y[�������u�m�� =pJ�b�"C�>����n���� ��b�F>!�����|���s�dN������=�������mV�D�-hi�[��.��p[�����'���x�4��#[�e�N>���/�^��*����!<��X��?W(Uf��u�<����6�=��8�x�"=������^�C�==rY'H��D�G���7!{$u`��"�����"p�+��WJ\4����l�|//��}����O���>q9�=�L���@?P$l��)�k���y^�[�K�;�E*l�x5+�uY��C���Ci�x��^������G��H��o���C���C���:�D�=��+�J{��,M�C�J�����cXQ���������3F�Xd�%���U��>����WD(��![�\���M�/�uN���R�L4c07�C�������u�Jm�&B����XOG���
�2�o��%�
(^��|�[�[�����7T��/h-�����_�*t�~�C�zu��'��B^��:Pw��<+����m�Yp_��(+q��fk�#�����'�p�L=ST���=�4���@#*T�^�5�x�V���%s	+L36A���������P	�dc���/k�'�xC��i��>��{�����"E�_�������7cKm�D�8PXl,������m��� �$�3�P8z2������o�f����S�����_���?2FtBZ�C���v���`).���j2�yp��-�t�����p�iFKf{� Hx��|e���_[K�Q��^�i+>e�0���S��}i�t���p\)yJ�-C�e)�nf4���������jV
�1�9�
����G�q8A5�u����-� ��p�<�qc�oh1�`�^b`'���c_16]d�
d0�L7�v�K��3��e��5_�aJ�}��,+�'�9|s� t
&��`wB���a�A�F�T�n+������KF�'��L=}��u �����`�p�p�vi�e��;C�ka$3(l��,l#�d�!�CY��<[,�
73?����m�k^�1�<]�{:y���#pA: 8�h�*�S��G�pXz�1������������)��h��R)���3�����H)8�"N��Jg�0w�Kvg��6�]��Pt�`�P�3����Z/�_?�[�����}7O���������cit��z��O3��9��^�8��w|�K��"
e��pRO�����:���
��c�J�@���')��N(����vY�������`o��&U aE�"��������K�pC/��^�`����<����*�>���2���rF��QD�
'4$��n��r�7��R���&���A5�|9����Z���H�]��rZ�>�<�6L�)�.	�p�k�u���f�Z�����=�`���8�]�=P��a�����l���=7`�=�Uw����
��B;��c]Q�8�(u�Z��Z�}j�y
�%Zf#���J�\,�5�_��9"�v�CU�q��Q"�^�w�
�//��z�"_e�pRF ~q��������B8�s����q'wO"�2����Aa�	�F��
���Gw�Q��`p�J_/�i8�]��,g!�uH�|���JMmyJ!��&��y �����T%"K%IM��{������������D���h�;n\��)O��U28�Q�����w~�db��c�z���Fh�
�
������#��v�����@�q� ��
����d���x�b/�n�I�����l�+�����@7
�X	�yN�1���
;��s4QN���g�!������T�,d���z������������#��a~��]���-d���]w����Qx���r���V�T&������r�+�L���9v��d3���b�gp�� N�o��je+��������~^.��&�N�k4���%���h�r0'!"�|���e��2�\|�T�����g^����;���	�["c�V��gO���]%��U@��|�m��������h�f("\����E����-��*�Y�Z�	iZ_�KKfx� fP=��k��>�)R���Cp����{���G[bR����V4M�u������/�aL*rlS��=�G�Z���Z�;fG��c�Yq���!�#$Gn	�|�Ld;'����9��X_��g_��n��)+S`>���lZPG��M����({�vw��g��cB���1a�����_�.�'��hR6w�����BD���GL�6
#wm: ���E���]�^nl������Y�v��`O�	.[_�p,v��~�K����k������j�����m��oA�Wi
1>�,W�-'_<HRx�������]_2Q�������RM�7;����=VH����E��
��a�G_G5��6��5$���:U�M�,(t�:j�ig��E��+v��tS�r�m�����/'p������!�(B��@vn[��;��P�r����T�tB ��mC{1�o��-v��c�A3T�rA��o��w��T�(��^Y��0�lts��T��c��;�&�
F�E�n�=���tP���~������<�e���:_�,�xL�<t3��M�f/%O�����bgP@�K���O`i�!e�.
�ap1hpc.9��������J��6x��v���w��1�0�2��
<-�r�9,�
��[Fxi�]H'�P
��`�xfZa���@��6-�7�W�������������q�^��fB\��������$�8�V�A
[�����.!��
	$m$8�%c�/���������U7��PN$���/kh�u��0����X�����������3�p��
��q�?�
�p* �%�B���b�/�D�~��G�A��a�2<�7i��� 6��0��v� j��$���{�����8RP�m'��j;�X��>bKK�F��ha��9p�1/>L���^�%����1Z_��]�uCk���Q|WLkT��,�|�:G�I��F�#�����XL��<�gZo#
��/��b�O��G�'��f?9����y��[8qo���#��AY���	�$$�<�c8C���pq��?�[�.
%s����^j�;��\0�%P2�l�A&���h�
PcY-���'�����0�E��[����p���d�R�9H8��"�]2_lVQV����@�]��k����L����E��Q��d"^"�-\���bx#mY���G4���
��K�3'���|���&5�����Kz8v��FJ��������b����W��^G{��*������.�Gn0�X8��/���W�����P,�����hu�����>�����R�6�FjVU�)��^�4[������pg��a�a
�57�xg�20��_�nt�]����ydyA������*W�O��k@5�m+�0�e�����z�4q]������y�a�o�J�5r��cX=:�z����<�X��~A�#���#z����4�~�����]Q=����ZdW;�������5��h���OX���,&�T��
d��
��\
���8�)<8����37����H(Vj���q+�����D	1	P^d��A���zn��"�������ag��]i�{!����Fb�$S�8)��Eb��b���a������6����w���_+�R���������a7���ab��{h�TO*��`���$%�a�L�;�Zf-�����19�^��-�Ng�e'����@�]7���M��|��=�E�0�UG���#��Cs�8�|
)���Q��a��)I9��C�X�"%�(L`XsTQ��#���'��-��S�������*[�����^�MY�'{(<���������r^�y���#���4���m�!H.�z�&��R	�'H�~a�p�h�v�B�����f�i#��&l����ku���K���d�����w�5*����6	���
��[�#���t5��f]Xu���1�;��x�X�_3���d,]n~����^p7��_����xD9���F��P;���P����Y����'�rC!
i��!��m�1H��/�h�S`��JF6'^���\*�]gU�>�2^BkQ�7����Qd����+�0���C	'{�H>'pr��y��B���0� n���H�q7o�v�k�N��ed��\2T��H�,}�,;%���D�Kt}jK%��&�$�F	�H���,[^8������k�o)s#\���w*rKLG�(���Td�#�������F%-���&z.#���]1�#���}�4D���e�r^$�3���7���6�TV��v�U�����p7�q�&�7j���E������h38@�bA���cq}dU�E&I%0��D@5|����������C�AH�ud�����
�H)q���d��4�y7L�L�%��T�[���&�PQ��oGoOm�B��T�RN �BK������|E����e^q��C��3����&��-�`���{_�J��o�����$�E����o�
1/��9uP���v@h8�WP&�R�,bZ� k��t{��@�~/<$�����"C�jlKl��v��&��
R�b0�!r��[�D6o�o���M���N'��3�8�s6�%��x��r;K
�f�b�����H�����N���
�]�T�Lw7N�rjKg}y��l��X��K)~�3����'E�9���KB
����^(��J���>-+H
�s�5��n�������=���)���n�
��7��Z�J"��}PeVV����:9��\�z�����v�����$����l����b�mN7���3��+���OXD���C���9�[�hs�������ja������s��P��.P,�SI��e0@2��T���|�U��������6���f���S�A��X�k�-��~��;��z�	]<���t��2�~�?1L�Di�!����-	�H
�1g�cL����O����<�bn��z<�u��������g<e���FDgR�o����c�p?���:f��Y<�������C����@b���_��W��p��9D
�!%=��;��MW0�����&��PG�{�1JYC%,lR����QA�,]p��0��;.\�'u�*��;�:j�9F�e2L��������y��N*�I����g�T�#��k�+��.�,���O��?��v���-��{���FchI#�����g	�y�F;�n���ca���!t����ia��w��f�.f�dRgi����}`�����d���q`����_o��V�����U��O�qH?}��������<�`�},�&�c�:�����].�th#f��-��^Xy���33t���j�p`��Oc"�$�t��+_D����qka-����^<.z����S�Q4������R�$L�����C�:>G��T��������Dv�MqL�y~��L����w@���b9�hM,�����/@�R���"R�
	���I�*Z����	X����k�[�����+��si������[+��W]��LS/&�bEH��U��	���P��������%� �K�X��S����BbGO��3$^�[�M��Mr��|�5B~�[����k�Ev3�V����"b��`�����1���Qd���0���G3����a�_^N�+6@a��9m�+|�=V">��f����`����b����������|=�%��~�'�C���*���0Zev�{�yPz����������go.�
C��R�OT*����p��J�2��+0L�1j��`4�|dg�8�g�u�l6�+�#��mzj�=h�����e��dsA^7l�0��N�c2�&����i���^Fr�|��������4_��W��\�B���g��H%�����VFo�m���m�K��m�����:���WW~��m����8�f�iT�4����Y�/y�X?���0W,\�2���*##_���F���1��A������\@������A�������`r�V�C�#�6����	3�c�gU%'�"�fh�C�f(���1��W�c��]
'�n��y�Y0(����N��{H/�a�������A��Th���kQ�g
Gi����g�%1�]������Phvp���������j��?�J�E�e�i��+�]���	�z�:�j8#�(��/��39`���:R|�	��@�S2R���o:����	|�}<0R(��>�U~s����������������,Z���H��y����O
R�;��p�(��(�DR���\A����Ey9�$����2��YxP��g��VU+��`�����=����O *\�������u�K��;=r���O�kq4b��)Wz?B�4��!`������|��
&�V/��h���p��{����F� G=<�xD��!��-�`�z�����O/��*���7�ah�]���(P��j#q��@��'����(�rIo!���X�"��d�"���)J���t�`�U�yX0T��,����-�s��+Iu���v�����>���4O�������,�?e|UT�
0001-Added-various-optimizations-to-tuplesort-quicksort.patchtext/x-patch; charset=US-ASCII; name=0001-Added-various-optimizations-to-tuplesort-quicksort.patchDownload
From 043b92f519630cb1fd7c84c1461e6ab420f5d7eb Mon Sep 17 00:00:00 2001
From: Peter Geoghegan <peter@peter.laptop>
Date: Mon, 26 Sep 2011 03:02:19 +0100
Subject: [PATCH] Added various optimizations to tuplesort quicksort.

We now take advantage of optimizations from knowing which comparator to
use at compile time, as well as from avoiding using the general SQL
callable function machinery for comparators
---
 src/backend/utils/sort/tuplesort.c     |   55 +++++++-
 src/include/utils/template_qsort_arg.h |  217 ++++++++++++++++++++++++++++++++
 2 files changed, 266 insertions(+), 6 deletions(-)
 create mode 100644 src/include/utils/template_qsort_arg.h

diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 3505236..b006c3e 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -112,6 +112,7 @@
 #include "utils/memutils.h"
 #include "utils/pg_rusage.h"
 #include "utils/rel.h"
+#include "utils/template_qsort_arg.h"
 #include "utils/tuplesort.h"
 
 
@@ -1225,7 +1226,27 @@ puttuple_common(Tuplesortstate *state, SortTuple *tuple)
 }
 
 /*
- * All tuples have been provided; finish the sort.
+ * Manufacture type specific sorting specialisations
+ * with inline comparators
+ */
+static inline int
+compare_int4(Datum first, Datum second)
+{
+	int32		first_int = DatumGetInt32(first);
+	int32		second_int = DatumGetInt32(second);
+
+	if (first_int > second_int)
+		return 1;
+	else if (first_int == second_int)
+		return 0;
+	else
+		return -1;
+}
+
+TEMPLATE_QSORT_ARG(int4, compare_int4);
+
+/*
+ * All tuples have been provided; perform the sort.
  */
 void
 tuplesort_performsort(Tuplesortstate *state)
@@ -1247,11 +1268,30 @@ tuplesort_performsort(Tuplesortstate *state)
 			 * amount of memory.  Just qsort 'em and we're done.
 			 */
 			if (state->memtupcount > 1)
-				qsort_arg((void *) state->memtuples,
-						  state->memtupcount,
-						  sizeof(SortTuple),
-						  (qsort_arg_comparator) state->comparetup,
-						  (void *) state);
+			{
+				/*
+				 * Choose between various type-specific qsort variants.
+				 * Do so to avail of per-type compiler optimizations, in
+				 * particular, inlining of comparators.
+				 */
+
+				if (state->scanKeys && state->scanKeys->sk_func.fn_oid == 351 &&
+					state->comparetup == &comparetup_heap)
+					int4_qsort_arg((void *) state->memtuples,
+								  state->memtupcount,
+								  sizeof(SortTuple),
+								  (void *) state);
+				else
+					/*
+					 * Finally, fall back on type-neutral qsort with
+					 * function-pointer comparator
+					 */
+					qsort_arg((void *) state->memtuples,
+								  state->memtupcount,
+								  sizeof(SortTuple),
+								  (qsort_arg_comparator) state->comparetup,
+								  (void *) state);
+			}
 			state->current = 0;
 			state->eof_reached = false;
 			state->markpos_offset = 0;
@@ -2657,6 +2697,9 @@ myFunctionCall2Coll(FmgrInfo *flinfo, Oid collation, Datum arg1, Datum arg2)
  * and return a 3-way comparison result.  This takes care of handling
  * reverse-sort and NULLs-ordering properly.  We assume that DESC and
  * NULLS_FIRST options are encoded in sk_flags the same way btree does it.
+ *
+ * Note that this logic is more-or-less duplicated in template_qsort_arg.h,
+ * and "instantiated" per-type there.
  */
 static inline int32
 inlineApplySortFunction(FmgrInfo *sortFunction, int sk_flags, Oid collation,
diff --git a/src/include/utils/template_qsort_arg.h b/src/include/utils/template_qsort_arg.h
new file mode 100644
index 0000000..41f8237
--- /dev/null
+++ b/src/include/utils/template_qsort_arg.h
@@ -0,0 +1,217 @@
+/*-------------------------------------------------------------------------
+ *	template_qsort_arg.h: "template" version of qsort_arg.c
+ *
+ *	This version of qsort_arg is exclusively used within tuplesort.c to
+ *	more efficiently sort common types such as integers and floats. In 
+ *	providing this version, we seek to take advantage of compile-time 
+ *	optimisations for specific types, in particular, the inlining of 
+ *	comparators, as well as reducing the overhead of comparisons that
+ *	the general SQL-callable-function API imposes.
+ *
+ *	It is assumed that any types that use this infrastructure have at 
+ *	most one element in their ScanKeys array.
+ *
+ *	CAUTION: if you change this file, see also qsort_arg.c as well as 
+ *	qsort.c. qsort_arg.c should be considered authoratative.
+ *
+ *	Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group
+ *	Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *	src/include/utils/template_qsort_arg.h
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+
+#define swapcode(TYPE, parmi, parmj, n) 	\
+do {						\
+	size_t i = (n) / sizeof (TYPE);		\
+	TYPE *pi = (TYPE *)(void *)(parmi);	\
+	TYPE *pj = (TYPE *)(void *)(parmj);	\
+	do {					\
+		TYPE	t = *pi;		\
+		*pi++ = *pj;			\
+		*pj++ = t;			\
+		} while (--i > 0);		\
+} while (0)
+
+#define SWAPINIT(a, es) swaptype = ((char *)(a) - (char *)0) % sizeof(long) || \
+	(es) % sizeof(long) ? 2 : (es) == sizeof(long)? 0 : 1;
+
+#define thistype_vecswap(TYPE, a, b, n) if ((n) > 0) TYPE##_swapfunc((a), (b), (size_t)(n), swaptype) 	
+
+#define thistype_swap(TYPE, a, b)				\
+if (swaptype == 0) {						\
+		long t = *(long *)(void *)(a);			\
+		*(long *)(void *)(a) = *(long *)(void *)(b);	\
+		*(long *)(void *)(b) = t;			\
+	} else							\
+		TYPE##_swapfunc(a, b, es, swaptype)	
+
+/* 
+ * This macro manufactures a type-specific implementation of qsort_arg with 
+ * the comparator, COMPAR, known at compile time. COMPAR is typically an
+ * inline function.
+ *
+ * COMPAR should take as its arguments two Datums, and return an int, in
+ * line with standard qsort convention.
+ * 
+ * We have void* parameters for TYPE##AppSort just to shut up the compiler. 
+ * They could be SortTuple pointers instead, but that would make it more 
+ * difficult to keep template_qsort_arg.h consistent with qsort_arg.c. 		
+ *
+ */
+
+#define TEMPLATE_QSORT_ARG(TYPE, COMPAR)				\
+void TYPE##_qsort_arg(void *a, size_t n, size_t es, void *arg);		\
+												\
+static inline int32										\
+TYPE##AppSort(const void *a, const void *b, Tuplesortstate *state)				\
+{												\
+	int32		compare;								\
+	const SortTuple* aT = a;								\
+	const SortTuple* bT = b;								\
+	/* Allow interrupting long sorts */							\
+	CHECK_FOR_INTERRUPTS();									\
+												\
+	Assert(state->nKeys == 1);								\
+	Assert(state->scanKeys);								\
+												\
+	if ( aT->isnull1)									\
+	{											\
+		if (bT->isnull1)								\
+			compare = 0;		/* NULL "=" NULL */				\
+		else if (state->scanKeys->sk_flags & SK_BT_NULLS_FIRST)				\
+			compare = -1;		/* NULL "<" NOT_NULL */				\
+		else										\
+			compare = 1;		/* NULL ">" NOT_NULL */				\
+	}											\
+	else if (bT->isnull1)									\
+	{											\
+		if (state->scanKeys->sk_flags & SK_BT_NULLS_FIRST)				\
+			compare = 1;		/* NOT_NULL ">" NULL */				\
+		else										\
+			compare = -1;		/* NOT_NULL "<" NULL */				\
+	}											\
+	else											\
+	{											\
+		compare = COMPAR(aT->datum1, bT->datum1);					\
+												\
+		if (state->scanKeys->sk_flags & SK_BT_DESC)					\
+			compare = -compare;							\
+	}											\
+												\
+	return compare;										\
+}												\
+									\
+static inline void							\
+TYPE##_swapfunc(char *a, char *b, size_t n, int swaptype)		\
+{									\
+	if (swaptype <= 1)						\
+		swapcode(long, a, b, n); 				\
+	else								\
+		swapcode(char, a, b, n);				\
+}									\
+									\
+static inline char *										\
+TYPE##_med3(char *a, char *b, char *c, void *arg)						\
+{												\
+	return TYPE##AppSort(a, b, arg) < 0 ?							\
+		(TYPE##AppSort(b, c, arg) < 0 ? b : (TYPE##AppSort(a, c, arg) < 0 ? c : a))	\
+		: (TYPE##AppSort(b, c, arg) > 0 ? b : (TYPE##AppSort(a, c, arg) < 0 ? a : c));	\
+}												\
+												\
+void 												\
+TYPE##_qsort_arg(void *a, size_t n, size_t es, void *arg)					\
+{												\
+	char	   *pa,										\
+			   *pb,									\
+			   *pc,									\
+			   *pd,									\
+			   *pl,									\
+			   *pm,									\
+			   *pn;									\
+	int			d,								\
+				r,								\
+				swaptype,							\
+				presorted;							\
+												\
+loop:SWAPINIT(a, es); 										\
+	if (n < 7)										\
+	{											\
+		for (pm = (char *) a + es; pm < (char *) a + n * es; pm += es)			\
+			for (pl = pm; pl > (char *) a && TYPE##AppSort(pl - es, pl, arg) > 0;	\
+				 pl -= es)							\
+				thistype_swap(TYPE, pl, pl - es);				\
+		return;										\
+	}											\
+	presorted = 1;										\
+	for (pm = (char *) a + es; pm < (char *) a + n * es; pm += es)				\
+	{											\
+		if (TYPE##AppSort(pm - es, pm, arg) > 0)					\
+		{										\
+			presorted = 0;								\
+			break;									\
+		}										\
+	}											\
+	if (presorted)										\
+		return;										\
+	pm = (char *) a + (n / 2) * es;								\
+	if (n > 7)										\
+	{											\
+		pl = (char *) a;								\
+		pn = (char *) a + (n - 1) * es;							\
+		if (n > 40)									\
+		{										\
+			d = (n / 8) * es;							\
+			pl = TYPE##_med3(pl, pl + d, pl + 2 * d, arg);				\
+			pm = TYPE##_med3(pm - d, pm, pm + d, arg);				\
+			pn = TYPE##_med3(pn - 2 * d, pn - d, pn, arg);				\
+		}										\
+		pm = TYPE##_med3(pl, pm, pn, arg);						\
+	}											\
+	thistype_swap(TYPE, a, pm);								\
+	pa = pb = (char *) a + es;								\
+	pc = pd = (char *) a + (n - 1) * es;							\
+	for (;;)										\
+	{											\
+		while (pb <= pc && (r = TYPE##AppSort(pb, a, arg)) <= 0)			\
+		{										\
+			if (r == 0)								\
+			{									\
+				thistype_swap(TYPE, pa, pb);					\
+				pa += es;							\
+			}									\
+			pb += es;								\
+		}										\
+		while (pb <= pc && (r = TYPE##AppSort(pc, a, arg)) >= 0)			\
+		{										\
+			if (r == 0)								\
+			{									\
+				thistype_swap(TYPE, pc, pd);					\
+				pd -= es;							\
+			}									\
+			pc -= es;								\
+		}										\
+		if (pb > pc)									\
+			break;									\
+		thistype_swap(TYPE, pb, pc);							\
+		pb += es;									\
+		pc -= es;									\
+	}											\
+	pn = (char *) a + n * es;								\
+	r = Min(pa - (char *) a, pb - pa);							\
+	thistype_vecswap(TYPE, a, pb - r, r);							\
+	r = Min(pd - pc, pn - pd - es);								\
+	thistype_vecswap(TYPE, pb, pn - r, r);							\
+	if ((r = pb - pa) > es)									\
+		TYPE##_qsort_arg(a, r / es, es, arg);						\
+	if ((r = pd - pc) > es)									\
+	{											\
+		/* Iterate rather than recurse to save stack space */				\
+		a = pn - r;									\
+		n = r / es;									\
+		goto loop;									\
+	}											\
+}												\
+
-- 
1.7.6

#30Robert Haas
robertmhaas@gmail.com
In reply to: Peter Geoghegan (#29)
Re: Inlining comparators as a performance optimisation

On Sun, Sep 25, 2011 at 10:12 PM, Peter Geoghegan <peter@2ndquadrant.com> wrote:

[ new results ]

Nice results. I think these are far more convincing than the last
set, because (1) the gains are bigger and (2) they survive -O2 and (3)
you tested an actual query, not just qsort() itself.

I don't want to take time to review this in detail right now, because
I don't think it would be fair to put this ahead of things that were
submitted for the current CommitFest, but I'm impressed.

On the subject of highly ambitious optimisations to sorting, one
possibility I consider much more practicable than GPU-accelerated
sorting is simple threading; quicksort can be parallelised very
effectively, due to its divide-and-conquer nature. If we could agree
on a threading abstraction more sophisticated than forking, it's
something I'd be interested in looking at. To do so would obviously
entail lots of discussion about how that relates to whatever way we
eventually decide on implementing parallel query, and that's obviously
a difficult discussion.

I have the same feeling about about this that I do about almost every
executor optimization that anyone proposes: the whole project would be
entirely simple and relatively painless if it weren't for the need to
make planner changes. I mean, deciding on a threading interface is
likely to be a somewhat contentious discussion, with differences of
opinion on whether we should do it and what the API should look like.
But at the end of the day it's not rocket science, and I expect that
we would end up with something reasonable. What seems much harder is
figuring out how to decide when to perform quicksort in parallel vs.
single-threaded, and how much parallelism would be appropriate. I
haven't seen anyone propose even a shadow of an idea about how to make
such decisions intelligently, either in general or in specific cases.

The other issue is that, while threading would be possibly suitable
for this particular case, at least for built-in datatypes with
comparison operations that basically reduce to single machine-language
comparison instructions, it's hard to see how we could take it much
further. It would be unsafe for these multiple threads of execution
to do anything that could possibly throw an error or anything that
touches a lightweight lock or, really, just about anything at all.
Trying to make the entire backend thread-safe - or even any
significant portion of it - seems like a colossal effort that will
most likely fail, but maybe not without eating an enormous amount of
developer time first. And without doing that, I don't think we could
even extend this as far as, say, numeric, whose functions do things
like palloc() and ereport() internally. So I feel like this whole
approach might be a dead-end - there's a narrow range of cases where
it could be made to work, I think, but after that I think you hit a
titanium wall.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#31Peter Geoghegan
peter@2ndquadrant.com
In reply to: Robert Haas (#30)
Re: Inlining comparators as a performance optimisation

On 26 September 2011 04:46, Robert Haas <robertmhaas@gmail.com> wrote:

I don't want to take time to review this in detail right now, because
I don't think it would be fair to put this ahead of things that were
submitted for the current CommitFest, but I'm impressed.

Thank you.

Now that I think about it, the if statement that determines if the
int4 specialisation will be used may be okay - it sort of documents
the conditions under which the int4 specialisation should be used with
reference to how the "else" generic case actually works, which is
perhaps no bad thing. I now realise that I should be using constants
from fmgroids.h though.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

#32Bruce Momjian
bruce@momjian.us
In reply to: Peter Geoghegan (#29)
Re: Inlining comparators as a performance optimisation

Peter Geoghegan wrote:

On the subject of highly ambitious optimisations to sorting, one
possibility I consider much more practicable than GPU-accelerated
sorting is simple threading; quicksort can be parallelised very
effectively, due to its divide-and-conquer nature. If we could agree
on a threading abstraction more sophisticated than forking, it's
something I'd be interested in looking at. To do so would obviously
entail lots of discussion about how that relates to whatever way we
eventually decide on implementing parallel query, and that's obviously
a difficult discussion.

I agree that the next big challenge for Postgres is parallel operations.
With the number of cores increasing, and with increased memory and SSD,
parallel operation is even more important. Rather than parallelizing
the entire backend, I imagine adding threading or helper processes for
things like sorts, index scans, executor nodes, and stored procedure
languages. I expect final code to be 2-3 years in the future.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

#33Greg Stark
stark@mit.edu
In reply to: Bruce Momjian (#32)
Re: Inlining comparators as a performance optimisation

On Wed, Oct 5, 2011 at 2:49 AM, Bruce Momjian <bruce@momjian.us> wrote:

Rather than parallelizing
the entire backend, I imagine adding threading or helper processes for
things like sorts, index scans, executor nodes, and stored procedure
languages.  I expect final code to be 2-3 years in the future.

I don't actually see it would be a big problem quicksort to start up a
bunch of threads which do some of the work and go away when the
tuplesort is done. As long as the code they're executing is well
defined and limited to a small set of code that can be guaranteed to
be thread-safe it should be reasonably simple.

The problem is that in most of the Postgres core there aren't many
places where much code fits that description. Even in tuplesort it can
call out to arbitrary user-defined functions so we would need a way to
mark which functions are thread-safe. Beyond integer and floating
point comparisons it may not be much -- certainly not Numeric or
strings due to detoasting... And then there are things like handling
the various signals (including SI invalidations?) and so on.

I agree that if we wanted to farm out entire plan nodes we would
probably end up generating new partial plans which would be handed to
other backend processes. That's not unlike what Oracle did with
Parallel Query. But i'm a bit skeptical that we'll get much of that
done in 2-3 years. The main use case for Parallel Query in Oracle is
for partitioned tables -- and we haven't really polished that after
how many years?

--
greg

#34Robert Haas
robertmhaas@gmail.com
In reply to: Greg Stark (#33)
Re: Inlining comparators as a performance optimisation

On Tue, Oct 4, 2011 at 10:55 PM, Greg Stark <stark@mit.edu> wrote:

I agree that if we wanted to farm out entire plan nodes we would
probably end up generating new partial plans which would be handed to
other backend processes. That's not unlike what Oracle did with
Parallel Query. But i'm a bit skeptical that we'll get much of that
done in 2-3 years. The main use case for Parallel Query in Oracle is
for partitioned tables -- and we haven't really polished that after
how many years?

Partitioning hasn't been completely neglected; 9.1 adds support for
something called Merge Append. You may have heard of (or, err,
authored) that particular bit of functionality. :-)

Of course, it would be nice to have a better syntax, but I don't think
the lack of it should discourage us from working on parallel query,
which is a muti-part problem. You need to:

- have planner support to decide when to parallelize
- have a mechanism for firing up worker processes and synchronizing
the relevant bits of state between the master and the workers
- have an IPC mechanism for streaming data between the master process
and the workers
- figure out how to structure the executor so that the workers neither
stall nor run too far ahead of the master (which might have LIMIT 10
or something)

Markus Wanner took a crack at generalizing the autovacuum machinery
that we have now into something that could be used to fire up
general-purpose worker processes, but it fell down mostly because I
(and, I think, others) weren't convinced that imessages were something
we wanted to suck into core, and Markus reasonably enough wasn't
interested in rewriting it to do something that wouldn't really help
his work with Postgres-R. I'm not sure where Bruce is getting his
timeline from, but I think the limiting factor is not so much that we
don't have people who can write the code as that those people are
busy, and this is a big project. But you can bet that if it gets to
the top of Tom's priority list (just for example) we'll see some
motion...!

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#35Bruce Momjian
bruce@momjian.us
In reply to: Robert Haas (#34)
Re: Inlining comparators as a performance optimisation

Robert Haas wrote:

Markus Wanner took a crack at generalizing the autovacuum machinery
that we have now into something that could be used to fire up
general-purpose worker processes, but it fell down mostly because I
(and, I think, others) weren't convinced that imessages were something
we wanted to suck into core, and Markus reasonably enough wasn't
interested in rewriting it to do something that wouldn't really help
his work with Postgres-R. I'm not sure where Bruce is getting his
timeline from, but I think the limiting factor is not so much that we
don't have people who can write the code as that those people are
busy, and this is a big project. But you can bet that if it gets to
the top of Tom's priority list (just for example) we'll see some
motion...!

I was thinking of setting up a team to map out some strategies and get
community buy-in, and then we could attack each issue. I got the 2-3
years from the Win32 timeline.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

#36Robert Haas
robertmhaas@gmail.com
In reply to: Peter Geoghegan (#29)
Re: Inlining comparators as a performance optimisation

On Sun, Sep 25, 2011 at 10:12 PM, Peter Geoghegan <peter@2ndquadrant.com> wrote:

I've produced something much neater than my first patch, attached,
although I still consider this to be at the POC stage, not least since
I'm not exactly sure how I should be selecting the right
specialisation in tuplesort_performsort() (a hack is used right now
that does a direct pg_proc OID comparison), and also because I haven't
implemented anything other than qsort for heap tuples yet (a minor
detail, I suppose). I'm pleased to say that a much clearer picture of
what's really going on here has emerged.

Statistics: Total runtime according to explain analyze for query
"select * from orderlines order by prod_id" (dellstore2 sample db), at
GCC 4.5's -02 optimisation level, after warming the cache, on my
desktop:

Without the patch:

~82ms

With the patch, but with the "inline" keyword commented out for all
new functions/meta-functions:

~60ms

with the patch, unmodified:

~52ms

Reviewing away when I should be sleeping, wahoo...!

I think that we should really consider doing with this patch what Tom
suggested upthread; namely, looking for a mechanism to allow
individual datatypes to offer up a comparator function that doesn't
require bouncing through FunctionCall2Coll(). It seems to me that
duplicating the entire qsort() algorithm is a little iffy. Sure, in
this case it works out to a win. But it's only a small win -
three-quarters of it is in the uncontroversial activity of reducing
the impedance mismatch - and how do we know it will always be a win?
Adding more copies of the same code can be an anti-optimization if it
means that a smaller percentage of it fits in the instruction cache,
and sometimes small changes in runtime are caused by random shifts in
the layout of memory that align things more or less favorably across
cache lines rather than by real effects. Now it may well be that this
is a real effect, but will it still look as good when we do this for
10 data types? For 100 data types?

In contrast, it seems to me that reducing the impedance mismatch is
something that we could go and do across our entire code base, and
every single data type would benefit from it. It would also be
potentially usable by other sorting algorithms, not just quick sort.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#37Simon Riggs
simon@2ndQuadrant.com
In reply to: Robert Haas (#36)
Re: Inlining comparators as a performance optimisation

On Fri, Nov 18, 2011 at 5:20 AM, Robert Haas <robertmhaas@gmail.com> wrote:

I think that we should really consider doing with this patch what Tom
suggested upthread; namely, looking for a mechanism to allow
individual datatypes to offer up a comparator function that doesn't
require bouncing through FunctionCall2Coll().  It seems to me that
duplicating the entire qsort() algorithm is a little iffy.  Sure, in
this case it works out to a win.  But it's only a small win -
three-quarters of it is in the uncontroversial activity of reducing
the impedance mismatch - and how do we know it will always be a win?
Adding more copies of the same code can be an anti-optimization if it
means that a smaller percentage of it fits in the instruction cache,
and sometimes small changes in runtime are caused by random shifts in
the layout of memory that align things more or less favorably across
cache lines rather than by real effects.  Now it may well be that this
is a real effect, but will it still look as good when we do this for
10 data types?  For 100 data types?

In contrast, it seems to me that reducing the impedance mismatch is
something that we could go and do across our entire code base, and
every single data type would benefit from it.  It would also be
potentially usable by other sorting algorithms, not just quick sort.

I don't think its credible to implement that kind of generic
improvement at this stage of the release cycle. That has a much bigger
impact since it potentially effects all internal datatypes and
external ones also. Definitely a longer term way forward.

If individual datatypes offer up a comparator function that is easily
going to result in more code than is being suggested here. So the
argument about flooding the CPU cache works against your alternate
proposal, not in favour of it.

We have no proof that we need to do this for 10 or 100 data types. We
only currently have proof that there is gain for the most common
types. Of course, it sounds like it might be useful to allow any data
type to gain an advantage, but we shouldn't be blind to the point that
almost nobody will use such a facility, and if they do the code won't
be written for a long time yet. If this came as a request from custom
datatype authors complaining of slow sorts it would be different, but
it didn't so we don't even know if anybody would ever write user
defined comparator routines. Rejecting a patch because of a guessed
user requirement is not good.

Peter's suggested change adds very few lines of code and those compile
to some very terse code, a few hundred instructions at very most.
Requesting an extra few cachelines to improve qsort by so much is
still an easy overall win.

The OP change improves qsort dramatically, and is a small, isolated
patch. There is no significant downside. We also have it now, so lets
commit this, chalk up another very good performance improvement and
use our time on something else this commitfest, such as the flexlocks
idea.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

#38Robert Haas
robertmhaas@gmail.com
In reply to: Simon Riggs (#37)
Re: Inlining comparators as a performance optimisation

On Fri, Nov 18, 2011 at 3:53 AM, Simon Riggs <simon@2ndquadrant.com> wrote:

We have no proof that we need to do this for 10 or 100 data types. We
only currently have proof that there is gain for the most common
types.

Well, that's kind of my point. I think this needs more work before we
decide what the best approach is. So far, the ONLY test result we
have that compares inlining to not inlining shows a speedup from 60 ms
to 52 ms. I think that an 8 ms speedup on one test with one datatype
on one platform/compiler combination isn't sufficient evidence to
conclude that this is the best possible approach.

I think the way to look at this is that this patch contains two
possibly good ideas: one of which is to make the qsort() argument
match the qsort() calling convention, and the other of which is to
have multiple copies of the qsort() logic that inline the comparators
for their respective datatypes. Tom hypothesized that most of the
benefit was in the first idea, and the numbers that Peter posted seem
to support that conclusion. The first idea is also less invasive and
more broadly applicable, so to me that seems like the first thing to
pursue.

Now, that doesn't mean that we shouldn't consider the second idea as
well, but I don't believe that the evidence presented thus far is
sufficient to prove that we should go that route. It seems entirely
possible that inlining any non-trivial comparator function could work
out to a loss, or that the exact choice of compiler flags could affect
whether inlining works out to a plus or a minus (what happens with -O3
vs. -O2? what about -O1? what about -O2 -fno-omit-frame-pointer?
what if they're using HP's aCC or Intel's icc rather than gcc?).
There's no point in installing an optimization that could easily be a
pessimization on some other workload, and that hasn't been tested, or
at least no results have been posted here. On the other hand,
matching the calling convention of the comparator function to what
qsort() wants and eliminating the trampoline seems absolutely certain
to be a win in every case, and based on the work Peter has done it
seems like it might be a quite large win.

In fact, you have to ask yourself just exactly how much our
function-calling convention is costing us in general. We use that
mechanism an awful lot and whatever loss of cycles is involved would
be spread all over the code base where oprofile or gprof won't easily
be able to pick it out. Even the cost at any particular call site
will be split between the caller and the callee. There might be more
stuff we could optimize here than just sorting...

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#39Tom Lane
tgl@sss.pgh.pa.us
In reply to: Simon Riggs (#37)
Re: Inlining comparators as a performance optimisation

Simon Riggs <simon@2ndQuadrant.com> writes:

On Fri, Nov 18, 2011 at 5:20 AM, Robert Haas <robertmhaas@gmail.com> wrote:

I think that we should really consider doing with this patch what Tom
suggested upthread; namely, looking for a mechanism to allow
individual datatypes to offer up a comparator function that doesn't
require bouncing through FunctionCall2Coll().

I don't think its credible to implement that kind of generic
improvement at this stage of the release cycle.

Er, *what*? We're in mid development cycle, we are nowhere near
release. When exactly would you have us make major changes?

In any case, what I understood Robert to be proposing was an add-on
feature that could be implemented in one datatype at a time. Not
a global flag day. We couldn't really do the latter anyway without
making life very unpleasant for authors of extension datatypes.

regards, tom lane

#40Peter Geoghegan
peter@2ndquadrant.com
In reply to: Robert Haas (#36)
Re: Inlining comparators as a performance optimisation

On 18 November 2011 05:20, Robert Haas <robertmhaas@gmail.com> wrote:

I think that we should really consider doing with this patch what Tom
suggested upthread; namely, looking for a mechanism to allow
individual datatypes to offer up a comparator function that doesn't
require bouncing through FunctionCall2Coll().  It seems to me that
duplicating the entire qsort() algorithm is a little iffy.

I understand that we highly value extensibility and genericity (yes,
that's a real word). We may not always be well served by that
tendency.

Sure, in this case it works out to a win.  But it's only a small win -
three-quarters of it is in the uncontroversial activity of reducing
the impedance mismatch - and how do we know it will always be a win?

Firstly, 1/4 of a quite large gain is still a pretty good gain.
Secondly, I probably didn't actually isolate the effects of inlining,
nor the overall benefit of the compiler knowing the comparator at
compile-time. I just removed the inline keyword. Those are two
different things. The inline keyword serves as a request to the
compiler to inline. The compiler can and often will ignore that
request. Most people know that. What isn't so widely known is that
modern compilers may equally well inline even when they haven't been
asked to (but only when they can). When you also consider, as I've
already pointed out several times, that individual call sites are
inlined, it becomes apparent that there may well still be a certain
amount of inlining and/or other optimisations like procedural
integration going on at some call sites even without the encouragement
of the inline keyword, that would not have been performed without the
benefit of compile-time comparator knowledge. The addition of the
inline keyword may just, in this particular case, have the compiler
inline even more call sites. Posting the ~8ms difference was
motivated by a desire to prove that inlining had *some* role to play,
without actually going to the trouble of implementing Tom's idea as a
basis of comparison, because Tom was very sceptical of inlining.

The long and the short of it is that I'm going to have to get my hands
dirty with a dissembler before we really know exactly what's
happening. That, or I could use an optimisation fence of some type.

Adding more copies of the same code can be an anti-optimization if it
means that a smaller percentage of it fits in the instruction cache,
and sometimes small changes in runtime are caused by random shifts in
the layout of memory that align things more or less favorably across
cache lines rather than by real effects.  Now it may well be that this
is a real effect, but will it still look as good when we do this for
10 data types?  For 100 data types?

I'd favour limiting it to just the common integer and float types.

In contrast, it seems to me that reducing the impedance mismatch is
something that we could go and do across our entire code base, and
every single data type would benefit from it.  It would also be
potentially usable by other sorting algorithms, not just quick sort.

Suppose that we went ahead and added that infrastructure. What you
must acknowledge is that one reason that this speed-up is so dramatic
is that the essential expense of a comparison is already so low - a
single instruction - and therefore the overall per-comparison cost
goes way down, particularly if the qsort inner loop can store the code
across fewer cache lines. For that reason, any absolute improvement
that you'll see in complex datatypes will be smaller, maybe much
smaller, because for each comparison we'll execute many more
instructions that are essential to the comparison. In my estimation,
all of this work does not point to there being an undue overhead in
the function calling convention as you suggested. Still, I'm not
opposed to investigating generalising this in some way, reservations
notwithstanding, unless we have to block-wait on it. I don't want to
chase diminishing returns too far.

Well, that's kind of my point.  I think this needs more work before we
decide what the best approach is.

Agreed.

So far, the ONLY test result we
have that compares inlining to not inlining shows a speedup from 60 ms
to 52 ms.  I think that an 8 ms speedup on one test with one datatype
on one platform/compiler combination isn't sufficient evidence to
conclude that this is the best possible approach.

Fair enough, but it's not the only test I did - I posted other numbers
for the same query when the table was 48mb, and we saw a proportional
improvement, consistent with a per-comparison win. I'm supposed to be
on leave for a few days at the moment, so I won't be very active this
weekend, but I'm rather curious as to where you or others would like
to see me go with benchmarks.

I should point out that we currently don't have much idea how big of a
win applying these principles could be for index creation times...it
could possibly be very significant. My profiling of index creation
makes this looks promising.

On 18 November 2011 13:39, Robert Haas <robertmhaas@gmail.com> wrote:

It seems entirely possible that inlining any non-trivial comparator function could work
out to a loss,

Compilers weigh the size of the function to be inlined heavily when
deciding, on a call site by call site basis, whether or not to inline.
Much effort has been expended on making these heuristics work well.
Besides, you're the one advocating pursuing this for types with
non-trivial comparators. This will certainly be less effective for
such non-trivial types, perhaps quite a lot less effective, as already
noted.

It's worth acknowledging that sorting of integers and floats is in
general very important, probably more important than any other sorting
case.

or that the exact choice of compiler flags could affect
whether inlining works out to a plus or a minus (what happens with -O3
vs. -O2?  what about -O1?  what about -O2 -fno-omit-frame-pointer?
what if they're using HP's aCC or Intel's icc rather than gcc?).

If you'd like to see numbers for another platform, I can get those.
How about Windows? I only have access to x86_64 boxes. I don't see
that the exact compiler flags used will influence whether or not
inlining is a win, so much as whether or not it occurs at all. As to
-fno-omit-frame-pointer or something nullifying the benefits, well,
that's a bit fantastical. We're already at the mercy of the compiler
to do the right thing at this level of granularity. I really just want
to help/enable it.

There's no point in installing an optimization that could easily be a
pessimization on some other workload, and that hasn't been tested, or
at least no results have been posted here.

Hmm. Well, I accept the burden of proof lies with me here. I have a
hard time believing that inlining/compile-time optimisation benefits
(benefits which, to repeat for emphasis, have not been
isolated/measured yet) could be entirely nullified by CPU caching
effects on any plausible workload. What does a benchmark that
alleviates your concerns here look like?

The way that I understand inlining to interact with CPU cache is that
it will increase the number of cache misses if it causes a cache line
to be crossed, but will decrease the number of cache misses when
locality of reference is improved such that less cache lines are
crossed within an inner loop. Well, the overall cost of a sort is
measured in the number of comparisons performed, so locality of
reference is particularly important here - in general, the comparator
is called lots of times.

What I'm having a hard time with is general scepticism of the value of
inlining, when the self-contained test case I posted showed an
inlining qsort() radically out-performing my system's qsort(), without
any impedance mismatch to muddy the waters. Sure, you can hold me to
account and get me to prove my claim, but don't you think it's curious
that that effect was observed there?

Incidentally, I'd like to call this fast path sorting, if it ends up
being committed in a form that is not significantly different to what
we have now.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

#41Martijn van Oosterhout
kleptog@svana.org
In reply to: Robert Haas (#36)
Re: Inlining comparators as a performance optimisation

On Fri, Nov 18, 2011 at 12:20:26AM -0500, Robert Haas wrote:

I think that we should really consider doing with this patch what Tom
suggested upthread; namely, looking for a mechanism to allow
individual datatypes to offer up a comparator function that doesn't
require bouncing through FunctionCall2Coll(). It seems to me that
duplicating the entire qsort() algorithm is a little iffy. Sure, in
this case it works out to a win. But it's only a small win -
three-quarters of it is in the uncontroversial activity of reducing
the impedance mismatch - and how do we know it will always be a win?

There's always the old idea of a data type providing a function mapping
f:(type -> int64) in such a way that it preserves order. That is, in the
sense that:

f(x) < f(y) => x < y

When sorting, you add f(x) as hidden column and change "ORDER BY x" to
"ORDER BY f(x), x". Then you only need to special case the int64
version. This would mean that in most cases you may be able to skip
the call because you're comparing integers. The downside is you need
to call f on each input. It depends on the datatype if that's cheaper
or not, but for all numerics types I think it's an easy win.

I don't think anyone has written a proof of concept for this. It does
have the advantage of scaling better than coding a qsort for each
individual type.

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

He who writes carelessly confesses thereby at the very outset that he does
not attach much importance to his own thoughts.

-- Arthur Schopenhauer

#42Robert Haas
robertmhaas@gmail.com
In reply to: Peter Geoghegan (#40)
Re: Inlining comparators as a performance optimisation

On Fri, Nov 18, 2011 at 4:38 PM, Peter Geoghegan <peter@2ndquadrant.com> wrote:

I understand that we highly value extensibility and genericity (yes,
that's a real word). We may not always be well served by that
tendency.

True (except that genericity is not a synonym for generality AFAICT).
A good fraction of the optimizations that we've done over the last,
well, pick any arbitrary period of time consists in adding special
cases that occur frequently enough to be worth optimizing - e.g. HOT,
or fast relation locks - often to extremely good effect.

Firstly, 1/4 of a quite large gain is still a pretty good gain.
Secondly, I probably didn't actually isolate the effects of inlining,
nor the overall benefit of the compiler knowing the comparator at
compile-time. I just removed the inline keyword.

Maybe we should look at trying to isolate that a bit better.

It strikes me that we could probably create an API that would support
doing either of these things depending on the wishes of the underlying
datatype. For example, imagine that we're sorting with <(int4, int4).
We associate a PGPROC-callable function with that operator that
returns "internal", really a pointer to a struct. The first element
of the struct is a pointer to a comparison function that qsort() (or a
tape sort) can invoke without a trampoline; the second is a wholesale
replacement for qsort(); either or both can be NULL. Given that, it
seems to me that we could experiment with this pretty easily, and if
it turns out that only one of them is worth doing, it's easy to drop
one element out of the structure.

Or do you have another plan for how to do this?

Fair enough, but it's not the only test I did - I posted other numbers
for the same query when the table was 48mb, and we saw a proportional
improvement, consistent with a per-comparison win. I'm supposed to be
on leave for a few days at the moment, so I won't be very active this
weekend, but I'm rather curious as to where you or others would like
to see me go with benchmarks.

I should point out that we currently don't have much idea how big of a
win applying these principles could be for index creation times...it
could possibly be very significant. My profiling of index creation
makes this looks promising.

Have you done any benchmarks where this saves seconds or minutes,
rather than milliseconds? That would certainly make it more exciting,
at least to me.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#43Peter Geoghegan
peter@2ndquadrant.com
In reply to: Robert Haas (#42)
Re: Inlining comparators as a performance optimisation

On 19 November 2011 02:55, Robert Haas <robertmhaas@gmail.com> wrote:

Maybe we should look at trying to isolate that a bit better.

Indeed. Fortunately, GCC has options to disable each optimisation.
Here's potentially relevant flags that we're already implicitly using
at -02:

-finline-small-functions <-- this is the one that inlined functions
without an "inline" keyword
-findirect-inlining
-finline-functions-called-once
-fearly-inlining
-fipa-sra (Perform interprocedural scalar replacement of aggregates,
removal of unused parameters and replacement of parameters passed by
reference by parameters passed by value).

In an effort to better isolate the effects of inlining, I tried this:

./configure CFLAGS="-fno-inline -fno-inline-small-functions" (I could
have disabled more -02 optimisations, but this proved sufficient to
make my point)

Unsurprisingly, this makes things slower than regular -02.

With the patch, the same query (once again, using explain analyze)
with the same fast path quicksort stabilises around 92ms +/- 5ms
(recall that the original figure was ~82ms). Our gains evaporate, and
then some. Take away the additional CLAGS, and we're predictably back
to big gains, with the query taking ~52ms just as before.

What happens to this query when we build an unmodified postgres with
these same CFLAGS? Well, we see the query take ~116ms after a few
runs. It seems that the impedance mismatch matters, but inlining and
other optimisations look to be at least as important. This isn't
surprising to me, given what I was able to do with the isolated test.
Maybe I should have tried it with additional disabling of
optimisations named above, but that would have perhaps made things
less clear.

I'd probably have been better off directly measuring qsort speed and
only passing those flags when compiling tuplesort.c (maybe impedance
mismatch issues would have proven to have been even less relevant),
but I wanted to do something that could be easily recreated, plus it's
late.

It strikes me that we could probably create an API that would support
doing either of these things depending on the wishes of the underlying
datatype.  For example, imagine that we're sorting with <(int4, int4).
 We associate a PGPROC-callable function with that operator that
returns "internal", really a pointer to a struct.  The first element
of the struct is a pointer to a comparison function that qsort() (or a
tape sort) can invoke without a trampoline; the second is a wholesale
replacement for qsort(); either or both can be NULL.  Given that, it
seems to me that we could experiment with this pretty easily, and if
it turns out that only one of them is worth doing, it's easy to drop
one element out of the structure.

Or do you have another plan for how to do this?

I haven't given it much thought. Let me get back to you on that next week.

Have you done any benchmarks where this saves seconds or minutes,
rather than milliseconds?  That would certainly make it more exciting,
at least to me.

Right. Well, I thought I'd use pgbench to generate a large table in a
re-creatable way. That is:

pgbench -i -s 60

This puts pgbench_accounts at 769MB. Then, having increased work_mem
to 1GB (enough to qsort) and maintenance_work_mem to 756mb, I decided
to test this query with the patch:

explain analyze select * from pgbench_accounts order BY abalance;

This stabilised at ~3450ms, through repeatedly being executed.

How does this compare to unpatched postgres? Well, it stabilised at
about ~3780ms for the same query.

This patch is obviously less of a win as the number of tuples to sort
goes up. That's probably partly explained by the cost of everything
else going up at a greater rate than the number of comparisons. I
suspect that if we measure qsort in isolation, we'll see better
results, so we may still see a good win on index creation time as a
result of this work.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

#44Peter Geoghegan
peter@2ndquadrant.com
In reply to: Peter Geoghegan (#43)
Re: Inlining comparators as a performance optimisation

On 20 November 2011 03:29, Peter Geoghegan <peter@2ndquadrant.com> wrote:

./configure CFLAGS="-fno-inline -fno-inline-small-functions" (I could
have disabled more -02 optimisations, but this proved sufficient to
make my point)

I'll isolate this further to tuplesort.c soon, which ought to be a lot
more useful.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

#45Peter Geoghegan
peter@2ndquadrant.com
In reply to: Peter Geoghegan (#44)
Re: Inlining comparators as a performance optimisation

Part of my problem with not producing specialisations that I really
neglected to complain about until now is the inconsistency between
types, and the need to deal with that.

We benefit from assuming in our specialisation that we're only dealing
with POD types, simply not considering things that we would otherwise
have had to for the benefit of types that have a Datum representation
that is pass-by-reference, or have collations, or have multiple
scankeys. Leaving aside the question of straight compiler-optimisation
benefits for the moment, the remaining benefit of what I've done comes
not so much from avoiding the usual function call machinery per se, as
from doing so *as well as* cutting down on what currently happens in
comparetup_heap to handle every single compound and scalar type.
Compare the function comparetup_heap with my meta-function
TYPE##AppSort to see what I mean. The function comparetup_heap is the
comparator directly used by qsort_arg when sorting heap tuples, and
qsort_arg outsources to comparetup_heap some things that you might not
expect it to (very little has changed about qsort_arg since we lifted
it from NetBSD back in 2006). So while you might imagine that that
loop in comparetup_heap and things like its use of the heap_getattr
macros won't expand to that many instructions, you really don't want
them in the inner loop of a long operation like qsorting, paid for up
to O(n ^ 2) times. There's also the obvious implications for compiler
optimisations, particularly relating to effective usage of CPU cache.

I'm trying to get you what you asked for: A straight choice between
what Tom suggested and what I suggested, with perhaps some compromises
between the two positions. That's sort of tricky though, especially
considering the issues raised above.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

#46Robert Haas
robertmhaas@gmail.com
In reply to: Peter Geoghegan (#4)
Re: Inlining comparators as a performance optimisation

On Tue, Sep 20, 2011 at 7:53 PM, Peter Geoghegan <peter@2ndquadrant.com> wrote:

I don't think that the fact that that happens is at all significant at
this early stage, and it never even occurred to me that you'd think
that it might be. I was simply disclosing a quirk of this POC patch.
The workaround is probably to use a macro instead. For the benefit of
those that didn't follow the other threads, the macro-based qsort
implementation, which I found to perform significantly better than
regular qsort(), runs like this on my laptop when I built at 02 with
GCC 4.6 just now:

C stdlib quick-sort time elapsed: 2.092451 seconds
Inline quick-sort time elapsed: 1.587651 seconds

Results on my machine, for what they're worth:

[rhaas inline_compar_test]$ gcc -O0 qsort-inline-benchmark.c
[rhaas inline_compar_test]$ ./a.out
C stdlib quick-sort time elapsed: 2.366762 seconds
Inline quick-sort time elapsed: 1.807951 seconds
[rhaas inline_compar_test]$ gcc -O1 qsort-inline-benchmark.c
[rhaas inline_compar_test]$ ./a.out
C stdlib quick-sort time elapsed: 1.970473 seconds
Inline quick-sort time elapsed: 1.002765 seconds
[rhaas inline_compar_test]$ gcc -O2 qsort-inline-benchmark.c
[rhaas inline_compar_test]$ ./a.out
C stdlib quick-sort time elapsed: 1.966408 seconds
Inline quick-sort time elapsed: 0.958999 seconds
[rhaas inline_compar_test]$ gcc -O3 qsort-inline-benchmark.c
[rhaas inline_compar_test]$ ./a.out
C stdlib quick-sort time elapsed: 1.988693 seconds
Inline quick-sort time elapsed: 0.975090 seconds

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#47Peter Geoghegan
peter@2ndquadrant.com
In reply to: Robert Haas (#46)
1 attachment(s)
Re: Inlining comparators as a performance optimisation

On 21 November 2011 22:55, Robert Haas <robertmhaas@gmail.com> wrote:

Results on my machine, for what they're worth:

I was curious as to how much of an improvement I'd made to sorting in
isolation. I've adding simple sort timing instrumentation to the code
in a revised patch, attached, for the convenience of reviewers.

This patch adds an int8 specialisation, but it also supports fast path
sorting of timestamps by using that same int8 specialisation (for
HAVE_INT64_TIMESTAMP builds at least - didn't know if it was worth
handling the NaN crud for the other case). ISTM that various different
types with identical internal representations can share
specialisations like this, which I hope will alleviate some earlier
concerns about both the limited applicability of this optimisation and
the possible proliferation of specialisations.

Here's some raw qsort figures in seconds for the query "select * from
orderlines order by test" (where test is a timestamptz column I added
and updated with some data, executing repeatedly on the same machine
as before):

Before:

DEBUG: elapsed: 0.031738
DEBUG: elapsed: 0.031595
DEBUG: elapsed: 0.031502
DEBUG: elapsed: 0.031378
DEBUG: elapsed: 0.031359
DEBUG: elapsed: 0.031413
DEBUG: elapsed: 0.031499
DEBUG: elapsed: 0.031394
DEBUG: elapsed: 0.031368

After:

DEBUG: elapsed: 0.013341
DEBUG: elapsed: 0.014608
DEBUG: elapsed: 0.013809
DEBUG: elapsed: 0.013244
DEBUG: elapsed: 0.014307
DEBUG: elapsed: 0.013207
DEBUG: elapsed: 0.013227
DEBUG: elapsed: 0.013264
DEBUG: elapsed: 0.013143
DEBUG: elapsed: 0.013455
DEBUG: elapsed: 0.013447

I wonder, is it worth qualifying that the "Sort Method" was a
"quicksort (fast path)" sort within explain analyze output? This is a
rather large improvement, so It might actually be something that
people look out for, as it might be tricky to remember the exact
circumstances under which the optimisation kicks in by the time we're
done here.

I haven't had as much time as I'd like to polish this patch, or to get
clearer answers. I expect to spend more time on it over the weekend.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

Attachments:

0001-Initial-commit-of-optimization.patchtext/x-patch; charset=US-ASCII; name=0001-Initial-commit-of-optimization.patchDownload
From 687779e799eb0d0064f86192cc74ea473ff9a607 Mon Sep 17 00:00:00 2001
From: peter2ndQuadrant <peter@2ndquadrant.com>
Date: Sun, 20 Nov 2011 20:36:25 +0000
Subject: [PATCH] Initial commit of optimization

Stop directly using oid

Added int8 quicksort fast path specialisation, which can also be used in
place of F_TIMESTAMP_CMP for HAVE_INT64_TIMESTAMP builds.

Rebased, revised patch for -hackers, with timestamp and int8 fast path
sorting using the same int8 specialization.

Remove unneeded line

Rebase for -hackers
---
 src/backend/utils/sort/tuplesort.c     |  105 +++++++++++++++-
 src/include/utils/template_qsort_arg.h |  217 ++++++++++++++++++++++++++++++++
 2 files changed, 316 insertions(+), 6 deletions(-)
 create mode 100644 src/include/utils/template_qsort_arg.h

diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 3505236..4962973 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -107,11 +107,13 @@
 #include "miscadmin.h"
 #include "pg_trace.h"
 #include "utils/datum.h"
+#include "utils/fmgroids.h"
 #include "utils/logtape.h"
 #include "utils/lsyscache.h"
 #include "utils/memutils.h"
 #include "utils/pg_rusage.h"
 #include "utils/rel.h"
+#include "utils/template_qsort_arg.h"
 #include "utils/tuplesort.h"
 
 
@@ -1225,12 +1227,60 @@ puttuple_common(Tuplesortstate *state, SortTuple *tuple)
 }
 
 /*
- * All tuples have been provided; finish the sort.
+ * Manufacture type specific sorting specialisations
+ * with inline comparators
+ */
+static inline int
+compare_int4(Datum first, Datum second)
+{
+	int32		first_int = DatumGetInt32(first);
+	int32		second_int = DatumGetInt32(second);
+
+	if (first_int > second_int)
+		return 1;
+	else if (first_int == second_int)
+		return 0;
+	else
+		return -1;
+}
+
+static inline int
+compare_int8(Datum first, Datum second)
+{
+	int64		first_int = DatumGetInt64(first);
+	int64		second_int = DatumGetInt64(second);
+
+	if (first_int > second_int)
+		return 1;
+	else if (first_int == second_int)
+		return 0;
+	else
+		return -1;
+}
+
+TEMPLATE_QSORT_ARG(int4, compare_int4);
+TEMPLATE_QSORT_ARG(int8, compare_int8);
+
+double timeval_subtract(struct timeval *x, struct timeval *y);
+double timeval_subtract(struct timeval *x, struct timeval *y)
+{
+	struct timeval result;
+	/* Compute the time remaining to wait. tv_usec is certainly positive. */
+	result.tv_sec = x->tv_sec - y->tv_sec;
+	result.tv_usec = x->tv_usec - y->tv_usec;
+
+	/* return difference in seconds */
+	return result.tv_sec + ((double) result.tv_usec / 1000000);
+}
+
+/*
+ * All tuples have been provided; perform the sort.
  */
 void
 tuplesort_performsort(Tuplesortstate *state)
 {
 	MemoryContext oldcontext = MemoryContextSwitchTo(state->sortcontext);
+	struct timeval begin, end;
 
 #ifdef TRACE_SORT
 	if (trace_sort)
@@ -1247,11 +1297,51 @@ tuplesort_performsort(Tuplesortstate *state)
 			 * amount of memory.  Just qsort 'em and we're done.
 			 */
 			if (state->memtupcount > 1)
-				qsort_arg((void *) state->memtuples,
-						  state->memtupcount,
-						  sizeof(SortTuple),
-						  (qsort_arg_comparator) state->comparetup,
-						  (void *) state);
+			{
+				/*
+				 * Choose between various type-specific qsort variants.
+				 * Do so to avail of per-type compiler optimizations, in
+				 * particular, inlining of comparators.
+				 */
+				gettimeofday(&begin, NULL);
+				if (state->scanKeys && state->scanKeys->sk_func.fn_oid == F_BTINT4CMP &&
+					state->comparetup == &comparetup_heap)
+				{
+					int4_qsort_arg((void *) state->memtuples,
+								  state->memtupcount,
+								  sizeof(SortTuple),
+								  (void *) state);
+				}
+				else if (state->scanKeys &&
+					  (
+						  /* Some comparison that has an underlying int8 representation */
+						  state->scanKeys->sk_func.fn_oid == F_BTINT8CMP
+#ifdef HAVE_INT64_TIMESTAMP
+						  || state->scanKeys->sk_func.fn_oid == F_TIMESTAMP_CMP
+#endif
+					  )
+					&& state->comparetup == &comparetup_heap)
+				{
+					int8_qsort_arg((void *) state->memtuples,
+								  state->memtupcount,
+								  sizeof(SortTuple),
+								  (void *) state);
+				}
+				else
+				{
+					/*
+					 * Finally, fall back on type-neutral qsort with
+					 * function-pointer comparator
+					 */
+					qsort_arg((void *) state->memtuples,
+								  state->memtupcount,
+								  sizeof(SortTuple),
+								  (qsort_arg_comparator) state->comparetup,
+								  (void *) state);
+				}
+				gettimeofday(&end, NULL);
+				elog(DEBUG1, "elapsed: %f", timeval_subtract(&end, &begin));
+			}
 			state->current = 0;
 			state->eof_reached = false;
 			state->markpos_offset = 0;
@@ -2657,6 +2747,9 @@ myFunctionCall2Coll(FmgrInfo *flinfo, Oid collation, Datum arg1, Datum arg2)
  * and return a 3-way comparison result.  This takes care of handling
  * reverse-sort and NULLs-ordering properly.  We assume that DESC and
  * NULLS_FIRST options are encoded in sk_flags the same way btree does it.
+ *
+ * Note that this logic is more-or-less duplicated in template_qsort_arg.h,
+ * and "instantiated" per-type there.
  */
 static inline int32
 inlineApplySortFunction(FmgrInfo *sortFunction, int sk_flags, Oid collation,
diff --git a/src/include/utils/template_qsort_arg.h b/src/include/utils/template_qsort_arg.h
new file mode 100644
index 0000000..41f8237
--- /dev/null
+++ b/src/include/utils/template_qsort_arg.h
@@ -0,0 +1,217 @@
+/*-------------------------------------------------------------------------
+ *	template_qsort_arg.h: "template" version of qsort_arg.c
+ *
+ *	This version of qsort_arg is exclusively used within tuplesort.c to
+ *	more efficiently sort common types such as integers and floats. In 
+ *	providing this version, we seek to take advantage of compile-time 
+ *	optimisations for specific types, in particular, the inlining of 
+ *	comparators, as well as reducing the overhead of comparisons that
+ *	the general SQL-callable-function API imposes.
+ *
+ *	It is assumed that any types that use this infrastructure have at 
+ *	most one element in their ScanKeys array.
+ *
+ *	CAUTION: if you change this file, see also qsort_arg.c as well as 
+ *	qsort.c. qsort_arg.c should be considered authoratative.
+ *
+ *	Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group
+ *	Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *	src/include/utils/template_qsort_arg.h
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+
+#define swapcode(TYPE, parmi, parmj, n) 	\
+do {						\
+	size_t i = (n) / sizeof (TYPE);		\
+	TYPE *pi = (TYPE *)(void *)(parmi);	\
+	TYPE *pj = (TYPE *)(void *)(parmj);	\
+	do {					\
+		TYPE	t = *pi;		\
+		*pi++ = *pj;			\
+		*pj++ = t;			\
+		} while (--i > 0);		\
+} while (0)
+
+#define SWAPINIT(a, es) swaptype = ((char *)(a) - (char *)0) % sizeof(long) || \
+	(es) % sizeof(long) ? 2 : (es) == sizeof(long)? 0 : 1;
+
+#define thistype_vecswap(TYPE, a, b, n) if ((n) > 0) TYPE##_swapfunc((a), (b), (size_t)(n), swaptype) 	
+
+#define thistype_swap(TYPE, a, b)				\
+if (swaptype == 0) {						\
+		long t = *(long *)(void *)(a);			\
+		*(long *)(void *)(a) = *(long *)(void *)(b);	\
+		*(long *)(void *)(b) = t;			\
+	} else							\
+		TYPE##_swapfunc(a, b, es, swaptype)	
+
+/* 
+ * This macro manufactures a type-specific implementation of qsort_arg with 
+ * the comparator, COMPAR, known at compile time. COMPAR is typically an
+ * inline function.
+ *
+ * COMPAR should take as its arguments two Datums, and return an int, in
+ * line with standard qsort convention.
+ * 
+ * We have void* parameters for TYPE##AppSort just to shut up the compiler. 
+ * They could be SortTuple pointers instead, but that would make it more 
+ * difficult to keep template_qsort_arg.h consistent with qsort_arg.c. 		
+ *
+ */
+
+#define TEMPLATE_QSORT_ARG(TYPE, COMPAR)				\
+void TYPE##_qsort_arg(void *a, size_t n, size_t es, void *arg);		\
+												\
+static inline int32										\
+TYPE##AppSort(const void *a, const void *b, Tuplesortstate *state)				\
+{												\
+	int32		compare;								\
+	const SortTuple* aT = a;								\
+	const SortTuple* bT = b;								\
+	/* Allow interrupting long sorts */							\
+	CHECK_FOR_INTERRUPTS();									\
+												\
+	Assert(state->nKeys == 1);								\
+	Assert(state->scanKeys);								\
+												\
+	if ( aT->isnull1)									\
+	{											\
+		if (bT->isnull1)								\
+			compare = 0;		/* NULL "=" NULL */				\
+		else if (state->scanKeys->sk_flags & SK_BT_NULLS_FIRST)				\
+			compare = -1;		/* NULL "<" NOT_NULL */				\
+		else										\
+			compare = 1;		/* NULL ">" NOT_NULL */				\
+	}											\
+	else if (bT->isnull1)									\
+	{											\
+		if (state->scanKeys->sk_flags & SK_BT_NULLS_FIRST)				\
+			compare = 1;		/* NOT_NULL ">" NULL */				\
+		else										\
+			compare = -1;		/* NOT_NULL "<" NULL */				\
+	}											\
+	else											\
+	{											\
+		compare = COMPAR(aT->datum1, bT->datum1);					\
+												\
+		if (state->scanKeys->sk_flags & SK_BT_DESC)					\
+			compare = -compare;							\
+	}											\
+												\
+	return compare;										\
+}												\
+									\
+static inline void							\
+TYPE##_swapfunc(char *a, char *b, size_t n, int swaptype)		\
+{									\
+	if (swaptype <= 1)						\
+		swapcode(long, a, b, n); 				\
+	else								\
+		swapcode(char, a, b, n);				\
+}									\
+									\
+static inline char *										\
+TYPE##_med3(char *a, char *b, char *c, void *arg)						\
+{												\
+	return TYPE##AppSort(a, b, arg) < 0 ?							\
+		(TYPE##AppSort(b, c, arg) < 0 ? b : (TYPE##AppSort(a, c, arg) < 0 ? c : a))	\
+		: (TYPE##AppSort(b, c, arg) > 0 ? b : (TYPE##AppSort(a, c, arg) < 0 ? a : c));	\
+}												\
+												\
+void 												\
+TYPE##_qsort_arg(void *a, size_t n, size_t es, void *arg)					\
+{												\
+	char	   *pa,										\
+			   *pb,									\
+			   *pc,									\
+			   *pd,									\
+			   *pl,									\
+			   *pm,									\
+			   *pn;									\
+	int			d,								\
+				r,								\
+				swaptype,							\
+				presorted;							\
+												\
+loop:SWAPINIT(a, es); 										\
+	if (n < 7)										\
+	{											\
+		for (pm = (char *) a + es; pm < (char *) a + n * es; pm += es)			\
+			for (pl = pm; pl > (char *) a && TYPE##AppSort(pl - es, pl, arg) > 0;	\
+				 pl -= es)							\
+				thistype_swap(TYPE, pl, pl - es);				\
+		return;										\
+	}											\
+	presorted = 1;										\
+	for (pm = (char *) a + es; pm < (char *) a + n * es; pm += es)				\
+	{											\
+		if (TYPE##AppSort(pm - es, pm, arg) > 0)					\
+		{										\
+			presorted = 0;								\
+			break;									\
+		}										\
+	}											\
+	if (presorted)										\
+		return;										\
+	pm = (char *) a + (n / 2) * es;								\
+	if (n > 7)										\
+	{											\
+		pl = (char *) a;								\
+		pn = (char *) a + (n - 1) * es;							\
+		if (n > 40)									\
+		{										\
+			d = (n / 8) * es;							\
+			pl = TYPE##_med3(pl, pl + d, pl + 2 * d, arg);				\
+			pm = TYPE##_med3(pm - d, pm, pm + d, arg);				\
+			pn = TYPE##_med3(pn - 2 * d, pn - d, pn, arg);				\
+		}										\
+		pm = TYPE##_med3(pl, pm, pn, arg);						\
+	}											\
+	thistype_swap(TYPE, a, pm);								\
+	pa = pb = (char *) a + es;								\
+	pc = pd = (char *) a + (n - 1) * es;							\
+	for (;;)										\
+	{											\
+		while (pb <= pc && (r = TYPE##AppSort(pb, a, arg)) <= 0)			\
+		{										\
+			if (r == 0)								\
+			{									\
+				thistype_swap(TYPE, pa, pb);					\
+				pa += es;							\
+			}									\
+			pb += es;								\
+		}										\
+		while (pb <= pc && (r = TYPE##AppSort(pc, a, arg)) >= 0)			\
+		{										\
+			if (r == 0)								\
+			{									\
+				thistype_swap(TYPE, pc, pd);					\
+				pd -= es;							\
+			}									\
+			pc -= es;								\
+		}										\
+		if (pb > pc)									\
+			break;									\
+		thistype_swap(TYPE, pb, pc);							\
+		pb += es;									\
+		pc -= es;									\
+	}											\
+	pn = (char *) a + n * es;								\
+	r = Min(pa - (char *) a, pb - pa);							\
+	thistype_vecswap(TYPE, a, pb - r, r);							\
+	r = Min(pd - pc, pn - pd - es);								\
+	thistype_vecswap(TYPE, pb, pn - r, r);							\
+	if ((r = pb - pa) > es)									\
+		TYPE##_qsort_arg(a, r / es, es, arg);						\
+	if ((r = pd - pc) > es)									\
+	{											\
+		/* Iterate rather than recurse to save stack space */				\
+		a = pn - r;									\
+		n = r / es;									\
+		goto loop;									\
+	}											\
+}												\
+
-- 
1.7.7.3

#48Simon Riggs
simon@2ndQuadrant.com
In reply to: Tom Lane (#39)
Re: Inlining comparators as a performance optimisation

On Fri, Nov 18, 2011 at 2:11 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Simon Riggs <simon@2ndQuadrant.com> writes:

On Fri, Nov 18, 2011 at 5:20 AM, Robert Haas <robertmhaas@gmail.com> wrote:

I think that we should really consider doing with this patch what Tom
suggested upthread; namely, looking for a mechanism to allow
individual datatypes to offer up a comparator function that doesn't
require bouncing through FunctionCall2Coll().

I don't think its credible to implement that kind of generic
improvement at this stage of the release cycle.

Er, *what*?  We're in mid development cycle, we are nowhere near
release.  When exactly would you have us make major changes?

In any case, what I understood Robert to be proposing was an add-on
feature that could be implemented in one datatype at a time.  Not
a global flag day.  We couldn't really do the latter anyway without
making life very unpleasant for authors of extension datatypes.

Tom, whenever you think I've said something you really disagree with,
just assume there's a misunderstanding. Like here.

Of course it is OK to make such changes at this time.

Given we have <2 months to the last CF of this release, inventing a
generic infrastructure is unlikely to be finished and complete in this
dev cycle, so requesting that isn't a practical suggestion, IMHO.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

#49Robert Haas
robertmhaas@gmail.com
In reply to: Peter Geoghegan (#47)
Re: Inlining comparators as a performance optimisation

On Tue, Nov 22, 2011 at 8:09 PM, Peter Geoghegan <peter@2ndquadrant.com> wrote:

I wonder, is it worth qualifying that the "Sort Method" was a
"quicksort (fast path)" sort within explain analyze output? This is a
rather large improvement, so It might actually be something that
people look out for, as it might be tricky to remember the exact
circumstances under which the optimisation kicks in by the time we're
done here.

Well, right now the decision as to which mechanism should be used here
gets made in tuplesort_performsort(), which has no good way of
communicating with EXPLAIN anyway. Actually, I think that's a
modularity violation; using the address of comparetup_heap as a flag
value seems quite ugly. How about moving that logic up to
tuplesort_begin_heap() and having it set some state inside the
Tuplesort, maybe based on a flag in the opclass (or would it have to
attach to the individual operator)?

At least on my machine, your latest patch reliably crashes the
regression tests in multiple places.

The following test case also crashes them for me (perhaps for the same
reason the regression tests crash):

create table i4 (a int, b int);
insert into i4 values (4, 1), (2, 1), (0, 1), (null, 1), (-2, 1), (-7,
1), (4, 2), (4, 3), (4, 4);
select * from i4 order by 1, 2;
TRAP: FailedAssertion("!(state->nKeys == 1)", File: "tuplesort.c", Line: 1261);

The formatting of src/include/utils/template_qsort_arg.h is hard to
read. At ts=8, the backslashes line up, but the code doesn't fit in
80 columns. If you set ts=4, then it fits in 80 columns, but the
backslashes don't line up any more, and the variable declarations
don't either. I believe ts=4 is project standard.

I still think it would be a good idea to provide a mechanism to
override heap_comparetup() with a type-specific function. I don't
think that would take much extra code, and then any data type could
get at least that much benefit out of this.

It seems like it could be a good idea to do some
per-assembler-instruction profiling of this code, and perhaps also of
the original code. I'm curious where the time is being spent.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#50Peter Geoghegan
peter@2ndquadrant.com
In reply to: Robert Haas (#49)
1 attachment(s)
Re: Inlining comparators as a performance optimisation

On 23 November 2011 19:24, Robert Haas <robertmhaas@gmail.com> wrote:

Well, right now the decision as to which mechanism should be used here
gets made in tuplesort_performsort(), which has no good way of
communicating with EXPLAIN anyway.

You could pretty easily add something to Tuplesortstate to accomplish
this. That isn't an endorsement of doing so, but I'm not sure that it
isn't appropriate.

Actually, I think that's a
modularity violation; using the address of comparetup_heap as a flag
value seems quite ugly. How about moving that logic up to
tuplesort_begin_heap()

I'll post a patch soon that does just that in the next day or two.
Tuplesortstate has a pointer to a sort specialisation.

and having it set some state inside the
Tuplesort, maybe based on a flag in the opclass (or would it have to
attach to the individual operator)?

I'm not sure that there's much point in such a flag.

At least on my machine, your latest patch reliably crashes the
regression tests in multiple places.

TRAP: FailedAssertion("!(state->nKeys == 1)", File: "tuplesort.c", Line: 1261);

Yes, sorry about that. Should have been discriminating against nKeys > 1 cases.

As of this evening, for sorts with multiple scankeys, I'm using
optimisations for the first scankey but not subsequent scankeys, which
is frequently almost as good as having optimisations for all scanKeys.

The formatting of src/include/utils/template_qsort_arg.h is hard to
read.  At ts=8, the backslashes line up, but the code doesn't fit in
80 columns.  If you set ts=4, then it fits in 80 columns, but the
backslashes don't line up any more, and the variable declarations
don't either.  I believe ts=4 is project standard.

Fair enough. My working copy and .vimrc have been updated.

I still think it would be a good idea to provide a mechanism to
override heap_comparetup() with a type-specific function.  I don't
think that would take much extra code, and then any data type could
get at least that much benefit out of this.

It seems like it could be a good idea to do some
per-assembler-instruction profiling of this code, and perhaps also of
the original code.  I'm curious where the time is being spent.

How would you go about doing that? The instrumentation that profilers
use actually caused a big drop in performance here when I attempted it
a few weeks ago. There's a kind of Heisenberg effect.

This optimisation *more than doubles* raw sort performance for the
cases. There is nothing contrived or cherry picked about the query
that I selected to represent this optimisation - it was literally the
first one that I selected.

Sometimes, I see even a markedly better gain than a doubling of raw
sort performance - I think my earlier experiments that indicated a
much smaller improvement past a certain point may have been
methodologically flawed. Sorry about that.

If I double-up the data in the orderlines table a few times, until it
reaches 385 MB (duplicate ever tuple with an insert into ...select ),
then warm the cache, I get very interesting results. Here, we see a
few runs of the same old query unoptimised (note that I've excluded
some cold-cache runs before these runs):

Before optimisation
==============
Total runtime: 7785.473 ms - 3.517310 secs just sorting
Total runtime: 8203.533 ms - 3.577193 secs just sorting
Total runtime: 8559.743 ms - 3.892719 secs just sorting

Total runtime: 9032.564 ms - 3.844746 secs just sorting

Total runtime: 9637.179 ms - 4.434431 secs just sorting
Total runtime: 9647.215 ms - 4.440560 secs just sorting
Total runtime: 9669.701 ms - 4.448572 secs just sorting

After optimisation
==============
Total runtime: 5462.419 ms - 1.169963 secs just sorting
Total runtime: 5510.660 ms - 1.234393 secs just sorting
Total runtime: 5511.703 ms - 1.208377 secs just sorting

Total runtime: 5588.604 ms - 1.175536 secs just sorting

Total runtime: 5899.496 ms - 1.250403 secs just sorting
Total runtime: 6023.132 ms - 1.338760 secs just sorting
Total runtime: 6717.177 ms - 1.486602 secs just sorting

This is a 800109kB sort.

So, taking the median value as representative here, that looks to be
just shy of a 40% improvement, or 3.4 seconds. My /proc/cpuinfo is
attached on the off chance that someone is interested in that. More
work is needed here, but this seems promising.

It will be interesting to see how far all of this can be taken with
comparetup_index_btree. Certainly, I'm sure there's some gain to be
had there by applying lessons learned from comparetup_heap.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

Attachments:

cpuinfo.txttext/plain; charset=US-ASCII; name=cpuinfo.txtDownload
#51Peter Geoghegan
peter@2ndquadrant.com
In reply to: Peter Geoghegan (#50)
3 attachment(s)
Re: Inlining comparators as a performance optimisation

3 items are attached:

1. A spreadsheet, results.ods, which has results for automated
multiple runs of the "doubled up" dellstore orderlines queries (where
orderlines is 385 MB). Results are sorted, with the median (or the
lower middle value, didn't get the mean of the two middle runs) result
for each run highlighted as representative. There are 3 builds of
Postgres (HEAD, not inlined, optimized), each on its own sheet in the
spreadsheet. The cache was warmed for each query, and subsequently the
query was run 20 times.

2. A python script I hacked together that should run on anything
around Python 2.5+, with a single dependency, psycopg2. Look at --help
to see how it works. This is the script that actually generated the
figures that went into the spreadsheet, by being run once for each
type of build. You can fairly easily play along at home with this, for
these and other queries. It will spit out CSV files. It is designed to
warm the cache (by default 3 times before each query) to eliminate
caching effects. You can add your own query to the python list to have
it run by the script, to generate the same set of figures for that
query. I'm rather curious as to how much of an impact this
optimisation will have on queries with unique nodes, joins and
grouping when they rely on a sort node for their input, in the real
world. Certainly, a query need not have an ORDER BY clause to see
benefits, perhaps substantial benefits. The logic to parse sort node
details is rather unsophisticated right now though, due to my focus on
the obvious ORDER BY case.

3. The revision of the patch that was actually tested, now with
inlining specialisations for the single scanKey case, and a
non-inlined specialisation for multiple scanKeys where we can still
benefit from eliding the SQL function call machinery for the first
scanKey, which is often almost as useful as being able to do so for
all scanKeys. It also selects a sorting specialisation when it can in
tuplesort_begin_heap, per Robert's suggestion.

Reviewers will want to comment out line 731 of tuplesort.c, "#define
optimize", to quickly get unoptimized behaviour for comparative
purposes. Furthermore, if you'd like to see what this would look like
without inlining, you can simply comment out assignments of inline
variants of specialisations (i.e. int4inlqsort_arg and
int8inlqsort_arg) in tuplesort_begin_heap.

It has been suggested that I'm chasing diminishing returns by
inlining, as I go further for a smaller benefit. Of course, that's
true. However, I believe that I'm not chasing them past the point
where that ceases to make sense, and these figures support that
contention - chasing diminishing returns in the nature of this kind of
work. Here, the additional benefit of inlining accounts for over an
entire second shaved off a query that was originally 7056ms, so that's
not to be sniffed at.

I'll reiterate that qsort_arg has only been modified 4 times after its
initial commit in 2006, and these were all trivial changes. While the
way that I ape generic programming with the preprocessor is on the
ugly side, what I've done can be much more easily understood with
reference to qsort_arg itself. Robert said that duplicating the sort
function was "iffy". However, that already happened long ago, as we're
already maintaining both qsort_arg.c and qsort.c, and comments already
urge maintainers to keep the two consistent. That's not been a problem
though, because, as I've said, there has never been any call to make
substantive changes to either in all those years.

We could probably further reduce the code footprint of all of this by
having template_qsort_arg.h generate comparators directly. That might
be inflexible in a way that turns out to matter though, like if we
wanted to use this for non-HAVE_INT64_TIMESTAMP timestamps and had to
add NaN crud.

My next step is to see how this goes with hundreds of sorts in the
hundreds of megabytes on a high-end server. I don't have immediate
access to one, but I'm working that out.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

Attachments:

0001-Initial-commit-of-optimization.patchtext/x-patch; charset=US-ASCII; name=0001-Initial-commit-of-optimization.patchDownload
From 92185762297ba9819eb7f1b208db03663d9fdfa8 Mon Sep 17 00:00:00 2001
From: peter2ndQuadrant <peter@2ndquadrant.com>
Date: Sun, 20 Nov 2011 20:36:25 +0000
Subject: [PATCH] Initial commit of optimization

Stop directly using oid

Added int8 quicksort fast path specialisation, which can also be used in
place of F_TIMESTAMP_CMP for HAVE_INT64_TIMESTAMP builds.

Rebased, revised patch for -hackers, with timestamp and int8 fast path
sorting using the same int8 specialization.

Remove unneeded line

Rebase for -hackers

Descriminate against cases where nKeys > 1 when selecting sort
specialisation.

We now support fast path sorting with multiple scanKeys.

We now inline for one scanKey, but do not inline our comparetup_heap
replacement when multiple scanKeys are used (although this is at the
compiler's discretion). However, when multiple scanKeys are used, we
still use a specialisation that will elide the SQL-function-call
machinery for the first scanKey, which is almost as good for most cases.

Further formatting clean-ups

Move specialization selection logic to tuplesort_begin_heap(), per
suggestion of Robert Haas

Add simple switch for testing optimization.

Improvements to template qsort_arg comments. Support fast path sorting
of dates.

Added float fast path sorting

Patch mimimization.

Additional clean-up.
---
 src/backend/utils/sort/tuplesort.c     |  156 ++++++++++++++++-
 src/include/utils/template_qsort_arg.h |  290 ++++++++++++++++++++++++++++++++
 src/port/qsort.c                       |    3 +-
 3 files changed, 440 insertions(+), 9 deletions(-)
 create mode 100644 src/include/utils/template_qsort_arg.h

diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 3505236..eb36e17 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -107,11 +107,13 @@
 #include "miscadmin.h"
 #include "pg_trace.h"
 #include "utils/datum.h"
+#include "utils/fmgroids.h"
 #include "utils/logtape.h"
 #include "utils/lsyscache.h"
 #include "utils/memutils.h"
 #include "utils/pg_rusage.h"
 #include "utils/rel.h"
+#include "utils/template_qsort_arg.h"
 #include "utils/tuplesort.h"
 
 
@@ -226,6 +228,13 @@ struct Tuplesortstate
 										   Tuplesortstate *state);
 
 	/*
+	 * This specialization function pointer is sometimes used as an alternative
+	 * to the standard qsort_arg, when it has been determined that we can 
+	 * benefit from various per-type performance optimizations.
+	 */
+	void (*qsort_arg_spec)(void *a, size_t n, size_t es, void *arg);
+
+	/*
 	 * Function to copy a supplied input tuple into palloc'd space and set up
 	 * its SortTuple representation (ie, set tuple/datum1/isnull1).  Also,
 	 * state->availMem must be decreased by the amount of space used for the
@@ -457,6 +466,11 @@ static void tuplesort_heap_insert(Tuplesortstate *state, SortTuple *tuple,
 static void tuplesort_heap_siftup(Tuplesortstate *state, bool checkIndex);
 static unsigned int getlen(Tuplesortstate *state, int tapenum, bool eofOK);
 static void markrunend(Tuplesortstate *state, int tapenum);
+static inline int32 inlineApplySortFunction(FmgrInfo *sortFunction, 
+						int sk_flags, Oid collation,
+						Datum datum1, bool isNull1,
+						Datum datum2, bool isNull2);
+
 static int comparetup_heap(const SortTuple *a, const SortTuple *b,
 				Tuplesortstate *state);
 static void copytup_heap(Tuplesortstate *state, SortTuple *stup, void *tup);
@@ -576,6 +590,71 @@ tuplesort_begin_common(int workMem, bool randomAccess)
 	return state;
 }
 
+/*
+ * Manufacture type specific sorting specialisations
+ * with inline comparators
+ */
+static inline int
+compare_int4(Datum first, Datum second)
+{
+	int32           first_int = DatumGetInt32(first);
+	int32           second_int = DatumGetInt32(second);
+	if (first_int > second_int)
+		return 1;
+	else if (first_int == second_int)
+		return 0;
+	else
+		return -1;
+}
+
+static inline int
+compare_int8(Datum first, Datum second)
+{
+	int64           first_int = DatumGetInt64(first);
+	int64           second_int = DatumGetInt64(second);
+
+	if (first_int > second_int)
+		return 1;
+	else if (first_int == second_int)
+		return 0;
+	else
+		return -1;
+}
+
+static inline int
+compare_float4(Datum first, Datum second)
+{
+	float4 f = DatumGetInt64(first);
+	float4 s = DatumGetInt64(second);
+
+
+	if (f > s)
+		return 1;
+	else if (f == s)
+		return 0;
+	else
+		return -1;
+}
+
+static inline int
+compare_float8(Datum first, Datum second)
+{
+	float8 f = DatumGetInt64(first);
+	float8 s = DatumGetInt64(second);
+
+	if (f > s)
+		return 1;
+	else if (f == s)
+		return 0;
+	else
+		return -1;
+}
+
+TEMPLATE_QSORT_ARG(int4, compare_int4);
+TEMPLATE_QSORT_ARG(int8, compare_int8);
+TEMPLATE_QSORT_ARG(float4, compare_float4);
+TEMPLATE_QSORT_ARG(float8, compare_float8);
+
 Tuplesortstate *
 tuplesort_begin_heap(TupleDesc tupDesc,
 					 int nkeys, AttrNumber *attNums,
@@ -649,7 +728,43 @@ tuplesort_begin_heap(TupleDesc tupDesc,
 							   sortFunction,
 							   (Datum) 0);
 	}
-
+#define optimize
+
+#ifdef optimize
+	/* Select a qsort specialization, if possible */
+	if (state->scanKeys &&
+		  /* Some comparison that has an underlying int4 representation */
+		(
+		  state->scanKeys->sk_func.fn_oid == F_BTINT4CMP
+			|| state->scanKeys->sk_func.fn_oid == F_DATE_CMP
+		)
+		)
+	{
+		if (state->nKeys == 1)
+			state->qsort_arg_spec = int4inlqsort_arg;
+		else
+			state->qsort_arg_spec = int4regqsort_arg;
+	}
+	else if (state->scanKeys &&
+			  /* Some comparison that has an underlying int8 representation */
+			(
+			  state->scanKeys->sk_func.fn_oid == F_BTINT8CMP
+#ifdef HAVE_INT64_TIMESTAMP
+			  || state->scanKeys->sk_func.fn_oid == F_TIMESTAMP_CMP
+#endif
+		
+			))
+	{
+		if (state->nKeys == 1)
+			state->qsort_arg_spec = int8inlqsort_arg;
+		else
+			state->qsort_arg_spec = int8regqsort_arg;
+	}
+	else
+	{
+		state->qsort_arg_spec = NULL;
+	}
+#endif
 	MemoryContextSwitchTo(oldcontext);
 
 	return state;
@@ -1225,7 +1340,7 @@ puttuple_common(Tuplesortstate *state, SortTuple *tuple)
 }
 
 /*
- * All tuples have been provided; finish the sort.
+ * All tuples have been provided; perform the sort.
  */
 void
 tuplesort_performsort(Tuplesortstate *state)
@@ -1247,11 +1362,28 @@ tuplesort_performsort(Tuplesortstate *state)
 			 * amount of memory.  Just qsort 'em and we're done.
 			 */
 			if (state->memtupcount > 1)
-				qsort_arg((void *) state->memtuples,
-						  state->memtupcount,
-						  sizeof(SortTuple),
-						  (qsort_arg_comparator) state->comparetup,
-						  (void *) state);
+			{
+				/* Consider a sorting specialization */
+				if (state->qsort_arg_spec)
+				{
+					state->qsort_arg_spec((void *) state->memtuples,
+								  state->memtupcount,
+								  sizeof(SortTuple),
+								  (void *) state);
+				}
+				else
+				{
+					/*
+					 * Fall back on type-neutral qsort with
+					 * function-pointer comparator
+					 */
+					qsort_arg((void *) state->memtuples,
+								  state->memtupcount,
+								  sizeof(SortTuple),
+								  (qsort_arg_comparator) state->comparetup,
+								  (void *) state);
+				}
+			}
 			state->current = 0;
 			state->eof_reached = false;
 			state->markpos_offset = 0;
@@ -2657,6 +2789,9 @@ myFunctionCall2Coll(FmgrInfo *flinfo, Oid collation, Datum arg1, Datum arg2)
  * and return a 3-way comparison result.  This takes care of handling
  * reverse-sort and NULLs-ordering properly.  We assume that DESC and
  * NULLS_FIRST options are encoded in sk_flags the same way btree does it.
+ *
+ * Note that this logic is more-or-less duplicated in template_qsort_arg.h,
+ * and "instantiated" per-type there.
  */
 static inline int32
 inlineApplySortFunction(FmgrInfo *sortFunction, int sk_flags, Oid collation,
@@ -2708,10 +2843,15 @@ ApplySortFunction(FmgrInfo *sortFunction, int sortFlags, Oid collation,
 }
 
 
+
+
 /*
  * Routines specialized for HeapTuple (actually MinimalTuple) case
+ *
+ * N.B. : Some of this code is partially duplicated for the purposes 
+ * of heap comparison specializations. Take care to keep the code
+ * within template_qsort_arg.h consistent with this code.
  */
-
 static int
 comparetup_heap(const SortTuple *a, const SortTuple *b, Tuplesortstate *state)
 {
diff --git a/src/include/utils/template_qsort_arg.h b/src/include/utils/template_qsort_arg.h
new file mode 100644
index 0000000..3554707
--- /dev/null
+++ b/src/include/utils/template_qsort_arg.h
@@ -0,0 +1,290 @@
+/*-------------------------------------------------------------------------
+ *	template_qsort_arg.h: "template" version of qsort_arg.c
+ *
+ *	This version of qsort_arg is exclusively used within tuplesort.c to
+ *	more efficiently sort common types such as integers and floats. In
+ *	providing this version, we seek to take advantage of compile-time
+ *	optimisations for specific types, in particular, the inlining of
+ *	comparators, as well as reducing the overhead of comparisons that
+ *	the general SQL-callable-function API imposes.
+ *
+ *  The TEMPLATE_QSORT_ARG() macro generates an inlining variant (for
+ *  clients with a single scanKey) and non-inlining variant for sorts
+ *  with multiple scanKeys.
+ *
+ *  We rely on the availability of various functions within tuplesort.c,
+ *  and indeed we partially duplicate some code from there, so this file
+ *  should be considered private to that module, rather than a generic
+ *  piece of infrastructure.
+ *
+ *	CAUTION: if you change this file, see also qsort_arg.c as well as
+ *	qsort.c. qsort_arg.c should be considered authoratative.
+ *
+ *	Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group
+ *	Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *	src/include/utils/template_qsort_arg.h
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+
+#define swapcode(TYPE, parmi, parmj, n) \
+do {									\
+	size_t i = (n) / sizeof (TYPE);		\
+	TYPE *pi = (TYPE *)(void *)(parmi);	\
+	TYPE *pj = (TYPE *)(void *)(parmj);	\
+	do {								\
+		TYPE	t = *pi;				\
+		*pi++ = *pj;					\
+		*pj++ = t;						\
+		} while (--i > 0);				\
+} while (0)
+
+#define SWAPINIT(a, es) swaptype = ((char *)(a) - (char *)0) % sizeof(long) || \
+	(es) % sizeof(long) ? 2 : (es) == sizeof(long)? 0 : 1;
+
+#define thistype_vecswap(TYPE, a, b, n, SPEC_VAR)							\
+	if ((n) > 0) TYPE##SPEC_VAR##swapfunc((a), (b), (size_t)(n), swaptype) 	
+
+#define thistype_swap(TYPE, a, b, SPEC_VAR)			\
+if (swaptype == 0) {								\
+		long t = *(long *)(void *)(a);				\
+		*(long *)(void *)(a) = *(long *)(void *)(b);\
+		*(long *)(void *)(b) = t;					\
+	} else											\
+		TYPE##SPEC_VAR##swapfunc(a, b, es, swaptype)	
+
+/* 
+ * This macro manufactures a type-specific implementation of qsort_arg with 
+ * the comparator, COMPAR, known at compile time. COMPAR is typically an
+ * inline function.
+ *
+ * COMPAR should take as its arguments two Datums, and return an int, in
+ * line with standard qsort convention.
+ * 
+ * We have void* parameters for TYPE##AppSort just to shut up the compiler. 
+ * They could be SortTuple pointers instead, but that would make it more 
+ * difficult to keep template_qsort_arg.h consistent with tuplesort.c. 		
+ *
+ */
+
+#define DO_TEMPLATE_QSORT_ARG(TYPE, COMPAR, SPEC_VAR, REG_ADDITIONAL_CODE)			\
+void TYPE##SPEC_VAR##qsort_arg(void *a, size_t n, size_t es, void *arg);			\
+																					\
+static inline int32																	\
+TYPE##SPEC_VAR##AppSort(const void *a, const void *b, Tuplesortstate *state)		\
+{																					\
+	int32		compare;															\
+	const SortTuple* aT = a;														\
+	const SortTuple* bT = b;														\
+																					\
+	/* Allow interrupting long sorts */												\
+	CHECK_FOR_INTERRUPTS();															\
+																					\
+	Assert(state->scanKeys);														\
+																					\
+	if ( aT->isnull1)																\
+	{																				\
+		if (bT->isnull1)															\
+			compare = 0;		/* NULL "=" NULL */									\
+		else if (state->scanKeys->sk_flags & SK_BT_NULLS_FIRST)						\
+			compare = -1;		/* NULL "<" NOT_NULL */								\
+		else																		\
+			compare = 1;		/* NULL ">" NOT_NULL */								\
+	}																				\
+	else if (bT->isnull1)															\
+	{																				\
+		if (state->scanKeys->sk_flags & SK_BT_NULLS_FIRST)							\
+			compare = 1;		/* NOT_NULL ">" NULL */								\
+		else																		\
+			compare = -1;		/* NOT_NULL "<" NULL */								\
+	}																				\
+	else																			\
+	{																				\
+		compare = COMPAR(aT->datum1, bT->datum1);									\
+																					\
+		if (state->scanKeys->sk_flags & SK_BT_DESC)									\
+			compare = -compare;														\
+	}																				\
+	if (compare != 0) 																\
+		return compare;																\
+	/* Additional code for variants where we have more than one scanKey */ 			\
+	REG_ADDITIONAL_CODE 															\
+	return 0;																		\
+}																					\
+																					\
+static inline void													\
+TYPE##SPEC_VAR##swapfunc(char *a, char *b, size_t n, int swaptype)	\
+{																	\
+	if (swaptype <= 1)												\
+		swapcode(long, a, b, n); 									\
+	else															\
+		swapcode(char, a, b, n);									\
+}																	\
+																	\
+static inline char *														\
+TYPE##SPEC_VAR##med3(char *a, char *b, char *c, void *arg)					\
+{																			\
+	return TYPE##SPEC_VAR##AppSort(a, b, arg) < 0 ?							\
+		(TYPE##SPEC_VAR##AppSort(b, c, arg) < 0 ? 							\
+					b : (TYPE##SPEC_VAR##AppSort(a, c, arg) < 0 ? c : a))	\
+		: (TYPE##SPEC_VAR##AppSort(b, c, arg) > 0 ? 						\
+					b : (TYPE##SPEC_VAR##AppSort(a, c, arg) < 0 ? a : c));	\
+}																			\
+																			\
+void 																	\
+TYPE##SPEC_VAR##qsort_arg(void *a, size_t n, size_t es, void *arg)		\
+{																		\
+	char	   *pa,														\
+			   *pb,														\
+			   *pc,														\
+			   *pd,														\
+			   *pl,														\
+			   *pm,														\
+			   *pn;														\
+	int			d,														\
+				r,														\
+				swaptype,												\
+				presorted;												\
+																		\
+loop:SWAPINIT(a, es); 													\
+	if (n < 7)															\
+	{																	\
+		for (pm = (char *) a + es; pm < (char *) a + n * es; pm += es)	\
+			for (pl = pm; pl > (char *) a && 							\
+					TYPE##SPEC_VAR##AppSort(pl - es, pl, arg) > 0;		\
+				 pl -= es)												\
+				thistype_swap(TYPE, pl, pl - es, SPEC_VAR);				\
+		return;															\
+	}																	\
+	presorted = 1;														\
+	for (pm = (char *) a + es; pm < (char *) a + n * es; pm += es)		\
+	{																	\
+		if (TYPE##SPEC_VAR##AppSort(pm - es, pm, arg) > 0)				\
+		{																\
+			presorted = 0;												\
+			break;														\
+		}																\
+	}																	\
+	if (presorted)														\
+		return;															\
+	pm = (char *) a + (n / 2) * es;										\
+	if (n > 7)															\
+	{																	\
+		pl = (char *) a;												\
+		pn = (char *) a + (n - 1) * es;									\
+		if (n > 40)														\
+		{																\
+			d = (n / 8) * es;											\
+			pl = TYPE##SPEC_VAR##med3(pl, pl + d, pl + 2 * d, arg);		\
+			pm = TYPE##SPEC_VAR##med3(pm - d, pm, pm + d, arg);			\
+			pn = TYPE##SPEC_VAR##med3(pn - 2 * d, pn - d, pn, arg);		\
+		}																\
+		pm = TYPE##SPEC_VAR##med3(pl, pm, pn, arg);						\
+	}																	\
+	thistype_swap(TYPE, a, pm, SPEC_VAR);								\
+	pa = pb = (char *) a + es;											\
+	pc = pd = (char *) a + (n - 1) * es;								\
+	for (;;)															\
+	{																	\
+		while (pb <= pc && 												\
+					(r = TYPE##SPEC_VAR##AppSort(pb, a, arg)) <= 0)		\
+		{																\
+			if (r == 0)													\
+			{															\
+				thistype_swap(TYPE, pa, pb, SPEC_VAR);					\
+				pa += es;												\
+			}															\
+			pb += es;													\
+		}																\
+		while (pb <= pc && 												\
+				(r = TYPE##SPEC_VAR##AppSort(pc, a, arg)) >= 0)			\
+		{																\
+			if (r == 0)													\
+			{															\
+				thistype_swap(TYPE, pc, pd, SPEC_VAR);					\
+				pd -= es;												\
+			}															\
+			pc -= es;													\
+		}																\
+		if (pb > pc)													\
+			break;														\
+		thistype_swap(TYPE, pb, pc, SPEC_VAR);							\
+		pb += es;														\
+		pc -= es;														\
+	}																	\
+	pn = (char *) a + n * es;											\
+	r = Min(pa - (char *) a, pb - pa);									\
+	thistype_vecswap(TYPE, a, pb - r, r, SPEC_VAR);						\
+	r = Min(pd - pc, pn - pd - es);										\
+	thistype_vecswap(TYPE, pb, pn - r, r, SPEC_VAR);					\
+	if ((r = pb - pa) > es)												\
+		TYPE##SPEC_VAR##qsort_arg(a, r / es, es, arg);					\
+	if ((r = pd - pc) > es)												\
+	{																	\
+		/* Iterate rather than recurse to save stack space */			\
+		a = pn - r;														\
+		n = r / es;														\
+		goto loop;														\
+	}																	\
+}
+
+/*
+ * This code becomes part of the comparator meta-function for the "reg" 
+ * specialization variant of each datatype-specific specialization.
+ *
+ * For "reg", we can handle multiple scankeys, but the function generally
+ * will not be inlined. Inlining has been found to offer additional performance 
+ * benefits beyond eliding the regular SQL function-call machinery.
+ *
+ * We do not elide the regular SQL function-call machinery in the "reg" case
+ * for the second and subsequent scanKeys. The higher the cardinality of the
+ * first scanKey column, the less frequently we'll need to fall back.
+ */ 
+
+#define REG_ADDITIONAL_CODE 														\
+{												 									\
+	ScanKey		scanKey = state->scanKeys;											\
+	HeapTupleData ltup;																\
+	HeapTupleData rtup;																\
+	TupleDesc	tupDesc;															\
+	int			nkey;																\
+																					\
+	Assert(state->nKeys > 1)														\
+	/* Compare additional sort keys */												\
+	ltup.t_len = ((MinimalTuple) aT->tuple)->t_len + MINIMAL_TUPLE_OFFSET;			\
+	ltup.t_data = (HeapTupleHeader) ((char *) aT->tuple - MINIMAL_TUPLE_OFFSET);	\
+	rtup.t_len = ((MinimalTuple) bT->tuple)->t_len + MINIMAL_TUPLE_OFFSET;			\
+	rtup.t_data = (HeapTupleHeader) ((char *) bT->tuple - MINIMAL_TUPLE_OFFSET);	\
+	tupDesc = state->tupDesc;														\
+	scanKey++;																		\
+	for (nkey = 1; nkey < state->nKeys; nkey++, scanKey++)							\
+	{																				\
+		AttrNumber	attno = scanKey->sk_attno;										\
+		Datum		datum1,															\
+					datum2;															\
+		bool		isnull1,														\
+					isnull2;														\
+																					\
+		datum1 = heap_getattr(&ltup, attno, tupDesc, &isnull1);						\
+		datum2 = heap_getattr(&rtup, attno, tupDesc, &isnull2);						\
+																					\
+		compare = inlineApplySortFunction(&scanKey->sk_func,						\
+										  scanKey->sk_flags,						\
+										  scanKey->sk_collation,					\
+										  datum1, isnull1,							\
+										  datum2, isnull2);							\
+		if (compare != 0)															\
+			return compare;															\
+	}																				\
+}
+
+/* 
+ * Manufacture inlining variant for nKeys=1 case, and non-inlining variant
+ * for nKeys > 1 case
+ */	
+#define TEMPLATE_QSORT_ARG(TYPE, COMPAR)	\
+DO_TEMPLATE_QSORT_ARG(TYPE, COMPAR, inl, ;) 	\
+DO_TEMPLATE_QSORT_ARG(TYPE, COMPAR, reg, REG_ADDITIONAL_CODE)	\
+	
diff --git a/src/port/qsort.c b/src/port/qsort.c
index 8e2c6d9..c949c8e 100644
--- a/src/port/qsort.c
+++ b/src/port/qsort.c
@@ -7,7 +7,8 @@
  *	  Remove ill-considered "swap_cnt" switch to insertion sort,
  *	  in favor of a simple check for presorted input.
  *
- *	CAUTION: if you change this file, see also qsort_arg.c
+ *	CAUTION: if you change this file, see also qsort_arg.c and 
+ *	template_qsort_arg.h
  *
  *	src/port/qsort.c
  */
-- 
1.7.7.3

results.odsapplication/vnd.oasis.opendocument.spreadsheet; name=results.odsDownload
tuplesort_test.pytext/x-python; charset=US-ASCII; name=tuplesort_test.pyDownload
#52Peter Geoghegan
peter@2ndquadrant.com
In reply to: Peter Geoghegan (#51)
1 attachment(s)
Re: Inlining comparators as a performance optimisation

Attached are the results from performing a similar process to the
prior benchmark, but on Greg Smith's high-end server, and with an
orderlines table that has been "doubled-up" until it is 1538 MB,
making the same old query perform a quicksort that's over 3GB. Short
version: HEAD is 20468.0ms, with my patch it's 13689.698ms, so these
gains hold-up very well for very large in-memory sorts, possibly even
perfectly.

Note that the "doubling up" had an interesting and desirable effect
for the purposes of this patch review: It's approaching a worst case
due to the low cardinality of the product + quantity columns, which
crops up with the non-inlined functions only (int4regqsort_arg, etc).
All too frequently, evaluating the first scankey results in equality,
and so we cannot elide the SQL function call machinery. This is going
to dampen the improvement for the multiple scanKey case considerably
(and it looks like a smaller relative improvement than when the
orderlines table was quite a lot smaller). As I said from the outset,
there is no worst case for the single scanKey case that occurs to me.
Multiple scanKeys are likely to be a problem for any effort to offer
user-defined, per-type sort functions. I could probably make
int4regqsort_arg and friends perform a bit better than we see here by
increasing the cardinality of those two columns for the second query,
so the first scanKey comparison would frequently suffice.

Incidentally, I'm pretty sceptical of the idea of any effort being
made to provide user-defined per-type sort functions or anything like
that. No one has come forward with a novel sorting implementation that
looks useful, although there has been some talk of some. It's
relatively easy to hack on tuplesort to build a prototype, so I don't
think the lack of this functionality is the blocker here. Besides, we
will probably end up doing a terrible job of anticipating how invasive
such a facility needs to be to facilitate these novel sorting
implementations.

While I'm very encouraged by these results, I must admit that there
are a few things that I cannot currently explain. I don't know why the
second query on the "Optimized" sheet outperforms the same query on
the "Not inlined" page. When you consider how small the standard
deviation is for each set of results, it's fairly obvious that it
cannot be explained by noise. I thought it was weird, so I re-ran both
benchmarks, with much the same outcome. It may be that the compiler
gets lucky for the optimized case, resulting in a bit of an extra
gain, and that effect is ridiculously magnified. In all cases, the
regression tests pass.

The single key/inlining optimisation seems to account for a big part
of the gains, and the "Not inlined" figures for the first query would
seem to suggest that the inlining can account for more than half of
gains seen, whereas that was previously assumed to be less. What I
think is happening here is that we're benefiting both from inlining as
well as not having to even consider multiple scanKeys (we have to at
least consider them even if we know in advance from the nScankeys
variable that there'll never be multiple scanKeys). Again, if the
cardinality was higher, we'd probably see "Not inlined" do better
here.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

Attachments:

results_server.odsapplication/vnd.oasis.opendocument.spreadsheet; name=results_server.odsDownload
#53Bruce Momjian
bruce@momjian.us
In reply to: Peter Geoghegan (#52)
Re: Inlining comparators as a performance optimisation

Peter Geoghegan wrote:

Attached are the results from performing a similar process to the
prior benchmark, but on Greg Smith's high-end server, and with an
orderlines table that has been "doubled-up" until it is 1538 MB,
making the same old query perform a quicksort that's over 3GB. Short
version: HEAD is 20468.0ms, with my patch it's 13689.698ms, so these
gains hold-up very well for very large in-memory sorts, possibly even
perfectly.

Note that the "doubling up" had an interesting and desirable effect
for the purposes of this patch review: It's approaching a worst case
due to the low cardinality of the product + quantity columns, which
crops up with the non-inlined functions only (int4regqsort_arg, etc).
All too frequently, evaluating the first scankey results in equality,
and so we cannot elide the SQL function call machinery. This is going
to dampen the improvement for the multiple scanKey case considerably
(and it looks like a smaller relative improvement than when the
orderlines table was quite a lot smaller). As I said from the outset,
there is no worst case for the single scanKey case that occurs to me.
Multiple scanKeys are likely to be a problem for any effort to offer
user-defined, per-type sort functions. I could probably make
int4regqsort_arg and friends perform a bit better than we see here by
increasing the cardinality of those two columns for the second query,
so the first scanKey comparison would frequently suffice.

These are exciting advanced you are producing and I am hopeful we can
get this included in Postgres 9.2. I have mentioned already that I
think parallelism is the next big Postgres challenge, and of course, one
of the first areas for parallelism is sorting. If you can speed up
sorting as you have reported, this allows us to provide faster sorting
now, and get even larger benefits if we implement sorting parallelism.

In fact, because Postgres 9.2 has so many performance improvements, we
might find that we have a more limited need for parallelism.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

#54Peter Geoghegan
peter@2ndquadrant.com
In reply to: Bruce Momjian (#53)
Re: Inlining comparators as a performance optimisation

On 29 November 2011 15:31, Bruce Momjian <bruce@momjian.us> wrote:

These are exciting advanced you are producing and I am hopeful we can
get this included in Postgres 9.2.

Thanks Bruce.

I have mentioned already that I
think parallelism is the next big Postgres challenge, and of course, one
of the first areas for parallelism is sorting.

I'm not sure that sorting has that much to recommend it as an initial
target of some new backend parallelism other than being easy to
implement. I've observed the qsort_arg specialisations in this patch
out-perform stock qsort_arg by as much as almost 3 times. However, the
largest decrease in a query's time that I've observed was 45%, and
that was for a contrived worst-case for quicksort, but about 25% is
much more typical of queries similar to the ones I've shown, for more
normative data distributions. While that's a respectable gain, it
isn't a paradigm shifting one, and it makes parallelising qsort itself
for further improvements quite a lot less attractive - there's too
many other sources of overhead.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

#55Bruce Momjian
bruce@momjian.us
In reply to: Peter Geoghegan (#54)
Re: Inlining comparators as a performance optimisation

Peter Geoghegan wrote:

On 29 November 2011 15:31, Bruce Momjian <bruce@momjian.us> wrote:

These are exciting advanced you are producing and I am hopeful we can
get this included in Postgres 9.2.

Thanks Bruce.

I have mentioned already that I
think parallelism is the next big Postgres challenge, and of course, one
of the first areas for parallelism is sorting.

I'm not sure that sorting has that much to recommend it as an initial
target of some new backend parallelism other than being easy to
implement. I've observed the qsort_arg specialisations in this patch
out-perform stock qsort_arg by as much as almost 3 times. However, the
largest decrease in a query's time that I've observed was 45%, and
that was for a contrived worst-case for quicksort, but about 25% is
much more typical of queries similar to the ones I've shown, for more
normative data distributions. While that's a respectable gain, it
isn't a paradigm shifting one, and it makes parallelising qsort itself
for further improvements quite a lot less attractive - there's too
many other sources of overhead.

Agreed. I think your improvements make it likely we will address not
address sort parallelism first.

With all the improvements coming in Postgres 9.2, we might need to look
at I/O parallelism first.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

#56Andres Freund
andres@anarazel.de
In reply to: Peter Geoghegan (#54)
Re: Inlining comparators as a performance optimisation

On Tuesday, November 29, 2011 07:48:37 PM Peter Geoghegan wrote:

On 29 November 2011 15:31, Bruce Momjian <bruce@momjian.us> wrote:

These are exciting advanced you are producing and I am hopeful we can
get this included in Postgres 9.2.

Thanks Bruce.

I have mentioned already that I

think parallelism is the next big Postgres challenge, and of course, one
of the first areas for parallelism is sorting.

I'm not sure that sorting has that much to recommend it as an initial
target of some new backend parallelism other than being easy to
implement. I've observed the qsort_arg specialisations in this patch
out-perform stock qsort_arg by as much as almost 3 times. However, the
largest decrease in a query's time that I've observed was 45%, and
that was for a contrived worst-case for quicksort, but about 25% is
much more typical of queries similar to the ones I've shown, for more
normative data distributions. While that's a respectable gain, it
isn't a paradigm shifting one, and it makes parallelising qsort itself
for further improvements quite a lot less attractive - there's too
many other sources of overhead.

I think that logic is faulty.

For one I doubt that anybody is honestly suggesting paralellism inside qsort
itself. It seems more likely/sensible to implement that on the level of
mergesorting.
Index builds for example could hugely benefit from improvements on that level.
With index build you often get pretty non-optimal data distributions btw...

I also seriously doubt that you will find an area inside pg's executor where
you find that paralellizing them will provide a near linear scale without
much, much more work.

Also I wouldn't consider sorting the easiest target - especially on a qsort
level - for parallelization as you constantly need to execute user defined
operators with multiple input tuples which has the usual problems.
COPY parsing + inserting or such seems to be way easier target for example.
Even doing hashing + aggregation in different threads seems likely to be
easier.

Andres

#57Greg Jaskiewicz
gj@pointblue.com.pl
In reply to: Peter Geoghegan (#52)
Re: Inlining comparators as a performance optimisation

On 28 Nov 2011, at 02:15, Peter Geoghegan wrote:

Attached are the results from performing a similar process to the
prior benchmark, but on Greg Smith's high-end server, and with an
orderlines table that has been "doubled-up" until it is 1538 MB,
making the same old query perform a quicksort that's over 3GB. Short
version: HEAD is 20468.0ms, with my patch it's 13689.698ms, so these
gains hold-up very well for very large in-memory sorts, possibly even
perfectly.

This is some really good stuff.
Has to be said, that there must be quite few other places where inlining like that could have quite a positive benefit.
But - I also have to say that both template functions and template classes in C++ give you pretty much the same speed improvement, with much better clarity and readability of the code. (I've tested it with the example Peter presented, if someone is interested in the code).
One more reason why PostgreSQL project should look into the future and get some of the bits simplified and optimised by switching to C++.

#58Peter Geoghegan
peter@2ndquadrant.com
In reply to: Robert Haas (#49)
1 attachment(s)
Re: Inlining comparators as a performance optimisation

Attached is revision of my patch with some clean-ups. In particular,
I'm now using switch statements for greater readability, plus
supporting fast path sorting of the time datatype. I've also updated
the documentation on "Date/Time Types" to note the additional
disadvantage of using the deprecated "store timestamp + friends as
double precision floating-point numbers" compile time option.

There is one aspect to this optimisation that I haven't touched on,
which is the effect on memory consumption. I think that much of the
value that this patch will deliver will come from being able to
release sort memory earlier. Consider that the substantial
improvements in raw sorting speed (far more substantial than the
improvements in query runtime) will sometimes result in a concomitant
reduction in the time that the executor holds onto memory allocated
for sorting. Maybe the effect will only be really noticeable for plans
with a sort node as their root node, but that isn't exactly a rare
occurrence, particularly among large, expensive sorts.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

Attachments:

readability_inline_compar.patchtext/x-patch; charset=US-ASCII; name=readability_inline_compar.patchDownload
From cffad91578edd698e89ef2bf7384d82aee26b57e Mon Sep 17 00:00:00 2001
From: peter2ndQuadrant <peter@2ndquadrant.com>
Date: Sun, 20 Nov 2011 20:36:25 +0000
Subject: [PATCH] Initial commit of optimization

Stop directly using oid

Added int8 quicksort fast path specialisation, which can also be used in
place of F_TIMESTAMP_CMP for HAVE_INT64_TIMESTAMP builds.

Rebased, revised patch for -hackers, with timestamp and int8 fast path
sorting using the same int8 specialization.

Remove unneeded line

Rebase for -hackers

Descriminate against cases where nKeys > 1 when selecting sort
specialisation.

We now support fast path sorting with multiple scanKeys.

We now inline for one scanKey, but do not inline our comparetup_heap
replacement when multiple scanKeys are used (although this is at the
compiler's discretion). However, when multiple scanKeys are used, we
still use a specialisation that will elide the SQL-function-call
machinery for the first scanKey, which is almost as good for most cases.

Further formatting clean-ups

Move specialization selection logic to tuplesort_begin_heap(), per
suggestion of Robert Haas

Add simple switch for testing optimization.

Improvements to template qsort_arg comments. Support fast path sorting
of dates.

Added float fast path sorting

Patch mimimization.

Additional clean-up.

Indentation issue.

Specialization clean-ups

Use switch statement for sort specialization selection

Further comment clean-up.

Additional clean-up

Make comparators consistent

Added date fast path support, documentation updates

Fix template_qsort_arg.h assertion

Assert nScanKeys == 1 for inlining case
---
 doc/src/sgml/datatype.sgml             |    6 +
 src/backend/utils/sort/tuplesort.c     |  223 ++++++++++++++++++++++++-
 src/include/utils/template_qsort_arg.h |  288 ++++++++++++++++++++++++++++++++
 src/port/qsort.c                       |    3 +-
 4 files changed, 512 insertions(+), 8 deletions(-)
 create mode 100644 src/include/utils/template_qsort_arg.h

diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index fe59a1c..f5dfbbb 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -1619,6 +1619,12 @@ SELECT E'\\xDEADBEEF';
     floating-point case, large <type>interval</type> values degrade in
     precision as the size of the interval increases.
    </para>
+   <para>
+    Note that if the server is built to represent these types as
+    floating point numbers, that will prevent it from using fast path
+    sort specializations for the types, and sorting columns of those
+    types will be noticeably slower.
+   </para>
    </note>
 
    <para>
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 3505236..0a14a90 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -107,11 +107,13 @@
 #include "miscadmin.h"
 #include "pg_trace.h"
 #include "utils/datum.h"
+#include "utils/fmgroids.h"
 #include "utils/logtape.h"
 #include "utils/lsyscache.h"
 #include "utils/memutils.h"
 #include "utils/pg_rusage.h"
 #include "utils/rel.h"
+#include "utils/template_qsort_arg.h"
 #include "utils/tuplesort.h"
 
 
@@ -226,6 +228,13 @@ struct Tuplesortstate
 										   Tuplesortstate *state);
 
 	/*
+	 * This specialization function pointer is sometimes used as an alternative
+	 * to the standard qsort_arg, when it has been determined that we can 
+	 * benefit from various per-type performance optimizations.
+	 */
+	void (*qsort_arg_spec)(void *a, size_t n, size_t es, void *arg);
+
+	/*
 	 * Function to copy a supplied input tuple into palloc'd space and set up
 	 * its SortTuple representation (ie, set tuple/datum1/isnull1).  Also,
 	 * state->availMem must be decreased by the amount of space used for the
@@ -457,6 +466,11 @@ static void tuplesort_heap_insert(Tuplesortstate *state, SortTuple *tuple,
 static void tuplesort_heap_siftup(Tuplesortstate *state, bool checkIndex);
 static unsigned int getlen(Tuplesortstate *state, int tapenum, bool eofOK);
 static void markrunend(Tuplesortstate *state, int tapenum);
+static inline int32 inlineApplySortFunction(FmgrInfo *sortFunction, 
+						int sk_flags, Oid collation,
+						Datum datum1, bool isNull1,
+						Datum datum2, bool isNull2);
+
 static int comparetup_heap(const SortTuple *a, const SortTuple *b,
 				Tuplesortstate *state);
 static void copytup_heap(Tuplesortstate *state, SortTuple *stup, void *tup);
@@ -576,6 +590,99 @@ tuplesort_begin_common(int workMem, bool randomAccess)
 	return state;
 }
 
+/*
+ * Manufacture type specific sorting specialisations
+ * with inline comparators
+ */
+static inline int
+compare_int4(Datum first, Datum second)
+{
+	int32           a = DatumGetInt32(first);
+	int32           b = DatumGetInt32(second);
+	if (a > b)
+		return 1;
+	else if (a == b)
+		return 0;
+	else
+		return -1;
+}
+
+static inline int
+compare_int8(Datum first, Datum second)
+{
+	int64           a = DatumGetInt64(first);
+	int64           b = DatumGetInt64(second);
+
+	if (a > b)
+		return 1;
+	else if (a == b)
+		return 0;
+	else
+		return -1;
+}
+
+static inline int
+compare_float4(Datum first, Datum second)
+{
+	float4		a = DatumGetFloat4(first);
+	float4		b = DatumGetFloat4(second);
+
+	if (isnan(a))
+	{
+		if (isnan(b))
+			return 0;			/* NAN = NAN */
+		else
+			return 1;			/* NAN > non-NAN */
+	}
+	else if (isnan(b))
+	{
+		return -1;				/* non-NAN < NAN */
+	}
+	else
+	{
+		if (a > b)
+			return 1;
+		else if (a < b)
+			return -1;
+		else
+			return 0;
+	}
+}
+
+static inline int
+compare_float8(Datum first, Datum second)
+{
+	float8		a = DatumGetFloat8(first);
+	float8		b = DatumGetFloat8(second);
+
+	if (isnan(a))
+	{
+		if (isnan(b))
+			return 0;			/* NAN = NAN */
+		else
+			return 1;			/* NAN > non-NAN */
+	}
+	else if (isnan(b))
+	{
+		return -1;				/* non-NAN < NAN */
+	}
+	else
+	{
+		if (a > b)
+			return 1;
+		else if (a < b)
+			return -1;
+		else
+			return 0;
+	}
+}
+
+/* Instantiate sorting specializations */
+TEMPLATE_QSORT_ARG(int4, compare_int4);
+TEMPLATE_QSORT_ARG(int8, compare_int8);
+TEMPLATE_QSORT_ARG(float4, compare_float4);
+TEMPLATE_QSORT_ARG(float8, compare_float8);
+
 Tuplesortstate *
 tuplesort_begin_heap(TupleDesc tupDesc,
 					 int nkeys, AttrNumber *attNums,
@@ -650,6 +757,83 @@ tuplesort_begin_heap(TupleDesc tupDesc,
 							   (Datum) 0);
 	}
 
+	/* Select a qsort specialization, if possible */
+	if (state->scanKeys)
+	{
+		if (state->nKeys == 1)
+		{
+			/* Single scankey, use an inlining specialization */
+			switch (state->scanKeys->sk_func.fn_oid)
+			{
+				/* Some comparison that has an underlying int4 representation */
+				case F_BTINT4CMP:
+					state->qsort_arg_spec = int4inlqsort_arg;
+					break;
+				case F_DATE_CMP:
+					state->qsort_arg_spec = int4inlqsort_arg;
+					break;
+				/* Some comparison that has an underlying int8 representation */
+				case F_BTINT8CMP:
+					state->qsort_arg_spec = int8inlqsort_arg;
+					break;
+#ifdef HAVE_INT64_TIMESTAMP
+				case F_TIMESTAMP_CMP:
+					state->qsort_arg_spec = int8inlqsort_arg;
+					break;
+				case F_TIME_CMP:
+					state->qsort_arg_spec = int8inlqsort_arg;
+					break;
+#endif
+				/* floating point types */
+				case F_BTFLOAT4CMP:
+					state->qsort_arg_spec = float4inlqsort_arg;
+					break;
+				case F_BTFLOAT8CMP:
+					state->qsort_arg_spec = float8inlqsort_arg;
+					break;
+				default:
+					state->qsort_arg_spec = NULL;
+			}
+		}
+		else
+		{
+			/*
+			 * Multiple scankeys, use a specialisation to elide
+			 * SQL function call machinery for the first
+			 */
+			switch (state->scanKeys->sk_func.fn_oid)
+			{
+				/* Some comparison that has an underlying int4 representation */
+				case F_BTINT4CMP:
+					state->qsort_arg_spec = int4regqsort_arg;
+					break;
+				case F_DATE_CMP:
+					state->qsort_arg_spec = int4regqsort_arg;
+					break;
+				/* Some comparison that has an underlying int8 representation */
+				case F_BTINT8CMP:
+					state->qsort_arg_spec = int8regqsort_arg;
+					break;
+#ifdef HAVE_INT64_TIMESTAMP
+				case F_TIMESTAMP_CMP:
+					state->qsort_arg_spec = int8regqsort_arg;
+					break;
+				case F_TIME_CMP:
+					state->qsort_arg_spec = int8regqsort_arg;
+					break;
+#endif
+				/* floating point types */
+				case F_BTFLOAT4CMP:
+					state->qsort_arg_spec = float4regqsort_arg;
+					break;
+				case F_BTFLOAT8CMP:
+					state->qsort_arg_spec = float8regqsort_arg;
+					break;
+				default:
+					state->qsort_arg_spec = NULL;
+			}
+		}
+	}
 	MemoryContextSwitchTo(oldcontext);
 
 	return state;
@@ -1225,7 +1409,7 @@ puttuple_common(Tuplesortstate *state, SortTuple *tuple)
 }
 
 /*
- * All tuples have been provided; finish the sort.
+ * All tuples have been provided; perform the sort.
  */
 void
 tuplesort_performsort(Tuplesortstate *state)
@@ -1247,11 +1431,28 @@ tuplesort_performsort(Tuplesortstate *state)
 			 * amount of memory.  Just qsort 'em and we're done.
 			 */
 			if (state->memtupcount > 1)
-				qsort_arg((void *) state->memtuples,
-						  state->memtupcount,
-						  sizeof(SortTuple),
-						  (qsort_arg_comparator) state->comparetup,
-						  (void *) state);
+			{
+				/* Consider a sorting specialization */
+				if (state->qsort_arg_spec)
+				{
+					state->qsort_arg_spec((void *) state->memtuples,
+								  state->memtupcount,
+								  sizeof(SortTuple),
+								  (void *) state);
+				}
+				else
+				{
+					/*
+					 * Fall back on type-neutral qsort with
+					 * function-pointer comparator
+					 */
+					qsort_arg((void *) state->memtuples,
+								  state->memtupcount,
+								  sizeof(SortTuple),
+								  (qsort_arg_comparator) state->comparetup,
+								  (void *) state);
+				}
+			}
 			state->current = 0;
 			state->eof_reached = false;
 			state->markpos_offset = 0;
@@ -2657,6 +2858,9 @@ myFunctionCall2Coll(FmgrInfo *flinfo, Oid collation, Datum arg1, Datum arg2)
  * and return a 3-way comparison result.  This takes care of handling
  * reverse-sort and NULLs-ordering properly.  We assume that DESC and
  * NULLS_FIRST options are encoded in sk_flags the same way btree does it.
+ *
+ * Note that this logic is partially duplicated in template_qsort_arg.h,
+ * from which it is "instantiated" per-type.
  */
 static inline int32
 inlineApplySortFunction(FmgrInfo *sortFunction, int sk_flags, Oid collation,
@@ -2708,10 +2912,15 @@ ApplySortFunction(FmgrInfo *sortFunction, int sortFlags, Oid collation,
 }
 
 
+
+
 /*
  * Routines specialized for HeapTuple (actually MinimalTuple) case
+ *
+ * N.B. : Some of this code is partially duplicated for the purposes 
+ * of heap comparison specializations. Take care to keep the code
+ * within template_qsort_arg.h consistent with this code.
  */
-
 static int
 comparetup_heap(const SortTuple *a, const SortTuple *b, Tuplesortstate *state)
 {
diff --git a/src/include/utils/template_qsort_arg.h b/src/include/utils/template_qsort_arg.h
new file mode 100644
index 0000000..9fee892
--- /dev/null
+++ b/src/include/utils/template_qsort_arg.h
@@ -0,0 +1,288 @@
+/*-------------------------------------------------------------------------
+ *	template_qsort_arg.h: "template" version of qsort_arg.c
+ *
+ *	This version of qsort_arg is exclusively used within tuplesort.c to	more
+ *	efficiently sort common types such as integers and floats. In providing
+ *	this version, we seek to take advantage of compile-time	optimizations,
+ *	in particular, the inlining of comparators, as well as reducing the
+ *	overhead of comparisons that the general SQL-callable-function API imposes.
+ *
+ *	The TEMPLATE_QSORT_ARG() macro generates an inlining variant (for sorts
+ *	with a single scanKey) and non-inlining variant for sorts with multiple
+ *	scanKeys.
+ *
+ *	We rely on various function declarations, and indeed partially duplicate
+ *	some code from tuplesort.c, so this file should be considered private to
+ *	that module, rather than a generic piece of infrastructure.
+ *
+ *	CAUTION: if you change this file, see also qsort_arg.c as well as
+ *	qsort.c. qsort_arg.c should be considered authoratative.
+ *
+ *	Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group
+ *	Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *	src/include/utils/template_qsort_arg.h
+ *-------------------------------------------------------------------------
+ */
+
+#include "c.h"
+
+#define swapcode(TYPE, parmi, parmj, n) \
+do {									\
+	size_t i = (n) / sizeof (TYPE);		\
+	TYPE *pi = (TYPE *)(void *)(parmi);	\
+	TYPE *pj = (TYPE *)(void *)(parmj);	\
+	do {								\
+		TYPE	t = *pi;				\
+		*pi++ = *pj;					\
+		*pj++ = t;						\
+		} while (--i > 0);				\
+} while (0)
+
+#define SWAPINIT(a, es) swaptype = ((char *)(a) - (char *)0) % sizeof(long) || \
+	(es) % sizeof(long) ? 2 : (es) == sizeof(long)? 0 : 1;
+
+#define thistype_vecswap(TYPE, a, b, n, SPEC_VAR)							\
+	if ((n) > 0) TYPE##SPEC_VAR##swapfunc((a), (b), (size_t)(n), swaptype) 	
+
+#define thistype_swap(TYPE, a, b, SPEC_VAR)			\
+if (swaptype == 0) {								\
+		long t = *(long *)(void *)(a);				\
+		*(long *)(void *)(a) = *(long *)(void *)(b);\
+		*(long *)(void *)(b) = t;					\
+	} else											\
+		TYPE##SPEC_VAR##swapfunc(a, b, es, swaptype)	
+
+/* 
+ * This macro manufactures a type-specific implementation of qsort_arg with
+ * the comparator, COMPAR, known at compile time. COMPAR is typically an
+ * inline function.
+ *
+ * COMPAR should take as its arguments two Datums, and return an int, in
+ * line with standard qsort convention.
+ * 
+ * We have void* parameters for TYPE##AppSort just to shut up the compiler.
+ * They could be SortTuple pointers instead, but that would make it more
+ * difficult to keep template_qsort_arg.h consistent with tuplesort.c.
+ *
+ */
+
+#define DO_TEMPLATE_QSORT_ARG(TYPE, COMPAR, SPEC_VAR, REG_ADDITIONAL_CODE)			\
+void TYPE##SPEC_VAR##qsort_arg(void *a, size_t n, size_t es, void *arg);			\
+																					\
+static inline int32																	\
+TYPE##SPEC_VAR##AppSort(const void *a, const void *b, Tuplesortstate *state)		\
+{																					\
+	int32		compare;															\
+	const SortTuple* aT = a;														\
+	const SortTuple* bT = b;														\
+																					\
+	/* Allow interrupting long sorts */												\
+	CHECK_FOR_INTERRUPTS();															\
+																					\
+	Assert(state->scanKeys);														\
+																					\
+	if ( aT->isnull1)																\
+	{																				\
+		if (bT->isnull1)															\
+			compare = 0;		/* NULL "=" NULL */									\
+		else if (state->scanKeys->sk_flags & SK_BT_NULLS_FIRST)						\
+			compare = -1;		/* NULL "<" NOT_NULL */								\
+		else																		\
+			compare = 1;		/* NULL ">" NOT_NULL */								\
+	}																				\
+	else if (bT->isnull1)															\
+	{																				\
+		if (state->scanKeys->sk_flags & SK_BT_NULLS_FIRST)							\
+			compare = 1;		/* NOT_NULL ">" NULL */								\
+		else																		\
+			compare = -1;		/* NOT_NULL "<" NULL */								\
+	}																				\
+	else																			\
+	{																				\
+		compare = COMPAR(aT->datum1, bT->datum1);									\
+																					\
+		if (state->scanKeys->sk_flags & SK_BT_DESC)									\
+			compare = -compare;														\
+	}																				\
+	if (compare != 0) 																\
+		return compare;																\
+	/* Additional code for variants where we have more than one scanKey */ 			\
+	REG_ADDITIONAL_CODE 															\
+	return 0;																		\
+}																					\
+																					\
+static inline void													\
+TYPE##SPEC_VAR##swapfunc(char *a, char *b, size_t n, int swaptype)	\
+{																	\
+	if (swaptype <= 1)												\
+		swapcode(long, a, b, n); 									\
+	else															\
+		swapcode(char, a, b, n);									\
+}																	\
+																	\
+static inline char *														\
+TYPE##SPEC_VAR##med3(char *a, char *b, char *c, void *arg)					\
+{																			\
+	return TYPE##SPEC_VAR##AppSort(a, b, arg) < 0 ?							\
+		(TYPE##SPEC_VAR##AppSort(b, c, arg) < 0 ? 							\
+					b : (TYPE##SPEC_VAR##AppSort(a, c, arg) < 0 ? c : a))	\
+		: (TYPE##SPEC_VAR##AppSort(b, c, arg) > 0 ? 						\
+					b : (TYPE##SPEC_VAR##AppSort(a, c, arg) < 0 ? a : c));	\
+}																			\
+																			\
+void 																	\
+TYPE##SPEC_VAR##qsort_arg(void *a, size_t n, size_t es, void *arg)		\
+{																		\
+	char	   *pa,														\
+			   *pb,														\
+			   *pc,														\
+			   *pd,														\
+			   *pl,														\
+			   *pm,														\
+			   *pn;														\
+	int			d,														\
+				r,														\
+				swaptype,												\
+				presorted;												\
+																		\
+loop:SWAPINIT(a, es); 													\
+	if (n < 7)															\
+	{																	\
+		for (pm = (char *) a + es; pm < (char *) a + n * es; pm += es)	\
+			for (pl = pm; pl > (char *) a && 							\
+					TYPE##SPEC_VAR##AppSort(pl - es, pl, arg) > 0;		\
+				 pl -= es)												\
+				thistype_swap(TYPE, pl, pl - es, SPEC_VAR);				\
+		return;															\
+	}																	\
+	presorted = 1;														\
+	for (pm = (char *) a + es; pm < (char *) a + n * es; pm += es)		\
+	{																	\
+		if (TYPE##SPEC_VAR##AppSort(pm - es, pm, arg) > 0)				\
+		{																\
+			presorted = 0;												\
+			break;														\
+		}																\
+	}																	\
+	if (presorted)														\
+		return;															\
+	pm = (char *) a + (n / 2) * es;										\
+	if (n > 7)															\
+	{																	\
+		pl = (char *) a;												\
+		pn = (char *) a + (n - 1) * es;									\
+		if (n > 40)														\
+		{																\
+			d = (n / 8) * es;											\
+			pl = TYPE##SPEC_VAR##med3(pl, pl + d, pl + 2 * d, arg);		\
+			pm = TYPE##SPEC_VAR##med3(pm - d, pm, pm + d, arg);			\
+			pn = TYPE##SPEC_VAR##med3(pn - 2 * d, pn - d, pn, arg);		\
+		}																\
+		pm = TYPE##SPEC_VAR##med3(pl, pm, pn, arg);						\
+	}																	\
+	thistype_swap(TYPE, a, pm, SPEC_VAR);								\
+	pa = pb = (char *) a + es;											\
+	pc = pd = (char *) a + (n - 1) * es;								\
+	for (;;)															\
+	{																	\
+		while (pb <= pc && 												\
+					(r = TYPE##SPEC_VAR##AppSort(pb, a, arg)) <= 0)		\
+		{																\
+			if (r == 0)													\
+			{															\
+				thistype_swap(TYPE, pa, pb, SPEC_VAR);					\
+				pa += es;												\
+			}															\
+			pb += es;													\
+		}																\
+		while (pb <= pc && 												\
+				(r = TYPE##SPEC_VAR##AppSort(pc, a, arg)) >= 0)			\
+		{																\
+			if (r == 0)													\
+			{															\
+				thistype_swap(TYPE, pc, pd, SPEC_VAR);					\
+				pd -= es;												\
+			}															\
+			pc -= es;													\
+		}																\
+		if (pb > pc)													\
+			break;														\
+		thistype_swap(TYPE, pb, pc, SPEC_VAR);							\
+		pb += es;														\
+		pc -= es;														\
+	}																	\
+	pn = (char *) a + n * es;											\
+	r = Min(pa - (char *) a, pb - pa);									\
+	thistype_vecswap(TYPE, a, pb - r, r, SPEC_VAR);						\
+	r = Min(pd - pc, pn - pd - es);										\
+	thistype_vecswap(TYPE, pb, pn - r, r, SPEC_VAR);					\
+	if ((r = pb - pa) > es)												\
+		TYPE##SPEC_VAR##qsort_arg(a, r / es, es, arg);					\
+	if ((r = pd - pc) > es)												\
+	{																	\
+		/* Iterate rather than recurse to save stack space */			\
+		a = pn - r;														\
+		n = r / es;														\
+		goto loop;														\
+	}																	\
+}
+
+/*
+ * This code becomes part of the comparator meta-function for the "reg" 
+ * specialization variant of each datatype-specific specialization.
+ *
+ * For "reg", we can handle multiple scankeys, but the function generally
+ * will not be inlined. Inlining has been found to offer additional performance 
+ * benefits beyond eliding the regular SQL function-call machinery.
+ *
+ * We do not elide the regular SQL function-call machinery in the "reg" case
+ * for the second and subsequent scanKeys. The higher the cardinality of the
+ * first scanKey column, the less frequently we'll need to fall back.
+ */ 
+
+#define REG_ADDITIONAL_CODE 														\
+{												 									\
+	ScanKey		scanKey = state->scanKeys;											\
+	HeapTupleData ltup;																\
+	HeapTupleData rtup;																\
+	TupleDesc	tupDesc;															\
+	int			nkey;																\
+																					\
+	Assert(state->nKeys > 1);														\
+	/* Compare additional sort keys */												\
+	ltup.t_len = ((MinimalTuple) aT->tuple)->t_len + MINIMAL_TUPLE_OFFSET;			\
+	ltup.t_data = (HeapTupleHeader) ((char *) aT->tuple - MINIMAL_TUPLE_OFFSET);	\
+	rtup.t_len = ((MinimalTuple) bT->tuple)->t_len + MINIMAL_TUPLE_OFFSET;			\
+	rtup.t_data = (HeapTupleHeader) ((char *) bT->tuple - MINIMAL_TUPLE_OFFSET);	\
+	tupDesc = state->tupDesc;														\
+	scanKey++;																		\
+	for (nkey = 1; nkey < state->nKeys; nkey++, scanKey++)							\
+	{																				\
+		AttrNumber	attno = scanKey->sk_attno;										\
+		Datum		datum1,															\
+					datum2;															\
+		bool		isnull1,														\
+					isnull2;														\
+																					\
+		datum1 = heap_getattr(&ltup, attno, tupDesc, &isnull1);						\
+		datum2 = heap_getattr(&rtup, attno, tupDesc, &isnull2);						\
+																					\
+		compare = inlineApplySortFunction(&scanKey->sk_func,						\
+										  scanKey->sk_flags,						\
+										  scanKey->sk_collation,					\
+										  datum1, isnull1,							\
+										  datum2, isnull2);							\
+		if (compare != 0)															\
+			return compare;															\
+	}																				\
+}
+#define INL_ADDITIONAL_CODE Assert(state->nKeys == 1);
+/* 
+ * Manufacture inlining variant for nKeys=1 case, and non-inlining variant
+ * for nKeys > 1 case
+ */	
+#define TEMPLATE_QSORT_ARG(TYPE, COMPAR)						\
+DO_TEMPLATE_QSORT_ARG(TYPE, COMPAR, inl, INL_ADDITIONAL_CODE)	\
+DO_TEMPLATE_QSORT_ARG(TYPE, COMPAR, reg, REG_ADDITIONAL_CODE)	\
+
diff --git a/src/port/qsort.c b/src/port/qsort.c
index 8e2c6d9..c949c8e 100644
--- a/src/port/qsort.c
+++ b/src/port/qsort.c
@@ -7,7 +7,8 @@
  *	  Remove ill-considered "swap_cnt" switch to insertion sort,
  *	  in favor of a simple check for presorted input.
  *
- *	CAUTION: if you change this file, see also qsort_arg.c
+ *	CAUTION: if you change this file, see also qsort_arg.c and 
+ *	template_qsort_arg.h
  *
  *	src/port/qsort.c
  */
-- 
1.7.7.3

#59Robert Haas
robertmhaas@gmail.com
In reply to: Peter Geoghegan (#58)
Re: Inlining comparators as a performance optimisation

On Thu, Dec 1, 2011 at 11:44 AM, Peter Geoghegan <peter@2ndquadrant.com> wrote:

Attached is revision of my patch with some clean-ups. In particular,
I'm now using switch statements for greater readability, plus
supporting fast path sorting of the time datatype. I've also updated
the documentation on "Date/Time Types" to note the additional
disadvantage of using the deprecated "store timestamp + friends as
double precision floating-point numbers" compile time option.

One thing I'm starting to get a bit concerned about is the effect of
this patch on the size of the resulting binary. The performance
results you've posted are getting steadily more impressive as you get
into this, which is cool, but it seems like the number of copies of
the qsort logic that you're proposing to include in the resulting
binary is also steadily climbing. On my system, a stripped postgres
binary built with my usual compile options (except --enable-cassert,
which I took out) is 49336 bytes bigger with this patch applied, an
increase of close to 1%. We might need to be a little bit choosy
about this, because I don't think that we want to end up with a
situation where some noticeable percentage of the final binary
consists of differently-inlined versions of the quicksort algorithm -
especially because there may be other places where we want to do
similar kinds of inlining.

Thoughts?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#60Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#59)
Re: Inlining comparators as a performance optimisation

Robert Haas <robertmhaas@gmail.com> writes:

On Thu, Dec 1, 2011 at 11:44 AM, Peter Geoghegan <peter@2ndquadrant.com> wrote:

Attached is revision of my patch with some clean-ups.

One thing I'm starting to get a bit concerned about is the effect of
this patch on the size of the resulting binary.

Regardless of that, I'm still of the opinion that this patch is
fundamentally misdesigned. The way it's set up, it is only possible to
have a fast path for types that are hard-wired into tuplesort.c, and
that flies in the face of twenty years' worth of effort to make Postgres
an extensible system. I really don't care about performance
measurements: this is not an acceptable design, period.

What I want to see is some mechanism that pushes this out to the
individual datatypes, so that both core and plug-in datatypes have
access to the benefits. There are a number of ways that could be
attacked, but the most obvious one is to invent additional, optional
support function(s) for btree opclasses.

I also still think that we should pursue the idea of a lower-overhead
API for comparison functions. It may be that it's worth the code bulk
to have an inlined copy of qsort for a small number of high-usage
datatypes, but I think there are going to be a lot for which we aren't
going to want to pay that price, and yet we could get some benefit from
cutting the call overhead. A lower-overhead API would also save cycles
in usage of the comparison functions from btree itself (remember that?).

I think in particular that the proposed approach to multiple sort keys
is bogus and should be replaced by just using lower-overhead
comparators. The gain per added code byte in doing it as proposed
has got to be too low to be reasonable.

regards, tom lane

#61Peter Geoghegan
peter@2ndquadrant.com
In reply to: Robert Haas (#59)
Re: Inlining comparators as a performance optimisation

On 1 December 2011 17:15, Robert Haas <robertmhaas@gmail.com> wrote:

One thing I'm starting to get a bit concerned about is the effect of
this patch on the size of the resulting binary.  The performance
results you've posted are getting steadily more impressive as you get
into this, which is cool, but it seems like the number of copies of
the qsort logic that you're proposing to include in the resulting
binary is also steadily climbing.  On my system, a stripped postgres
binary built with my usual compile options (except --enable-cassert,
which I took out) is 49336 bytes bigger with this patch applied, an
increase of close to 1%.

Do your usual compile options include debug symbols? I've been using
standard compile options for development of this patch, for obvious
reasons. I get 36690 bytes (just under 36 KiB, or a 0.644% increase).

Binary bloat is a legitimate concern. I thought that I was
conservative in my choice of specialisations though.

We might need to be a little bit choosy
about this, because I don't think that we want to end up with a
situation where some noticeable percentage of the final binary
consists of differently-inlined versions of the quicksort algorithm -
especially because there may be other places where we want to do
similar kinds of inlining.

Thoughts?

A simple caveat in a comment along the lines of "this mechanism
instantiates 2 copies of qsort_arg per type, please use judiciously"
sounds like the right balance. It could also be possible to be penny
wise and pound foolish here though.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

#62Robert Haas
robertmhaas@gmail.com
In reply to: Peter Geoghegan (#61)
Re: Inlining comparators as a performance optimisation

On Thu, Dec 1, 2011 at 8:29 PM, Peter Geoghegan <peter@2ndquadrant.com> wrote:

Do your usual compile options include debug symbols? I've been using
standard compile options for development of this patch, for obvious
reasons. I get 36690 bytes (just under 36 KiB, or a 0.644% increase).

They do, but I ran "strip" on the resulting binary before taking those
measurements, so I thought that would undo it...

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#63Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#60)
Re: Inlining comparators as a performance optimisation

On Thu, Dec 1, 2011 at 8:13 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

What I want to see is some mechanism that pushes this out to the
individual datatypes, so that both core and plug-in datatypes have
access to the benefits.  There are a number of ways that could be
attacked, but the most obvious one is to invent additional, optional
support function(s) for btree opclasses.

I feel like what's missing here is a way for the catalogs to refer to
a function that doesn't take FunctionCallInfo as an argument. But it
strikes me that it wouldn't be very hard to design such a mechanism.
It seems like all we really need to do is decide on a nomenclature.
The simplest possible approach would probably be to just refer to
functions by name. In other words, we add a text or name column to
pg_amproc, and then inside the backend there's a function whose job it
is to map the string to a function address. However, that would have
the annoying effect of causing catalog bloat, so I'm inclined to
instead propose an 8-byte integer. (A 4-byte integer is enough, but
would increase the risk of collisions between values that might be
chosen by third party extensions.)

So, the API to this would look something like this:

void register_arbitrary_function_pointer(int64 handle, void *funcptr);
void *get_arbitrary_function_pointer(int64 handle);

So the idea is that if you need to refer to an arbitrary function in a
system catalog (such as pg_amproc), you generate a random non-zero
64-bit integer, stuff it into the catalog, and then register the same
64-bit integer with the appropriate function pointer. To avoid
increasing backend startup time, we could have a Perl script generate
a prefab hash table for the built-in functions; loadable modules would
add their entries at PG_init() time using
register_arbitrary_function_pointer.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#64Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#63)
Re: Inlining comparators as a performance optimisation

Robert Haas <robertmhaas@gmail.com> writes:

On Thu, Dec 1, 2011 at 8:13 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

What I want to see is some mechanism that pushes this out to the
individual datatypes, so that both core and plug-in datatypes have
access to the benefits. There are a number of ways that could be
attacked, but the most obvious one is to invent additional, optional
support function(s) for btree opclasses.

I feel like what's missing here is a way for the catalogs to refer to
a function that doesn't take FunctionCallInfo as an argument. But it
strikes me that it wouldn't be very hard to design such a mechanism.

IMO, entries in pg_proc ought to refer to functions that are callable
by the standard interface: breaking that basic contract is not going to
lead anywhere pleasant. Nor do I really want yet more columns in
pg_proc. Nor does the "register a pointer" scheme you suggest make
any sense to me: you still need to connect these things to catalog
entries, pg_opclass entries in particular, and the intermediate handle
adds nothing to the situation except for creating a risk of collisions.

The scheme that was rolling around in my mind was about like this:

* Define btree opclasses to have an optional support function,
amprocnum 2, that has a SQL signature of func(internal) returns void.

* The actual argument represented by the "internal" parameter would
be a pointer to a struct which is to be filled in by the support
function. We'd call the support function once, during tuplesort
setup or btree index relcache entry setup, and save the struct
somewhere.

* The struct contents would be pointers to functions that have a
non-FunctionCallInfo interface. We know we need at least two:
a full qsort replacement, and a non-FCI comparator. We might want
more in future, if someone convinces us that additional specializations
of sorting are worth the trouble. So I imagine a struct like this:

typedef struct SortSupportFunctions {
void (*inline_qsort) (Datum *elements, int nelements);
int (*comparator) (Datum a, Datum b);
} SortSupportFunctions;

with the agreement that the caller must zero out the struct before call,
and then the btree support function sets the function pointers for any
specializations it's going to offer. If we later add a third or fourth
function pointer, datatypes that know about that could fill in those
pointers, but datatypes that haven't been updated aren't broken.

One thing I'm not too certain about is whether to define the APIs just
as above, or to support a passthrough argument of some sort (and if so,
what does it reference)? Possibly any datatype that we'd actually care
about this for is going to be simple enough to not need any state data.
Or possibly not. And what about collations?

regards, tom lane

#65Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#64)
Re: Inlining comparators as a performance optimisation

On Thu, Dec 1, 2011 at 9:46 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

IMO, entries in pg_proc ought to refer to functions that are callable
by the standard interface: breaking that basic contract is not going to
lead anywhere pleasant.  Nor do I really want yet more columns in
pg_proc.

I wasn't proposing to create pg_proc entries for this.

Nor does the "register a pointer" scheme you suggest make
any sense to me: you still need to connect these things to catalog
entries, pg_opclass entries in particular, and the intermediate handle
adds nothing to the situation except for creating a risk of collisions.

I think you might be misinterpreting what I had in mind. Right now,
pg_amproc entries have an "amproc" column that points to a pg_proc
entry that in turn points to a function that takes
FunctionCallInfoData as an argument. What I'm proposing to do is add
an additional column to that catalog that points more or less directly
to a non-SQL-callable function, but it can't actually just be the
address of the function because that's not stable. So what I'm
proposing is that we interpose the thinnest possible shim layer
between the catalog and a function pointer, and an int64 -> function
pointer mapping seemed to me like something that would fit the bill.

The scheme that was rolling around in my mind was about like this:

* Define btree opclasses to have an optional support function,
amprocnum 2, that has a SQL signature of func(internal) returns void.

* The actual argument represented by the "internal" parameter would
be a pointer to a struct which is to be filled in by the support
function.  We'd call the support function once, during tuplesort
setup or btree index relcache entry setup, and save the struct
somewhere.

* The struct contents would be pointers to functions that have a
non-FunctionCallInfo interface.  We know we need at least two:
a full qsort replacement, and a non-FCI comparator.  We might want
more in future, if someone convinces us that additional specializations
of sorting are worth the trouble.  So I imagine a struct like this:

       typedef struct SortSupportFunctions {
               void    (*inline_qsort) (Datum *elements, int nelements);
               int     (*comparator) (Datum a, Datum b);
       } SortSupportFunctions;

with the agreement that the caller must zero out the struct before call,
and then the btree support function sets the function pointers for any
specializations it's going to offer.  If we later add a third or fourth
function pointer, datatypes that know about that could fill in those
pointers, but datatypes that haven't been updated aren't broken.

One thing I'm not too certain about is whether to define the APIs just
as above, or to support a passthrough argument of some sort (and if so,
what does it reference)?  Possibly any datatype that we'd actually care
about this for is going to be simple enough to not need any state data.
Or possibly not.  And what about collations?

Maybe there should be a comparator_setup function that gets the
collation OID and returns void *, and then that void * value gets
passed as a third argument to each call to the comparator function.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#66Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#65)
Re: Inlining comparators as a performance optimisation

Robert Haas <robertmhaas@gmail.com> writes:

On Thu, Dec 1, 2011 at 9:46 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Nor does the "register a pointer" scheme you suggest make
any sense to me: you still need to connect these things to catalog
entries, pg_opclass entries in particular, and the intermediate handle
adds nothing to the situation except for creating a risk of collisions.

I think you might be misinterpreting what I had in mind. Right now,
pg_amproc entries have an "amproc" column that points to a pg_proc
entry that in turn points to a function that takes
FunctionCallInfoData as an argument. What I'm proposing to do is add
an additional column to that catalog that points more or less directly
to a non-SQL-callable function, but it can't actually just be the
address of the function because that's not stable. So what I'm
proposing is that we interpose the thinnest possible shim layer
between the catalog and a function pointer, and an int64 -> function
pointer mapping seemed to me like something that would fit the bill.

But you still need a lot of mechanism to do that mapping, including an
initialization function that has noplace to be executed (unless every
.so that defines a btree opclass has to be listed in preload_libraries),
so it doesn't seem like the "thinnest possible shim" to me.

One thing I'm not too certain about is whether to define the APIs just
as above, or to support a passthrough argument of some sort (and if so,
what does it reference)? �Possibly any datatype that we'd actually care
about this for is going to be simple enough to not need any state data.
Or possibly not. �And what about collations?

Maybe there should be a comparator_setup function that gets the
collation OID and returns void *, and then that void * value gets
passed as a third argument to each call to the comparator function.

Maybe. Or perhaps we could merge that work into the
function-pointer-setup function --- that is, give it the collation and
some extra workspace to fool with. We would always know the
collation at the time we're doing that setup.

At this point the struct filled in by the setup function is starting
to feel a lot like FmgrInfo, and history teaches us a lot about what
needs to be in there. So maybe

typedef struct SortSupportInfoData *SortSupportInfo;

typedef struct SortSupportInfoData
{
MemoryContext info_cxt; /* where to allocate subsidiary data */
void *extra; /* any data needed by functions */
Oid collation; /* provided by caller */

void (*inline_qsort) (Datum *elements, int nelements,
SortSupportInfo info);
int (*comparator) (Datum a, Datum b,
SortSupportInfo info);
/* possibly other function pointers in future */
} SortSupportInfoData;

I am thinking that the btree code, at least, would want to just
unconditionally do

colsortinfo->comparator(datum1, datum2, colsortinfo)

so for an opclass that fails to supply the low-overhead comparator,
it would insert into the "comparator" pointer a shim function that
calls the opclass' old-style FCI-using comparator. (Anybody who
complains about the added overhead would be told to get busy and
supply a low-overhead comparator for their datatype...) But to do
that, we have to have enough infrastructure here to cover all cases,
so omitting collation or not having a place to stash an FmgrInfo
won't do.

regards, tom lane

#67Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#66)
Re: Inlining comparators as a performance optimisation

On Thu, Dec 1, 2011 at 10:48 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

But you still need a lot of mechanism to do that mapping, including an
initialization function that has noplace to be executed (unless every
.so that defines a btree opclass has to be listed in preload_libraries),
so it doesn't seem like the "thinnest possible shim" to me.

PG_init?

One thing I'm not too certain about is whether to define the APIs just
as above, or to support a passthrough argument of some sort (and if so,
what does it reference)?  Possibly any datatype that we'd actually care
about this for is going to be simple enough to not need any state data.
Or possibly not.  And what about collations?

Maybe there should be a comparator_setup function that gets the
collation OID and returns void *, and then that void * value gets
passed as a third argument to each call to the comparator function.

Maybe.  Or perhaps we could merge that work into the
function-pointer-setup function --- that is, give it the collation and
some extra workspace to fool with.  We would always know the
collation at the time we're doing that setup.

Ah! That seems quite nice.

At this point the struct filled in by the setup function is starting
to feel a lot like FmgrInfo, and history teaches us a lot about what
needs to be in there.  So maybe

typedef struct SortSupportInfoData *SortSupportInfo;

typedef struct SortSupportInfoData
{
       MemoryContext   info_cxt;       /* where to allocate subsidiary data */
       void            *extra;         /* any data needed by functions */
       Oid             collation;      /* provided by caller */

       void    (*inline_qsort) (Datum *elements, int nelements,
                                SortSupportInfo info);
       int     (*comparator) (Datum a, Datum b,
                              SortSupportInfo info);
       /* possibly other function pointers in future */
} SortSupportInfoData;

I am thinking that the btree code, at least, would want to just
unconditionally do

       colsortinfo->comparator(datum1, datum2, colsortinfo)

so for an opclass that fails to supply the low-overhead comparator,
it would insert into the "comparator" pointer a shim function that
calls the opclass' old-style FCI-using comparator.  (Anybody who
complains about the added overhead would be told to get busy and
supply a low-overhead comparator for their datatype...)  But to do
that, we have to have enough infrastructure here to cover all cases,
so omitting collation or not having a place to stash an FmgrInfo
won't do.

I'm slightly worried about whether that'll be adding too much overhead
to the case where there is no non-FCI comparator. But it may be no
worse than what we're doing now.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#68Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#67)
Re: Inlining comparators as a performance optimisation

Robert Haas <robertmhaas@gmail.com> writes:

On Thu, Dec 1, 2011 at 10:48 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I am thinking that the btree code, at least, would want to just
unconditionally do

colsortinfo->comparator(datum1, datum2, colsortinfo)

so for an opclass that fails to supply the low-overhead comparator,
it would insert into the "comparator" pointer a shim function that
calls the opclass' old-style FCI-using comparator. (Anybody who
complains about the added overhead would be told to get busy and
supply a low-overhead comparator for their datatype...) But to do
that, we have to have enough infrastructure here to cover all cases,
so omitting collation or not having a place to stash an FmgrInfo
won't do.

I'm slightly worried about whether that'll be adding too much overhead
to the case where there is no non-FCI comparator. But it may be no
worse than what we're doing now.

It should be the same or better. Right now, we are going through
FunctionCall2Coll to reach the FCI-style comparator. The shim function
would be more or less equivalent to that, and since it's quite special
purpose I would hope we could shave a cycle or two. For instance, we
could probably afford to set up a dedicated FunctionCallInfo struct
associated with the SortSupportInfo struct, and not have to reinitialize
one each time.

regards, tom lane

#69Peter Geoghegan
peter@2ndquadrant.com
In reply to: Tom Lane (#60)
Re: Inlining comparators as a performance optimisation

On 2 December 2011 01:13, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Regardless of that, I'm still of the opinion that this patch is
fundamentally misdesigned.  The way it's set up, it is only possible to
have a fast path for types that are hard-wired into tuplesort.c, and
that flies in the face of twenty years' worth of effort to make Postgres
an extensible system.

What the user doesn't know can't hurt them. I'm not of the opinion
that an internal optimisations flies in the face of anything - is it
really so hard for you to hold your nose for a fairly isolated patch
like this?

I really don't care about performance
measurements: this is not an acceptable design, period.

If ever there was a justification for doing something so distasteful,
I would think that a 60% decrease in the time taken to perform a raw
sort for POD types (plus common covariants) would be it.

What I want to see is some mechanism that pushes this out to the
individual datatypes, so that both core and plug-in datatypes have
access to the benefits.  There are a number of ways that could be
attacked, but the most obvious one is to invent additional, optional
support function(s) for btree opclasses.

I also still think that we should pursue the idea of a lower-overhead
API for comparison functions.  It may be that it's worth the code bulk
to have an inlined copy of qsort for a small number of high-usage
datatypes, but I think there are going to be a lot for which we aren't
going to want to pay that price, and yet we could get some benefit from
cutting the call overhead.

I'm not opposed to that. It just seems fairly tangential to what I've
done here, as well as considerably less important to users,
particularly when you look at the numbers I've produced. It would be
nice to get a lesser gain for text and things like that too, but other
than that I don't see a huge demand.

A lower-overhead API would also save cycles
in usage of the comparison functions from btree itself (remember that?).

Yes, I remember that. Why not do something similar there, and get
similarly large benefits?

I think in particular that the proposed approach to multiple sort keys
is bogus and should be replaced by just using lower-overhead
comparators.  The gain per added code byte in doing it as proposed
has got to be too low to be reasonable.

Reasonable by what standard? We may well be better off with
lower-overhead comparators for the multiple scanKey case (though I
wouldn't like to bet on it) but we would most certainly not be better
off without discriminating against single scanKey and multiple scanKey
cases as I have.

Look at the spreadsheet results_server.ods . Compare the "not inlined"
and "optimized" sheets. It's clear from those numbers that a
combination of inlining and of simply not having to consider a case
that will never come up (multiple scanKeys), the inlining
specialisation far outperforms the non-inlining, multiple scanKey
specialization for the same data (I simply commented out inlining
specializations so that non-inlining specialisations would always be
called for int4 and friends, even if there was only a single scanKey).
That's where the really dramatic improvements are seen. It's well
worth having this additional benefit, because it's such a common case
and the benefit is so big, and while I will be quite happy to pursue a
plan to bring some of these benefits to all types such as the one you
describe, I do not want to jettison the additional benefits described
to do so. Isn't that reasonable?

We're talking about taking 6778.302ms off a 20468.0ms query, rather
than 1860.332ms. Not exactly a subtle difference. Assuming that those
figures are representative of the gains to be had by a generic
mechanism that does not inline/specialize across number of scanKeys,
are you sure that that's worthwhile?

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

#70Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#68)
Re: Inlining comparators as a performance optimisation

On Fri, Dec 2, 2011 at 12:16 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

It should be the same or better.  Right now, we are going through
FunctionCall2Coll to reach the FCI-style comparator.  The shim function
would be more or less equivalent to that, and since it's quite special
purpose I would hope we could shave a cycle or two.  For instance, we
could probably afford to set up a dedicated FunctionCallInfo struct
associated with the SortSupportInfo struct, and not have to reinitialize
one each time.

OK, but I think it's also going to cost you an extra syscache lookup,
no? You're going to have to check for this new support function
first, and then if you don't find it, you'll have to check for the
original one. I don't think there's any higher-level caching around
opfamilies to save our bacon here, is there?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#71Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#70)
Re: Inlining comparators as a performance optimisation

Robert Haas <robertmhaas@gmail.com> writes:

OK, but I think it's also going to cost you an extra syscache lookup,
no? You're going to have to check for this new support function
first, and then if you don't find it, you'll have to check for the
original one. I don't think there's any higher-level caching around
opfamilies to save our bacon here, is there?

[ shrug... ] If you are bothered by that, get off your duff and provide
the function for your datatype. But it's certainly going to be in the
noise for btree index usage, and I submit that query parsing/setup
involves quite a lot of syscache lookups already. I think that as a
performance objection, the above is nonsensical. (And I would also
comment that your proposal with a handle is going to involve a table
search that's at least as expensive as a syscache lookup.)

regards, tom lane

#72Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#71)
Re: Inlining comparators as a performance optimisation

Tom Lane wrote:

Robert Haas <robertmhaas@gmail.com> writes:

OK, but I think it's also going to cost you an extra syscache lookup,
no? You're going to have to check for this new support function
first, and then if you don't find it, you'll have to check for the
original one. I don't think there's any higher-level caching around
opfamilies to save our bacon here, is there?

[ shrug... ] If you are bothered by that, get off your duff and provide
the function for your datatype. But it's certainly going to be in the
noise for btree index usage, and I submit that query parsing/setup
involves quite a lot of syscache lookups already. I think that as a
performance objection, the above is nonsensical. (And I would also
comment that your proposal with a handle is going to involve a table
search that's at least as expensive as a syscache lookup.)

Agreed. Doing something once and doing something in the sort loop are
two different overheads.

I am excited by this major speedup Peter Geoghegan has found. Postgres
doesn't have parallel query, which is often used for sorting, so we are
already behind some of the databases are compared against. Getting this
speedup is definitely going to help us. And I do like the generic
nature of where we are heading!

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

#73Pavel Stehule
pavel.stehule@gmail.com
In reply to: Bruce Momjian (#72)
Re: Inlining comparators as a performance optimisation

[ shrug... ] If you are bothered by that, get off your duff and provide
the function for your datatype.  But it's certainly going to be in the
noise for btree index usage, and I submit that query parsing/setup
involves quite a lot of syscache lookups already.  I think that as a
performance objection, the above is nonsensical.  (And I would also
comment that your proposal with a handle is going to involve a table
search that's at least as expensive as a syscache lookup.)

Agreed.  Doing something once and doing something in the sort loop are
two different overheads.

I am excited by this major speedup Peter Geoghegan has found.  Postgres
doesn't have parallel query, which is often used for sorting, so we are
already behind some of the databases are compared against.  Getting this
speedup is definitely going to help us.  And I do like the generic
nature of where we are heading!

Oracle has not or had not parallel sort too, and I have a reports so
Oracle does sort faster then PostgreSQL (but without any numbers). So
any solution is welcome

Regards

Pavel

#74Robert Haas
robertmhaas@gmail.com
In reply to: Bruce Momjian (#72)
1 attachment(s)
Re: Inlining comparators as a performance optimisation

On Fri, Dec 2, 2011 at 2:35 PM, Bruce Momjian <bruce@momjian.us> wrote:

Agreed.  Doing something once and doing something in the sort loop are
two different overheads.

OK, so I tried to code this up. Adding the new amproc wasn't too
difficult (see attached). It wasn't obvious to me how to tie it into
the tuplesort infrastructure, though, so instead of wasting time
guessing what a sensible approach might be I'm going to use one of my
lifelines and poll the audience (or is that ask an expert?).
Currently the Tuplesortstate[1]Consistent capitalization is for wimps. has a pointer to an array of
ScanKeyData objects, one per column being sorted. But now instead of
"FmgrInfo sk_func", the tuplesort code is going to want each scankey
to contain "SortSupportInfo(Data?) sk_sortsupport"[2]Hey, we could abbreviate that "SSI"! Oh, wait..., because that's
where we get the comparison function from. Should I just go ahead
and add one more member to that struct, or is there some more
appropriate way to handle this?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

[1]: Consistent capitalization is for wimps.
[2]: Hey, we could abbreviate that "SSI"! Oh, wait...

Attachments:

sort-support.patchapplication/octet-stream; name=sort-support.patchDownload
diff --git a/doc/src/sgml/xindex.sgml b/doc/src/sgml/xindex.sgml
index 2832436..5f67624 100644
--- a/doc/src/sgml/xindex.sgml
+++ b/doc/src/sgml/xindex.sgml
@@ -758,7 +758,8 @@ DEFAULT FOR TYPE int4 USING btree FAMILY integer_ops AS
   OPERATOR 3 = ,
   OPERATOR 4 >= ,
   OPERATOR 5 > ,
-  FUNCTION 1 btint4cmp(int4, int4) ;
+  FUNCTION 1 btint4cmp(int4, int4),
+  FUNCTION 2 btint4sortsupport(int4, int4) ;
 
 CREATE OPERATOR CLASS int2_ops
 DEFAULT FOR TYPE int2 USING btree FAMILY integer_ops AS
diff --git a/src/backend/access/nbtree/nbtcompare.c b/src/backend/access/nbtree/nbtcompare.c
index 23f2b61..f95bc9f 100644
--- a/src/backend/access/nbtree/nbtcompare.c
+++ b/src/backend/access/nbtree/nbtcompare.c
@@ -49,6 +49,7 @@
 #include "postgres.h"
 
 #include "utils/builtins.h"
+#include "utils/sortsupport.h"
 
 
 Datum
@@ -83,6 +84,28 @@ btint4cmp(PG_FUNCTION_ARGS)
 		PG_RETURN_INT32(-1);
 }
 
+static int
+btint4acmp(Datum a, Datum b, SortSupportInfo ss)
+{
+	int32		a1 = Int32GetDatum(a);
+	int32		b1 = Int32GetDatum(b);
+
+	if (a1 > b1)
+		return 1;
+	else if (a1 == b1)
+		return 0;
+	else
+		return -1;
+}
+
+Datum
+btint4sortsupport(PG_FUNCTION_ARGS)
+{
+	SortSupportInfo	ss = (SortSupportInfo) PG_GETARG_POINTER(0);
+	ss->extra = btint4acmp;
+	PG_RETURN_VOID();
+}
+
 Datum
 btint8cmp(PG_FUNCTION_ARGS)
 {
diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h
index 347d942..b17855e 100644
--- a/src/include/access/nbtree.h
+++ b/src/include/access/nbtree.h
@@ -417,13 +417,17 @@ typedef struct xl_btree_newroot
 
 /*
  *	When a new operator class is declared, we require that the user
- *	supply us with an amproc procedure for determining whether, for
- *	two keys a and b, a < b, a = b, or a > b.  This routine must
- *	return < 0, 0, > 0, respectively, in these three cases.  Since we
- *	only have one such proc in amproc, it's number 1.
+ *	supply us with an amproc procedure (BTORDER_PROC) for determining
+ *  whether, for two keys a and b, a < b, a = b, or a > b.  This routine
+ *  must return < 0, 0, > 0, respectively, in these three cases.
+ *
+ *  To facilitate accelerated sorting, an operator class may choose to
+ *  offer a second procedure (BTSORTSUPPORT_PROC).  For full details, see
+ *  src/include/utils/sortsupport.h
  */
 
-#define BTORDER_PROC	1
+#define BTORDER_PROC			1
+#define BTSORTSUPPORT_PROC		2
 
 /*
  *	We need to be able to tell the difference between read and write
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 8b075d3..ddacdf2 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -117,7 +117,7 @@ typedef FormData_pg_am *Form_pg_am;
  * ----------------
  */
 
-DATA(insert OID = 403 (  btree	5 1 t f t t t t t t t f t t 0 btinsert btbeginscan btgettuple btgetbitmap btrescan btendscan btmarkpos btrestrpos btbuild btbuildempty btbulkdelete btvacuumcleanup btcostestimate btoptions ));
+DATA(insert OID = 403 (  btree	5 2 t f t t t t t t t f t t 0 btinsert btbeginscan btgettuple btgetbitmap btrescan btendscan btmarkpos btrestrpos btbuild btbuildempty btbulkdelete btvacuumcleanup btcostestimate btoptions ));
 DESCR("b-tree index access method");
 #define BTREE_AM_OID 403
 DATA(insert OID = 405 (  hash	1 1 f f t f f f f f f f f f 23 hashinsert hashbeginscan hashgettuple hashgetbitmap hashrescan hashendscan hashmarkpos hashrestrpos hashbuild hashbuildempty hashbulkdelete hashvacuumcleanup hashcostestimate hashoptions ));
diff --git a/src/include/catalog/pg_amproc.h b/src/include/catalog/pg_amproc.h
index e5d43f7..373191a 100644
--- a/src/include/catalog/pg_amproc.h
+++ b/src/include/catalog/pg_amproc.h
@@ -100,6 +100,7 @@ DATA(insert (	1976   21 21 1 350 ));
 DATA(insert (	1976   21 23 1 2190 ));
 DATA(insert (	1976   21 20 1 2192 ));
 DATA(insert (	1976   23 23 1 351 ));
+DATA(insert (	1976   23 23 2 324 ));
 DATA(insert (	1976   23 20 1 2188 ));
 DATA(insert (	1976   23 21 1 2191 ));
 DATA(insert (	1976   20 20 1 842 ));
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 28e53b7..e062b80 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -568,6 +568,8 @@ DATA(insert OID = 350 (  btint2cmp		   PGNSP PGUID 12 1 0 0 0 f f f t f i 2 0 23
 DESCR("less-equal-greater");
 DATA(insert OID = 351 (  btint4cmp		   PGNSP PGUID 12 1 0 0 0 f f f t f i 2 0 23 "23 23" _null_ _null_ _null_ _null_ btint4cmp _null_ _null_ _null_ ));
 DESCR("less-equal-greater");
+DATA(insert OID = 324 (  btint4sortsupport PGNSP PGUID 12 1 0 0 0 f f f t f i 1 0 2278 "2281" _null_ _null_ _null_ _null_ btint4sortsupport _null_ _null_ _null_ ));
+DESCR("sort support");
 DATA(insert OID = 842 (  btint8cmp		   PGNSP PGUID 12 1 0 0 0 f f f t f i 2 0 23 "20 20" _null_ _null_ _null_ _null_ btint8cmp _null_ _null_ _null_ ));
 DESCR("less-equal-greater");
 DATA(insert OID = 354 (  btfloat4cmp	   PGNSP PGUID 12 1 0 0 0 f f f t f i 2 0 23 "700 700" _null_ _null_ _null_ _null_ btfloat4cmp _null_ _null_ _null_ ));
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 47a1412..565e455 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -304,6 +304,13 @@ extern Datum btcharcmp(PG_FUNCTION_ARGS);
 extern Datum btnamecmp(PG_FUNCTION_ARGS);
 extern Datum bttextcmp(PG_FUNCTION_ARGS);
 
+/*
+ *		Per-opclass sort support functions for new btrees.  Like the
+ *		functions above, these are stored in pg_amproc and defined in
+ *		access/nbtree/nbtcompare.c
+ */
+extern Datum btint4sortsupport(PG_FUNCTION_ARGS);
+
 /* float.c */
 extern PGDLLIMPORT int extra_float_digits;
 
diff --git a/src/include/utils/sortsupport.h b/src/include/utils/sortsupport.h
new file mode 100644
index 0000000..a925ce1
--- /dev/null
+++ b/src/include/utils/sortsupport.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * sortsupport.h
+ *	  Framework for accelerated sorting.
+ *
+ * Sorting can be performed by repeatedly invoking an SQL-callable
+ * comparison function on the value to be compared; this module provides
+ * a way to avoid some of the function call overhead inherent in that
+ * interface.  A btree opclass may choose to offer a BTSORTSUPPORT_PROC,
+ * taking a single argument of type internal and returning void.
+ *
+ * This function will be called during sort setup and should initialize
+ * the extra and comparator functions of the SortSupportInfo structure
+ * that will be passed as an argument, using info_cxt for any required
+ * memory allocations.  The relevant collation OID will be initialized
+ * prior to invoking this function.
+ *
+ * After initialization, the comparator function will be called directly
+ * for each comparison, eliminating the overhead of the FunctionCallInfo
+ * interface.
+ *
+ * Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/utils/sortsupport.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef SORTSUPPORT_H
+#define SORTSUPPORT_H
+
+typedef struct SortSupportInfoData *SortSupportInfo;
+
+typedef struct SortSupportInfoData
+{
+	MemoryContext	info_cxt;
+	void		   *extra;
+	Oid				collation;
+
+	int	(*comparator)(Datum a, Datum b, SortSupportInfo info);
+} SortSupportInfoData;
+
+#endif   /* SORTSUPPORT_H */
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index b915134..1c8a634 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1186,40 +1186,28 @@ WHERE p1.amprocfamily = p3.oid AND p3.opfmethod = p2.oid AND
 
 -- Detect missing pg_amproc entries: should have as many support functions
 -- as AM expects for each datatype combination supported by the opfamily.
--- GiST/GIN are special cases because each has an optional support function.
+-- btree/GiST/GIN each allow an optional support function
 SELECT p1.amname, p2.opfname, p3.amproclefttype, p3.amprocrighttype
 FROM pg_am AS p1, pg_opfamily AS p2, pg_amproc AS p3
 WHERE p2.opfmethod = p1.oid AND p3.amprocfamily = p2.oid AND
-    p1.amname <> 'gist' AND p1.amname <> 'gin' AND
-    p1.amsupport != (SELECT count(*) FROM pg_amproc AS p4
-                     WHERE p4.amprocfamily = p2.oid AND
-                           p4.amproclefttype = p3.amproclefttype AND
-                           p4.amprocrighttype = p3.amprocrighttype);
- amname | opfname | amproclefttype | amprocrighttype 
---------+---------+----------------+-----------------
-(0 rows)
-
--- Similar check for GiST/GIN, allowing one optional proc
-SELECT p1.amname, p2.opfname, p3.amproclefttype, p3.amprocrighttype
-FROM pg_am AS p1, pg_opfamily AS p2, pg_amproc AS p3
-WHERE p2.opfmethod = p1.oid AND p3.amprocfamily = p2.oid AND
-    (p1.amname = 'gist' OR p1.amname = 'gin') AND
-    (SELECT count(*) FROM pg_amproc AS p4
-     WHERE p4.amprocfamily = p2.oid AND
-           p4.amproclefttype = p3.amproclefttype AND
-           p4.amprocrighttype = p3.amprocrighttype)
-      NOT IN (p1.amsupport, p1.amsupport - 1);
+    p1.amsupport - (SELECT count(*) FROM pg_amproc AS p4
+		WHERE p4.amprocfamily = p2.oid AND
+              p4.amproclefttype = p3.amproclefttype AND
+              p4.amprocrighttype = p3.amprocrighttype)
+	NOT BETWEEN 0 AND
+		CASE WHEN p1.amname IN ('gin', 'gist', 'btree') THEN 1 ELSE 0 END;
  amname | opfname | amproclefttype | amprocrighttype 
 --------+---------+----------------+-----------------
 (0 rows)
 
 -- Also, check if there are any pg_opclass entries that don't seem to have
--- pg_amproc support.  Again, GiST/GIN have to be checked specially.
+-- pg_amproc support.  Again, opclasses with an optional support proc have to be
+-- checked specially.
 SELECT amname, opcname, count(*)
 FROM pg_am am JOIN pg_opclass op ON opcmethod = am.oid
      LEFT JOIN pg_amproc p ON amprocfamily = opcfamily AND
          amproclefttype = amprocrighttype AND amproclefttype = opcintype
-WHERE am.amname <> 'gist' AND am.amname <> 'gin'
+WHERE am.amname <> 'gist' AND am.amname <> 'gin' AND am.amname <> 'btree'
 GROUP BY amname, amsupport, opcname, amprocfamily
 HAVING count(*) != amsupport OR amprocfamily IS NULL;
  amname | opcname | count 
@@ -1230,7 +1218,7 @@ SELECT amname, opcname, count(*)
 FROM pg_am am JOIN pg_opclass op ON opcmethod = am.oid
      LEFT JOIN pg_amproc p ON amprocfamily = opcfamily AND
          amproclefttype = amprocrighttype AND amproclefttype = opcintype
-WHERE am.amname = 'gist' OR am.amname = 'gin'
+WHERE am.amname = 'gist' OR am.amname = 'gin' OR am.amname = 'btree'
 GROUP BY amname, amsupport, opcname, amprocfamily
 HAVING (count(*) != amsupport AND count(*) != amsupport - 1)
     OR amprocfamily IS NULL;
@@ -1261,19 +1249,20 @@ WHERE p1.amprocfamily = p3.oid AND p4.amprocfamily = p6.oid AND
 (0 rows)
 
 -- For btree, though, we can do better since we know the support routines
--- must be of the form cmp(lefttype, righttype) returns int4.
+-- must be of the form cmp(lefttype, righttype) returns int4 or sortsupport(internal).
 SELECT p1.amprocfamily, p1.amprocnum,
 	p2.oid, p2.proname,
 	p3.opfname
 FROM pg_amproc AS p1, pg_proc AS p2, pg_opfamily AS p3
 WHERE p3.opfmethod = (SELECT oid FROM pg_am WHERE amname = 'btree')
     AND p1.amprocfamily = p3.oid AND p1.amproc = p2.oid AND
-    (amprocnum != 1
-     OR proretset
-     OR prorettype != 'int4'::regtype
-     OR pronargs != 2
-     OR proargtypes[0] != amproclefttype
-     OR proargtypes[1] != amprocrighttype);
+    (CASE WHEN amprocnum = 1
+          THEN prorettype != 'int4'::regtype OR pronargs != 2
+               OR proargtypes[0] != amproclefttype
+               OR proargtypes[1] != amprocrighttype
+          WHEN amprocnum = 2
+          THEN prorettype != 'void'::regtype OR pronargs != 1
+               OR proargtypes[0] != 'internal'::regtype END);
  amprocfamily | amprocnum | oid | proname | opfname 
 --------------+-----------+-----+---------+---------
 (0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index b0d1430..1146008 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -926,37 +926,27 @@ WHERE p1.amprocfamily = p3.oid AND p3.opfmethod = p2.oid AND
 
 -- Detect missing pg_amproc entries: should have as many support functions
 -- as AM expects for each datatype combination supported by the opfamily.
--- GiST/GIN are special cases because each has an optional support function.
+-- btree/GiST/GIN each allow an optional support function
 
 SELECT p1.amname, p2.opfname, p3.amproclefttype, p3.amprocrighttype
 FROM pg_am AS p1, pg_opfamily AS p2, pg_amproc AS p3
 WHERE p2.opfmethod = p1.oid AND p3.amprocfamily = p2.oid AND
-    p1.amname <> 'gist' AND p1.amname <> 'gin' AND
-    p1.amsupport != (SELECT count(*) FROM pg_amproc AS p4
-                     WHERE p4.amprocfamily = p2.oid AND
-                           p4.amproclefttype = p3.amproclefttype AND
-                           p4.amprocrighttype = p3.amprocrighttype);
-
--- Similar check for GiST/GIN, allowing one optional proc
-
-SELECT p1.amname, p2.opfname, p3.amproclefttype, p3.amprocrighttype
-FROM pg_am AS p1, pg_opfamily AS p2, pg_amproc AS p3
-WHERE p2.opfmethod = p1.oid AND p3.amprocfamily = p2.oid AND
-    (p1.amname = 'gist' OR p1.amname = 'gin') AND
-    (SELECT count(*) FROM pg_amproc AS p4
-     WHERE p4.amprocfamily = p2.oid AND
-           p4.amproclefttype = p3.amproclefttype AND
-           p4.amprocrighttype = p3.amprocrighttype)
-      NOT IN (p1.amsupport, p1.amsupport - 1);
+    p1.amsupport - (SELECT count(*) FROM pg_amproc AS p4
+		WHERE p4.amprocfamily = p2.oid AND
+              p4.amproclefttype = p3.amproclefttype AND
+              p4.amprocrighttype = p3.amprocrighttype)
+	NOT BETWEEN 0 AND
+		CASE WHEN p1.amname IN ('gin', 'gist', 'btree') THEN 1 ELSE 0 END;
 
 -- Also, check if there are any pg_opclass entries that don't seem to have
--- pg_amproc support.  Again, GiST/GIN have to be checked specially.
+-- pg_amproc support.  Again, opclasses with an optional support proc have to be
+-- checked specially.
 
 SELECT amname, opcname, count(*)
 FROM pg_am am JOIN pg_opclass op ON opcmethod = am.oid
      LEFT JOIN pg_amproc p ON amprocfamily = opcfamily AND
          amproclefttype = amprocrighttype AND amproclefttype = opcintype
-WHERE am.amname <> 'gist' AND am.amname <> 'gin'
+WHERE am.amname <> 'gist' AND am.amname <> 'gin' AND am.amname <> 'btree'
 GROUP BY amname, amsupport, opcname, amprocfamily
 HAVING count(*) != amsupport OR amprocfamily IS NULL;
 
@@ -964,7 +954,7 @@ SELECT amname, opcname, count(*)
 FROM pg_am am JOIN pg_opclass op ON opcmethod = am.oid
      LEFT JOIN pg_amproc p ON amprocfamily = opcfamily AND
          amproclefttype = amprocrighttype AND amproclefttype = opcintype
-WHERE am.amname = 'gist' OR am.amname = 'gin'
+WHERE am.amname = 'gist' OR am.amname = 'gin' OR am.amname = 'btree'
 GROUP BY amname, amsupport, opcname, amprocfamily
 HAVING (count(*) != amsupport AND count(*) != amsupport - 1)
     OR amprocfamily IS NULL;
@@ -990,7 +980,7 @@ WHERE p1.amprocfamily = p3.oid AND p4.amprocfamily = p6.oid AND
     (p2.proretset OR p5.proretset OR p2.pronargs != p5.pronargs);
 
 -- For btree, though, we can do better since we know the support routines
--- must be of the form cmp(lefttype, righttype) returns int4.
+-- must be of the form cmp(lefttype, righttype) returns int4 or sortsupport(internal).
 
 SELECT p1.amprocfamily, p1.amprocnum,
 	p2.oid, p2.proname,
@@ -998,12 +988,13 @@ SELECT p1.amprocfamily, p1.amprocnum,
 FROM pg_amproc AS p1, pg_proc AS p2, pg_opfamily AS p3
 WHERE p3.opfmethod = (SELECT oid FROM pg_am WHERE amname = 'btree')
     AND p1.amprocfamily = p3.oid AND p1.amproc = p2.oid AND
-    (amprocnum != 1
-     OR proretset
-     OR prorettype != 'int4'::regtype
-     OR pronargs != 2
-     OR proargtypes[0] != amproclefttype
-     OR proargtypes[1] != amprocrighttype);
+    (CASE WHEN amprocnum = 1
+          THEN prorettype != 'int4'::regtype OR pronargs != 2
+               OR proargtypes[0] != amproclefttype
+               OR proargtypes[1] != amprocrighttype
+          WHEN amprocnum = 2
+          THEN prorettype != 'void'::regtype OR pronargs != 1
+               OR proargtypes[0] != 'internal'::regtype END);
 
 -- For hash we can also do a little better: the support routines must be
 -- of the form hash(lefttype) returns int4.  There are several cases where
#75Bruce Momjian
bruce@momjian.us
In reply to: Robert Haas (#74)
Re: Inlining comparators as a performance optimisation

Robert Haas wrote:

On Fri, Dec 2, 2011 at 2:35 PM, Bruce Momjian <bruce@momjian.us> wrote:

Agreed. ?Doing something once and doing something in the sort loop are
two different overheads.

OK, so I tried to code this up. Adding the new amproc wasn't too
difficult (see attached). It wasn't obvious to me how to tie it into
the tuplesort infrastructure, though, so instead of wasting time
guessing what a sensible approach might be I'm going to use one of my
lifelines and poll the audience (or is that ask an expert?).
Currently the Tuplesortstate[1] has a pointer to an array of
ScanKeyData objects, one per column being sorted. But now instead of
"FmgrInfo sk_func", the tuplesort code is going to want each scankey
to contain "SortSupportInfo(Data?) sk_sortsupport"[2], because that's
where we get the comparison function from. Should I just go ahead
and add one more member to that struct, or is there some more
appropriate way to handle this?

Is this code immediately usable anywhere else in our codebasde, and if
so, is it generic enough?

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

#76Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#74)
Re: Inlining comparators as a performance optimisation

Robert Haas <robertmhaas@gmail.com> writes:

OK, so I tried to code this up. Adding the new amproc wasn't too
difficult (see attached). It wasn't obvious to me how to tie it into
the tuplesort infrastructure, though, so instead of wasting time
guessing what a sensible approach might be I'm going to use one of my
lifelines and poll the audience (or is that ask an expert?).

OK, I'll take a whack at improving this over the weekend.

regards, tom lane

#77Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#74)
Re: Inlining comparators as a performance optimisation

Robert Haas <robertmhaas@gmail.com> writes:

OK, so I tried to code this up. Adding the new amproc wasn't too
difficult (see attached). It wasn't obvious to me how to tie it into
the tuplesort infrastructure, though, so instead of wasting time
guessing what a sensible approach might be I'm going to use one of my
lifelines and poll the audience (or is that ask an expert?).

Here's another cut at this. I only went as far as converting the
heap-sort code path in tuplesort.c, so there's lots more to do, but
this does sort integers successfully. Before expanding the patch
to do more, I think we need to have consensus on some API details,
in particular:

* I invented a "SortKey" struct that replaces ScanKey for tuplesort's
purposes. Right now that's local in tuplesort.c, but we're more than
likely going to need it elsewhere as well. Should we just define it
in sortsupport.h? Or perhaps we should just add the few additional
fields to SortSupportInfoData, and not bother with two struct types?
Before you answer, consider the next point.

* I wonder whether it would be worthwhile to elide inlineApplyComparator
altogether, pushing what it does down to the level of the
datatype-specific functions. That would require changing the
"comparator" API to include isnull flags, and exposing the
reverse/nulls_first sort flags to the comparators (presumably by
including them in SortSupportInfoData). The main way in which that
could be a win would be if the setup function could choose one of four
comparator functions that are pre-specialized for each flags
combination; but that seems like it would add a lot of code bulk, and
the bigger problem is that we need to be able to change the flags after
sort initialization (cf. the reversedirection code in tuplesort.c), so
we'd also need some kind of "re-select the comparator" call API. On the
whole this doesn't seem promising, but maybe somebody else has a
different idea.

* We're going to want to expose PrepareSortSupportComparisonShim
for use outside tuplesort.c too, and possibly refactor
tuplesort_begin_heap so that the SortKey setup logic inside it
can be extracted for use elsewhere. Shall we just add those to
tuplesort's API, or would it be better to create a sortsupport.c
with these sorts of functions?

I have not done any performance testing on this patch, but it might be
interesting to check it with the same test cases Peter's been using.

regards, tom lane

#78Peter Geoghegan
peter@2ndquadrant.com
In reply to: Tom Lane (#77)
1 attachment(s)
Re: Inlining comparators as a performance optimisation

On 4 December 2011 19:17, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I have not done any performance testing on this patch, but it might be
interesting to check it with the same test cases Peter's been using.

I've attached a revision of exactly the same benchmark run to get the
results in results_server.ods .

You'll see very similar figures to results_server.ods for HEAD and for
my patch, as you'd expect. I think the results speak for themselves. I
maintain that we should use specialisations - that's where most of the
benefit is to be found.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

Attachments:

results_server_new.odsapplication/vnd.oasis.opendocument.spreadsheet; name=results_server_new.odsDownload
#79Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Peter Geoghegan (#78)
1 attachment(s)
Re: Inlining comparators as a performance optimisation

On 05.12.2011 02:14, Peter Geoghegan wrote:

On 4 December 2011 19:17, Tom Lane<tgl@sss.pgh.pa.us> wrote:

I have not done any performance testing on this patch, but it might be
interesting to check it with the same test cases Peter's been using.

I've attached a revision of exactly the same benchmark run to get the
results in results_server.ods .

You'll see very similar figures to results_server.ods for HEAD and for
my patch, as you'd expect. I think the results speak for themselves. I
maintain that we should use specialisations - that's where most of the
benefit is to be found.

I'm still not convinced the big gain is in inlining the comparison
functions. Your patch contains a few orthogonal tricks, too:

* A fastpath for the case of just one scankey. No reason we can't do
that without inlining.

* inlining the swap-functions within qsort. Thaẗ́'s different from
inlining the comparison functions, and could be done regardless of the
data type. The qsort element size in tuplesort.c is always sizeof(SortTuple)

And there's some additional specializations we can do, not implemented
in your patch, that don't depend on inlining the comparison functions:

* A fastpath for the case of no NULLs
* separate comparetup functions for ascending and descending sorts (this
allows tail call elimination of the call to type-specific comparison
function in the ascending version)
* call CHECK_FOR_INTERRUPTS() less frequently.

To see how much difference those things make, I hacked together the
attached patch. I didn't base this on Robert's/Tom's patch, but instead
just added a quick & dirty FunctionCallInvoke-overhead-skipping version
of btint4cmp. I believe that aspect of this patch it's similar in
performance characteristics, though.

In my quick tests, it gets quite close in performance to your inlining
patch, when sorting int4s and the input contains no NULLs. But please
give it a try in your test environment, to get numbers comparable with
your other tests.

Perhaps you can get even more gain by adding the no-NULLs, and asc/desc
special cases to your inlining patch, too, but let's squeeze all the fat
out of the non-inlined version first. One nice aspect of many of these
non-inlining optimizations is that they help with external sorts, too.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Attachments:

misc-qsort-optimizations-1.patchtext/x-diff; name=misc-qsort-optimizations-1.patchDownload
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 3505236..85bec20 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -114,6 +114,8 @@
 #include "utils/rel.h"
 #include "utils/tuplesort.h"
 
+#include "utils/builtins.h"
+
 
 /* sort-type codes for sort__start probes */
 #define HEAP_SORT		0
@@ -273,6 +275,8 @@ struct Tuplesortstate
 	int			memtupcount;	/* number of tuples currently present */
 	int			memtupsize;		/* allocated length of memtuples array */
 
+	bool		hasnulls_initial;	/* any NULLs in the first key col? */
+
 	/*
 	 * While building initial runs, this is the current output run number
 	 * (starting at 0).  Afterwards, it is the number of initial runs we made.
@@ -341,6 +345,8 @@ struct Tuplesortstate
 	TupleDesc	tupDesc;
 	ScanKey		scanKeys;		/* array of length nKeys */
 
+	int (*comparator) (Datum, Datum);
+
 	/*
 	 * These variables are specific to the CLUSTER case; they are set by
 	 * tuplesort_begin_cluster.  Note CLUSTER also uses tupDesc and
@@ -459,6 +465,11 @@ static unsigned int getlen(Tuplesortstate *state, int tapenum, bool eofOK);
 static void markrunend(Tuplesortstate *state, int tapenum);
 static int comparetup_heap(const SortTuple *a, const SortTuple *b,
 				Tuplesortstate *state);
+static int comparetup_heap_1key_asc_nonulls(const SortTuple *a, const SortTuple *b,
+				Tuplesortstate *state);
+static int comparetup_heap_1key_asc_nonulls_btint4cmp(const SortTuple *a, const SortTuple *b,
+				Tuplesortstate *state);
+static int compare_int4(Datum first, Datum second);
 static void copytup_heap(Tuplesortstate *state, SortTuple *stup, void *tup);
 static void writetup_heap(Tuplesortstate *state, int tapenum,
 			  SortTuple *stup);
@@ -551,6 +562,7 @@ tuplesort_begin_common(int workMem, bool randomAccess)
 	state->availMem = state->allowedMem;
 	state->sortcontext = sortcontext;
 	state->tapeset = NULL;
+	state->hasnulls_initial = false;
 
 	state->memtupcount = 0;
 	state->memtupsize = 1024;	/* initial guess */
@@ -1247,11 +1259,41 @@ tuplesort_performsort(Tuplesortstate *state)
 			 * amount of memory.  Just qsort 'em and we're done.
 			 */
 			if (state->memtupcount > 1)
+			{
+				int	(*comparetup) (const SortTuple *a, const SortTuple *b,
+								   Tuplesortstate *state);
+
+				/* If there is only one key col, the input contains no NULLs,
+				 * and it's ascending, we can use a fastpath version of
+				 * comparetup_heap that skips the over those things.
+				 */
+				if (state->comparetup == comparetup_heap &&
+					state->nKeys == 1 &&
+					!(state->scanKeys[0].sk_flags & SK_BT_DESC) &&
+					!state->hasnulls_initial)
+				{
+					/*
+					 * furthermore, if the comparison function happens to be
+					 * btint4cmp, we have an additional fastpath implementation
+					 * for that that skips the FunctionCallInvoke overhead
+					 */
+					if (state->scanKeys[0].sk_func.fn_addr == btint4cmp)
+					{
+						state->comparator = compare_int4;
+						comparetup = comparetup_heap_1key_asc_nonulls_btint4cmp;
+					}
+					else
+						comparetup = comparetup_heap_1key_asc_nonulls;
+				}
+				else
+					comparetup = state->comparetup;
+
 				qsort_arg((void *) state->memtuples,
 						  state->memtupcount,
 						  sizeof(SortTuple),
-						  (qsort_arg_comparator) state->comparetup,
+						  (qsort_arg_comparator) comparetup,
 						  (void *) state);
+			}
 			state->current = 0;
 			state->eof_reached = false;
 			state->markpos_offset = 0;
@@ -2713,6 +2755,44 @@ ApplySortFunction(FmgrInfo *sortFunction, int sortFlags, Oid collation,
  */
 
 static int
+compare_int4(Datum first, Datum second)
+{
+	int32           a = DatumGetInt32(first);
+	int32           b = DatumGetInt32(second);
+	if (a > b)
+		return 1;
+	else if (a == b)
+		return 0;
+	else
+		return -1;
+}
+
+static int
+comparetup_heap_1key_asc_nonulls(const SortTuple *a, const SortTuple *b, Tuplesortstate *state)
+{
+	ScanKey		scanKey = state->scanKeys;
+
+	/* Allow interrupting long sorts */
+	CHECK_FOR_INTERRUPTS();
+
+	/* Compare the sort key */
+	return DatumGetInt32(myFunctionCall2Coll(&scanKey->sk_func,
+											 scanKey->sk_collation,
+											 a->datum1, b->datum1));
+}
+
+
+static int
+comparetup_heap_1key_asc_nonulls_btint4cmp(const SortTuple *a, const SortTuple *b, Tuplesortstate *state)
+{
+	/* Allow interrupting long sorts */
+	CHECK_FOR_INTERRUPTS();
+
+	/* Compare the sort key */
+	return state->comparator(a->datum1, b->datum1);
+}
+
+static int
 comparetup_heap(const SortTuple *a, const SortTuple *b, Tuplesortstate *state)
 {
 	ScanKey		scanKey = state->scanKeys;
@@ -2784,6 +2864,10 @@ copytup_heap(Tuplesortstate *state, SortTuple *stup, void *tup)
 								state->scanKeys[0].sk_attno,
 								state->tupDesc,
 								&stup->isnull1);
+
+	/* update hasnulls_initial flag */
+	if (stup->isnull1)
+		state->hasnulls_initial = true;
 }
 
 static void
diff --git a/src/port/qsort_arg.c b/src/port/qsort_arg.c
index 28d1894..0fa0d75 100644
--- a/src/port/qsort_arg.c
+++ b/src/port/qsort_arg.c
@@ -99,6 +99,124 @@ med3(char *a, char *b, char *c, qsort_arg_comparator cmp, void *arg)
 		: (cmp(b, c, arg) > 0 ? b : (cmp(a, c, arg) < 0 ? a : c));
 }
 
+
+/*
+ * A specialized version for es == sizeof(SortTuple). We only use use the
+ * SortTuple struct here to get its size and alignment.
+ */
+typedef uintptr_t Datum; /* copied from postgres.h */
+typedef struct
+{
+	void	   *tuple;			/* the tuple proper */
+	Datum		datum1;			/* value of first key column */
+	bool		isnull1;		/* is first key column NULL? */
+	int32		tupindex;		/* see notes above */
+} SortTuple;
+
+
+#define swap_SortTuple(a, b)							\
+	do {												\
+		SortTuple tmp;									\
+		tmp = *(SortTuple *) (a);						\
+		*(SortTuple *) (a) = *(SortTuple *) (b);		\
+		*(SortTuple *) (b) = tmp;						\
+	} while(0)
+
+static void
+qsort_arg_SortTuple(void *a, size_t n, qsort_arg_comparator cmp, void *arg)
+{
+	size_t		es = sizeof(SortTuple);
+	char	   *pa,
+			   *pb,
+			   *pc,
+			   *pd,
+			   *pl,
+			   *pm,
+			   *pn;
+	int			d,
+				r,
+				swaptype,
+				presorted;
+
+loop:SWAPINIT(a, es);
+	if (n < 7)
+	{
+		for (pm = (char *) a + es; pm < (char *) a + n * es; pm += es)
+			for (pl = pm; pl > (char *) a && cmp(pl - es, pl, arg) > 0;
+				 pl -= es)
+				swap_SortTuple(pl, pl - es);
+		return;
+	}
+	presorted = 1;
+	for (pm = (char *) a + es; pm < (char *) a + n * es; pm += es)
+	{
+		if (cmp(pm - es, pm, arg) > 0)
+		{
+			presorted = 0;
+			break;
+		}
+	}
+	if (presorted)
+		return;
+	pm = (char *) a + (n / 2) * es;
+	if (n > 7)
+	{
+		pl = (char *) a;
+		pn = (char *) a + (n - 1) * es;
+		if (n > 40)
+		{
+			d = (n / 8) * es;
+			pl = med3(pl, pl + d, pl + 2 * d, cmp, arg);
+			pm = med3(pm - d, pm, pm + d, cmp, arg);
+			pn = med3(pn - 2 * d, pn - d, pn, cmp, arg);
+		}
+		pm = med3(pl, pm, pn, cmp, arg);
+	}
+	swap_SortTuple(a, pm);
+	pa = pb = (char *) a + es;
+	pc = pd = (char *) a + (n - 1) * es;
+	for (;;)
+	{
+		while (pb <= pc && (r = cmp(pb, a, arg)) <= 0)
+		{
+			if (r == 0)
+			{
+				swap_SortTuple(pa, pb);
+				pa += es;
+			}
+			pb += es;
+		}
+		while (pb <= pc && (r = cmp(pc, a, arg)) >= 0)
+		{
+			if (r == 0)
+			{
+				swap_SortTuple(pc, pd);
+				pd -= es;
+			}
+			pc -= es;
+		}
+		if (pb > pc)
+			break;
+		swap_SortTuple(pb, pc);
+		pb += es;
+		pc -= es;
+	}
+	pn = (char *) a + n * es;
+	r = Min(pa - (char *) a, pb - pa);
+	vecswap(a, pb - r, r);
+	r = Min(pd - pc, pn - pd - es);
+	vecswap(pb, pn - r, r);
+	if ((r = pb - pa) > es)
+		qsort_arg_SortTuple(a, r / es, cmp, arg);
+	if ((r = pd - pc) > es)
+	{
+		/* Iterate rather than recurse to save stack space */
+		a = pn - r;
+		n = r / es;
+		goto loop;
+	}
+}
+
 void
 qsort_arg(void *a, size_t n, size_t es, qsort_arg_comparator cmp, void *arg)
 {
@@ -114,6 +232,12 @@ qsort_arg(void *a, size_t n, size_t es, qsort_arg_comparator cmp, void *arg)
 				swaptype,
 				presorted;
 
+	if (es == sizeof(SortTuple))
+	{
+		qsort_arg_SortTuple(a, n, cmp, arg);
+		return;
+	}
+
 loop:SWAPINIT(a, es);
 	if (n < 7)
 	{
#80Peter Geoghegan
peter@2ndquadrant.com
In reply to: Heikki Linnakangas (#79)
1 attachment(s)
Re: Inlining comparators as a performance optimisation

On 5 December 2011 13:23, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:

I'm still not convinced the big gain is in inlining the comparison
functions. Your patch contains a few orthogonal tricks, too:

* A fastpath for the case of just one scankey. No reason we can't do that
without inlining.

True, though Tom did seem to not like that idea very much.

* inlining the swap-functions within qsort. Thaẗ́'s different from inlining
the comparison functions, and could be done regardless of the data type. The
qsort element size in tuplesort.c is always sizeof(SortTuple)

Sure. I think that Tom mostly objects to hard-coded intelligence about
which datatypes are used, rather than specialisations per se.

And there's some additional specializations we can do, not implemented in
your patch, that don't depend on inlining the comparison functions:

* A fastpath for the case of no NULLs
* separate comparetup functions for ascending and descending sorts (this
allows tail call elimination of the call to type-specific comparison
function in the ascending version)
* call CHECK_FOR_INTERRUPTS() less frequently.

All of those had occurred to me, except the last - if you look at the
definition of that macro, it looks like more trouble than it's worth
to do less frequently. I didn't revise my patch with them though,
because the difference that they made, while clearly measurable,
seemed fairly small, or at least relatively so. I wasn't about to add
additional kludge for marginal benefits given the existing
controversy, though I have not dismissed the idea entirely - it could
be important on some other machine.

Perhaps you can get even more gain by adding the no-NULLs, and asc/desc
special cases to your inlining patch, too, but let's squeeze all the fat out
of the non-inlined version first.

As I said, those things are simple enough to test, and were not found
to make a significant difference, at least to my exact test case +
hardware.

One nice aspect of many of these
non-inlining optimizations is that they help with external sorts, too.

I'd expect the benefits to be quite a lot smaller there, but fair point.

Results from running the same test on your patch are attached. I think
that while you're right to suggest that the inlining of the comparator
proper, rather than any other function or other optimisation isn't
playing a dominant role in these optimisations, I think that you're
underestimating the role of locality of reference. To illustrate this,
I've also included results for when I simply move the comparator to
another translation unit, logtape.c . There's clearly a regression
from doing so (I ran the numbers twice, in a clinical environment).
The point is, there is a basically unknown overhead paid for not
inlining - maybe inlining is not worth it, but that's a hard call to
make.

I didn't try to make the numbers look worse by putting the comparator
proper in some other translation unit, but it may be that I'd have
shown considerably worse numbers by doing so.

Why the strong aversion to what I've done? I accept that it's ugly,
but is it really worth worrying about that sort of regression in
maintainability for something that was basically untouched since 2006,
and will probably remain untouched after this work concludes, whatever
happens?

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

Attachments:

results_server_w_heikki.odsapplication/vnd.oasis.opendocument.spreadsheet; name=results_server_w_heikki.odsDownload
#81Tom Lane
tgl@sss.pgh.pa.us
In reply to: Peter Geoghegan (#80)
Re: Inlining comparators as a performance optimisation

Peter Geoghegan <peter@2ndquadrant.com> writes:

Why the strong aversion to what I've done? I accept that it's ugly,
but is it really worth worrying about that sort of regression in
maintainability for something that was basically untouched since 2006,
and will probably remain untouched after this work concludes, whatever
happens?

Maintainability is only part of the issue --- though it's definitely one
part, since this code has hardly gone "untouched since 2006", cf
http://git.postgresql.org/gitweb/?p=postgresql.git;a=history;f=src/backend/utils/sort/tuplesort.c;hb=HEAD

What is bothering me is that this approach is going to cause substantial
bloat of the executable code, and such bloat has got distributed costs,
which we don't have any really good way to measure but for sure
micro-benchmarks addressing only sort speed aren't going to reveal them.
Cache lines filled with sort code take cycles to flush and replace with
something else.

I think it's possibly reasonable to have inlined copies of qsort for a
small number of datatypes, but it seems much less reasonable to have
multiple copies per datatype in order to obtain progressively tinier
micro-benchmark speedups. We need to figure out what the tradeoff
against executable size really is, but so far it seems like you've been
ignoring the fact that there is any such tradeoff at all.

regards, tom lane

#82Nicolas Barbier
nicolas.barbier@gmail.com
In reply to: Tom Lane (#81)
Re: Inlining comparators as a performance optimisation

2011/12/5 Tom Lane <tgl@sss.pgh.pa.us>:

What is bothering me is that this approach is going to cause substantial
bloat of the executable code, and such bloat has got distributed costs,
which we don't have any really good way to measure but for sure
micro-benchmarks addressing only sort speed aren't going to reveal them.
Cache lines filled with sort code take cycles to flush and replace with
something else.

I think it's possibly reasonable to have inlined copies of qsort for a
small number of datatypes, but it seems much less reasonable to have
multiple copies per datatype in order to obtain progressively tinier
micro-benchmark speedups.  We need to figure out what the tradeoff
against executable size really is, but so far it seems like you've been
ignoring the fact that there is any such tradeoff at all.

[ Randomly spouting ideas here: ]

Might it not be a good idea to decide whether to use the inlined
copies vs. the normal version, based on how much data to sort? Surely
for a very large sort, the cache blow-out doesn't matter that much
relative to the amount of time that can be saved sorting?

Assuming that all types would have an inlined sort function, although
that would indeed result in a larger binary, most of that binary would
never touch the cache, because corresponding large sorts are never
performed. If they would sporadically occur (and assuming the points
at which inline sorting starts to get used are chosen wisely), the
possibly resulting cache blow-out would be a net win.

I am also assuming here that instruction cache lines are small enough
for case line aliasing not to become a problem; putting all sort
functions next to each other in the binary (so that they don't alias
with the rest of the backend code that might be used more often) might
alleviate that.

Nicolas

--
A. Because it breaks the logical sequence of discussion.
Q. Why is top posting bad?

#83Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#77)
Re: Inlining comparators as a performance optimisation

On Sun, Dec 4, 2011 at 2:17 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

* I invented a "SortKey" struct that replaces ScanKey for tuplesort's
purposes.  Right now that's local in tuplesort.c, but we're more than
likely going to need it elsewhere as well.  Should we just define it
in sortsupport.h?  Or perhaps we should just add the few additional
fields to SortSupportInfoData, and not bother with two struct types?
Before you answer, consider the next point.

+1 for not bothering with two struct types. We might want to consider
calling the resulting structure SortKey rather than
SortSupportInfoData, however.

* I wonder whether it would be worthwhile to elide inlineApplyComparator
altogether, pushing what it does down to the level of the
datatype-specific functions.  That would require changing the
"comparator" API to include isnull flags, and exposing the
reverse/nulls_first sort flags to the comparators (presumably by
including them in SortSupportInfoData).  The main way in which that
could be a win would be if the setup function could choose one of four
comparator functions that are pre-specialized for each flags
combination; but that seems like it would add a lot of code bulk, and
the bigger problem is that we need to be able to change the flags after
sort initialization (cf. the reversedirection code in tuplesort.c), so
we'd also need some kind of "re-select the comparator" call API.  On the
whole this doesn't seem promising, but maybe somebody else has a
different idea.

I thought about this, too, but it didn't seem promising to me, either.

* We're going to want to expose PrepareSortSupportComparisonShim
for use outside tuplesort.c too, and possibly refactor
tuplesort_begin_heap so that the SortKey setup logic inside it
can be extracted for use elsewhere.  Shall we just add those to
tuplesort's API, or would it be better to create a sortsupport.c
with these sorts of functions?

Why are we going to want to do that? If it's because there are other
places in the code that can make use of a fast comparator that don't
go through tuplesort.c, then we should probably break it off into a
separate file (sortkey.c?). But if it's because we think that clients
of the tuplesort code are going to need it for some reason, then we
may as well keep it in tuplesort.c.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#84Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#83)
Re: Inlining comparators as a performance optimisation

Robert Haas <robertmhaas@gmail.com> writes:

On Sun, Dec 4, 2011 at 2:17 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

* We're going to want to expose PrepareSortSupportComparisonShim
for use outside tuplesort.c too, and possibly refactor
tuplesort_begin_heap so that the SortKey setup logic inside it
can be extracted for use elsewhere. Shall we just add those to
tuplesort's API, or would it be better to create a sortsupport.c
with these sorts of functions?

Why are we going to want to do that? If it's because there are other
places in the code that can make use of a fast comparator that don't
go through tuplesort.c, then we should probably break it off into a
separate file (sortkey.c?). But if it's because we think that clients
of the tuplesort code are going to need it for some reason, then we
may as well keep it in tuplesort.c.

My expectation is that nbtree, as well as mergejoin and mergeappend,
would get converted over to use the fast comparator API. I looked at
that a little bit but didn't push it far enough to be very sure about
whether they'd be able to share the initialization code from
tuplesort_begin_heap. But they're definitely going to need the shim
function for backwards compatibility, and
PrepareSortSupportComparisonShim was my first cut at a wrapper that
would be generally useful.

regards, tom lane

#85Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#84)
Re: Inlining comparators as a performance optimisation

On Tue, Dec 6, 2011 at 12:06 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Sun, Dec 4, 2011 at 2:17 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

* We're going to want to expose PrepareSortSupportComparisonShim
for use outside tuplesort.c too, and possibly refactor
tuplesort_begin_heap so that the SortKey setup logic inside it
can be extracted for use elsewhere.  Shall we just add those to
tuplesort's API, or would it be better to create a sortsupport.c
with these sorts of functions?

Why are we going to want to do that?  If it's because there are other
places in the code that can make use of a fast comparator that don't
go through tuplesort.c, then we should probably break it off into a
separate file (sortkey.c?).  But if it's because we think that clients
of the tuplesort code are going to need it for some reason, then we
may as well keep it in tuplesort.c.

My expectation is that nbtree, as well as mergejoin and mergeappend,
would get converted over to use the fast comparator API.  I looked at
that a little bit but didn't push it far enough to be very sure about
whether they'd be able to share the initialization code from
tuplesort_begin_heap.  But they're definitely going to need the shim
function for backwards compatibility, and
PrepareSortSupportComparisonShim was my first cut at a wrapper that
would be generally useful.

OK. Well, then pushing it out to a separate file probably makes
sense. Do you want to do that or shall I have a crack at it? If the
latter, what do you think about using the name SortKey for everything
rather than SortSupport?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#86Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#85)
Re: Inlining comparators as a performance optimisation

Robert Haas <robertmhaas@gmail.com> writes:

OK. Well, then pushing it out to a separate file probably makes
sense. Do you want to do that or shall I have a crack at it? If the
latter, what do you think about using the name SortKey for everything
rather than SortSupport?

I'll take another crack at it. I'm not entirely sold yet on merging
the two structs; I think first we'd better look and see what the needs
are in the other potential callers I mentioned. If we'd end up
cluttering the struct with half a dozen weird fields, it'd be better to
stick to a minimal interface struct with various wrapper structs, IMO.

OTOH it did seem that the names were getting a bit long. If we do
keep the two-struct-levels approach, what do you think of
s/SortSupportInfo/SortSupport/g ?

regards, tom lane

#87Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#86)
Re: Inlining comparators as a performance optimisation

On Tue, Dec 6, 2011 at 1:07 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

OK.  Well, then pushing it out to a separate file probably makes
sense.  Do you want to do that or shall I have a crack at it?  If the
latter, what do you think about using the name SortKey for everything
rather than SortSupport?

I'll take another crack at it.  I'm not entirely sold yet on merging
the two structs; I think first we'd better look and see what the needs
are in the other potential callers I mentioned.  If we'd end up
cluttering the struct with half a dozen weird fields, it'd be better to
stick to a minimal interface struct with various wrapper structs, IMO.

OK. I'll defer to whatever you come up with after looking at it.

OTOH it did seem that the names were getting a bit long.  If we do
keep the two-struct-levels approach, what do you think of
s/SortSupportInfo/SortSupport/g ?

+1. I had that thought when you originally suggested that name, but
it didn't seem worth arguing about.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#88Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#87)
Re: Inlining comparators as a performance optimisation

Robert Haas <robertmhaas@gmail.com> writes:

On Tue, Dec 6, 2011 at 1:07 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I'll take another crack at it. I'm not entirely sold yet on merging
the two structs; I think first we'd better look and see what the needs
are in the other potential callers I mentioned. If we'd end up
cluttering the struct with half a dozen weird fields, it'd be better to
stick to a minimal interface struct with various wrapper structs, IMO.

OK. I'll defer to whatever you come up with after looking at it.

OK, it looks like nodeMergeAppend.c could use something exactly like the
draft SortKey struct, while nodeMergejoin.c could embed such a struct in
MergeJoinClauseData. The btree stuff needs something more nearly
equivalent to a ScanKey, including a datum-to-compare-to and a flags
field. I'm inclined to think the latter would be too specialized to put
in the generic struct. On the other hand, including the reverse and
nulls_first flags in the generic struct is clearly a win since it allows
ApplyComparator() to be defined as a generic function. So the only
thing that's really debatable is the attno field, and I'm not anal
enough to insist on a separate level of struct just for that.

I am however inclined to stick with the shortened struct name SortSupport
rather than using SortKey. The presence of the function pointer fields
(especially the inlined-qsort pointers, assuming we adopt some form of
Peter's patch) changes the struct's nature in my view; it's not really
describing just a sort key (ie an ORDER BY column specification).

regards, tom lane

#89Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#88)
Re: Inlining comparators as a performance optimisation

On Tue, Dec 6, 2011 at 4:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Tue, Dec 6, 2011 at 1:07 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I'll take another crack at it.  I'm not entirely sold yet on merging
the two structs; I think first we'd better look and see what the needs
are in the other potential callers I mentioned.  If we'd end up
cluttering the struct with half a dozen weird fields, it'd be better to
stick to a minimal interface struct with various wrapper structs, IMO.

OK.  I'll defer to whatever you come up with after looking at it.

OK, it looks like nodeMergeAppend.c could use something exactly like the
draft SortKey struct, while nodeMergejoin.c could embed such a struct in
MergeJoinClauseData.  The btree stuff needs something more nearly
equivalent to a ScanKey, including a datum-to-compare-to and a flags
field.  I'm inclined to think the latter would be too specialized to put
in the generic struct.  On the other hand, including the reverse and
nulls_first flags in the generic struct is clearly a win since it allows
ApplyComparator() to be defined as a generic function.  So the only
thing that's really debatable is the attno field, and I'm not anal
enough to insist on a separate level of struct just for that.

I am however inclined to stick with the shortened struct name SortSupport
rather than using SortKey.  The presence of the function pointer fields
(especially the inlined-qsort pointers, assuming we adopt some form of
Peter's patch) changes the struct's nature in my view; it's not really
describing just a sort key (ie an ORDER BY column specification).

Works for me. I think we should go ahead and get this part committed
first, and then we can look at the inlining stuff as a further
optimization for certain cases...

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#90Peter Geoghegan
peter@2ndquadrant.com
In reply to: Robert Haas (#89)
Re: Inlining comparators as a performance optimisation

On 7 December 2011 00:18, Robert Haas <robertmhaas@gmail.com> wrote:

Works for me.  I think we should go ahead and get this part committed
first, and then we can look at the inlining stuff as a further
optimization for certain cases...

Do you mean just inlining, or inlining and the numerous other
optimisations that my patch had?

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

#91Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#89)
Re: Inlining comparators as a performance optimisation

Robert Haas <robertmhaas@gmail.com> writes:

Works for me. I think we should go ahead and get this part committed
first, and then we can look at the inlining stuff as a further
optimization for certain cases...

Yeah. Attached is a second-draft patch, which I'm fairly happy with
except that the documentation hasn't been updated. It extends the
changes into the executor as well as analyze.c, with the result that
we can get rid of some of the old infrastructure altogether.
(inlineApplySortFunction is still there for the moment, though.)
Also, I adopted a style similar to what we've done for inlined list
functions to make ApplyComparatorFunction inline-able by all callers.

There are three somewhat-independent lines of work to pursue from here:

1. Adding sortsupport infrastructure for more datatypes.
2. Revising nbtree and related code to use this infrastructure.
3. Integrating Peter's work into this framework.

I'll try to take care of #1 for at least a few key datatypes before
I commit, but I think #2 is best done as a separate patch, so I'll
postpone that till later.

regards, tom lane

#92Robert Haas
robertmhaas@gmail.com
In reply to: Peter Geoghegan (#90)
Re: Inlining comparators as a performance optimisation

On Tue, Dec 6, 2011 at 8:13 PM, Peter Geoghegan <peter@2ndquadrant.com> wrote:

On 7 December 2011 00:18, Robert Haas <robertmhaas@gmail.com> wrote:

Works for me.  I think we should go ahead and get this part committed
first, and then we can look at the inlining stuff as a further
optimization for certain cases...

Do you mean just inlining, or inlining and the numerous other
optimisations that my patch had?

Whichever you like. But I think part of the point here is to
disentangle those optimizations from each other and decide how broadly
it makes sense to apply each one. Avoiding the FunctionCallInfo stuff
is one, and it seems like we can apply that to a wide variety of data
types (maybe all of them) for both in-memory and on-disk sorting, plus
btree index ops, merge joins, and merge append. The gains may be
modest, but they will benefit many use cases. Your original patch
targets a much narrower use case (in-memory sorting of POD types) but
the benefits are larger. We don't have to pick between a general but
small optimization and a narrower but larger one; we can do both.

In this regard, I think Heikki's remarks upthread are worth some
thought. If inlining is a win just because it avoids saving and
restoring registers or allows better instruction scheduling, then
inlining is the (probably?) the only way to get the benefit. But if
most of the benefit is in having a separate path for the
single-sort-key case, we can do that without duplicating the qsort()
code and get the benefit for every data type without much code bloat.
I'd like to see us dig into that a little, so that we get the broadest
possible benefit out of this work. It doesn't bother me that not
every optimization will apply to every case, and I don't object to
optimizations that are intrinsically narrow (within some reasonable
limits). But I'd rather not take what could be a fairly broad-based
optimization and apply it only narrowly, all things being equal.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#93Peter Geoghegan
peter@2ndquadrant.com
In reply to: Robert Haas (#92)
Re: Inlining comparators as a performance optimisation

On 7 December 2011 03:45, Robert Haas <robertmhaas@gmail.com> wrote:

In this regard, I think Heikki's remarks upthread are worth some
thought.  If inlining is a win just because it avoids saving and
restoring registers or allows better instruction scheduling, then
inlining is the (probably?) the only way to get the benefit.  But if
most of the benefit is in having a separate path for the
single-sort-key case, we can do that without duplicating the qsort()
code and get the benefit for every data type without much code bloat.
I'd like to see us dig into that a little, so that we get the broadest
possible benefit out of this work.  It doesn't bother me that not
every optimization will apply to every case, and I don't object to
optimizations that are intrinsically narrow (within some reasonable
limits).  But I'd rather not take what could be a fairly broad-based
optimization and apply it only narrowly, all things being equal.

I think we're in agreement then.

It's important to realise, though I'm sure you do, that once you've
done what I did with sorting single scanKey integers, that's it -
there isn't really any way that I can think of to speed up quick
sorting further, other than parallelism which has major challenges,
and isn't particularly appropriate at this granularity. The real
significance of inlining is, well, that's it, that's all there is -
after inlining, I predict that the well will run dry, or practically
dry. Inlining actually is a pretty great win when you look at sorting
in isolation, but it's just that there's been a number of other
significant wins in my patch, and of course there is overhead from a
number of other areas, so perhaps it's easy to lose sight of that.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

#94Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#91)
Re: Inlining comparators as a performance optimisation

On Tue, Dec 6, 2011 at 8:46 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

1. Adding sortsupport infrastructure for more datatypes.
2. Revising nbtree and related code to use this infrastructure.
3. Integrating Peter's work into this framework.

I'll try to take care of #1 for at least a few key datatypes before
I commit, but I think #2 is best done as a separate patch, so I'll
postpone that till later.

I see you've committed a chunk of this now. Does it make sense to do
#1 for every data type we support, or should we be more selective than
that? My gut feeling would be to add it across the board and
introduce an opr_sanity check for it. The general utility of adding
it to deprecated types like abstime is perhaps questionable, but it
strikes me that the value of making everything consistent probably
outweighs the cost of a few extra lines of code.

Are you planning to do anything about #2 or #3?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#95Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#94)
Re: Inlining comparators as a performance optimisation

Robert Haas <robertmhaas@gmail.com> writes:

On Tue, Dec 6, 2011 at 8:46 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

1. Adding sortsupport infrastructure for more datatypes.
2. Revising nbtree and related code to use this infrastructure.
3. Integrating Peter's work into this framework.

I'll try to take care of #1 for at least a few key datatypes before
I commit, but I think #2 is best done as a separate patch, so I'll
postpone that till later.

I see you've committed a chunk of this now. Does it make sense to do
#1 for every data type we support, or should we be more selective than
that?

Basically, I tried to do #1 for every datatype for which the comparator
was cheap enough that reducing the call overhead seemed likely to make a
useful difference. I'm not in favor of adding sortsupport functions
where this is not true, as I think it'll be useless code and catalog
bloat. I don't want to add 'em for cruft like abstime either.

There's some stuff that's debatable according to this criterion --- in
particular, I wondered whether it'd be worth having a fast path for
bttextcmp, especially if we pre-tested the collate_is_c condition and
had a separate version that just hardwired the memcmp code path. (The
idea of doing that was one reason I insisted on collation being known at
the setup step.) But it would still have to be prepared for detoasting,
so in the end I was unenthused. Anyone who feels like testing could try
to prove me wrong about it though.

Are you planning to do anything about #2 or #3?

I am willing to do #2, but not right now; I feel what I need to do next
is go review SPGist. I don't believe that #2 blocks progress on #3
anyway. I think #3 is in Peter's court, or yours if you want to do it.

(BTW, I agree with your comments yesterday about trying to break down
the different aspects of what Peter did, and put as many of them as we
can into the non-inlined code paths.)

regards, tom lane

#96Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#95)
Re: Inlining comparators as a performance optimisation

On Wed, Dec 7, 2011 at 10:09 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

There's some stuff that's debatable according to this criterion --- in
particular, I wondered whether it'd be worth having a fast path for
bttextcmp, especially if we pre-tested the collate_is_c condition and
had a separate version that just hardwired the memcmp code path.  (The
idea of doing that was one reason I insisted on collation being known at
the setup step.)  But it would still have to be prepared for detoasting,
so in the end I was unenthused.  Anyone who feels like testing could try
to prove me wrong about it though.

I think that'd definitely be worth investigating (although I'm not
sure I have the time to do it myself any time real soon).

Are you planning to do anything about #2 or #3?

I am willing to do #2, but not right now; I feel what I need to do next
is go review SPGist.

Yeah, makes sense. That one seems likely to be a challenge to absorb.

I don't believe that #2 blocks progress on #3
anyway.  I think #3 is in Peter's court, or yours if you want to do it.

(BTW, I agree with your comments yesterday about trying to break down
the different aspects of what Peter did, and put as many of them as we
can into the non-inlined code paths.)

Cool. Peter, can you rebase your patch and integrate it into the
sortsupport framework that's now committed?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#97Peter Geoghegan
peter@2ndquadrant.com
In reply to: Robert Haas (#96)
Re: Inlining comparators as a performance optimisation

On 7 December 2011 15:15, Robert Haas <robertmhaas@gmail.com> wrote:

On Wed, Dec 7, 2011 at 10:09 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

 But it would still have to be prepared for detoasting,
so in the end I was unenthused.  Anyone who feels like testing could try
to prove me wrong about it though.

I think that'd definitely be worth investigating (although I'm not
sure I have the time to do it myself any time real soon).

I'll at least take a look at it. Sorting text is a fairly common case.
I'm not hugely enthused about spending too much time on work that will
only be useful if collate_is_c.

I don't believe that #2 blocks progress on #3
anyway.  I think #3 is in Peter's court, or yours if you want to do it.

(BTW, I agree with your comments yesterday about trying to break down
the different aspects of what Peter did, and put as many of them as we
can into the non-inlined code paths.)

I'm confident that we should have everything for the simple case of
ordering by a single int4 and int8 column, and I think you'd probably
agree with that - they're extremely common cases. Anything beyond that
will need to be justified, probably in part by running additional
benchmarks.

Cool.  Peter, can you rebase your patch and integrate it into the
sortsupport framework that's now committed?

Yes, I'd be happy to, though I don't think I'm going to be getting
around to it this side of Friday. Since it isn't a blocker, I assume
that's okay.

The rebased revision will come complete with a well thought-out
rationale for my use of inlining specialisations, that takes account
of the trade-off against binary bloat that Tom highlighted. I wasn't
ignoring that issue, but I did fail to articulate my thoughts there,
mostly because I felt the need to do some additional research to
justify my position.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

#98Robert Haas
robertmhaas@gmail.com
In reply to: Peter Geoghegan (#97)
Re: Inlining comparators as a performance optimisation

On Wed, Dec 7, 2011 at 10:58 AM, Peter Geoghegan <peter@2ndquadrant.com> wrote:

On 7 December 2011 15:15, Robert Haas <robertmhaas@gmail.com> wrote:

On Wed, Dec 7, 2011 at 10:09 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

 But it would still have to be prepared for detoasting,
so in the end I was unenthused.  Anyone who feels like testing could try
to prove me wrong about it though.

I think that'd definitely be worth investigating (although I'm not
sure I have the time to do it myself any time real soon).

I'll at least take a look at it. Sorting text is a fairly common case.
I'm not hugely enthused about spending too much time on work that will
only be useful if collate_is_c.

Well, text_pattern_ops is not exactly an uncommon case, but sure. And
there might be some benefit for other collations, too.

Cool.  Peter, can you rebase your patch and integrate it into the
sortsupport framework that's now committed?

Yes, I'd be happy to, though I don't think I'm going to be getting
around to it this side of Friday. Since it isn't a blocker, I assume
that's okay.

Sounds fine to me.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#99Greg Smith
greg@2ndQuadrant.com
In reply to: Robert Haas (#98)
Re: Inlining comparators as a performance optimisation

I think we can call a new sorting infrastructure popping out and what
looks to be over 90 messages on this topic as successful progress on
this front. Peter's going to rev a new patch, but with more performance
results to review and followup discussion I can't see this one as
wrapping for the current CommitFest. Marking it returned and we'll
return to this topic during or before the next CF. With the majority of
the comments here coming from committers, I think continuing progress in
that direction won't be a problem.

--
Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us

#100Pierre C
lists@peufeu.com
In reply to: Tom Lane (#23)
Re: Inlining comparators as a performance optimisation

On Wed, 21 Sep 2011 18:13:07 +0200, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:

On 21.09.2011 18:46, Tom Lane wrote:

The idea that I was toying with was to allow the regular SQL-callable
comparison function to somehow return a function pointer to the
alternate comparison function,

You could have a new function with a pg_proc entry, that just returns a
function pointer to the qsort-callback.

Yeah, possibly. That would be a much more invasive change, but cleaner
in some sense. I'm not really prepared to do all the legwork involved
in that just to get to a performance-testable patch though.

A few years ago I had looked for a way to speed up COPY operations, and it
turned out that COPY TO has a good optimization opportunity. At that time,
for each datum, COPY TO would :

- test for nullness
- call an outfunc through fmgr
- outfunc pallocs() a bytea or text, fills it with data, and returns it
(sometimes it uses an extensible string buffer which may be repalloc()d
several times)
- COPY memcpy()s returned data to a buffer and eventually flushes the
buffer to client socket.

I introduced a special write buffer with an on-flush callback (ie, a close
relative of the existing string-buffer), in this case the callback was
"flush to client socket", and several outfuncs (one per type) which took
that buffer as argument, besides the datum to output, and simply put the
datum inside the buffer, with appropriate transformations (like converting
to bytea or text), and flushed if needed.

Then the COPY TO BINARY of a constant-size datum would turn to :
- one test for nullness
- one C function call
- one test to ensure appropriate space available in buffer (flush if
needed)
- one htonl() and memcpy of constant size, which the compiler turns out
into a couple of simple instructions

I recall measuring speedups of 2x - 8x on COPY BINARY, less for text, but
still large gains.

Although eliminating fmgr call and palloc overhead was an important part
of it, another large part was getting rid of memcpy()'s which the compiler
turned into simple movs for known-size types, a transformation that can be
done only if the buffer write functions are inlined inside the outfuncs.
Compilers love constants...

Additionnally, code size growth was minimal since I moved the old outfuncs
code into the new outfuncs, and replaced the old fmgr-callable outfuncs
with "create buffer with on-full callback=extend_and_repalloc() - pass to
new outfunc(buffer,datum) - return buffer". Which is basically equivalent
to the previous palloc()-based code, maybe with a few extra instructions.

When I submitted the patch for review, Tom rightfully pointed out that my
way of obtaining the C function pointer sucked very badly (I don't
remember how I did it, only that it was butt-ugly) but the idea was to get
a quick measurement of what could be gained, and the result was positive.
Unfortunately I had no time available to finish it and make it into a real
patch, I'm sorry about that.

So why do I post in this sorting topic ? It seems, by bypassing fmgr for
functions which are small, simple, and called lots of times, there is a
large gain to be made, not only because of fmgr overhead but also because
of the opportunity for new compiler optimizations, palloc removal, etc.
However, in my experiment the arguments and return types of the new
functions were DIFFERENT from the old functions : the new ones do the same
thing, but in a different manner. One manner was suited to sql-callable
functions (ie, palloc and return a bytea) and another one to writing large
amounts of data (direct buffer write). Since both have very different
requirements, being fast at both is impossible for the same function.

Anyway, all that rant boils down to :

Some functions could benefit having two versions (while sharing almost all
the code between them) :
- User-callable (fmgr) version (current one)
- C-callable version, usually with different parameters and return type

And it would be cool to have a way to grab a bare function pointer on the
second one.

Maybe an extra column in pg_proc would do (but then, the proargtypes and
friends would describe only the sql-callable version) ?
Or an extra table ? pg_cproc ?
Or an in-memory hash : hashtable[ fmgr-callable function ] => C version
- What happens if a C function has no SQL-callable equivalent ?
Or (ugly) introduce an extra per-type function type_get_function_ptr(
function_kind ) which returns the requested function ptr

If one of those happens, I'll dust off my old copy-optimization patch ;)

Hmm... just my 2c

Regards
Pierre

#101Bruce Momjian
bruce@momjian.us
In reply to: Pierre C (#100)
Re: Inlining comparators as a performance optimisation

On Fri, Jan 13, 2012 at 10:48:56AM +0100, Pierre C wrote:

Maybe an extra column in pg_proc would do (but then, the proargtypes
and friends would describe only the sql-callable version) ?
Or an extra table ? pg_cproc ?
Or an in-memory hash : hashtable[ fmgr-callable function ] => C version
- What happens if a C function has no SQL-callable equivalent ?
Or (ugly) introduce an extra per-type function
type_get_function_ptr( function_kind ) which returns the requested
function ptr

If one of those happens, I'll dust off my old copy-optimization patch ;)

I agree that COPY is ripe for optimization, and I am excited you have
some ideas on this.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +