[WIP] Zipfian distribution in pgbench

Started by Alik Khilazhevover 8 years ago57 messages
#1Alik Khilazhev
a.khilazhev@postgrespro.ru
3 attachment(s)

Hello!

PostgreSQL shows very bad results in YCSB Workload A (50% SELECT and 50% UPDATE of random row by PK) on benchmarking with big number of clients using Zipfian distribution. MySQL also has decline but it is not significant as it is in PostgreSQL. MongoDB does not have decline at all. And if pgbench would have Zipfian distribution random number generator, everyone will be able to make research on this topic without using YCSB.

This is the reason why I am currently working on random_zipfian function.

The bottleneck of algorithm that I use is that it calculates zeta function (it has linear complexity - https://en.wikipedia.org/wiki/Riemann_zeta_function). It my cause problems on generating huge amount of big numbers.

That’s why I added caching for zeta value. And it works good for cases when random_zipfian called with same parameters in script. For example:


\set a random_zipfian(1, 100, 1.2)
\set b random_zipfian(1, 100, 1.2)

In other case, second call will override cache of first and caching does not make any sense:

\set a random_zipfian(1, 100, 1.2)
\set b random_zipfian(1, 200, 1.4)

That’s why I have a question: should I implement support of caching zeta values for calls with different parameters, or not?

P.S. I attaching patch and script - analogue of YCSB Workload A.
Run benchmark with command:
$ pgbench -f ycsb_read_zipf.sql -f ycsb_update_zipf.sql

On scale = 10(1 million rows) it gives following results on machine with 144 cores(with synchronous_commit=off):
nclients tps
1 8842.401870
2 18358.140869
4 45999.378785
8 88713.743199
16 170166.998212
32 290069.221493
64 178128.030553
128 88712.825602
256 38364.937573
512 13512.765878
1000 6188.136736

Attachments:

ycsb_read_zipf.sqlapplication/sql; name=ycsb_read_zipf.sql; x-unix-mode=0644Download
pgbench-zipf-01v.patchapplication/octet-stream; name=pgbench-zipf-01v.patch; x-unix-mode=0644Download
diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml
index 64b043b48a..1dea6e4b17 100644
--- a/doc/src/sgml/ref/pgbench.sgml
+++ b/doc/src/sgml/ref/pgbench.sgml
@@ -1014,6 +1014,14 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
        <entry>an integer between <literal>1</> and <literal>10</></>
       </row>
       <row>
+        <entry><literal><function>random_zipfian(<replaceable>lb</>, <replaceable>ub</>, <replaceable>parameter</>)</></></>
+        <entry>integer</>
+        <entry> Zipfian-distributed random integer in <literal>[lb, ub]</>,
+              see below</>
+        <entry><literal>random_zipfian(1, 10, 1.2)</></>
+        <entry>an integer between <literal>1</> and <literal>10</></>
+      </row>
+      <row>
        <entry><literal><function>sqrt(<replaceable>x</>)</></></>
        <entry>double</>
        <entry>square root</>
@@ -1094,6 +1102,24 @@ f(x) = PHI(2.0 * parameter * (x - mu) / (max - min + 1)) /
       of the Box-Muller transform.
      </para>
     </listitem>
+
+    <listitem>
+        <para>
+          For Zipfian distribution, <replaceable>parameter</> 
+          defines how skewed the distribution. A physical sense of this parameter
+          is following: <literal>1</> generated <literal>N</> times, 
+          <literal>2</> generated <literal>2 ^ parameter</> less, 
+          <literal>3</> generated <literal>3 ^ parameter</> less, ...
+          <literal> X </> generated <literal>X ^ parameter</> less times.
+        </para>
+        <para>
+          Intuitively, the larger the <replaceable>parameter</>, the more
+          frequently value values to the beginning of the interval are drawn. 
+          The closer to 0 <replaceable>parameter</> is, 
+          the flatter (more uniform) the access distribution.
+
+        </para>
+    </listitem>
    </itemizedlist>
 
   <para>
diff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.y
index b3a2d9bfd3..25d5ad48e5 100644
--- a/src/bin/pgbench/exprparse.y
+++ b/src/bin/pgbench/exprparse.y
@@ -191,6 +191,9 @@ static const struct
 	{
 		"random_exponential", 3, PGBENCH_RANDOM_EXPONENTIAL
 	},
+	{
+		"random_zipfian", 3, PGBENCH_RANDOM_ZIPFIAN
+	},
 	/* keep as last array element */
 	{
 		NULL, 0, 0
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 4d364a1db4..9e515abe9b 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -94,6 +94,7 @@ static int	pthread_join(pthread_t th, void **thread_return);
 #define DEFAULT_NXACTS	10		/* default nxacts */
 
 #define MIN_GAUSSIAN_PARAM		2.0 /* minimum parameter for gauss */
+#define MIN_ZIPFIAN_PARAM		1.0000001 /* minimum parameter for zipfian */
 
 int			nxacts = 0;			/* number of transactions per client */
 int			duration = 0;		/* duration in seconds */
@@ -184,6 +185,14 @@ char	   *dbName;
 char	   *logfile_prefix = NULL;
 const char *progname;
 
+/* Variables used in zipfian_random */
+double		zipf_zetan;				/* zeta(n) */
+double		zipf_prev_theta;		/* theta parameter of previous execution */
+double		zipf_prev_nitems;		/* n(ub - lb + 1) parameter of previous execution */
+double		zipf_alpha;
+double		zipf_beta;
+double		zipf_eta;
+
 #define WSEP '@'				/* weight separator */
 
 volatile bool timer_exceeded = false;	/* flag from signal handler */
@@ -737,6 +746,58 @@ getPoissonRand(TState *thread, int64 center)
 	return (int64) (-log(uniform) * ((double) center) + 0.5);
 }
 
+/* helper function for getZipfianRand */
+static double 
+zeta(int64 n, double theta)
+{
+	int			i;
+	double		ans = 0.0;
+
+	for (i = 1; i <= n; i++)
+		ans += pow(1. / (double)i, theta);
+	return (ans);
+}
+
+/* helper function for getZipfianRand. Computes random variable in range 1..n */
+static int64 
+zipfn(TState *thread, int64 n, double theta)
+{
+	double		u = pg_erand48(thread->random_state);
+	double		uz = u * zipf_zetan;
+
+	if (uz < 1.)
+		return 1;
+	if (uz < 1 + zipf_beta)
+		return 2;
+	return 1 + (int64)(n * pow(zipf_eta * u - zipf_eta + 1., zipf_alpha));
+}
+
+/* random number generator: zipfian distribution from min to max inclusive */
+static int64
+getZipfianRand(TState *thread, int64 min, int64 max, double theta)
+{
+	Assert(theta > MIN_ZIPFIAN_PARAM);
+
+	int64		n = max - min + 1;
+	double		zipf_zeta2;
+
+	/* update cached variables if current parameters are different from previous */
+	if (n != zipf_prev_nitems || theta != zipf_prev_theta)
+	{
+		zipf_prev_theta = theta;
+		zipf_prev_nitems = n;
+		
+		zipf_zeta2 = zeta(2, theta);
+		zipf_zetan = zeta(n, theta);
+
+		zipf_alpha = 1. / (1 - theta);
+		zipf_beta = pow(0.5, theta);
+		zipf_eta = (1. - pow(2. / n, 1 - theta)) / (1. - zipf_zeta2 / zipf_zetan);
+	}
+
+	return min + zipfn(thread, n, theta) - 1;
+}
+
 /*
  * Initialize the given SimpleStats struct to all zeroes
  */
@@ -1567,6 +1628,7 @@ evalFunc(TState *thread, CState *st,
 		case PGBENCH_RANDOM:
 		case PGBENCH_RANDOM_EXPONENTIAL:
 		case PGBENCH_RANDOM_GAUSSIAN:
+		case PGBENCH_RANDOM_ZIPFIAN:
 			{
 				int64		imin,
 							imax;
@@ -1617,6 +1679,18 @@ evalFunc(TState *thread, CState *st,
 						setIntValue(retval,
 									getGaussianRand(thread, imin, imax, param));
 					}
+					else if (func == PGBENCH_RANDOM_ZIPFIAN)
+					{
+						if (param <= MIN_ZIPFIAN_PARAM)
+						{
+							fprintf(stderr,
+									"zipfian parameter must be greater than %f "
+									"(not %f)\n", MIN_ZIPFIAN_PARAM, param);
+							return false;
+						}
+						setIntValue(retval,
+									getZipfianRand(thread, imin, imax, param));
+					}
 					else		/* exponential */
 					{
 						if (param <= 0.0)
diff --git a/src/bin/pgbench/pgbench.h b/src/bin/pgbench/pgbench.h
index abc13e9463..1a29f1260c 100644
--- a/src/bin/pgbench/pgbench.h
+++ b/src/bin/pgbench/pgbench.h
@@ -75,7 +75,8 @@ typedef enum PgBenchFunction
 	PGBENCH_SQRT,
 	PGBENCH_RANDOM,
 	PGBENCH_RANDOM_GAUSSIAN,
-	PGBENCH_RANDOM_EXPONENTIAL
+	PGBENCH_RANDOM_EXPONENTIAL,
+	PGBENCH_RANDOM_ZIPFIAN
 } PgBenchFunction;
 
 typedef struct PgBenchExpr PgBenchExpr;
ycsb_update_zipf.sqlapplication/sql; name=ycsb_update_zipf.sql; x-unix-mode=0644Download
#2Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Alik Khilazhev (#1)
Re: [WIP] Zipfian distribution in pgbench

Hello Alik,

PostgreSQL shows very bad results in YCSB Workload A (50% SELECT and 50%
UPDATE of random row by PK) on benchmarking with big number of clients
using Zipfian distribution. MySQL also has decline but it is not
significant as it is in PostgreSQL. MongoDB does not have decline at
all. And if pgbench would have Zipfian distribution random number
generator, everyone will be able to make research on this topic without
using YCSB.

Your description is not very precise. What version of Postgres is used? If
there is a decline, compared to which version? Is there a link to these
results?

This is the reason why I am currently working on random_zipfian function.

The bottleneck of algorithm that I use is that it calculates zeta
function (it has linear complexity -
https://en.wikipedia.org/wiki/Riemann_zeta_function). It my cause
problems on generating huge amount of big numbers.

Indeed, the function computation is over expensive, and the numerical
precision of the implementation is doubtful.

If there is no better way to compute this function, ISTM that it should be
summed in reverse order to accumulate small values first, from (1/n)^s +
... + (1/2)^ s. As 1/1 == 1, the corresponding term is 1, no point in
calling pow for this one, so it could be:

double ans = 0.0;
for (i = n; i >= 2; i--)
ans += pow(1. / i, theta);
return 1.0 + ans;

That’s why I added caching for zeta value. And it works good for cases
when random_zipfian called with same parameters in script. For example:

That’s why I have a question: should I implement support of caching zeta
values for calls with different parameters, or not?

I do not envision the random_zipfian function to be used widely, but if it
is useful to someone this is fine with me. Could an inexpensive
exponential distribution be used instead for the same benchmarking
purpose?

If the functions when actually used is likely to be called with different
parameters, then some caching beyond the last value would seem in order.
Maybe a small fixed size array?

However, it should be somehow thread safe, which does not seem to be the
case with the current implementation. Maybe a per-thread cache? Or use a
lock only to update a shared cache? At least it should avoid locking to
read values...

P.S. I attaching patch and script - analogue of YCSB Workload A.
Run benchmark with command:
$ pgbench -f ycsb_read_zipf.sql -f ycsb_update_zipf.sql

Given the explanations, the random draw mostly hits values at the
beginning of the interval, so when the number of client goes higher one
just get locking contention on the updated row?

ISTM that also having the tps achieved with a flat distribution would
allow to check this hypothesis.

On scale = 10(1 million rows) it gives following results on machine with
144 cores(with synchronous_commit=off):

nclients tps
1 8842.401870
2 18358.140869
4 45999.378785
8 88713.743199
16 170166.998212
32 290069.221493
64 178128.030553
128 88712.825602
256 38364.937573
512 13512.765878
1000 6188.136736

--
Fabien.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Robert Haas
robertmhaas@gmail.com
In reply to: Alik Khilazhev (#1)
Re: [WIP] Zipfian distribution in pgbench

On Fri, Jul 7, 2017 at 3:45 AM, Alik Khilazhev
<a.khilazhev@postgrespro.ru> wrote:

PostgreSQL shows very bad results in YCSB Workload A (50% SELECT and 50% UPDATE of random row by PK) on benchmarking with big number of clients using Zipfian distribution. MySQL also has decline but it is not significant as it is in PostgreSQL. MongoDB does not have decline at all.

How is that possible? In a Zipfian distribution, no matter how big
the table is, almost all of the updates will be concentrated on a
handful of rows - and updates to any given row are necessarily
serialized, or so I would think. Maybe MongoDB can be fast there
since there are no transactions, so it can just lock the row slam in
the new value and unlock the row, all (I suppose) without writing WAL
or doing anything hard. But MySQL is going to have to hold the row
lock until transaction commit just like we do, or so I would think.
Is it just that their row locking is way faster than ours?

I'm more curious about why we're performing badly than I am about a
general-purpose random_zipfian function. :-)

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

In reply to: Robert Haas (#3)
Re: [WIP] Zipfian distribution in pgbench

On Fri, Jul 7, 2017 at 5:17 AM, Robert Haas <robertmhaas@gmail.com> wrote:

How is that possible? In a Zipfian distribution, no matter how big
the table is, almost all of the updates will be concentrated on a
handful of rows - and updates to any given row are necessarily
serialized, or so I would think. Maybe MongoDB can be fast there
since there are no transactions, so it can just lock the row slam in
the new value and unlock the row, all (I suppose) without writing WAL
or doing anything hard.

If you're not using the Wired Tiger storage engine, than the locking
is at the document level, which means that a Zipfian distribution is
no worse than any other as far as lock contention goes. That's one
possible explanation. Another is that indexed organized tables
naturally have much better locality, which matters at every level of
the memory hierarchy.

I'm more curious about why we're performing badly than I am about a
general-purpose random_zipfian function. :-)

I'm interested in both. I think that a random_zipfian function would
be quite helpful for modeling certain kinds of performance problems,
like CPU cache misses incurred at the page level.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

In reply to: Alik Khilazhev (#1)
Re: [WIP] Zipfian distribution in pgbench

On Fri, Jul 7, 2017 at 12:45 AM, Alik Khilazhev
<a.khilazhev@postgrespro.ru> wrote:

On scale = 10(1 million rows) it gives following results on machine with 144 cores(with synchronous_commit=off):
nclients tps
1 8842.401870
2 18358.140869
4 45999.378785
8 88713.743199
16 170166.998212
32 290069.221493
64 178128.030553
128 88712.825602
256 38364.937573
512 13512.765878
1000 6188.136736

Is it possible for you to instrument the number of B-Tree page
accesses using custom instrumentation for pgbench_accounts_pkey?

If that seems like too much work, then it would still be interesting
to see what the B-Tree keyspace looks like before and after varying
the "nclient" count from, say, 32 to 128. Maybe there is a significant
difference in how balanced or skewed it is in each case. Or, the index
could simply be more bloated.

There is a query that I sometimes use, that itself uses pageinspect,
to summarize the keyspace quickly. It shows you the highkey for every
internal page, starting from the root and working down to the lowest
internal page level (the one just before the leaf level -- level 1),
in logical/keyspace order. You can use it to visualize the
distribution of values. It could easily include the leaf level, too,
but that's less interesting and tends to make the query take ages. I
wonder what the query will show here.

Here is the query:

with recursive index_details as (
select
'pgbench_accounts_pkey'::text idx
),
size_in_pages_index as (
select
(pg_relation_size(idx::regclass) / (2^13))::int4 size_pages
from
index_details
),
page_stats as (
select
index_details.*,
stats.*
from
index_details,
size_in_pages_index,
lateral (select i from generate_series(1, size_pages - 1) i) series,
lateral (select * from bt_page_stats(idx, i)) stats),
internal_page_stats as (
select
*
from
page_stats
where
type != 'l'),
meta_stats as (
select
*
from
index_details s,
lateral (select * from bt_metap(s.idx)) meta),
internal_items as (
select
*
from
internal_page_stats
order by
btpo desc),
-- XXX: Note ordering dependency within this CTE, on internal_items
ordered_internal_items(item, blk, level) as (
select
1,
blkno,
btpo
from
internal_items
where
btpo_prev = 0
and btpo = (select level from meta_stats)
union
select
case when level = btpo then o.item + 1 else 1 end,
blkno,
btpo
from
internal_items i,
ordered_internal_items o
where
i.btpo_prev = o.blk or (btpo_prev = 0 and btpo = o.level - 1)
)
select
idx,
btpo as level,
item as l_item,
blkno,
btpo_prev,
btpo_next,
btpo_flags,
type,
live_items,
dead_items,
avg_item_size,
page_size,
free_size,
-- Only non-rightmost pages have high key.
case when btpo_next != 0 then (select data from bt_page_items(idx,
blkno) where itemoffset = 1) end as highkey
from
ordered_internal_items o
join internal_items i on o.blk = i.blkno
order by btpo desc, item;

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Peter Geoghegan (#5)
Re: [WIP] Zipfian distribution in pgbench

Peter Geoghegan wrote:

Here is the query:

with recursive index_details as (
select
'pgbench_accounts_pkey'::text idx
), [...]

Hmm, this seems potentially very useful. Care to upload it to
https://wiki.postgresql.org/wiki/Category:Snippets ?

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

In reply to: Alvaro Herrera (#6)
Re: [WIP] Zipfian distribution in pgbench

On Fri, Jul 7, 2017 at 12:59 PM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:

Hmm, this seems potentially very useful. Care to upload it to
https://wiki.postgresql.org/wiki/Category:Snippets ?

Sure. I've added it here, under "index maintenance":

https://wiki.postgresql.org/wiki/Index_Maintenance#Summarize_keyspace_of_a_B-Tree_index

It would be a nice additional touch if there was an easy way of taking
the on-disk representation of index tuples (in this case that would be
little-endian signed integers from bt_page_items()), and from that
output actual typed values. Maybe just for a few select datatypes.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#8Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Fabien COELHO (#2)
2 attachment(s)
Re: [WIP] Zipfian distribution in pgbench

Hello, Fabien!

Your description is not very precise. What version of Postgres is used? If there is a decline, compared to which version? Is there a link to these results?

Benchmark have been done in master v10. I am attaching image with results:
.

Indeed, the function computation is over expensive, and the numerical precision of the implementation is doubtful.

If there is no better way to compute this function, ISTM that it should be summed in reverse order to accumulate small values first, from (1/n)^s + ... + (1/2)^ s. As 1/1 == 1, the corresponding term is 1, no point in calling pow for this one, so it could be:

double ans = 0.0;
for (i = n; i >= 2; i--)
ans += pow(1. / i, theta);
return 1.0 + ans;

You are right, it’s better to reverse order.

If the functions when actually used is likely to be called with different parameters, then some caching beyond the last value would seem in order. Maybe a small fixed size array?

However, it should be somehow thread safe, which does not seem to be the case with the current implementation. Maybe a per-thread cache? Or use a lock only to update a shared cache? At least it should avoid locking to read values…

Yea, I forget about thread-safety. I will implement per-thread cache with small fixed array.

Given the explanations, the random draw mostly hits values at the beginning of the interval, so when the number of client goes higher one just get locking contention on the updated row?

Yes, exactly.

ISTM that also having the tps achieved with a flat distribution would allow to check this hypothesis.

On Workload A with uniform distribution PostgreSQL shows better results than MongoDB and MySQL(see attachment). Also you can notice that for small number of clients type of distribution does not affect on tps on MySQL.

And it’s important to mention that postgres run with option synchronous_commit=off, to satisfy durability MongoDB writeConcern=1&journaled=false. In this mode there is possibility to lose all changes in the last second. If we run postgres with max durability MongoDB will lag far behind.
---
Thanks and Regards,
Alik Khilazhev
Postgres Professional:
http://www.postgrespro.com <http://www.postgrespro.com/&gt;
The Russian Postgres Company

Attachments:

ycsb_workload_a.pngimage/png; name=ycsb_workload_a.pngDownload
�PNG


IHDR��z��3sRGB���@IDATx��`T������^�@��#X��E}������������>���.���E�.���!=�d�������$��n�
�k\���9s����s���L:$	H�$ I@��$I�$	H�$ I@��$I�$	H�$ I@��$I�$	H�$ I@��$I�$	H�$ I@��$I�$	H�$ I@��$I�$	H�$���M^m��qqq��u��|A7�T*���x����HK���Pz��&B/��Vi�����*
��|��d'zP����m�{:����v��w?ty(��r�9��*K�G���hB�&������������1c�������/h
�S��299�h4F>�%%%���s�����jll,b���>71�V&L��c��K�i���oJJJqqq;�b]�tIKK�A
�N;T���w:�V���
�	�^�u���&�)h:��w�i�����pLU�HF�{��
�y	!�����5���k���i'�������E�T���y�"�O�
������y�G�KU��dee���TVV�*�e���AW?�*v0�n�B��`��
&�#��J�D��F>��M���/��H#�U��N�������oM[��g�`&EQ�������QC�F�a��N��G{&�����t������]���w_z:���8�L	H�T{���h��.&M�4r�������?��i��q;_|������q��v�\�>�h$��pB��{��j);���X@RB��$I�$	4 ��h�'�x�<���h�`0\t�EO>��_|1}��=z�=������};�b1������N!�3	�N3��g\����K�z��F<��X��x�n+2��	c�odr��S0����,OigyD���Y�*�P!�V!V�/�H"<0���=��)����8�����/���4n����������������O���=��0��J���i�������*�r�K.W�|^29S�>����r�<��6Y�2�3�r�L
��p��g���Mf5+�J�E+��d.��k��2���i�<c{��!��N�~���<����S0�i��o����U�V����`������v�I(Mp��t�-��>��G(������l`�����|KyP
d���c�n����h��s���a���0 M���#9<v�!�*|(�
*s{@e<�����#��-�E+P��`B��0�v_L��p�k����DeT����c��Dg�5�re�B����1���t�A��IT�zMe�����Vht;��|���5*�eo��U�uV��&2�����@����@Y��9�.�_���� ���-���H��������<�v)�<#�UY^(��vIH��'���s�D��+�������$�3hV��<�?m�[�Bx���Xr���',n'��'N�Xjj*P�3,��GVoa�Y��{����g�yeZ��G����N����+��O�q���)rc7�����U��U=�V���_pN��y���=����C�Igi��Rh���]��e����CsT�b�M�5��U2y��V��8m�.k��e���nW�OV�;�I���p�������f��k�i$\�D��8���N�	�T$�h<,
<��6m
�Dle��#b9C��Ov���A�z�fvvv��4r��������7�����o������'���l��}��[o:t�{��G���P����,Et7O��>��i7�-�-�[�UQ�)7V���0.�Y+gD�e�����e�+q>g�0�%=�c���/'A�92��C�u�h������)��m(u��Q��4gTo�:��2�d>���q�N��Q��m�N�AW�$m��b����|�����4�I�Rm�b���qD>�>��|VEsn��)>c�����~����i�!��"�&z��.�
�	�����s^	�PX�p~fV�u��-����?��V�X�f��vPV�����d1��{/4�5�P��U;V�����\���$K���C�����i��K��0�T5����
����?����Uo���Ee��6.��dM��*�TGE��h��r��r�������RQ]�F��24Q�5�'*�6���&M��}g�hU�]��}G��e��<�n�m�������ZK�$	H�$ics&*$���������?��S@G��������i�A�����M7������LH(�������KV��.�L?X?��L}�z�SEy�W������U������3{�b7�O���BG�Ng���ms*�!u���q��N��>.��)2�������*�-��q��F,!�&I�O�jS��,M����Q
�pU�"��D�J�u�lE.�~����z��z��z���x=���QG�,�#I@��$�N*��phD`?� �����z������)U���������q����D
}"C���xA�<���;W�V:f����Y==1G����P-��=n�9oyu���Uo��������~Bt����W�����;l�+����s7����p��;�I��x�����FYtw
��C�_�|�A��jp���"[:�\ 9�a�szl�Y%���_��$	HhTa�F[���l6�<�����M�f�0Z�5*f�I1CNWF)�<YX�[���vV�/������O��R����Y����������"�a:P��G�6F��;F��O0t�o��u��|�9���?�����{�������/_ce��B-G-����zi��5Q�4�LM����S�^�����*=�}N��e��Z���;��������~%	H�$ I��I���
f�]1/�1�����
G
���;��t&e7=RX�����[�&��[�m3~��|j�N���z?�Z^�P�N�f�v������ur�J�ul�$C�Q�c�K�O[&��r����i�?�"�Q�����k+uWn��K��QJu�B�E�G�~l��f"9il�B#�X��j���]��Y�����V(xx	���C��$I�:��	�F�X4	-N����.g�
z(H��xa�2��I*7���{��W&u81b7|������l���[���.V��'a��)'�&xayY~<�������qA�&��(cV�|�_EV�4�*u�
�8��6��V��Jc�rf,��J����(r�sU;��Z;J\���4�|X�RJ��$I�)�c��1]4�!W��@�mZ���2��M�Y�F_����kJ�3~�c�)����&�(!7\2��]m���Z��D�dG>�������~S�����>� �'a:Y���d�����B�-�A�M�O�,rl_]Tw���c���'���27�}2�u�hm/r�hm/s������,I�$	H�(	����+e2S!;bg]�0��{�c����VT�`��"�������^�������fO�v�DW�ik���k?�z��������q��q}�(��Qk��f��d����\����Ud Y�2��������>:C7u�(}���^�����]��5�U��n���1Y��M���0$]�$ I@��$�0H�������BO�������n9���@�i����+J^�_}�/{?;��[�e;�9?��Zt��g3V���'��y����NqqT��E�]��]��!U�1�@��?���o��<���,�bY������������
�g�X�l/w�
�%8Z�%W�8��E����%�&(��
^s*fJ��$	H�$v	����0����A�������s�E_Q�����W���������}���_������0% ��t��9��v��e��Y����}Z��YC��(���P��3W;��!�V�]�L�2��6�b�U�C�H��Zq
W�^�L��Y���b\R���u�;�'��6cj�_�����N"(I@��$IG?�����:��IR��s������B�U�������������~?P�JZL�*6e9�&c��Ic�0tU�����o�Nn�UA��+���0
\��&��R��.I������:��Oh������C�vO�����.��j����s?6�"�m>����
qB�NqC)����X���C��$I�Z+���e2�\F,��2�
D�t����|��_����m�{��x������������g���|��n6�.���h�����1U�V�����/p��3���6{,�e��Y���R��Q=�Q=4�=c�iT�j�,8Nn�����P��:���1C<n�������q������>��N�^Vn�n�z�~�&���;"���C��$I���~V���[�����EY�rsXn6�}N��7�[��@���9��,��,��2n��:oE��%���5�x�JW��u�1�f��"�8Zk�������Z��]�q��VN��8N����-���U:�o*�X����cV�p�P���d�N-S���� db�&����
f�KP�����b�HP����v[}��F��=.���	Eb��c���u�FK����o��[R����K�$���`��Am�G�Z�S�I��\�m���J����=j}��B�����z|zF��_r�`���u{��l��n���k��Ot����>S��?d�_.V��sHE��
�7?^�:��,r��*�����]�q!���S��U`D�����%��J��V��7J6�p�V�\e.@��k�Jj��HK. �Z���s�ga;i~=���.X�2?*�^3hP�T��V���Svr; 
T�
�{�E��KEV����7/�z%����u�i�`�cED�H�SVd2DVC��x;u��`�>��tl_(�Mu@�]����i	2m�V��cr�eZ����b�Jhc_���/-yaB�'�g_���ue�����'�w�GD<Ea����@����w�����et���T���0�Z�[)5X1���<j� ��)~f�i'.��:P�I
�hB~6dA0S,�?`(y�aV������?����w��Z�"3�`��f�
�V��V�m�a�V��s�\��3���Bx��>���b�d��J3:�����>���G_�������l�.��Z@n��-h��� 7\Byq�/ib�1����?5?u�^���kj1�s�Z->�x�C����HmR��/�������4�������w�i�uV���h8�<*��@��OQ����V�]"<<�����FB�#�W(&&�
��]H���ji��:mZw�R���6g�6VV�g�����U�O�I�������o.o�y��@�	��s��?�8�,��{���D��3M���\!�8���EEE)����Y
�?,o|�����[�%���D����~�b�?��Tm�����2���q�o�(Z����F����������drU�W
k�pC;���.�0A�VO��(��b	`���:.�����Z����^~����L��^�Ku��-�����.��S�cU�2�
S��)g�`�b�����&�u���)��!��J��*
E�Z��oV�p�F������g����!������X$#|��L&)��kNc_���� #:::��MS>Z�v$;��r��p�`�X����dC�L���W����l�P��b�`,�Y-/���������YZ��*(@eee<�
8"������'��<������E���x
��#��?�`0������P�cI|l������ ����Vf
(�*��
[�z�c��0�-��%[�0(T��Li�y���(�20�VZ������&_�o�FLr{�������.�d�'�����X?��*��d�
�$�a\ �'wX-LC����6v(	�|���x*���5{�k��-����S��{�	�\'���]�}�]X<SP�D��kT��P%���������������f�K9�4�	�>����P�j4[��k��c,�#Q��E��R0K&S�J����Z(�5^������T�T<=��'g�������ubk�|������tG�����>�u�1��V��c�V4�Bx��6+��44��G����w%k�����z1�� ��a�����OY�����@��{�1o��la�rp�4.1���Z�J��|��L <Q�b�I�5.[�������Egi��!�<��6v��V=ZC�~K����U���v)�)~�~�A^�j�C�B0�L@�?  �����Zm�9C��?��u��	�	�����(.(�Bsh��`�S��I�vi�$qCU1������BY.����_!����>I��'���B�zU���"�G�c��?"8j�	����}�Ca5��
�uT�v0��luq]��:�����i������3����������9��:�J�|�l���M;����o6tS����e/z�]v���;K�C��:~�Aj�} �R<:�x�a��\\���;p��d��(�P���9y��m�����YO�<3u��'��Q���`����z1U���g����sa4��9tq���Sq��B=��q2
@��q���J�����0�^S������XY�0�����/" C�����B&������1�PSE�G�|����r���I�XF��o��-O�\Q_i��`���y}Z������
B5.":WA;p��* �0�q��������A�MN3��8t`L��	d�1����_M�%a8�_y/��M?�kb��iV��	��-���8�g�\��cz�T��V�>[%k6T(����	�n=	�,�����T��J���z�)�������q�O�~�������Hy������y���<�_�=�����%oeX���L���b�P������+���������f�de;��yf����
�Z��������
�����Dj5�5����;����A��G�������0/\��0��������k�[8XR0�����r*��	U����S&�1����ZBTl5�P���a��5
BC~R�/��2�W��ea�#�B��	JS�?���J�� ��������
^����������\��C���.����I��/d�">�'Uc���.�-D�a�?�.�) ���`N����i?�����������V
���E��~C�p��D��q�_"HC�4u��R<�%�&$:��.ku`��t�RG?���>���.u��,J���=��@���1�1}^n��?*n���U�a���m���������������,y����<�6���������a�������1�������I ��V���g�F����������[FC�q8�p�P�z9��:yb�0��
�?������OZz��e��*@��
���:��y�WG%�JDh��� .,D�g� ��0��R�!�������XA�$�-�6��
B�W��O����^��a0����&��y�=�O�h��s�������?@�/�.	�ei@(�(GhL`�_E�j����`��
���B~M��E&k
p�&G������%$���Q��
*}�V^U�29����e`�#��5J�Mc�������������*?��{�{����x�3��5�`�#����C"�f	��a&r~���j�X���g�K�M��jY��U>�O�]!�����F��@�ab�
�Z�5����O���{��0��9������Iv�����4��@���N�	A�TQ�j��+��6�*~�>|��dq��fX�8@�10�@Ua$!�#,���``��x����7M?-V�B�����#0�IG�H�(`����N����_�Bp4{/l��K��u����d�+���:�l��
�p�����I�pW���{_�c���KV����lc����K�4M@i�h�U�U�V�!RB^�"uN�Q7_XG�2����Pj��u��PS�^���G�32�vg����[Rt�[��VO�muz��J��J�Qn+�����VRl�����wq�N)�1<�-���-�2�dOCe[��W����.SC��{f���r���TV��$���W[�r�X����i�5��,\���Ef��}l��4	�j�$p��L��i��*";}�nV1���/:���1<E���_Uar�����e	w^P�M��nH?���q7������//}�c����z5$:��1	0J�OM�a��v��n�����f���J)�*�W�u7m/^L�����\6-[���>/�����4.���?���i�����������='���DE��$�cS���}?�5`UL&C���^���m�tx^^_:�W\�^�l�=?�����6w��]du�Z��6w��U�������&��%A����V�%���RO������.�h�<6|T�C��$I�"WG9G�u���������d{��
����
�N�d�Z����>����,��z��c�T.���
����-����
��m~�,"<�	9l.( N��e8HgL�#��I��o�����'�#I@��$I�����	]��E�h.k�(M�~o������4�/�+�#���S�F���t�:��I�+��D�zX�a�N��(	 ���z=������f'
6�
���Vg��Ui���*���6���������pk��7d]��i�EK�w[J�`L�$I@��$���@�~�z���h����:k�����a�LFf��)�$A�����U�:A@������J�W�5����Y�E��ym���
�����F��-����
$�_��T�*�^��R+��������0�u�8��g�vX���W��y0<�������>�lvZ_/��y�����|���$��Gt�������5�g:#%$	H�$�&M�t��W�w�yo���g�����_|�������!����
7��&t����7%%��Geg+�,���;����.]����'�|�`���n_Kk{��2�v������a��c{!2�p�)��?�������<������h�"J����b��T�2#J���p:^�L�����b��G��c��+���u������~v^��:gq�C�o�
�]X�*�8Ae&����j�����g������O�Z1!�m�d����q�k����$ I@�@�`��n����Nb#;v]c���k����'��={6�Po���#G�{��lq��+���7���N���i���_����������fee>��C`PRR���HLL|����c�l6�]�������6�
|���[��7���[n����+{Z�����&0n��A�Q&==����7��bt���tp��(jV��PZ�����`��\�e"�H��7��mqy�m���A����dqZ�q�Z�U$i�&�7$&�T:����fZ�O������?OV4j�W�L�n�����{�`��c������|��;������|��[���}��j��4��j��N����v�v>y �B���(��t�s�9�|���]��E<o��\�-[����W����W�X�i�& f���o��8�>���x����gCh��'�|r��������^8w��Gy\�����?��#(�Ib�lD%��4�j��3�8��������{�n����?���q��}���
v�|���"`@�� p���H�
>�!I�-/��8�l�|;���~y5�Bg��S&��h[��Y��
�F�8�6��J����k����~���������X��hoE�^��W�G�2�y�(u��V��9>a�E�B#����.�������JP�}���������w���������yP(�QYM�P�T��"O��w�w�?��~�r	!>�/U�R}�+"k���p�	�������A�.��������.�9k��!C��`t����_��[�n���/P���/��|������P�>������:55�b(xD���1c��ik���`v��txf��2�������t��z�� nNN��3g�gdq�M7!r��kW�&nX����c�����@1�Ga���M���vi�&����2���p� �vl��-����6[��W�����e�g��+(�j,.��,WY"+�U�b��8�"Y��(�<E�������=�\�{u�n����v��m�>4��;.��5|9�r{�qaIL5�������O�",K�O�0!��j�`��|�0��1"�w�nfgg7��LP(��<��=�-f>��zv�fT�d���Lh�������|��T�s��1	��5�v�����6�
��V�-[6x�`t���#'6�LH�C� ��@�0�3<A��Q`-���P\\�t��� G�������C��|{�|^}�"qL�N]R������ ��5@��Yt0!!�������Ra�f��[U��N��3���]?�>�0����y����w������?�k�uC�mj��s���%w,+�}���8�cx$�0o#���b	t!b�w4����_1��������u��'�k�_�v�#B@�i��k�1
_T�%&M�� ~�y���"����^^[3���2My������e��|���eZ�}��wV�^�1�yL�<������kDe�I.UUU��8��W_}��c.E�CW�����A��cX�/8&6���m���
�r�}tAdTPP�W	��g��m����c�r���x�����?�<;x@��{�X��l��%���{����8�A������R(�dd��Y�`7��=^�{�KO����I�t�9�8�/��w6��q����g||<�����<����7��F���S�	�1���'��� 3P��������J��|As�}�io�:��M�	
�;~�x�����A����@&P�
��a��\�'_�t���F+W]u�8��>}:DH����1��`a��+����00o�s�=��3�i���q\��f01,���O?�t���c����O�)��I���1
��tTk���a��o����	V9�x4ko���Q�o���ii�nL�pg�+�)[�n��g\U�G00I�>����-�N���=��s+O���wz���-oo.�%�
?�6c���w�h�\\�O�o���/����)D
���)X�I��P�Hc���G�:i�8~����0i����on�[�1$�r����@��h�������s���o��&@$�a`��������~�a����#������������Q����P�[Mm��V��y���p�s��E�S�L�8+�����?l��c���j�w�~����)�+'��'��3{m/����������������Dw�K�$	t���D1k�Q�D�7�nP/��|�)�Nkg6�F�[�d	9����
wT~x���+vCL����a@���C":��NFh�
{�a6����o��q���w��S��6m�]�.���
?�4��;���& �_E��J-�m(�|P��}�_�������0~����J~I�$I��@rr2X��{dp��v>&�z�h���f���hzh�`vmj�df6������Ct��3��t �-oZ��V8�.	�������-y�����SK�p}l��y��U�`��P��<"����Me���������>�WLzt|���$�����e;;h��&x�.I�$	A�~�c<�4���G"z���O�f��d��
����F[�f
a����[q�3�*��ct�R"�^U����=��|��]&?�{�����y�b�n�u`��Y;��'V�������o���2o����[�KWv��[��KW%	H�$��h����b�r�N�kZ[�,���|j���\&YFk������,��U������L��s��G$k���2��j�<�Q�N���~gK�g;�g���yx��a)3'}��`�����uJK'�$	H�$��@3���s�NJ�3�G����x�6^�*/��62�����Kw�|�����q����C�&W�JW���u����L��������b��r^��i}��z}S��{+�@�����K�$ I��@3�"""H���}����v��gbkD��t�08F�sOe8�F�|���u�w|��31?�N�rY��K���`/��`�#3�X������W���Q����'g���g~u}�9&v}8���#I@��$�cS�0���x/3�M�)a�;w
I)u�Jm�A]l/n���)��nL*Tzo�Fw�Auf�`���{�������?����@nf�7,E���K$}6���"���J������������$Im(����8���8y�&S�!��/�zY����_Y,s<�8�Z��;���}k��\��W����|�W�
�-~E���u���]?`�`�n����6��#?�����D����z��l��c3��:ng���/��$ I��$���-�D3��h@�Lvv
ZD�|OQ|���
(���(���=���n�=+*�R�\��(tJ&�V�]�P�s�|~�����t��7����>m`��sz_�ct�	��'�kr��4�g��Yp�[���m�%�_��m@�������$	tz	�x���z�^y�,�#��a�^N� jfvv6�'I��Ylv�J'��<y2Q8�A��~������+�4��E]2�C�K2333!2j��C����2
q�����&A�e"Z���!�`%�e3&h6q�:�,q?�K.���k�]�pa
k��!���'����R���&9���DrV���|G������8rz|����}V��Y[q�6��v�gM��.��0kX��c3��<{�g����c^����jH�$PW���'�{��n]"3���V)�v��*�q�Z���C�\�s
�x��W/v`/B��C%4������K�+6`2� f�,���b����>� � 2���D�&F�%�F�-��%�0������C |����7����m��v�����D����{�'�4�/�����6���	����0q9Z�HGT�M�f���m�+k������h�V�)��+J���q���cf�>�l��%�_uT�����Yb���;��f��4<����WJ�nO��J�TKK�Z%L��$�N'�[�cU�QU����>�����-O��.5�l�:���u�f��;�Y�	��7��O>����+������_��$��'��%���*��4Ba�h}���\=��S�/_����w�����W/�����;�M p7Y��]6�@fO��]���g�y1EKd*�Xp��0��w����w��4�i��=��S�����4�]5
��Q��|KGx$P��x������N?���Iw$���x�K�+�$�V�`C���2~�������H��_"�r*_�T��~�����!I@���(���3+��5�X���rZ�B�+W�JU���/�OV&���q $0�b�g,���|v�^�n�hfC�b��]�v��8�l!O|�w�}:{���r$�����X��^�w�����o��^���|&aol��w�gK�DL���{��37vp�Dz������`���7F���P�������!�-FDk��%�Q�"��Kl�������5�J���z��I��(Y�R��������?����5kh�E}N���`��J��]���Z��%J�VFD';����w
>����O��N���d��6_])�n������U�>�)f��o���K���;��v���@<K"
�9�����ok?����R�+C0f��S�Yt_r0�2=��n�\/��~����P��N��7n<�J�F���F��9s&�h���'���������t�q���(_kY

H��K
��Vj|�d���Wm���Sz�e\��{��Yd���T���g�;7�*K�w}QR�0y��Q#��q�k�#./_�����>�]��,E�v��l��;kE���\�
J��5�����
���\�[����U+�*E���"������6�Ax�l����i�Y����<�M���K�w^|^��)�b�[f���	�k�D�7�[+h�k�G�;"�ac
���{�9u�=��������X�'���E<�d�O��-�.�������~�aT�3�<��������H���6?��#p�=�E�$�	c�i4���<vquh��m���� ����>0-��
?���3`�����p3x�EJd2h�j�K@�����+�	����j�jM�B���d�1��5n�C�OD>��l�p&	�N��n�i9�>�����R��0v��I�����{���e��s��y=h�n�l�B�+w�A�������+��^�/��3��M��w�>�U��� 02�8P3�js|jtp"�p����#_�a�hA�:���*�.��"
Z�"�H�����os_l�u��9���{,���yRl�8*]y���L�+�x��l^���)[�%&t�W�=���L��P�r	}R�c�k�/W�N���
Q�I��"5~Q����9�J����D��7o�,b��+W�l�N����L0����I����e�n61cP��W��Z�j�?�~h�}��������v-l����^�ZVu��2&��8b���
�q��mja�G�����t����d]I���-�n+�]Y'�7�/|��R�s&s7�n�Gv6rrr��4X���'����+9�9m6v��a�AN���@������2�fr_x��{��K�y4hJ]�W���
x�l��
0�:v�X�fm^�uw��a�/� �1x3���'Y��X�Z�����"b
R�D%I��,��DD��u�C�'����b�^�5��+K���1���Q�%�{��wq�r����X�������76����uCR����i�m/����b� ���pU���d�����Y�Y���L��j�����5M���>c�~���2a|�]wu��dS��/7*3��X1�t�"��G�#���ir�	���<T�[�g���>�g��@����]R����Uo�����N���������O����Y������.��K�T���NmJU$	H8�%`�X0AGB'��~��9AF&�iiih�LAG�M���K)s�
�U���n�E�R@�vU��[��g����'����G�G�,Z�\��Z�&-�49=o/��.a��^:0���s�V|�����v��$I�:��`z�D ��3�8������"e\�#����.*��JV���t��d���sC��+��I����wf_�J����K����u�7����e[��VN�&lw��>	g�U.��zm��|7XK��$ I@�@�J�o3��/����{,77������W�X���*��%���&�� u����+��>�����{J�������^�{l�S�f���y�������6��N����^�.�Ol�x�2h�REI��A�Qq�
��B;��nM4����E��_����������"_��fj��^k�'�0a�v����������E����FL���_��Q��yK���{���z�_��w[����> ����o��v�_�4Z"��.��I%	�`yNE�]a��b���I4��|�~/m&�y���F�rrQ�}��d���D;[E��vl��7���
��'��<��C.(]�N���{�kT��o,�/�)~~m�U���{ar���������m������J�$	Hh#	���n0hFu�@����I3�p�%S,f���a�����m�G8i��M�Wm��0���kXhU�k�������>���	�q�^�r��w����/���ov>�W��K/������w�eD�'���l.� ���X�*��F��MBY�bz��m�]Pm����E�Uq0l!5qFa�:���4��JV��=��U&Sk�Ie�M>���{��� c����|G-w����9���(��X�����i=�o�r���N�zw��J�JSA�U�(I l r �E��=�b[�j�YqD4/L�l���e�Q���������_����
�~ll�N��+K+��\`(�l��^����S���@��n����)�]S����-_��u"^���J�����_��8�{,�^60���I���^�P� ��)���*iJ�%	HhK	4
���o?����O?��gv<���o�`nK~�@[���h����Rip���$D7i�����nL=����?6���p���v��~7A7ms{��f)�tr�h`����g��_����Z?���� �A�#U�$ I@���hfRzz:�h}��i�&I%�r��D�Z�p�����uge�e.�����7�/}+������K��s���h��]`�,�����������Qi����Zrq�����j�5K-�TJ��$�6�@3��7Nk���>�8XL�.v�h�bD�U�K��D`w6Y�nFP�Pvn�M���K��U�zXa�@IDAT��Bwu�*T��Km${���s�������_'>��u�{��}]����[Sl�z��Q�m������g����#�����K$	H�$��`6��5k���fo�7�|����>�%��x��w�{�=U�>��C}��+��r���y�m���������EB��*_�[��i��1Mk�����z�,K���q3��+s�������n���������Y���S=?���K�kL;/�N\� I@�@'�����Z���jc0�o�������k��%����?���g�y��_�����5���W_}5���Z��J�����oGo~���p=gW����o�����/)����z�$&u5��U�
W���H�z
�8���j���/:v8���sw�5q����fNy�����'���'��
�����l6�kr��,����[�kq�����������:(I�l�P
�j(�W�/��1q���������ku���h�j(��>.����+�����c�jc�����@�G}t����s��n�	K����K2{02���_��0����*+�0;<�T:t(�=��JLL������4���%W��H[�<���
ELr���v{�S����#������t8�5+�.��0q��|%���yy��k�����1��-{N�?_�������X���	9k��xC�y��}Y�E��W����x�
��?�Y�����V�G&�(��.?�:,p�l|����'|zD>#�U^|�N����O0-�������9"Y���<�\�=z����������+V[� �B�����+�4/�."��xJ	JA�
 ��?�X4�����=|�p���u�V*��.���w�i�OK�.��?����1b��|������T����l� ���lHy�C��K0���7`ee�I'�����e�qq}����Ys�F�����0���>m��|%x����	�-tsq�'>^�B.����q���r�-�}���n�����DOx�N}IH7�@"�87�PA�B���g+|��r���������P����;M�#�
e�<�]�V}^_������3�Y�g_���bgL�w��E*�e����{|r��
��q����^5<}�Y;�����yd�\�'�/Z��=�u���^�Z�M���oU�)�S����h����F���)��U�]��j�~��RG /x����f�<}�uV"`uITy�����^�W�_���W��+v)��j �G��EV���Q�V�^
�8���^x!Kx@��^z� �(p��7�et�|l�\ps�`s��r���|�I��Yy���_}�7��Dh���@,&^(���1E��������6��V�?������`��
F<x(e'�7
�p��,D��*�4	��D`������1�}�$�%%%A0�(���+@A�6����Y�}���@~�Rh����]���/!w��/��E�"�U�G��J��t�����\m�w_�|�}�k�����K��1�������?�d�Q+��in�|fv�X�e��Nh-�;���UJ���h�5�)�K���Kc�<����'�11|6V B�yP��������"EbX�� ~#D�
���bf|�,�l&����<�M���|R�sf[}C3Z�W.��d�����r�j�����!p�;A%�8�y4h�o��v�u�����r�����I�&��?��vf>k7�t�H�ew��U`V�[ 	�����_�%����}��������"w�����,/���K�O=�`�MgB6Tw���������|���v��s������N����M���]F�t��1�h��V����X��<����G���H
�Am
Zm�O��VV�k���f��:[�f�QC!��O��D"�%;-?�\����	w&�Gt�)���V/|�^"��B<�.��\��\�I���M��O�gu[�gf��w{+M��oi
!R�D������h�j�v�h��G4�EJ�D�F�TA�A
�U�B�������4�.em5��i����
�6���y�c	Vq
�0�����)�'�`F%t'O�����K�V�T���0���6M��"�R4�������(��Q������h����<�_~QZ�����b�����P2�Z�
������/����V�fP�R�2~d14�)S�P�p���)0����e������x����	P}�Z�B|B�N�U�KMF��!L�G�����U���bx�Z�6��K�V��q�}����t^�_o��������T�����k��\> ��sz�Sdy{S��;��lA��"\��u�����Slg�jX��f?J��Y���:�#�������rl�������9�Zn���H:�	p�Lz@h���s��X�Ak4F�|h� �	6Ap����-[�����M�h��D���i�7T�0�N@�^x��"�-�TfL�L1���Z�z��:R�}AL��csfh���9s�, �����s�4:>Fs�t�6Zc��D��p�bR�jE���^��(,\�It:J���������#feN�;i�������z�m-KKm7�q��u��L�vp��S��>*�@��n)�_��lX$ �$�F������YoQ�����_���8p���z`'	�E�2/^������JtHP��R(�����
H&|���]��C�I&6]&���C
�:M�,M���*�m�����B<u�T�[�_~�%�N����5�o�ECe0&0v��4�D/��~Ce[��uM?��&��(5�����b����a��%��V��F�y��y5m�i��[E�R999���A��m(X���T�mL��b�L���e�|��U���2�=��?�YQiu
K5L��4�ob�NYP�j�6���w�7�6�L�*]�[=��KPnrss��G�)�uPT�"��K�
_T�����Mp_���C����2>/�}[px�x�p
<�|�#1�#����T"M���z��w1 #v�)���c�����T

�����7Tl}�Wd��,/Z2J0�f&[!.�B�L�R�f�Q;�mh�%������N��kF�i�����0g�<&�:4"���I��eNGU��h��\����+K�z�b��R���2����?��{c��O�v�,��4q�^~������n(������wB�m#R?�n|{K��"�9�IJHh'	0J���h�)g�Vl��o�!�����'�x�6g�%9(���?��C�� �����a�@>��im�.�/�(����+uW�L4��&�	BgF���HZ����J�*dfw�G�0F��h/\������� ���S����U?Tl��h�3������
���M��l/��+~����G��������6�,�37e/I���6<�F��-��y����6G�
�7�LA���X@��K��f @�Im�WP���K#���F�4O�;u%g����n+[�A����F\�������=m+��~Y\��;�l3<�[,�_�7��~	����R�/�Li��p	Z�#I (	����m���AK�Q����
6��%��8��(k��`bk�v���7X�c3�*uT|���e��������hc���w�5�����1����W��/iz��'����t��=�����6���b�;�����]90)^��_E��HI�$��%�������PVb'Y��r^��~�����G�[�\��Gg�e��Ri�N�X��y��%��������e�|k��K����l�[�r�Y�,�X��zp�t�,����B���K?�a,	a�R�/��$ I �%����Vh����/�zYb��x�3Si��*TM�LYn/�tw�H�Q���������wO�����)���co����Wh�����%��80f���[���W�tj�U�xh\F���b�����$���6T�#�C�F������f4`&�Y��������k5x��6���F�gV�h�b*]�K��VX���nR����.K���+��<rf��;�/~#y�uE��B���]��������~�xi=>��-#Rgo3�{�D�x5�]�HIh�p&>Tc���%~���b��@3LO��Ad+bl���fg�z���������M�Tx�e��:[[�?'R��&���x�s[�L{C��7���+���K�7��%���W��[[�����z%�4"��������m~j�|^�:��6(Q�$�i$��hV�m=E�E{�X�����4��}/1��:�,Glb�G&��jL|�F%�Y��Vx������@����x�������:��Us�/�m���U�=�o-;�������6�P�������>13� I:UW�:v�K#^x�L�pAb���hKl��Zr�5����+��%K�;��O-�������
�LYU�O��v���`�H�V�)���w�=�r�O��/�?ke�9�j�z��q�������������F�i<��^�o����fzv|���(�$�����l{�0�BN�D�������U
�ga�.��]�Bo�����_�����y��=�L���Tk����7�ae���U�ia�����qD����I_��)�tB��.���������J���o~&(�\I%	H�p	4������]tv"S?����-F�
�H�6�6���U;|�F|!v�����-�vh4�&D�W����K����$�$}�]��>�<zf��������3��F.l2����W��U��-#d�B���,��s���[�XQdYQh�k���Cu�lu��.����j��Vd2DVC����	+7�������C� �=�L��pu��+3��o[p4�����
j�,%��O�vh���)�y���9��}�>]��!�8+����@/�zB��������S��w�[����!W�]���WR�\[��U�����j�JK�V��[��D�]q�=2�k����]���!���������=�G�L�pY]�*�{L������}��b�����b�^����M]�j��2����vf�����������f	�r���nmjm��U��g�)��}����<�m��QF�Q�f�S:�
3���E��@v�Q�B<K�����VF��Z�V���T�+�Lr������}@g#���`�F8��[����=>SN�����K}]�����}����x��M{��<.��7%�'�A�����s�**�����A��]�A�3V=8Q;*-����)�b��V�T
���������������\js��^j)"��4�z��|6p-����@/��(�Hb�^�C|EC����m��o={v-U�}�p5�
$<�F�-��x�
v{`/B�u6��������<|��jr���O5x	E
J�N�O�z��UN�S���j�Z�'��F����oQ'b5����<����ek�MO�������������X���#Rx�K�.e�Q=`���0�FJU
y���D����4���{EO����sC�uK�}}�{u��^c5P��^��"�&����6D�D����4�������"U�	�d.�
~�{d�<���z+���V�+E7;��4
��7o�4i\����p�F��"ex�A�q��S�\��b��,�\��n���G>d���D����"5{����]S���46uvA��ie�>*Z�,;
�Es�h����&?�
�h�Z�3N;(I7<��~<.#�����:���V96��P������r�� ����������U�
w
��L��j�������z��!�1W6;6,]p���[�y�v�e7�V���n����W�����o4�V�����w_��AM�C���[�������\�3���uS3N�/��&:���KW����oO���n*�����
���rP�~L�aD����=�����K����n�/+��+�l/��69���=�,�uH��H�Q�=.s���2m��}Wi/����������;�:`�<��o���z��=e�1���������wN<�D�����R���c:�b"+���
>�.//?��3{���i��E�Q����S���r����C2����h��B0�ZL,�ao���v��I'%uU*d6E��J��D�m����L;~���[��3&������G]U�����z�U��7�c��4�G�Qj���$���������id*��>�����Rn[RP�~����
����a�EI�%�u��Ord�!���5W`M�;a��'s�=1����:,W�*7~-�2d�Y�4�B�
�E1��A��/1b�9s��C&fN.\��������>�5��W^���u���g�������c2�{��g�y�|qq�s�=��/|������y3���gV�q�t����~���Zl��	�]�*����)��V2>F�����OA�$!����/�o�q�-z�r�W^�������s_���a���D���p�����?Y0V�6w~�zq~��2k���}����>�%��y�I���2<k
��Z)�:z�]�j��=|�r�����bO�5k��/��bu����q8G��$��L���F�q}������K�.?��h�&�����
l�u�]���^z)�O(���m#
*�r�)/��"���1P��L7`��s�MNNf���\@��P�d�`c�W�H�7G?�.Y�R������>���{O�������OU��5B��0������u�����K1��=8Y?%+��AI�+�]������m���jg��&v:^�����5���GJ��x��|e��N^�h�s�;n��^�����������=��;���K=�,�0�n +�`*�/��W\q����~��A�Z����������Y��/X�y���I��1�v��
G-l��
�"�g�������&H3(h�j$\r�	���.�I��H�G).sa������U8i���W����]���-y�P�������-��~uC�V)���������������?q��4���W��U�3��xw�������DA�@���]���.��Fo���Sa/RX�e�"�zk�j�������c�z#G�D�e��G���_~��/��Hc�f���^��322X�~���7��
Z����=����N:	S���K�y����z�y����
���zKmM��N�W���B�4����>��;�w��3�t���'��8���S��~D-{�0�������0����o���({�kY�42M?>3fd��oY1w�IX�]U�M���K��`������Y���X�`	���pr&t��Jm\������M�"P��peNw��X��+�a�X���+�Z�d������t��;z���s���W_}5�{X�.��8E�h�b�
h>��,�)������7��$S�\<JX��G��x��
gq0b��Hh��kq�.[5%���SO�:q�E���.^�_W���k�8H�-������fP�O���5*5jbV�9��/��Z)�Z���2��2�����*���X��*5vI@���5-�/�����~����/�6E����_(����\o�,����h������&O�6��(���US?j���6l ��������`������2G	�t���T��,�X7�A��!�$�z	T��o�Y7���IdN�+q�%%K_.]���V�zbS�����)a�~����*��&���)����(b`�YY�k���������gv�����cn�1�*V`0�+�(
,�,�y��7�XB���Z�x��C�
���Ro��6��r#1f1R�	Tg���s��>JX��R�����w&�mTg��hF�d�v��N�}#!+[�@�Z��n��RJ[Z��Ny�p��,�����h?���])�������$l�����E��K3��MP��c��$�9e,����?w��>�}�75xx���H����@&�
l�sh�5k>U�������Y������o�CF+%�@x6��t�������'�L��]�k��74����CV5} �z�;�Jo�c\��p!�[�t���G��	4�R��z*��������_���L8�85�,��=!�h�Z�WLK������������jO�?���Mp����|7��o��!3���`��K(�a��N�-f	O�U�S���r.�����������t8��
�q;����������v���@l�k��%�a��'�q��*;c������
tq���R+52���71�����Y��Zx�c�o>�e�M��J$\�O�cYM[��Y��.��1f\�
�U[����4����-�	&v��X��z�E�G�`�-8�i8
����~��uXx�e���X_���+>��%���<��n�;��I��{�'��� ������C�Yw��U+/��w���O���V����bd"���O�d�Ci�X��8��W���<}n%^��h������U�Ib0y ����x����@a�����q��q47n�M8J�~���/���I��w�}��������q����kf�Ze���'���E�@�k����U�|����S��
���^����f�&�����L�|��`LeG�%�����U���&7&]����0a@L��`�6���$DW��2�I�)�j ��M�q�\;�;N�U `��1������[H��R/���nC��$�&����~V�qs6Z��Bs>-�yP%�<
����g�	&l�4vj���Fm��v����j>%L��#��������-�E��=������V��jGM���5i��Mx�
�K�z��YM���y�E�vd���Cf��+?�����l,b���P2f�y<����Q�s����ytS�*�����-��Li���15�L�]
@�>��3h�X$����{��I�0�y[H@q�UW���^x���$�3��5�v�v���q;���E3Z���f�*����6yW�@����V�Cqdpc�]���J����&���d��^�c�;set�UjF����]i�lJ���r�_N����J��l�tR��J@H ����a$=Nn�_�*6�:*O��1���'<��p{:s@���*�d�xi�m�3��"u�,����Lj&����>��v�-�I%���|C%� HT� [��������nSp9���%�Sp�|n�X���@�����a� �����%����{��O�� `@�w�����M�L����I��P�]tQoo/��/��r����	��0�f\>��q;���t�V���j�o&�rD��	]�Kh&���Y�x��JUyp��Yh0
PUR
�����vS%���B#�W�������z����=�4i�Y�=f��@�D�W�r�/'����B]�j�K�d�DvOrm��Y�L2f����C/�$@��lG��������d)c�2n)�lI�*�e��z��c�$^anZ���t��;B���R{C�p2�z�����8������gI�Pt �0�����7ef�_F����H�OL���y���,�b~E�(]�t)3#�[s���fc��7t>Q��8�����{����KQ������`q>@�h2/aqV/W���f\�I�Y�U�z��k$��U�hjkk�\DD
=�������^1������Wq����X���-�Z��I!���A�X&U�L�:O��<��&��	��T�3����V#��N�tk�����o.�/�T!
;�2��s�SY_�$&��j��Y�����jL�j�@L{�'�ro�����T����F:���f��`1�Q���S}�c��:R!,����������1�����=(C��
����F�d�;|�]w�
>�c�E��{����[n9��s|��*d��1e�gE�����!E�{�{�Gp0���<��PO*`|NoM���K���V�j������I��a~�m)��4	�lqH����Y���Z�b�j���jw�z���K*U8�t*�E����}����#]�p��U�Cj�/��.�J������d2���/����K�=WmWZ���kj�Kj�g�T\�d�IK��xP�����W�PY���.^�4�:��g<�<;@�����erb$�|F�E��B�?���C�~��Q>c����F@N$�B���(�J�����L
�R��B� �*)t^|j����aS2������2[]��m�����]n��m��).��Uk�-����w:�������d8�M�;R��D�]
v���JE������L�r�c=+�Ef�4W�_�����e�Sa[[�D2^^_yj�m��[5��q17��^��@+k���tF��sV$X���qq!P�M���F�P]����c���o�O40���Ma�y�,�������
�,�0���`��;)M�tM����������Z�l��Bs����mU��
��k,��ZRN'C$6Vc~w&����3d�F��D ��6_�#+bg�� f>�{#F��P���.]sp��E+�����tb6���J�.��|�OI���=�4{�/�V �J�$��c�����d���/
F���$	U���P�����2]��L���P�>���o1KfY�W*��U#��Xc�U�+���f���4�	��0B�k�-�W�A�oW2�������N��,ag�7iR��������3��#�x���>��0 ���v��u����71��S�Tg$�k(A�W��#	��	-��MP��������%O���*<>�l���QT�?�"@�5:�n�3�,��]�XfV����F��U�h�W��x���o�j&�����H�XZ�&#�4���D`?&`�Pw�e�D-7{V����s�B{�7��e�\��6M�o��l��������N�5�
M���{e�4I�$/r�����I�y���@�|�������
e@��_�bd���M�������6h���t��W�i���P&V������f]h��o�8q2+���2k��TlP�{�'���H:�x�G[N���������J�� �1�d�3t�������Y���������\6�q��j�=�k����-}��1��&
e�A��>��(�M���'����m!el`Pw��@@ 0Q�����v���V�������x��&kE�����m���t6���\�77#!���T�;�Wj@�� �`=�NQ�n��tU�n eX~��V��cA� �� ��Dfv[�^��j�q��kk�+}�SZ\��V�2o�$v�a2�����U#SQ�K���Dv��'`�����Q�E��V  � z��#���T�%S�~���GR�f�Cq����0�hR�uO#�����5g=�%���22Sx-#(�#Qe�FzQk�v�#=,3��E��B�0�PeC��s��������;q���Te�	U��x�e������z�L&��ppzu F*F���$5��>�$F�q<aJ��0��z�OD'���P  8Vt�T�!jd(>�3w�N���b�m����[_c���}z�o��e�n�M������&5�a������Lh�,1���T�_7��F����9�����q����Ot��S�U�9��>��Z�2=X���������bz��M��������[�x\����V:%�[�v�`�!P�����nM�}I5NL<z�{����ID@'��q���7�N�"�6\�]����+���
Cg]�-�:���,.BsF5k(��x&N��RD!
X�(`]�+s2����1Y���d��
b�~����#��/{\7y��|N�/����'��0b&�Zo$�
�+�Q����8i'�nN����3���&`��]�rU�Z���;#6��@�h@\f�z�Gn��%�1Fg��B_`������d[��e�e�Yf���L���7m���-��������n�`���t��mb�.0�����b*�k����a@���V�uM�sm�kQ���V��KjdU�����k����O'���a^%�x|p���J����Z^��k��N�E���E�K��O���I�3��I
����Jv�(�\���fc�xd����K�l���R��t,��u��/#t6!?Yc��~�uWf�YW|s�5?���~�;LQV���s+�k�]k}��U��z	"'��a����+�4�$<��9dt�)A���U���&`�%�V��V*��T���FZ'#�d��av��[c.]�|uu�����Y\�
��X~��Fpen��.�0[�Y�V�M6bq*�����>$��	F�=�+s��b#oK���W������To��?�����P�*l��W����9U�Njt]��^���wF���C/T��	��c�0 z�]!�y�3��'`S:e�P-vS�[����M  �H����vo�Z=0����1��N�]���U�����fG�2s�	YWf=���4k(�c��4\�	0�3NS������[a���O3��sK�������E������B�}Y��x���N��[�l���|���;�>s(��/�J���,���+�G�Z�_�6���b�K�h�Kt�����������������#��;�N���C�ppz�-�Y����f�X`�!�������,2$7\��D$�+�HO*�C�/����,��lk��2��lH�c���Y�T�nYm����u����N��Z=��
��r�F1I������V<��[����T8�#.���x|���t0=���l�����SO����/�r&\�q^������.rD�(�,��2��s�u����*����������t\��-���`2:����{��X���l���l!���
:=��F�u,�u&��3��:M�S�N��9�u�
�15H�G���<i�@����k�%F�[�4i!�X���Y<^\m_Q�XY�DD^������u1[�������������`\M�q�����3������T��>��j �S�2,u���?<oRU3�Nx��[���z�drFb�j*%�5�ZMF	4��t�����S���
��R�U�1�.��"}��;`��������&��������M�-
R2������3�0c�����L�����/3�b2C:G9��b�6[�UH�:�Q+��%g�Lfw������=��1#	k�L*���gF�2��L�_��L�cq������H����d�wG���K{�;\aOTY����b���RY������/��USTM���I�I��{�	"W�H%zX�I�F@j������9�;0��D��CN'3�����x��6�VkUR����o$,zsCC�%e�����������H����jUU�F P�gL~�T��������l==�E9����Lx����NM�=Q�e�������l1\�����G��h&+������Y�.�e�U)�B�Md$�����1������R���H ���_�	\_c���>�����9���&��l�����s�?�R_����~��Kd���	$�<���
!��~(�M���5���?tL�u�p:	�����z�)�&�������@@ �#�i��N��<����tm��")6<�d�KvT���5f=�W���gs�9W���O����0���9�gTbD��oO;�#�}��}C�G�2�Y�t�KZVc;�N����~r�G��X\��v���7cd;�?���.���$���&`����Z0�\�}MTP P�d��Z����h�;l�k��&%g���fen!�#�������-w��,�����E1����l��4w:w��x����S��J`�;��Va���&�V�s8����+�b_x� n��A�X��{���<ZV�l��	�w�0X��E+��@:�OFFcc#�69I3���+pe&�W�����w�����6�����u"2�Vh`O���p���a�o�y`�=����S�ZX���RZ�&1���������`Rw���ZXVa�*m�d�Y�[��<�N=�`Q%��@`�#@�d��dK���b?r����,����+�w�����m�����&���a��U�����55�$��T����E�����\JX�D�;��?�=0�R_do ��g"���	$���#6����p����� P��*.MqX���Q�@@  �^0�fm������Q����n���f����Vo�Z�����;�T�Y���j1�D���5��:��o�p�6���^9����?�>4�Z��7"�SDO���1V`��H�j$�e2��'h���O�_%L��V����t��!�:&�{�R�"A�T����w��y��	zl�w�������J��������Z@�/��l��mM��@�����>%������$��kO�ax��>���Go���HP(�j�0��TqMK��������������Y���:/�Mj\�C{�������9
�l���B���Zv����=����Q��l���))�JH��T��>�������������!P������/����~���u�����;6\���&
�������klVfX��V��r�������g���5����d���n��v��<�R%`�����$`�&
�����\���6%����F@�M����� ��e�gM�s�����g��X������y��J�����l&���%%3�I:�&��>$���c���8��M/��XlS�@�0�{J����U�@�c�����-q��@@ P�`��{������������*?u��������:�+0�KH�S�e�}�rD@��W��(I���E���0)thP*E�����q/3�$!��^U4��#�y�����hO��0e��8��d"P&��^�>��###zj������??��%��-:��3����x���p0�.�v�:���w��qw�)�bF �T���������e��e�^{��-I����/�V��N��'>�v���������.;��3_��Q���l�!�.Y����l�E����@$`2�#�~���$��O�SEQp�����o�q���P�o�188����������{�?��4�l������s�r
�����wW����v*/����zUY���K��i	u�������RU���������Q)�x4S9j���
@�d`�e7c@t8�P�u�(������F><<O�|>NX�d���'����3g���8�F�4�%�/��lo��.�����*�����1�����,�������N�|c�ES5�k��z)���t���QO���OSw2��n8�w�f2�3��#)������8�;6p�����D��'j�a-���C\244���u�9��3*1�q;��h�,�������y:%��bj@ee%f�t�)k
���^��Hb�3S�L��� ���n��?�Y�X�� `t�����k�X�{C��g��K��;�����wwwo����+�hll\�n�j��ov��%��m����3��e���2@�x��5�7o6���������!������K;::m�����]���]�=�.��0�����$O:��^x��:����3���G#P�/�x��W�|�=���~}�{�9F�O�����,s��k,^����?�H�����y�Pb��F\����5c���b	���
0���-^��?���6440mbM��h���w����!���Qa���a��y�[/�:sM���I?���5��P��Q9f�O����R0�7�A�g,5D�*k�83\�
v<,,"e�a�Xq�����MA6T�<-C��3C���b��orc7O�����|�36l��?��O����b��kv�w0��By�]w�t�-�����;���0��Q����?�����7��A���G?�/��*���������{����/��g|�����wT��q�>�Nc����Y\�6��_��3{�lLh����R������o���C��Z������?���y�c���/���}i���������UW�a����	g#d��O�6�-F��	M��6�u�Y����y!�n��v�������=��~�,2��yn2*P�e~�[����>b�Y�=l��~���!���w�C��_���������SN9�K_�S������,�� �z�7��
�{p}����M���0���}��+h�����,�n��a�i"��`f6�����0;IO�k��T��.�
)����wC�����������+�l��	�e�`���Lh��QG����|`Y�G1��>J�����"�������<���x��QY����-�����0�s���=�K.��I!�6VI�&7����Q����H��f�=J?�F�v�m�R��������2k��k���c1IDAT��dX@#��
�V%���w�R%`&���e������q�p2�
7����W��z�|�M�����.�@�`G���w��q(l�P�c�z���/Q��0VjH����oy�v������CV��W���p�����r"����K
F)o�o@�i������X
��o��g���%#�f�L	�: �at��y�0�P��'��[o�)
��$+C���/�r�����|����o����3�P>�'���	��n��l�-���JV4�Fa��������O>�%������tZ�:��R>�C��!�LcV=00�����
����V�OV����L���sL����R��Q>�>e�N����u<M��0z'\�0dc��fu*>���0 B��������o����0G1&.� �,��
�_����K�\���U���w�M7�0&0� �Q����w;gF}�k+����{�����.�����6 �TpF��gc
I����5�\3���s������������3����e���|�+�OH�<o'./+P�V����Z5���.���36�"�m/����.�`��^�q7���'�l�P�J�>����
�������~:FC�5E>��������S����o�v�O3�������Y"9��
4��

���L����/7�Z�����LnN;�4C��0��h�'v;q�@@  �F%�#��<x���^l�Y
�h��_���p0�|~B��\��?���l��&�o��J�|+-���RUA%��Re�d�h�b$����a4u��(R/l
�3 ���>����W_�m����>��`������>�9�Z�������m��O?���4���?��bgD�����+sXE�3�R5����K�{:@������m�0�G����?�L�B�:w��U4�W�S���W'�|2���a����/#�b)�'���Rzt���/���Q�����@@ PN	����h��"���|�������������4T��4�����B����������B>��/~�1�H0��
;y����!�_���1	����lJ&*��>��x`�|<�-�S�� �)[�����
�f�>���'1��JX�1{�����3�
8�|�}5�lh�1o���F;M�l�sq��cc�	�S&F[B]��FT_ ����B#~�A�e2�+�X��z��	(���A��>��������B���.�3��i�vY����k��em8w�� ut�04����9p��@��p�=P��ID����-�?�^���qY�5ne�������c:vX�� >9��9�1��q���_��3N_jQ�$�U�3	rI�qG��Ov.nQ�Lq ���+�L�1��c�ORQ�
c��
G#L����a���
7k"�q9�?S�,DcS�!����s_���@@  ��@@  ��@@  ��@@  ��@@  ��@@  ��@@  f��{]�|i��IEND�B`�
ycsb_workloada_with_uniform.pngimage/png; name=ycsb_workloada_with_uniform.pngDownload
�PNG


IHDR��z��3sRGB���@IDATx�������7ow���<Z1@EQ���
!(�������	"(
��C����n������=������s�w��g�w�����R)�B�

(P(�P@��B�

(P(�P@��B�

(P(�P@��B�

(P(�P@��B�

(P(�P@��B�

(P(�P@��B�
��������0<<<::���4�Rj�Z�Z�t:�md�����6�P]��7c�5���^h4�����]#%c��r����Mk���>m��~��H�5��.�]��������*,,�X,���W�^[�nm|�.��`0���t��4����#F���q#�H�su��|,L�.���:���SSS���;�6��AAA��lnR��$�E���K�|�������W��#���-��}�����7��@Qpp���������t�����Yo�����-�KKK����������>��w���6��0u4�1J���@��2�I�6�75K{����?�j�Z���*���et�������J/����n0]f���K��
'����:x�B�\
:��:Z������pw�.����yKV������V���6��i�Qcj���

�-X���b�v�h4�T��
����+�"�A�*� ����j�z���WW|�����|��C���Z�N��(�?���g��S�4d��5�pB����E,WCt��'�4�J�6��%+W�<������'�|������[�����/���O>A��R�v�r�G�U����E(��JO�����1��_��Gm4��1/�j�^�C�������j���5���4~=��{�$�T��A�d�����\�K%����q:=v���P�&,��n����8��	
�=67��.���G��}|
���X���Mi���,J�������hy���V\.[-W$�L^��&?�@�m�:D�r��+�z�R���H��UR9U�UTM��A%J��P}�������r���_�Ov����N��!��\mDx����S�o��aaa111�'O...f�A�F�i��c���m�����Sv($&&���?��o��c�=+QJX�n]HH��7�XTT�U��
0����'}rr2�����6��f��9F��<���z���&��S�z����>@��F�|��&�K_�3j��$5i��D�y�j���G�����������:�J�Vk��=*p�T��.��::�6��fR��-��Vi-K����������Z�:�`�I�Uq+����W��x�Z#�J�|3�\p�:U ��E��6z4
%�F�F2�,9!a��G$����z���K������`��P�X����a"�yo��Ckb����{����=�o�t��������:2�^�6[�ee�b����]lq[,��[|_CrjpR�����c�w�~��I]�6(�fS��SO=u��'������K[����Gc
7h� ��>��r�<z���333{��=e��~��-X������^���?���5k������?��wA���,v1�p�
�|�
y�"�z}hh��Q��L[�
W#�r{��H�t,��a%`����������&�`B�
N�H?����}��(��
D!���G#���%5����v����N��Y^�Dow���Am�����.-w��{��.3P�*�����*+�X�!��-ey����5�'����=
}~/��l��Bg.��:Z .48��$����ZjcEB��C���YM	Y�{��
��a&�.Q���*����bM��=��3����B:�<�4|�z$E2��P2���2 �P`�**US�A������r2�{�I����$B=:h���R8r��%F��A�����F.���Z��2������v����.xz��yv��a���#F��*e�j~-�R���_���3��x
���{�n�T� ��$��p��s�AD��~�z�&���w�������������^	������������>K�i����K�.}��������>�������N�J�{����~��W����SO��
v�E��7������_��
W�H�G���Eo.��aO��w��:	/a����0�'��)c���� �����yP���J��$��Q�"
!�A�C8������:UBH����Pi�:��\e��,��\x&�`�������L�r��IV`�n	��Ub
�rK�2�&~+D���B�+��&�� %UE ��~����tQ9�86QB�<�V*z�����Jd�,�9��
��������BUT�;|#B��*�p��q�j�>q8T��p�	�tb�o�]0>x��e��N�[�*�
L�bYf4�Z�Qm�k�&��������sJ���o�6�E��6��1���m �`}�zN�4>b���#$��������k�j�>���=��/K-s��ymr{,fW�!8�����eA&!��c&����\����E�-))�����p���0���
r��y
�Lz��D�^��O����%�?�_�0���P�n3�F�Zo��,]'������=����(��n����e��`����@M��d��8�/�S�Hd�����NA�[l���&,�����v��C3J�b�c�s�K�E�b^�.S�����g���v�Yn���*P�fsY��rc�q��V�����O�D�[�2:
�)�A�;9M�x���t�<�[L�r*�pu�C
�9��3��.��UH����p%M����������I�����U[�Vy,�D���|h��j����mZaaiM ������ 8��>��j|�B��:��E!���&���I [ �j^u�4��z_V�Z���hN�	���x4j����%F����<^QT�6���;i�z,�Z`�e�v��|��
x[�pE��v;=.�m>�o��~����KV�<��V��8COJ+t�U�o���~��w��###�[�� ����0�Z�d����������{��/��b���W^y����?k�,�d����P!��N��d��'#�M�6��hD;<�Vr�t���pg��n�T?��<CLG?��U�����)��0����q2�0�0�������'Z:��q,V���S��[ $������d���]<%F�aq#�����X)*��C��}� w�U[&���o�!��*+�kP@�����Uy��#fj	i5�����\����-p�"F�j9Li" ��	���Z�F��e))�L�������kMi=*���_��cU_�T� ��j�p��e�f9*j�x��pS�����V�i�5�+�mE���V�fGi�6v\��B�������pb����%���#w��~�\#cr��2'.[��(������U������}��-[����_XTa���+���=z�@���1����N��g���.�;w��3g�ZlL��j%+�P@����tV��?�>&���Q�F��G:��i�^S`��Xn�D�.7�)Y�y��0�y8e@�;��Z��bN�&�(K)��S@Zc��:~W���KW#��W�)^�
�9%���r�H �=Ba��Df�}�gK(�m�%���6��5��6��B`�/ig`�}��a�
�k���R7��g"3�*�%���I�.P��G��n���s�*���m���w�y�j�:���� �&�����i3O������g��	���H/���B��xy��h����r�y�����O���B��ee���z�5�Q�k�c�^0k\1����l:Y����Z��x������ �'_y.�W/E�W(��( #Yu<�����EVGn����2~��F��B��3X����qm��1x�j��M`v�n�!�'z���,�9r�8�i3^m���,f�W,M*��j)�R�V���t^,��E�� U���1�3f�`������(V�m�p]5)�mF� ]���G.��&���(�����2O���E��fmR*R(��) !��T=���pZ<�%���\�@u��������.#7�}s���!��]�/g.��\8Ts��EA;x�`t�2�6XN�����5�� ;���r��UlR��n���A�x(�%E	�1
�/��qA����H��T��C�Ft�*�P(�	)PY�$5W��n\-!��i>�y���8s[��`0W=\�Q5��=5c�i$��'�'Y�}�p���A����L���+�O���x�3��W�]�^Jg
�$����1�H�h�/�~���i�]Rb����u��E�'���Y�c�w�]��n^��>�qg���^�
�}�hK
(���n���uaw�?vY�����>�:��������h���P@��B����-D��+&�t��=^<)p�w������j�������f�

(h��Q��6��j���,\�O�5R6��	!��X���si������]��W�@7�
��#.�uw���_��n������}SXYw�^�s�7���m�>�r�������e_��*
cE:���j
�|;����o�\����,�����6P����s"q�{��7�������������'B$�p�]0��Sd$��n���V������4�n�6/9������B7�H�/I�<��4�ga����w�yg~~~�{�N9[e����1�TB��21)�W���W\'yF��Oc����w�pG��g����y��s�vl�<�h�,T������v��w��O���6[�>���vX�Y�H��tf�,]`<����9�W��Fhz-��_^�F�!�{�I���.�W'�B3�-�/�%����B:]^r��X���o8%�/R�4�2UH��\\w��
����EV��_:�r�TG��c����j�x��+�86����c��~�����a����$�&�L&;� W�%ek4����d�Z����c��8�t�7��M�3�����UjT�I�/���B#.X��z�V���a�u����\�9�a�sY���2������8������r�m���M�����{(kyz���M2����t��^��Pkk���&��O�h�]��fe����(��D3���}�?�Wy
�\7k�&<�@aa�QG���	��&M��g<rn%�YrV�>}8z�������<��H�t�I�HZ�h��m���1�=8��/����>���AS$������G���h��Hq�)�T��K/����q��IWo��u����0����2;�Sf���<3��y�OF�D?�������V��7����{�$�Y��a.5'����j���o�`M�Q���G�n�����lN��L�^����eNK.s����4�:-�.k1�,����	��r�p�(Pm��WH��%����=�����^Q�rU���R����9�L3�Z�mj�����
.�����G���??�|�.��=��s��������=��y������^s�5x����hh��O�����QP��g���u+�Mz��~���p�
���3��� =	�J�=�����S�SRR���K��I�#��������7+g�]}�m������9x����p�W9������F���(�-�w�Z�/�����k�l����1jt�:���O��aR����
��Gi�!Z�p�_��k����+~����G�%�k�y\V��Q���]�b7��b8i���Q��������
B��n\�!�v)��.%^�@�R���B��{YU�6�Uza����h�<%���<(Vp����/_����0��b�3d�4��#�
q�����r�J�{�.q(4�0���������/)���?��K����$�����r�" ��a'T��	���z�";�7������|-W��-G��,)B3'����/���������l�g��o����O8�d���Us�sw�n��]��m/SY�%��c�(K��r+�~%M����5�������1�[�0�)DN�B�~>qj�Qx[��k�G�Ui0���!��O[���"�d��e+u�J�u�����ks���]V�T����wX�8Vw���hMJ+e+��� I��?
�����T��P��{�?���VOa�b�B2RZ�M����KE��W�\A�@�<E�R��C�b&FY�"k�nI,� �G�l�|����� �����)���^�9s�9����YF�N���/���cC�;���=�?����y�=�>/��&;5���!G_��>A	�������Ns�dfU3�,���U��^���^�,��IL��|{�E��h5>�p�Z�����1��L���:?p:X�f����4��hrc��K2�r�\�B$�n[���V�s����#G��f:yT^�0g#��b��i�����;%F��B��)�G���������
"���^�u��4
��
����
�.�����:�������l�y*���7����b#FFL� Nz���8��3F������.�-&b��������w���/�����������,�;�H9���o�)fn�.������z��U��p���q�E���./>�lj��/��:���!j��)�t�fN�:3->LaM����z���3��AZ�@�o��'@�"��H�p���`���}�K4����$�b��z�NdB	m+q�#�.���[�8�����,�[��	H�D���z3��dS(��( �i}�[��O�<(���c�[VRa'����	�N��0
�b���G��~�|�I�����$��g�-1���XLi\r� :	�r*�G#���ac���|��~+�$m�hf�SN9�{���Y+�S��_~i���o]����{f���s.N�����z������IO��T��w�����-�:�U��	�l�����3Bl�Z��b����N����=���x���t��'�Fu���e�����f��U.D���<F!�
*#���MBok1�p���e-��]u�V5����.�6~�Z�#������a�/K����N�����N; ]���-����P��������>����mE|��'6�a�E  DDL����� ���r ��53���n �xd����hm�i�R��W^�z���E������6�����HW��G�C!h���YN@]QQQ0��|�Z�~�p�t�"}���{������n����Q�
�����g%���1�o�oo��]���+����2�	�
o*���p�.���g,.�g��l��B.�����Yg�h��L3
i�qGjQK��!����k t�OpOc��h�h��g��N�IKlaE;��QE#f��@�9��E���i-�-z�}�����/u�l!�W.�
Q�U�r0�"�����1ee�MCFV��1��
c���%Kx
�������%g���l��\�G�E.�m�
(���h��xa��q��~�r��{,;�R�b�u���q�\.[�����^{m�m9:��-�f������J$��e�H*�3LsB�Lj�A����M��ZS�`���6�k%#2���}B����A[�O#�l������?P����(��S��oz�W

4H�ZqT��������8��<�'W�5v�
w�!�v���'�u�KS|R�2Nu��!�W������S��o�����6w��>��98��rZ��_aNq�$�TK1����
'���1��B'
m��G�6���
S��3N������\��peJH��B�#�\,�U�E�G��*Y�B���1i|o�����9��9���h���2���|��A�&
�st��?>I�~��$��C����t�����^�v�8z�0HKAI3-�h�L#���5��n�9x��~��3�=P��[��

��3��T�E^�($�������
���l�C�3v�os��F����*4r������<��w�����>[���9���8�^^������{�:������c��}��������ql�����+�$!��l�u$y�Ye���(�n
�z||<r;���.5��g�NY���`�A���B��9�L�,���'(>n��a�_h����rvi�F�ykK��1Ps��9Yw����I����(f�����7�4��1)�:
������V���*�kK
g�b�]\�����Oo��&����P���^)
\~�q�iM��a�K6��^Ai
���~	C���,}�|{a�"v�B����,�9������.�yk��;�^�2g�3Y�����Q�R�B�V����1���w{n�4�;�s@6G�p����^����;w^q�
������s���YY��I7�\������eN�	7����C�}�`����vE��R���r8��,+c������&<7a�������3�6.��J�@�S&*66�����-C�������99�������n�L�{�;���q�^��?yv�����=�2{_�v��5���	q��b�OO����}k���H��]���-X�G�:�������=un����Gd��H��W�����SSi|{Q���p�3�Wg;r�l������(Y�~�,��)#�)���<kF��J�7�B��0����/�S�o��������{�W#�*��v�{g�]�$�
6=6������F-����*������(�H���_�2�$����������^y�P�#Q@A��3ur�����/�'�=�X�D��"����3���/�FL�3n�^�8��Or^�y��M�A��v��*�_��Gr6=��b|[�z�.R�Y�+�\�[34z`f�w�e���R�o
<��]��+s�q����7ad���nyS��k
��
�5�`�6�Z�������f�4�sB�u��)��#y_���3�]9-�����������]A��E��w��|3�h�7Mb|i�VcJ>/.h����m�Ng8:f�?��I���)�v����V�Z|������������������p%�P@��B�:(P'�f��������#Gb;s)����d��=�������36o\�2#���V�3���p�3��FY�m,X��p��^W�
W#N~���90jfL�h��bw����6[vL���~#������B������i������,3�n�w���|��e�{�������P^��B���u0z�Tl��
������$��P���n��Y�:?*���=�K4�#���3)t��g@�,�3�&=�k���K��x�W�h�F��<0������vo�{{r^ry�}DI��p�a#oO��G�>���rq���
Pb�J����K^~c��u��?�������a��0��E)�
tC
���s���/�����c���a�����1Ga2��zc�@�N������[o�F����W��m�Iym��`���g��I
S�n�}��"g��
�J� y�WyoK;}k�"���dF�4�U�p���;��nuPp-Ik���j`���!��B/�����/��mvypN�Q[���(�hMV����3D�6 ��^!��/�`o��%�=����5��py�_X�����~��~lR����m�4�%�B�v���8�2�V��6���&�����*�1��T�������3���w�
0+���z
W��-c��7���W^y�Z�t:�I'�����K��h����b���E��Bu��wBM\^`�*W���o�t���.�:rz�-�������sxjw�k�?��������#���-�	�WW�Q�5)�g��~c��iF����_wf?�Y���m�z���a�KS��,Y�3����7
���W�e{�^=�����Z�d%�I ��=��k��7��6��eo����~��8�I�(�
�*0
qx��W��7��uj�C�V�R�O�ikn��'���������a�6m�t��g�_}����������
4p����hI���L<^����9��]�v�����`���	��������4��F�bC��}�pe�������=����V�B@K�@�	'���Cb�?����o�#�k���q*��>}zM��=��_	�"Y������=��)S�p���5k'j�y��<nC\�&��\\M����{.���h�1?��p�������~p�->���IK�A=3�����b�5_v�� �����aW�����k�������W����^-�#���@��I�_�
�x@���D��;���o���b���w�Z�V~7����v�[>o������%I��MX>����3��/��u�%�z78������e���d�����U����Z5��~N������<9�v|�8act���|c�Q�vz��Y���NL�J�����z�M7��a�;v��pZ"|*�W1�Ez���� [X9�	<~���_�u���w��yd|�����p���K@e�^u�U�_='hCl������/��|��'�wTl���?������:�������?&N�x����rU+���6����D����?�M���z�Lo����wjj*�!
�8���P$�A|Mv��
���Di7�:�mQ���U�oQ�b�|�u�t��>��~�Y���)������[�v:UzG��'�l�y4�>��ihb�9%�7�����������6.���c��XZ�~#��E@\������O�.��M9��Z��B<���	�:���E�F�/��}p
��bZ:���!C�tr�{�v�b?m������?.��=��w|\�*m���Y�������?{�4���<��			����������z{uv-[yDDD=o.<���9�e�zw�]�:�J���?G����q��,@�%]|�+V�@8���YP���k��vT>��3`U��/^|�����>��������}}}��
	�.��Lw�S6(�Bs������
y���A��~�
M�� ������/P2�tpnx�y��d���:�0���3�}�Y�Q����OL���W�\I�!
�=X����q��O���2�@�O�������[��Oq����jM�H����x4&K{�qy\Z����s-S�V������s���<AUZ�����
�=�<lhy�6������F���3�������\rw�q{�:�)�0��~�����</��5���.��>� ��^�H$����,��A�s���������s�[
+r����9��S0������Z1�|fU����8����N�;����jO�!x���!}��fM�A���5�P���/B�x��w��2�L���A��Y�-e�~e�b�����2�d���y�4jU�K�N�/����V�'��}Df��_�]�*ky8��F���������[�t���eP����?N��,>��CNQ|���8��O�U���#F)`�,X@x��%�������?���?�fN`Vd3NFF}�XM����x��TG����p�B�����WM�' �_9@�A�j��(!���ax�B��L�6�'fZ��N @h��?88x�����r��T�s��0����]����Z����]��wB��B����:W��x��x��}��B�.o��##|���|��Y�+���9/�9�U�
��Gp�U�������_�'���
�e#�r�O.���jk%
�������:���kV���_WE��9j���A��e�����7�����������=c}����]�g�u�������D�}�����s�[�}��*�����2�N��4�������j�s���<�<�{'o>
�M�
�}F#��J���^{
��9��
a�:��$��V�=�i�
��������-o��1����H
�H53\���~*�1`0I2Bv��w�y������~��^,z������c�r���s���������D�5K�E���,�������>����F|���C���9�j�B�3��v��{\���Oo���V����W�r���N}O��\n���w��*�����R���3��}�_�M�#o9������v�>�T���n;�9����|���������������8���3�	�f��w����rJ���]���;Wg���������j�^Sr�����/��/��S��7�LDjT�ZAW��]������-�d�C2Q��{���`j	#C���?���0�q�T�k@F���s&�Z��o��s ���� �z��r-T���4���W�����{��G�7B��<d�>v�p-r��M��(������ +V���aM*���H!�k�w!�z��_���/�����B�w�^3T��B.�$�C���;<5��1c���1z���1w%,�g�T��GSg���km�!�O�K��<����?�xI��:�u����A�w'�_:��3!�!I���|={�lNq6Yk\wiv�O)���]�������D��p����z��[�^o�4oK+MO���8�(�(��3Wyy�������%m���J�Xf.�2����.��k�9a�8q�Z�V���p���k6���Xf:k>E:
�G��tg���aj�8Q�"��D��~��=�0��1����/)6S,�[[z�}2�`
FE����G���_|�x����?NQ\�H-�brnHL���k�o������xD��
�_8��"��%P7���`~��3��q�5bO��^x$m�4,dX5I����
�~$U�F��~'��m���Eg��`a�M�~VWE����y����^�_��a���_WJ9>���1	��������?���PV�O�gi�������q�A��Q�qG�����,�[��j�!ug������j���-v��s�Y��-����%����5
�K�7��L��53o���qY;w*�h`�5�����>&k��~Q���F&���`�k���h��/XG����Z���	��Z�v�����Z:��+D<�SA������j� �c���P��c�R��5��S��w�r�_X������~�5�BG�9���>���?��b��3�}�h�u��iw~�k����"�G����������V��r�4
�Kx�;gq��_�����!������b��VR+h}
�� ��8`�����ZC:`����14G����	K���vj��V��e��Z�2o�������������=i�tsr����8Q�r���y����mI���_���6T���BQ��_��������;z�^tVR�;���3+��jR�Q18�����&-�y��A�&�����L�i�����D
:^~��F6�0W#w�d
0�b
�9
m4���uw=*I���3j�?�Y�����_�,*�����;0w�i��t�e�L#��D!!����w��,�e��� �����}����|�������W�E(�\�lG�S�{�(�&Q�d����[b�?y�m�o0o^��v��%�^(�B�]�
0�R�^z)lX?~��W��#���tidCt7D�����w��|gude�g���������������z<�D�vT�����mN���O��~�lO�D
�o(��F��
����y�E�9��}�����@�{�����6���U�&"e���}k�'-
�`���Dj�Q�dt��Vz�P�:`LB�?�x68c�����1��^F7��4|�`��S����:�\
F�p�%s_���J����N!��
���O�������O��s�����@��d���R:&h����D\�;t��<vF=[jS�:4g��v��"���O3e`�����my���~Qs�V�(�J�y��ms���U[��	
40�c��.^��4NNx�M������qu���,�?�}�M0s�y��~=��m�,u�GIZ��m����������	�E+��z<��>�p���W�G��(^�Y�&>��!�w��E�p�[����wo�Vu�z��Y��f���"��%�/y1�����<eA�y��#�f�������{�I;HSf�MS's�����.8�y`�P�
�dbg�u[zU�����	��Y�+�����:b����K�:�������D�|�I; ��7���%?���XF��#>��p#�,������2��M>>&��>�W��}cO�KG-� ��a��"�!q�g�?�J��w���#��O��f~��ow������4��PX)7��@8{s��f��d�R�N�����OB@���t�&n�By3w�@?��)�7����o;|������8�k�����bbS�DzmPb����f���mN�5��KNx�����<���O��W��dp�����q������fu�v���sS��V����~v/O�YK�\�f`���k���mS��`��>{��4[���e��.A�v�D����n�����k"������X���sl@����������T�������������	!=/�������)w��}�^��Y���m��m���Nj�r�y����Yj�[�`��vWQcKQ�I���s����K"FL��qb��s�����f
O�1r��������N�>$hFO��yA�?��c�rc�����5�Gs<���L�����q;��$�
�44f~L�If[��i����d�a5���t�,`���g�����qr���[)����U����Dh�&Ysw�����}����`��Osp��v�����[�F��
��
����M�,uiT�i1����W3v���N1�D���K�6�����F��4.q��W���#�N���qWE_� ��������9bK�BN����3��'l�N#�}*Wc)�q������O+��&��L[4����U�)P(��)�p��gT��C���`����l����O��*KF�#]B|�����7���?��ow��9��R[u�t�[�y2�;2�e>����������N�s��K�����w���iK-��{������D���NUc�[��@�L��B�V�@��x��>}:��Z�1��H�az��r�����n������{���3�=(��������'
�-1�����kv������:A[��f{����W��_�3A����[v��u,Jp��
�u�"=N��_?����������c`�]��J�
t
4
�q�b��~��������JG�
�2&������h��G��k���������k�Es�Y�63���m��~�wvN��*�t�X�?Ro_��������?�������
<��F�%=��iK����LN��>Sx�~���;�Q�1�9)y
�A��5�M���G�C�����rg�uV�#���ccc+�i��q���S����r�5�p[�����Yw�����o\(����Rk:x�~���������Tj�:v��:�P��w�
���L�.�f�����w��vOr��.wy�T7�����7��}�H��_|zr��;s��\�6nI'��m7g�_\r����O��97����Uw�\�I��4��(��b��K�r:U�F�S��U���)uk-�A�SO=�RST@r�=*�����+��'}��V
y����(q�g�}���V��>6l�#'M���K/��X�u�Q'�|�\�J����^v��~�N<E��bz�%�]��+���#y4�����)i������7��f���_������`�^����s�����4����\�#/�)��}{b�)w������5Y����q����	o�J��P@B���v�F�p\v~�N���[���> �r������UW^yeFF��M�R<e�vsLgt��o��&>�G���W�O>��q'�w��7�x<KHH���������/N���@5����'\<
'&&~��GJ���#G�<�����������(�|��1c�pJ#erb#��Lnq2���I]EE�7@R,��_��1�����/�����}���4������z���s�9x!:��c?��S�O��O�a�����,��A������i����q��>q�D\���m�m���������`���/����O��t�����{eM�(�y��5��:mf����i.�Q����o������]�3�o��K��M5�w�E	!���{��,��*��
���pY�3~�_v`}��'�'-�=&u����;u����nU��g��������zF
s�����x�	CN���+.��fS����Yq�E��_��u�]�Y�����`���3g���k��i�6m�g���[a%�4@l~����~���m��C�T��q �H��w�r<���=��G!�G��3g��U��D�4��3�$��5k` )��k���F�D#/<d�����?��7�f�����,���'8q/����3����m��q���H�W�^M��v�����G�~�i@�xVIII�/��u+o�}�@IDAT�j��v[/]�����:t�O?�T-[7��qK?S�{�fe��	��'LH���"����4]�K���/�m���Q�]f��qA�E�1(rz�����l�����������_����1���:i����X����m��f$Q��P�����#"����/=�j8]��3f���2X�Q���o��o������>�`�kq����\ ��
P�`�	�s9a�����1�r0'���"��<��L�@ �e���9���G%�s�1p�111$ {ii�������.�U�%''s�Wzz:xf��B���q"4x���Q~��'�K�c�p�p���,h-<��S��G2�
w�u��h�"��t7E���c�����deDL�Y� �={6�����V��t�@���UQ��k���_�l�3�K����O]�U���������v7!H�v�c	��38��>��;�����>{r^�V���GU���Y����IO������g�|3����y�)��<�ZU~�f��
�E�.�����������5��2z�_h+��j��
������l'@��������i��}���u��{��|��DH�E@0����%�$a�
x�ia�;v��3�s�A�$�K��@��m������u����a�������%�,�����D�\��95R�
�#�&H&�u`,����G�hK
byVv}0%R�5��\���v�-;�_���/���_rcRN��=V�H��2�p�s����+�G��}kZ��u�&�0ihL�
������������b��J��|D�G������{E�u���hU�0y��=����)K�{���Q7
��1s���_��:��r����8��k�<��K��T
�(�-�
�R%o/3����N�����;*+|��j�h���2"�����f�6,\��������?�h����<<.\&�%�G�u���H�	��b��p7�����2��DV�!��8c9��(Ek�euM�-yI�EE��i��y�r���R��)7-D���0���& ���jE��m���l&���8���5)�bS��,���H�oBI��;���������c���h�5.|����?�n�k�[e�]t����5,yG�x���IC���q�T�Za��%o��t��q������rr�;vO�F�aHM���mB����p3���Mj��]���O���	�������gd���U�[��Q!@����5j9D���:d?��(��@�����
e�Ej(�
�����y�-l1&��XAW� �������R6�?T��&�^��:��}-h:u�T��0�hj�H^5�����8~�0#L�;����`9�k�����<xpXX��&8u-[����\p�H����E���!�v�@�����"�
fm,Jn���"�&1
��Xo��%��%�\l]����Z:z>h� ���.��/���j��tKi��~:�	�-��?������Y�/��O]*�C�oO2�73��_l�}Bc.�Z�/��3���	������=�i����5�gD!T���S�k�I�;n���n��������}-�������?,�wM��F��X���\Vu@���w��Je),�����or�>�����@��Q�&b�X�i��z�����Y���.{�(���1�z-;�Q2��E�IkR�)��Y��5�6&�f���� %� 2Xd����2�#%�M���?)����FY�&��v�o���o��������#Sd� ���CS�0��h�hX�n��=��e���Z����L�T����c��M�-V���������k^Bj�k�LEH�����64��NF����
C@�@��{zM��&��S��E��j�|��]�����yf�G}t��m��hj����\�b�1%�		���1�[0�0�S>�vC�O7�;0�3��������s��3��g�k��3���5V.�e��������t�{V��t���������=
,�wf?}��#��.J�J���Zd|C����IK|Bb���8}�����)�����56�������#�j������`?�U�)�h,��j+�������Z���Y�f��{(��II������Ha
Cj��Ed}$e�`��h��S�N)�����o�T��u_S��H#�j�M����_�z�5�%��zWy���_���V����z�K�u�t�E���B�.{;�_@7����a���9��+��fn�~2)�c�����t��b����V������qZV�	Sz�N�n1-Ux�r�Z�]s ��G������|����ib���i��X�4��e����v�`��?��#��8���u�p��uG�	��[n��A���H����^���]����������W]SKn�)�8�t�������a��7��N�~t�����X�������a�������U3�.�	:�����e�v��u�f�;���z��"g$�_;������m�^�V���N���i��`��O./=�s��������?=�(�E2YW{����j���V(�U
h�AZ�`�O��'D��5����Z�AB-�t��{JH�KB��g��be.`\*~]e.]@Qd�V���!>w���\f���&�d�r�aF������w
|��W�l
��^��FfiR2�\M����`db�v#%@��j��;������m�����.X\���%q`����O���\�{�1^�)�w{��
�$hY
����L�cw�#g�����/2J���z2��[X����k������%���OZ1�&,��V�^�keE��k��u�Z�U�@��_��`d
��5Dg(������_�����,nG��^��d:�������P��(�'L��z.+(�t�Z?�����j����F�)Z��3���'�#tQ�5zI�9XK�+0��D-�.�������Q���l�����?0[����@�������P��R�tQ�cSN�Q������v���51���G����.���o����������0���iw!m�:��E�2��-������D����W��=kif������6u�DZ��v�iwI�~}g��������3^�G<c��7F�	h4h�z����_��~����b��.�l���^��8]V��.O�93�YkK�G������*vOV�@}��5p�3hG�::-3%-/�'P*��5,d&[G�����Z_��W���"�Z��/���
Z;<x���8=np�_�X�i3h]�tY<N-@���4�2ls+�_�
�`�0	y��8�4�hN3�
M0b^�z[����	�O��/���v���1bZ@�	)kN���;����2J��]k��j�@�u�������:��,C��NH+^�_������Z%�H�Mj�'�V��0!6J���T@|����G���Q��t�\������������,��Y�(q��+0��:A\��B,,���j�:��wZ���1,.P-��t.��2<e�l���#�J^0�G��A_`[����.H���4�q]�>�%s��Lj�����d�R�`�fK�M3*�b��o`���?AP��b����&�^ �U���N�dc�����������ow�i�]���H�q6a�����-��=����F�t���%i��r�U�����=�����}V`��-kQ��B7i!~6��;��!Qw��0.�,�:��~���;�	7���.5}�����k�(X��u�0!��XC���NQ��lZG�*d�����Q�2���S�.[�������7��3f�=���?_H_}�����d����(6p��B���0fm u�c(�<�_��"��u�{���Zm�������(�_Oo^���S"��@\��E	K��������[����e
�qYH.����������h���9��I�9	|8pc�f��d�vxE#�U��N��l�f�����R�a������9s_y�G9/b�}�}>!Q��;��A�k��V�8�L�:���>pq�����w���G��>���L�EhO:���b���
�T��S��@��8���x�������5����Q��r�z���%�!���%���
�Uf�fBW��w�~�l�����F���5:��;��CC�Ou5�BOrK -H��[����d���6H(�E��2�'D���Cb��`�e�vK�-Xg!H��[�xk]:��U^�W�mKl7Jn��7�G�����$+uEug>�
h��M���#����<j�	-��I`�Bs����=��"yf�W���UQ�{c�'�����;>������M3��y���79.�su����O��t���s�a�������m���1:����aS{_�?��]9�����%N���y���2R!���B��2Mc�$L���T1��oa���8�UR^��B��v�4���<�.���MaD�b�X���~�J>�R���YQ��3�>f��4��Gq6\W��C<bvbk[c
�q�������F�-��c-������K�D�Nk��(�	[R��m�(�yXo����24�����Q���(��Y����h�9N�����D�#:����k�����������/��P�Mqq�����FG>^lM��y�2��93p����s�g���L���Q�^�A��������^���y��Tz��V:w�������r,?�M:*����{^q���]y����"E��P��Z.C�n�m��
��{-!%rKV3b���8B
�/a*�,���]��D��&B��W�Q���9K���T[��Q�V����i�����s���q=������r�J2�.,��6����&��xz��;��76����KR!�s#�/�i�ZcT������x�Z���;�����(� ������u*�:.`D��@s\K��O��=\� �1��,��[��������3�C�6���"�"���Y�e0a}���b��w��}���
�*�_�Y��6�p����P@V2���qGT�:��
-N);�{TZ���r��������^�:VFV�0vs��M��!�88���aWd��Q6�cwD���S��OSQ\E��O�/C�	&�b���a����y(�\�0)�T�X�s=h�mv�.|-�"�_)�v*�����F��Emm�S��a��c�.�=��u�e|� w���C�Z��#y�����c�v;��RK-vj�T[�y�l�~!��kTJ�����}q��	^����e���>p������t$m�{�%S7�lZW��>(!v����=�m��"f,��`�GRx����B���;1�������c��oA��MU��6K�z�jX�{A���H��1��,����q1&��T����#�3h8�V��d����Kp����s`n�|V>d����o�Sx���,c�+;�3�1U���F�*�]�p��(RE����=f?�%�Q���Y�P�	������p9�q ��*�WmS=wB0HKhQ#�����GNs^��Y�{�K��d��
89������5�(%K#)P�K+r'R��e�X�6�Ujd�d�O�:�IP�����4�
��Dz���o{K�����{�����c�9�>}HT�/�������v7f�u����p�^]O�,���nP�0���cIZ^������8���&wa�S�U����P`k�)U�.��L0[)�{��N=u�ok���0���(�@�J+f(��K��l�:U��]k{=A�������|�_�U�����m�,���5��r�����8�I}���'ON+S�	qO�:c�?�bw��c��dh��Q��=����Z����i[��?y�-�qh�����_=��Z��U����8*�U)P���jZ��S�N���+2=c���4L0V�x����;q�k4��S��pM��M%kO�yc���������tK]��Mz�S������d��H��i-y:�����+z���F�?�wVP�h�x��,���3V�n��ax������?��?K%���@tV�f�#�{V��J�e����S$���
Vi�]�~:=���3���"����K5�$��O��c$���>Z�UZ
I��k=:�W���n����2��	������_�
|4��d)��M���B
��|
b��V��C{�v�0�������%�Kl�E�.8a�=*�����ImR<YH�(�;�/	e����WT��DyTN��N�"��C�����b����L�rH����xD<�)�" i�8\zDQ?e�
\�f��7���x����'�Q�n�:w�S�A�/RH���p5h������7��l��?����i�L'�5����F?�x]��iG��|L��b��M�;j�QwF2!I�����;;c�[��u�\%������wo�� h|r&%���/^��S����S��E��"���E�>��G�N����}���}�/�N��7!����0��7����~;�������9�5f6��_�@����m����@�.5����Lp���@�V�w�����X���L��!��~,o9"Sg�������e2�j4&�2�����9�$���@������9�q@a�u�pxn�i��;<Z������T������q((�q�u �
g�3�"Ie�%H)�����G��T+XN2V\��X=	+�M��������haI��0Rc��O���7P��_F((�d���h> ��n�����o�f^����L������A�q��p�f�A�?�*(A�
U�s<��eNxOU�Kn��V��i��8 F����� �#XL�h���&�x� ��[o��_D�{��I��Ln�WJ�W�LAI��k)�6����_a]��j.��s���������3�����-�F]�K����|9@�UZ"P��^sb�?%
37�,��\�]TYf0�#��]�k �!}I#�@6��|{{X�����58��E?6��O���|_�����%�-��r|v-9��w��*���s^{{���l~g+�����9s���[a��M���U�.�
w%q�������q�{�X~�5#����M��Ov�����U�q�=�v���+v�!v��$�fO�� ���f1��S/=������
��HsKf89
����,�_2��+�D�j`�����a|H�V�$��\�x��`V�OJC���_�S9�ry*�R�T�����o�����t�pX��7��R����T��oz&rT�0�yTY9����pzX=X9���1���_NB@\����}��)��2�`�<v�BG�������+s�o|���<��2���C��JK�:�Km��'|���`�#������;��3'��o����s ab��B���7x�\L�n
l�x8�H,pF_���>!)F$�k5���5�x�$��@�C������[w�5Hd�H�i���"����RE�����^@Y`@
��'e�*�]�]���_d3�zy�S��]��(�/���r�B����_��]���
6D�������n�� ��u&�h�o^�D��{�*g��3(�*�_0�����*�����^j�+��G������Q���%o��K^~~~�^��v��^9�w���-8�)88�_9�_�
�����aA�.9r��q��-[����1�P����0��N�O
x��������F�F��)�/��dxf	����Cb��eL�M��1��%of>������e���sc��eY�[)	9�����N�381�?��`�Y��(� )Kl���;]��;/��;���2�>���������keTJ��[K�$�R�E�[���*k������� c�k�c������+jE����V`ix�-��y����N~	/h��}J9 5e�A�1�uv�E���#<��4u�@CD�����u�wZ�:�6�V���Oy�
2[�����@m�Ej�V&�#���������IrQ�K���� ��N���0'�a1L����b���
��J�]����rV��\5�N�P`�Z�!*�S�U*~��R�@)���&-C�Ni0�mB�Ka^���Y`+���^��~��r�LnJ���W�)��P�DA(��xv6�0,�����?~5m���Qeou��u��E�O].7:d.2����L� ���,+!	�y����Q�u���|��v.d^�B����d� ���!�T>�W��NE�v7;At^������7���]$�^�
�E���f����,��}9��O
=W�W��]�E���F'|
"���S��������J���W�Pj�GC�r���������y'~�Q��8�2��d�v�=uuu�����T
��;G�\�6�q��&Wx���/��=��O?����S�_�r��?6����p4M���
X�p�0~��4�O?�4���]wi�I��AC��`q��g����'�-82x�`��|�f_�o���L�D��t��������;GYg�������%.2�?[m��`H����7�$4����A�������I��\�����9���g�8vfL}UeJ�|�A������$�i�"?}g����n]U����]{�?�u�x�6*"�l.���jN��%��%��u�K���H4��h�I�|#73&^�������Yxs��x9�St����x�����W�`�����U���9�o�����T@�N���o�Q�0nBM�O�)��L����~��?Wj��n�5{�>�s����Wj���UN9'^F$%�z����"��������W��(Xm�����e��PX3��j��������_��7���C��%� ��2�����n����nT^`�_���8�B�)������E� `��V�:@�|������������Uc��u���x��|i��':O��cr��sdn�d��M�6����[�O^�����wSv�N�~�7�����w�Q�OKc�k@�;��}��+�����I�~�W���HW;�)���-���=(vr���H��}�����nk�������hjj�+����r��1O>�$��M7�t��W������_NHH��L�/f����P����g��f~��Epq�O?�����i���[����G�\F��g������O?���y\rr�2p���C�F��f��&L@��J��.�`����ap���*vf�}y�����4t�0�����V��}�v{}��H�������o��coR��Ch�T��z���x!V���}��le��m������A�o3����F
a�m��+����"s�	|'|�zoi���SZX\+��5���3�6���5���p���[���������v[��H�>������~x�zV	j�]���,���f�����dM�T�E��gbB��iU)}Z-G�s�CD�
���+>�
�V�~�G
Ee`��
�~�����j?����n�������(�\������^��o���'�f,AK`w*�@�������S���c�u��dC�����
K\U��2���KJ��~��	�IZf1�*�A��:���<s�����PW�����I�]�#���6q�4e+}��M8K���Z�}��L����������{������w������A����9s�P��W_�I�#Z������$iH�5��[�������K��4��0��{�$�9
Z�LzOf���������i>�5��}��-�A�o��]�G��_ys���='>��zb�Z���n*��r���������H��t]9�M�������'F�c�"W��
�To�5����y�|���{D�h,B<~����������{%�=+qz������SGV�����2��f�I@�z�q��EX����b��Eua��_,`^��a�������v
P�/�
�k����-�BK�C��#��p�C����u���&��
����`X��_�5�V�����e�K�:n����V��E�����ZK
N��:F�7��%`���yC����A�����y��u��]V��$������k����z�� �T��)�@af�~�����Je�|1�������M��6Avp�((�(I���K�2�b�G�*����A���O����0��b��<6|y����������P��_\�r%��WZzR'i�/��"w��A_h�zu;�3#^�����'��������c�]�s�����m��:�m+n�<��a��d��o��l��	��OS'��L?�W���z�#���������!F�m����l��]e�Gd�m��.!*st�;9U�����,�\�I��S������>��������������B���K�����|?����|��O�Ow�w��Q�0Q)��y�n�?�)9S��x�y��b�%%��bU���a�gO��E	��H�G�m����+�����Bg~����U]���+�iO���4��K�/P��"W�T��PW�g�$w[H���@0i���S��{z��?�����������c4���b#���y*)�@V�����[�g�
@�X
-��w���"��?������C�<v������&F��5Eb&�r#X�~�%�����&���/K�k�6��=3k"u�&�u�����'�����NO�9��t5��JfD���[s��m�r�7#v�!�S��F}���������G�7_]ZT���CmA���������I�V�'�5I�:9��]mq�WX����6�w�9s��m1�0i��#X����[���7��9+��[�'����?K�s�K�9����V��]�3'Dm�f�����T���A��<:J����H���h�N7����S�fP}��dJ�Y��k!$��SY�),�����bWQ����SBW{������{��-�@=����Pc�,��Shr}1� �����/�;��=����6���;�K�.��?���M�6��E
�c�_�U�����2@���'m.X��+a����$����'X�@�z;?Y�<��3�X��(CS�u���r1D�B�f��t��I�m]���U.L;w�:6����x_�-e���-op��������O����
�0�����R��2��������������4J�)6���H;��{�_1j:%F/��9���&
N0�����{e���\��Ybq��� �����������z`'�}"E��|�.wE���:M���}\�7�k�-
��6>�{���:�����A��Q�{����3E��"��s|�:%I������IO�P��0N#$���a��R��K��u�������g^����
6�k�Ou��$�c�1 8�(���G1�����!/��y��}��w�g�l�{�Bc�+�!��X;�`�qA��sO�r3��8E�]�����
���S��=������q�C��J ��@5{��(��V���������|o�2�����NN���W�^����}S��
6�����������/����V��	���gG��4�b����K�5����_G�������Q�|;�{�h������1���-�l�mKa��NQ��i3������$GDh�����/�bT�:��:��l�+��W���\%�F�(�fno��K3�� ����w�19y�=)g�����%�<U�|��y����Tz��*�@%,���tj��L��X����8UZ�:5E�xF_�������G{����
x�NU��qWT	r��b7��!4�W]kA���7�/9x�mUn�k�W��T��y�oW�U�a����D
���y����`���9�PX
�h����H����7,�P:�E��-[�����D���`0.���4���q�"�)"��Von���|�y������3vk��E*�"e�J�0������S�;w���B��1���U��Y���I��9�
a&�������qc��}�J���[Q���|��^���%���@��g�;U?��2S�~�g?<�M,���(c}�2���\����@�N�������P���+�������q�:v�9����f��w�Li�)-��Jp�Ok�����'�L���vpY�|���=fn��9Q�'�W>�(_G
{�6��f���`��������&e���J��}>j����F���U������U��)I��duZ�.9I��	�1�R���TEL������	�e��.*��yK���%��j\����%|�	���.}��nB��
��A�n��������/5�}�{y��G]�
����������R�	
4��H�o����={������5G<������:��-v�T��kj
<4�(wVpm�_x�Z��5 �K oA�f��43�QY�?���<����Q1[�cj��o�2s���������u�m���-+|���d%`,��M8��
ibU=�������8�<�o��m��b]�S�$�����a���Z���QM0�Q*q?��������}�R�lU��-�;F�L�;3T�:]�(�_2�������r��<�*����R����.O-p���?"����������.��q�9u����W|U��z�G
�s��X-� ����/��"����A���G�f���WZ)X����|c8���"��7:�oOt�7!���vb�2-E���W.<�d\1Z�Cn�UT��5������
Ee���JQc����h*h��B�Hp�����!����P�r�s{s��G[Y��i��[90��zhF��/\����Q��\Z��kE�	�"��2�t��p���]����#�
A	�r�S���G;��}��5e@��������{�������X������lB�����g��[S'\����v�5��p���d����yD��yS&E���6O�*k��5����j?�4�	s�b�$��o�t��7n��?�������W�����p��TI&��h}�X��8�H�f�R>�n������VVo��������$&��������q��0��y��|�j�����Ng���qq�#����JC��l�%��aT��,����d�����������z����8�����H;5	�6hUF��{2S��\��b
%wZa��E����P(�����=��+��x9c���
������ T&� k��M�}aCEC����@3L�4�'Ny,��07�p�%�x�
l�� s���[A\X�w�}70�D6hG�FclGuJ��z�\w]��D|�x�i��t��K��.���9����#����	�>�R��������dG-�����;5"urt���G�r{��(Zb6����[�J^?���P�V�oA������,�������|�D��H�&#F?8�8���g2�O0��~~���-������uU����j"�x�����K��<3e���Sfef���E%����U�>mz��z���%p�/������bD�3;pD(�\���D)����S��� 4"n�$&�����MxR��6������j��."�aF�������2os��[<�u��������A����	`3[�u�8��Ol��{��?&�V�K�.xq5F�U�V8��/}�{�=�w��}0���Y��x������U���D]�QwD����E���8���#3m\�s�eXl��f�[��rd��Dv�%��L;+:���c����+Xl&|���/��@���g���4���f\�����I�g�U�v�&����J��r�\�k�(�s���s��}�W���2���+����
�a+�l+����qV���B.�&������8O�a�e����S�������Z���u��he��TO*�iNT�����������'s"�#1��BF���>T~Sm���/-u�x��QE[m�����c#a�Z^r+ �!���������b��!��#:F���� v1p��
�Ce�@�z?�O`��4�J���U�V��)�����I;<W�����|������eF�����vE��"}���_o/u��8c�:�k�;��1)kw��_.+ZRc�&��	���u�Q���]���*��Q�5)�S��]bt���������0M����t"B������:��������q��'j�4f���I5<v��V�����v�K��J��jx��>Eq8O'xl0�v�4 ��>,-H�r�0(QG0C��ss'�'6b]:k�
6�cU2�BHSf��l>���kVe��`3`|��1=����<�TQp��/�����Ay�3x�~�'Y�0���_}���}����%����m�T8'������y������&����������9���T�t���v���~��o}QQ�h��>U=��N1�
�*�#]Q=u�O�Z��5g���H%�g�#��a-)��R)��Gj��O�3��)+��>�3����8;�9.��Vm�6����{������dV��K-J~}VN@��M|7Cb]R?ML����1}�Q������5�gQ�����m��M��=��Wm�1x4�������|��=���/�#�(qt��$���{���$eB�'=E���6E*Ob���(%��x9�y��s��q�=SZ)5�_�Q�5�D����(<�������c��]��q%j�Se�������g����u����e�������V,�7��8��/gujx���y�6�<��W�:�R'���v>_Z���vo���^��O��(�a��CR�r#��Z���|
jSM����v��u��M����1r��o����ov�� ��w��\E.O���z=��O��U��U>�.������bD7"
����$�)S���S��Jm�!eHT�3U�i�����w�����	��6�����8�S��hp�|�1��o��Q��:�����`#8S���{����4��d���6��=�!��.�s��c��#(��GD�Ob��
�e�|��.�h���?�������p����-q5�[��������7�.�<9��W�f?q@�����)GG��Thd��>��~����1��mE�����+��>t��H]�(]�x��q�����i��HMRg�UW��������������x����5S������|,���[d�������Z���S�6��!Y`���N�Y�E�Z+��U�����������{M��d�\!�����O�'Q<�
_}Y\a���J���
H���`\�p�EdAhlD�/p�)\�4�H�d\�t�f��������h���N����j�Ct��i��W�<j�i�g����ko<��>E������Q�[\����Ea'<
Iu��vi�++��mo��8����&}�h]�����Q�L�u������m��o��m������5;��zk�>a������>����!!�E��dj��v�'��'����s��#�����������o/�Q�6��^'�����O�|[2�������T��S���"(���
���s�2��tM�3Es���5�*26Gv���]����_w�O�9~���W�����,�~���F���{�|�Ken���Z�~J8�!v�Sd���b5���>8��q|��n�k�E������v���TX�Wg�s]��]{��.���U[����J$"=�:�66K�H�%��K��:��[/x����8��� 1�l/�a/���Vx����d���~��hOh��]|y�,����������c���uY����]�Y��yj�2��Y��s=��W$��@r�V����?P�ll�IgD���k��'��-j�>�������^���z�������HMbV��t}cCR��Fx��\��x:��������F�
I����V�e��T�t
m�6��>��.��&��6�{d���~S��G_���V�
E��b� �����M�dHR#ZI�f���7�x#I��LO���(xi+�����IwE+������-q�<�J����
�����7~Fm�����7�mXs�+E�+�z����v��3����6��He��A��SGN>�G.Wi��FMg����t}�<�w.�0n���.,�[
��$�qxL��Q�����W��j�R��N�%��'���w��2#���t��u�%[�u%x$��L�.gM�:~����c��a�H��n�a��V����(5`�rdff����0�f�#�j�^}��n�t��������������{d*��e���kw���%v���W^������)��

��*B������To�o�[P��0/�0��4��Q���]��d�����
�c.m�x��8�P<�2b�����*<IDATH�L�yL�4�eu�c<�q�V����x1Y
��cT��*}�*"Q��[��O��Gc�d�tZL���*��!s������m��[c���D]sh�Y�����'�#g!t���m�����	�a����\y�������8v�X��'���n���=�(�y��1X�c����o2t15
F�+V�8��lUo�d������}O�<z�]��x����ry������_���*o�)�������07%qLd��5[/��f8��)�N�aK��nu��)���A�yL�LR+�s,���fe�]K�c�Gfs!��9.���r9r�����0���eq�1�n��P�U	jc�6�+��m�^��!��&�~��� \�]�U��:���%7q9���\df�C��xO��d�L����B�~LW �Rv2������w�������mk4��9$ ���]������G���o���|v�5d��������CE�)��
K���
�A8��,AtA(FF����@1���H�D"�����4���EdF�YFg�s����B�u���.���0��322`>)�����?�i�
M�\��vX�~�L�B�������xt��I�SW�o��Q=��e��e�)�]�`�$�;�tI���w_��v""����R#�4�[e�T�qv���
�)�<���SP�
S��b#4�1��5��	����5J�r7#�l�Sm����W<���bBl��Y{�6�a�����y�:2E��1u�Cr?�P����M
�%����Xx�J���X���������m��������o����u\����}C�~�}C,���7�������52a����Tr��v�������.��#z1�:���
�\�	��-�&b���3)���0���+@��_��>�,����{���~?�0���/��[���+��2u�TBl�g���?pK�
`���Y�X.�w�}�=�?Ae"%��1�1�4{��w���b�DM�����~�i�����c1�Y ��E`F!)4
�������� ��@9�m���@������	��$�����n-~"0rS��������%s4�����H����,������s�
��Nwe%�����q.�N��Q�q"��J�8=�4E����	��wVZ�,�0�vW��'D���cg8<�4u�����d����U[>z��?*M�	�d�X��K6���6A����u���:W(A����*W�K��H��5��/������A?~�M��]g����X����n��	��0�@8�!L&^|���_��j@��&���|�d�b�*q��g����~��7@&M]s�5�� ��[o�E8���l��+�pip�����o����aUI�&�bT�����i��y��A����
�����` i���W]u|3	�p��Y�;c?~<�������D��={6 �1��`|p��/7
�4}�c�����G�M`��w||Z�s�����
��g���
�w;=�F�b�S���k�����=���P�Z��3��o-tmy��|���[�E�@XQ��*���h��u�O�(]<���=WF�yY���dw[���p>�KN���5�������������u��������K�������K��u�'��#:���>Y�KV��6��j��b����]�^;�v���q�	��iC+W'�y����>D�y������@p�*{��x����=2w��P�\R��Q���R�Q���WL����!�}��G��^x\A_���D��}�vK�S�V#�})�LSs�4n'�;��C}XO�g������k�!��Zc��4?�y�f��:#�5���Q�n�J�������X7������Q*����`f$��E���[l$��M�� ���Og�g�y�8��6�����H�if��%�����R�}��d�+_/��N�T%�/x8�����}Ss���k����51�~�&��S��n�C����v�hSJJ�K��W��Xd���
�6B�I$��bq;6��1���'3�{�����n9�m����r�nq�=��#���n��/Y�KV�1��N�|����:��8e�)���d�W`��%�(���������<�.���
�!�m�,_��������*�z�fi�2���RY������"�+Q���	��fggO�4	��
E�\Z(\��
n.�w����QK�@^v���0�������ud���i�=���z�����1�o����)�kW�d�18{�u��S�N
b|�
�	I�)~�������/[�,$��F���r�����o��N��VVc�������qS��[�6���5����EUD���?S�GF�Q�m^���~�@;RA�@G������]|�d��jV)#��.qW�O�-������qW/4??0�*'���S�!Rh��;���	�,s�+�$Q�yy}nrF�m
�e)�c9p�_rABzat�#fHlW�dS�����7�]2�jk�V���V��Y��i."3V�[�M�en�
V�4G�n�W-���~�O������@z���<��p��y��4o�<g.��RL�@5��S��q���me'�a �.RU*�j���}��G(�
�r�)��?���������8V��Fa�F��h]|���(�����
@�����^*�l��������OL�+�.p.Vn�=�t�BSL�)����nH�{����AI���u��E���]?=��������;���4F1��%�w=_B>A�% m$
4G���cr�Em�9���M����d�zU�N�bP��5�Ze�V�S�>S���4����J`�=>��D�r�+��B��p[��0{*�(����py����>~�/�O.�O��N�?�������9&f�9��T���l�k�	�%��k�	��������Gl%��m�)��j|����@#��?�p����r,X���3Y1BS4�X0������n�JCh�����"��7l���U����,bG+��-0x���EN7�e�fbC����=�H	c��x���N�����I:`�i��.|���a�?��s���K�,��&��/���Xf�E&�`	D������b�����wqQ���������5��$���2��&uv�����`5�"��)�o���R����}��^<�V�9=����.�w��%�v4���)���6E\s|��^��`�$J|�dTm=����������������������Q)j�q����T�9
�T`[�N��5����1�FT��
<f�%��M�n����J�]lw���o��S��T9�57�����x�e�~�O&�r�

~�&�1!��`U|�1C������������2����Z���t���a�X����x�ye�?�������Y,!#&Q``��G�������>r���������������fO���p�'jX��������7lQm����SSc����!�c�`O�z���.���;sC_�L�����.��V��o{��j�`/ m$
l��&N$0 ���=��Z������+������V*"��8�
lN�k���t�ZW���UUJV/r�W��x�>.U��W���b��x.��+�^��cv��i3L���,h��vR���o�~���SS���qo����r�Ve�W�5�����dJ��=U!W2<��	`�����i~��?���\�x���?�����6
v������)����'!#E!�Q�"����#������Es���<�����2���n?���A�|���������q��?�n�W���Q��RU���d=��T�gx��6|���5� ��9�LzU"��A�:U���iRH��VD�0�^������D�q�k��Q9[�E�,d�(vxj<�:�S�����Y��6:���g��8r��(��LP��M2��H��.�8����R��s�fA�@@ ����b��l�v��'�l�m��vp���c/_]�W���������U�@��4������S������;$�%�K���{���S���2W���=�����D��) ��^��`U��5*Bl���
��p��Y�N2�:�PY���H��@��z��fO�l�T��l.q�R����!x�������3�4�x)��0�a�����'
N#�h"*e�������KR1��u����s��L�U�1����<W��
<J�jy���>�&)5����r>�D%m'�mto��II�O
 a��J�4��\��E&5�Z�E�,X���{�K�1B��tz�\��S�P`3���������ow��S�g�K����t��cG9����R�f�g 9����'��4��:\���|��|5D����0�5��X��|�,S9��S�b<�/Q���s@tl�gm��:���h����=��������}����������=f%�+|[.)YV��Coj��Y��	��x���N�����shz������E&��;����Z�������_��S��{�(�#��	�0��d����0����J���o��@�S�
��rL���O����f��`v��);\�6w����K�b�O�{���]���7j^�mt������F"�����>�?�">AM�s����`k�mbH���`,�p��^�����i�G��?��Vho����U����W���T��NKNz������o]j����^�)1A�g�6V�z�>f����Y����C����������nk��p�q�
�����]w��276�����-;x`@K�"�8<.��mX��U�d���!sc��V8�
�ZALZ�V%7���jy�J�����l���e�I�[���E��uU�8���)+��*��2��\�4c�mw������x�+��t�'F�wy�����D����xipw����(�<T�:��yh�mP:�����`�N���rC"^	7���������lD��e&gU�Vf+Y���:g��Y�W_���������bIb�a��v��{�����O�J��h�������!�f@��vm�x��|��!���.W�(�\����:�B��3&��Jn�i���$�*I�=�O%�>kT]��!��C.d��W������!0G�
'GM2�c�����{���0e1�c�'�1��V�!5�a7����H
�>r�����.iv�^�N������Z�����?�pjY���[���J;����dC�v�k�����D�����R:4P�z|��!����H�\���`���+�VF'�v��|�h�&
��F
M:H��4����
��������%�
&}P+8ld���	j��`9��`��o���E���]e/�=!j�S��m��-�(����?����O���������4�:A����( Q�cP��!N����|�l��i�=a�"B�$+s���_��w��|���}EJ#^�c#<����YGR�$�u���;����n��Z�6���C=�>�C���sG����_�Q����n7$X�����������u$��=$
H8f
�(�|�0�x@�!���}�t2�es�������tb�S�d`�/������29-�=��i�k���I�<uo����b��W����m*_]�����}B�Oi�( Q@�@�Q����O��JO[P�d�4M�U�7�T��=���5u���{�k�~���(y��"���{�{��;���y��g���R������6%
H�( Q�((&v�G1�zU1R�2��8m��^Y���g���3#����r��
_�������zq���B	}�P�)Q@��D�������s��xC7u�e�&�1�����x�10\�����o��]�����W�d�o����wl�za��/�f��3{��L�������>V��J����&Q@��D����
&eop������_���2���s�%&&>��3�
"w�����z��}��2���w��Wze��W\���7%�Yn�����<.�
v�*��m������Rj���.�%
H�(P�B.�����OV����T�#aM|$��	�@�xJ�hTL(f�� 6���z��&����������K��D�-P�&?�����6�#�"�<n����}�e�q`�iDt�bf�?����&�+cfT��@GT8>�F�7�8eR�����j�m��7��.���qK�>�g�c����J��6���w��0��Ie�$
tX
�}�����^�{d��*�Lm�}dS��@.9�;9"������\��{���O<�q�F2
�q��������tI�Y�~��}�Y������^������m�n��F�u���o���m��6|�p\�I-���"
��^h���^"	q=?���:k��Qw�u����s������9s2�/����z�������������?��~���{��,�9
r.q:��G3���{�8���U2�
Iw�SEnw������V[�T����^�:u~ztoC�{�;�J����m$u!Q@�@���W���������=L+���Tz"��(��:����I�nb������g'�5r��e��Z�'O�7o�����_~y��M���f���Y���������jkk_y��]�v���;��g�}6t�P$����7�\u�U=z�x����
G�k��a�)
����O��k��d�������J���������'N$���s�97�|���~��������h��X������r�-�|��Wf�9���(�c=�����]�����oC���3���{����
����[��v{��6%
H�(��(�#�����f%D��j9��vO�I=��O��>U��
������3//6����a[9p�>}:!��L���X�������/_�Q���N;
�EnL#��;v�X�h�~��g�n������a=��3���f��I�7|j=��}���3����<;;�B�n���^�z�����j4r����#�o}��\�����D�bq�����+�����Y#�T+=3�Y������'xNy8����(,���/m}m��%
H�(�Pz�5��$7N0M��l�}
���e��~�8XL��*j������%xR�U���6�c={�����7��^�{l���(-�������?�����(�i� ^�n-gee��cOA��[UU�d�:�z��a��t�:tA�\��^#(�/���s�=��/��,��z��g{���}nT��
.��QkG��r����R�}�J��u���	}��v���( Q������(�(��?^E�E��*�������=�"��$��GQ����F�dp])��N�:]|����?,,{.��"*��>��8������F�e�M*`�L!@���WO�4I�Z���G�
��1�q�fp�����m�}��������O�<G�7Xjv2��>�S,��}������>�)�~�~�?X!SD(#�O�g�WaH����W.+?E�vZL�����mEa���M/���D���������=��Q��;���x�(�aL�(c�o�����og?*X����=E��Ql����;�����q���be��T�DK.|�&?������o��F�Rk5�����0��q��`|�����n���/�Q�al��x���Rx_��8J�X�!���e��={XI����^��,^h�.�7�F��a���JG�����j��~�a��^��x�w^����<h�?+}�)����v�X��[
�����F��"������5f����b�g�5���sy.:f:B�5�9��u�Z9Zx,��md��}/�H(D�$�bl�y���GVkb��� �����^��D�G����+|0����[�bh�~9F�B`���08�R�����G�`��f)��>����fY�N��"`��(�f�@�����9�)�\����Lk|� ����Z�����4���'E_0��>]��Nvj��}���OVf���/�y_�� m$
H�(p(`�X�O@N���]�]
��x�'^�x�zPh3P3�g���8/��!��8/!7@��_��R���(�6�[��Q�g�;)��l��6�nO����u���"���n%�K�W����Z���di�JA�.m$
H�(p(ha��#���FO0�����1TC�������N{`�O���J��D�1Ef����a����������T�D�:q�a���c{`9>h�:%AK|
��S���qQ����]��"�#Q@��D����	�]UF�[&}?�;!!z��5�I�������D��)p��[��He�m����6o��c+��?�������e�9������.�D�$
4M^���3�L���E:�
�	�����/�7�x���[K��l�F^�������5���=��J��QI$
H�(�^(�b6�t��L�������8C��R����Z�G�?�0T���j6����M��aB����d
�0q�p�m:�&�P�
���tJh)z&�'�����"?F=F4���g���+���p���$��a��'��j��Mh%��O�����3~��6T#d�\kf�`��P�n��6���+��m� �E�W��e[4+�r
��D�0\���,�CZ(�<�2����B4�iJr�s���Y���y�!/eg��
kb!����)f���h��U��K����m>��5#a��J�-[��h�m�rk�+�� B��.O����c�x��3���x����48����c
��������	�L���L9�2����|�����7y.������W	�{�����!`d Th�Id�������l-�L���$x\4�����'����%0�&�UV�-<��~��Nf���:��Y��P�+���\nnN�"�5h_7jkF�d��/�:��f�\k���!�5
����Mk.�tn8R�lS/���#�<�^'T�z����jPj') .8x7��a�AJC
-������m�Rk��m���)
���72��?����)D���	���@wE��D��!kXFF�4
!��B>l�����3�f(�8%��Q��d�w�^�2�+bR�|�CI���R���mb�iB��\r�%�_~�����h����O���%� �'r^�����]w�u�9���@��%MIu��@���dvH8���k�8q"W�)	eI�N*��~��^�����40�'3x��'0CC�]�h�$�<Y/6����F�����>�����.#��:�8/�v��e\k���U�g���?�0�H����/���Hi���@��I�WR����g<�h�#��u�{���>���g�}vzz:���N�,�;:�W�\��C��$�srr����]�=0���`s!�4��L��[A��������F�)
��=����'�xQG��M�y��
�������=����F7,���j������D��P�p�k�����!�d�,�!
���Fpw&8��Q�r����VX��	�K�9f
�����������*����;��s�s��������Ap�����{/O����K��$�4�c�N��'OFH�U�Va�^ZZz��w#�|��7���f���O��WK�+x��XR
]	|^��kv$�4��������{�a��UF��2x���YYY�B�v(�H��(^@�y�i��^��FR
�aqJ9$�k�~�����C�gY7�?�z�:t��������G?���3f4[�cV�����]�;F��&M������M�:URw�C��D�'3�v�^X^�T�~��]�v��	C?���m�(?QI���2;�O��W_�@�2�as9�Hz�L��2�S�����$
�h��!�F��IIPs��Ad�F2��+kQ�,B)��7a?����W]u��#F�Gq�A1y�)��#!�>.v	(G����`�5�L�6m��yt4p��5k�����&%Q@�@�)���Z?I��Pf'�=����������������qw�����^�?�����;�`4�>�`nn.�A7n�(����f'1an��6")�3"$�@��m���|H*K�( Q �SC*��bq0�7�3g����b��bZ{�5���������0���[o���f�6l�n��1	�9s&���T?~<��&�2fey�Q�,Q@��D�$�B*����E�,N������%����I�v����=��&������#��L0�����"J��?��0�L?�����Cs����!�[��D��( p=�H?OZ
dgg>��-��`j��-a�A\�����-[��{�����g��
`0Z���5�E� ��`����7�x��?���4G��D��) aSC*���������k������dR)�f6l�+��9@�l�$o<��y3���qi_)�Fj�F�bx����L�P�z��N�:RA��D�$
t8
��`$�pf�X5���R�G��������^�2
0S�����������b����?+b��$���	(����D�$
H�( Q@��D�$
H�( Q@��D�$
H�( Q@��D�$
H�( Q@��D�$
H�( Q@��D�$
H�( Q@��D�$
H�( Q�y
�?�)Fh�AIEND�B`�
#9Amit Kapila
amit.kapila16@gmail.com
In reply to: Alik Khilazhev (#8)
1 attachment(s)
Re: [WIP] Zipfian distribution in pgbench

On Mon, Jul 10, 2017 at 12:19 PM, Alik Khilazhev <a.khilazhev@postgrespro.ru

wrote:

Hello, Fabien!

Your description is not very precise. What version of Postgres is used? If
there is a decline, compared to which version? Is there a link to these
results?

Benchmark have been done in master v10. I am attaching image with results:
.

It will be interesting to see what the profiling data with perf says about
this for PostgreSQL. Can you try to get the perf report?

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Attachments:

ycsb_workload_a.pngimage/png; name=ycsb_workload_a.pngDownload
�PNG


IHDR��z��3sRGB���@IDATx��`T������^�@��#X��E}������������>���.���E�.���!=�d�������$��n�
�k\���9s����s���L:$	H�$ I@��$I�$	H�$ I@��$I�$	H�$ I@��$I�$	H�$ I@��$I�$	H�$ I@��$I�$	H�$ I@��$I�$	H�$���M^m��qqq��u��|A7�T*���x����HK���Pz��&B/��Vi�����*
��|��d'zP����m�{:����v��w?ty(��r�9��*K�G���hB�&������������1c�������/h
�S��299�h4F>�%%%���s�����jll,b���>71�V&L��c��K�i���oJJJqqq;�b]�tIKK�A
�N;T���w:�V���
�	�^�u���&�)h:��w�i�����pLU�HF�{��
�y	!�����5���k���i'�������E�T���y�"�O�
������y�G�KU��dee���TVV�*�e���AW?�*v0�n�B��`��
&�#��J�D��F>��M���/��H#�U��N�������oM[��g�`&EQ�������QC�F�a��N��G{&�����t������]���w_z:���8�L	H�T{���h��.&M�4r�������?��i��q;_|������q��v�\�>�h$��pB��{��j);���X@RB��$I�$	4 ��h�'�x�<���h�`0\t�EO>��_|1}��=z�=������};�b1������N!�3	�N3��g\����K�z��F<��X��x�n+2��	c�odr��S0����,OigyD���Y�*�P!�V!V�/�H"<0���=��)����8�����/���4n����������������O���=��0��J���i�������*�r�K.W�|^29S�>����r�<��6Y�2�3�r�L
��p��g���Mf5+�J�E+��d.��k��2���i�<c{��!��N�~���<����S0�i��o����U�V����`������v�I(Mp��t�-��>��G(������l`�����|KyP
d���c�n����h��s���a���0 M���#9<v�!�*|(�
*s{@e<�����#��-�E+P��`B��0�v_L��p�k����DeT����c��Dg�5�re�B����1���t�A��IT�zMe�����Vht;��|���5*�eo��U�uV��&2�����@����@Y��9�.�_���� ���-���H��������<�v)�<#�UY^(��vIH��'���s�D��+�������$�3hV��<�?m�[�Bx���Xr���',n'��'N�Xjj*P�3,��GVoa�Y��{����g�yeZ��G����N����+��O�q���)rc7�����U��U=�V���_pN��y���=����C�Igi��Rh���]��e����CsT�b�M�5��U2y��V��8m�.k��e���nW�OV�;�I���p�������f��k�i$\�D��8���N�	�T$�h<,
<��6m
�Dle��#b9C��Ov���A�z�fvvv��4r��������7�����o������'���l��}��[o:t�{��G���P����,Et7O��>��i7�-�-�[�UQ�)7V���0.�Y+gD�e�����e�+q>g�0�%=�c���/'A�92��C�u�h������)��m(u��Q��4gTo�:��2�d>���q�N��Q��m�N�AW�$m��b����|�����4�I�Rm�b���qD>�>��|VEsn��)>c�����~����i�!��"�&z��.�
�	�����s^	�PX�p~fV�u��-����?��V�X�f��vPV�����d1��{/4�5�P��U;V�����\���$K���C�����i��K��0�T5����
����?����Uo���Ee��6.��dM��*�TGE��h��r��r�������RQ]�F��24Q�5�'*�6���&M��}g�hU�]��}G��e��<�n�m�������ZK�$	H�$ics&*$���������?��S@G��������i�A�����M7������LH(�������KV��.�L?X?��L}�z�SEy�W������U������3{�b7�O���BG�Ng���ms*�!u���q��N��>.��)2�������*�-��q��F,!�&I�O�jS��,M����Q
�pU�"��D�J�u�lE.�~����z��z��z���x=���QG�,�#I@��$�N*��phD`?� �����z������)U���������q����D
}"C���xA�<���;W�V:f����Y==1G����P-��=n�9oyu���Uo��������~Bt����W�����;l�+����s7����p��;�I��x�����FYtw
��C�_�|�A��jp���"[:�\ 9�a�szl�Y%���_��$	HhTa�F[���l6�<�����M�f�0Z�5*f�I1CNWF)�<YX�[���vV�/������O��R����Y����������"�a:P��G�6F��;F��O0t�o��u��|�9���?�����{�������/_ce��B-G-����zi��5Q�4�LM����S�^�����*=�}N��e��Z���;��������~%	H�$ I��I���
f�]1/�1�����
G
���;��t&e7=RX�����[�&��[�m3~��|j�N���z?�Z^�P�N�f�v������ur�J�ul�$C�Q�c�K�O[&��r����i�?�"�Q�����k+uWn��K��QJu�B�E�G�~l��f"9il�B#�X��j���]��Y�����V(xx	���C��$I�:��	�F�X4	-N����.g�
z(H��xa�2��I*7���{��W&u81b7|������l���[���.V��'a��)'�&xayY~<�������qA�&��(cV�|�_EV�4�*u�
�8��6��V��Jc�rf,��J����(r�sU;��Z;J\���4�|X�RJ��$I�)�c��1]4�!W��@�mZ���2��M�Y�F_����kJ�3~�c�)����&�(!7\2��]m���Z��D�dG>�������~S�����>� �'a:Y���d�����B�-�A�M�O�,rl_]Tw���c���'���27�}2�u�hm/r�hm/s������,I�$	H�(	����+e2S!;bg]�0��{�c����VT�`��"�������^�������fO�v�DW�ik���k?�z��������q��q}�(��Qk��f��d����\����Ud Y�2��������>:C7u�(}���^�����]��5�U��n���1Y��M���0$]�$ I@��$�0H�������BO�������n9���@�i����+J^�_}�/{?;��[�e;�9?��Zt��g3V���'��y����NqqT��E�]��]��!U�1�@��?���o��<���,�bY������������
�g�X�l/w�
�%8Z�%W�8��E����%�&(��
^s*fJ��$	H�$v	����0����A�������s�E_Q�����W���������}���_������0% ��t��9��v��e��Y����}Z��YC��(���P��3W;��!�V�]�L�2��6�b�U�C�H��Zq
W�^�L��Y���b\R���u�;�'��6cj�_�����N"(I@��$IG?�����:��IR��s������B�U�������������~?P�JZL�*6e9�&c��Ic�0tU�����o�Nn�UA��+���0
\��&��R��.I������:��Oh������C�vO�����.��j����s?6�"�m>����
qB�NqC)����X���C��$I�Z+���e2�\F,��2�
D�t����|��_����m�{��x������������g���|��n6�.���h�����1U�V�����/p��3���6{,�e��Y���R��Q=�Q=4�=c�iT�j�,8Nn�����P��:���1C<n�������q������>��N�^Vn�n�z�~�&���;"���C��$I���~V���[�����EY�rsXn6�}N��7�[��@���9��,��,��2n��:oE��%���5�x�JW��u�1�f��"�8Zk�������Z��]�q��VN��8N����-���U:�o*�X����cV�p�P���d�N-S���� db�&����
f�KP�����b�HP����v[}��F��=.���	Eb��c���u�FK����o��[R����K�$���`��Am�G�Z�S�I��\�m���J����=j}��B�����z|zF��_r�`���u{��l��n���k��Ot����>S��?d�_.V��sHE��
�7?^�:��,r��*�����]�q!���S��U`D�����%��J��V��7J6�p�V�\e.@��k�Jj��HK. �Z���s�ga;i~=���.X�2?*�^3hP�T��V���Svr; 
T�
�{�E��KEV����7/�z%����u�i�`�cED�H�SVd2DVC��x;u��`�>��tl_(�Mu@�]����i	2m�V��cr�eZ����b�Jhc_���/-yaB�'�g_���ue�����'�w�GD<Ea����@����w�����et���T���0�Z�[)5X1���<j� ��)~f�i'.��:P�I
�hB~6dA0S,�?`(y�aV������?����w��Z�"3�`��f�
�V��V�m�a�V��s�\��3���Bx��>���b�d��J3:�����>���G_�������l�.��Z@n��-h��� 7\Byq�/ib�1����?5?u�^���kj1�s�Z->�x�C����HmR��/�������4�������w�i�uV���h8�<*��@��OQ����V�]"<<�����FB�#�W(&&�
��]H���ji��:mZw�R���6g�6VV�g�����U�O�I�������o.o�y��@�	��s��?�8�,��{���D��3M���\!�8���EEE)����Y
�?,o|�����[�%���D����~�b�?��Tm�����2���q�o�(Z����F����������drU�W
k�pC;���.�0A�VO��(��b	`���:.�����Z����^~����L��^�Ku��-�����.��S�cU�2�
S��)g�`�b�����&�u���)��!��J��*
E�Z��oV�p�F������g����!������X$#|��L&)��kNc_���� #:::��MS>Z�v$;��r��p�`�X����dC�L���W����l�P��b�`,�Y-/���������YZ��*(@eee<�
8"������'��<������E���x
��#��?�`0������P�cI|l������ ����Vf
(�*��
[�z�c��0�-��%[�0(T��Li�y���(�20�VZ������&_�o�FLr{�������.�d�'�����X?��*��d�
�$�a\ �'wX-LC����6v(	�|���x*���5{�k��-����S��{�	�\'���]�}�]X<SP�D��kT��P%���������������f�K9�4�	�>����P�j4[��k��c,�#Q��E��R0K&S�J����Z(�5^������T�T<=��'g�������ubk�|������tG�����>�u�1��V��c�V4�Bx��6+��44��G����w%k�����z1�� ��a�����OY�����@��{�1o��la�rp�4.1���Z�J��|��L <Q�b�I�5.[�������Egi��!�<��6v��V=ZC�~K����U���v)�)~�~�A^�j�C�B0�L@�?  �����Zm�9C��?��u��	�	�����(.(�Bsh��`�S��I�vi�$qCU1������BY.����_!����>I��'���B�zU���"�G�c��?"8j�	����}�Ca5��
�uT�v0��luq]��:�����i������3����������9��:�J�|�l���M;����o6tS����e/z�]v���;K�C��:~�Aj�} �R<:�x�a��\\���;p��d��(�P���9y��m�����YO�<3u��'��Q���`����z1U���g����sa4��9tq���Sq��B=��q2
@��q���J�����0�^S������XY�0�����/" C�����B&������1�PSE�G�|����r���I�XF��o��-O�\Q_i��`���y}Z������
B5.":WA;p��* �0�q��������A�MN3��8t`L��	d�1����_M�%a8�_y/��M?�kb��iV��	��-���8�g�\��cz�T��V�>[%k6T(����	�n=	�,�����T��J���z�)�������q�O�~�������Hy������y���<�_�=�����%oeX���L���b�P������+���������f�de;��yf����
�Z��������
�����Dj5�5����;����A��G�������0/\��0��������k�[8XR0�����r*��	U����S&�1����ZBTl5�P���a��5
BC~R�/��2�W��ea�#�B��	JS�?���J�� ��������
^����������\��C���.����I��/d�">�'Uc���.�-D�a�?�.�) ���`N����i?�����������V
���E��~C�p��D��q�_"HC�4u��R<�%�&$:��.ku`��t�RG?���>���.u��,J���=��@���1�1}^n��?*n���U�a���m���������������,y����<�6���������a�������1�������I ��V���g�F����������[FC�q8�p�P�z9��:yb�0��
�?������OZz��e��*@��
���:��y�WG%�JDh��� .,D�g� ��0��R�!�������XA�$�-�6��
B�W��O����^��a0����&��y�=�O�h��s�������?@�/�.	�ei@(�(GhL`�_E�j����`��
���B~M��E&k
p�&G������%$���Q��
*}�V^U�29����e`�#��5J�Mc�������������*?��{�{����x�3��5�`�#����C"�f	��a&r~���j�X���g�K�M��jY��U>�O�]!�����F��@�ab�
�Z�5����O���{��0��9������Iv�����4��@���N�	A�TQ�j��+��6�*~�>|��dq��fX�8@�10�@Ua$!�#,���``��x����7M?-V�B�����#0�IG�H�(`����N����_�Bp4{/l��K��u����d�+���:�l��
�p�����I�pW���{_�c���KV����lc����K�4M@i�h�U�U�V�!RB^�"uN�Q7_XG�2����Pj��u��PS�^���G�32�vg����[Rt�[��VO�muz��J��J�Qn+�����VRl�����wq�N)�1<�-���-�2�dOCe[��W����.SC��{f���r���TV��$���W[�r�X����i�5��,\���Ef��}l��4	�j�$p��L��i��*";}�nV1���/:���1<E���_Uar�����e	w^P�M��nH?���q7������//}�c����z5$:��1	0J�OM�a��v��n�����f���J)�*�W�u7m/^L�����\6-[���>/�����4.���?���i�����������='���DE��$�cS���}?�5`UL&C���^���m�tx^^_:�W\�^�l�=?�����6w��]du�Z��6w��U�������&��%A����V�%���RO������.�h�<6|T�C��$I�"WG9G�u���������d{��
����
�N�d�Z����>����,��z��c�T.���
����-����
��m~�,"<�	9l.( N��e8HgL�#��I��o�����'�#I@��$I�����	]��E�h.k�(M�~o������4�/�+�#���S�F���t�:��I�+��D�zX�a�N��(	 ���z=������f'
6�
���Vg��Ui���*���6���������pk��7d]��i�EK�w[J�`L�$I@��$���@�~�z���h����:k�����a�LFf��)�$A�����U�:A@������J�W�5����Y�E��ym���
�����F��-����
$�_��T�*�^��R+��������0�u�8��g�vX���W��y0<�������>�lvZ_/��y�����|���$��Gt�������5�g:#%$	H�$�&M�t��W�w�yo���g�����_|�������!����
7��&t����7%%��Geg+�,���;����.]����'�|�`���n_Kk{��2�v������a��c{!2�p�)��?�������<������h�"J����b��T�2#J���p:^�L�����b��G��c��+���u������~v^��:gq�C�o�
�]X�*�8Ae&����j�����g������O�Z1!�m�d����q�k����$ I@�@�`��n����Nb#;v]c���k����'��={6�Po���#G�{��lq��+���7���N���i���_����������fee>��C`PRR���HLL|����c�l6�]�������6�
|���[��7���[n����+{Z�����&0n��A�Q&==����7��bt���tp��(jV��PZ�����`��\�e"�H��7��mqy�m���A����dqZ�q�Z�U$i�&�7$&�T:����fZ�O������?OV4j�W�L�n�����{�`��c������|��;������|��[���}��j��4��j��N����v�v>y �B���(��t�s�9�|���]��E<o��\�-[����W����W�X�i�& f���o��8�>���x����gCh��'�|r��������^8w��Gy\�����?��#(�Ib�lD%��4�j��3�8��������{�n����?���q��}���
v�|���"`@�� p���H�
>�!I�-/��8�l�|;���~y5�Bg��S&��h[��Y��
�F�8�6��J����k����~���������X��hoE�^��W�G�2�y�(u��V��9>a�E�B#����.�������JP�}���������w���������yP(�QYM�P�T��"O��w�w�?��~�r	!>�/U�R}�+"k���p�	�������A�.��������.�9k��!C��`t����_��[�n���/P���/��|������P�>������:55�b(xD���1c��ik���`v��txf��2�������t��z�� nNN��3g�gdq�M7!r��kW�&nX����c�����@1�Ga���M���vi�&����2���p� �vl��-����6[��W�����e�g��+(�j,.��,WY"+�U�b��8�"Y��(�<E�������=�\�{u�n����v��m�>4��;.��5|9�r{�qaIL5�������O�",K�O�0!��j�`��|�0��1"�w�nfgg7��LP(��<��=�-f>��zv�fT�d���Lh�������|��T�s��1	��5�v�����6�
��V�-[6x�`t���#'6�LH�C� ��@�0�3<A��Q`-���P\\�t��� G�������C��|{�|^}�"qL�N]R������ ��5@��Yt0!!�������Ra�f��[U��N��3���]?�>�0����y����w������?�k�uC�mj��s���%w,+�}���8�cx$�0o#���b	t!b�w4����_1��������u��'�k�_�v�#B@�i��k�1
_T�%&M�� ~�y���"����^^[3���2My������e��|���eZ�}��wV�^�1�yL�<������kDe�I.UUU��8��W_}��c.E�CW�����A��cX�/8&6���m���
�r�}tAdTPP�W	��g��m����c�r���x�����?�<;x@��{�X��l��%���{����8�A������R(�dd��Y�`7��=^�{�KO����I�t�9�8�/��w6��q����g||<�����<����7��F���S�	�1���'��� 3P��������J��|As�}�io�:��M�	
�;~�x�����A����@&P�
��a��\�'_�t���F+W]u�8��>}:DH����1��`a��+����00o�s�=��3�i���q\��f01,���O?�t���c����O�)��I���1
��tTk���a��o����	V9�x4ko���Q�o���ii�nL�pg�+�)[�n��g\U�G00I�>����-�N���=��s+O���wz���-oo.�%�
?�6c���w�h�\\�O�o���/����)D
���)X�I��P�Hc���G�:i�8~����0i����on�[�1$�r����@��h�������s���o��&@$�a`��������~�a����#������������Q����P�[Mm��V��y���p�s��E�S�L�8+�����?l��c���j�w�~����)�+'��'��3{m/����������������Dw�K�$	t���D1k�Q�D�7�nP/��|�)�Nkg6�F�[�d	9����
wT~x���+vCL����a@���C":��NFh�
{�a6����o��q���w��S��6m�]�.���
?�4��;���& �_E��J-�m(�|P��}�_�������0~����J~I�$I��@rr2X��{dp��v>&�z�h���f���hzh�`vmj�df6������Ct��3��t �-oZ��V8�.	�������-y�����SK�p}l��y��U�`��P��<"����Me���������>�WLzt|���$�����e;;h��&x�.I�$	A�~�c<�4���G"z���O�f��d��
����F[�f
a����[q�3�*��ct�R"�^U����=��|��]&?�{�����y�b�n�u`��Y;��'V�������o���2o����[�KWv��[��KW%	H�$��h����b�r�N�kZ[�,���|j���\&YFk������,��U������L��s��G$k���2��j�<�Q�N���~gK�g;�g���yx��a)3'}��`�����uJK'�$	H�$��@3���s�NJ�3�G����x�6^�*/��62�����Kw�|�����q����C�&W�JW���u����L��������b��r^��i}��z}S��{+�@�����K�$ I��@3�"""H���}����v��gbkD��t�08F�sOe8�F�|���u�w|��31?�N�rY��K���`/��`�#3�X������W���Q����'g���g~u}�9&v}8���#I@��$�cS�0���x/3�M�)a�;w
I)u�Jm�A]l/n���)��nL*Tzo�Fw�Auf�`���{�������?����@nf�7,E���K$}6���"���J������������$Im(����8���8y�&S�!��/�zY����_Y,s<�8�Z��;���}k��\��W����|�W�
�-~E���u���]?`�`�n����6��#?�����D����z��l��c3��:ng���/��$ I��$���-�D3��h@�Lvv
ZD�|OQ|���
(���(���=���n�=+*�R�\��(tJ&�V�]�P�s�|~�����t��7����>m`��sz_�ct�	��'�kr��4�g��Yp�[���m�%�_��m@�������$	tz	�x���z�^y�,�#��a�^N� jfvv6�'I��Ylv�J'��<y2Q8�A��~������+�4��E]2�C�K2333!2j��C����2
q�����&A�e"Z���!�`%�e3&h6q�:�,q?�K.���k�]�pa
k��!���'����R���&9���DrV���|G������8rz|����}V��Y[q�6��v�gM��.��0kX��c3��<{�g����c^����jH�$PW���'�{��n]"3���V)�v��*�q�Z���C�\�s
�x��W/v`/B��C%4������K�+6`2� f�,���b����>� � 2���D�&F�%�F�-��%�0������C |����7����m��v�����D����{�'�4�/�����6���	����0q9Z�HGT�M�f���m�+k������h�V�)��+J���q���cf�>�l��%�_uT�����Yb���;��f��4<����WJ�nO��J�TKK�Z%L��$�N'�[�cU�QU����>�����-O��.5�l�:���u�f��;�Y�	��7��O>����+������_��$��'��%���*��4Ba�h}���\=��S�/_����w�����W/�����;�M p7Y��]6�@fO��]���g�y1EKd*�Xp��0��w����w��4�i��=��S�����4�]5
��Q��|KGx$P��x������N?���Iw$���x�K�+�$�V�`C���2~�������H��_"�r*_�T��~�����!I@���(���3+��5�X���rZ�B�+W�JU���/�OV&���q $0�b�g,���|v�^�n�hfC�b��]�v��8�l!O|�w�}:{���r$�����X��^�w�����o��^���|&aol��w�gK�DL���{��37vp�Dz������`���7F���P�������!�-FDk��%�Q�"��Kl�������5�J���z��I��(Y�R��������?����5kh�E}N���`��J��]���Z��%J�VFD';����w
>����O��N���d��6_])�n������U�>�)f��o���K���;��v���@<K"
�9�����ok?����R�+C0f��S�Yt_r0�2=��n�\/��~����P��N��7n<�J�F���F��9s&�h���'���������t�q���(_kY

H��K
��Vj|�d���Wm���Sz�e\��{��Yd���T���g�;7�*K�w}QR�0y��Q#��q�k�#./_�����>�]��,E�v��l��;kE���\�
J��5�����
���\�[����U+�*E���"������6�Ax�l����i�Y����<�M���K�w^|^��)�b�[f���	�k�D�7�[+h�k�G�;"�ac
���{�9u�=��������X�'���E<�d�O��-�.�������~�aT�3�<��������H���6?��#p�=�E�$�	c�i4���<vquh��m���� ����>0-��
?���3`�����p3x�EJd2h�j�K@�����+�	����j�jM�B���d�1��5n�C�OD>��l�p&	�N��n�i9�>�����R��0v��I�����{���e��s��y=h�n�l�B�+w�A�������+��^�/��3��M��w�>�U��� 02�8P3�js|jtp"�p����#_�a�hA�:���*�.��"
Z�"�H�����os_l�u��9���{,���yRl�8*]y���L�+�x��l^���)[�%&t�W�=���L��P�r	}R�c�k�/W�N���
Q�I��"5~Q����9�J����D��7o�,b��+W�l�N����L0����I����e�n61cP��W��Z�j�?�~h�}��������v-l����^�ZVu��2&��8b���
�q��mja�G�����t����d]I���-�n+�]Y'�7�/|��R�s&s7�n�Gv6rrr��4X���'����+9�9m6v��a�AN���@������2�fr_x��{��K�y4hJ]�W���
x�l��
0�:v�X�fm^�uw��a�/� �1x3���'Y��X�Z�����"b
R�D%I��,��DD��u�C�'����b�^�5��+K���1���Q�%�{��wq�r����X�������76����uCR����i�m/����b� ���pU���d�����Y�Y���L��j�����5M���>c�~���2a|�]wu��dS��/7*3��X1�t�"��G�#���ir�	���<T�[�g���>�g��@����]R����Uo�����N���������O����Y������.��K�T���NmJU$	H8�%`�X0AGB'��~��9AF&�iiih�LAG�M���K)s�
�U���n�E�R@�vU��[��g����'����G�G�,Z�\��Z�&-�49=o/��.a��^:0���s�V|�����v��$I�:��`z�D ��3�8������"e\�#����.*��JV���t��d���sC��+��I����wf_�J����K����u�7����e[��VN�&lw��>	g�U.��zm��|7XK��$ I@�@�J�o3��/����{,77������W�X���*��%���&�� u����+��>�����{J�������^�{l�S�f���y�������6��N����^�.�Ol�x�2h�REI��A�Qq�
��B;��nM4����E��_����������"_��fj��^k�'�0a�v����������E����FL���_��Q��yK���{���z�_��w[����> ����o��v�_�4Z"��.��I%	�`yNE�]a��b���I4��|�~/m&�y���F�rrQ�}��d���D;[E��vl��7���
��'��<��C.(]�N���{�kT��o,�/�)~~m�U���{ar���������m������J�$	Hh#	���n0hFu�@����I3�p�%S,f���a�����m�G8i��M�Wm��0���kXhU�k�������>���	�q�^�r��w����/���ov>�W��K/������w�eD�'���l.� ���X�*��F��MBY�bz��m�]Pm����E�Uq0l!5qFa�:���4��JV��=��U&Sk�Ie�M>���{��� c����|G-w����9���(��X�����i=�o�r���N�zw��J�JSA�U�(I l r �E��=�b[�j�YqD4/L�l���e�Q���������_����
�~ll�N��+K+��\`(�l��^����S���@��n����)�]S����-_��u"^���J�����_��8�{,�^60���I���^�P� ��)���*iJ�%	HhK	4
���o?����O?��gv<���o�`nK~�@[���h����Rip���$D7i�����nL=����?6���p���v��~7A7ms{��f)�tr�h`����g��_����Z?���� �A�#U�$ I@���hfRzz:�h}��i�&I%�r��D�Z�p�����uge�e.�����7�/}+������K��s���h��]`�,�����������Qi����Zrq�����j�5K-�TJ��$�6�@3��7Nk���>�8XL�.v�h�bD�U�K��D`w6Y�nFP�Pvn�M���K��U�zXa�@IDAT��Bwu�*T��Km${���s�������_'>��u�{��}]����[Sl�z��Q�m������g����#�����K$	H�$��`6��5k���fo�7�|����>�%��x��w�{�=U�>��C}��+��r���y�m���������EB��*_�[��i��1Mk�����z�,K���q3��+s�������n���������Y���S=?���K�kL;/�N\� I@�@'�����Z���jc0�o�������k��%����?���g�y��_�����5���W_}5���Z��J�����oGo~���p=gW����o�����/)����z�$&u5��U�
W���H�z
�8���j���/:v8���sw�5q����fNy�����'���'��
�����l6�kr��,����[�kq�����������:(I�l�P
�j(�W�/��1q���������ku���h�j(��>.����+�����c�jc�����@�G}t����s��n�	K����K2{02���_��0����*+�0;<�T:t(�=��JLL������4���%W��H[�<���
ELr���v{�S����#������t8�5+�.��0q��|%���yy��k�����1��-{N�?_�������X���	9k��xC�y��}Y�E��W����x�
��?�Y�����V�G&�(��.?�:,p�l|����'|zD>#�U^|�N����O0-�������9"Y���<�\�=z����������+V[� �B�����+�4/�."��xJ	JA�
 ��?�X4�����=|�p���u�V*��.���w�i�OK�.��?����1b��|������T����l� ���lHy�C��K0���7`ee�I'�����e�qq}����Ys�F�����0���>m��|%x����	�-tsq�'>^�B.����q���r�-�}���n�����DOx�N}IH7�@"�87�PA�B���g+|��r���������P����;M�#�
e�<�]�V}^_������3�Y�g_���bgL�w��E*�e����{|r��
��q����^5<}�Y;�����yd�\�'�/Z��=�u���^�Z�M���oU�)�S����h����F���)��U�]��j�~��RG /x����f�<}�uV"`uITy�����^�W�_���W��+v)��j �G��EV���Q�V�^
�8���^x!Kx@��^z� �(p��7�et�|l�\ps�`s��r���|�I��Yy���_}�7��Dh���@,&^(���1E��������6��V�?������`��
F<x(e'�7
�p��,D��*�4	��D`������1�}�$�%%%A0�(���+@A�6����Y�}���@~�Rh����]���/!w��/��E�"�U�G��J��t�����\m�w_�|�}�k�����K��1�������?�d�Q+��in�|fv�X�e��Nh-�;���UJ���h�5�)�K���Kc�<����'�11|6V B�yP��������"EbX�� ~#D�
���bf|�,�l&����<�M���|R�sf[}C3Z�W.��d�����r�j�����!p�;A%�8�y4h�o��v�u�����r�����I�&��?��vf>k7�t�H�ew��U`V�[ 	�����_�%����}��������"w�����,/���K�O=�`�MgB6Tw���������|���v��s������N����M���]F�t��1�h��V����X��<����G���H
�Am
Zm�O��VV�k���f��:[�f�QC!��O��D"�%;-?�\����	w&�Gt�)���V/|�^"��B<�.��\��\�I���M��O�gu[�gf��w{+M��oi
!R�D������h�j�v�h��G4�EJ�D�F�TA�A
�U�B�������4�.em5��i����
�6���y�c	Vq
�0�����)�'�`F%t'O�����K�V�T���0���6M��"�R4�������(��Q������h����<�_~QZ�����b�����P2�Z�
������/����V�fP�R�2~d14�)S�P�p���)0����e������x����	P}�Z�B|B�N�U�KMF��!L�G�����U���bx�Z�6��K�V��q�}����t^�_o��������T�����k��\> ��sz�Sdy{S��;��lA��"\��u�����Slg�jX��f?J��Y���:�#�������rl�������9�Zn���H:�	p�Lz@h���s��X�Ak4F�|h� �	6Ap����-[�����M�h��D���i�7T�0�N@�^x��"�-�TfL�L1���Z�z��:R�}AL��csfh���9s�, �����s�4:>Fs�t�6Zc��D��p�bR�jE���^��(,\�It:J���������#feN�;i�������z�m-KKm7�q��u��L�vp��S��>*�@��n)�_��lX$ �$�F������YoQ�����_���8p���z`'	�E�2/^������JtHP��R(�����
H&|���]��C�I&6]&���C
�:M�,M���*�m�����B<u�T�[�_~�%�N����5�o�ECe0&0v��4�D/��~Ce[��uM?��&��(5�����b����a��%��V��F�y��y5m�i��[E�R999���A��m(X���T�mL��b�L���e�|��U���2�=��?�YQiu
K5L��4�ob�NYP�j�6���w�7�6�L�*]�[=��KPnrss��G�)�uPT�"��K�
_T�����Mp_���C����2>/�}[px�x�p
<�|�#1�#����T"M���z��w1 #v�)���c�����T

�����7Tl}�Wd��,/Z2J0�f&[!.�B�L�R�f�Q;�mh�%������N��kF�i�����0g�<&�:4"���I��eNGU��h��\����+K�z�b��R���2����?��{c��O�v�,��4q�^~������n(������wB�m#R?�n|{K��"�9�IJHh'	0J���h�)g�Vl��o�!�����'�x�6g�%9(���?��C�� �����a�@>��im�.�/�(����+uW�L4��&�	BgF���HZ����J�*dfw�G�0F��h/\������� ���S����U?Tl��h�3������
���M��l/��+~����G��������6�,�37e/I���6<�F��-��y����6G�
�7�LA���X@��K��f @�Im�WP���K#���F�4O�;u%g����n+[�A����F\�������=m+��~Y\��;�l3<�[,�_�7��~	����R�/�Li��p	Z�#I (	����m���AK�Q����
6��%��8��(k��`bk�v���7X�c3�*uT|���e��������hc���w�5�����1����W��/iz��'����t��=�����6���b�;�����]90)^��_E��HI�$��%�������PVb'Y��r^��~�����G�[�\��Gg�e��Ri�N�X��y��%��������e�|k��K����l�[�r�Y�,�X��zp�t�,����B���K?�a,	a�R�/��$ I �%����Vh����/�zYb��x�3Si��*TM�LYn/�tw�H�Q���������wO�����)���co����Wh�����%��80f���[���W�tj�U�xh\F���b�����$���6T�#�C�F������f4`&�Y��������k5x��6���F�gV�h�b*]�K��VX���nR����.K���+��<rf��;�/~#y�uE��B���]��������~�xi=>��-#Rgo3�{�D�x5�]�HIh�p&>Tc���%~���b��@3LO��Ad+bl���fg�z���������M�Tx�e��:[[�?'R��&���x�s[�L{C��7���+���K�7��%���W��[[�����z%�4"��������m~j�|^�:��6(Q�$�i$��hV�m=E�E{�X�����4��}/1��:�,Glb�G&��jL|�F%�Y��Vx������@����x�������:��Us�/�m���U�=�o-;�������6�P�������>13� I:UW�:v�K#^x�L�pAb���hKl��Zr�5����+��%K�;��O-�������
�LYU�O��v���`�H�V�)���w�=�r�O��/�?ke�9�j�z��q�������������F�i<��^�o����fzv|���(�$�����l{�0�BN�D�������U
�ga�.��]�Bo�����_�����y��=�L���Tk����7�ae���U�ia�����qD����I_��)�tB��.���������J���o~&(�\I%	H�p	4������]tv"S?����-F�
�H�6�6���U;|�F|!v�����-�vh4�&D�W����K����$�$}�]��>�<zf��������3��F.l2����W��U��-#d�B���,��s���[�XQdYQh�k���Cu�lu��.����j��Vd2DVC����	+7�������C� �=�L��pu��+3��o[p4�����
j�,%��O�vh���)�y���9��}�>]��!�8+����@/�zB��������S��w�[����!W�]���WR�\[��U�����j�JK�V��[��D�]q�=2�k����]���!���������=�G�L�pY]�*�{L������}��b�����b�^����M]�j��2����vf�����������f	�r���nmjm��U��g�)��}����<�m��QF�Q�f�S:�
3���E��@v�Q�B<K�����VF��Z�V���T�+�Lr������}@g#���`�F8��[����=>SN�����K}]�����}����x��M{��<.��7%�'�A�����s�**�����A��]�A�3V=8Q;*-����)�b��V�T
���������������\js��^j)"��4�z��|6p-����@/��(�Hb�^�C|EC����m��o={v-U�}�p5�
$<�F�-��x�
v{`/B�u6��������<|��jr���O5x	E
J�N�O�z��UN�S���j�Z�'��F����oQ'b5����<����ek�MO�������������X���#Rx�K�.e�Q=`���0�FJU
y���D����4���{EO����sC�uK�}}�{u��^c5P��^��"�&����6D�D����4�������"U�	�d.�
~�{d�<���z+���V�+E7;��4
��7o�4i\����p�F��"ex�A�q��S�\��b��,�\��n���G>d���D����"5{����]S���46uvA��ie�>*Z�,;
�Es�h����&?�
�h�Z�3N;(I7<��~<.#�����:���V96��P������r�� ����������U�
w
��L��j�������z��!�1W6;6,]p���[�y�v�e7�V���n����W�����o4�V�����w_��AM�C���[�������\�3���uS3N�/��&:���KW����oO���n*�����
���rP�~L�aD����=�����K����n�/+��+�l/��69���=�,�uH��H�Q�=.s���2m��}Wi/����������;�:`�<��o���z��=e�1���������wN<�D�����R���c:�b"+���
>�.//?��3{���i��E�Q����S���r����C2����h��B0�ZL,�ao���v��I'%uU*d6E��J��D�m����L;~���[��3&������G]U�����z�U��7�c��4�G�Qj���$���������id*��>�����Rn[RP�~����
����a�EI�%�u��Ord�!���5W`M�;a��'s�=1����:,W�*7~-�2d�Y�4�B�
�E1��A��/1b�9s��C&fN.\��������>�5��W^���u���g�������c2�{��g�y�|qq�s�=��/|������y3���gV�q�t����~���Zl��	�]�*����)��V2>F�����OA�$!����/�o�q�-z�r�W^�������s_���a���D���p�����?Y0V�6w~�zq~��2k���}����>�%��y�I���2<k
��Z)�:z�]�j��=|�r�����bO�5k��/��bu����q8G��$��L���F�q}������K�.?��h�&�����
l�u�]���^z)�O(���m#
*�r�)/��"���1P��L7`��s�MNNf���\@��P�d�`c�W�H�7G?�.Y�R������>���{O�������OU��5B��0������u�����K1��=8Y?%+��AI�+�]������m���jg��&v:^�����5���GJ��x��|e��N^�h�s�;n��^�����������=��;���K=�,�0�n +�`*�/��W\q����~��A�Z����������Y��/X�y���I��1�v��
G-l��
�"�g�������&H3(h�j$\r�	���.�I��H�G).sa������U8i���W����]���-y�P�������-��~uC�V)���������������?q��4���W��U�3��xw�������DA�@���]���.��Fo���Sa/RX�e�"�zk�j�������c�z#G�D�e��G���_~��/��Hc�f���^��322X�~���7��
Z����=����N:	S���K�y����z�y����
���zKmM��N�W���B�4����>��;�w��3�t���'��8���S��~D-{�0�������0����o���({�kY�42M?>3fd��oY1w�IX�]U�M���K��`������Y���X�`	���pr&t��Jm\������M�"P��peNw��X��+�a�X���+�Z�d������t��;z���s���W_}5�{X�.��8E�h�b�
h>��,�)������7��$S�\<JX��G��x��
gq0b��Hh��kq�.[5%���SO�:q�E���.^�_W���k�8H�-������fP�O���5*5jbV�9��/��Z)�Z���2��2�����*���X��*5vI@���5-�/�����~����/�6E����_(����\o�,����h������&O�6��(���US?j���6l ��������`������2G	�t���T��,�X7�A��!�$�z	T��o�Y7���IdN�+q�%%K_.]���V�zbS�����)a�~����*��&���)����(b`�YY�k���������gv�����cn�1�*V`0�+�(
,�,�y��7�XB���Z�x��C�
���Ro��6��r#1f1R�	Tg���s��>JX��R�����w&�mTg��hF�d�v��N�}#!+[�@�Z��n��RJ[Z��Ny�p��,�����h?���])�������$l�����E��K3��MP��c��$�9e,����?w��>�}�75xx���H����@&�
l�sh�5k>U�������Y������o�CF+%�@x6��t�������'�L��]�k��74����CV5} �z�;�Jo�c\��p!�[�t���G��	4�R��z*��������_���L8�85�,��=!�h�Z�WLK������������jO�?���Mp����|7��o��!3���`��K(�a��N�-f	O�U�S���r.�����������t8��
�q;����������v���@l�k��%�a��'�q��*;c������
tq���R+52���71�����Y��Zx�c�o>�e�M��J$\�O�cYM[��Y��.��1f\�
�U[����4����-�	&v��X��z�E�G�`�-8�i8
����~��uXx�e���X_���+>��%���<��n�;��I��{�'��� ������C�Yw��U+/��w���O���V����bd"���O�d�Ci�X��8��W���<}n%^��h������U�Ib0y ����x����@a�����q��q47n�M8J�~���/���I��w�}��������q����kf�Ze���'���E�@�k����U�|����S��
���^����f�&�����L�|��`LeG�%�����U���&7&]����0a@L��`�6���$DW��2�I�)�j ��M�q�\;�;N�U `��1������[H��R/���nC��$�&����~V�qs6Z��Bs>-�yP%�<
����g�	&l�4vj���Fm��v����j>%L��#��������-�E��=������V��jGM���5i��Mx�
�K�z��YM���y�E�vd���Cf��+?�����l,b���P2f�y<����Q�s����ytS�*�����-��Li���15�L�]
@�>��3h�X$����{��I�0�y[H@q�UW���^x���$�3��5�v�v���q;���E3Z���f�*����6yW�@����V�Cqdpc�]���J����&���d��^�c�;set�UjF����]i�lJ���r�_N����J��l�tR��J@H ����a$=Nn�_�*6�:*O��1���'<��p{:s@���*�d�xi�m�3��"u�,����Lj&����>��v�-�I%���|C%� HT� [��������nSp9���%�Sp�|n�X���@�����a� �����%����{��O�� `@�w�����M�L����I��P�]tQoo/��/��r����	��0�f\>��q;���t�V���j�o&�rD��	]�Kh&���Y�x��JUyp��Yh0
PUR
�����vS%���B#�W�������z����=�4i�Y�=f��@�D�W�r�/'����B]�j�K�d�DvOrm��Y�L2f����C/�$@��lG��������d)c�2n)�lI�*�e��z��c�$^anZ���t��;B���R{C�p2�z�����8������gI�Pt �0�����7ef�_F����H�OL���y���,�b~E�(]�t)3#�[s���fc��7t>Q��8�����{����KQ������`q>@�h2/aqV/W���f\�I�Y�U�z��k$��U�hjkk�\DD
=�������^1������Wq����X���-�Z��I!���A�X&U�L�:O��<��&��	��T�3����V#��N�tk�����o.�/�T!
;�2��s�SY_�$&��j��Y�����jL�j�@L{�'�ro�����T����F:���f��`1�Q���S}�c��:R!,����������1�����=(C��
����F�d�;|�]w�
>�c�E��{����[n9��s|��*d��1e�gE�����!E�{�{�Gp0���<��PO*`|NoM���K���V�j������I��a~�m)��4	�lqH����Y���Z�b�j���jw�z���K*U8�t*�E����}����#]�p��U�Cj�/��.�J������d2���/����K�=WmWZ���kj�Kj�g�T\�d�IK��xP�����W�PY���.^�4�:��g<�<;@�����erb$�|F�E��B�?���C�~��Q>c����F@N$�B���(�J�����L
�R��B� �*)t^|j����aS2������2[]��m�����]n��m��).��Uk�-����w:�������d8�M�;R��D�]
v���JE������L�r�c=+�Ef�4W�_�����e�Sa[[�D2^^_yj�m��[5��q17��^��@+k���tF��sV$X���qq!P�M���F�P]����c���o�O40���Ma�y�,�������
�,�0���`��;)M�tM����������Z�l��Bs����mU��
��k,��ZRN'C$6Vc~w&����3d�F��D ��6_�#+bg�� f>�{#F��P���.]sp��E+�����tb6���J�.��|�OI���=�4{�/�V �J�$��c�����d���/
F���$	U���P�����2]��L���P�>���o1KfY�W*��U#��Xc�U�+���f���4�	��0B�k�-�W�A�oW2�������N��,ag�7iR��������3��#�x���>��0 ���v��u����71��S�Tg$�k(A�W��#	��	-��MP��������%O���*<>�l���QT�?�"@�5:�n�3�,��]�XfV����F��U�h�W��x���o�j&�����H�XZ�&#�4���D`?&`�Pw�e�D-7{V����s�B{�7��e�\��6M�o��l��������N�5�
M���{e�4I�$/r�����I�y���@�|�������
e@��_�bd���M�������6h���t��W�i���P&V������f]h��o�8q2+���2k��TlP�{�'���H:�x�G[N���������J�� �1�d�3t�������Y���������\6�q��j�=�k����-}��1��&
e�A��>��(�M���'����m!el`Pw��@@ 0Q�����v���V�������x��&kE�����m���t6���\�77#!���T�;�Wj@�� �`=�NQ�n��tU�n eX~��V��cA� �� ��Dfv[�^��j�q��kk�+}�SZ\��V�2o�$v�a2�����U#SQ�K���Dv��'`�����Q�E��V  � z��#���T�%S�~���GR�f�Cq����0�hR�uO#�����5g=�%���22Sx-#(�#Qe�FzQk�v�#=,3��E��B�0�PeC��s��������;q���Te�	U��x�e������z�L&��ppzu F*F���$5��>�$F�q<aJ��0��z�OD'���P  8Vt�T�!jd(>�3w�N���b�m����[_c���}z�o��e�n�M������&5�a������Lh�,1���T�_7��F����9�����q����Ot��S�U�9��>��Z�2=X���������bz��M��������[�x\����V:%�[�v�`�!P�����nM�}I5NL<z�{����ID@'��q���7�N�"�6\�]����+���
Cg]�-�:���,.BsF5k(��x&N��RD!
X�(`]�+s2����1Y���d��
b�~����#��/{\7y��|N�/����'��0b&�Zo$�
�+�Q����8i'�nN����3���&`��]�rU�Z���;#6��@�h@\f�z�Gn��%�1Fg��B_`������d[��e�e�Yf���L���7m���-��������n�`���t��mb�.0�����b*�k����a@���V�uM�sm�kQ���V��KjdU�����k����O'���a^%�x|p���J����Z^��k��N�E���E�K��O���I�3��I
����Jv�(�\���fc�xd����K�l���R��t,��u��/#t6!?Yc��~�uWf�YW|s�5?���~�;LQV���s+�k�]k}��U��z	"'��a����+�4�$<��9dt�)A���U���&`�%�V��V*��T���FZ'#�d��av��[c.]�|uu�����Y\�
��X~��Fpen��.�0[�Y�V�M6bq*�����>$��	F�=�+s��b#oK���W������To��?�����P�*l��W����9U�Njt]��^���wF���C/T��	��c�0 z�]!�y�3��'`S:e�P-vS�[����M  �H����vo�Z=0����1��N�]���U�����fG�2s�	YWf=���4k(�c��4\�	0�3NS������[a���O3��sK�������E������B�}Y��x���N��[�l���|���;�>s(��/�J���,���+�G�Z�_�6���b�K�h�Kt�����������������#��;�N���C�ppz�-�Y����f�X`�!�������,2$7\��D$�+�HO*�C�/����,��lk��2��lH�c���Y�T�nYm����u����N��Z=��
��r�F1I������V<��[����T8�#.���x|���t0=���l�����SO����/�r&\�q^������.rD�(�,��2��s�u����*����������t\��-���`2:����{��X���l���l!���
:=��F�u,�u&��3��:M�S�N��9�u�
�15H�G���<i�@����k�%F�[�4i!�X���Y<^\m_Q�XY�DD^������u1[�������������`\M�q�����3������T��>��j �S�2,u���?<oRU3�Nx��[���z�drFb�j*%�5�ZMF	4��t�����S���
��R�U�1�.��"}��;`��������&��������M�-
R2������3�0c�����L�����/3�b2C:G9��b�6[�UH�:�Q+��%g�Lfw������=��1#	k�L*���gF�2��L�_��L�cq������H����d�wG���K{�;\aOTY����b���RY������/��USTM���I�I��{�	"W�H%zX�I�F@j������9�;0��D��CN'3�����x��6�VkUR����o$,zsCC�%e�����������H����jUU�F P�gL~�T��������l==�E9����Lx����NM�=Q�e�������l1\�����G��h&+������Y�.�e�U)�B�Md$�����1������R���H ���_�	\_c���>�����9���&��l�����s�?�R_����~��Kd���	$�<���
!��~(�M���5���?tL�u�p:	�����z�)�&�������@@ �#�i��N��<����tm��")6<�d�KvT���5f=�W���gs�9W���O����0���9�gTbD��oO;�#�}��}C�G�2�Y�t�KZVc;�N����~r�G��X\��v���7cd;�?���.���$���&`����Z0�\�}MTP P�d��Z����h�;l�k��&%g���fen!�#�������-w��,�����E1����l��4w:w��x����S��J`�;��Va���&�V�s8����+�b_x� n��A�X��{���<ZV�l��	�w�0X��E+��@:�OFFcc#�69I3���+pe&�W�����w�����6�����u"2�Vh`O���p���a�o�y`�=����S�ZX���RZ�&1���������`Rw���ZXVa�*m�d�Y�[��<�N=�`Q%��@`�#@�d��dK���b?r����,����+�w�����m�����&���a��U�����55�$��T����E�����\JX�D�;��?�=0�R_do ��g"���	$���#6����p����� P��*.MqX���Q�@@  �^0�fm������Q����n���f����Vo�Z�����;�T�Y���j1�D���5��:��o�p�6���^9����?�>4�Z��7"�SDO���1V`��H�j$�e2��'h���O�_%L��V����t��!�:&�{�R�"A�T����w��y��	zl�w�������J��������Z@�/��l��mM��@�����>%������$��kO�ax��>���Go���HP(�j�0��TqMK��������������Y���:/�Mj\�C{�������9
�l���B���Zv����=����Q��l���))�JH��T��>�������������!P������/����~���u�����;6\���&
�������klVfX��V��r�������g���5����d���n��v��<�R%`�����$`�&
�����\���6%����F@�M����� ��e�gM�s�����g��X������y��J�����l&���%%3�I:�&��>$���c���8��M/��XlS�@�0�{J����U�@�c�����-q��@@ P�`��{������������*?u��������:�+0�KH�S�e�}�rD@��W��(I���E���0)thP*E�����q/3�$!��^U4��#�y�����hO��0e��8��d"P&��^�>��###zj������??��%��-:��3����x���p0�.�v�:���w��qw�)�bF �T���������e��e�^{��-I����/�V��N��'>�v���������.;��3_��Q���l�!�.Y����l�E����@$`2�#�~���$��O�SEQp�����o�q���P�o�188����������{�?��4�l������s�r
�����wW����v*/����zUY���K��i	u�������RU���������Q)�x4S9j���
@�d`�e7c@t8�P�u�(������F><<O�|>NX�d���'����3g���8�F�4�%�/��lo��.�����*�����1�����,�������N�|c�ES5�k��z)���t���QO���OSw2��n8�w�f2�3��#)������8�;6p�����D��'j�a-���C\244���u�9��3*1�q;��h�,�������y:%��bj@ee%f�t�)k
���^��Hb�3S�L��� ���n��?�Y�X�� `t�����k�X�{C��g��K��;�����wwwo����+�hll\�n�j��ov��%��m����3��e���2@�x��5�7o6���������!������K;::m�����]���]�=�.��0�����$O:��^x��:����3���G#P�/�x��W�|�=���~}�{�9F�O�����,s��k,^����?�H�����y�Pb��F\����5c���b	���
0���-^��?���6440mbM��h���w����!���Qa���a��y�[/�:sM���I?���5��P��Q9f�O����R0�7�A�g,5D�*k�83\�
v<,,"e�a�Xq�����MA6T�<-C��3C���b��orc7O�����|�36l��?��O����b��kv�w0��By�]w�t�-�����;���0��Q����?�����7��A���G?�/��*���������{����/��g|�����wT��q�>�Nc����Y\�6��_��3{�lLh����R������o���C��Z������?���y�c���/���}i���������UW�a����	g#d��O�6�-F��	M��6�u�Y����y!�n��v�������=��~�,2��yn2*P�e~�[����>b�Y�=l��~���!���w�C��_���������SN9�K_�S������,�� �z�7��
�{p}����M���0���}��+h�����,�n��a�i"��`f6�����0;IO�k��T��.�
)����wC�����������+�l��	�e�`���Lh��QG����|`Y�G1��>J�����"�������<���x��QY����-�����0�s���=�K.��I!�6VI�&7����Q����H��f�=J?�F�v�m�R��������2k��k���c1IDAT��dX@#��
�V%���w�R%`&���e������q�p2�
7����W��z�|�M�����.�@�`G���w��q(l�P�c�z���/Q��0VjH����oy�v������CV��W���p�����r"����K
F)o�o@�i������X
��o��g���%#�f�L	�: �at��y�0�P��'��[o�)
��$+C���/�r�����|����o����3�P>�'���	��n��l�-���JV4�Fa��������O>�%������tZ�:��R>�C��!�LcV=00�����
����V�OV����L���sL����R��Q>�>e�N����u<M��0z'\�0dc��fu*>���0 B��������o����0G1&.� �,��
�_����K�\���U���w�M7�0&0� �Q����w;gF}�k+����{�����.�����6 �TpF��gc
I����5�\3���s������������3����e���|�+�OH�<o'./+P�V����Z5���.���36�"�m/����.�`��^�q7���'�l�P�J�>����
�������~:FC�5E>��������S����o�v�O3�������Y"9��
4��

���L����/7�Z�����LnN;�4C��0��h�'v;q�@@  �F%�#��<x���^l�Y
�h��_���p0�|~B��\��?���l��&�o��J�|+-���RUA%��Re�d�h�b$����a4u��(R/l
�3 ���>����W_�m����>��`������>�9�Z�������m��O?���4���?��bgD�����+sXE�3�R5����K�{:@������m�0�G����?�L�B�:w��U4�W�S���W'�|2���a����/#�b)�'���Rzt���/���Q�����@@ PN	����h��"���|�������������4T��4�����B����������B>��/~�1�H0��
;y����!�_���1	����lJ&*��>��x`�|<�-�S�� �)[�����
�f�>���'1��JX�1{�����3�
8�|�}5�lh�1o���F;M�l�sq��cc�	�S&F[B]��FT_ ����B#~�A�e2�+�X��z��	(���A��>��������B���.�3��i�vY����k��em8w�� ut�04����9p��@��p�=P��ID����-�?�^���qY�5ne�������c:vX�� >9��9�1��q���_��3N_jQ�$�U�3	rI�qG��Ov.nQ�Lq ���+�L�1��c�ORQ�
c��
G#L����a���
7k"�q9�?S�,DcS�!����s_���@@  ��@@  ��@@  ��@@  ��@@  ��@@  ��@@  f��{]�|i��IEND�B`�
#10Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Alik Khilazhev (#8)
Re: [WIP] Zipfian distribution in pgbench

Hello Alik,

Your description is not very precise. What version of Postgres is used?
If there is a decline, compared to which version? Is there a link to
these results?

Benchmark have been done in master v10. I am attaching image with results:
.

Ok, thanks.

More precision would be helpful, such as the exact pgbench option used (eg
how many client per thread in pgbench, how long does it run, prepared
transactions, ...).

Intuitively, contention should explain a saturation of the tps
performance, because more clients are not effective to improve tps as the
wait for other clients, and the latency would degrade.

But it is unclear to me why the tps would be reduced even with lock
contention, so something seems amiss.

Performance debugging by mail is an uneasy task.

Maybe you could try zipf with unlogged tables, to check whether skipping
the WAL write does something.

Also Amit advice about the perf report looks useful.

Given the explanations, the random draw mostly hits values at the
beginning of the interval, so when the number of client goes higher one
just get locking contention on the updated row?

Yes, exactly.

Ok. The uniform distribution run, if all other parameters are equal, gives
a hint about the potential performance when the performance bottleneck is
hit.

On Workload A with uniform distribution PostgreSQL shows better results
than MongoDB and MySQL(see attachment). Also you can notice that for
small number of clients type of distribution does not affect on tps on
MySQL.

Ok. I assume that you use pgbench for pg and other ad-hoc tools for the
other targets (mysql & mongodb).

--
Fabien.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#11Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Peter Geoghegan (#5)
1 attachment(s)
Re: [WIP] Zipfian distribution in pgbench

On 7 Jul 2017, at 21:53, Peter Geoghegan <pg@bowt.ie> wrote:

Is it possible for you to instrument the number of B-Tree page
accesses using custom instrumentation for pgbench_accounts_pkey?

If that seems like too much work, then it would still be interesting
to see what the B-Tree keyspace looks like before and after varying
the "nclient" count from, say, 32 to 128. Maybe there is a significant
difference in how balanced or skewed it is in each case. Or, the index
could simply be more bloated.

There is a query that I sometimes use, that itself uses pageinspect,
to summarize the keyspace quickly. It shows you the highkey for every
internal page, starting from the root and working down to the lowest
internal page level (the one just before the leaf level -- level 1),
in logical/keyspace order. You can use it to visualize the
distribution of values. It could easily include the leaf level, too,
but that's less interesting and tends to make the query take ages. I
wonder what the query will show here.

Here is the query:

I am attaching results of query that you sent. It shows that there is nothing have changed after executing tests.


Thanks and Regards,
Alik Khilazhev
Postgres Professional:
http://www.postgrespro.com <http://www.postgrespro.com/&gt;
The Russian Postgres Company

Attachments:

result.txttext/plain; name=result.txt; x-unix-mode=0644Download
#12Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Fabien COELHO (#10)
1 attachment(s)
Re: [WIP] Zipfian distribution in pgbench

Hello!

I want to say that our company is already engaged in the search for the causes of the problem and their solution. And also we have few experimental patches that increases performance for 1000 clients by several times.

In addition, I have fixed threadsafety issues and implemented per-thread cache for zeta values. See attached patch.

Attachments:

pgbench-zipf-02v.patchapplication/octet-stream; name=pgbench-zipf-02v.patch; x-unix-mode=0644Download
diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml
index 64b043b48a..1dea6e4b17 100644
--- a/doc/src/sgml/ref/pgbench.sgml
+++ b/doc/src/sgml/ref/pgbench.sgml
@@ -1014,6 +1014,14 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
        <entry>an integer between <literal>1</> and <literal>10</></>
       </row>
       <row>
+        <entry><literal><function>random_zipfian(<replaceable>lb</>, <replaceable>ub</>, <replaceable>parameter</>)</></></>
+        <entry>integer</>
+        <entry> Zipfian-distributed random integer in <literal>[lb, ub]</>,
+              see below</>
+        <entry><literal>random_zipfian(1, 10, 1.2)</></>
+        <entry>an integer between <literal>1</> and <literal>10</></>
+      </row>
+      <row>
        <entry><literal><function>sqrt(<replaceable>x</>)</></></>
        <entry>double</>
        <entry>square root</>
@@ -1094,6 +1102,24 @@ f(x) = PHI(2.0 * parameter * (x - mu) / (max - min + 1)) /
       of the Box-Muller transform.
      </para>
     </listitem>
+
+    <listitem>
+        <para>
+          For Zipfian distribution, <replaceable>parameter</> 
+          defines how skewed the distribution. A physical sense of this parameter
+          is following: <literal>1</> generated <literal>N</> times, 
+          <literal>2</> generated <literal>2 ^ parameter</> less, 
+          <literal>3</> generated <literal>3 ^ parameter</> less, ...
+          <literal> X </> generated <literal>X ^ parameter</> less times.
+        </para>
+        <para>
+          Intuitively, the larger the <replaceable>parameter</>, the more
+          frequently value values to the beginning of the interval are drawn. 
+          The closer to 0 <replaceable>parameter</> is, 
+          the flatter (more uniform) the access distribution.
+
+        </para>
+    </listitem>
    </itemizedlist>
 
   <para>
diff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.y
index b3a2d9bfd3..25d5ad48e5 100644
--- a/src/bin/pgbench/exprparse.y
+++ b/src/bin/pgbench/exprparse.y
@@ -191,6 +191,9 @@ static const struct
 	{
 		"random_exponential", 3, PGBENCH_RANDOM_EXPONENTIAL
 	},
+	{
+		"random_zipfian", 3, PGBENCH_RANDOM_ZIPFIAN
+	},
 	/* keep as last array element */
 	{
 		NULL, 0, 0
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 4d364a1db4..9f7cc5f018 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -93,7 +93,10 @@ static int	pthread_join(pthread_t th, void **thread_return);
 #define LOG_STEP_SECONDS	5	/* seconds between log messages */
 #define DEFAULT_NXACTS	10		/* default nxacts */
 
+#define ZIPF_CACHE_SIZE	15		/* cache cells number */
+
 #define MIN_GAUSSIAN_PARAM		2.0 /* minimum parameter for gauss */
+#define MIN_ZIPFIAN_PARAM		1.0000001 /* minimum parameter for zipfian */
 
 int			nxacts = 0;			/* number of transactions per client */
 int			duration = 0;		/* duration in seconds */
@@ -334,6 +337,28 @@ typedef struct
 } CState;
 
 /*
+ * Cache cell for zipfian_random call
+ */
+typedef struct
+{
+	double		zetan;		/* zeta(n) */
+	double		theta;		/* theta parameter of previous execution */
+	int64		nitems;		/* n(ub - lb + 1) parameter of previous execution */
+	double		alpha;
+	double		beta;
+	double		eta;
+} ZipfData;
+
+/*
+ * Zipf cache for zeta values
+ */
+typedef struct
+{
+	int			cells_inited;
+	ZipfData   *cells;
+} ZipfCache;
+
+/*
  * Thread state
  */
 typedef struct
@@ -345,6 +370,7 @@ typedef struct
 	unsigned short random_state[3]; /* separate randomness for each thread */
 	int64		throttle_trigger;	/* previous/next throttling (us) */
 	FILE	   *logfile;		/* where to log, or NULL */
+	ZipfCache  zipf_cache;
 
 	/* per thread collected stats */
 	instr_time	start_time;		/* thread start time */
@@ -737,6 +763,101 @@ getPoissonRand(TState *thread, int64 center)
 	return (int64) (-log(uniform) * ((double) center) + 0.5);
 }
 
+/* helper function for getZipfianRand */
+static double 
+zipfZetaRange(int64 from, int64 to, double theta)
+{
+	int			i;
+	double		ans = 0.0;
+
+	for (i = to; i > from; i--)
+		ans += pow(i, -theta);
+	return ans;
+}
+
+/* helper function for getZipfianRand. Computes random variable in range 1..n */
+static int64 
+zipfn(TState *thread, ZipfData *zipf, int64 n, double theta)
+{
+	double		u = pg_erand48(thread->random_state);
+	double		uz = u * zipf->zetan;
+
+	if (uz < 1.)
+		return 1;
+	if (uz < 1 + zipf->beta)
+		return 2;
+	return 1 + (int64)(n * pow(zipf->eta * u - zipf->eta + 1., zipf->alpha));
+}
+
+/* set zeta value and other parametrs to cache cell */
+static void
+zipfSetCacheCell(ZipfData *cell, ZipfData *nearest, int64 n, double theta)
+{
+	double		zipf_zeta2;
+	double		sub_zeta;
+
+	cell->nitems = n;
+	cell->theta = theta;
+
+	zipf_zeta2 = zipfZetaRange(1, 2, theta) + 1;
+
+	if (nearest != NULL && nearest->nitems - n < n)
+	{
+		sub_zeta = zipfZetaRange(Min(n, nearest->nitems), Max(n, nearest->nitems), theta);
+		cell->zetan = nearest->zetan + (n - nearest->nitems > 0 ? sub_zeta : -sub_zeta);
+	}
+	else
+		cell->zetan = zipfZetaRange(1, n, theta) + 1;
+
+	cell->alpha = 1. / (1 - theta);
+	cell->beta = pow(0.5, theta);
+	cell->eta = (1. - pow(2. / n, 1 - theta)) / (1. - zipf_zeta2 / cell->zetan);
+}
+
+static ZipfData *
+zipfFindOrCreateCacheCell(ZipfCache *cache, int64 n, double theta)
+{
+	int			i;
+	ZipfData   *cell;
+	ZipfData   *nearest = NULL;
+
+	if (!cache->cells)
+		cache->cells = (ZipfData *) pg_malloc(sizeof(ZipfData) * ZIPF_CACHE_SIZE);
+
+	/* search zeta for cached for given parameters */
+	for(i = 0; i < cache->cells_inited; i++)
+	{
+		cell = &cache->cells[i];
+		if (cell->nitems == n && cell->theta == theta) 
+			return &cache->cells[i];
+		
+		if (cell->theta == theta &&
+			(nearest == NULL || Abs(cell->nitems - n) < Abs(nearest->nitems - n)))
+			nearest = cell;
+	}
+
+	/* create new one if it is not exist */
+	if (cache->cells_inited != ZIPF_CACHE_SIZE)
+		i = cache->cells_inited++;
+	else
+		i = pg_lrand48() % ZIPF_CACHE_SIZE;
+	
+	zipfSetCacheCell(&cache->cells[i], nearest, n, theta);
+
+	return &cache->cells[i];
+}
+
+/* random number generator: zipfian distribution from min to max inclusive */
+static int64
+getZipfianRand(TState *thread, int64 min, int64 max, double theta)
+{
+	/* Assert(theta > MIN_ZIPFIAN_PARAM); */
+	int64		n = max - min + 1;
+	ZipfData   *cell = zipfFindOrCreateCacheCell(&thread->zipf_cache, n, theta);
+
+	return min + zipfn(thread, cell, n, theta) - 1;
+}
+
 /*
  * Initialize the given SimpleStats struct to all zeroes
  */
@@ -1567,6 +1688,7 @@ evalFunc(TState *thread, CState *st,
 		case PGBENCH_RANDOM:
 		case PGBENCH_RANDOM_EXPONENTIAL:
 		case PGBENCH_RANDOM_GAUSSIAN:
+		case PGBENCH_RANDOM_ZIPFIAN:
 			{
 				int64		imin,
 							imax;
@@ -1617,6 +1739,18 @@ evalFunc(TState *thread, CState *st,
 						setIntValue(retval,
 									getGaussianRand(thread, imin, imax, param));
 					}
+					else if (func == PGBENCH_RANDOM_ZIPFIAN)
+					{
+						if (param <= MIN_ZIPFIAN_PARAM)
+						{
+							fprintf(stderr,
+									"zipfian parameter must be greater than %f "
+									"(not %f)\n", MIN_ZIPFIAN_PARAM, param);
+							return false;
+						}
+						setIntValue(retval,
+									getZipfianRand(thread, imin, imax, param));
+					}
 					else		/* exponential */
 					{
 						if (param <= 0.0)
@@ -4262,8 +4396,10 @@ main(int argc, char **argv)
 		thread->random_state[0] = random();
 		thread->random_state[1] = random();
 		thread->random_state[2] = random();
-		thread->logfile = NULL; /* filled in later */
+		thread->logfile = NULL;	/* filled in later */
 		thread->latency_late = 0;
+		thread->zipf_cache.cells = NULL;	 /* filled in later */
+		thread->zipf_cache.cells_inited = 0;
 		initStats(&thread->stats, 0);
 
 		nclients_dealt += thread->nstate;
diff --git a/src/bin/pgbench/pgbench.h b/src/bin/pgbench/pgbench.h
index abc13e9463..1a29f1260c 100644
--- a/src/bin/pgbench/pgbench.h
+++ b/src/bin/pgbench/pgbench.h
@@ -75,7 +75,8 @@ typedef enum PgBenchFunction
 	PGBENCH_SQRT,
 	PGBENCH_RANDOM,
 	PGBENCH_RANDOM_GAUSSIAN,
-	PGBENCH_RANDOM_EXPONENTIAL
+	PGBENCH_RANDOM_EXPONENTIAL,
+	PGBENCH_RANDOM_ZIPFIAN
 } PgBenchFunction;
 
 typedef struct PgBenchExpr PgBenchExpr;
In reply to: Alik Khilazhev (#11)
Re: [WIP] Zipfian distribution in pgbench

On Wed, Jul 12, 2017 at 4:28 AM, Alik Khilazhev
<a.khilazhev@postgrespro.ru> wrote:

I am attaching results of query that you sent. It shows that there is
nothing have changed after executing tests.

But something did change! In the case where performance was good, all
internal pages on the level above the leaf level have exactly 285
items, excluding the rightmost page, which unsurprisingly didn't fill.
However, after increasing client count to get the slow case, the "hot"
part of the keyspace (the leaf pages owned by the first/leftmost
internal page on that level) has 353 items rather than 285.

Now, that might not seem like that much of a difference, but if you
consider how duplicates are handled in the B-Tree code, and how unique
index enforcement works, I think it could be. It could lead to heavy
buffer lock contention, because we sometimes do a lot of work with an
exclusive buffer lock held.

This is something that I go into a lot of detail on in the Wiki page
on Key normalization:
https://wiki.postgresql.org/wiki/Key_normalization#Avoiding_unnecessary_unique_index_enforcement

Now, I'm not saying that I know for sure that that's what it is. It
seems like a good area to investigate, though. Even if it wasn't
buffer lock contention, we're still talking about the difference
between the hot part of the B-Tree being about 353 pages, versus 285.
Buffer lock contention could naturally limit the growth in size to
"only" 353, by slowing everything down.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

In reply to: Peter Geoghegan (#13)
Re: [WIP] Zipfian distribution in pgbench

On Wed, Jul 12, 2017 at 12:30 PM, Peter Geoghegan <pg@bowt.ie> wrote:

On Wed, Jul 12, 2017 at 4:28 AM, Alik Khilazhev
<a.khilazhev@postgrespro.ru> wrote:

I am attaching results of query that you sent. It shows that there is
nothing have changed after executing tests.

But something did change! In the case where performance was good, all
internal pages on the level above the leaf level have exactly 285
items, excluding the rightmost page, which unsurprisingly didn't fill.
However, after increasing client count to get the slow case, the "hot"
part of the keyspace (the leaf pages owned by the first/leftmost
internal page on that level) has 353 items rather than 285.

Could you please run the query again for both cases, but this time
change it to get items from the leaf level too, and not just internal
levels? Just remove the "wheretype != 'l' " clause in one of the CTEs.
That should do it.

It's probably the case that the hot part of the B-tree is actually
significantly less than 353 items (or 285 items), and so the "before"
and "after" difference in how many pages are used for the hot part of
the keyspace could be quite a lot larger than my initial estimate. It
could be that the granularity that is shown on the internal pages is
insufficient to see just how bad the problem is.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#15Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Peter Geoghegan (#13)
Re: [WIP] Zipfian distribution in pgbench

Peter Geoghegan wrote:

Now, that might not seem like that much of a difference, but if you
consider how duplicates are handled in the B-Tree code, and how unique
index enforcement works, I think it could be. It could lead to heavy
buffer lock contention, because we sometimes do a lot of work with an
exclusive buffer lock held.

Not to mention work done with a "buffer cleanup lock" held -- which is
compounded by the fact that acquiring such a lock is prone to starvation
if there are many scanners of that index. I've seen a case where a very
hot table is scanned so heavily that vacuum is starved for days waiting
to acquire cleanup on a single page (vacuum was only able to finish
because the app using the table was restarted). I'm sure that a uniform
distribution of keys, with a uniform distribution of values scanned,
would give a completely different behavior than a highly skewed
distribution where a single key receives a large fraction of the scans.

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

In reply to: Alvaro Herrera (#15)
Re: [WIP] Zipfian distribution in pgbench

On Wed, Jul 12, 2017 at 1:55 PM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:

Not to mention work done with a "buffer cleanup lock" held -- which is
compounded by the fact that acquiring such a lock is prone to starvation
if there are many scanners of that index. I've seen a case where a very
hot table is scanned so heavily that vacuum is starved for days waiting
to acquire cleanup on a single page (vacuum was only able to finish
because the app using the table was restarted). I'm sure that a uniform
distribution of keys, with a uniform distribution of values scanned,
would give a completely different behavior than a highly skewed
distribution where a single key receives a large fraction of the scans.

Good point.

I'd be interested in seeing the difference it makes if Postgres is
built with the call to _bt_check_unique() commented out within
nbtinsert.c. The UPDATE query doesn't ever change indexed columns, so
there should be no real need for the checking it does in this case.
We're seeing a lot of duplicates in at least part of the keyspace, and
yet the queries themselves should be as HOT-safe as possible.

Another possibly weird thing that I noticed is latency standard
deviation. This is from Alik's response to my request to run that SQL
query:

latency average = 1.375 ms
tps = 93085.250384 (including connections establishing)
tps = 93125.724773 (excluding connections establishing)
SQL script 1: /home/nglukhov/ycsb_read_zipf.sql
- weight: 1 (targets 50.0% of total)
- 2782999 transactions (49.8% of total, tps = 46364.447705)
- latency average = 0.131 ms
- latency stddev = 0.087 ms
SQL script 2: /home/nglukhov/ycsb_update_zipf.sql
- weight: 1 (targets 50.0% of total)
- 2780197 transactions (49.8% of total, tps = 46317.766703)
- latency average = 2.630 ms
- latency stddev = 14.092 ms

This is from the 128 client case -- the slow case.

Notice that the standard deviation is very high for
ycsb_update_zipf.sql. I wonder if this degrades because of some kind
of feedback loop, that reaches a kind of tipping point at higher
client counts.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

In reply to: Peter Geoghegan (#16)
Re: [WIP] Zipfian distribution in pgbench

On Wed, Jul 12, 2017 at 2:17 PM, Peter Geoghegan <pg@bowt.ie> wrote:

I'd be interested in seeing the difference it makes if Postgres is
built with the call to _bt_check_unique() commented out within
nbtinsert.c.

Actually, I mean that I wonder how much of a difference it would make
if this entire block was commented out within _bt_doinsert():

if (checkUnique != UNIQUE_CHECK_NO)
{
...
}

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#18Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Peter Geoghegan (#17)
5 attachment(s)
Re: [WIP] Zipfian distribution in pgbench

On 13 Jul 2017, at 00:20, Peter Geoghegan <pg@bowt.ie> wrote:

Actually, I mean that I wonder how much of a difference it would make
if this entire block was commented out within _bt_doinsert():

if (checkUnique != UNIQUE_CHECK_NO)
{

}

I am attaching results of test for 32 and 128 clients for original and patched(_bt_doinsert) variants.

Thanks and Regards,
Alik Khilazhev
Postgres Professional:
http://www.postgrespro.com <http://www.postgrespro.com/&gt;
The Russian Postgres Company

Attachments:

pgbench_index_after_32_patched.txttext/plain; name=pgbench_index_after_32_patched.txt; x-unix-mode=0600Download
pgbench_index_after_32.txttext/plain; name=pgbench_index_after_32.txt; x-unix-mode=0600Download
pgbench_index_after_128_patched.txttext/plain; name=pgbench_index_after_128_patched.txt; x-unix-mode=0600Download
pgbench_index_after_128.txttext/plain; name=pgbench_index_after_128.txt; x-unix-mode=0600Download
pgbench_index_before.txttext/plain; name=pgbench_index_before.txt; x-unix-mode=0600Download
#19Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Alik Khilazhev (#12)
4 attachment(s)
Re: [WIP] Zipfian distribution in pgbench

Hello Alik,

A few comments about the patch v2.

Patch applies and compiles.

Documentation says that the closer theta is from 0 the flatter the distribution
but the implementation requires at least 1, including strange error messages:

zipfian parameter must be greater than 1.000000 (not 1.000000)

Could theta be allowed between 0 & 1 ? I've tried forcing with theta = 0.1
and it worked well, so I'm not sure that I understand the restriction.
I also tried with theta=0.001 but it seemed less good.

I have also tried to check the distribution wrt the explanations, with the
attached scripts, n=100, theta=1.000001/1.5/3.0: It does not seem to work,
there is repeatable +15% bias on i=3 and repeatable -3% to -30% bias for
values in i=10-100, this for different values of theta (1.000001,1.5,
3.0).

If you try the script, beware to set parameters (theta, n) consistently.

About the code:

Remove spaces and tabs at end of lines or on empty lines.

zipfn: I would suggest to move the pg_erand48 out and pass the result
instead of passing the thread. the erand call would move to getZipfianRand.

I'm wondering whether the "nearest" hack could be removed so as to simplify
the cache functions code...

Avoid editing lines without changes (spacesn/tabs?)
   -  thread->logfile = NULL; /* filled in later */
   +  thread->logfile = NULL; /* filled in later */

The documentation explaining the intuitive distribution talks about a N
but does not provide any hint about its value depending on the parameters.

There is an useless empty lien before "</para>" after that.

--
Fabien.

Attachments:

compte_init.sqlapplication/x-sql; name=compte_init.sqlDownload
compte_bench.sqlapplication/x-sql; name=compte_bench.sqlDownload
compte_expected.sqlapplication/x-sql; name=compte_expected.sqlDownload
compte_results.sqlapplication/x-sql; name=compte_results.sqlDownload
In reply to: Alik Khilazhev (#18)
Re: [WIP] Zipfian distribution in pgbench

On Thu, Jul 13, 2017 at 4:38 AM, Alik Khilazhev
<a.khilazhev@postgrespro.ru> wrote:

I am attaching results of test for 32 and 128 clients for original and
patched(_bt_doinsert) variants.

Thanks.

The number of leaf pages at the left hand side of the leaf level seems
to be ~50 less than the unpatched 128 client case was the first time
around, which seems like a significant difference. I wonder why. Maybe
autovacuum ran at the right/wrong time this time around?

Did you happen to record TPS for this most recent set of tests?

I notice one possibly interesting thing from these new numbers: the 32
client case is slightly more bloated when unique index enforcement is
removed.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

In reply to: Peter Geoghegan (#20)
Re: [WIP] Zipfian distribution in pgbench

On Thu, Jul 13, 2017 at 10:02 AM, Peter Geoghegan <pg@bowt.ie> wrote:

The number of leaf pages at the left hand side of the leaf level seems
to be ~50 less than the unpatched 128 client case was the first time
around, which seems like a significant difference. I wonder why. Maybe
autovacuum ran at the right/wrong time this time around?

To reiterate what I say above:

The number of leaf pages with dead items is 20 with this most recent
run (128 clients, patched + unpatched). The leftmost internal page one
level up from the leaf level contains 289 items. Whereas last time it
was 353 items.

That's a difference between having 20 hot/bloated leaf pages, and
probably 84 hot/bloated pages, which I infer must have been the total
number of bloated leaf pages within "result.txt". I think that
something about all the "pgbench_index_*txt" tests are very different
to what we see within "result.txt". It's as if there was a problem
when "result.txt" ran, but that problem somehow didn't come up when
you did new tests.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

In reply to: Peter Geoghegan (#21)
Re: [WIP] Zipfian distribution in pgbench

On Thu, Jul 13, 2017 at 12:49 PM, Peter Geoghegan <pg@bowt.ie> wrote:

To reiterate what I say above:

The number of leaf pages with dead items is 20 with this most recent
run (128 clients, patched + unpatched). The leftmost internal page one
level up from the leaf level contains 289 items. Whereas last time it
was 353 items.

That's a difference between having 20 hot/bloated leaf pages, and
probably 84 hot/bloated pages, which I infer must have been the total
number of bloated leaf pages within "result.txt". I think that
something about all the "pgbench_index_*txt" tests are very different
to what we see within "result.txt". It's as if there was a problem
when "result.txt" ran, but that problem somehow didn't come up when
you did new tests.

I just figured out that "result.txt" is only a 60 second pgbench run.
Is the same true for other tests?

It would be much more interesting to see runs that lasted 10 minutes
or more. Maybe with pgbench-tools. I would expect that a decline in
performance due to bloating the index could happen only after a
certain threshold was crossed. Things like HOT pruning and LP_DEAD
cleanup could be pretty effective, until finally a tipping point is
reached and they're much less effective.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#23Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Fabien COELHO (#19)
Re: [WIP] Zipfian distribution in pgbench

On 13 Jul 2017, at 19:14, Fabien COELHO <coelho@cri.ensmp.fr> wrote:

Documentation says that the closer theta is from 0 the flatter the distribution
but the implementation requires at least 1, including strange error messages:

zipfian parameter must be greater than 1.000000 (not 1.000000)

Could theta be allowed between 0 & 1 ? I've tried forcing with theta = 0.1
and it worked well, so I'm not sure that I understand the restriction.
I also tried with theta=0.001 but it seemed less good.

Algorithm works with theta less than 1. The only problem here is that theta can not be 1, because of next line of code

cell->alpha = 1. / (1 - theta);

That’s why I put such restriction. Now I see 2 possible solutions for that:
1) Exclude 1, and allow everything in range (0;+∞).
2) Or just increase/decrease theta by very small number if it is 1.

I have also tried to check the distribution wrt the explanations, with the attached scripts, n=100, theta=1.000001/1.5/3.0: It does not seem to work, there is repeatable +15% bias on i=3 and repeatable -3% to -30% bias for values in i=10-100, this for different values of theta (1.000001,1.5, 3.0).

If you try the script, beware to set parameters (theta, n) consistently.

I've executed scripts that you attached with different theta and number of outcomes(not n, n remains the same = 100) and I found out that for theta = 0.1 and big number of outcomes it gives distribution very similar to zipfian(for number of outcomes = 100 000, bias -6% to 8% in whole range and for NOO = 1000 000, bias is -2% to 2%).

By, number of outcomes(NOO) I mean how many times random_zipfian was called. For example:
pgbench -f compte_bench.sql -t 100000

So, I think it works but works worse for small number of outcomes. And also we need to find optimal theta for better results.


Thanks and Regards,
Alik Khilazhev
Postgres Professional:
http://www.postgrespro.com <http://www.postgrespro.com/&gt;
The Russian Postgres Company

#24Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Peter Geoghegan (#22)
2 attachment(s)
Re: [WIP] Zipfian distribution in pgbench

On 13 Jul 2017, at 23:13, Peter Geoghegan <pg@bowt.ie> wrote:

I just figured out that "result.txt" is only a 60 second pgbench run.
Is the same true for other tests?

Yes, other tests ran 60 seconds too.

It would be much more interesting to see runs that lasted 10 minutes
or more.

I am attaching results of tests for 32 and 128 clients that were running for 10 minutes, and TPS remains 305 and 115 ktps respectively.

Tests was executed with configuration set for YCSB. And there is very aggressively autovacuum, this can be reason why there is no decline appears with increasing working time.
Autovacuum config:

autovacuum_max_workers = 8
autovacuum_naptime = 10s
autovacuum_vacuum_scale_factor = 0.1
autovacuum_vacuum_cost_delay = 0ms
autovacuum_vacuum_cost_limit = 10000

Attachments:

pgbench_index_after_128_10min.txttext/plain; name=pgbench_index_after_128_10min.txt; x-unix-mode=0600Download
pgbench_index_after_32_10min.txttext/plain; name=pgbench_index_after_32_10min.txt; x-unix-mode=0600Download
#25Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Alik Khilazhev (#23)
Re: [WIP] Zipfian distribution in pgbench

Algorithm works with theta less than 1. The only problem here is that
theta can not be 1, because of next line of code

cell->alpha = 1. / (1 - theta);

That’s why I put such restriction. Now I see 2 possible solutions for that:
1) Exclude 1, and allow everything in range (0;+∞).

Yep.

2) Or just increase/decrease theta by very small number if it is 1.

Nope, this seems quite arbitrary.

I've executed scripts that you attached with different theta and number
of outcomes(not n, n remains the same = 100) and I found out that for
theta = 0.1 and big number of outcomes it gives distribution very
similar to zipfian(for number of outcomes = 100 000, bias -6% to 8% in
whole range and for NOO = 1000 000, bias is -2% to 2%).

Ok, so you did not get the large bias for i=3. Strange.

By, number of outcomes(NOO) I mean how many times random_zipfian was
called. For example: pgbench -f compte_bench.sql -t 100000 So, I think
it works but works worse for small number of outcomes. And also we need
to find optimal theta for better results.

Hmmm. I've run one million outcomes each time.

I'll check on the next version.

--
Fabien.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

In reply to: Alik Khilazhev (#24)
Re: [WIP] Zipfian distribution in pgbench

On Fri, Jul 14, 2017 at 6:37 AM, Alik Khilazhev
<a.khilazhev@postgrespro.ru> wrote:

I am attaching results of tests for 32 and 128 clients that were running for 10 minutes, and TPS remains 305 and 115 ktps respectively.

Tests was executed with configuration set for YCSB. And there is very aggressively autovacuum, this can be reason why there is no decline appears with increasing working time.
Autovacuum config:

autovacuum_max_workers = 8
autovacuum_naptime = 10s
autovacuum_vacuum_scale_factor = 0.1
autovacuum_vacuum_cost_delay = 0ms
autovacuum_vacuum_cost_limit = 10000

I think that what this probably comes down to, more than anything
else, is that you have leftmost hot/bloated leaf pages like this:

idx | level | l_item | blkno | btpo_prev |
btpo_next | btpo_flags | type | live_items | dead_items |
avg_item_size | page_size | free_size | highkey
-----------------------+-------+--------+-------+-----------+-----------+------------+------+------------+------------+---------------+-----------+-----------+-------------------------
...
pgbench_accounts_pkey | 0 | 1 | 1 | 0 |
2751 | 65 | l | 100 | 41 | 16 |
8192 | 5328 | 11 00 00 00 00 00 00 00
pgbench_accounts_pkey | 0 | 2 | 2751 | 1 |
2746 | 65 | l | 48 | 90 | 16 |
8192 | 5388 | 32 00 00 00 00 00 00 00
...

The high key for the leftmost shows that only values below 0x11 belong
on the first page. This is about 16 or 17 possible distinct values,
and yet the page has 100 live items, and 41 dead items; in total,
there is room for 367 items. It's only slightly better with other
nearby pages. This is especially bad because once the keyspace gets
split up this finely, it's *impossible* to reverse it -- it's more or
less a permanent problem, at least until a REINDEX. You cannot merge
pages back together until one is completely empty, which in this case
and many cases will in fact never happen. Aggressive vacuuming is
probably helpful in part because it prevents the problem from ever
getting completely out of hand. That doesn't seem like a very
practical solution, though.

We should probably be treating unique indexes in a special way, since
inserting a new conflicting tuple necessarily supersedes whatever it
conflicted with. Postgres holds on to the old tuple today, but only
because the transaction might abort, or because an old snapshot might
be interested in old tuple versions. However, the general pattern with
unique indexes is that there must be one tuple visible to new
snapshots, and old versions are almost certainly going to became
garbage very quickly. Unique indexes really are quite special, which
nbtree doesn't take advantage of at all. If you read the coverage of
B-Trees within "Transaction Processing: Concepts and Techniques", and
many other publications, the general trend seems to be that unique
indexes have truly unique keys, based only on the user-visible key
values. They make a sharp distinction between primary and secondary
indexes, which doesn't really exist in Postgres (at least, not at the
access method level).

I suspect that the best long term solution is to add GIN-style
duplicate handling within nbtree for unique indexes, with special
pruning style optimizations to the posting list structure. This would
probably only happen with unique indexes. The useful thing about this
approach is it separates these two problems:

1. Representing what values are in the index for lookup, and their
latest row version.

2. Finding old row versions, in the less common case where you have an
old snapshot.

With a design like that, nbtree would never "cut up the keyspace too
finely" because of a temporary surge of UPDATE insertions. You still
get bloat, but you add it to a place where garbage collection can be
much better targeted. Under this scheme, it doesn't really matter so
much if garbage collection doesn't happen very frequently, because
there could be LP_DEAD style hints for the auxiliary posting list
structure. That could probably work based on atomic ops, and would
greatly reduce the number of pages that UPDATE workloads like this
dirtied.

It probably wouldn't make sense to add things like GIN's compression
of item pointers, since most data within the auxiliary posting list
structure is removed very quickly. I wouldn't expect btree_gin to be
faster for this workload today, because it doesn't have
kill_prior_tuple/LP_DEAD support, and because it doesn't support
unique indexes, and so cannot exploit the special situation that
exists with unique indexes.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#27Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Fabien COELHO (#25)
1 attachment(s)
Re: [WIP] Zipfian distribution in pgbench

Hello Fabien,

On 14 Jul 2017, at 17:51, Fabien COELHO <coelho@cri.ensmp.fr> wrote:

Ok, so you did not get the large bias for i=3. Strange.

I got large bias for i=3 and theta > 1 even with a million outcomes, but for theta < 1 (I have tested on theta = 0.1 and 0.3) it showed quite good results.

I am attaching patch v3. Among other things I fixed small typo in description of random_exponential function in pgbench.sgml file.


Thanks and Regards,
Alik Khilazhev
Postgres Professional:
http://www.postgrespro.com <http://www.postgrespro.com/&gt;
The Russian Postgres Company

Attachments:

pgbench-zipf-03v.patchapplication/octet-stream; name=pgbench-zipf-03v.patch; x-unix-mode=0644Download
diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml
index 64b043b48a..68a0f1a698 100644
--- a/doc/src/sgml/ref/pgbench.sgml
+++ b/doc/src/sgml/ref/pgbench.sgml
@@ -1014,6 +1014,14 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
        <entry>an integer between <literal>1</> and <literal>10</></>
       </row>
       <row>
+        <entry><literal><function>random_zipfian(<replaceable>lb</>, <replaceable>ub</>, <replaceable>parameter</>)</></></>
+        <entry>integer</>
+        <entry> Zipfian-distributed random integer in <literal>[lb, ub]</>,
+              see below</>
+        <entry><literal>random_zipfian(1, 10, 1.2)</></>
+        <entry>an integer between <literal>1</> and <literal>10</></>
+      </row>
+      <row>
        <entry><literal><function>sqrt(<replaceable>x</>)</></></>
        <entry>double</>
        <entry>square root</>
@@ -1045,7 +1053,7 @@ f(x) = exp(-parameter * (x - min) / (max - min + 1)) / (1 - exp(-parameter))
 </literallayout>
       Then value <replaceable>i</> between <replaceable>min</> and
       <replaceable>max</> inclusive is drawn with probability:
-      <literal>f(x) - f(x + 1)</>.
+      <literal>f(i) - f(i + 1)</>.
      </para>
 
      <para>
@@ -1094,6 +1102,21 @@ f(x) = PHI(2.0 * parameter * (x - mu) / (max - min + 1)) /
       of the Box-Muller transform.
      </para>
     </listitem>
+    <listitem>
+        <para>
+          For Zipfian distribution, <replaceable>parameter</> 
+          defines how skewed the distribution. A physical sense of this parameter
+          is following: probability of <literal>i</>th element <literal>f(i) = (1 / (i ^ parameter)) /
+          (H(N, parameter))</> where <literal>N</> is number of elements and <literal>H(N, parameter)</>
+          is <literal>N</>th generalized harmonic number. 
+        </para>
+        <para>
+          Intuitively, the larger the <replaceable>parameter</>, the more
+          frequently value values to the beginning of the interval are drawn. 
+          The closer to 0 <replaceable>parameter</> is, 
+          the flatter (more uniform) the access distribution.
+        </para>
+    </listitem>
    </itemizedlist>
 
   <para>
diff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.y
index b3a2d9bfd3..25d5ad48e5 100644
--- a/src/bin/pgbench/exprparse.y
+++ b/src/bin/pgbench/exprparse.y
@@ -191,6 +191,9 @@ static const struct
 	{
 		"random_exponential", 3, PGBENCH_RANDOM_EXPONENTIAL
 	},
+	{
+		"random_zipfian", 3, PGBENCH_RANDOM_ZIPFIAN
+	},
 	/* keep as last array element */
 	{
 		NULL, 0, 0
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 4d364a1db4..6126d9a21d 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -93,6 +93,8 @@ static int	pthread_join(pthread_t th, void **thread_return);
 #define LOG_STEP_SECONDS	5	/* seconds between log messages */
 #define DEFAULT_NXACTS	10		/* default nxacts */
 
+#define ZIPF_CACHE_SIZE	15		/* cache cells number */
+
 #define MIN_GAUSSIAN_PARAM		2.0 /* minimum parameter for gauss */
 
 int			nxacts = 0;			/* number of transactions per client */
@@ -334,6 +336,29 @@ typedef struct
 } CState;
 
 /*
+ * Cache cell for zipfian_random call
+ */
+typedef struct
+{
+	double		zetan;			/* zeta(n) */
+	double		theta;			/* theta parameter of previous execution */
+	int64		nitems;			/* n(ub - lb + 1) parameter of previous
+								 * execution */
+	double		alpha;
+	double		beta;
+	double		eta;
+} ZipfData;
+
+/*
+ * Zipf cache for zeta values
+ */
+typedef struct
+{
+	int			cells_inited;
+	ZipfData   *cells;
+} ZipfCache;
+
+/*
  * Thread state
  */
 typedef struct
@@ -345,6 +370,7 @@ typedef struct
 	unsigned short random_state[3]; /* separate randomness for each thread */
 	int64		throttle_trigger;	/* previous/next throttling (us) */
 	FILE	   *logfile;		/* where to log, or NULL */
+	ZipfCache	zipf_cache;
 
 	/* per thread collected stats */
 	instr_time	start_time;		/* thread start time */
@@ -737,6 +763,87 @@ getPoissonRand(TState *thread, int64 center)
 	return (int64) (-log(uniform) * ((double) center) + 0.5);
 }
 
+/* helper function for getZipfianRand */
+static double
+zipfZeta(int64 n, double theta)
+{
+	int			i;
+	double		ans = 0.0;
+
+	for (i = n; i > 1; i--)
+		ans += pow(i, -theta);
+	return ans + 1;
+}
+
+/* helper function for getZipfianRand. Computes random variable in range 1..n */
+static int64
+zipfn(double uniform_random, ZipfData * zipf, int64 n, double theta)
+{
+	double		uz = uniform_random * zipf->zetan;
+
+	if (uz < 1.)
+		return 1;
+	if (uz < 1 + zipf->beta)
+		return 2;
+	return 1 + (int64) (n * pow(zipf->eta * uniform_random - zipf->eta + 1., zipf->alpha));
+}
+
+/* set zeta value and other parameters to cache cell */
+static void
+zipfSetCacheCell(ZipfData * cell, int64 n, double theta)
+{
+	double		zipf_zeta2;
+
+	cell->nitems = n;
+	cell->theta = theta;
+
+	zipf_zeta2 = zipfZeta(2, theta);
+	cell->zetan = zipfZeta(n, theta);
+
+	cell->alpha = 1. / (1 - theta);
+	cell->beta = pow(0.5, theta);
+	cell->eta = (1. - pow(2. / n, 1 - theta)) / (1. - zipf_zeta2 / cell->zetan);
+}
+
+static ZipfData *
+zipfFindOrCreateCacheCell(ZipfCache * cache, int64 n, double theta)
+{
+	int			i;
+	ZipfData   *cell;
+
+	if (!cache->cells)
+		cache->cells = (ZipfData *) pg_malloc(sizeof(ZipfData) * ZIPF_CACHE_SIZE);
+
+	/* search cached zeta for given parameters */
+	for (i = 0; i < cache->cells_inited; i++)
+	{
+		cell = &cache->cells[i];
+		if (cell->nitems == n && cell->theta == theta)
+			return &cache->cells[i];
+	}
+
+	/* create new one if it is not exist */
+	if (cache->cells_inited != ZIPF_CACHE_SIZE)
+		i = cache->cells_inited++;
+	else
+		i = pg_lrand48() % ZIPF_CACHE_SIZE;
+
+	zipfSetCacheCell(&cache->cells[i], n, theta);
+
+	return &cache->cells[i];
+}
+
+/* random number generator: zipfian distribution from min to max inclusive */
+static int64
+getZipfianRand(TState *thread, int64 min, int64 max, double theta)
+{
+	int64		n = max - min + 1;
+	ZipfData   *cell = zipfFindOrCreateCacheCell(&thread->zipf_cache, n, theta);
+	double		uniform = pg_erand48(thread->random_state);
+
+	return min + zipfn(uniform, cell, n, theta) - 1;
+}
+
 /*
  * Initialize the given SimpleStats struct to all zeroes
  */
@@ -1567,6 +1674,7 @@ evalFunc(TState *thread, CState *st,
 		case PGBENCH_RANDOM:
 		case PGBENCH_RANDOM_EXPONENTIAL:
 		case PGBENCH_RANDOM_GAUSSIAN:
+		case PGBENCH_RANDOM_ZIPFIAN:
 			{
 				int64		imin,
 							imax;
@@ -1617,6 +1725,18 @@ evalFunc(TState *thread, CState *st,
 						setIntValue(retval,
 									getGaussianRand(thread, imin, imax, param));
 					}
+					else if (func == PGBENCH_RANDOM_ZIPFIAN)
+					{
+						if (param <= 0 || param == 1)
+						{
+							fprintf(stderr,
+									"zipfian parameter must be greater than zero and can not be 1"
+									" (got %f)\n", param);
+							return false;
+						}
+						setIntValue(retval,
+									getZipfianRand(thread, imin, imax, param));
+					}
 					else		/* exponential */
 					{
 						if (param <= 0.0)
@@ -4264,6 +4384,8 @@ main(int argc, char **argv)
 		thread->random_state[2] = random();
 		thread->logfile = NULL; /* filled in later */
 		thread->latency_late = 0;
+		thread->zipf_cache.cells = NULL;	/* filled in later */
+		thread->zipf_cache.cells_inited = 0;
 		initStats(&thread->stats, 0);
 
 		nclients_dealt += thread->nstate;
diff --git a/src/bin/pgbench/pgbench.h b/src/bin/pgbench/pgbench.h
index abc13e9463..1a29f1260c 100644
--- a/src/bin/pgbench/pgbench.h
+++ b/src/bin/pgbench/pgbench.h
@@ -75,7 +75,8 @@ typedef enum PgBenchFunction
 	PGBENCH_SQRT,
 	PGBENCH_RANDOM,
 	PGBENCH_RANDOM_GAUSSIAN,
-	PGBENCH_RANDOM_EXPONENTIAL
+	PGBENCH_RANDOM_EXPONENTIAL,
+	PGBENCH_RANDOM_ZIPFIAN
 } PgBenchFunction;
 
 typedef struct PgBenchExpr PgBenchExpr;
#28Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Alik Khilazhev (#27)
Re: [WIP] Zipfian distribution in pgbench

Ok, so you did not get the large bias for i=3. Strange.

I got large bias for i=3 and theta > 1 even with a million outcomes,

Ok. So this is similar to what I got.

Is this bias expected from the drawing method, say because it is
approximated and the approximation is weak at some points, or is there an
issue with its implementation, says some shift which gets smoothed down
for higher indexes?

I am attaching patch v3. Among other things I fixed small typo in
description of random_exponential function in pgbench.sgml file.

I'll look into that.

--
Fabien.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#29Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Fabien COELHO (#28)
1 attachment(s)
Re: [WIP] Zipfian distribution in pgbench

On 17 Jul 2017, at 13:51, Fabien COELHO <coelho@cri.ensmp.fr> wrote:

Is this bias expected from the drawing method, say because it is approximated and the approximation is weak at some points, or is there an issue with its implementation, says some shift which gets smoothed down for higher indexes?

I have checked paper where such implementation was proposed and there theta allowed only on range between 0 and 1. It seems like it is not guaranteed that it should work well when theta is more than 1.

I am attaching paper, see page 23.

Attachments:

syntheticdatagen.pdfapplication/pdf; name=syntheticdatagen.pdf; x-unix-mode=0600Download
%PDF-1.2 
%����
 
14 0 obj
<<
/Length 15 0 R
/Filter /FlateDecode 
>>
stream
H��Wio�H���1��9�����9g�Ibe�,ZdKb,�J��W;����I��4���Dj�Q��x������8��rB@��L~~M ��9��4�S����;x?������o��B�n�����(��_�����+�����J*��!��"��	���j��jU��9��E���=5;��gE��0K���2�U���]����R�b&��������h��.������4����Af��'x	M��'na4����f�]�p_�.�!I��A�[�����>��`0�a�I4��]������H���������p�?���fL9�Y�G�iDw�w?SN�2j��
�`	��M#����(�=����h�p�U�P�0F�\y��SFc�K���h�	 n�,e+Vp-*x�D��M^#�M+�
\���
-�����������8N �(�mI`:�#���u�'}U���*�������������JmP�<���%>J�����Q���������J��j��������"�K"���@������
���z����U9�H�0��b��(|D����.%C�����4sU������ee��Q�	���,������>������{�^��F�"�����&�a�	@����UVp.���knJs6N����{h��x�0�`>p�����	|���5�*�R������	��]�{X[Wpv�y����:�0J]��p�{���q�����e�7A%�@�9��#�����C8�����R;�>����fR-���"%Ge|�=�hH���=���&f5�@�J��@B\���(5��U���"A` �cM�q��,���I�m�Y���/6��0o����v��!�>�*_�m�f���tS�l��=�$��BG%n��R��sk�s
����Gi��6�_���V�������$��J�Nq��n���|\8���t~�62�����z�b*��+��`1�x�s�?����GcHC���Xbq��=
3'�SW�PWx�R������I���5��;�����;����E[�In�j���Y��sLpW�K(��zee�nQ�_����T��R!%p�)�q�B���a������O�J�N�D��9c��Sw�����F`M���$�)�K���������8t���������}X�b�mu%:p�]l��|�D�kR���/���KWu�{G�l�����# ��g?��le����l��w�}T���FlW������P:���A\~��,;���]�����4M��D~A��dw��-���\J��T!Z�|$�*�����d�i*��Kv���k6�bc2�����&p���rX����Q(����;S�3~������0OJ��X���VV�0�I���Y�V�X��@cU���������7mx�� ~
������M����v_��f�h����y����Na��?a(�w���P����N)���6y]���������`�ks5.��i���p����<���b�G�C���������u*0���H�g��br�����;a&a�4����'Lq����{0R;�/������v�r]���@��pk�v�6��.���n�.�o������
����w�����M����IL%�Zp���au�~�B��B	�AuU���4�'��F:
�7{1���BU�����`���;B��E9�S�SDX!����9����
��c5��N��X>���
���0�{Y�@b�Y"o���;��]!�
cP�2gP\O����;2Y��(����B��w,l�|]+i����V��%����c*�m�/�,+�}���G=�;5
F
h�P���qV�xo�����o�,�a��hU]ty�S>>F-�${B��=y�t��)-*�qWA,0YdgU�����IK�$vFG��?���Lik7�\���(����7=�8R=b��8�=bu�7��C�i��MDx����,<�4�G:�6G�u����4���D����kc�=��� �	y�����������v�d<�6�G}*�N`���t;������O|���3�'#���`l�;'������#���v�g��x��pgv�c�����V���q�r�M<��A7H�����0�1N<�����N�^�����������=i��8r���Y~S�w+Y,�=����7���
e���$1�a�+#��r.q(������3I����W���� @m����^�j�{���w�b�~���~� �8�6�*g���'��r�-�,�(dz��,�+T09��=��ib�X%A"�kTh����+�����>�{�v
~���x co��\,[��P�1B��Fz��l�a�L'X_��u�]���T�l!9�j���v��kq�%��l�[=X��V3r�d����G�D�]�������Z�e��q��(�&��=�@����5�f}�q�u|�!��
����8���1�?�+�4��V�!��.�
P
F\�(f�Rj�z����{D���y���)��i��s��������������s9qN�ca\P��4�����*��G�+�j�����_��N( !�r�!�N�v�s��H���A���}�JV��;�j�o��}'����Mn��u�a�l�1F�As�������v��v4X�fbi(��}En��_HVH������8�I����:��Vvf���i>v�+)d�wa�sW-��eB���j�D]G�k��}��2nQ�MmI`�T��QWD��D����
yY�;�{���Lj�=x/�����m������6�������~z�`�O�\
j][�>���?�����z����V���1���9�l�����b
�b�Z&�[�������k���������2	�d�����ASh�!�	�f�'���)��G���C���E��/������g�yM.u�2�MX��y���=���>D��<W)��-P����2������PJ����E^����S\d��1XL�&��d	V��}��������S��b�B���J`pEq�|�M�G2����tU�N���F��j�2f}���/(�N7m�N�I;��yQ��{} %�;_�i�����11(b�Y�L=�5(�=����bx�2��a~�L��7����.e��y�W��K-u����Z@�����fY.H��U���*HR�MR&J�H�v�A�(�J��\����H��(N=U�0i.6;� �������������E�/#�O�
endstream
endobj
15 0 obj
3247
endobj
4 0 obj
<<
/Type /Page
/Parent 5 0 R
/Resources <<
/Font <<
/F0 6 0 R 
/F1 8 0 R 
/F2 10 0 R 
/F3 12 0 R 
>>
/ProcSet 2 0 R
>>
/Contents 14 0 R
>>
endobj
19 0 obj
<<
/Length 20 0 R
/Filter /FlateDecode 
>>
stream
H��W�r�6���>��XI���\��L'Mu:��DBb�P0���]�x���?������gW/�W5���U��__�jW��j������gR\��~����4)�~�z��Wy���O��UR�y4A�2�6~D����'�s���w�Rt������b4FV�d��wo��l�:�4>���?�o�j���7{nD��a�����rt���Zt����AvS�4I�^���Wh!�b��L��g�����,�|�jJ�iC-�9(�;C'�b4��������?�4:N�b�2�]���(��������dY�x���I�1��J��'�K���3�O�6�
�Y~�#W[��o8���=(�i*�RM�X?��v��U��h�k�MD���Y���=
�����$s��N��)�S��#������X@������E�.`��"0�$��=h�~j��upT�7���H�Y����r�-���p8�A�&���{���:o�l-)A��eIY���X��n����r�r-v=z%�6z�p��:��o���E7C��mt�g�>[��?:��4��W����6�T/�e9�
K<<�H�k�����e����X8##dXe���d��
��M�Y@�����'�g%u��|�Pb	2'���1r������
d�fRq�'����z�a'md	�m�������d��NC�	c�@Fw6g�-�#�Dv����B�B2��f�[KT�/�2�hdU���"�pK�M�����Q��Oof_Mh���"21l�,��o*����.��>�����������z�������V3	��,��iU�U�������l�����l	IDd��&wT��ml!OwXj}�?0����y
�aKY�M���"�l�,��=���9�}�5�,��`�|�����Tf�}�_���?[�x\9�b���ugI+��}/�����	J��zX�um��L�wm�SE�����2-���%�T�PY?��h+P��?n�^��%��
�����\�,�`F���[��t��Bq��P�#(T���f�0~%��r���ty8���g�=1�C�����\yO�d&�
��40$����X�i/�cF-H�~�����F��07^6h��?���y�$rJnFP��h%��Y��XQ��4��@q�Q�p�+d� 8|���`��U*��|*�gVtB<m�1~O�'�?��"�HT�\Fo-����\6L?h��i����uq��g�0���������X��2��d��j�c��jxn'�����������pV?�"��$i���'�i�>j>��3}�.����-��)��
2B�;J���%[��D�M�$�{�ND�e��h�}�5~1����#����yn3Z���=���M,��Ib�F������6�����F
R�bc���@�S����=n��G���d�-}D���+�?�q+�,d��w[���</��&q�K�X�A�L;�P��Q5N�3�W�R^.Z@������xX��{��8(i�";S�^`�4CwK~,]`�����j�c�/�j������~k{����d�
endstream
endobj
20 0 obj
1564
endobj
16 0 obj
<<
/Type /Page
/Parent 5 0 R
/Resources <<
/Font 23 0 R
/ProcSet 2 0 R
>>
/Contents 19 0 R
>>
endobj
23 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F4 17 0 R
/F5 21 0 R
>>
endobj
25 0 obj
<<
/Length 26 0 R
/Filter /FlateDecode 
>>
stream
H��W�n������~0���7�Y���b�v4�8�8���h
�����O�����F6rY����U�:u��t5�0J8^����#���A��we�������
�t�^���%1sg�C�����w��^�8���$�:���r}�yD���h���o�i���M-�"8����L�����\��4��\��X��(�M{���u[����}~�wE��'�5�����9O$d�S}E�B�v=�Hj��;���Z(�����w+���
\M"*��5����9�S����#Bu�0O(S�����-�����z�US�����($y�Xe7���S������/�x�����6~.P�u�
~��6o�+M:@%��x�
��D-�@?�����Vpn-}*Z�����/���-P��-���
]>"�Z��UQ�P�
xt�����*7���uO�J�6lc�}�r��.�cq�C��u��f���n�
��}s�>����?����b�9F`Z�j5����Y]�����C���P�`��!�{�#�����`j8����
��q[���e�������l�������]D���e�#bG���u��`���F�yfb��G ��6c��6�������UE���	�ID�.�U��;�U��n�}0�n��1�/s8����U\�����j���X��L�I�-����Eh�q���D�vu��*�����Oy�o6���*J<���JQ)QC�W�����*��kP	!k�����X��Z1��-�8S�	���+�xUt�M�n�|#����.�T��}Or��gYbx&"fNW�������������)�|_�O�'�`�����=_������}��J�79����J�A*��bV��#,��w��������v�y�6�����b_�b�>
$:M	���!��P^`���H�"n�Vki���Q�[a7���B����[p��U���[��I��}�6uGwH�A�"��E�"��I��U#�m���[6c�L���}U^?���a�A�1xi��7��HaH������f~�6�xAr���M�Q-qJ57��]}��Flp+��V���(������ :DK�6��xPh&��
eD[�!tY��#����VO�}�Z6x���Z�E�X�(1����� ��Y�2�;���;��rS�z$R$����@����;K=X0�,)2��!��q��kQO?|Z�=n�\|���o�Q��@Yv��c��L�Z��e�AZJ��,�����M�!foOU�/H2���9�Q�42=���y��m������|��|{,}������TU#M%�%^/�f�����s�O���l>:�������-�D�����]�������Z�5��IU"���v�A0�����raD��Q/�[^��"s�j�J���E�C���+F/N�m��a�����4�#N��Ej���36#������??E&V�L	�m.����c}+O���4����	�D���F�&��N��E�m�I8�@~uT��U@Yl�H�����X2SC-L�i�ae�����	M��P�Wx���2hBS���@��'&���.���se����	}���CUwwWx�t�����L�W��2�Y��&���W(�!��6�l��R��{�b���o��^�������]	W#N�*�<�G��]A��Y7��=���/�$��$�����a#d��z�f5F5,90��l����a���y�q�#i��D�c����6�&sS��e�����P��*.���BB��)��Fe�P'M-u�X���_u��w��C������ ��c����o�.�/����N^��D�.Z��//����?q��YF��{�txj)�<z�Y���;rFe�����{������u��V�b��C LFaH=��a	���K�
'��l�yXM�:�`�p��z��/�E�G�(��76%�'���PQ6����=�x�;4�P��S8qv�xr�a�f��3������4h��[!p�0Z�'��������
�[M�q-�r
����4��y
4O�ia��'��hF�/��W������2��J�Ll��������L��E����U�E�JI���Nzb5g���Y%�f���<���a���~�s�VJ�xH��0sW2���.�L��qf��t+��u���=���
u����U����rvc"��#�[e�����w�+_B���}�N>����d�Df�������]�CX2���������Z ���l��a,�����PH�:����9�_��PMD�����,q,�VC��jE��B�Q�?N�]56}��J1�[��N-��}��Ho:��T�+����R�$Yl:�;��n;z1��'�W)w
Y�R�m��<{
}?���Y87��f"�er>�P|9���c�7R�s8���2k#���V���G������&o�r](����10�m�(�R���BP��������xO�A]��q��+R����
���
>}(nk"%,1�j��6;�;�j���>�n�� m�}�<������9I.��PC��c���)@ib��yj�I��B��J��e)��W���}9N��S�a�`k1�RjR���^�0����"�o�BL�
&�����q��=�%���IxR2.7$Q��XB�y1������k��)�<��>��AZ�s��@����y��*O�&A������������)������?�mC�E�|`F�Z=L��^�q4��)dF��e���^5�q�@����J��asNZ)�J������K|	{���w���#�U�vOf����yof@<6����>/�L�<^�r�O�����7���S��yk���Gv;<Z�9W ��X�$�<(��c���H�h$Q��4�-��}#f ��4DLU����y�3���BsO���=.Y��G{,/P���l�����V=�-p�����h~�D��<�����G����5��4uot'�����Bn�����`K��7NC%�&��S�5��M.�X-�z���Dh�x���M��QJ���	[���������n	��{���/�afH�$������n:�������&�E�x��u��uHIza/zr1���IjiDh�$ID���v��1�j�d�(&��s+&�jF�����M�w�������1��2�����O�����0���������9a��r-��5��u]_�4���(.mB��y����
Vx����i�q#�R�YY��!f����W������x1c��U&,��p1�-���<���y����g��S�L�$p����>���>���.��%�V0���\���,#�g��z��K��4����r�j��� �-�i������.����g�Avy�[��P���s�Wf���%��:������wa(V=�ml�4|��a���U�F�Y!G������@+A��?<������"��f:WV�@� 5����O_l�t��_#x
endstream
endobj
26 0 obj
3478
endobj
24 0 obj
<<
/Type /Page
/Parent 5 0 R
/Resources <<
/Font 29 0 R
/ProcSet 2 0 R
>>
/Contents 25 0 R
>>
endobj
29 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F4 17 0 R
/F5 21 0 R
/F6 27 0 R
>>
endobj
31 0 obj
<<
/Length 32 0 R
/Filter /FlateDecode 
>>
stream
H��W�n�~���48����e���9�N[�^�k[�ZJ%9���g�\�l��V�y��?��~�<�R+��	q��L����g�RDX��5q��$���Q�?��/gZy�x����;^9�PY{��s��U��������'��[u�v�\�����_�WhKI_����-H����)lemv`�'����zs-�xZ����r!>�����v[��B	��DSuF��l�����lD�8��n���jr��L�_�A�����q5�y��
k�jpuQ���j9��Xf�j������_/��g
@��{��������E0D~������&���6>���^�q!�D����gW�~��z���:}y���Qa�{��[�)�&�����2��7*�Q�dP�}���g�����K�m�=�����/6������O���FhP��T��J���X��
��$���}���_:�{��D�a�f���G�5fDU�1%!"x����5f����5j�Q
�m��Q�dK���e8�����'%uD��gG�h�e�\*e���'-�1�8K���I�GK����n�������DP����[��7�L"4]|������Y�#Qx����v�\{�N�$����h$4�Att�����'��#3Pe�c�%�b�cG��q$�f^��:t� 2%��M��oY�H�����,�nA�z4(�Y���G5
�M�Vi��i0��)(��yPP�H���PhL�������I�9S�@��|R�5MS��<�F�o����Hs��D����ZvOy��~����-�T>2�If=T�U����e���	�&��ya�dmD�9T���x��
���<<6�J
l`�r<�k���kF� [S��w�h����d����a�h�sM:xN�w$����W���So����#M�����;=Z��`��:�84��Bj%L>���]G�2���~���4(�<�i�t	�V�����r4\�Hi���8���\�����s����4reEM�`@�=�l�O 2�M�����ROWa����J|�<
���f���|(_���h�����T�n����a?h����[]zM7p��,RM���#9g�9���t�j�v����`e=tS8�rc�B�����SF4
|F�Ke��PW��4pC�����4����������*v��=,$P�4�;
"n8gb>qh����f4��LB�'M2����4$�4�iPT
B�^���t:
��V�����&_�*P�]����D)|)T�'������Lk�4��t0�y�����!AJ	>�yk������CCcCOv��/�f��i,���tv~aT[�
��p����:��G��Hq����I\~�f�������&WR��#
�z��'�\Z��Y#`y�Po��F���i�v�n\#I>mD�Q�LT�B�,i7�BN������^K���*/��@Y����
_bQFd���5�"�$�+Vd-/�8Z$%��shV��L���l��T�*A�8���V�$���c9���3U��W ��}]

UE((+������r�����DW�e�����Y��T�^�TX�$�U���VJhV��<A�%��:'�����e�����?SC�e%���������9��"���<�g�%���R�������
��"!]�qT��&[�O���AA���	rm|�k�R��r
~/� ��	�n���+�����dN��%F��C\'���U�H��RW���kH:��i���U���@���%���$�D�pN�+�U��J��8�J����V�����
��C�e��hr��dN��%F��C\'@5���xW�@oB��VC�����C��3Gb��DZD9�
J&%�����(��2��2��3�<rh�u��.y���2D6���X�MU�h�T1��m�R(�C����F�S���Z�|+jd�g"������T��'��4Q���%�,��1^<F���S&��O��b+p���e��<y[m<��kI���80�������-��*}�|'���[Q��=W�M�u�(���G�6�#�^4��K�%���
���b���w�F(p�
��~_��P�3�p���DHz3aR�\�P�g���f9nv^���b���2����Mx�������G���Ay��f���)�
.����^����^�oD+���j"e���>�/�9$�	%�������{�X�}�u��=@� �����YF���&��z�]nw��m�*�-���}'�vq���v�J���������*����u�iw]�$�����C/�n��n�]oD����[�w����������,_~H=��RF
���������-�+�4F3&�c��c���zJ�V�M�N,W��u�E�\2S�KzY���Z��%��m*���j��x� ��v����_o�^���C��X��*K�l7��)%w�����"��Q�8@��I
��k�ny��:����Z\=A$w��;���M�/�w���h����v��k�+U��U���x.}'��Q��=��oW�;(��w�v�
Z���=�� R*����Gp����~��}�r��a�k��G�M�U���m����o�����v�]E^��i�����-%�[� ~%�9J�!��d34�}���4���1�����9����y��,cU���3V-z<�������x��E��>O�gj/���B��{Y/���^���8�%��pm���j����_��C�n��F��-��8j���d�>d�@KZ��.[Q������n��.`�EP�hI�9�93Y�
��f��|m���e��nIq�����9T��[(���&�1"sN����S�&�[�LX�TM��U�|��$�F� +�U�S�j7������j44��N,V�\*-T���,F�K	�Rr��u`����k���T�31*�(K,���S9��O��B�w!:�0�(i	_��J�/q��5\WtQ.�z����\������Q@�^��*�QB�)iD�����,�E����*�Y_�T2eG�����:�m��J|]������`,����u*��:yH[��M��SC�>iH��+qv~��X�\U�66peq{�A#��#2O@�g![����|�)������Z�����fvrzz�����
�p�����N�����-3A��� ���������.?�j5>�\�Wad0������VQ`����Q����9�@^�E|����|i7y��)>L�!�&�������u��?�Eex���^v����m�d���$]�����V������������xc���j��d!zW &��j��k
`��7�����7���8`�z�&�}Ft���&��&�f����}�<w����I�d�\�d�Pk1n,�)��M���n
��s� �O.N�����Ps�X��$i����M�t�bG��c� ����������>>��a`v������#�����&��e�{|���1��j5���������=�ES���j*���C���h8V�`RV:tP
�~�o0uh������P#����F��`��K�s�C5�4�9��'������m���Y���e�8�^�r��K��A��X��E�?������8��u����d<��l	���g~tG���z"y"6�l'X�!�^����`�����(�����������uQL����6�(��>��@���X���^k%��w�G��N��x"���&�p<_��`Xe��l239��fi\�90����f�@����$x�5aK��JFs���V���S{~��i�j���=����=����v":<��_�
m��}d@k������++c�NJs����5�d���Kn���C�� M���������cyW�Z��:HO�Uz���H/%��%A6Uy� %w����8��+V(�~0�\��S+I\I�z��j�E�d��� ���M�o/$��q�c�Y�����V��%�U2�f������&K��A�Oh�e��Pf�*�aG��xsY�����X�D�����U�l�~�����Z6�xC~�_�S������_�=��+y��� %�����R�n�av�����g9�.�R���*7i���i�����;���
�%T
t�&	���;�	���HV3��{�J`�R�Y-M
1�'��z�z1'���F�p������^I��S�[0��������Bd�g��% �� ��4)�K�Jo��;[��/�$
endstream
endobj
32 0 obj
4243
endobj
30 0 obj
<<
/Type /Page
/Parent 5 0 R
/Resources <<
/Font 35 0 R
/ProcSet 2 0 R
>>
/Contents 31 0 R
>>
endobj
35 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F5 21 0 R
/F7 33 0 R
>>
endobj
37 0 obj
<<
/Length 38 0 R
/Filter /FlateDecode 
>>
stream
H��W�r�6���w����.;��4Mgg����}��A�i[�-������	R[v�d&� �����"c�(	
0��``�5\\�2�pm�k0_���_����_��`u7����_���aw�c���M>�>���~1����(�p]l�EU:_���31&�F{g�Dvj�T8@f1��fU���}�nL[,�&o���1
��I���ZtF��T� �%Lx'�����@<
y�����i��+c4�NJ�k�j���(g�z�B6���T�B�	�..B�2�PsD�����Zd�*�-l�����4���x,�?�b��c���l<YJQ�R�z���TW�;:��x���|�k+���7OmY��Xb�)r�������M��@����E��P�@R������N�=��}��u�;�x��U�M[?/,��%��� e�`��\"R+���B����^��sU��Kx25<�[�^��>�hQ>�z}	��y|����H�}�3�����r�Em�Q8l�	%$�/��|^l`U��X�1��b���S���a��y	���o�U]�<mcj��,v���)�vRj��������K�f���$�D����>��w^,����^����e�9�q<�����(%��b�p�����M�������F�h6��==w�d�l ���cgt��$�f/�q��C��%���ak@D~�����:��*��:>�\gh�u�C��h�3�
Q��B�����Y�F�1��N���i�O�zqx����u�
��������E���"u�$����A�Kku^�F�Ja<�F�,:����N�Y���]��+eR�Ni���S��#eDs6z����!��/]�
���Eg�F�Fo�]��3�&K36�p���S�ss��0�R&�I�|I�Kvpy��L�
C�m%���)�������A�M��[8:M�����I>51������h�@_2i8���4���8�P^���>}!t�sT���IW�����J����|��!|/�QY.��k��m�
��(=��������q�~/��<�]�g��?NN0�q��_���N4�$~&u�%����'�AmJ�
^���E��V��\���lG���7V?�ND3
w�W�Y
�c���G���I���}����+�~r#{BK�N+=�������0D&C�y1`�`W���4��%�C�D���#��b���$�A�C�~�c�9�G�#|��������C������e������G�}:z���!�W����5�� ���D@��h����I���W�f2�/����p7�I����������M����P4��������J�~9;wy����JO4�p�?^�=<�XT�vV't�Pu���n��uGy�-w4J���hxB�>q�9]�:��7?���;������
endstream
endobj
38 0 obj
1405
endobj
36 0 obj
<<
/Type /Page
/Parent 5 0 R
/Resources <<
/Font <<
/F0 6 0 R 
/F1 8 0 R 
/F2 10 0 R 
/F3 12 0 R 
/F4 17 0 R 
>>
/ProcSet 2 0 R
>>
/Contents 37 0 R
>>
endobj
40 0 obj
<<
/Length 41 0 R
/Filter /FlateDecode 
>>
stream
H��W�r��~����R�-������k����������!�5hPZ%�wO���T�FV�`��������%�E>�d����3��_�bg��s���%�c���p���=���l���!�o�u//�o^���'� p"|$��m�})�]�:m�r�.������<��q�se$p:;3[T�Y��S��������^����]�6��wi����nH��:�'
EvF����P6���B$K<c��$����'��{�o��T�f���V��;05
��f�������2����`@`��LH�+ ��R%>�S���lu��ia��>�=js6�M�,R�9�:7X��#��H0f"�X�(1���s�������-�&�6
(b7z��
�y`�������X[��3�oE�4K���n��y`+��@�:oXZ��:o���!�H$��t�^���T\�������6L�����u�#9,/���uWl�~�3#����j����`��_Qr��"��(��Fg���@�E��}�E
���7r��������#��#��c���P�/�=G�%��V�A����~���������������m��^HK'�J��Y�-K$C��nn��'�������(�dH���8��[n��\�uz��co�x���\�zA�)�=�%$�����6��a�O�(:RQL5��z���49s�&�/�������@XDv��vq�n*�sn��A�����v@Pw�:�2�w��C�;(0p~����jU��^\�d�Mu���=�N�4�)��m�YL�w����fD�E���]�w�p*��"k�W�f��&I��k��B�{����SMh��08�WP��7
N��.:� �m�5���m�����L[-�?/w�.������F�������r�Cy����M�o7S�p�������8���#���p���wq
c�+�����T2d�k%���* j��7�.?|^���I����	��k��F��0ul%����~��~x4�U��y�K��cYz�fy�`h�@��L�����4/�v9���t���1m��J����k(��u�a����6�-��P��_���n��Y�J�dN�2J2�c�
]i�5M����R�*m��.��	���c<h��b�	�;h��iFT68?���Zh��6��~���0�������K8v���[*���LD���B����������6*�3?�L����|�jLRQ����������l �p��������-�����6�����������v���W�my<%d�+*��^��e�U��V�pX�b���7�_h6u�x��a�k��40�.[�5{�165b��@��8��!���}p=����`p�9"_�;}���!��C��u�D�?�����g�=������*P�i�q�a~�0l�4��@�;���Q�_��^]�hL]k��b���Pk&��J%N�H%�4J��'�?T�O�IE�������5����
��Y����'�������U���D�t����	��G'�4�SA�����Q(�a����5�z"�MJ�����������4�wz�	�w&"���YV5�����a�{cJ����S��w���)��1$}u"�vk�qN���P2�|pA��N�]]��u�[�\O.��q������7n?�Z�S��P���	��#��;'c��;?�l�f�/q����I���2&������6�"`x���p�v�V����Iw.{�x���4�����i�x��yAU�#H�$�v��89��v�����43���0���8��@>��oW!�WW����=S�h3�?�yF�p��9w�����rS�e����K�qO�5�&��`t��bw�
}{?�H����YW�
�?8�n+]��g������%��ts�Y���6����B�,�qb�R���W����0��[(����sa��^�eZ�m���X`AsF��.�&.�_��S�6�s�n�`un��}���F#������n�$v�[�f��@G�rU�bw��p`���E�Z���5��I�]c��v��ic,j5XF�{�r�I{y.����7,���;�]s����D"	�\o�E
�l��%��pa���UTz<��f��b�����B�9j��
|n�H�����B/�ATl����R0����y�5�
`+��Ec��4V�� L��)�����G'dj���6���!�P:�G<����E8��0����$�4�{j�c��f�[g��
��_�\ 8�H2u���hTjB����/�
���]O�sq�r�"�Q����������%��<u����������%s�)
����oJ�	�������viHb�4d�����X��������<��xZ�=w��{�&�=3hR�A<02
����{w���g���z�`f5S��QC�<P��1i������	��������>�Q����8s��.AA�0)�����Cb/	���`Kdg<u�H���
�=�������B1��{M�&e��J�=�a]��$x	��y�/�����24��}Pw�(3&o):�d����
NDl������_N����-_�~	3��0��������w�#lC$����aDmvEX����"	�#�������+*Q!\PR�U��l���#I�������pvRGB��o����R���2�M�����rJ(t�.M���%��'`�D�Ba�{����-E���R���r�}����G.�[����������uN�+�6'�|�Zg�����2�s]��"���q�SEz!/��K��=3�����=����6Q������S*(R)��7���9������I��2M�R������
��]����Q��	���q ��c�N���4����0�Qw��w#I)������s��a^QB��}�\ 4�M�]
`���-��C"
��Q��6 �tq���R�v�k
�* 
���N�#I�%b�?��i��2!��d�����I��������bAR�`�
��`�^�42���]�e�VL���w9��W�sYTh��c_l�h2f�|�v��B�o@jf�c	���<�:�}{�� ��H����*����v�M��b��L�9���~7�&D�-���}�
endstream
endobj
41 0 obj
3129
endobj
39 0 obj
<<
/Type /Page
/Parent 5 0 R
/Resources <<
/Font 44 0 R
/ProcSet 2 0 R
>>
/Contents 40 0 R
>>
endobj
44 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F4 17 0 R
/F5 21 0 R
/F6 27 0 R
/F7 33 0 R
/F8 42 0 R
>>
endobj
47 0 obj
<<
/Length 48 0 R
/Filter /FlateDecode 
>>
stream
H��W�n�F��C����F�oy��q��&�&
�@�NK��C���*_�U]���hdd����:u������D�&Y�T��W����Q\�~LD	������
�Q0<��x#�~�R2��!|H�)��(������_ad���:6�S�,~��������m��sw�$�%�$�������-:,Kc�]f��g[��V���M;;5��PM���H��<@�����B��;�n�D�%J�k���� ���+n?��5P�����R�I�7P5O��dn���ym�����(�2#�b[%y1���Y��%*����(i�Z|���mm���&�sg����d���Y�����u������h��E���C�8T�Q����Qu[i
V<4���h�j��6������;�iWu�{�D18'��\��q�U�LH��������*�u�@@%F�Zq����i��p�Dxv�n;����=���d���~h���Lb�|M���h<� ��{��+K3�D�s�2�^�K�L�1����r]fs
V�(lU�k�|�o�Y-���zk�m����Q��:I6r�$O��$�/�~;��@�c��vN���������q<��^���Rh�:aM7N���u8��0]�@�Q��A�N�9�?}��bpy������.T4���nl����3�NRz
�����H���3K�*�E�2������C�G�f���+r����F��c�Z=�������y�c,�&�%��g�)��u�(jw-8�t���1�' �y[�3�C6��	�2�4�mB��ub'��u{'0\=4�
���V���sVY�K�l��v�yvN ���>���2�u/au)��2�a�nH�!s����{l�E��Y@��;�Y ��n�H�c�����
������zY�b��*���y��z!�a�Z��
��k�|aEH��,��.j���p&)�N����C�o��*Q�h����4(�)��xVjup�}IaM�W��r���JS��=D��*�(��W�dj�o!�_��T�t��e���Y��������������r��=kt�$5�H5F�`������?�*�Y��R����y��[�o��:�W���=�F���x�>:�aD�����B�-�}Q�����b���6O����O����d��"�_�yuvEY�HR������l�������gW9�B~�[L��0��-�b�7z��}����q�������W�K����O��x�]����{���r��5��j�J:S���!B�d%N�P�
���e:��W�!)�X�X0��G�.�v
�J��wZ�����*+K���/\���z�;Ov�O�I!j�1�"���<ah�'��3�_����z��^�uQ������8y'r����GGv�Z�h�\'����Z(�FxD�w�,�Ea�.�?rV��^4!,�:��\u_���� w}9?�,�.->A���Y�K��Z����Zg�o,����O��W����&��:��j���������X�^�o�`���0+%��x��f�]<J��`��t����^A�e}�p�?�k�~|�,�-�)�]�2�_�5��j��A����z��:�sb(��raKU���O���\�p�r's��e?�E�R�Z����i������=�����B�GV8W�v�����h�]��k���5m������r�G�"<x��W\����$����������K��a�0q����<�x���h�W��L}C,�E���������EfY��g\�E�:��8p{�����Gt��kf�w����3f>x4��&�� ����{�b�K.[�rQ~r�3��R;)��%E+j��1R*l�I���.T8eR.�<��'j�����N���M+�(�\cG��G�����`w���3���@OYHH����;B���o��b�!������Y�,B?fR�����E�	��+!������?;|��Y��Z����1�����T-���!d�O�C�w��)6X���'R�YV��>x�b]l�ijmg�'�C�nR��ZH�[�"{�p��������Q@�+'_��?���?� ������,x�w���D
�\?��?�4ND
��bM���2�E��t@&�F��d�*��3n�������XB����`ePQQ�y4�g�������P_f�Be��
0��� ����C�*N�4�:,��T�s<�[i/!F�d��������by���9�H�*����"��=I�@�3�����0��)Y9��u�E�N��s}����������+M�R:����/-�R�&i���I2Sr��D�L'�v�H����C��v�T>`��mn���,Ze<ZDdK6�a����Uy�EK�I��-�N�PKe�e�x��1�)D0������E��G��,*(UI^Mpy�����d���/U\K�������d:@xd4��F�h���b,������ki"�:)���DF�)��<<����Z�����zY��x�����"�����#�GLz)��'��#��]4���C��B����Y��>���;Hc���y>����4#C�u��D�3��l����s�e���f�,�:.+�{:��(/|a��(���gJ�d^g�L_�����WYn�8=�����F[�"��rg0��#'�(v`;���k!Y��`�#m.��+�XUbGRn���#���;�(��������.��C�V����(�PU�(A:���T�����H�c#>yC�/���P�C�V��ld�a�dIB\�'+��8$P���
5B��]s���aM����J�'H��Q�v����Y!!���`W7d8cg��4>��>J�y$\�3�d����d���iE�D�]{	y���Q\B��K�X���G��<Cz�R��p�hi�HD`+X8�����qNg���%�p&�w���a���JCC���g{��x%k"T���.�Yg��-��x�RF��L3�ofF���d�Gl���[�������l���j�h�����E����/�*�J�lc�_fI��s!&I��\��p<�nxY�m�.������VCl>J�4d�&���~RV7�$�����:�R�	�>r��,��b����:9���CMG�*��Zb<_#�]�/-)���+-�J����O�E��Z��AV��k/;� +�M�����(k����
�3�Cs�
�22�g�����z�&������23������j���B
D���8Xse���/�.���kI��o!���/�{�)g\���eD���u��
.�X�o,<,:�D_N�i���@�V;����m).�6�-��������9R+Yd���7��x�<�,T�����X#E�+�[^ic�m�X�I|�.UBN�e�in��E?)I���/�0�|m���==;�/B����R�S��73��o�
RP������
H�r�E\����o_�9M!��\�Z����M17��_�g�y>���E�O���7���;�	->T�I\3"hz�i:3�n��\��:�S�e�����j����^��:�/�����U�������/�����>�~���|d�H3�^T������dO>�d�j���t����Q�$��1����`���&C�0�O����yO�!��N��!���y��&���0����H�<D$Nxr���7��K��>O�����)^y��]�e����.���_�7��x�>���l��H����V�u}�C
�l���������0�vH�B�Q���w��?L/5����>���T�� �OWu�$����O���5�~0Z~�cz~�+ ?N��)�x�?IE��������0����yG�������M����OHF]��oHA?��?:=�������}J>N�_�hJ$�����c7�:��&���	��1��yu��s��[v��
endstream
endobj
48 0 obj
3962
endobj
45 0 obj
<<
/Type /Page
/Parent 46 0 R
/Resources <<
/Font 49 0 R
/ProcSet 2 0 R
>>
/Contents 47 0 R
>>
endobj
49 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F4 17 0 R
/F5 21 0 R
/F8 42 0 R
>>
endobj
51 0 obj
<<
/Length 52 0 R
/Filter /FlateDecode 
>>
stream
H��W�r����Cg�J")����e�k��o�}
S.��
�Rq�������%�7�U��AO��������\gL=������\��s �w3|�	�����0�K���%>���m^xc���yG��c�{�5����6~^��1[�{��2�E�?�,E�l�o�9c�a�����#��5q]{�|}��<)���y=��H�m\������O����@%:���@A6
j�P��*��Z ��s���;�m�d��&�;�R�k�a���%Pg�5��/#����A�Gmb��@���sXV|�I�r�%,�2�2�AV�)����"����^slD���������w�������S>�5�4��z�E�ql���=�c�����7��@GC��Ch�#����	��T�%fp����\�����@D�i#B�Css{L���`�k�*�
D^����<^��0:�f�~`�z!�	���H����������H�DY������<���jH�"KaY	�P�@���?��tHNtF��`��#F�����V<��`�IPKE��Z��&*(f�FT�,W��hJ�����>"5�����!�G�V��{�!�d����W{E�����[M
�V��8It,L�4��"���[/�N�1���������qX��4A6�pK
��?������Y��|p�I�Dw���R�k�VuaJ�����G�8��>������JL���@*�T�Z��u�-]���u���Q��b�/����Q7�
 -)�> �#��#����zL���O.o
��UYao�/qXQ���F_���u�	�+��c�7P�X�?�~�fYQ��9�N�U�'����M��b��N�#����\S��������1��7rLo\��
��S��	��c.�S7�u���
�x�W�8�`�J��f����|�Wm����j����������_��<��v��$���7�������7@��wa�_</ �>K^���Y�bj��,�z���K��*mD-�'7�="��Jhe ������L����u ��C�c����1�o��]]l���#w���s=�����dMJ��\��=����<K�GV���"��v[S}����P�B~b��
T���Q���?�h�MF;$�
`�G�o����R��� �*�7��Z����9:��E�BR�fX_��;�!a��v|��,g�>��q2��i��x�����v���g�05sp+t�����p){)'>o{t�.�9w�Dv����%����mm��F�(�rY�����I����
�,	��+k���ta3^]��s
f���-����E�/tqW����+:��������N�5���&�i����=��m���T�U ��9������	*S=��l�^4g��d:Q�*�i�Myw\�k�a���+��J����G��Iq��DF������!��8
�U�[����m���U��"����T��-����7��D/���DVF�$�*����"�#�3]%����D��K�o��)v���|g���J���!��?�r�D�3��uk8�?�1j�+41P=K�$O���y��!#��l��	�2��������	)������������E*f��VwZ����}��6�B����""<]����^��nXvBub�����a?��!��I�}�RQ%P���%�u�oy�����w�!��}��+X<b���t��D����~�Z�C�� .��tZ�5J0O�/j���q��L����A�&s'�p�VV����r�X���p��v��:�'0<�}�pA�8��<O��Kt�~n�7�S������T�$n\���f�n�)nq�l�C��<�"��+/0q���S[�F�7�MB�"�3���4iw�`��d��18��Q@
FT?�����j�=��9�:�/+���B��`-*�/�![-� r� ��Mc�&�#������C�[�z�o����x(��v��Jz�?2��>m�<�"l�����~�nU^�5b����'���KS� �j�~���!dk��j]�|��(��'�V�4:��c���eW��ML��j(
������<���P���2��&���H�y��J���bR��X�����1�d���D��3;n�6#j�sC�����7���:�)�,b�e�~���j���P�&�����1����b���dMO>�P�|����l��EUd�<ojZ�XJ��F$"������{*�^d+�uL%�E�M�	i�J�)��e��zBU4��dTW*m��0����@�F�W�����F�����s�
���u��V���%!�D�*i�����������9c� ���������Kt��?�	yo�����k��h���rL
O���-@�R�Z,ey����Z�AV����g�>$�b���W����`��^�=F�H��^<i;��|X-8�a�^=���*KU��\�z)fj<5�[;���K?�_�ph-N
endstream
endobj
52 0 obj
2464
endobj
50 0 obj
<<
/Type /Page
/Parent 46 0 R
/Resources <<
/Font 53 0 R
/ProcSet 2 0 R
>>
/Contents 51 0 R
>>
endobj
53 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F5 21 0 R
/F7 33 0 R
/F8 42 0 R
>>
endobj
55 0 obj
<<
/Length 56 0 R
/Filter /FlateDecode 
>>
stream
H��RKs�0���=&3���z\��C[z���2V�!��t����8q��eg��^�m�UJ�
��]��c�f��r���A}���W��x��wF��o�4x���f��v��	���G���>��������X����]��>`a$0T D��y��o�X"���3?�a�����ge�v�m^�L���"��F}:7NsV�����@Uo��!`�|~����ut����m�X	����Z�u(2��kj��{-9�3�=�$�Ua��Wq�����1��N�6e�<�e(��YAc�ty�����u;�
4����S@�����Fm�0�a������@ ���Sg4���Y}��c�D`&���t"��sd��=������&�o��>��Z�#������l��c����Q�O}�+�H(��QI��7���)�C��qI6=P�OV�":`��,���?�)��
endstream
endobj
56 0 obj
445
endobj
54 0 obj
<<
/Type /Page
/Parent 46 0 R
/Resources <<
/Font <<
/F0 6 0 R 
/F1 8 0 R 
/F2 10 0 R 
/F3 12 0 R 
>>
/ProcSet 2 0 R
>>
/Contents 55 0 R
>>
endobj
58 0 obj
<<
/Length 59 0 R
/Filter /FlateDecode 
>>
stream
H��Wk�����|����ymo� �n��?PJ�=�Q���
��{I^rf���MwD���:������$�4�)!wo�(����z����m����k�������r���������i���W�X8��,M�	�����e4���#�Q7�S�iv����ig��,�	��Y�����"��L��@�������]En�a��!o���������x�g��YDz-s����r4�I����W`�!�'&^����Ja�,���+���
B��yRJ�GNY����Zf� ��#	�X+��"���.My��^�6���&���j.���X8.g��;�-�����7��������4)����^���T]��T���;n;�n�`���i���zc�C�:XB����y��u�I��%�A��r{
�6�Q5Q�M;6i��Zw}���aX�yY:�<2�}=��2i�!\\;�������9g�R������m��H$+|$,)�E�?�qR��`]�����k{(��K���D5!(����E��H&���u8��f?����Z��l��8�UB85��t�`���
s�q��������X4\���L��5�����G���Z��Me<�����`�����������8�n0���Bd��r��\�y?����C��o�r#���l�a�.�;F���_�=����4��������I���9��@W����m��r0y�r��K8����Q��z�ff��Yq��1s��oIxOq�V�|Z�%��@�!�~}����*X`�p��f����:��J;��f��w� �c��<S�������Ia����%�k�����&�u�����{=V-b�Y��S�����S������N>A���b����.������l\��4bK�ueq�%��Vu�iTm|E��Q0Q`8�I���U� �������|jM�@�lw���g
/�
�n��t���l+k=3�h)���B8G���ks���u$@������#�^��Y��@}������}��'�!)"�H�`�T�]��8�����J
��,U�����"�������^kQ�����p1��]�>e��f��qi+}�[�������t����%X�Q>��G5B�T��6������_��u_m��g7s�l�K|4���L��D��I�p6~��s�>���w�R%He9�_����>?�Y��e5Nx��)���s�������!@C�������B���\A�,��=C�c��"�������_6CY�}t�.�N*Y`����Xtm;����)��Q�4�h�KL�$q���T�f�
t
$��zP�g����H>��`��i�m?�+���XfbR�Y9O��u�g�%-��"�WH�#e[�����/�������,�H[E��v���z�l����c����c?�^t��=r���3��/�v t-Lu,E�;�������z%�����5������V�>6zh�����I�j p�?�����~.��|Z%�#D�X�)���G�~�*}?n��� �M��Mu�z��?*d����,����b���P?h�%[������@����1e/�$!he�x�O����cJJ�B	��5��g%r��&����)���yg*G%���}��A`T�m�vm5Z����&�Sv�QYd�-p	��P�!�����_�$��k�$3*���He�k��j�`u���	�
>A��+8��iQ���Q�tYF�.���8�O#���� ����MV|"�kq��9O�_��S�<h�����R�v��t9���
���QV��%�.H���r�����Z�C���{-X2l��Ed�������G'4��1�z��d�\E�a�m�0�����a>g�%)��O>��(E�,����m��4������A.�AD}�K8��/�X��p���!d<�M =AQd��E��i2!���������G�kaY������r�*�LW�:��t%x'F�k����zo��arA��<)�Z��UZ�>4��c1�>6M���`�N\wClk�� �C�]~|�'���L�H"<&�vk6�:���+��1@���(���F%[HqQ�"�y���{N9���7���o��z�hm~��z���k��^����/�3���.^����du.p���m�,,��k�}�}zd�y�� %�_����y4,�0G�.i� Tkd���l�z<��hzP����K�	�*���'���`:^��_����A�X��Ti�C�d�o��IE��q���$�}5�b5��`	T$>���p"�2_QXP�,h����i�2�Q�7��kw�:-��+}���`��j0�����W�������G�
endstream
endobj
59 0 obj
2377
endobj
57 0 obj
<<
/Type /Page
/Parent 46 0 R
/Resources <<
/Font 60 0 R
/ProcSet 2 0 R
>>
/Contents 58 0 R
>>
endobj
60 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F4 17 0 R
/F6 27 0 R
>>
endobj
62 0 obj
<<
/Length 63 0 R
/Filter /FlateDecode 
>>
stream
H��WYo�H���, g,����d����� ;0/��:�H��mav��o��K�(�IY]]��W���Y`�g3���;3���k8��6 �w+|��\���;��9�?�����]���9���5��g�u��&�#u��b��d74�yX�t
oY��,�������aXB�!�N��U&��/�W��F�,���>-7�d��p����W�:�(G�����dS�0����mu����<Pqym5�5��E��V�TKq_=��
1�T�a���m����gJj�L	Q<@#
W�9N���1�����	D9
K
�wBa�IA�V4�(���k�_@����8���W�)�����:�"�.u,��!�^]*2���L��d��U��S��+?��G������
���~Z�����6�^�-����"�+�-`%��r������8c��&���P�E�>����(�XZ������]�aR����]�k��l��,��U*k����"t*Md�� P�w������d<�6a	u**���BLQ�Y�?����dXM����^�}8��a�|��+�&@s�r��SYwV�&��{������mgJ���o����8�����i���FU^`�
�����K��`�{f����%�+�"�d���bnJ]�;Z��(0-%����4*hB�V���l�G0���3�J���.�-2kZB�
0!����5s�~q�l6�������lGS��+�GFc�`/<H�g�C5b���QnY(��$����F�������c����>���,�9��pg8��g��M�}��h*2QN���N���P�tE�h�k�4^7�]�����E|�8�a�����
tU\��a�g-
���F���Gk��������������)	u*L� ��?�>T�+�Z�L��M�o�IC3�L�=�o��_��h�B����?�Wg\��:���zw.,�R!j��w`\,�����c|��r�������B���5���Vc�}P�5����qH.�����f��dc�(�d����$��������y������t�d[�����9�7L�k��_	?���3���}4���vg���L�s�!Bl�[l�`'p�����x8��|����@h��;!t%��!��Z\E���6��q�5�����c��T�(���HK��g�!KaK�Y��b�t����U�t�8��S$��^�k��#7BF/����������qK�++2��r3fE��Wfy����T]����
��r<��(&��tq>���,qSeY����B��0��y�������S��
��D�#�%�,q��t��U���(L!��e<B8A����������UI�s\H�F|��y�B�����/��-Q_��r,��T���G ��
�|��MV�39X��;���m2���;�n'���q=m��
'�"��-�5�`l�_X��������%��
�"��@e^��#�b��@�����oMS*��A���Q	�x���}	wY��x!`��iP�&D�x)��0�E��1b,�j�'���V9����
��:��W�]j}�l�i�r�k_+�"����u�O�X�����r]gI�L1��CP$��a�r�����T�s4��4��z�Sw����OE%�V�d",]P����T�%���B�>JXkDw����@��D��pK+�t������VI�v�Hs�}
�t������PIcZ6��5�0
�+�,����%��s�P������n�@�Mk:������;.����A;�G=�7u3��NC�.������K�����]U�>��B��9�dfw*t���i;�����@���n2I�cxe	���5x�I
��?\-�����-�����Z����Ln���3�9Q�����"-���,��L�5���G<��f"�%u�3�	��A�����i��Q���a����!�����

Paq����kI�'LEp*��	�����wo�}$���
G7h��/��7@.f���d�1�_�cKC��I��w����~�S����@�`O�5����O�z$8�u��|��|6Y��W����_�T{�wO���x����i�������U��=j}iw=�vj��V�y��l������2[7��G���>vq4��3��~L��T;Zk�1�+��$��Pe��?P������TU�����w���'�\�BH)��V����I}&-T�z�+���W�R�x���k��Z�b��!N������)'�f�����w�p��i��q��Y���c���#^�e��d�M�����1���+q.�g�!��[Y�y����K������q�1����s�0|I�[��.�}����!B/[�9;�!L2����\�:^�k0}_�U��MOZ�5���l��J��6��	���`��!���?5�XD:���?�#\i��cw��(����A�+���|����[F1��0�P��t����Wz�4-2�Y�>�����:������?I�r����|�"��w�(r9-�<��J�Z#�z#D{fb+�Rz�J� E�k��%+�(��
4ZM��Q/�#���J��Y��@�ih[;�=[nP�~���
+��6tK/ �(��j���2h�p�|;�l�������<��$~M�	�����*�hQ��$�CU`�h/�4�C�k
?����K�j�����hl4>�d��G�~K�h�
�/?@��2,��f��"��%M�����j����&A��'���
���{��?�HE�����4h������Iy���p���&��#5�������	�-� ��$�O���c��s�����'
�A���w�g/��$vh����:�ji"���h�YN�1oY]������")�a��I����I)|u>�����ZF������^��������7
	�O�T�Z�BR�J����;7�O��_.��e
endstream
endobj
63 0 obj
2937
endobj
61 0 obj
<<
/Type /Page
/Parent 46 0 R
/Resources <<
/Font 64 0 R
/ProcSet 2 0 R
>>
/Contents 62 0 R
>>
endobj
64 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F5 21 0 R
/F6 27 0 R
/F7 33 0 R
/F8 42 0 R
>>
endobj
66 0 obj
<<
/Length 67 0 R
/Filter /FlateDecode 
>>
stream
H��W����������ElcDu7�y��O�$A���`GjI�)RKR����QM��8^g7^�M���������EJ �$`����_����5���w3��3?�9���O�F���t"vezM���FB8������6�}.�F�����\�U^yUjY�D�0j	����#a�M9w�id��$gU=���e��m>�7Y��g�l�|w��5T���t�c
�$&�
a����J����@��:����J��W��;�l���
��T�`q���2�Q :}(s��#4H�41���G�%�#������9��)��>��V��sFC�|�|��X��A�Y�B�2���Z:����7E>C}�����`V�M�hbUVm�T$zX��A`x��6���}���������Z��H�?m�0 �%6�4%�C���uV>�qU[���@r-���nY�9K��@,��(���VU#aS=��Yn�J�� ����j�B-7��J��<����|��6o1�a[�y�du�Jt,���y���&�e���G������-dV���
4�"H�M-�x_H���AE�3o{q�Ra�$�Z�@�B�}��&b���S6+���f���2�
�,�>m����H���Z�Q�kd���]S�TA������i�v^Y&�K������!'�=B\
�Q�j
��e+K���J�������	�@]����l�g�l�E{z�bQl��O|d���vU���3�k�B�ja�3�����u
���L�4\p������V����h����r���������%��<���yY����`��������R�vNC~\����*�!/Q�"g�A��i@i��
GX�*��P��y�n��l��i��F�r*���6�-����;��b�o������i��KT�]e���i[Yj������8�+����a���v�CR��o�h�A�.���`id��>Q�\6�:�G�W�����m(�!RM+���aL�[:�\x�}|����>Pv����K�S��	5���32�>��(����[�x
?eV�t���r��3�2��`.g�\��F�R���J	��-�:��*���ow��u�<����0M,N���6�4����pt���n�j��k��WJ�c���K��l����h���:
N��v:�$�,v��d���r8 -���$�.�����I��.m��qwp���r��1�5u��=-�A��x9}��%�����������9����9��#
o:�H7�Y�I�z�dP��y���� :Z�I�����q��jh����G)N���y��"���,��������(���<�P�$:���l:��x/�����B&}R�`�#2�������;��H������a�s�EQ?gE�?��Sc���0<'O���i���(/4��1D�'2�lt�`���Twd�+�^��BO;�s�|��|LE.��
G�:����o�M��@�q�q��%��T�"=�89�'c�t��S�'-�[���������.�������ahy��5�c�Ii�Hi'�OG���������xO7���e{=��U ��` w��1������K��K�����K=�\E�+WlDW��i�=�\���d��&��K'���N4�G6���N�{@���5N���j*�Sn	��77��==�b|����Fvt�y%KhW����Q2|�����i7�k��Tuz��k^�?�p[��j-�|v���R�y��mg4�{����'���V�Ve��VB��J�K���cK��G "�XCo�jYgkx�<}��(_|;[�*��~|P����h�k�l���E]���"�EQ=��2��v��'���O��Y�I,,�7��6-�#,��A�J���
�i��^�h�[��C^��c���5*�/LC���'�Ve�4�X?,<z{G#HM~�fIjC)��S����+�xHtae����BJ�\P����\��H��P��C��B�Sk��}B�����{K���������� ���������G0T]q���	g�.���:(�S����������$Nm$u����LH(g�+��a2��O�#_��a��c�v������r�=h���1�4���k���V����	���3�Orx���3�lV?�bB�R�P=6N�YT�Vy�^��^��=k��21��H�������?��!������L���[����++�Eu�k=F��*��.���H	w�}!!�f�eU��KVlec.�x��[!����Hj�������#t����D/sH��(AW�a%��wo���[-����_���H��@�����0�G�1���zS�y���1/��Y6E��~�����t%v8-�^�g���+��{���`g�<�~��:���!��s�
�2���VWP�m�*��25���M�����l��nuO�����\���fq��@WT��07M��`���k�Q�}���0�i-�Gl�f���;����6�J���	b����k;�i��I|�������9aD
���z�x�Q�U������W6���*<(P'�w`+��o�J}���/�;��5a��#��0��%x���k��_A�+_c����^�w�bg��Wc���r��_����M���@�0gSLR�������������	Q���p ]A�#��
vC�$H�p-NUN�(JLOU/F)��n� ��tfh�#�'w�`��N441DJA���9�c4�`:!%8P��R!���hl�B�� ��2� ��
H��(�
#0��`Mtad�������AyfIF~i�(P�r���E�I�5:�\�ByFfN*<��l���;�����~&��L�,NN,�`sr@NAo]k��[��Y��E�%�Ey���F�q��4,����K-Gj�"W0�-�j��_��f j���P��g���e��rA%������ �
endstream
endobj
67 0 obj
2941
endobj
65 0 obj
<<
/Type /Page
/Parent 46 0 R
/Resources <<
/Font 72 0 R
/ProcSet 2 0 R
>>
/Contents 66 0 R
>>
endobj
72 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F5 21 0 R
/F6 27 0 R
/F7 33 0 R
/F8 42 0 R
/F9 68 0 R
/F10 70 0 R
>>
endobj
75 0 obj
<<
/Length 76 0 R
/Filter /FlateDecode 
>>
stream
H��W�r�8��*���(�jw&�v�LM�t��7U�h	�hS�BRv<]��s/�����j'qH�������I���ul�2}��S�����C"�v��tN�����#��)���0�\{_�lO�\Rs����.
<%��M2�O���"E\%�-y��i�gRu�Z�c;�uv,��z���Q�#_�� �y� WOY�U2'��*��KQ*�4�M��(���t��q�0-��������?"&���ZC��K��Vq�w����F�2�+2�o���]?$>8�i{\B���C������:���0}��E��x*(=�1�����Q�w�iX����fJ#����Q�N��$�$�!D�.�C�oJR�y�B�qF���\��i"����2��y�Z�!���	���o�xEf����:�>��Lf���q!y�#�s�^P(>�(��m@)�<�JQ�E���#��v�B�����j��jI
���RV ���-���>�<*�2��w0�n����J���\>���"����J��(���<�d�X,�,N�*�2���4��x�0/	=��uF�#�_/���X���;;����J�+$��cf�L>"�����"�W�/������6I!�k�Y���b�T����WA�I�g�����>Q���5���@�������6�� {:1�o�
�/ H�&
�M�|�`��7�H&�2�x���Ba�:�$��1_n��]�������![�-
�;�
���k��E^�d�E�&��|�8��yo|�����fe��	�2m�~{b�U�b8�"�y��5:�2n�`������LO(�?8�h(�����\������+��`�v���;�G��_�Is�aS�W����y�Wv�#�/��/��*��w�����u�uk�C��8�_z�5Fk<9<�U3�mf��f�5�5=T����S<���Z�q`�+����An���Z�y*����T���2{=�+p����L[oh������%9�9�q�cc5<~��T|hd�����?�:����>���$*V��::��3�� x���6I����xj��,&�6�e�[��X��:X��8E4����U�&U�v�"���1n�^��������1��b}�v�
<�qY�a�q������&�`�^	:���rNev���\�h�g���j�t��*|��(;�r��f9m�)����
<�k�d�P�a���v��e��}���6c5qE�;���`-p�����6�-�EN�i�-��bm~�4c?k[vL�|����������"!wV���A5	t�Ejx��b�����D���g������+4���IZ��d�j�����?Jd{�����CMG=��L���M��X�������i��gH���d��8Y�$���+�a�#x��������]�P5JXu.W����G�~e)��DMh��?s���N�A�Bde�6e��5����i�?���n��H��`?�t7������8�����wv.S�3%��U	��D�<^@��N������q� d�J��� �\�:�x�;^����t6Q�8��c�s�SaY�!�����5 V��G�]���]F;(A2_�9�)_�}�R�����m�CDPt
4�E��p�F���em�i�������~�GP�:�F�xv��80U��MZkd���.��2Y���S6���n��{�k����M0���Y������b^8�w�h�g��e/������=q�������m\���}�`^��P��Uf_2"w
_����v�o��b����UE����H��� �7����b/�u���er�Z����\��YnU�Nr/���@P���0���`	���?��<<������(g�M��@�Y�I�)�v6/�<�a�N�����(�~�W�f����~-^|x}�<*u��F�-����Q]Fr��<QsM�%����6_p�HM.�m��E������vs����2R�3��i.���$���oDDz�W�8fV��� 	��'xy����t�j)H��_�sQ�`Ty��K`�?8�����-�����C�n���ckvj��6��"���~+2Q�U^�.!�?mD6��@�7�9�9�L6�cW);>����D����zl�,��X�3��&��p��9�?�L���r���p������\���5���,B/
�������64SJ1*�}�^�,z���z�$Y)�T'Y���
9M���f���qC�5sy�d1�����p~����x�����e��3�\$gt� @^��89�K�N2��z&�=hl��K�_��;51�����W���G��A8����F���o@-(4��e���'���z~�{9�v��6Q@�w��r��E�����p ������W�;_"�����?d�EU�$N?�y��Y��&�\Lw
��c��L.9��y�����`�G�@j����Tv���������C�����#��t�P�cE"���!����C�,������r@��W#�K�~�`c`��_���s��� ��<
|�u��_���@�g�,7�����@Iar�[�7���8j�8[�+��HD)y!dS����g�u�V�1\?
|��=H�OM����UQ�	���}�)����TD��M�H�,}"���tU#������1��)vzsz�T��#�n�b���>o*hr���G�����rc���������zSg�Jm��/�p�H�\����V�����T�8�U��l�)����%2?J��j����a z���bU���>7�M��%��Ro�x<Nl�i��������{o�/|r�wMs� �������5��Lop��D����$5���63a�uDO�j��1wX(���.4M@���������o&&%��e$��4�d2��4(��W�SM��N�����.(?vJ�G��r���*��ON���\��G�VA4P9
3S����aa�,�
[�`|�_ ��=_��\��`L1�S��}�c*s#^�Y��q��.�#���r1;��?�!��/��z<��!q]����:�0w<��A6�� �U�2d6l��q�tJj���TS 4��_��+v�'�.��?>�,��%����E8%��,�8�!,a�������Y���
endstream
endobj
76 0 obj
3137
endobj
73 0 obj
<<
/Type /Page
/Parent 74 0 R
/Resources <<
/Font 77 0 R
/ProcSet 2 0 R
>>
/Contents 75 0 R
>>
endobj
77 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F4 17 0 R
/F5 21 0 R
/F6 27 0 R
/F7 33 0 R
/F8 42 0 R
>>
endobj
79 0 obj
<<
/Length 80 0 R
/Filter /FlateDecode 
>>
stream
H��Wk������R���R�^���
�P��f���M���%G�q\���$/iJ�f&	R82�q�9���y�S�r�1!7o_P��5[���;Jr���o�fE�jO�#Y�����E��~{�>�A�_^�c�t��8�Wp���������D~��lDWT[��(����]�&�2P��9��["�3��>����/���5�x�����y+:q'Z���Y�L}�Q���K���-S�%!������2�����E��.���2����!���R����]�
��fl��'I�����Bkx��\d�b,�&Gu��f'+"6�lHQ��������"l����(�V3�����w�9`���J~��xe/o�/��4���M�|���YO�z��G�W\�����������������+�9�����%�\3���,���S�R��q�w}S��NA#I�B�����#l�kr��?!+p~\���V��5��n���z��������5id��P���]_�8�]"�Z)V;/��Am�	(�C����T�ai��(��q�=0��7J��r��/Z�j���S���w����AD�l��,D|���@n����wM������������R��v���hx���w���j�s�D�<�1s����\�%���
�1�;'���}��x��~%	3�&3��p�d~����t�X��r1K0C��c)[�@��Y~~U�Kr����l��P~|b��x����N����8t� ����|c�7�dU�b�+��+���Y���7���TK����6f��q,��E���E������N:�I����9s2��S_4�4���B[i�����G
���?A!���a��a�����#�������`+�{Yu�*��g��N�8�p�<��_2"^��Q��?�;}�f���E�-��s���qi��*Q(
Me�;��e�E~���$ �fg����2�1S�Kc�N�:�[��S��U�Ce�-�;Q�4�dX�����Q���I���,%��lX����l8Y�L4?�����-�]�H]��,������9�l��2����a)p��??JR��B?�e���F��*�"�S/�������-� ��!�B�[S�o5��lAn!��p�.���X��c8����m�Rb�<U�����uz_��a����R�h��2�����,�����[�f�u�*���Q������i��.6h�K�4]��x��ukD�����5T�
@�l�*7����7\�S����	Y���W����t��]�$�F�����59���K��t��d�[4haS\R�f�9I�i1Q�YbIP�>\rm�e��x���s��0��B�jW����=�`m���_mL��d�4�g��Q>,����q�Y*S��;B[b��������p���54���!����x3.�������+�R}.N�|���h>���0��[,�	p7���m	e)M�^������a�cj@\��Bt�Wvi�����������6E[�L3��/m�8v��8������	=�"�m��f��y^���k_���t�eWN�8�tI��+3��A�C
��>q�sX�
���%�f:jz�i7B���*dnl�&S�����gbu%�vx���@��v%�5P���^��2�����1v��)Z��r�VreaG�N�x0��y%Nb�Q4AK�}�����mI�������XM��ga�
�rq�/��^�����������#g&�����@��<��bTa���#����.��f�"�����16�M�
=������������Q:��a�||���O�R�F)�h�b[��o����
>+-�h�_X(������2�+F���x�v�%��k����P����p���?S������.�����#Q4Tg�J����C�9��7��>\'��K�C~��=���z�[���e���%���s���Yi���0F���6�K�*�J�&�,������Y:����L/�5��O����V�	q�H�m�`��Q������F	�?�ds�iO�}v�z�$�3[���K���c��	��rA�S,��h�}.:��5���Z�B���+uR�����>�y�/��sg(�zv�vWh��{�f�)m�������=��4q&��)����g#���f��.Y�'�}l���r�X�\!�Ku������r�G��fDn���0c��� B_���a�T���[`�������
��x��0�u��T��R��j
nM
;�f�$-���V������K��vgJ2�/o"b�XB�4�@�^W�����x(�R�i��!�X����$�#4�4'����o������������l~�.G!F�G�oe9�E��s����������FN|f4x>����q87�}�|^O0j	����]��v��;����.b�5��������,��x���8���+U�T�D�"�D@�a�������w���i	����U��e����4�Y�)c������a�l�eQ����NO!<e��8��0��
9�e��<t��F��3�����2����W-��\Q���{qR����v��'��������e9�9��
+��V�P:k����Pk�,����}��*gSu2���K��l����S��K��J5k��,I�7�^L��M
������pa��s&.WT�be��}��������1��$k~�/,������t��r���O�4r�^-�|��Qu����������d#��-DD�������+�k�Zz���/�?D=��}��=R�C�P��7q�������xg�]�Fz�����o�{���������*��i�:��ec�]�W�����?m��7�c��!0��H�h=?�����G��k!�
���{&.G�t�Tg������m|�zq��r�6����{]�K������w��������GE���#4����"5]���W*##~�%^'f���e�1�s�k
b�Ss��0Yp��%T5H��iK�h����!z6e�e���Q�D�x��QP�Z�3"����=�������2�D��S_��[����R<�����@fC!B[�y���[���C����*�h���\���[��MF���;v(�j�.��F#���P2_�bZ���`�dh�����(�F��1���]��w��R�"Wz�����z��> �7������mWP����<���g�0���������k�",����kD�R�t*��$S�6��6��*[
�R�����?��P��|7e��������=e��h{=\������]J��^xd:��{�?��3{d}�7V���:7���l���M�&T���-�#��@���ix���N�����a��]�T�Ww*���?��:
��V�L���c�a��������W��
endstream
endobj
80 0 obj
3420
endobj
78 0 obj
<<
/Type /Page
/Parent 74 0 R
/Resources <<
/Font 83 0 R
/ProcSet 2 0 R
>>
/Contents 79 0 R
>>
endobj
83 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F5 21 0 R
/F6 27 0 R
/F11 81 0 R
>>
endobj
85 0 obj
<<
/Length 86 0 R
/Filter /FlateDecode 
>>
stream
H��WY��6���9%�8x:)qvj�;��<eR)��4��P�����@�cd�V�W
��>�������*�$	i�#B�o�(1��[r��-%|��7�,����p| 3��,?]q����CD������XE^D��e�����|$��Z�y��-y��R5���h<���J��va���g�^��b��AM�&��Nv� o�._�Zj�������t&�.��B�H(wB8����
$���y$��[��:@)��)�GZ9�i�e|M�1*,��2�����E�$"gOHw�����(f�.	�a������45�������:?��O��+�P8�p3bF�"L���eI �d�6�6�4�
��-��?KR����%d��l�wDU�R�+o��-X�F%�Jc7�"A5��\�5�AT
�T����^>H"���
���y/P�w�He4sRY��E�@t���4��ya>��]G�M'[�����^7k���:���0>��0�Y���x�{]+%8D�����5h7!�dq��JW��O����(����w�P%�Z��y7��
���X��{Yw*/_yd"k���.��pT�bR_<N�����=J������N:UI0S����y?r�%	2
2n���n:��yJd����&J�^�2 �]���G
�[�|��v�������SW�AG�M���;�9r�1h`���a��uA�o9���x�YD3�sA�Q�����0���j�J�A�lH�w�Rbk]se���z�<cl_���
�]�v&A!�z
I��?U�'����T]m	���;��F�R���7�s
?��d�h8�kx�!2�Ytl"qA$g����t
�I��'�|+��/������8u�4A�C��@@m�o@�&/:S�� 5���+L�;�=$�Q��u�7��.$:���j���Z��9i����du���Qum~�W���<�
��H
��������P���&,�����5B�����g2_5����$�!l�:�,�&��#�����	gA�G��jtg�t�Ec���E�G[/`��fNTgt`e����Q��TL&�5������I{~���d��}5&�S��%��p���$0��.2�[<LU�:
%m��5�r�?Y5�#��)C���/x5�S����T��������Q�����'�H�c���$��"g/_�����K����>L����E2����G�4�NRQrI�����~��H�{%	.xu�8%���)��;F����ol:�D�D�}](��A4��%�5����y�����.���{�����"G8�e�Wvf��9�8�����*��A�6��/�*���z-���X�{J�
�8������Y#�3������ZC�5>N���v_�����zVq�`����y ����Ms������q��d����\\0�i7�V�)O����^D�������<���2?:���_���PR�����G�!��#�r�j�����Z��8x$����1���a�_�S d����UksX���eb���b�tGM��>��A�k_+h9���(������|w����W����8�{��~rX}d����$�Qm��G��N"�q�)�����;BV]��*w���{���_V�����|h�n�3|��\DL�-N���I�����V[L�NnM���S�e9w�$.�-�6qd��f����+�}��Z�Bz�P��.
��a��6������<��D�o���pL��v"0p��ZU��#����l�;�8���A)�>����b��b��PkK[	���%J�:���_�W~o�<_.��h����
���7�6�(�is|z����s���>9p�3,�d~�pG;nh���Z�\�����8qL���O���-�������an4�8��sx�����*��5��pt����)����S
���P�8��i$M|X��6����y	�B+��5��zt�~�\W�/�_V����]60[2}��<��d��D�x�c��>��{���������_������O6����p���Y�����D&����V�d�����^Lk�T�(o%#�r�$�dZ/��GN��;PE�L��e�F���>�3
��7��E�G�q��*`1\U���Z<��j��=.U!F���)�ITX^=*�0��� �4)��2S�����TZ<i�9	�h��s�Zi�TaP-�m���\��L��������i��A����N��A���i
��8��g:.Z���/��d���rW�le-[��m�����%N�%TD���r
{��D]���w?rS�ngW���v��=U.��3�D�2�l�a��U��	�8U�M��#2����������usb��,�>�x2x���*Kx�m�j��}a�h�����x#%<�e��n�� ��r���j��m�]X$|�`�}MN���?X��hF�����/����l�e!4
s0Ur�jy�E�Q7�\
~�����)^�D�L�R�um��.Ci��KRx)<'���+��s�K��}%1mq-�1��/������)iX[!{;e��W
t&��@5���	��w-��{6!��]�"6��`\�q�X\�.�507R
��)�;��B���[����YR���bk;����U�M]`���R
endstream
endobj
86 0 obj
2658
endobj
84 0 obj
<<
/Type /Page
/Parent 74 0 R
/Resources <<
/Font 87 0 R
/ProcSet 2 0 R
>>
/Contents 85 0 R
>>
endobj
87 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F4 17 0 R
/F5 21 0 R
/F6 27 0 R
/F7 33 0 R
/F11 81 0 R
>>
endobj
89 0 obj
<<
/Length 90 0 R
/Filter /FlateDecode 
>>
stream
H��W�n������F����2����	$;����,�n�j�+��c �"Y��n���
fh5/U��N��xy�S�J�����O'���-99�LI�n��,�����|$���\��p�����Am~9����
K��WH�*w�����o��?t��b��[��n��k�]�&�2��T��;��[D��I�y��|�e�W����z�K����b�������W5�D��ej![���%�����	��2�������5@)����W9�)�2��j�
SvW�a�����IFp@{�'J��SW����eYp��C�G�\�E�O�r��=>x�'Rf������^;)uo�]?���f����H����%��|�EE I��I�hR���������<���.m�v��n;R���uEnj�T�����"��d��`����c�������/��{��o�����6���%��~������\k��=���(��M1�B:�
�$���������7���$��g_:���x���{��(K�;�f	�����r?��N���TXy������1���^��V�������t�%���+|��jb���������jP����Y���+��)�WG?���&�M���
�/Q�9D��8�*�������DL����-��?x�������y����r�E�5�SB�w�����eK���F���
 �� ���s?uJ-���$M34��nU��e�������_�3�j��h� ���{�.{��x���,�]La��7���4���w�gA��q���+����|.8�����TLo�1�X����q[������Q������1;M�'��AQ��(\)#\������kA�8�6j����D���Q�h�7�DDhN��]N�
��k'����n��o�#d���$��d^8���<Z���y�������[<���3��d���\t+WV`��jb������|z&<NO��@��"������C�IF��-z��b��ot����#y�l2U�w��(�D���L��4��rg<���|�3���|�?r'Q�qi���]�?�a����y��h��^�%}���d	"9��d���<�>
2�s:)1����P����W}u�������,y��&AZE���@����K������[����	_|x�5����a���~�-<����F8Ai����'�<+�49j7��q�m!MH�W�_����~������u���]��}_��]o<&2!��8c1�V�$8�_��SC�����~������{�"��xH�V���0�{HP�#�Y������(��C������� ��)L�V�K�c9k!C.V�3%�n.�yh
&V���/�Q'fUT�"�x�B6;k��es��@<�cP���jFjZ�$�!X�[0��u���~���5���G�$���	)"�U�(�LbJY��.z
�
�;���I�=P��Lz78����O@��ya���m7�'����X-e9����%�d%�!�PU�>��U�J���g�	s7����W�u���-2���2��%�3�����zU���%�*����*����pa���~M\�c��*L�(�������;�:���pi�>f�����*T��|���t.0��5��~�m�@F;����&�/��UC����%��\�(]��I�?f��y�S����l�<�m������}Q���W�k���W�J�Z���F!��$����g�w�01��v\���W���&jo[^��)M�?'��B%f B����r;,`�1�b.\MD���3�+��"��<n����2��dfK"�����3xI����M�?�@�	��O��0g����F3�+/(�Q�~��{ {�����O7�<�w��xu�p��|�1)�8�r����,N���i�D���KZ��R���I(rK��LU�/�/P,����,���/�t�7O����%�i�
 '\������y��c���o)6�;�/����������p4�`���1{��<t3���������t�Yr�Du9���8�MD
-a�d~���/����_��LV�NB?�,����'?�<+�%_���k�W��_��: $.�e����|�x�U�6�e!��"6Wg?���O����/
!C�<��4�Vc&�S��Y���i����+���\����'��Z���9u��ty��5bq�@�o+4uP�
�)7�\%��m+F�4��"����x�K�N��x���v�O$nh�����QeD�c�v�+�I����u�����t��An����= ����qk�=>�v���������H�_p	Lt����.K"�u��YM�^x������e������Q4�0s���k��Al��j�����ia��
endstream
endobj
90 0 obj
2392
endobj
88 0 obj
<<
/Type /Page
/Parent 74 0 R
/Resources <<
/Font 91 0 R
/ProcSet 2 0 R
>>
/Contents 89 0 R
>>
endobj
91 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F5 21 0 R
/F6 27 0 R
/F11 81 0 R
>>
endobj
93 0 obj
<<
/Length 94 0 R
/Filter /FlateDecode 
>>
stream
H��Wmo�����+P(�$�\����\��>�E?��`E�%�������wf_(R/�;��6j���yf��g~�<�,\k�x��O,��b'�o,�p���2��q�qx�7'��}��Y���^9}c�����y��<u��u��kx+2Q�*�f�K��I���l�o�����	�zw$���k�u�,��������j.�$����^�R�o����:OGn !��q K��O�;4~�q��m�6P2Z���s��qjv����F������b���e��!���l �<
m�y�fi>�)��"��T�/�b
C������q��wCX�?��ZY�K7�t!�v�Y���%�OuZ�{������m���=����g���\@RB!s���(`��*���e~���j�+�����]�L��~;�C�5�$��O��O�.�i��I��� ���5dW����>���0�����!�?���^5�9�	z�
��`.��=�t��j/n5,���(���P5�[� ����{��C~
2k�O`�y�h�n�!� D������(FQ�K~��=(�VyqK�ae�w_d�[�	��O"��c
1K2��8���>S�u^�N/h����������,��>��*���i<��:�4`��[�����_�<O��jI���Z������X���r�8[-��-�����G��U�n�����.��B]�g�z~*x6�?5F@2���np:w�#���{�_�S�/�P b���b|�G:����B\�v��	6�������"r��t?�)���{NofE;S��M,��oaU^�H�l&kR��*���c�����i�;���^�����t�����AB����H�3H^��z������K����AV/&����_/�}�0�+�����L��A3����� ��J^�����k�sj���������E3x�8�=S������@�HJS��<g���!b�{������������\ ��8�4U���f�B�CA!��Xk'X�S�#�������"����������i]06���Fo���_@�������o�C������X����J���(����+
���P�Es'�Rhg�+v�o�����
����J����(hr�7���dRW6%��U�����X�A u�?��W2,w�t�z���N���w-7�6*t�	�D�a��� ~�I�H���
�$$��3��q���{T���r������$��O����w!���C���h����-���{x}fUU-�
�5m��%���[c)�d�C�8��GN(�	��RO6��@���	���0R�C��G]blg"�^e%�P��:�B�o�>F0�����m,��������0=
�6Lp�s�-�)?�,��Icb�����6
[����7��ei�,��9���?�T%���x���v��v�NU�F�m��{��H��b�,Q����R�����D��������b*V�g��-��c�Z{lY��%x�fD�T|��$	e���f;��`�����N;TW
�4�]
2�`Z�b�f�9�>YB ��	�%�$���&J�������H��#���b�U�I����`�ke����BQ��W����No;d�5���,i�}�j����m+/I��)�U��3w��e���c���	�0���Zc�3L:
~���^��dZ#����@i_�I�1G�������Y��W����!#���T���'�R�^6+D�Ym,[�c����H��2d�r[�<e���+���.����X!��cD�Ap����)wd�5����u'��aY��W)e!y�����x���S�H�<�8���lI�>�|�k��_����K��P�V�=��1�wyyd�E������k%t#�!xJ����J��,�XS���f��:�o���<J�i�i��H�v��D�Te=�Yr���Sd��k!�����q���#���	��7M�5�p+��hg�;��������e+�*�o7��dV�M�������2�U?���635�B2�e�����0��BW�~,��@=J�����T���-::�0r\wW�[��t?��������~��;Q���5-��F���F���!~���`|����m�<���V����0u��]jK�Q�-�cYas�z�;���:6�:�O���1��>5�]3EF[�V�V*��oZ�zD��!];T�>�������b�T����~�����U$hP�VD���x�za5G���q7�c�u:Iq~o����wM�H��M/ha"�D�agD�����`�P�e'E0E]4T�h�y[��i�C��Z������J�2/�����g_������8G�Y
��L]PG����"���s��E��N+���"��3��{L���H7���#k�a�\���P=�	1O�*��N�z'6����4�~/��=f����-h��#��N�uWh������h/�-4H:�Mg�-�%�E:����W+;2���V���3�-�`����������[��j�4�5��%��r�c��It*Q��r7�������E���ELU:��0��yI��n:�t�r��i�F�{<�e��U�N�`���A���c8+���,�T��|������(_+���T7��������hR`�m�?b{�c�t��.������W��U�������:��_��N�p|��g�
���p�NO�Y8/��*3c�v�>|�3Y�I��dT�����`a���Q��@�d�|	������Z(�2M4��h)?����u�?�<�y ~�>�������D�����.�����
�����O0HER+������^���T�����q�kh��1>b�
u�9�U��k�Xx	�Y����u����@�$�WS?#/~.�=��
+��i���T����3������ng
�����u��iCx�k\*��A���Pnw��U`�VM3e��)��T�
endstream
endobj
94 0 obj
2987
endobj
92 0 obj
<<
/Type /Page
/Parent 74 0 R
/Resources <<
/Font 95 0 R
/ProcSet 2 0 R
>>
/Contents 93 0 R
>>
endobj
95 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F5 21 0 R
/F6 27 0 R
/F7 33 0 R
/F8 42 0 R
/F11 81 0 R
>>
endobj
97 0 obj
<<
/Length 98 0 R
/Filter /FlateDecode 
>>
stream
H��W[o�X�����K �Y6����Z5�F�je�!qbl�I���97��6nB���R1���|3���������
t�����_rG������������w��c��9��
����k`�7��D�����8���w�n���&^DW�&� ��,���0m�i����}~�(~&��C[��@gq����(��Y0��^�M���B>q
S�:��)y�7Y��t)D����6?B	�U~��8=7�G(
�����V�T�b�:�&*����
���-}�v��ma�0���rSs]!u`[�/^^SX&�4�R��������������&~���d��DM/b%Q����C0�+H;�����#`�� �[�nK�b��@l���j���J�|fJ(\qZ)���lW��8�N�N���c�8����\��:����eh�#�%�0�Y]%9�0����7�v�����k��X��H�#^�t�~L� Q	j��'n���.J�aqkD^�	]&4E��B�(�D�J#�&�<��2s�+O��`��gl�/�4Ia����OW4|�*�Q���q*��eA
Na�ViY�y�$k�����T��
�ne���#�Y�����2�h���t���B_�ri�:U5���������ve�tWWf��a�aJE�D����.U�����3
����G�F�'���d-�
e;���aM���aS69<lmV$)�/>���_�����?C��M�&q�Pclu������.��a�����h������+����:5���~��Xj�(D���b�?�!���Y8Ak4�'X	�2O�\z�q@R�k7*����2va���w!P8�e|7��'<K&�#a���k#v�����Z�z!w���������R\�_b��Q9�OO����2Z�Vq���r��;-��L�
&��

�ci�"�`3��NXKb��5���7��9���'��`>6�6�t��W���E)���M���.p�'���/����3�Gz���o����Y~H&��.��arz�lP��a�+���1U������v]���\��'�D�T�9d�yRm�u�*U@���d|�$�D�I��I$�*9��h}��rt9����5�j>�U��"(9'~a�V���UO��8�d�x$]�_�/IQuN��p��p����t�9��7r�n��>;P��G�q�6p��h�%��K��^�q�d�/�c�8�����%�����f��3NOk&5�kYo�]�D�}��f�4�R�|	��Mbh��9l��JQD��eY����lNsNoSPC�hcz���L���m��&t�@��?D�g������;�L?2�����F������"^Q��ni�4H�������s��O�7�������nN��aVi�6�h�*Pb��y���Y�y�&������a���7��@�4�X[r��������8�#S�=�m����9V�%z�Q�P�����eN��A����95��|p�a��\�����{=�H��YV+�`
�7�f�#DK�w��#RM��]3�A0GP������@�����F�*:�����U��w%�V3��W#\**���<�� m����Z�)����l[��pAI��<�g���Ns]�XM?
��!Z�!]�<T19�w�u���&+���mO.��v�����:����>:x�x�j���hFK=J���,��0�� �����/7����]>����^c4;��v��Y�I
����q�Fb�lf���
{h^����r�AcH'��r+� 6k�s�Tc>�L1�>�	�a��^?�pN�����2s�$_�w�#���YB3F~Wl�[�
�z���;�h���X�;TH�<a�J�UN�m��Gs������:��*�P�m|��Y������ ~����>�t�h��n�\�������O:j��3������w�G�J��q
endstream
endobj
98 0 obj
1951
endobj
96 0 obj
<<
/Type /Page
/Parent 74 0 R
/Resources <<
/Font 99 0 R
/ProcSet 2 0 R
>>
/Contents 97 0 R
>>
endobj
99 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F5 21 0 R
/F6 27 0 R
/F7 33 0 R
>>
endobj
102 0 obj
<<
/Length 103 0 R
/Filter /FlateDecode 
>>
stream
H����r�����0�r�p�30�*Y����*7�YS+�	-�H���>�zN������$Q����������N4����Ms��J4���O���w����c�57��u?F��l^5?47��R�5���`�;�-��IV�����0��4�������_�������������������K
7L&vBh�A$�5
��[�t����������������y����:<���R}���;�J����]{��$��vT���8�\g1��wz:�R��y7��*~�aT_�����@�:nW�������X�������iR�1�n�����=5��6����S����c�������������s����y�<�?7���������h84O_����������K���1-%'�\KUv��0�_���Z�
�5?=�yx|�kP����
�����}������������!��}��o`��Rm��7W���u��=��G]��=��S�5a�[��:S��v�(e?v���77WR�^�,o�B�?=��g���{ww����v�?j��Z���)"� a�[u.�~y���?i��C�a�����������C�;-��/F	�[��2�g���Q��Je���]O&�Z�I]?@�!qS�4��>yy%S[b��z}��b����+-���S�r�Z�^p*�X3�[WSXj�!�n�U�����0-UE=�L��HSF�L8�d7�-���N%�����7��������?�71&�����������~#�i��=#��|o��%���,��v*��$ 4�t}F�������A%�{E���	�E-�������gD����.�\=��U�@u+'8��u���[����������#zj;G)��a /���@��PJx7 	����avK#��SK#��4r"K;q����.�Hv���b���n8���1����kP��KP�mx���<3,�R�SR���������'R0dq��m���n)78g�;�\T�S����!HW6��Z'!8����Xn�tZ�tz���9���3)�z
��-eB��;!9J��� �j"!XJ��g�r D�1|Y���mH-~��+�\j���J3e\�9��3@g;y2q�R��V�qk�9�85It9�q:�?�j�����9���%g
:(%�@��H����`<(d�X���4�b��>d�5#W_���#{N
�F���)�KAhZ��� 4�:r������q����b�����\@��&{O? ��+��Y�.�I�-Il���/fK#��SK#��4r"K�����?��c��Uhe'7�����Z�"v:��a:�m��%�LR~�-�U^w�S����q��Fk*8���B�����JCR��j��s��E�;%��p�8j*�T�w��������a��=��;G��Mp4�������}������,���"�e��\�x�0B"S�eB���"��3����C�f��4sR�g�O�����AO��;�z~��&�Y�J���	zZ���bO	Vd����� XZ�H�r0� 2�,DV����G;�\T�S���������9>w�m�^d�1���q������;�b*5d1�K��La��,E��2@^�5�(���0s$A���ij��
c.����>���)�#[>��� �-������{��u8�=�����,a-[��c�;�1������8�1([�O9Mg*���)��(|�Y�K{Q����"�8,ba�q"k
I��>��k.��)q`���k����S��p)"c��4z"�8���������Zm��UM��?�����4��#�e��`�x&1B "s�eC����X��4s���**��qi��J����'��]
>�L���>y6|�������w�=��
�/_.��"�����D �,R0�A�L!����n�0�g ;�\T�S�����H�Rg<�c�{���_��ZaBk�����Zd0�����V�Q��Q�f2�(N�N����+n�e��"(��$��8FO��RD .��ZC� �J	�GD*95C	�����-�T��H9.����>B&��t���S��|+���1�	c����q������M�]�%�<%iX�aK�oPJ��`�f����`�=(d�X��4$ukj�!�s��E�;%��p
X���N�R]�K����Z�-�4p����j.]w������m��aj�����9B� �%�p|��|5,���,���3�9B���,C@�6Pzs.x������sR�3������aZ�-M�����n���	�S1T��M��wP)�4�/[��^�����W�bOAhE���
�X�C���5��!�3���ZkH����}�*�s��E�;%��p�����|Of����8]\���OI��t9���[�:���x�������"�����_��%�`l�2zjD�"�q ���QXJA� �P����4raK;U�4R�K#'�����p���^�4�[k�=N�^��I���Z��F��RX��O ���2{J��"#��5�������E
���!��!d!�����-����f�����vN
�F8�9�Nw1:�o�mpz:�tv����ig���f������������%�p|&�|50���,���3�9����,CH�6�	���K-=TQi��K3'UzU'O�Gc|}�Jm�m��>���B=�a��C1gG�yJ�`���tLv�v�#�r�F3!Q8�:R�)Y��cr�h�R�K�)H���BF����JC(�
a�O�D�\3r��!{�8����kD��s���bt*�
����
��b:CkE������u�{����\g�����\`��&&{OH �i,�[�0I&0I~���/xK#��SK#��4r"K�H��g����fu��m'���
@S���<��j�Q�:/�!���7�5���{��;��u��d��4bE29kLB)a����-9r^��B����d��f�!N��*�F�
��\C��S�!~B�PT������n�#����.5W��8�Kj�uP�si������.���(�36{����X�5Y���=�^�\�����8���������k|�h�x��MP�~��}3z" �S�G�n�@��&�0DL���"��D	�R��pE��k�$�=#)-�������v�oR���?@�������_�9%#�����VOp�r��d�������B�� X?�!��`��(� "��!�[�w����.5W��8�Kj��s�j��H�_1�L�`\
�����!z����i����2���L,��@+�%y��f��k3�gFK
bN^�c;c#@T��!C	Q��5U@?e�)�V�2n9*�HD����>X�W�/���1�>�X���t�����i���a�eT�������p�R��5�j�q�:l�`~�&���(�M�w�9��n�C���E���=�!&�@@�YCz�r��#�K����;'��p��f���)���8�`j�Z���j���'�=K�,m/���}��n�sB��)A�.��t?�/�^I{
}
��V�4������$G<�v���~��f"���!:P'X/
2!A0��C����[�
@���R��pE��k�$�=�nZ��k�A�����#8(��j{�����������&p��I��.�vWK�f�Mh�n���t@b�=ZM0b���L�BYP@��?�I�`HS PDE�HU&
���e��t��f�����vI
�qq|*V�%���\%��L���M���GIyBEj2���*��QBVYGSV�����~AK|�B��ZE���������x\~�d� ��� �@%�f8t�j���v��j��FI�j?��n�pU�O��{Q�a|6��*j�M�����uI����u=".k��$h�,@�_�t|�b��,P��vQ�����C�s&�*�1� B�B�!��@Q"��4���N��R3pqe�����\���lUI��(�����M�����Q�`P�X�=y��)��$�r�~�m�E(�%98vh��_G���  ��������@HALd��~�6p���]�+��5��Wd��]z��o�m��8���oym�Ao�V���9��9z3����/���/������N!�
endstream
endobj
103 0 obj
4175
endobj
100 0 obj
<<
/Type /Page
/Parent 101 0 R
/Resources <<
/Font <<
/F0 6 0 R 
/F1 8 0 R 
/F2 10 0 R 
/F3 12 0 R 
>>
/ProcSet 2 0 R
>>
/Contents 102 0 R
>>
endobj
105 0 obj
<<
/Length 106 0 R
/Filter /FlateDecode 
>>
stream
H����r���@����h��1�8���r�du�74EI�J�CR��<}08�	="
�a������3�
����1b$k@�p�H�g�Ln>}a�y�����%�k�������_���o�k���;�jT�|�����Z����F�=���^���'��j��-��3��~}]o7a/�t�ka�wa���\�@8��k����on/��zI�����~���s��z���;�J��	-�3�&@\j{��;p��,���E�K�V��2�f�s�V��5�
wa��m��Ls��|o�V����l�O��@�x��i vM�	����V��w��;h�o��:�?��HWB��T���F)j�����\z4����5�6J���������p+4�:����E~�Q��,�]��we�a����\?*U\�J����
�L������]�U2t��k�"O�U���
O�;99c���v�HA�>��.8�(�g�wk�[r9#���p�S�
|�����A���sCW(���{N
q}�����me��}
�$��$j��
�F�Q�U�S
��"���m?��D#�y	1��[Ax
v
��\*�����ZS�?[J���G�p� 9F�����Ot
�98a���2P�Qd}&S
��	�e.%F�
!���L&��bN�"��"��f��SR��G�����W�
P�y����?���?�7�Gy��5�M\��n�TZu����t���j���G��������1�����")9g��K[&a��c�D�.�%\���	0�5$vK��>Bb�5#�Pb�#�����$M��U!
��(I���I�E��� J�{$*�����t��HW�k��>MS��)]q$��5
 o.���M���9����i�4�s�	0���� �(���!1�L����=T05�������lS�q5�Xj���,��=@����D��A�g�U�4��
X�kB�S���z���	��4�L���8����	Q���
�H\��9���@$XOL���a�A��4Y��f�!�[�;�9�=���Be�=$��sj��'kj$&����Q��*��^�sHJ*y�gH�K I�j��'�	��@BUc�����j)@��T��$G�t}�O��������"�3zR �<�,�'�0��m�-b���}�Di����PQi���T��������m�����(�YK�����9�6�ja.@��h������V*��u�a��	n���
�H���9����$X�PL����a�A�;Y�qg�!!�������S�������c�����c�[QK }5F���.���P�����rTjE��'��O>��r,O�A�VH�����r���Ok.�����i=3�AD-s�i,gh �A$�3Q�@-96�0/Sc.ljL�!��1'2��<Q�3�����8I������������f���
��d��>�_���F�A��x.���=�H!���$|����u����������:*��Bx7�u3<!N�����;�9�T����*2�r0C�0�(1�4$vK��FHB��f�*C�!q��SC\=�D(�Q���(5>XN���M
gT��}���,u�S~:��� %c�O0�@5RTuT%l��Bya<N#�qc��(Y��O(
�9a��2P�Vd}�P
�Q�e%� �
�E�	B'Js.����Js��4��JO�����F��:��S�J�Tp~������v�t��������������
)w:!������u��#x7j�DO�\���s�A�=H�8��"�'���
3~���LCb����#!�\3�
�!��8b��!�C��p�$�?�D>���39J�rP�:�"���~�!&c��5�b��"��<��	,�Mgw��`p��%����R��OANi5!�qq����K!k��gA�(�g ��yB;�$3'H0�`%���CPSSa����0�9��Ha��blS�~-����h�\Pm�����"�G�?y)�s��/X�ms\������d`�:f�/t�T'�
�t���1������"�)9g��K[>a���c�D�.*\�X�	\�5$v���9�b�5#�Pb�#�������Z��� &�bb�Q�.1y>��������@3���O=�@5�T=�����iU}�P�J�S���YH��a������#&+�)G�g|�@�0Y�W�a���[���U�s��'TT�C��9'UzzhI����0cb���wb�mf�C8&��1G3���!LR#u��:�c��@27��O�9FO������<����h)^"�����S�Qu
�"~0��[_A1\�0���*
 C����e9�2T�a���f�%lg�x���Eh��l. ��:�=s�F6�]�c����M��=���T���}�Q�����l�������Fp������O7w����lw��������{�n��6�R��	o�?�~/+��_�����}]��d�Z���|]�����?���������#������}}�������������x������e7�Y�}]�r�����i!��o��a�|]�(Y?�o�|=>}�}�X�$�a���@��o	o�����m�z�+�O�p��l��������{"��<?�v�q�Mzl���?f�.5�H_`=�l���R����n�#�-%o�������rqh������u�������
����~�YO}'bv�[���%��f�=��+�����i}x��|�o+�m�}�-����gr�����?6@
�-�py�����/����/�E]-_6����(Y���e�}C���?�����+mSi����#9l������������Wz��6�_��C���c��	�!���Q��w���!E���6������!9,�3�ap�7'|�'�a���<����)g�{�;]NZ��K�_W�?a�qT�7Lu�& �n�	������,NV��W��L����HE��*OJ���%����t�-��6��;^��a��g1�qQ�����8[e~N�4�b�<�`�z�G\��y:�����W��q��>���Q��q ���9��!����:�.�b���@O�MS����j�BE�<1��E��p��;�o$����
>�9����@7M��du�u��:b��M]@�>��3���NP���P�i�}�.y�1XeWX4A��Y{���?d���������e�����
��U'
endstream
endobj
106 0 obj
3201
endobj
104 0 obj
<<
/Type /Page
/Parent 101 0 R
/Resources <<
/Font <<
/F0 6 0 R 
/F1 8 0 R 
/F2 10 0 R 
/F3 12 0 R 
/F6 27 0 R 
>>
/ProcSet 2 0 R
>>
/Contents 105 0 R
>>
endobj
108 0 obj
<<
/Length 109 0 R
/Filter /FlateDecode 
>>
stream
H��W�r�F��CW�(K�0��Nb{�K�:^�yY3�Cr� @c@��T�}{n ��@������l��9}��t���I�B���_�� ��s8��r!�w3|�.�o`�0���G"��^�/B'4o.�����(��������5����+YM^��9/
^��q�������,�z��Z���I��4�k����sx{[6��^��N�`B�'qu�Q�N��� V�]��g�x�:���	Z ��s�����Y�R��7}6wN�0N�*�kvY�\�*��Xo��\QQ���#�H��x�Q��I��&N7T�L��^��WU	��|V�Kx�ES����8j������$�\����xm<�En�B�J�N�D��8�����d9,�h*�k��%��f�#�s(��
�q���v��Pa��j�]�P�O�0N4�N����0^0@���	����-
���5�R�J�t�q/�^��x�tVId�mq����k&j�����f�y�m��V�:���u%c�0����[�����D�U�t��@��PV-��]�$,I���fP� �
g�
V�����n�V�
-s�I�X�����x��*e4gUQ`����\C��$Il���|5�t����-Y�����0��8Z������z5B�/V���=��L8h]s$������ga��J]����v�E6"~�@������~�<���1�jL�9�����UU�j���C	o���j�U�I^�j*8J�F�!�������[[�9LoA�g72��,Q�#[g���e�'d��E5*���P�^��8�sgO�D�:�,
o����v��y��`�L��Y>*�Q���y�u�/yAk�R�H:����*�(5p_G~�w�]4�n���WNc9MIb0���yM�0wr���e�g��)��R��!�9�����O�JG0�R`1F
���e�l���=5�� �]�}i��x�Y�;�S�J9�S�fY����E������Qg/#�����[3�G��q��
�h�X7>��['$�� �}yn�<v�^���/����[��;l�st�t���`�������_��� 9�Y�X���g�)��!Y�����^�*a�I�-eIm�x�U��7ZL�h��c�S@�M����	�����E�n�E���4�R��0':��XV���2?��8:8z�eV����
L17W��Z��-
0W������MT����$���:�����P'�&#XRa[	����?��&d���O,��(�M�����1y����Rb�?��X
��f|�o?:��FU0c��XHx��lA��(�&��:V�F�:����>���=KZ��{�&#������#���3@��=����C�I�I��nv�������`RwJ��V�+�Ldb�X�z���������5��gN�_"I�wI )��p�cl��>^�$�����-���g��Me��u�����`S���������Q�A0��u�Gzy������7�Y(fp�{7�����L��H�U�e��K3,�I2����
�p�\�Z���]W<�8��t�
3qK��{c�n���:��d����`�1.�T��������Ew�<�����w��� ���������b��{��W����N}�<v���0�wU�������+W
IC�F�R���c�d�Q/�of��K.e�!)��ZhJ�;S��WG���?��=���G�����0|$^�=n����>����l�K�8�;�r��/��e
��g����Y��vD�GB���f ��{1�j�L���R;W
�����A��
U���X3��D� ��CyG�.�bH�iv�;}��pr����*4���w�g��=	MBo0�������.�=����w��?�����	g�w���L�*�cp+���.�������0����z�����K�p�P���fh�u4=]R���5���o�:%Y�yn	���f]���D��b�s.��O��Z&���-	>�d5�
�fA���5c5�i�,�?e��d�r�������������3�j����O��=L�����d��P��S����j�6��S��(_���������cU�4�UT����G}�|zH?��`�H�,��56N���}L���<"�.�����+���f��M���9~�z��x��Y�������@�����D��%�h�����L8�SUY���/)V>`��Ld���
�rk-�~�i)0�!�zD0sV��6�r	`U�N�"���YU��ZI�����xh���a��]w���$��� �<��Hp��S�Y��r
�ZO�q���Q�7c�=�l���tz�Cm���E�n�e�������v�~F��l:�^�3�75�-+�{�VJ����O���
endstream
endobj
109 0 obj
2363
endobj
107 0 obj
<<
/Type /Page
/Parent 101 0 R
/Resources <<
/Font 110 0 R
/ProcSet 2 0 R
>>
/Contents 108 0 R
>>
endobj
110 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F4 17 0 R
/F5 21 0 R
/F7 33 0 R
>>
endobj
114 0 obj
<<
/Length 115 0 R
/Filter /FlateDecode 
>>
stream
H��W�n[7~��NY�
���"(�)���]l,d��Q���v\}��3�9CI�(1���o�83$��O:���3i;;�,�mn���;����!���kv12|~aS����1������a��LZy�Nd�LXc�
-�A�>,���'��_����ru��,oo�������s8r���������,b~�����������~y�����W�m�E��
�V��)"=�Rv��LJ$�f��I� |�sO��wj��Tq����U�i@�X��Y�P]NK�f��3m=�!3�#dv\oQ���B ��c�>m�7����]���/l��>���P�16�uU���|��p1�x#g4�y�e6�>�����a������C�e���~��z���a������BZ��{>��KL��I>ksm}7�f��g�K��3�qa'�>���Q�Wx�y��J�n�]$��NK���?�>	B����d��@�d�
�����?�=}Z���[�X��M���.���fNI���`B��K2�\Q2U��:����������iZ�Cf9d�L����X$"�s��g����1����vM35����@6����vg�Ga``�i�I�~��b,4�������zB����$C$�pb����uMz+ST��.y��� .�F�d��l��@��=�Hu����]?8�!�gfc������4
����f5 ���sG����Ca��E����T/����&��WCt)8a��K�����P��3i����~���vI��e�U����I������a���k6
�K�ADF�)���!�#���3�E#(q �2@�5�����l�t�8D����U�IZ f�n�kH�����b�pu�NZe�n�tO��;������7{�}�:8$�����`�������b�w���|��M\:���;6�	�%��'��K���~a!<�7�����la�����l�+�z��3c�A:}����J��*�t�B/��Q'.���3e� ���������
�*�O��U��S�AU�T������V{�gc/�*)-|�Z�����1����:n3>K���������2�j������|���$�y��2	���%	�c��3�hqJQyO�p��t�N�t�J�Q�I:M��Te�NS�x*@P���T5MK�T���F�j���\����t�=?���\q�j�w���	���Z�le�k=����
Np2��}�F�j��a81Z�^����
��:���*��B��t���:�:]�V�N#N�i�=���=���#��Q(>��#�5}R��k��)?B���(��t��������&�4{�?}Z����r~�.��/A���X��Lj|���3w�o�I�mzDE������C~��+�Sx��F*�����L�������L|��/'���c��X��������'����� �������a��|�q|���������"���gt>\���	���n0�:���k����
�~J���L����O������LH���{<r��52������d~��Q�I�O=��]��c�g�t
\�=
�N����D����e,$���w�<�0�����X��<��?k+�>5��QK4�w�J�������4�l
���(#Sem����|�`��q�[�^�l����C*�8��0f"-��!�:VA�r�+u&��r��o�=[,�����CJG�m�2�N�m�;������O�l��a��������E��2����0�S������"X>��Q7C������gl�;"8���J\�]D'�wY�H���x�~e)c��R&O������bN�y��X��N�����3/D���9�^/�W�UZh�[�C�g�e�~����<��$|�>]�7��
��=���b��������X��P��$�zz�*����������b�
!���d�G�7����E�`(�<S�:��Jk��
���Hws�uw�*�Tl8�r��@spV�<����@��4���$7�Q1�@l����H
BE��E�����-�K[(7}��(6�:J�VY�����J,>D�T54�j^�S���$W��"�Y�CBe��u*�	5/����zr�AB*�����0�����9t�4���aV�p�8Ja I���gW��m�kUMp j�@W�|����rSm!
��]��	F���j�A���\O&F���BUh#$	Jd��Q �E[KW��A�c\K]% ������p���z6R ����#8;T�+�6�X-*7�@px�k�A�i����F� �����(���r�(����W������L��@����\4��jE�����	]�6���c��6)` 6\2)d�SW7I� �,
����j�R �V8��4�I�����@����7�@Th�~.��m�WF��F�r]�K�0��8�"a���WhB������m��#8%��_u
$,(��` �Wb��Q �}��jh2�@���A�i���HX���B�Hx)�����` ���B` )��vQ�8Fx2NQ<l�3)p �P)|�U�km��������l�3�@�Yhk���� �D�ho���s�V/!%`"�0	���f�cS�&�I4:J� �d�B` �Y'�@0mvLa�60�wMn9�V�@������]J)D��
<Y6�$��V[z�V����t�2��E���obyB�i�U�%DO$�
4Y5;�v5�6��M�l���tm<W�|���t��&�I4Z�@�ekZ
$,�6�*H��Q��h�}��A|���2
��pH]��V4�a ����%��nsKn�0�i��ra0|��G|�F������N�
L|�^�Qt�h\��F�������G������
_� =T��Fd�`��N2
Zi����V2�Bq�d������Z'�tB���v���#Jl-�.8EE�2��p��	�Yu��xS� ^!�������D��r� �w.c�bSS���L7a_T�c}d�75�iF��
��Y��k/�����4��q������C����PF-�2��m~
��%�'��^�
endstream
endobj
115 0 obj
3059
endobj
111 0 obj
<<
/Type /Page
/Parent 101 0 R
/Resources <<
/Font 116 0 R
/ProcSet 2 0 R
>>
/Contents 114 0 R
>>
endobj
116 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F5 21 0 R
/F8 42 0 R
/F12 112 0 R
>>
endobj
118 0 obj
<<
/Length 119 0 R
/Filter /FlateDecode 
>>
stream
H��W[o[���|) '���e�b���}(�^��� �r����lo�o/9C��GR�E�8��������b�(1���"dq6�$��|"����8�v����������dJN����3M�ogyA�U\9}����i��
��B?����y��|X=�6����'�n}w�~|]�����)����("���L���9_]=n��/��oW��+r�|^^.�VO����js�!:�Ng�@�f����7���L�f�>�T��5�R��:�[V�?��
{5��
��r),i7c_fR[��F�w���\�Z����|yz"#�x��/�����y��|y����S��5P(1�RT������)�}&xL�:V^��9����~{Y�#�H>� ��
+������X�U�]��q�.��.N|$�@`�9�['�p[R��!���.�!%F��|&�V� ���������34_��;2��Z������!���T��\$��H��!�D��dtL)
SLHYV�C�S
����Ih�k|�����D[�^8 ����R�]�d�9����P<n)xj���K�:;.��ec����m���'����K,�����f�q����6��2 ���8�h�VS��ct���X����N'Y�"�f���Z]��(�ra���W��/��*6�q���3��-�K��������.f��Z.��_��
�7S����"�Fj��;1���Xw����q���sUF�������I5���������B;RjO�������fq�v9L)2{,T
�����z���������XV���i��%{w�s%��N���~_m�V����q|���8�����m�iv�!�;���b�{o�L��j�~�.��1wyL��2��JKl��P�BXRE}�7#zY�L�9i8�_���) d�g�������!9^����hP�Vi�i������Z�HvzqR��U����d�3v���F�q>z���{�����	*��>�������C��}����]mV�����=�izL8T��N�7u�5�}$�	����aN�!�Tl��_M�C]#�'����yZ�����/w��o��/D���6������z$U���FWo������TFF�ecC�]n������n��m������A����d�������
yZ�_��urG��20�����/�~x�roJ�oI��|�.�
o�����{�Js0,��<n��t4�%��2���~7��n������������/L�21I�ZR4�PB��{�`�zd��CNmV�/�������{v�/NR\U�4�s�z����8���w�,?j��2�W\�;��}�N�[xM�R��|;n�piq!!lp?�]r����A*��B���AC��_�%��WZC=�R�r4V.���,�F��|H|K�Q��@�,��e�������2gfbk���e�pcE9B�>IIG�����k��FT�,���S��C�#gh��=���$!��saKb:�4���)��IQ�7��9a��D�#�y���YR������I��t��VD�O�Tk��f����y
j�r3,`�����X3<�����r[�|X� �����lA����)7�Z���)�s�s�
 ��f��������-�i�B����7�,����N�m-����\;�%�o`�#����`
h���[s,������:�2��.��Uj5��2
�Jt3�@�J��KE�[G�����T�+&,pP%Z�<����T�2~�p
	��g1�4�F������]�vZ�D�"��r�S�8k���]���@�\6>��Q�Y�3��1�f�������%�h�T������l^����G
%9�z�2����H�^:$V?�Q�N���t�4c�����mZ�k�b���Y�n�blS���#�����4�6������F6��Y�^A>6�)�H����]�T�"��]��G�dr��C�/��A$�|�*~�W�0qSW�M	��Z�@=���M���G@������6*��'
%�M[��]��o�d2�F�`���4D����K[X�����c��)��.;�n��y��a����G�8�L<!���E�9}/��;���7��&q�^�)�*��U����Q�i����ea�o*�+�iq ��J/4�l���5�}E�_����f��BU�'E���Q�A1�p�A�R0%Z4- ���y�f����5%WD�P�4o�l1��y��@�m�>��	�#��L���j����`�G��MAd��8��W1a! ���4@X�l;�|���x4@�L;�+fX��qf�y=��|h�"?�4�9�����RK� "&J���@D=�MS����G5�WIl<�
��'����!�i��cN��g�@����4@�e �*DC(��~�������Z�KF��@
!��g�;���$����_�J#g[�d����Z����n �!�z|�6�CI��j[� ��im�����|�$�����%:�*!�'$������u��B�������)W	�n{jl�W������^������Cx���8?n���USH�@���v�Ph�������XH�c��U�XL=����@W����Sj��M���3�@���s��#t������`��������!�K(��q�������-��Mq��uj�����3�2--�j�����
�`#%��q�W�Pz���A�����_�X+���?��������?�U���J���*\]��+�Z�h���g�{�S�v��=>��������PB���E��**8Y�MN������E7��^���T�%SrB�'x���������"i���{�,�q��/�?0��2m.�&i���T�s*�[���q2�<3�%����H6�j5�8$z�AD���7�������;��Bxo��@�B,��������NU���{4��k�������k����%tii�@���E�����*���~{{��SUAP�:S�Ak��1 �7��6EA(k��������~�������N_�w�m`KLS!�
�~0�
�s��D{�����>�N{���;�{H������x�`��W1c���q%x�v�M�ta�+gprV�2�����Up�a/?�a�����|��|�8���i-l�/�5 dU|����n�l�����%Fu��b���!_��.�H��Y�s�p#b�|����At��4����g.�I7���v����B����������
�����l��c�����n��Eo�e#-np�����L�_:������Ab�Q�����b�y��e�J�c���[���s�Ue��~�k���l"6ZZ�>3�9�:�]��>�_����Au�c1��.�~�j��y�u��a`��Be#�����V0c,��������z�.B?<�NW�6�s�a�E��c�H�Qg�3G�Q����7J�m�����}��]��@����P��Z�>E���%�9�]���>� $�+�i��IE�Zv�^�#:z{�|�o6w�7;c,�A=�~"e>��}��:�>��=m�S�C	e����O����{|����iz�M��8*_"Yy�c��qZ�i<�-�|>�r$����J>g+�����n�9���
�C9�4��B��jg�a$*�.I
4��O�@C��)�',����RM������Rm�E��@�"s������8-�;������#u�GC�!.�w��$��?V
F�F/dW�e��L}���x�������Xg^TP�xT�=��:2�J �
�B^v	9Ht�0���m+����JkHc�U#]O����������Q XhD��$$?gJ�1�V���j���W��{�E������hB�)M���l��#$��!�1�����I%Zd�����*$($4t:K ��c��-�*vH�`2�q1��x4�(�5~�b����?���8FXh�<��[��K ��[fdH�Wz9�$���-FT~�B�����*�0��	
�
ISK$(�AR�!�L-�A!1�$'�=�b2C* �8�Y�Q @S�O��5��
jx�0�����FC�#��F�����\�#*=Lf���Q ���J��
	�3�1���p��+�B�0<��x�����
r$�t���pQA�2�x��������L�Z!A1�L��k���bD�P^�@���G�	
��j��m�����!�%d���X#�������Q��c�w���p���fa�����l�;������v������/?:�
endstream
endobj
119 0 obj
4275
endobj
117 0 obj
<<
/Type /Page
/Parent 101 0 R
/Resources <<
/Font 120 0 R
/ProcSet 2 0 R
>>
/Contents 118 0 R
>>
endobj
120 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F5 21 0 R
/F8 42 0 R
/F12 112 0 R
>>
endobj
122 0 obj
<<
/Length 123 0 R
/Filter /FlateDecode 
>>
stream
H��W�n�~�C�����gf�Y8^d�Z����P�5Ejy,c���������A$@��������~��URV���CO���|���RT�7�=qw/�F��V����������C�pC�;7U�=T�������O��������������?����Y����>1�C)
r�tw�$&+k�Sy����_,��o���^7���x=�2^�+�����YC:K�@����D�j��Mz�A���=7MK��4a����T
~JT�����h�TU�%Z�����Rx�I�]iv��:����M�j�I��#U�DG-W�Z���S��u�a�����K1����e�g����p��������]����������dUF���/�6L<���|]?����R��u=j8���
����E)�1IJ)K����b�,Wk1i�nE�>�k�8o�]����S�"��/���XL��xhV�e�e���$s���'��t2��}���UD�q����O����L��x/@���AL�����A�9�T���
4-��� 2����S����m�<N��qQ��z�[�K�9l�)�-GM����1��\�`���UaEH-�%z0Y��.|���/����C�������q��b�BPu��l����<#Kf������5h�j�����G�y~-�/�r���T�!�v�b�t^p������bY�7�9�1%��_���:l7v�+�{@������x^lG`�hW�����&����������]�w��e(����`4I]j�W1��"�W\�z�}��\>!��A�'�����!mQ�w��`����������l[��6�?�	��Y��/_h$��v+zJ���&�;��'�7J��'�43����)z���yQt�q��WE~Z������uvn���h#�p9����
|H�c��(�]����E{�A���7h:���Xe�r����A�G`)'� �2�2i�6��g{D�d��G����Q���&�C/$��T6��v�^���7����)��n��NU�K����kt��N����H�A��0J�������
�f��t�p
�����
xI3�L���y���%��c���#�����lof1��.������oM
q�z��_��Q}"��<@�]e��������(�!������G$b�l�`���[�DO��-��E�=W�DZ��U:SU{f�!,��� f�U�1\*!���5T�K��,��!j�N�a+��HjP��Bc����J��0��	^"�*�_��f&e�`�(	�>���~���J�8Q�5���sQCWQ�����l?���8: �/f�,b�6N��o�^U�M���In\��$h�����r�'�}.��l������V���9����:��@��n�~nP�t��:�%�{�.�����#��6��������������gG��O���A'�2����2�||f���*�/����r��vo]�a�*���xM�`]�Q�y�x^,��8���f����T���X�|�):�1=�2�������SO�PHl���(��)����>�O�8���bHZOt�8}�T����y)Y��zL�`�50/@�����=?QS����Kr�6E>�n�5�9j1:��-m�`*����@��?�����5�_HE��{��Y��b����u�/��v�?�^>�~����/]�4<�����u>��N��;?���1�����c����zI���y3���h��;gt)U������L�?��Q��g1,/���x�`{�s�I �2���������K�Ri�7�������e�%u��N��bN�����N8�

�M��	i+)�� ���ONB�B	���.��#�B��8T7t������r<���Q���3���d~�8��7d����8*(N�o&40��
�-��f	]C�/��$�Ep�40z�<w����o)"�9����U����GW��h���c����������%�.��f#�q����B���/�a��B������P�T (�D��w�ilS��[����M	�9C����/�����~8
;|�{]U�i�*��Q[���&8~�B���x��\�
�0��^�H�aUb8aC�\���������,c"+G}l_�0�(�*/h�Q��"�m���4�X�s(�����W�Y"n7���")������	�CL����R��4v=��h��x�%�8f�XY����EwK
��oTe.�W
��������������'����d�
�bu��"��t�:V2��y�Ek��qPP4���|�u�SE�#c`���M1�
Y(��29��ok�U��^��Zf��g�,k��.��
%�����n%Es��d+W3^����/������,�����
��V:�W�/��-X��6x���������%f������8����R*J%��^�,Vm|c��74R�����ja������t��c�]x���OH|>����a��p�F��|�M����W��Y������Y���3��Z�&��
E��9=�M���3
_���	t���������m���c?��b^_��d]��fU?��b)V�z3��`9�-�
L�A�R�h)�hT9�5��e��\�`T�d��<�b�<N���z�Z��h{�+���������A�8t���5Z��\���j�m���r�!������F�vI���n�~}v���9�F$/��c���?���li,�;�c�pwx6�1�P�=y��c�s}��G�W�XQVp)�Cx��6����]y��>��&/��Kuc��%���l�@^J���'m���9ia��oO�s�����G��A��=6`�������?��Va[�NSU���D���T���[�K><�5H����4�v�FT�V�9�M5�$��6��@�n�����aeO)�Y��<�M�X�u��?��i��$�6�"��Y�Nn�����&�4��6a�����K����H�-����#b�B�u�&U��s,Q������y���QV�?"y���[�D�N����`�EV�-� }5�����Y�O�����
endstream
endobj
123 0 obj
3098
endobj
121 0 obj
<<
/Type /Page
/Parent 101 0 R
/Resources <<
/Font 124 0 R
/ProcSet 2 0 R
>>
/Contents 122 0 R
>>
endobj
124 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F5 21 0 R
/F6 27 0 R
/F8 42 0 R
>>
endobj
127 0 obj
<<
/Length 128 0 R
/Filter /FlateDecode 
>>
stream
H��W�n����Cc� vb��4o���Lv�� ��������d��5�����j��530������z����O7e(rq*����P��^�<~
E��v�N<U����8����O���Q&����i��7��"�:��4�"d��N�?��������Vj�����n��k��(�faa������J2�I�$e���SW���zm���J|T��(������?u�����A�6dy��X���mw	Q��|#��Sr�:�2�o��|�5F��[�k�.�K{��a��\�� �Bdp ���bo�s�8�R{��mm���L#�c���jP���������@�E?t�F��F��n���t[���QC��gU��h�V��A���P<D�w1���.�M��'���|e��|��q�eP�D����4[���[P�`qC�,�p��~P�!p��U��U�������Yx�Kw���;�C7
��"e���|;��bP��;�pL?h�oC7��4�S�������L���0���iu��Q�+	��<��U��Q�&E����q�5���bY��[��+(C������1�f���[�pR>�+3$�.�X�C�Vu��{�?�S��u%Ya�GA������	|���������[�k�iGB���G��:��,�q��#���u'����s���_\h�nv�����4�<z��L}�5X/�]����j	A���8vpW+SM��8�K�d���F�Q�-G�Y��<���! S�LG}����~�xr=<�W5)�Ye�f���2�����R�%�5Kz`Q��0N<��X��hF����7���!d��0��\�������N��������y����u��Do����&�G�4W�^���`��k;7z ��!��k�����W��aT���;�
�)R���;���zUn2J}l�H%���4�a�'����N�E�������a��qH{���v[m!�:����h`7�,v���zz0��X�/*R_�{Y��}7������E�J��k�(�
����/ku&��;wa�LX����(|��������k���	$	�3�nO�,��y���<�u�d��������?������4����k�
^��<�.e�)�Mz�`N����p����o)�������:8���z��Mp.?m�h�
�H�5asei��e����Iy���{��An��h�M��'7��$���O�h9'���4�D+s}��|n���:8��d=���J���v�����2���&�s�1:������.u,[�6��<��u������@��"��S��L� �����w�0�*
������G�<"���$��b����?��,t�UY���u���}G�G]�k	�=�����E����
^�1zxA��`�2`^�HD��N�b�i�f��0���&��_����
j�-������r�+�|�Wby�Vda��|���Q\ev�5Mw���LS1L��/��aG%�)�����������,&u��O�x&�|��TU7q�� ��~���],�'GFg��zeq���?,`*��H�s��;]Q���\0�������)f�/�o�$3*��z#HC8���e�X��,_B5�A��Y����E���I-�
�Lo��7
#��4H��#�1�#l�\��/mX�O�������[��b�]5����p�����4�������O��,����X.���
}s��<��r���z�Z<���SGb���}0�L�MmeA���U��L�����-��=)#��^��ihq��"@.~\P+��0�/H����"�R�8I��m�=����������?�a$�KSHU�Y�;��?[U1�J2�������|��� p[����]�^snR�lv\'.E�%�%~DY�����Y�(�uuhkP"v�c4����\�*E{br/z���e�|w/���AW���^��#�����>�N�a���gQ/i`���G�����x�f�E����C��Di���&����H��i�a[rT�\2��l�������<��;�v�����8��q�C�_m�����QL�lN���Q4�6��S�p;+r�_�����0�
{��cm0�*�;=�P�<X���x���-�5��j���&�{\�3X��|�w~��GK�������gV���I�{mA�	/~q1�,�W`��/�6��*���q��m���8L�qu_�I<��;�x7 ��#e���L�����e�������������Ed6
��yJD@��`j5A ��2�d~��T��0���N�}���{q�:J�NPz�H�+L��4
��L�x^1���o��������������_}�b���s���@]����E�9vp�v���N�M��k--_#�:iB�l�:�L~2���EG������&j
endstream
endobj
128 0 obj
2400
endobj
125 0 obj
<<
/Type /Page
/Parent 126 0 R
/Resources <<
/Font <<
/F0 6 0 R 
/F1 8 0 R 
/F2 10 0 R 
/F3 12 0 R 
/F6 27 0 R 
>>
/ProcSet 2 0 R
>>
/Contents 127 0 R
>>
endobj
130 0 obj
<<
/Length 131 0 R
/Filter /FlateDecode 
>>
stream
H��R�n�0������LR$%]�E�Mt���i��+"S#���v|t�vggg�|*��!�,
(�C�u�������j(w�{������5\�����J�TYm�<�p��L!y�F���n�v��O����Txr��k���3�����t``�l<@�������1�v�v{�|4�����6�l��~���"�K�}:7Nc�
���@��1�|�yG����e�Wn���*��T�5S��5��9���g-��94P�.f���a���MI�������er%@H}-7�4")�S%2T�9�h��N�o{�C�����.�8�����v��:_��f���	�+�0
S��Uj5	����wa�����uU�4�"E������4;i�4�G�C��g��&j�G���~�����rQ�:������!��2
endstream
endobj
131 0 obj
417
endobj
129 0 obj
<<
/Type /Page
/Parent 126 0 R
/Resources <<
/Font <<
/F0 6 0 R 
/F1 8 0 R 
/F2 10 0 R 
/F3 12 0 R 
>>
/ProcSet 2 0 R
>>
/Contents 130 0 R
>>
endobj
133 0 obj
<<
/Length 134 0 R
/Filter /FlateDecode 
>>
stream
H��W�n�6���<����D���c��
����p%�.kI�PT����/����0`����9s���quW'P��"+V����l����j|��g�j��l2\�
|������[��E��}�����E<"O���������� ��j����:�wV����d�$����������<���{��d�MO����U
,�k1�������������!/de��C2�.����V1��#��g'�JFO.���5�<�VkY���qgUGXb4��<������,:��+3�i^��������N��)7�1��8���
w�xGrn��Y�3/XU�>��K�����b/
l�-�;}�-�o�=|����Q��N�(?Mr�Jt�i���J�	Ft�=���*���_Y��P/xY���������A�-��,WY�o�_X�|��2�f�8�����O�\TV��1N=H�Gu"��r��u88���������,`I
 ��w�`�?8��HY��rF���k=y���6���I�+/xgC~.t3�x���/P�+��<	���$�\�E��g�.�
�#�����������������m�t~�V�G���+����7�������C^����n?����b�?�ba�+��<u*o����}(�F��
�y�Z������XO�k	I5�
�)�S�QW;|���u������-�
I*<��z&~��"�������l��v��n��i=�i�/�3O���z.�No���{����0r3uU'�'�K����+�C��k��A8���TY��T�F4F���u#XK�Ze����r�����g�S�";�Xpe���@�I�g�-����^uS����aI���$x����
<��l���w�u�g���0d�N�:��;���YR���Y��e�2���7H��Q-f#T�M��I����#��A�|�����:s=O�,��A	Q[Q�]�����_��/Z�#��h���GYF��Bu�^a��$�X�Ha���'�u��uR��\G��n�:ab�X})AyY���%!�����%�#	�Yr�������<)��l��,����������2�1b��m��9���~e'��*�n]T�o�����������+�/M��D��~�������sv	cE,<�tE��������`���<|(\M�W�_rn����9>�erX����w����l�=�x����)�L0d?N_��/���Xy
�b�,YK�nG��Q�QFT�@�_;Wi�\^��Q�0��9#���!���f��d<SQ����=�b8b�4M�;�Y�����J�R8�	�$�����]]������t/IH%�0��N�Q��<cq������T��/5k��h�Qn6�3�:���S�4�������E���^T+�(�_RO���NO�;�S��(��zp;5��pZ�?�[f��@
��}&��J ���m?�w�a,c�337(�������8 }2�5�W�Tl%q��~���It_��l�-��
&��{KG�3�-|�3��g]������������t?Y���Q�76xN5���3Y�5EyA��������Z�CI����0�<V
endstream
endobj
134 0 obj
1598
endobj
132 0 obj
<<
/Type /Page
/Parent 126 0 R
/Resources <<
/Font 135 0 R
/ProcSet 2 0 R
>>
/Contents 133 0 R
>>
endobj
135 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F4 17 0 R
/F6 27 0 R
>>
endobj
137 0 obj
<<
/Length 138 0 R
/Filter /FlateDecode 
>>
stream
H��W]S�J���)�H�Q4���d � 	v��
<y�u�%�H�����3�#Kv����J���������>$>���������O�5&��}���'xF�9��0��J��;2������������������a��F���m�g��%�(��:/��$�N��0��6�|��������Wx�5�����s+�R��`Y�Y�9K��1�de���Iu�Euu:7=�)�q�3�H���=@�{n�x�W�6���I���2�S���k���"�V���n����c��G��W>KhS�0��w�z���'�d�a�h�Ks�a�$��1�7��� H�/�Gi,�/��u]�>�&'�8�7���Q���b��l�9������y��G���|���)|3(*�$+G�<����UVU^4�qp�/GyU��qQ��|e6��\JfGJg�
>Y%6��U�(�d��g2L���)gs�� �u�{�����G�S8]����Gh[+y[ku���@�q�a�����^0���[��aY�
��f3dM�)�{��@3���Q��7E���2XV��U�>~L����sU��*J�zB~\����M�b��=�q(�O�e�V� �0
���������fy=��
�=�E���4�����d���d-�U9N�0�����������?��y��P{fL�d���
F��op��2	�@�	������{d>�Zw��(��mB�DB����vk��n���j��@ ��N��s
�"��f�>�J��e�\x��?9��"u	~���t�k��<�E<�/]�5@}�g|�(�15h�f��|<A���zL���Kb�W�%���_K�\N���I�h���]��_N3�%<[ CQ��71�E�	~�B�A���$�%Vl���l�������	v�c�:~x�����#~��8�|_b�~D1*�����z�[3�1��R�#��(��c[E�\6}4�U*��Y
�b��iZ��d�X3/�}Ue�`
}���r���������"���������s��'h�>�����u����7:�R'���3)��������7��t������>���N��Jt�����#M�>|�`�������x�\`U�P�E�iv�\��S9K�#�;�����c4#�.��b��
$^{�M����P'������,tp���p��d��K(�M�O��������&��GR��9���t:�S���,%p��&��#����E�`b�Bm5�K7e=���P����/�$�+�9��<$wB%	24��1}��`D���S�(�h��<WUM>5hM�d��\�4��`?O%N5�]��}���=������d�/��1����������A_��9:��T��������s��~1$ o�
���Z�'p���.���I��!������]���_+�^�V�����[9���L�������*t#t/��UN#�@.���Oa���d17�C��`�=t���,p�d�T�/�W���=��P8�&X�(�e�h�aFpE�q�
b4���,\��x&a~����TzQ���q�;���c�����w
���T�R!�q�<�3�.�����#������0|h�������i(��m!��thA��R��bZ�s��T�"�&��co[#�XQ�[�k'
\�l=������
R�}�
�\:�t���Kjd�G��$����{�/����26��x�|�j�W�Aa`s���%�I��W���
�?)�r��������G>{�1P0��a{����MEAm���+��m4�H����8����|`�IH�����������X�>�����d�w�}98T��������I����3f��n��![��)�8x���kZC���������������Q;��\o�ic�W���N3�]�P�5HA�U'���J�U�m�o:"����"I:-�L��a�8wl	�
��� ����%�}U��5���u��~�N�fF/�I#9�K�/�m<����OX|����A���lI���p�|�g����L���e��`v�?5Kp�,��JH����$l�'�N�YB*���1~9� ;����6A�(h��[�J}�h��&!�Qz��_� _�Dv�K�>�c��8����0�`U�D���bQOPK���� 8��v��&���}�5%����>l�@c��^����}�%�N�u�q���+�on�~��#�m�#���J������#�Cd����
�S���g�a\wEf�"�\+abd6f�@V�����G���>�d�V2{�5Q��W��Z�#�{��f���d�E�/���y�o���E��+PT�~A��H���@����]ZO����v62�y$&�����`�g�g��R%l�9h`���DfycSX���J�������"jn��$r�����a�G-sv!�L�h/��3a�����1��6HY�}�,
����|ql��!^������*��KkE>*��>]��bXJ�%q:��������\s�(� ,{�!���p���C�:�3�1��F��O�S0��G
��Hc�e�����m������]D�0�������iN�h�;�l����}���M]���Bf��{�9����N�,=��2��e
�W��S�\*�;j�TVj[���	?�L�������U�Z{������&�~�n� ���	VM&�$�S�_���,?�������h^�i�����lq %�����%�~/�W}��].�a���$�e�0	�	�������l(���q������z����!��g��=��K���dD��������E����I���/��?7������F��Uir
|i^��>�
����|�fIU�[W���oq>�{u���6%�;�������
��`��l
t���$��nhm���=��Hg�fXN�@L���l+^��f��e8��4��~�Fiu@�����M!i�#]��;J��$�����7������+P�*���c16���w�,:������@q0��
����L���^�}����BU��L#�G��gWasrn��,de��z%5������Q��*���lI*��*HW	�'�c*�
�N���=)g�y�������N ������R��
t��{9G�AX� �Y���K5�1v�0,����>�L
endstream
endobj
138 0 obj
3234
endobj
136 0 obj
<<
/Type /Page
/Parent 126 0 R
/Resources <<
/Font 139 0 R
/ProcSet 2 0 R
>>
/Contents 137 0 R
>>
endobj
139 0 obj
<<
/F0 6 0 R
/F1 8 0 R
/F2 10 0 R
/F3 12 0 R
/F4 17 0 R
/F6 27 0 R
>>
endobj
141 0 obj
<<
/Length 142 0 R
/Filter /FlateDecode 
>>
stream
H��V�R�H��CF��Y�"�������i��������.9$����E��k���{�u��d���,X�M��j	���=�,r��3?��>���	���Y���_9������8nMD4����vE��~���V�l
���b�.JmmQ":c$$�[����#���Q�:��pg~����a����j��d#�d�jg�&]�o&j�e:��4!�a��6��	Z�i��+��?�A����W^��;erM�!*,����6��X��HA$��`��c|o�M��]%Uuk���[V���(���=
N&i��.D��W\�|Z���f�PV0_�J�{v�#$X$��#~�7���*���S�q����JU����������*���C�@�-l}$���]�uD}�w�C�Y*Bg�0=\M*,�"v�Ut!��wr��}��qIc�C�Sag6��p�%�>hbMIWn.R_�?�m������_�L��R����m�]���
)u�J�����a���8b���+�Q�
���:��$���>��!{c��$qD?qja���Ow�J������|�suVn��FU_�����j�<�
��?�d������!m��������b
W*_)��M�#�]��<�t���R4��7�YYm��F���v~~�E�U�dY�P��h:���2g>X���������^��)�R�$���Z+��R���Rl$br;�1.HMX�.0���3|w{t�#l	�#��_�5���Z���<�A_:S
������D0�.���@���	�o���L���~b7��,f��,J]
���wz�I�f��d^��.��������K$JZYUR�2�z���\���?���Q��:7����p�t����N������q�����}D;����Odg!����yW��v�)?�AI�
u�0�>�����v������j��j��e������X0�_��Q�i���R<x�&����]�2�:�)����{�-�����S���K����B�M���&�����G�����3�-z���<��=��� ��tf���<���6��I�>�)��2g��Q�9�,c�����X�~�|1�8���
endstream
endobj
142 0 obj
1100
endobj
140 0 obj
<<
/Type /Page
/Parent 126 0 R
/Resources <<
/Font <<
/F0 6 0 R 
/F1 8 0 R 
/F2 10 0 R 
/F3 12 0 R 
/F6 27 0 R 
>>
/ProcSet 2 0 R
>>
/Contents 141 0 R
>>
endobj
6 0 obj
<<
/Type /Font
/Subtype /TrueType
/Name /F0
/BaseFont /Arial
/FirstChar 32
/LastChar 255
/Widths [ 278 278 355 556 556 889 667 191 333 333 389 584 278 333 278 278 
556 556 556 556 556 556 556 556 556 556 278 278 584 584 584 556 
1015 667 667 722 722 667 611 778 722 278 500 667 556 833 722 778 
667 778 722 667 611 722 667 944 667 667 611 278 278 278 469 556 
333 556 556 500 556 556 278 556 556 222 222 500 222 833 556 556 
556 556 333 500 278 556 500 722 500 500 500 334 260 334 584 750 
556 750 222 556 333 1000 556 556 333 1000 667 333 1000 750 611 750 
750 222 222 333 333 350 556 1000 333 1000 500 333 944 750 500 667 
278 333 556 556 556 556 260 556 333 737 370 556 584 333 737 552 
400 549 333 333 333 576 537 278 333 333 365 556 834 834 834 611 
667 667 667 667 667 667 1000 722 667 667 667 667 278 278 278 278 
722 722 778 778 778 778 778 584 778 722 722 722 722 667 667 611 
556 556 556 556 556 556 889 500 556 556 556 556 278 278 278 278 
556 556 556 556 556 556 556 549 611 556 556 556 556 500 556 500 
]
/Encoding /WinAnsiEncoding
/FontDescriptor 7 0 R
>>
endobj
7 0 obj
<<
/Type /FontDescriptor
/FontName /Arial
/Flags 32
/FontBBox [ -250 -212 1216 1000 ]
/MissingWidth 277
/StemV 80
/StemH 80
/ItalicAngle 0
/CapHeight 905
/XHeight 453
/Ascent 905
/Descent -212
/Leading 150
/MaxWidth 1013
/AvgWidth 441
>>
endobj
8 0 obj
<<
/Type /Font
/Subtype /TrueType
/Name /F1
/BaseFont /Arial,Italic
/FirstChar 32
/LastChar 255
/Widths [ 278 278 355 556 556 889 667 191 333 333 389 584 278 333 278 278 
556 556 556 556 556 556 556 556 556 556 278 278 584 584 584 556 
1015 667 667 722 722 667 611 778 722 278 500 667 556 833 722 778 
667 778 722 667 611 722 667 944 667 667 611 278 278 278 469 556 
333 556 556 500 556 556 278 556 556 222 222 500 222 833 556 556 
556 556 333 500 278 556 500 722 500 500 500 334 260 334 584 750 
556 750 222 556 333 1000 556 556 333 1000 667 333 1000 750 611 750 
750 222 222 333 333 350 556 1000 333 1000 500 333 944 750 500 667 
278 333 556 556 556 556 260 556 333 737 370 556 584 333 737 552 
400 549 333 333 333 576 537 278 333 333 365 556 834 834 834 611 
667 667 667 667 667 667 1000 722 667 667 667 667 278 278 278 278 
722 722 778 778 778 778 778 584 778 722 722 722 722 667 667 611 
556 556 556 556 556 556 889 500 556 556 556 556 278 278 278 278 
556 556 556 556 556 556 556 549 611 556 556 556 556 500 556 500 
]
/Encoding /WinAnsiEncoding
/FontDescriptor 9 0 R
>>
endobj
9 0 obj
<<
/Type /FontDescriptor
/FontName /Arial,Italic
/Flags 96
/FontBBox [ -250 -212 1216 1000 ]
/MissingWidth 277
/StemV 80
/StemH 80
/ItalicAngle -11
/CapHeight 905
/XHeight 453
/Ascent 905
/Descent -212
/Leading 150
/MaxWidth 1013
/AvgWidth 441
>>
endobj
10 0 obj
<<
/Type /Font
/Subtype /TrueType
/Name /F2
/BaseFont /TimesNewRoman,Bold
/FirstChar 32
/LastChar 255
/Widths [ 250 333 555 500 500 1000 833 278 333 333 500 570 250 333 250 278 
500 500 500 500 500 500 500 500 500 500 333 333 570 570 570 500 
930 722 667 722 722 667 611 778 778 389 500 778 667 944 722 778 
611 778 722 556 667 722 722 1000 722 722 667 333 278 333 581 500 
333 500 556 444 556 444 333 500 556 278 333 556 278 833 556 500 
556 556 444 389 333 556 500 722 500 500 444 394 220 394 520 778 
500 778 333 500 500 1000 500 500 333 1000 556 333 1000 778 667 778 
778 333 333 500 500 350 500 1000 333 1000 389 333 722 778 444 722 
250 333 500 500 500 500 220 500 333 747 300 500 570 333 747 500 
400 549 300 300 333 576 540 250 333 300 330 500 750 750 750 500 
722 722 722 722 722 722 1000 722 667 667 667 667 389 389 389 389 
722 722 778 778 778 778 778 570 778 722 722 722 722 722 611 556 
500 500 500 500 500 500 722 444 444 444 444 444 278 278 278 278 
500 556 500 500 500 500 500 549 500 556 556 556 556 500 556 500 
]
/Encoding /WinAnsiEncoding
/FontDescriptor 11 0 R
>>
endobj
11 0 obj
<<
/Type /FontDescriptor
/FontName /TimesNewRoman,Bold
/Flags 16418
/FontBBox [ -250 -216 1172 1000 ]
/MissingWidth 325
/StemV 136
/StemH 136
/ItalicAngle 0
/CapHeight 891
/XHeight 446
/Ascent 891
/Descent -216
/Leading 149
/MaxWidth 977
/AvgWidth 427
>>
endobj
12 0 obj
<<
/Type /Font
/Subtype /TrueType
/Name /F3
/BaseFont /TimesNewRoman
/FirstChar 32
/LastChar 255
/Widths [ 250 333 408 500 500 833 778 180 333 333 500 564 250 333 250 278 
500 500 500 500 500 500 500 500 500 500 278 278 564 564 564 444 
921 722 667 667 722 611 556 722 722 333 389 722 611 889 722 722 
556 722 667 556 611 722 722 944 722 722 611 333 278 333 469 500 
333 444 500 444 500 444 333 500 500 278 278 500 278 778 500 500 
500 500 333 389 278 500 500 722 500 500 444 480 200 480 541 778 
500 778 333 500 444 1000 500 500 333 1000 556 333 889 778 611 778 
778 333 333 444 444 350 500 1000 333 980 389 333 722 778 444 722 
250 333 500 500 500 500 200 500 333 760 276 500 564 333 760 500 
400 549 300 300 333 576 453 250 333 300 310 500 750 750 750 444 
722 722 722 722 722 722 889 667 611 611 611 611 333 333 333 333 
722 722 722 722 722 722 722 564 722 722 722 722 722 722 556 500 
444 444 444 444 444 444 667 444 444 444 444 444 278 278 278 278 
500 500 500 500 500 500 500 549 500 500 500 500 500 500 500 500 
]
/Encoding /WinAnsiEncoding
/FontDescriptor 13 0 R
>>
endobj
13 0 obj
<<
/Type /FontDescriptor
/FontName /TimesNewRoman
/Flags 34
/FontBBox [ -250 -216 1158 1000 ]
/MissingWidth 321
/StemV 73
/StemH 73
/ItalicAngle 0
/CapHeight 891
/XHeight 446
/Ascent 891
/Descent -216
/Leading 149
/MaxWidth 965
/AvgWidth 401
>>
endobj
17 0 obj
<<
/Type /Font
/Subtype /TrueType
/Name /F4
/BaseFont /Arial,Bold
/FirstChar 32
/LastChar 255
/Widths [ 278 333 474 556 556 889 722 238 333 333 389 584 278 333 278 278 
556 556 556 556 556 556 556 556 556 556 333 333 584 584 584 611 
975 722 722 722 722 667 611 778 722 278 556 722 611 833 722 778 
667 778 722 667 611 722 667 944 667 667 611 333 278 333 584 556 
333 556 611 556 611 556 333 611 611 278 278 556 278 889 611 611 
611 611 389 556 333 611 556 778 556 556 500 389 280 389 584 750 
556 750 278 556 500 1000 556 556 333 1000 667 333 1000 750 611 750 
750 278 278 500 500 350 556 1000 333 1000 556 333 944 750 500 667 
278 333 556 556 556 556 280 556 333 737 370 556 584 333 737 552 
400 549 333 333 333 576 556 278 333 333 365 556 834 834 834 611 
722 722 722 722 722 722 1000 722 667 667 667 667 278 278 278 278 
722 722 778 778 778 778 778 584 778 722 722 722 722 667 667 611 
556 556 556 556 556 556 889 556 556 556 556 556 278 278 278 278 
611 611 611 611 611 611 611 549 611 611 611 611 611 556 611 556 
]
/Encoding /WinAnsiEncoding
/FontDescriptor 18 0 R
>>
endobj
18 0 obj
<<
/Type /FontDescriptor
/FontName /Arial,Bold
/Flags 16416
/FontBBox [ -250 -212 1165 1000 ]
/MissingWidth 323
/StemV 153
/StemH 153
/ItalicAngle 0
/CapHeight 905
/XHeight 453
/Ascent 905
/Descent -212
/Leading 150
/MaxWidth 971
/AvgWidth 479
>>
endobj
21 0 obj
<<
/Type /Font
/Subtype /TrueType
/Name /F5
/BaseFont /CourierNew
/FirstChar 32
/LastChar 255
/Widths [ 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
]
/Encoding /WinAnsiEncoding
/FontDescriptor 22 0 R
>>
endobj
22 0 obj
<<
/Type /FontDescriptor
/FontName /CourierNew
/Flags 34
/FontBBox [ -250 -300 767 1000 ]
/MissingWidth 639
/StemV 109
/StemH 109
/ItalicAngle 0
/CapHeight 833
/XHeight 417
/Ascent 833
/Descent -300
/Leading 133
/MaxWidth 639
/AvgWidth 600
>>
endobj
27 0 obj
<<
/Type /Font
/Subtype /TrueType
/Name /F6
/BaseFont /TimesNewRoman,Italic
/FirstChar 32
/LastChar 255
/Widths [ 250 333 420 500 500 833 778 214 333 333 500 675 250 333 250 278 
500 500 500 500 500 500 500 500 500 500 333 333 675 675 675 500 
920 611 611 667 722 611 611 722 722 333 444 667 556 833 667 722 
611 722 611 500 556 722 611 833 611 556 556 389 278 389 422 500 
333 500 500 444 500 444 278 500 500 278 278 444 278 722 500 500 
500 500 389 389 278 500 444 667 444 444 389 400 275 400 541 778 
500 778 333 500 556 889 500 500 333 1000 500 333 944 778 556 778 
778 333 333 556 556 350 500 889 333 980 389 333 667 778 389 556 
250 389 500 500 500 500 275 500 333 760 276 500 675 333 760 500 
400 549 300 300 333 576 523 250 333 300 310 500 750 750 750 500 
611 611 611 611 611 611 889 667 611 611 611 611 333 333 333 333 
722 667 722 722 722 722 722 675 722 722 722 722 722 556 611 500 
500 500 500 500 500 500 667 444 444 444 444 444 278 278 278 278 
500 500 500 500 500 500 500 549 500 500 500 500 500 444 500 444 
]
/Encoding /WinAnsiEncoding
/FontDescriptor 28 0 R
>>
endobj
28 0 obj
<<
/Type /FontDescriptor
/FontName /TimesNewRoman,Italic
/Flags 98
/FontBBox [ -250 -216 1165 1000 ]
/MissingWidth 378
/StemV 73
/StemH 73
/ItalicAngle -11
/CapHeight 891
/XHeight 446
/Ascent 891
/Descent -216
/Leading 149
/MaxWidth 971
/AvgWidth 402
>>
endobj
33 0 obj
<<
/Type /Font
/Subtype /TrueType
/Name /F7
/BaseFont /CourierNew,Italic
/FirstChar 32
/LastChar 255
/Widths [ 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
]
/Encoding /WinAnsiEncoding
/FontDescriptor 34 0 R
>>
endobj
34 0 obj
<<
/Type /FontDescriptor
/FontName /CourierNew,Italic
/Flags 98
/FontBBox [ -250 -300 740 1000 ]
/MissingWidth 617
/StemV 109
/StemH 109
/ItalicAngle -11
/CapHeight 833
/XHeight 417
/Ascent 833
/Descent -300
/Leading 133
/MaxWidth 617
/AvgWidth 600
>>
endobj
42 0 obj
<<
/Type /Font
/Subtype /TrueType
/Name /F8
/BaseFont /CourierNew,Bold
/FirstChar 32
/LastChar 255
/Widths [ 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
]
/Encoding /WinAnsiEncoding
/FontDescriptor 43 0 R
>>
endobj
43 0 obj
<<
/Type /FontDescriptor
/FontName /CourierNew,Bold
/Flags 16418
/FontBBox [ -250 -300 713 1000 ]
/MissingWidth 594
/StemV 191
/StemH 191
/ItalicAngle 0
/CapHeight 833
/XHeight 417
/Ascent 833
/Descent -300
/Leading 133
/MaxWidth 594
/AvgWidth 600
>>
endobj
68 0 obj
<<
/Type /Font
/Subtype /TrueType
/Name /F9
/BaseFont /TimesNewRoman,BoldItalic
/FirstChar 32
/LastChar 255
/Widths [ 250 389 555 500 500 833 778 278 333 333 500 570 250 333 250 278 
500 500 500 500 500 500 500 500 500 500 333 333 570 570 570 500 
832 667 667 667 722 667 667 722 778 389 500 667 611 889 722 722 
611 722 667 556 611 722 667 889 667 611 611 333 278 333 570 500 
333 500 500 444 500 444 333 500 556 278 278 500 278 778 556 500 
500 500 389 389 278 556 444 667 500 444 389 348 220 348 570 778 
500 778 333 500 500 1000 500 500 333 1000 556 333 944 778 611 778 
778 333 333 500 500 350 500 1000 333 1000 389 333 722 778 389 611 
250 389 500 500 500 500 220 500 333 747 266 500 606 333 747 500 
400 549 300 300 333 576 500 250 333 300 300 500 750 750 750 500 
667 667 667 667 667 667 944 667 667 667 667 667 389 389 389 389 
722 722 722 722 722 722 722 570 722 722 722 722 722 611 611 500 
500 500 500 500 500 500 722 444 444 444 444 444 278 278 278 278 
500 556 500 500 500 500 500 549 500 556 556 556 556 444 500 444 
]
/Encoding /WinAnsiEncoding
/FontDescriptor 69 0 R
>>
endobj
69 0 obj
<<
/Type /FontDescriptor
/FontName /TimesNewRoman,BoldItalic
/Flags 16482
/FontBBox [ -250 -216 1178 1000 ]
/MissingWidth 327
/StemV 131
/StemH 131
/ItalicAngle -11
/CapHeight 891
/XHeight 446
/Ascent 891
/Descent -216
/Leading 149
/MaxWidth 982
/AvgWidth 412
>>
endobj
70 0 obj
<<
/Type /Font
/Subtype /TrueType
/Name /F10
/BaseFont /BookmanOldStyle
/FirstChar 32
/LastChar 255
/Widths [ 320 300 380 600 620 900 800 220 300 300 440 600 320 400 320 600 
620 620 620 620 620 620 620 620 620 620 320 320 600 600 600 540 
820 680 740 740 800 720 640 800 800 340 600 720 600 920 740 800 
620 800 720 660 620 780 700 960 720 640 640 300 600 300 600 500 
340 580 620 520 620 520 320 540 660 300 300 620 300 940 660 560 
620 580 440 520 380 680 520 780 560 540 480 280 600 280 600 750 
620 750 220 620 400 1000 540 540 420 1280 660 240 1240 750 640 750 
750 220 220 400 400 460 500 1000 440 980 520 240 900 750 480 640 
340 300 620 620 600 620 600 520 420 740 420 360 600 400 740 500 
400 549 372 372 340 576 600 320 320 372 420 360 930 930 930 540 
680 680 680 680 680 680 1260 740 720 720 720 720 340 340 340 340 
800 740 800 800 800 800 800 600 800 780 780 780 780 640 620 660 
580 580 580 580 580 580 860 520 520 520 520 520 300 300 300 300 
560 660 560 560 560 560 560 549 560 680 680 680 680 540 620 540 
]
/Encoding /WinAnsiEncoding
/FontDescriptor 71 0 R
>>
endobj
71 0 obj
<<
/Type /FontDescriptor
/FontName /BookmanOldStyle
/Flags 34
/FontBBox [ -250 -231 1463 1000 ]
/MissingWidth 286
/StemV 67
/StemH 67
/ItalicAngle 0
/CapHeight 894
/XHeight 447
/Ascent 894
/Descent -231
/Leading 125
/MaxWidth 1219
/AvgWidth 492
>>
endobj
81 0 obj
<<
/Type /Font
/Subtype /TrueType
/Name /F11
/BaseFont /Symbol,Italic
/FirstChar 30
/LastChar 255
/Widths [ 600 600 250 333 713 500 549 833 778 439 333 333 500 549 250 549 
250 278 500 500 500 500 500 500 500 500 500 500 278 278 549 549 
549 444 549 722 667 722 612 611 763 603 722 333 631 722 686 889 
722 722 768 741 556 592 611 690 439 768 645 795 611 333 863 333 
658 500 500 631 549 549 494 439 521 411 603 329 603 549 549 576 
521 549 549 521 549 603 439 576 713 686 493 686 494 480 200 480 
549 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 620 247 549 167 713 500 753 753 753 753 1042 987 603 
987 603 400 549 411 549 549 713 494 460 549 549 549 549 1000 603 
1000 658 823 686 795 987 768 768 823 768 768 713 713 713 713 713 
713 713 768 713 790 790 890 823 549 250 713 603 603 1042 987 603 
987 603 494 329 790 790 786 713 384 384 384 384 384 384 494 494 
494 494 600 329 274 686 686 686 384 384 384 384 384 384 494 494 
494 600 ]
/FontDescriptor 82 0 R
>>
endobj
82 0 obj
<<
/Type /FontDescriptor
/FontName /Symbol,Italic
/Flags 70
/FontBBox [ -250 -220 1255 1005 ]
/MissingWidth 334
/StemV 109
/StemH 109
/ItalicAngle -11
/CapHeight 1005
/XHeight 503
/Ascent 1005
/Descent -220
/Leading 225
/MaxWidth 1046
/AvgWidth 600
>>
endobj
112 0 obj
<<
/Type /Font
/Subtype /TrueType
/Name /F12
/BaseFont /Symbol
/FirstChar 30
/LastChar 255
/Widths [ 600 600 250 333 713 500 549 833 778 439 333 333 500 549 250 549 
250 278 500 500 500 500 500 500 500 500 500 500 278 278 549 549 
549 444 549 722 667 722 612 611 763 603 722 333 631 722 686 889 
722 722 768 741 556 592 611 690 439 768 645 795 611 333 863 333 
658 500 500 631 549 549 494 439 521 411 603 329 603 549 549 576 
521 549 549 521 549 603 439 576 713 686 493 686 494 480 200 480 
549 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 600 
600 600 600 620 247 549 167 713 500 753 753 753 753 1042 987 603 
987 603 400 549 411 549 549 713 494 460 549 549 549 549 1000 603 
1000 658 823 686 795 987 768 768 823 768 768 713 713 713 713 713 
713 713 768 713 790 790 890 823 549 250 713 603 603 1042 987 603 
987 603 494 329 790 790 786 713 384 384 384 384 384 384 494 494 
494 494 600 329 274 686 686 686 384 384 384 384 384 384 494 494 
494 600 ]
/FontDescriptor 113 0 R
>>
endobj
113 0 obj
<<
/Type /FontDescriptor
/FontName /Symbol
/Flags 6
/FontBBox [ -250 -220 1250 1005 ]
/MissingWidth 333
/StemV 109
/StemH 109
/ItalicAngle 0
/CapHeight 1005
/XHeight 503
/Ascent 1005
/Descent -220
/Leading 225
/MaxWidth 1042
/AvgWidth 600
>>
endobj
2 0 obj
[ /PDF /Text  ]
endobj
5 0 obj
<<
/Kids [4 0 R 16 0 R 24 0 R 30 0 R 36 0 R 39 0 R ]
/Count 6
/Type /Pages
/Parent 143 0 R
>>
endobj
46 0 obj
<<
/Kids [45 0 R 50 0 R 54 0 R 57 0 R 61 0 R 65 0 R ]
/Count 6
/Type /Pages
/Parent 143 0 R
>>
endobj
74 0 obj
<<
/Kids [73 0 R 78 0 R 84 0 R 88 0 R 92 0 R 96 0 R ]
/Count 6
/Type /Pages
/Parent 143 0 R
>>
endobj
101 0 obj
<<
/Kids [100 0 R 104 0 R 107 0 R 111 0 R 117 0 R 121 0 R ]
/Count 6
/Type /Pages
/Parent 143 0 R
>>
endobj
126 0 obj
<<
/Kids [125 0 R 129 0 R 132 0 R 136 0 R 140 0 R ]
/Count 5
/Type /Pages
/Parent 143 0 R
>>
endobj
143 0 obj
<<
/Kids [5 0 R 46 0 R 74 0 R 101 0 R 126 0 R ]
/Count 29
/Type /Pages
/MediaBox [ 0 0 612 792 ]
>>
endobj
1 0 obj
<<
/Creator <FEFF00530079006E007400680065007400690063004400610074006100470065006E002E0064006F00630020002D0020004D006900630072006F0073006F0066007400200057006F00720064>
/CreationDate (D:20000809185125)
/Title <FEFF00530079006E007400680065007400690063004400610074006100470065006E002E005000440046>
/Author <FEFF0047007200610079>
/Producer (Acrobat PDFWriter 4.0 for Windows NT)
>>
endobj
3 0 obj
<<
/Pages 143 0 R
/Type /Catalog
>>
endobj
xref
0 144
0000000000 65535 f 
0000104682 00000 n 
0000103975 00000 n 
0000105074 00000 n 
0000003365 00000 n 
0000104006 00000 n 
0000086366 00000 n 
0000087451 00000 n 
0000087704 00000 n 
0000088796 00000 n 
0000089058 00000 n 
0000090159 00000 n 
0000090430 00000 n 
0000091521 00000 n 
0000000019 00000 n 
0000003344 00000 n 
0000005182 00000 n 
0000091782 00000 n 
0000092873 00000 n 
0000003519 00000 n 
0000005161 00000 n 
0000093136 00000 n 
0000094221 00000 n 
0000005292 00000 n 
0000008955 00000 n 
0000005378 00000 n 
0000008934 00000 n 
0000094480 00000 n 
0000095576 00000 n 
0000009065 00000 n 
0000013504 00000 n 
0000009162 00000 n 
0000013483 00000 n 
0000095846 00000 n 
0000096938 00000 n 
0000013614 00000 n 
0000015204 00000 n 
0000013700 00000 n 
0000015183 00000 n 
0000018599 00000 n 
0000015371 00000 n 
0000018578 00000 n 
0000097206 00000 n 
0000098296 00000 n 
0000018709 00000 n 
0000022889 00000 n 
0000104115 00000 n 
0000018828 00000 n 
0000022868 00000 n 
0000023000 00000 n 
0000025660 00000 n 
0000023097 00000 n 
0000025639 00000 n 
0000025771 00000 n 
0000026411 00000 n 
0000025868 00000 n 
0000026391 00000 n 
0000029043 00000 n 
0000026567 00000 n 
0000029022 00000 n 
0000029154 00000 n 
0000032276 00000 n 
0000029240 00000 n 
0000032255 00000 n 
0000032387 00000 n 
0000035535 00000 n 
0000032495 00000 n 
0000035514 00000 n 
0000098563 00000 n 
0000099666 00000 n 
0000099945 00000 n 
0000101041 00000 n 
0000035646 00000 n 
0000039013 00000 n 
0000104226 00000 n 
0000035777 00000 n 
0000038992 00000 n 
0000039124 00000 n 
0000042762 00000 n 
0000039243 00000 n 
0000042741 00000 n 
0000101305 00000 n 
0000102379 00000 n 
0000042873 00000 n 
0000045728 00000 n 
0000042971 00000 n 
0000045707 00000 n 
0000045839 00000 n 
0000048450 00000 n 
0000045959 00000 n 
0000048429 00000 n 
0000048561 00000 n 
0000051745 00000 n 
0000048659 00000 n 
0000051724 00000 n 
0000051856 00000 n 
0000054026 00000 n 
0000051976 00000 n 
0000054005 00000 n 
0000054137 00000 n 
0000058511 00000 n 
0000104337 00000 n 
0000054234 00000 n 
0000058489 00000 n 
0000061973 00000 n 
0000058670 00000 n 
0000061951 00000 n 
0000064609 00000 n 
0000062144 00000 n 
0000064587 00000 n 
0000064724 00000 n 
0000067983 00000 n 
0000102647 00000 n 
0000103716 00000 n 
0000064822 00000 n 
0000067961 00000 n 
0000068098 00000 n 
0000072575 00000 n 
0000068198 00000 n 
0000072553 00000 n 
0000072690 00000 n 
0000075990 00000 n 
0000072790 00000 n 
0000075968 00000 n 
0000076105 00000 n 
0000078705 00000 n 
0000104455 00000 n 
0000076203 00000 n 
0000078683 00000 n 
0000079394 00000 n 
0000078876 00000 n 
0000079373 00000 n 
0000081253 00000 n 
0000079553 00000 n 
0000081231 00000 n 
0000081368 00000 n 
0000084791 00000 n 
0000081455 00000 n 
0000084769 00000 n 
0000084906 00000 n 
0000086195 00000 n 
0000084993 00000 n 
0000086173 00000 n 
0000104565 00000 n 
trailer
<<
/Size 144
/Root 3 0 R
/Info 1 0 R
/ID [<50565c3198381d503cd50caa99ad6817><50565c3198381d503cd50caa99ad6817>]
>>
startxref
105125
%%EOF
#30Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Alik Khilazhev (#29)
Re: [WIP] Zipfian distribution in pgbench

Hello,

Is this bias expected from the drawing method, say because it is
approximated and the approximation is weak at some points, or is there
an issue with its implementation, says some shift which gets smoothed
down for higher indexes?

I have checked paper where such implementation was proposed and there
theta allowed only on range between 0 and 1. It seems like it is not
guaranteed that it should work well when theta is more than 1.

Ok.

I see a significant issue with having a random_zipfian function which does
not really return a zipfian distribution under some parameter values. If
there is no better alternative, I would suggest to restrict the parameter
for values between 0 and 1, or to find a better approximation for theta >=
0.

I am attaching paper, see page 23.

Thanks for the paper. It reminds me that I intended to propose a
parametric pseudo-random permutation for pgbench, some day.

--
Fabien.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#31Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Alik Khilazhev (#27)
Re: [WIP] Zipfian distribution in pgbench

Hello Alik,

I am attaching patch v3.

Among other things I fixed small typo in description of
random_exponential function in pgbench.sgml file.

Ok. Probably this typo should be committed separatly and independently.

A few comments about v3:

Patch applies cleanly, compiles, works.

About the maths: As already said, I'm not at ease with a random_zipfian
function which does not display a (good) zipfian distribution. At the
minimum the documentation should be clear about the approximations implied
depending on the parameter value.

In the litterature the theta parameter seems to be often called alpha
or s (eg see https://en.wikipedia.org/wiki/Zipf%27s_law). I would suggest to
stick to "s" instead of "theta"?

About the code: looks simpler than the previous version, which is good.

Double constants should be used when the underlying type is a double,
instead of relying on implicit int to double promotion (0 -> 0.0, 1 -> 1.0).

Functions zipfZeta(n, theta) does not really computes the zeta(n) function,
so I think that a better name should be chosen. It seems to compute
H_{n,theta}, the generalized harmonic number. Idem "thetan" field in struct.

The handling of cache overflow by randomly removing one entry looks like
a strange idea. Rather remove the oldest entry?

ISTM that it should print a warning once if the cache array overflows as
performance would drop heavily.

If the zipf cache is constant size, there is no point in using dynamic
allocation, just declare an array...

Comments need updating: eg "theta parameter of previous execution" which
dates back when there was only one value.

There should be a comment to explain that the structure is in the thread
for thread safety.

There should be non regression tests somehow. If the "improve pgbench
tap test infrastructure" get through, things should be added there.

--
Fabien.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#32Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Fabien COELHO (#31)
1 attachment(s)
Re: [WIP] Zipfian distribution in pgbench

Hello Fabien,

I am attaching patch v4.

On 19 Jul 2017, at 17:21, Fabien COELHO <coelho@cri.ensmp.fr> wrote:

About the maths: As already said, I'm not at ease with a random_zipfian function which does not display a (good) zipfian distribution. At the minimum the documentation should be clear about the approximations implied depending on the parameter value.

I add one more sentence to documentation to emphasize that degree of proximity depends on parameter . And also I made restriction on parameter, now it can be only in range (0; 1)

In the litterature the theta parameter seems to be often called alpha
or s (eg see https://en.wikipedia.org/wiki/Zipf%27s_law). I would suggest to
stick to "s" instead of "theta”?

I have renamed it to “s”.

Functions zipfZeta(n, theta) does not really computes the zeta(n) function,
so I think that a better name should be chosen. It seems to compute
H_{n,theta}, the generalized harmonic number. Idem "thetan" field in struct.

Renamed zipfZeta to zipfGeneralizedHarmonic, zetan to harmonicn.

The handling of cache overflow by randomly removing one entry looks like
a strange idea. Rather remove the oldest entry?

Replaced with Least Recently Used replacement algorithm.

ISTM that it should print a warning once if the cache array overflows as performance would drop heavily.

Now it prints warning message if array overflowed. To print message only one time, it uses global flag, which is available for all threads.
And theoretically message can be printed more than one time.
It could be solved easily using pg_atomic_test_set_flag() from src/include/port/atomics.h but it can not be used in pgbench because of following lines of code there:

#ifdef FRONTEND
#error "atomics.h may not be included from frontend code"
#endif

Or it can be fixed by using mutexes from pthread, but I think code become less readable and more complex in this case.
So, should I spend time on solving this issue?

If the zipf cache is constant size, there is no point in using dynamic allocation, just declare an array…

Fixed. Does ZIPF_CACHE_SIZE = 15 is ok?

There should be non regression tests somehow. If the "improve pgbench
tap test infrastructure" get through, things should be added there.

I will send tests later, as separate patch.


Thanks and Regards,
Alik Khilazhev
Postgres Professional:
http://www.postgrespro.com
The Russian Postgres Company

Attachments:

pgbench-zipf-04v.patchapplication/octet-stream; name=pgbench-zipf-04v.patch; x-unix-mode=0644Download
diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml
index 64b043b48a..6f8244e4b9 100644
--- a/doc/src/sgml/ref/pgbench.sgml
+++ b/doc/src/sgml/ref/pgbench.sgml
@@ -1014,6 +1014,14 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
        <entry>an integer between <literal>1</> and <literal>10</></>
       </row>
       <row>
+        <entry><literal><function>random_zipfian(<replaceable>lb</>, <replaceable>ub</>, <replaceable>parameter</>)</></></>
+        <entry>integer</>
+        <entry> Zipfian-distributed random integer in <literal>[lb, ub]</>,
+              see below</>
+        <entry><literal>random_zipfian(1, 10, 0.5)</></>
+        <entry>an integer between <literal>1</> and <literal>10</></>
+      </row>
+      <row>
        <entry><literal><function>sqrt(<replaceable>x</>)</></></>
        <entry>double</>
        <entry>square root</>
@@ -1094,6 +1102,24 @@ f(x) = PHI(2.0 * parameter * (x - mu) / (max - min + 1)) /
       of the Box-Muller transform.
      </para>
     </listitem>
+    <listitem>
+        <para>
+          <literal>random_zipfian</> generates only approximate distribution and degree of proximity depends on <replaceable>parameter</>.
+        </para>
+        <para>
+          <replaceable>parameter</> 
+          defines how skewed the distribution. A physical sense of this parameter
+          is following: probability of <literal>i</>th element <literal>f(i) = (1 / (i ^ parameter)) /
+          (H(N, parameter))</> where <literal>N</> is number of elements in range and <literal>H(N, parameter)</>
+          is <literal>N</>th generalized harmonic number. 
+        </para>
+        <para>
+          Intuitively, the larger the <replaceable>parameter</>, the more
+          frequently value values to the beginning of the interval are drawn. 
+          The closer to 0 <replaceable>parameter</> is, 
+          the flatter (more uniform) the access distribution.
+        </para>
+    </listitem>
    </itemizedlist>
 
   <para>
diff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.y
index b3a2d9bfd3..25d5ad48e5 100644
--- a/src/bin/pgbench/exprparse.y
+++ b/src/bin/pgbench/exprparse.y
@@ -191,6 +191,9 @@ static const struct
 	{
 		"random_exponential", 3, PGBENCH_RANDOM_EXPONENTIAL
 	},
+	{
+		"random_zipfian", 3, PGBENCH_RANDOM_ZIPFIAN
+	},
 	/* keep as last array element */
 	{
 		NULL, 0, 0
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 4d364a1db4..5059de8282 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -93,6 +93,8 @@ static int	pthread_join(pthread_t th, void **thread_return);
 #define LOG_STEP_SECONDS	5	/* seconds between log messages */
 #define DEFAULT_NXACTS	10		/* default nxacts */
 
+#define ZIPF_CACHE_SIZE	15		/* cache cells number */
+
 #define MIN_GAUSSIAN_PARAM		2.0 /* minimum parameter for gauss */
 
 int			nxacts = 0;			/* number of transactions per client */
@@ -334,6 +336,36 @@ typedef struct
 } CState;
 
 /*
+ * Cache cell for zipfian_random call
+ */
+typedef struct
+{
+	/* cell keys */
+	double		s;				/* s - parameter of zipfan_random function */
+	int64		n;				/* number of elements in range (max - min + 1) */
+
+	double		harmonicn;		/* zipfGeneralizedHarmonic(n, s) */
+	double		alpha;
+	double		beta;
+	double		eta;
+
+	uint64		last_touch_time;	/* time when cell was used last time */
+} ZipfCell;
+
+/*
+ * Zipf cache for zeta values
+ */
+typedef struct
+{
+	uint64		current_time;	/* counter for LRU cache replacement algorithm */
+
+	int			cells_inited;	/* number of filled cells */
+	ZipfCell	cells[ZIPF_CACHE_SIZE];
+} ZipfCache;
+
+bool		zipfCacheOverflowMessagePrinted = false;
+
+/*
  * Thread state
  */
 typedef struct
@@ -345,6 +377,8 @@ typedef struct
 	unsigned short random_state[3]; /* separate randomness for each thread */
 	int64		throttle_trigger;	/* previous/next throttling (us) */
 	FILE	   *logfile;		/* where to log, or NULL */
+	ZipfCache	zipf_cache;		/* for thread-safe  zipfian random number
+								 * generation */
 
 	/* per thread collected stats */
 	instr_time	start_time;		/* thread start time */
@@ -737,6 +771,98 @@ getPoissonRand(TState *thread, int64 center)
 	return (int64) (-log(uniform) * ((double) center) + 0.5);
 }
 
+/* helper function for getZipfianRand */
+static double
+zipfGeneralizedHarmonic(int64 n, double s)
+{
+	int			i;
+	double		ans = 0.0;
+
+	for (i = n; i > 1; i--)
+		ans += pow(i, -s);
+	return ans + 1.;
+}
+
+/* helper function for getZipfianRand. Computes random variable in range 1..n */
+static int64
+zipfn(double uniform_random, ZipfCell * zipf, int64 n, double s)
+{
+	double		uz = uniform_random * zipf->harmonicn;
+
+	if (uz < 1.)
+		return 1;
+	if (uz < 1. + zipf->beta)
+		return 2;
+	return 1 + (int64) (n * pow(zipf->eta * uniform_random - zipf->eta + 1., zipf->alpha));
+}
+
+/* set harmonicn and other parameters to cache cell */
+static void
+zipfSetCacheCell(ZipfCell * cell, int64 n, double s)
+{
+	double		harmonic2;
+
+	cell->n = n;
+	cell->s = s;
+
+	harmonic2 = zipfGeneralizedHarmonic(2, s);
+	cell->harmonicn = zipfGeneralizedHarmonic(n, s);
+
+	cell->alpha = 1.0 / (1.0 - s);
+	cell->beta = pow(0.5, s);
+	cell->eta = (1.0 - pow(2.0 / n, 1.0 - s)) / (1.0 - harmonic2 / cell->harmonicn);
+}
+
+static ZipfCell *
+zipfFindOrCreateCacheCell(ZipfCache * cache, int64 n, double s)
+{
+	int			i,
+				least_recently_used_i = 0;
+	ZipfCell   *cell;
+
+	/* search cached cell for given parameters */
+	for (i = 0; i < cache->cells_inited; i++)
+	{
+		cell = &cache->cells[i];
+		if (cell->n == n && cell->s == s)
+			return &cache->cells[i];
+
+		if (cell->last_touch_time < cache->cells[least_recently_used_i].last_touch_time)
+			least_recently_used_i = i;
+	}
+
+	/* create new one if it is not exist */
+	if (cache->cells_inited != ZIPF_CACHE_SIZE)
+		i = cache->cells_inited++;
+	else
+	{
+		/* replace LRU cell if cache is full */
+		i = least_recently_used_i;
+		if (!zipfCacheOverflowMessagePrinted)
+		{
+			zipfCacheOverflowMessagePrinted = true;
+			fprintf(stderr, "zipfian cache array overflowed\n");
+		}
+	}
+
+	zipfSetCacheCell(&cache->cells[i], n, s);
+
+	return &cache->cells[i];
+}
+
+/* random number generator: zipfian distribution from min to max inclusive */
+static int64
+getZipfianRand(TState *thread, int64 min, int64 max, double s)
+{
+	int64		n = max - min + 1;
+	ZipfCell   *cell = zipfFindOrCreateCacheCell(&thread->zipf_cache, n, s);
+
+	cell->last_touch_time = thread->zipf_cache.current_time++;
+	double		uniform = pg_erand48(thread->random_state);
+
+	return min + zipfn(uniform, cell, n, s) - 1;
+}
+
 /*
  * Initialize the given SimpleStats struct to all zeroes
  */
@@ -1567,6 +1693,7 @@ evalFunc(TState *thread, CState *st,
 		case PGBENCH_RANDOM:
 		case PGBENCH_RANDOM_EXPONENTIAL:
 		case PGBENCH_RANDOM_GAUSSIAN:
+		case PGBENCH_RANDOM_ZIPFIAN:
 			{
 				int64		imin,
 							imax;
@@ -1617,6 +1744,18 @@ evalFunc(TState *thread, CState *st,
 						setIntValue(retval,
 									getGaussianRand(thread, imin, imax, param));
 					}
+					else if (func == PGBENCH_RANDOM_ZIPFIAN)
+					{
+						if (param <= 0.0 || param >= 1.0)
+						{
+							fprintf(stderr,
+									"zipfian parameter must be greater than 0 and less than 1"
+									" (got %f)\n", param);
+							return false;
+						}
+						setIntValue(retval,
+									getZipfianRand(thread, imin, imax, param));
+					}
 					else		/* exponential */
 					{
 						if (param <= 0.0)
@@ -4264,6 +4403,8 @@ main(int argc, char **argv)
 		thread->random_state[2] = random();
 		thread->logfile = NULL; /* filled in later */
 		thread->latency_late = 0;
+		thread->zipf_cache.cells_inited = 0;
+		thread->zipf_cache.current_time = 0;
 		initStats(&thread->stats, 0);
 
 		nclients_dealt += thread->nstate;
diff --git a/src/bin/pgbench/pgbench.h b/src/bin/pgbench/pgbench.h
index abc13e9463..1a29f1260c 100644
--- a/src/bin/pgbench/pgbench.h
+++ b/src/bin/pgbench/pgbench.h
@@ -75,7 +75,8 @@ typedef enum PgBenchFunction
 	PGBENCH_SQRT,
 	PGBENCH_RANDOM,
 	PGBENCH_RANDOM_GAUSSIAN,
-	PGBENCH_RANDOM_EXPONENTIAL
+	PGBENCH_RANDOM_EXPONENTIAL,
+	PGBENCH_RANDOM_ZIPFIAN
 } PgBenchFunction;
 
 typedef struct PgBenchExpr PgBenchExpr;
#33Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Alik Khilazhev (#32)
Re: [WIP] Zipfian distribution in pgbench

Hello Alik,

About the maths: As already said, I'm not at ease with a random_zipfian
function which does not display a (good) zipfian distribution. At the
minimum the documentation should be clear about the approximations
implied depending on the parameter value.

I add one more sentence to documentation to emphasize that degree of
proximity depends on parameter .
And also I made restriction on
parameter, now it can be only in range (0; 1)

Hmmm. On second thought, maybe one or the other is enough, either restrict
the parameter to values where the approximation is good, or put out a
clear documentation about when the approximation is not very good, but it
may be still useful even if not precise.

So I would be in favor of expanding the documentation but not restricting
the parameter beyond avoiding value 1.0.

[...]

Now it prints warning message if array overflowed. To print message only
one time, it uses global flag, which is available for all threads. And
theoretically message can be printed more than one time. [...]
So, should I spend time on solving this issue?

No. Just write a comment.

If the zipf cache is constant size, there is no point in using dynamic
allocation, just declare an array…

Fixed. Does ZIPF_CACHE_SIZE = 15 is ok?

My guess is yes, till someone complains that it is not the case;-)

There should be non regression tests somehow. If the "improve pgbench
tap test infrastructure" get through, things should be added there.

I will send tests later, as separate patch.

I think that developping a test would be much simpler with the improved
tap test infrastructure, so I would suggest to wait to know the result of
the corresponding patch.

Also, could you recod the patch to CF 2017-09?
https://commitfest.postgresql.org/14/

--
Fabien.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#34Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Fabien COELHO (#33)
Re: [WIP] Zipfian distribution in pgbench

I think that developping a test would be much simpler with the improved tap test infrastructure, so I would suggest to wait to know the result of the corresponding patch.

Ok, I will wait then.

Also, could you recod the patch to CF 2017-09?
https://commitfest.postgresql.org/14/ <https://commitfest.postgresql.org/14/&gt;

Yea, I will send it.


Thanks and Regards,
Alik Khilazhev
Postgres Professional:
http://www.postgrespro.com
The Russian Postgres Company

#35Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Alik Khilazhev (#34)
1 attachment(s)
Re: [WIP] Zipfian distribution in pgbench

Hmmm. On second thought, maybe one or the other is enough, either restrict the parameter to values where the approximation is good, or put out a clear documentation about when the approximation is not very good, but it may be still useful even if not precise.

So I would be in favor of expanding the documentation but not restricting the parameter beyond avoiding value 1.0.

I have removed restriction and expanded documentation in attaching patch v5.

Also I have recorded patch to CF 2017-09 — https://commitfest.postgresql.org/14/1206/.


Thanks and Regards,
Alik Khilazhev
Postgres Professional:
http://www.postgrespro.com
The Russian Postgres Company

Attachments:

pgbench-zipf-05v.patchapplication/octet-stream; name=pgbench-zipf-05v.patch; x-unix-mode=0644Download
diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml
index 64b043b48a..b84837f407 100644
--- a/doc/src/sgml/ref/pgbench.sgml
+++ b/doc/src/sgml/ref/pgbench.sgml
@@ -1014,6 +1014,14 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
        <entry>an integer between <literal>1</> and <literal>10</></>
       </row>
       <row>
+        <entry><literal><function>random_zipfian(<replaceable>lb</>, <replaceable>ub</>, <replaceable>parameter</>)</></></>
+        <entry>integer</>
+        <entry> Zipfian-distributed random integer in <literal>[lb, ub]</>,
+              see below</>
+        <entry><literal>random_zipfian(1, 10, 0.5)</></>
+        <entry>an integer between <literal>1</> and <literal>10</></>
+      </row>
+      <row>
        <entry><literal><function>sqrt(<replaceable>x</>)</></></>
        <entry>double</>
        <entry>square root</>
@@ -1094,6 +1102,25 @@ f(x) = PHI(2.0 * parameter * (x - mu) / (max - min + 1)) /
       of the Box-Muller transform.
      </para>
     </listitem>
+    <listitem>
+        <para>
+          <literal>random_zipfian</> generates only approximate distribution and degree of proximity depends on <replaceable>parameter</>.
+          For good accuracy it's better to choose <replaceable>parameter</> in range (0, 1). 
+        </para>
+        <para>
+          <replaceable>parameter</> 
+          defines how skewed the distribution. A physical sense of this parameter
+          is following: probability of <literal>i</>th element <literal>f(i) = (1 / (i ^ parameter)) /
+          (H(N, parameter))</> where <literal>N</> is number of elements in range and <literal>H(N, parameter)</>
+          is <literal>N</>th generalized harmonic number. 
+        </para>
+        <para>
+          Intuitively, the larger the <replaceable>parameter</>, the more
+          frequently value values to the beginning of the interval are drawn. 
+          The closer to 0 <replaceable>parameter</> is, 
+          the flatter (more uniform) the access distribution.
+        </para>
+    </listitem>
    </itemizedlist>
 
   <para>
diff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.y
index b3a2d9bfd3..25d5ad48e5 100644
--- a/src/bin/pgbench/exprparse.y
+++ b/src/bin/pgbench/exprparse.y
@@ -191,6 +191,9 @@ static const struct
 	{
 		"random_exponential", 3, PGBENCH_RANDOM_EXPONENTIAL
 	},
+	{
+		"random_zipfian", 3, PGBENCH_RANDOM_ZIPFIAN
+	},
 	/* keep as last array element */
 	{
 		NULL, 0, 0
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 4d364a1db4..fe523c46a1 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -93,6 +93,8 @@ static int	pthread_join(pthread_t th, void **thread_return);
 #define LOG_STEP_SECONDS	5	/* seconds between log messages */
 #define DEFAULT_NXACTS	10		/* default nxacts */
 
+#define ZIPF_CACHE_SIZE	15		/* cache cells number */
+
 #define MIN_GAUSSIAN_PARAM		2.0 /* minimum parameter for gauss */
 
 int			nxacts = 0;			/* number of transactions per client */
@@ -334,6 +336,36 @@ typedef struct
 } CState;
 
 /*
+ * Cache cell for zipfian_random call
+ */
+typedef struct
+{
+	/* cell keys */
+	double		s;				/* s - parameter of zipfan_random function */
+	int64		n;				/* number of elements in range (max - min + 1) */
+
+	double		harmonicn;		/* zipfGeneralizedHarmonic(n, s) */
+	double		alpha;
+	double		beta;
+	double		eta;
+
+	uint64		last_touch_time;	/* time when cell was used last time */
+} ZipfCell;
+
+/*
+ * Zipf cache for zeta values
+ */
+typedef struct
+{
+	uint64		current_time;	/* counter for LRU cache replacement algorithm */
+
+	int			cells_inited;	/* number of filled cells */
+	ZipfCell	cells[ZIPF_CACHE_SIZE];
+} ZipfCache;
+
+bool		zipfCacheOverflowMessagePrinted = false;
+
+/*
  * Thread state
  */
 typedef struct
@@ -345,6 +377,8 @@ typedef struct
 	unsigned short random_state[3]; /* separate randomness for each thread */
 	int64		throttle_trigger;	/* previous/next throttling (us) */
 	FILE	   *logfile;		/* where to log, or NULL */
+	ZipfCache	zipf_cache;		/* for thread-safe  zipfian random number
+								 * generation */
 
 	/* per thread collected stats */
 	instr_time	start_time;		/* thread start time */
@@ -737,6 +771,106 @@ getPoissonRand(TState *thread, int64 center)
 	return (int64) (-log(uniform) * ((double) center) + 0.5);
 }
 
+/* helper function for getZipfianRand */
+static double
+zipfGeneralizedHarmonic(int64 n, double s)
+{
+	int			i;
+	double		ans = 0.0;
+
+	for (i = n; i > 1; i--)
+		ans += pow(i, -s);
+	return ans + 1.;
+}
+
+/* helper function for getZipfianRand. Computes random variable in range 1..n */
+static int64
+zipfn(double uniform_random, ZipfCell * zipf, int64 n, double s)
+{
+	double		uz = uniform_random * zipf->harmonicn;
+
+	if (uz < 1.)
+		return 1;
+	if (uz < 1. + zipf->beta)
+		return 2;
+	return 1 + (int64) (n * pow(zipf->eta * uniform_random - zipf->eta + 1., zipf->alpha));
+}
+
+/* set harmonicn and other parameters to cache cell */
+static void
+zipfSetCacheCell(ZipfCell * cell, int64 n, double s)
+{
+	double		harmonic2;
+
+	cell->n = n;
+	cell->s = s;
+
+	harmonic2 = zipfGeneralizedHarmonic(2, s);
+	cell->harmonicn = zipfGeneralizedHarmonic(n, s);
+
+	cell->alpha = 1.0 / (1.0 - s);
+	cell->beta = pow(0.5, s);
+	cell->eta = (1.0 - pow(2.0 / n, 1.0 - s)) / (1.0 - harmonic2 / cell->harmonicn);
+}
+
+/*
+ * search for cache cell with keys (n, s)
+ * and create new cell if it is not exist
+ */
+static ZipfCell *
+zipfFindOrCreateCacheCell(ZipfCache * cache, int64 n, double s)
+{
+	int			i,
+				least_recently_used_i = 0;
+	ZipfCell   *cell;
+
+	/* search cached cell for given parameters */
+	for (i = 0; i < cache->cells_inited; i++)
+	{
+		cell = &cache->cells[i];
+		if (cell->n == n && cell->s == s)
+			return &cache->cells[i];
+
+		if (cell->last_touch_time < cache->cells[least_recently_used_i].last_touch_time)
+			least_recently_used_i = i;
+	}
+
+	/* create new one if it is not exist */
+	if (cache->cells_inited != ZIPF_CACHE_SIZE)
+		i = cache->cells_inited++;
+	else
+	{
+		/* replace LRU cell if cache is full */
+		i = least_recently_used_i;
+		if (!zipfCacheOverflowMessagePrinted)
+		{
+			/*
+			 * flag can be accessed in several threads simultaneously and
+			 * message can be printed more than one time
+			 */
+			zipfCacheOverflowMessagePrinted = true;
+			fprintf(stderr, "zipfian cache array overflowed\n");
+		}
+	}
+
+	zipfSetCacheCell(&cache->cells[i], n, s);
+
+	return &cache->cells[i];
+}
+
+/* random number generator: zipfian distribution from min to max inclusive */
+static int64
+getZipfianRand(TState *thread, int64 min, int64 max, double s)
+{
+	int64		n = max - min + 1;
+	ZipfCell   *cell = zipfFindOrCreateCacheCell(&thread->zipf_cache, n, s);
+
+	cell->last_touch_time = thread->zipf_cache.current_time++;
+	double		uniform = pg_erand48(thread->random_state);
+
+	return min + zipfn(uniform, cell, n, s) - 1;
+}
+
 /*
  * Initialize the given SimpleStats struct to all zeroes
  */
@@ -1567,6 +1701,7 @@ evalFunc(TState *thread, CState *st,
 		case PGBENCH_RANDOM:
 		case PGBENCH_RANDOM_EXPONENTIAL:
 		case PGBENCH_RANDOM_GAUSSIAN:
+		case PGBENCH_RANDOM_ZIPFIAN:
 			{
 				int64		imin,
 							imax;
@@ -1617,6 +1752,18 @@ evalFunc(TState *thread, CState *st,
 						setIntValue(retval,
 									getGaussianRand(thread, imin, imax, param));
 					}
+					else if (func == PGBENCH_RANDOM_ZIPFIAN)
+					{
+						if (param <= 0.0 || param == 1.0)
+						{
+							fprintf(stderr,
+									"zipfian parameter must be greater than 0 and cannot be 1"
+									" (got %f)\n", param);
+							return false;
+						}
+						setIntValue(retval,
+									getZipfianRand(thread, imin, imax, param));
+					}
 					else		/* exponential */
 					{
 						if (param <= 0.0)
@@ -4264,6 +4411,8 @@ main(int argc, char **argv)
 		thread->random_state[2] = random();
 		thread->logfile = NULL; /* filled in later */
 		thread->latency_late = 0;
+		thread->zipf_cache.cells_inited = 0;
+		thread->zipf_cache.current_time = 0;
 		initStats(&thread->stats, 0);
 
 		nclients_dealt += thread->nstate;
diff --git a/src/bin/pgbench/pgbench.h b/src/bin/pgbench/pgbench.h
index abc13e9463..1a29f1260c 100644
--- a/src/bin/pgbench/pgbench.h
+++ b/src/bin/pgbench/pgbench.h
@@ -75,7 +75,8 @@ typedef enum PgBenchFunction
 	PGBENCH_SQRT,
 	PGBENCH_RANDOM,
 	PGBENCH_RANDOM_GAUSSIAN,
-	PGBENCH_RANDOM_EXPONENTIAL
+	PGBENCH_RANDOM_EXPONENTIAL,
+	PGBENCH_RANDOM_ZIPFIAN
 } PgBenchFunction;
 
 typedef struct PgBenchExpr PgBenchExpr;
#36Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Alik Khilazhev (#35)
1 attachment(s)
Re: [WIP] Zipfian distribution in pgbench

Hello!

I realized that I was sending emails as HTML and latest patch is not visible in the archive now.
That’s why I am attaching it again.

I am sorry for that.

Attachments:

pgbench-zipf-05v.patchapplication/octet-stream; name=pgbench-zipf-05v.patch; x-unix-mode=0644Download
diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml
index 64b043b48a..b84837f407 100644
--- a/doc/src/sgml/ref/pgbench.sgml
+++ b/doc/src/sgml/ref/pgbench.sgml
@@ -1014,6 +1014,14 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
        <entry>an integer between <literal>1</> and <literal>10</></>
       </row>
       <row>
+        <entry><literal><function>random_zipfian(<replaceable>lb</>, <replaceable>ub</>, <replaceable>parameter</>)</></></>
+        <entry>integer</>
+        <entry> Zipfian-distributed random integer in <literal>[lb, ub]</>,
+              see below</>
+        <entry><literal>random_zipfian(1, 10, 0.5)</></>
+        <entry>an integer between <literal>1</> and <literal>10</></>
+      </row>
+      <row>
        <entry><literal><function>sqrt(<replaceable>x</>)</></></>
        <entry>double</>
        <entry>square root</>
@@ -1094,6 +1102,25 @@ f(x) = PHI(2.0 * parameter * (x - mu) / (max - min + 1)) /
       of the Box-Muller transform.
      </para>
     </listitem>
+    <listitem>
+        <para>
+          <literal>random_zipfian</> generates only approximate distribution and degree of proximity depends on <replaceable>parameter</>.
+          For good accuracy it's better to choose <replaceable>parameter</> in range (0, 1). 
+        </para>
+        <para>
+          <replaceable>parameter</> 
+          defines how skewed the distribution. A physical sense of this parameter
+          is following: probability of <literal>i</>th element <literal>f(i) = (1 / (i ^ parameter)) /
+          (H(N, parameter))</> where <literal>N</> is number of elements in range and <literal>H(N, parameter)</>
+          is <literal>N</>th generalized harmonic number. 
+        </para>
+        <para>
+          Intuitively, the larger the <replaceable>parameter</>, the more
+          frequently value values to the beginning of the interval are drawn. 
+          The closer to 0 <replaceable>parameter</> is, 
+          the flatter (more uniform) the access distribution.
+        </para>
+    </listitem>
    </itemizedlist>
 
   <para>
diff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.y
index b3a2d9bfd3..25d5ad48e5 100644
--- a/src/bin/pgbench/exprparse.y
+++ b/src/bin/pgbench/exprparse.y
@@ -191,6 +191,9 @@ static const struct
 	{
 		"random_exponential", 3, PGBENCH_RANDOM_EXPONENTIAL
 	},
+	{
+		"random_zipfian", 3, PGBENCH_RANDOM_ZIPFIAN
+	},
 	/* keep as last array element */
 	{
 		NULL, 0, 0
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 4d364a1db4..fe523c46a1 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -93,6 +93,8 @@ static int	pthread_join(pthread_t th, void **thread_return);
 #define LOG_STEP_SECONDS	5	/* seconds between log messages */
 #define DEFAULT_NXACTS	10		/* default nxacts */
 
+#define ZIPF_CACHE_SIZE	15		/* cache cells number */
+
 #define MIN_GAUSSIAN_PARAM		2.0 /* minimum parameter for gauss */
 
 int			nxacts = 0;			/* number of transactions per client */
@@ -334,6 +336,36 @@ typedef struct
 } CState;
 
 /*
+ * Cache cell for zipfian_random call
+ */
+typedef struct
+{
+	/* cell keys */
+	double		s;				/* s - parameter of zipfan_random function */
+	int64		n;				/* number of elements in range (max - min + 1) */
+
+	double		harmonicn;		/* zipfGeneralizedHarmonic(n, s) */
+	double		alpha;
+	double		beta;
+	double		eta;
+
+	uint64		last_touch_time;	/* time when cell was used last time */
+} ZipfCell;
+
+/*
+ * Zipf cache for zeta values
+ */
+typedef struct
+{
+	uint64		current_time;	/* counter for LRU cache replacement algorithm */
+
+	int			cells_inited;	/* number of filled cells */
+	ZipfCell	cells[ZIPF_CACHE_SIZE];
+} ZipfCache;
+
+bool		zipfCacheOverflowMessagePrinted = false;
+
+/*
  * Thread state
  */
 typedef struct
@@ -345,6 +377,8 @@ typedef struct
 	unsigned short random_state[3]; /* separate randomness for each thread */
 	int64		throttle_trigger;	/* previous/next throttling (us) */
 	FILE	   *logfile;		/* where to log, or NULL */
+	ZipfCache	zipf_cache;		/* for thread-safe  zipfian random number
+								 * generation */
 
 	/* per thread collected stats */
 	instr_time	start_time;		/* thread start time */
@@ -737,6 +771,106 @@ getPoissonRand(TState *thread, int64 center)
 	return (int64) (-log(uniform) * ((double) center) + 0.5);
 }
 
+/* helper function for getZipfianRand */
+static double
+zipfGeneralizedHarmonic(int64 n, double s)
+{
+	int			i;
+	double		ans = 0.0;
+
+	for (i = n; i > 1; i--)
+		ans += pow(i, -s);
+	return ans + 1.;
+}
+
+/* helper function for getZipfianRand. Computes random variable in range 1..n */
+static int64
+zipfn(double uniform_random, ZipfCell * zipf, int64 n, double s)
+{
+	double		uz = uniform_random * zipf->harmonicn;
+
+	if (uz < 1.)
+		return 1;
+	if (uz < 1. + zipf->beta)
+		return 2;
+	return 1 + (int64) (n * pow(zipf->eta * uniform_random - zipf->eta + 1., zipf->alpha));
+}
+
+/* set harmonicn and other parameters to cache cell */
+static void
+zipfSetCacheCell(ZipfCell * cell, int64 n, double s)
+{
+	double		harmonic2;
+
+	cell->n = n;
+	cell->s = s;
+
+	harmonic2 = zipfGeneralizedHarmonic(2, s);
+	cell->harmonicn = zipfGeneralizedHarmonic(n, s);
+
+	cell->alpha = 1.0 / (1.0 - s);
+	cell->beta = pow(0.5, s);
+	cell->eta = (1.0 - pow(2.0 / n, 1.0 - s)) / (1.0 - harmonic2 / cell->harmonicn);
+}
+
+/*
+ * search for cache cell with keys (n, s)
+ * and create new cell if it is not exist
+ */
+static ZipfCell *
+zipfFindOrCreateCacheCell(ZipfCache * cache, int64 n, double s)
+{
+	int			i,
+				least_recently_used_i = 0;
+	ZipfCell   *cell;
+
+	/* search cached cell for given parameters */
+	for (i = 0; i < cache->cells_inited; i++)
+	{
+		cell = &cache->cells[i];
+		if (cell->n == n && cell->s == s)
+			return &cache->cells[i];
+
+		if (cell->last_touch_time < cache->cells[least_recently_used_i].last_touch_time)
+			least_recently_used_i = i;
+	}
+
+	/* create new one if it is not exist */
+	if (cache->cells_inited != ZIPF_CACHE_SIZE)
+		i = cache->cells_inited++;
+	else
+	{
+		/* replace LRU cell if cache is full */
+		i = least_recently_used_i;
+		if (!zipfCacheOverflowMessagePrinted)
+		{
+			/*
+			 * flag can be accessed in several threads simultaneously and
+			 * message can be printed more than one time
+			 */
+			zipfCacheOverflowMessagePrinted = true;
+			fprintf(stderr, "zipfian cache array overflowed\n");
+		}
+	}
+
+	zipfSetCacheCell(&cache->cells[i], n, s);
+
+	return &cache->cells[i];
+}
+
+/* random number generator: zipfian distribution from min to max inclusive */
+static int64
+getZipfianRand(TState *thread, int64 min, int64 max, double s)
+{
+	int64		n = max - min + 1;
+	ZipfCell   *cell = zipfFindOrCreateCacheCell(&thread->zipf_cache, n, s);
+
+	cell->last_touch_time = thread->zipf_cache.current_time++;
+	double		uniform = pg_erand48(thread->random_state);
+
+	return min + zipfn(uniform, cell, n, s) - 1;
+}
+
 /*
  * Initialize the given SimpleStats struct to all zeroes
  */
@@ -1567,6 +1701,7 @@ evalFunc(TState *thread, CState *st,
 		case PGBENCH_RANDOM:
 		case PGBENCH_RANDOM_EXPONENTIAL:
 		case PGBENCH_RANDOM_GAUSSIAN:
+		case PGBENCH_RANDOM_ZIPFIAN:
 			{
 				int64		imin,
 							imax;
@@ -1617,6 +1752,18 @@ evalFunc(TState *thread, CState *st,
 						setIntValue(retval,
 									getGaussianRand(thread, imin, imax, param));
 					}
+					else if (func == PGBENCH_RANDOM_ZIPFIAN)
+					{
+						if (param <= 0.0 || param == 1.0)
+						{
+							fprintf(stderr,
+									"zipfian parameter must be greater than 0 and cannot be 1"
+									" (got %f)\n", param);
+							return false;
+						}
+						setIntValue(retval,
+									getZipfianRand(thread, imin, imax, param));
+					}
 					else		/* exponential */
 					{
 						if (param <= 0.0)
@@ -4264,6 +4411,8 @@ main(int argc, char **argv)
 		thread->random_state[2] = random();
 		thread->logfile = NULL; /* filled in later */
 		thread->latency_late = 0;
+		thread->zipf_cache.cells_inited = 0;
+		thread->zipf_cache.current_time = 0;
 		initStats(&thread->stats, 0);
 
 		nclients_dealt += thread->nstate;
diff --git a/src/bin/pgbench/pgbench.h b/src/bin/pgbench/pgbench.h
index abc13e9463..1a29f1260c 100644
--- a/src/bin/pgbench/pgbench.h
+++ b/src/bin/pgbench/pgbench.h
@@ -75,7 +75,8 @@ typedef enum PgBenchFunction
 	PGBENCH_SQRT,
 	PGBENCH_RANDOM,
 	PGBENCH_RANDOM_GAUSSIAN,
-	PGBENCH_RANDOM_EXPONENTIAL
+	PGBENCH_RANDOM_EXPONENTIAL,
+	PGBENCH_RANDOM_ZIPFIAN
 } PgBenchFunction;
 
 typedef struct PgBenchExpr PgBenchExpr;
In reply to: Alik Khilazhev (#36)
Re: [WIP] Zipfian distribution in pgbench

On Fri, Jul 21, 2017 at 4:51 AM, Alik Khilazhev
<a.khilazhev@postgrespro.ru> wrote:

(Latest version of pgbench Zipfian patch)

While I'm +1 on this idea, I think that it would also be nice if there
was an option to make functions like random_zipfian() actually return
a value that has undergone perfect hashing. When this option is used,
any given value that the function returns would actually be taken from
a random mapping to some other value in the same range. So, you can
potentially get a Zipfian distribution without the locality.

While I think that Zipfian distributions are common, it's probably
less common for the popular items to be clustered together within the
primary key, unless the PK has multiple attributes, and the first is
the "popular attribute". For example, there would definitely be a lot
of interest among contiguous items within an index on "(artist,
album)" where "artist" is a popular artist, which is certainly quite
possible. But, if "album" is the primary key, and it's a SERIAL PK,
then there will only be weak temporal locality within the PK, and only
because today's fads are more popular than the fads from previous
years.

Just another idea, that doesn't have to hold this patch up.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#38Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Peter Geoghegan (#37)
Re: [WIP] Zipfian distribution in pgbench

Hello Peter,

I think that it would also be nice if there was an option to make
functions like random_zipfian() actually return a value that has
undergone perfect hashing. When this option is used, any given value
that the function returns would actually be taken from a random mapping
to some other value in the same range. So, you can potentially get a
Zipfian distribution without the locality.

I definitely agree. This is a standard problem with all non uniform random
generators in pgbench, namely random_{gaussian,exponential}.

However hashing is not a good solution on a finite domain because of the
significant collision rate, so that typically 1/3 of values are empty and
collisions cause spikes. Also, collisions would break PKs.

The solution is to provide a (good) pseudo-random parametric permutation,
which is non trivial especially for non powers of two, so ISTM that it
should be a patch on its own.

The good news is that it is on my todo list and I have some ideas on how
to do it.

The bad news is that given the rate at which I succeed in getting things
committed in pgbench, it might take some years:-( For instance, simplistic
functions and operators to extend the current set have been in the pipe
for 15 months and missed pg10.

--
Fabien.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#39Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Alik Khilazhev (#35)
1 attachment(s)
Re: [WIP] Zipfian distribution in pgbench

Hello Alik,

So I would be in favor of expanding the documentation but not
restricting the parameter beyond avoiding value 1.0.

I have removed restriction and expanded documentation in attaching patch v5.

I've done some math investigations, which consisted in spending one hour
with Christian, a statistician colleague of mine. He took an old book out
of a shelf, opened it to page 550 (roughly in the middle), and explained
to me how to build a real zipfian distribution random generator.

The iterative method is for parameter a>1 and works for unbounded values.
It is simple to add a bound. In practice the iterative method is quite
effective, i.e. number of iterations is typically small, at least if the
bound is large and if parameter a is not too close to 1.

I've attached a python3 script which implements the algorithm. It looks
like magic. Beware that a C implementation should take care of float and
int overflows.

# usage: a, #values, #tests

sh> zipf.py 1.5 1000 1000000
# after 1.7 seconds
c = [391586, 138668, 75525, 49339, 35222, 26621, ...
... 11, 13, 12, 11, 16] (1.338591 iterations per draw)

sh> zipf.py 1.1 1000 1000000
# after 3.1 seconds
c = [179302, 83927, 53104, 39015, 30557, 25164, ...
... 82, 95, 93, 81, 80] (2.681451 iterations per draw)

I think that this method should be used for a>1, and the other very rough
one can be kept for parameter a in [0, 1), a case which does not make much
sense to a mathematician as it diverges if unbounded.

--
Fabien.

Attachments:

zipf.pytext/x-python; name=zipf.pyDownload
#40Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Fabien COELHO (#39)
Re: [WIP] Zipfian distribution in pgbench

Hello Fabien,

On 5 Aug 2017, at 12:15, Fabien COELHO <coelho@cri.ensmp.fr> wrote:

Hello Alik,

I've done some math investigations, which consisted in spending one hour with Christian, a statistician colleague of mine. He took an old book out of a shelf, opened it to page 550 (roughly in the middle), and explained to me how to build a real zipfian distribution random generator.

The iterative method is for parameter a>1 and works for unbounded values. It is simple to add a bound. In practice the iterative method is quite effective, i.e. number of iterations is typically small, at least if the bound is large and if parameter a is not too close to 1.

I've attached a python3 script which implements the algorithm. It looks like magic. Beware that a C implementation should take care of float and int overflows.

Thank you for the script. I will rewrite it to C and add to the patch soon.


Thanks and Regards,
Alik Khilazhev
Postgres Professional:
http://www.postgrespro.com
The Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#41Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Fabien COELHO (#39)
1 attachment(s)
Re: [WIP] Zipfian distribution in pgbench

Hello Fabien,

I think that this method should be used for a>1, and the other very rough one can be kept for parameter a in [0, 1), a case which does not make much sense to a mathematician as it diverges if unbounded.

Now “a” does not have upper bound, that’s why on using iterative algorithm with a >= 10000 program will stuck on infinite loop because of following line of code:
double b = pow(2.0, s - 1.0);
Because after overflow “b” becomes “+Inf”.

So should upper bound for “a" be set?

Should I mention in docs that there are two algorithms are used depending on values of a(s/theta)?

In attaching patch, I have added computeIterativeZipfian method and it’s usage in getZipfianRand.
Is it better to move code of computing via cache to new method, so that getZipfianRand will contain only 2 computeXXXZipfian method calls?

Attachments:

pgbench-zipf-06v.patchapplication/octet-stream; name=pgbench-zipf-06v.patch; x-unix-mode=0644Download
diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml
index 03e1212d50..8148364125 100644
--- a/doc/src/sgml/ref/pgbench.sgml
+++ b/doc/src/sgml/ref/pgbench.sgml
@@ -1014,6 +1014,14 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
        <entry>an integer between <literal>1</> and <literal>10</></>
       </row>
       <row>
+        <entry><literal><function>random_zipfian(<replaceable>lb</>, <replaceable>ub</>, <replaceable>parameter</>)</></></>
+        <entry>integer</>
+        <entry> Zipfian-distributed random integer in <literal>[lb, ub]</>,
+              see below</>
+        <entry><literal>random_zipfian(1, 10, 0.5)</></>
+        <entry>an integer between <literal>1</> and <literal>10</></>
+      </row>
+      <row>
        <entry><literal><function>sqrt(<replaceable>x</>)</></></>
        <entry>double</>
        <entry>square root</>
@@ -1094,6 +1102,25 @@ f(x) = PHI(2.0 * parameter * (x - mu) / (max - min + 1)) /
       of the Box-Muller transform.
      </para>
     </listitem>
+    <listitem>
+        <para>
+          <literal>random_zipfian</> generates only approximate distribution and degree of proximity depends on <replaceable>parameter</>.
+          For good accuracy it's better to choose <replaceable>parameter</> in range (0, 1). 
+        </para>
+        <para>
+          <replaceable>parameter</> 
+          defines how skewed the distribution. A physical sense of this parameter
+          is following: probability of <literal>i</>th element <literal>f(i) = (1 / (i ^ parameter)) /
+          (H(N, parameter))</> where <literal>N</> is number of elements in range and <literal>H(N, parameter)</>
+          is <literal>N</>th generalized harmonic number. 
+        </para>
+        <para>
+          Intuitively, the larger the <replaceable>parameter</>, the more
+          frequently value values to the beginning of the interval are drawn. 
+          The closer to 0 <replaceable>parameter</> is, 
+          the flatter (more uniform) the access distribution.
+        </para>
+    </listitem>
    </itemizedlist>
 
   <para>
diff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.y
index b3a2d9bfd3..25d5ad48e5 100644
--- a/src/bin/pgbench/exprparse.y
+++ b/src/bin/pgbench/exprparse.y
@@ -191,6 +191,9 @@ static const struct
 	{
 		"random_exponential", 3, PGBENCH_RANDOM_EXPONENTIAL
 	},
+	{
+		"random_zipfian", 3, PGBENCH_RANDOM_ZIPFIAN
+	},
 	/* keep as last array element */
 	{
 		NULL, 0, 0
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index ae78c7b1d4..f902fdd64e 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -93,6 +93,8 @@ static int	pthread_join(pthread_t th, void **thread_return);
 #define LOG_STEP_SECONDS	5	/* seconds between log messages */
 #define DEFAULT_NXACTS	10		/* default nxacts */
 
+#define ZIPF_CACHE_SIZE	15		/* cache cells number */
+
 #define MIN_GAUSSIAN_PARAM		2.0 /* minimum parameter for gauss */
 
 int			nxacts = 0;			/* number of transactions per client */
@@ -334,6 +336,36 @@ typedef struct
 } CState;
 
 /*
+ * Cache cell for zipfian_random call
+ */
+typedef struct
+{
+	/* cell keys */
+	double		s;				/* s - parameter of zipfan_random function */
+	int64		n;				/* number of elements in range (max - min + 1) */
+
+	double		harmonicn;		/* zipfGeneralizedHarmonic(n, s) */
+	double		alpha;
+	double		beta;
+	double		eta;
+
+	uint64		last_touch_time;	/* time when cell was used last time */
+} ZipfCell;
+
+/*
+ * Zipf cache for zeta values
+ */
+typedef struct
+{
+	uint64		current_time;	/* counter for LRU cache replacement algorithm */
+
+	int			cells_inited;	/* number of filled cells */
+	ZipfCell	cells[ZIPF_CACHE_SIZE];
+} ZipfCache;
+
+bool		zipfCacheOverflowMessagePrinted = false;
+
+/*
  * Thread state
  */
 typedef struct
@@ -345,6 +377,8 @@ typedef struct
 	unsigned short random_state[3]; /* separate randomness for each thread */
 	int64		throttle_trigger;	/* previous/next throttling (us) */
 	FILE	   *logfile;		/* where to log, or NULL */
+	ZipfCache	zipf_cache;		/* for thread-safe  zipfian random number
+								 * generation */
 
 	/* per thread collected stats */
 	instr_time	start_time;		/* thread start time */
@@ -737,6 +771,137 @@ getPoissonRand(TState *thread, int64 center)
 	return (int64) (-log(uniform) * ((double) center) + 0.5);
 }
 
+/* helper function for getZipfianRand */
+static double
+zipfGeneralizedHarmonic(int64 n, double s)
+{
+	int			i;
+	double		ans = 0.0;
+
+	for (i = n; i > 1; i--)
+		ans += pow(i, -s);
+	return ans + 1.;
+}
+
+/* helper function for getZipfianRand. Computes random variable in range 1..n */
+static int64
+zipfn(double uniform_random, ZipfCell * zipf, int64 n, double s)
+{
+	double		uz = uniform_random * zipf->harmonicn;
+
+	if (uz < 1.)
+		return 1;
+	if (uz < 1. + zipf->beta)
+		return 2;
+	return 1 + (int64) (n * pow(zipf->eta * uniform_random - zipf->eta + 1., zipf->alpha));
+}
+
+/* set harmonicn and other parameters to cache cell */
+static void
+zipfSetCacheCell(ZipfCell * cell, int64 n, double s)
+{
+	double		harmonic2;
+
+	cell->n = n;
+	cell->s = s;
+
+	harmonic2 = zipfGeneralizedHarmonic(2, s);
+	cell->harmonicn = zipfGeneralizedHarmonic(n, s);
+
+	cell->alpha = 1.0 / (1.0 - s);
+	cell->beta = pow(0.5, s);
+	cell->eta = (1.0 - pow(2.0 / n, 1.0 - s)) / (1.0 - harmonic2 / cell->harmonicn);
+}
+
+/*
+ * search for cache cell with keys (n, s)
+ * and create new cell if it is not exist
+ */
+static ZipfCell *
+zipfFindOrCreateCacheCell(ZipfCache * cache, int64 n, double s)
+{
+	int			i,
+				least_recently_used_i = 0;
+	ZipfCell   *cell;
+
+	/* search cached cell for given parameters */
+	for (i = 0; i < cache->cells_inited; i++)
+	{
+		cell = &cache->cells[i];
+		if (cell->n == n && cell->s == s)
+			return &cache->cells[i];
+
+		if (cell->last_touch_time < cache->cells[least_recently_used_i].last_touch_time)
+			least_recently_used_i = i;
+	}
+
+	/* create new one if it is not exist */
+	if (cache->cells_inited != ZIPF_CACHE_SIZE)
+		i = cache->cells_inited++;
+	else
+	{
+		/* replace LRU cell if cache is full */
+		i = least_recently_used_i;
+		if (!zipfCacheOverflowMessagePrinted)
+		{
+			/*
+			 * flag can be accessed in several threads simultaneously and
+			 * message can be printed more than one time
+			 */
+			zipfCacheOverflowMessagePrinted = true;
+			fprintf(stderr, "zipfian cache array overflowed\n");
+		}
+	}
+
+	zipfSetCacheCell(&cache->cells[i], n, s);
+
+	return &cache->cells[i];
+}
+
+static int64
+computeIterativeZipfian(unsigned short random_state[3], int64 n, double s)
+{
+	/* computing zipfian using rejection method */
+	double		b = pow(2.0, s - 1.0);
+	double		x,
+				t,
+				u,
+				v;
+
+	while (1)
+	{
+		/* random variates */
+		u = pg_erand48(random_state);
+		v = pg_erand48(random_state);
+
+		x = floor(pow(u, -1.0 / (s - 1.0)));
+
+		t = pow(1.0 + 1.0 / x, s - 1.0);
+		/* reject if too large or out of bound */
+		if (v * x * (t - 1.0) / (b - 1.0) <= t / b && x <= n)
+			break;
+	}
+	return (int64) x;
+}
+
+/* random number generator: zipfian distribution from min to max inclusive */
+static int64
+getZipfianRand(TState *thread, int64 min, int64 max, double s)
+{
+	int64		n = max - min + 1;
+
+	if (s > 1)
+		/* it is better to use iterative way to compute when s > 1 */
+		return min + computeIterativeZipfian(thread->random_state, n, s) - 1;
+
+	ZipfCell   *cell = zipfFindOrCreateCacheCell(&thread->zipf_cache, n, s);
+
+	cell->last_touch_time = thread->zipf_cache.current_time++;
+	double		uniform = pg_erand48(thread->random_state);
+
+	return min + zipfn(uniform, cell, n, s) - 1;
+}
+
 /*
  * Initialize the given SimpleStats struct to all zeroes
  */
@@ -1567,6 +1732,7 @@ evalFunc(TState *thread, CState *st,
 		case PGBENCH_RANDOM:
 		case PGBENCH_RANDOM_EXPONENTIAL:
 		case PGBENCH_RANDOM_GAUSSIAN:
+		case PGBENCH_RANDOM_ZIPFIAN:
 			{
 				int64		imin,
 							imax;
@@ -1617,6 +1783,18 @@ evalFunc(TState *thread, CState *st,
 						setIntValue(retval,
 									getGaussianRand(thread, imin, imax, param));
 					}
+					else if (func == PGBENCH_RANDOM_ZIPFIAN)
+					{
+						if (param <= 0.0 || param == 1.0)
+						{
+							fprintf(stderr,
+									"zipfian parameter must be greater than 0 and cannot be 1"
+									" (got %f)\n", param);
+							return false;
+						}
+						setIntValue(retval,
+									getZipfianRand(thread, imin, imax, param));
+					}
 					else		/* exponential */
 					{
 						if (param <= 0.0)
@@ -4273,6 +4451,8 @@ main(int argc, char **argv)
 		thread->random_state[2] = random();
 		thread->logfile = NULL; /* filled in later */
 		thread->latency_late = 0;
+		thread->zipf_cache.cells_inited = 0;
+		thread->zipf_cache.current_time = 0;
 		initStats(&thread->stats, 0);
 
 		nclients_dealt += thread->nstate;
diff --git a/src/bin/pgbench/pgbench.h b/src/bin/pgbench/pgbench.h
index abc13e9463..1a29f1260c 100644
--- a/src/bin/pgbench/pgbench.h
+++ b/src/bin/pgbench/pgbench.h
@@ -75,7 +75,8 @@ typedef enum PgBenchFunction
 	PGBENCH_SQRT,
 	PGBENCH_RANDOM,
 	PGBENCH_RANDOM_GAUSSIAN,
-	PGBENCH_RANDOM_EXPONENTIAL
+	PGBENCH_RANDOM_EXPONENTIAL,
+	PGBENCH_RANDOM_ZIPFIAN
 } PgBenchFunction;
 
 typedef struct PgBenchExpr PgBenchExpr;
#42Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Alik Khilazhev (#41)
Re: [WIP] Zipfian distribution in pgbench

Hello Alik,

Now “a” does not have upper bound, that’s why on using iterative algorithm with a >= 10000 program will stuck on infinite loop because of following line of code:
double b = pow(2.0, s - 1.0);
Because after overflow “b” becomes “+Inf”.

Yep, overflow can happen.

So should upper bound for “a" be set?

Yes, I agree. a >= 10000 does not make much sense... If you want uniform
you should use random(), not call random_zipfian with a = 10000. Basically
it suggests that too large values of "a" should be rejected. Not sure
where to put the limit, though.

Should I mention in docs that there are two algorithms are used
depending on values of a(s/theta)?

Yes, as a general principle I think that the documentation should reflect
the implementation.

In attaching patch, I have added computeIterativeZipfian method and it’s
usage in getZipfianRand. Is it better to move code of computing via
cache to new method, so that getZipfianRand will contain only 2
computeXXXZipfian method calls?

I have not looked in detail, but from what you say I would agree that the
implementation should be symmetric, so having one function calling one
method or the other sounds good.

--
Fabien.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#43Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Fabien COELHO (#42)
1 attachment(s)
Re: [WIP] Zipfian distribution in pgbench

Hello, Fabien

I am attaching patch v7.

Yes, I agree. a >= 10000 does not make much sense... If you want uniform you should use random(), not call random_zipfian with a = 10000. Basically it suggests that too large values of "a" should be rejected. Not sure where to put the limit, though.

I set upper bound for parameter to be equal to 1000.

Yes, as a general principle I think that the documentation should reflect the implementation.

Documentation have been updated, I have removed algorithms descriptions and put references to them there.

Attachments:

pgbench-zipf-07v.patchapplication/octet-stream; name=pgbench-zipf-07v.patch; x-unix-mode=0644Download
diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml
index 03e1212d50..64c3922134 100644
--- a/doc/src/sgml/ref/pgbench.sgml
+++ b/doc/src/sgml/ref/pgbench.sgml
@@ -1013,6 +1013,14 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
        <entry><literal>random_gaussian(1, 10, 2.5)</></>
        <entry>an integer between <literal>1</> and <literal>10</></>
       </row>
+      <row>
+        <entry><literal><function>random_zipfian(<replaceable>lb</>, <replaceable>ub</>, <replaceable>parameter</>)</></></>
+        <entry>integer</>
+        <entry> Zipfian-distributed random integer in <literal>[lb, ub]</>,
+              see below</>
+        <entry><literal>random_zipfian(1, 10, 0.5)</></>
+        <entry>an integer between <literal>1</> and <literal>10</></>
+      </row>
       <row>
        <entry><literal><function>sqrt(<replaceable>x</>)</></></>
        <entry>double</>
@@ -1094,6 +1102,23 @@ f(x) = PHI(2.0 * parameter * (x - mu) / (max - min + 1)) /
       of the Box-Muller transform.
      </para>
     </listitem>
+    <listitem>
+        <para>
+          <literal>random_zipfian</> generates only approximate distribution and degree of proximity depends on <replaceable>parameter</>.
+          For <replaceable>parameter</> in range (0, 1) random number generated by computing Harmonic numbers(algorithm taken from
+          "Quickly Generating Billion-Record Synthetic Databases", Jim Gray et al, SIGMOD 1994).
+          And for parameter bigger than 1 rejection method used(described in "Non-Uniform Random Variate Generation", Luc Devroye, p. 550-551, Springer 1986).
+          It's recommended to pick <replaceable>parameter</> in range (0, 1) for better accuracy. 
+          
+        </para>
+        <para>
+          <replaceable>parameter</> 
+          defines how skewed the distribution is. The larger the <replaceable>parameter</>, the more
+          frequently value values to the beginning of the interval are drawn. 
+          The closer to 0 <replaceable>parameter</> is, 
+          the flatter (more uniform) the access distribution
+        </para>
+    </listitem>
    </itemizedlist>
 
   <para>
diff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.y
index b3a2d9bfd3..25d5ad48e5 100644
--- a/src/bin/pgbench/exprparse.y
+++ b/src/bin/pgbench/exprparse.y
@@ -191,6 +191,9 @@ static const struct
 	{
 		"random_exponential", 3, PGBENCH_RANDOM_EXPONENTIAL
 	},
+	{
+		"random_zipfian", 3, PGBENCH_RANDOM_ZIPFIAN
+	},
 	/* keep as last array element */
 	{
 		NULL, 0, 0
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index ae78c7b1d4..5f30b07d5b 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -93,7 +93,10 @@ static int	pthread_join(pthread_t th, void **thread_return);
 #define LOG_STEP_SECONDS	5	/* seconds between log messages */
 #define DEFAULT_NXACTS	10		/* default nxacts */
 
+#define ZIPF_CACHE_SIZE	15		/* cache cells number */
+
 #define MIN_GAUSSIAN_PARAM		2.0 /* minimum parameter for gauss */
+#define MAX_ZIPFIAN_PARAM		1000 /* maximum parameter for zipfian */
 
 int			nxacts = 0;			/* number of transactions per client */
 int			duration = 0;		/* duration in seconds */
@@ -333,6 +336,36 @@ typedef struct
 	int			ecnt;			/* error count */
 } CState;
 
+/*
+ * Cache cell for zipfian_random call
+ */
+typedef struct
+{
+	/* cell keys */
+	double		s;				/* s - parameter of zipfan_random function */
+	int64		n;				/* number of elements in range (max - min + 1) */
+
+	double		harmonicn;		/* zipfGeneralizedHarmonic(n, s) */
+	double		alpha;
+	double		beta;
+	double		eta;
+
+	uint64		last_touch_time;	/* time when cell was used last time */
+} ZipfCell;
+
+/*
+ * Zipf cache for zeta values
+ */
+typedef struct
+{
+	uint64		current_time;	/* counter for LRU cache replacement algorithm */
+
+	int			cells_inited;	/* number of filled cells */
+	ZipfCell	cells[ZIPF_CACHE_SIZE];
+} ZipfCache;
+
+bool		zipfCacheOverflowMessagePrinted = false;
+
 /*
  * Thread state
  */
@@ -345,6 +378,8 @@ typedef struct
 	unsigned short random_state[3]; /* separate randomness for each thread */
 	int64		throttle_trigger;	/* previous/next throttling (us) */
 	FILE	   *logfile;		/* where to log, or NULL */
+	ZipfCache	zipf_cache;		/* for thread-safe  zipfian random number
+								 * generation */
 
 	/* per thread collected stats */
 	instr_time	start_time;		/* thread start time */
@@ -737,6 +772,143 @@ getPoissonRand(TState *thread, int64 center)
 	return (int64) (-log(uniform) * ((double) center) + 0.5);
 }
 
+/* helper function for getZipfianRand */
+static double
+zipfGeneralizedHarmonic(int64 n, double s)
+{
+	int			i;
+	double		ans = 0.0;
+
+	for (i = n; i > 1; i--)
+		ans += pow(i, -s);
+	return ans + 1.;
+}
+
+/* helper function for getZipfianRand. Computes random variable in range 1..n */
+static int64
+zipfn(double uniform_random, ZipfCell * zipf, int64 n, double s)
+{
+	double		uz = uniform_random * zipf->harmonicn;
+
+	if (uz < 1.)
+		return 1;
+	if (uz < 1. + zipf->beta)
+		return 2;
+	return 1 + (int64) (n * pow(zipf->eta * uniform_random - zipf->eta + 1., zipf->alpha));
+}
+
+/* set harmonicn and other parameters to cache cell */
+static void
+zipfSetCacheCell(ZipfCell * cell, int64 n, double s)
+{
+	double		harmonic2;
+
+	cell->n = n;
+	cell->s = s;
+
+	harmonic2 = zipfGeneralizedHarmonic(2, s);
+	cell->harmonicn = zipfGeneralizedHarmonic(n, s);
+
+	cell->alpha = 1.0 / (1.0 - s);
+	cell->beta = pow(0.5, s);
+	cell->eta = (1.0 - pow(2.0 / n, 1.0 - s)) / (1.0 - harmonic2 / cell->harmonicn);
+}
+
+/*
+ * search for cache cell with keys (n, s)
+ * and create new cell if it is not exist
+ */
+static ZipfCell *
+zipfFindOrCreateCacheCell(ZipfCache * cache, int64 n, double s)
+{
+	int			i,
+				least_recently_used_i = 0;
+	ZipfCell   *cell;
+
+	/* search cached cell for given parameters */
+	for (i = 0; i < cache->cells_inited; i++)
+	{
+		cell = &cache->cells[i];
+		if (cell->n == n && cell->s == s)
+			return &cache->cells[i];
+
+		if (cell->last_touch_time < cache->cells[least_recently_used_i].last_touch_time)
+			least_recently_used_i = i;
+	}
+
+	/* create new one if it is not exist */
+	if (cache->cells_inited != ZIPF_CACHE_SIZE)
+		i = cache->cells_inited++;
+	else
+	{
+		/* replace LRU cell if cache is full */
+		i = least_recently_used_i;
+		if (!zipfCacheOverflowMessagePrinted)
+		{
+			/*
+			 * flag can be accessed in several threads simultaneously and
+			 * message can be printed more than one time
+			 */
+			zipfCacheOverflowMessagePrinted = true;
+			fprintf(stderr, "zipfian cache array overflowed\n");
+		}
+	}
+
+	zipfSetCacheCell(&cache->cells[i], n, s);
+
+	return &cache->cells[i];
+}
+
+static int64
+computeIterativeZipfian(unsigned short random_state[3], int64 n, double s)
+{
+	/* computing zipfian using rejection method */
+	double		b = pow(2.0, s - 1.0);
+	double		x,
+				t,
+				u,
+				v;
+
+	while (true)
+	{
+		/* random variates */
+		u = pg_erand48(random_state);
+		v = pg_erand48(random_state);
+
+		x = floor(pow(u, -1.0 / (s - 1.0)));
+
+		t = pow(1.0 + 1.0 / x, s - 1.0);
+		/* reject if too large or out of bound */
+		if (v * x * (t - 1.0) / (b - 1.0) <= t / b && x <= n)
+			break;
+	}
+	return (int64) x;
+}
+
+static int64
+computeHarmonicZipfian(TState *thread, int64 n, double s)
+{
+	ZipfCell   *cell = zipfFindOrCreateCacheCell(&thread->zipf_cache, n, s);
+	
+	cell->last_touch_time = thread->zipf_cache.current_time++;
+	double		uniform = pg_erand48(thread->random_state);
+
+	return zipfn(uniform, cell, n, s);
+}
+
+/* random number generator: zipfian distribution from min to max inclusive */
+static int64
+getZipfianRand(TState *thread, int64 min, int64 max, double s)
+{
+	int64		n = max - min + 1;
+
+	if (s > 1)
+		/* it is better to use iterative way to compute when s > 1 */
+		return min + computeIterativeZipfian(thread->random_state, n, s) - 1;
+
+	return min + computeHarmonicZipfian(thread, n, s) - 1;
+}
+
 /*
  * Initialize the given SimpleStats struct to all zeroes
  */
@@ -1567,6 +1739,7 @@ evalFunc(TState *thread, CState *st,
 		case PGBENCH_RANDOM:
 		case PGBENCH_RANDOM_EXPONENTIAL:
 		case PGBENCH_RANDOM_GAUSSIAN:
+		case PGBENCH_RANDOM_ZIPFIAN:
 			{
 				int64		imin,
 							imax;
@@ -1617,6 +1790,18 @@ evalFunc(TState *thread, CState *st,
 						setIntValue(retval,
 									getGaussianRand(thread, imin, imax, param));
 					}
+					else if (func == PGBENCH_RANDOM_ZIPFIAN)
+					{
+						if (param <= 0.0 || param == 1.0 || param > MAX_ZIPFIAN_PARAM)
+						{
+							fprintf(stderr,
+									"zipfian parameter must be in range (0, 1) U (1, %d]"
+									" (got %f)\n", MAX_ZIPFIAN_PARAM,param);
+							return false;
+						}
+						setIntValue(retval,
+									getZipfianRand(thread, imin, imax, param));
+					}
 					else		/* exponential */
 					{
 						if (param <= 0.0)
@@ -4273,6 +4458,8 @@ main(int argc, char **argv)
 		thread->random_state[2] = random();
 		thread->logfile = NULL; /* filled in later */
 		thread->latency_late = 0;
+		thread->zipf_cache.cells_inited = 0;
+		thread->zipf_cache.current_time = 0;
 		initStats(&thread->stats, 0);
 
 		nclients_dealt += thread->nstate;
diff --git a/src/bin/pgbench/pgbench.h b/src/bin/pgbench/pgbench.h
index abc13e9463..1a29f1260c 100644
--- a/src/bin/pgbench/pgbench.h
+++ b/src/bin/pgbench/pgbench.h
@@ -75,7 +75,8 @@ typedef enum PgBenchFunction
 	PGBENCH_SQRT,
 	PGBENCH_RANDOM,
 	PGBENCH_RANDOM_GAUSSIAN,
-	PGBENCH_RANDOM_EXPONENTIAL
+	PGBENCH_RANDOM_EXPONENTIAL,
+	PGBENCH_RANDOM_ZIPFIAN
 } PgBenchFunction;
 
 typedef struct PgBenchExpr PgBenchExpr;
 
#44Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Alik Khilazhev (#43)
Re: [WIP] Zipfian distribution in pgbench

Hello Alik,

I am attaching patch v7.

Patch generates multiple warnings with "git apply", apparently because of
end-of-line spaces, and fails:

pgbench-zipf-07v.patch:52: trailing whitespace.
{
pgbench-zipf-07v.patch:53: trailing whitespace.
"random_zipfian", 3, PGBENCH_RANDOM_ZIPFIAN
pgbench-zipf-07v.patch:54: trailing whitespace.
},
pgbench-zipf-07v.patch:66: trailing whitespace.
#define ZIPF_CACHE_SIZE 15 /* cache cells number */
pgbench-zipf-07v.patch:67: trailing whitespace.

error: patch failed: doc/src/sgml/ref/pgbench.sgml:1013
error: doc/src/sgml/ref/pgbench.sgml: patch does not apply
error: patch failed: src/bin/pgbench/exprparse.y:191
error: src/bin/pgbench/exprparse.y: patch does not apply
error: patch failed: src/bin/pgbench/pgbench.c:93
error: src/bin/pgbench/pgbench.c: patch does not apply
error: patch failed: src/bin/pgbench/pgbench.h:75
error: src/bin/pgbench/pgbench.h: patch does not apply

It also complains with "patch", although it seems to work:

(Stripping trailing CRs from patch; use --binary to disable.)
patching file doc/src/sgml/ref/pgbench.sgml
(Stripping trailing CRs from patch; use --binary to disable.)
patching file src/bin/pgbench/exprparse.y
(Stripping trailing CRs from patch; use --binary to disable.)
patching file src/bin/pgbench/pgbench.c
(Stripping trailing CRs from patch; use --binary to disable.)
patching file src/bin/pgbench/pgbench.h
patch unexpectedly ends in middle of line
patch unexpectedly ends in middle of line

Please do not put spaces at the end of lines, eg there is a lonely tab on
the second line of "computeHarmonicZipfian".

It seems that "CR" characters are used:

sh> sha1sum ~/pgbench-zipf-07v.patch
439ad1de0a960b3b415ff6de9412b3cc4d6922e7

sh> file ~/pgbench-zipf-07v.patch
pgbench-zipf-07v.patch: unified diff output, ASCII text, with CRLF line terminators

Compilation issues one warning:

pgbench.c: In function οΏ½computeHarmonicZipfianοΏ½:
pgbench.c:894:2: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
double uniform = pg_erand48(thread->random_state);

Please conform to pg standard for portability, even if it is a very old one:-)

About the documentation:

I would suggest to replace 0.5 in the first example by 1.5, as a zipfian
distribution with a lower-than-one parameter is not that well defined.

I'm fine with using references to papers or books for the algorithm.

The documentation lines in the sgml file are too long. At least
limit text paragraphs to maybe 80 columns as done elsewhere.

typo: "<entry> Zipfian" (remove space)

typo: "used(described" (missing space)

typo: "numbers(algorithm" (again)

The sentence "It's recommended to pick <replaceable>parameter</> in range
(0, 1) for better accuracy." does not make sense with the updated
generation algorithm.

The text would benefit from a review by a native English speaker, who I am
not. Anyway here is an attempt at improving/simplifying the text (I have
removed sgml formatting). If some native English speaker could review it,
that would be great.

"""
"random_zipfian" generates an approximated bounded zipfian distribution.
For parameter in (0,1), an approximated algorithm is taken from
"Quickly Generating Billion-Record Synthetic Databases", Jim Gray et al, SIGMOD 1994.
For parameter in (1, 1000), a rejection method is used, based on
"Non-Uniform Random Variate Generation", Luc Devroye, p. 550-551, Springer 1986.
The distribution is not defined when the parameter's value is 1.0.
The drawing performance is poor for parameter values close and above 1.0
and on a small range.

"parameter" defines how skewed the distribution is.
The larger the "parameter", the more frequently values close to the beginning
of the interval are drawn.
The closer to 0 "parameter" is, the flatter (more uniform) the access distribution.
"""

English in comments and code:

"inited" is not English, I think you want "initialized". Maybe "nb_cells"
would do.

"it is not exist" (multiple instances) -> "it does not exist".

I think that the references to the paper/book should appear as comments in
the code, for instance in front of the functions which implement the
algorithm.

"current_time", "last_touch_time" are counter based, they are not "time".
I suggest to update the name to "current" and "last_touch" or "last_used".

"time when cell was used last time" -> "last used logical time"

About the code:

I would prefer that you use "1.0" instead of "1." for double constants.

I think that getZipfianRand should be symmetric. Maybe a direct formula
could do:

return min - 1 + (s > 1) ? xxx() : yyy();

or at least use an explicit "else" for symmetry.

Function "zipfn" asks for s and n as arguments, but ISTM that they are
already include as fields in cell, so this seems redundant. Moreover I do
not think that the zipfn function is that useful. I would suggest to
inline it in its caller.

"zipfGeneralizedHarmonic" is not really related to zipf, it is just used
there. I suggest to call it "generalizedHarmonicNumber" to reflect what it
does.

I suggest to put all LRU management in the getCell* function, that is to
move the last_use update from computeHarmonicZipfian to getCell.

computeHarminicZipfian and computeIterativeZipfian should have the same
signature, not one with a random state and the other with the thread. I
suggest to do the same as other random functions, whatever it is.

Otherwise, no action required:

The patch probably conflicst with other patches in the CF, which means
that there will be rebase needed. That is life. The queue is long and
slow.

Also, this function deserve some tap tests, that should be added if the
"pgbench tap test" patch get through. Otherwise it is as well tested as
everything else around, which is basically no test.

--
Fabien.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#45Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Fabien COELHO (#44)
1 attachment(s)
Re: [WIP] Zipfian distribution in pgbench

Hello Fabien,

Thank you for detailed review. I hope I have fixed all the issues you mentioned in your letter.

Attachments:

pgbench-zipf-08v.patchapplication/octet-stream; name=pgbench-zipf-08v.patch; x-unix-mode=0644Download
diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml
index 03e1212d50..f272b7dca4 100644
--- a/doc/src/sgml/ref/pgbench.sgml
+++ b/doc/src/sgml/ref/pgbench.sgml
@@ -1013,6 +1013,14 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
        <entry><literal>random_gaussian(1, 10, 2.5)</></>
        <entry>an integer between <literal>1</> and <literal>10</></>
       </row>
+      <row>
+        <entry><literal><function>random_zipfian(<replaceable>lb</>, <replaceable>ub</>, <replaceable>parameter</>)</></></>
+        <entry>integer</>
+        <entry>Zipfian-distributed random integer in <literal>[lb, ub]</>,
+              see below</>
+        <entry><literal>random_zipfian(1, 10, 1.5)</></>
+        <entry>an integer between <literal>1</> and <literal>10</></>
+      </row>
       <row>
        <entry><literal><function>sqrt(<replaceable>x</>)</></></>
        <entry>double</>
@@ -1094,6 +1102,28 @@ f(x) = PHI(2.0 * parameter * (x - mu) / (max - min + 1)) /
       of the Box-Muller transform.
      </para>
     </listitem>
+    <listitem>
+        <para>
+          <literal>random_zipfian</> generates an approximated bounded zipfian
+          distribution. For <replaceable>parameter</> in (0, 1), an
+          approximated algorithm is taken from
+          "Quickly Generating Billion-Record Synthetic Databases",
+          Jim Gray et al, SIGMOD 1994. For <replaceable>parameter</>
+          in (1, 1000), a rejection method used, based on
+          "Non-Uniform Random Variate Generation", Luc Devroye, p. 550-551,
+          Springer 1986. The distribution is not defined when the parameter's
+          value is 1.0. The drawing performance is poor for parameter values
+          close and above 1.0 and on a small range.
+          
+        </para>
+        <para>
+          <replaceable>parameter</>
+          defines how skewed the distribution is. The larger the <replaceable>parameter</>, the more
+          frequently values to the beginning of the interval are drawn.
+          The closer to 0 <replaceable>parameter</> is,
+          the flatter (more uniform) the access distribution.
+        </para>
+    </listitem>
    </itemizedlist>
 
   <para>
diff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.y
index b3a2d9bfd3..25d5ad48e5 100644
--- a/src/bin/pgbench/exprparse.y
+++ b/src/bin/pgbench/exprparse.y
@@ -191,6 +191,9 @@ static const struct
 	{
 		"random_exponential", 3, PGBENCH_RANDOM_EXPONENTIAL
 	},
+	{
+		"random_zipfian", 3, PGBENCH_RANDOM_ZIPFIAN
+	},
 	/* keep as last array element */
 	{
 		NULL, 0, 0
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index ae78c7b1d4..c0fc98975f 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -93,7 +93,10 @@ static int	pthread_join(pthread_t th, void **thread_return);
 #define LOG_STEP_SECONDS	5	/* seconds between log messages */
 #define DEFAULT_NXACTS	10		/* default nxacts */
 
+#define ZIPF_CACHE_SIZE	15		/* cache cells number */
+
 #define MIN_GAUSSIAN_PARAM		2.0 /* minimum parameter for gauss */
+#define MAX_ZIPFIAN_PARAM		1000	/* maximum parameter for zipfian */
 
 int			nxacts = 0;			/* number of transactions per client */
 int			duration = 0;		/* duration in seconds */
@@ -333,6 +336,36 @@ typedef struct
 	int			ecnt;			/* error count */
 } CState;
 
+/*
+ * Cache cell for zipfian_random call
+ */
+typedef struct
+{
+	/* cell keys */
+	double		s;				/* s - parameter of zipfan_random function */
+	int64		n;				/* number of elements in range (max - min + 1) */
+
+	double		harmonicn;		/* generalizedHarmonicNumber(n, s) */
+	double		alpha;
+	double		beta;
+	double		eta;
+
+	uint64		last_used;		/* last used logical time */
+} ZipfCell;
+
+/*
+ * Zipf cache for zeta values
+ */
+typedef struct
+{
+	uint64		current;		/* counter for LRU cache replacement algorithm */
+
+	int			nb_cells;		/* number of filled cells */
+	ZipfCell	cells[ZIPF_CACHE_SIZE];
+} ZipfCache;
+
+bool		zipfCacheOverflowMessagePrinted = false;
+
 /*
  * Thread state
  */
@@ -345,6 +378,8 @@ typedef struct
 	unsigned short random_state[3]; /* separate randomness for each thread */
 	int64		throttle_trigger;	/* previous/next throttling (us) */
 	FILE	   *logfile;		/* where to log, or NULL */
+	ZipfCache	zipf_cache;		/* for thread-safe  zipfian random number
+								 * generation */
 
 	/* per thread collected stats */
 	instr_time	start_time;		/* thread start time */
@@ -737,6 +772,140 @@ getPoissonRand(TState *thread, int64 center)
 	return (int64) (-log(uniform) * ((double) center) + 0.5);
 }
 
+/* helper function for getZipfianRand */
+static double
+generalizedHarmonicNumber(int64 n, double s)
+{
+	int			i;
+	double		ans = 0.0;
+
+	for (i = n; i > 1; i--)
+		ans += pow(i, -s);
+	return ans + 1.0;
+}
+
+/* set harmonicn and other parameters to cache cell */
+static void
+zipfSetCacheCell(ZipfCell * cell, int64 n, double s)
+{
+	double		harmonic2;
+
+	cell->n = n;
+	cell->s = s;
+
+	harmonic2 = generalizedHarmonicNumber(2, s);
+	cell->harmonicn = generalizedHarmonicNumber(n, s);
+
+	cell->alpha = 1.0 / (1.0 - s);
+	cell->beta = pow(0.5, s);
+	cell->eta = (1.0 - pow(2.0 / n, 1.0 - s)) / (1.0 - harmonic2 / cell->harmonicn);
+}
+
+/*
+ * search for cache cell with keys (n, s)
+ * and create new cell if it does not exist
+ */
+static ZipfCell *
+zipfFindOrCreateCacheCell(ZipfCache * cache, int64 n, double s)
+{
+	int			i,
+				least_recently_used_i = 0;
+	ZipfCell   *cell;
+
+	/* search cached cell for given parameters */
+	for (i = 0; i < cache->nb_cells; i++)
+	{
+		cell = &cache->cells[i];
+		if (cell->n == n && cell->s == s)
+			return &cache->cells[i];
+
+		if (cell->last_used < cache->cells[least_recently_used_i].last_used)
+			least_recently_used_i = i;
+	}
+
+	/* create new one if it does not exist */
+	if (cache->nb_cells != ZIPF_CACHE_SIZE)
+		i = cache->nb_cells++;
+	else
+	{
+		/* replace LRU cell if cache is full */
+		i = least_recently_used_i;
+		if (!zipfCacheOverflowMessagePrinted)
+		{
+			/*
+			 * flag can be accessed in several threads simultaneously and
+			 * message can be printed more than one time
+			 */
+			zipfCacheOverflowMessagePrinted = true;
+			fprintf(stderr, "zipfian cache array overflowed\n");
+		}
+	}
+
+	zipfSetCacheCell(&cache->cells[i], n, s);
+
+	cache->cells[i].last_used = cache->current++;
+	return &cache->cells[i];
+}
+/*
+ * Computing zipfian usinng rejection method, based on
+ * "Non-Uniform Random Variate Generation",
+ * Luc Devroye, p. 550-551, Springer 1986.
+ */
+static int64
+computeIterativeZipfian(TState *thread, int64 n, double s)
+{
+	double		b = pow(2.0, s - 1.0);
+	double		x,
+				t,
+				u,
+				v;
+
+	while (true)
+	{
+		/* random variates */
+		u = pg_erand48(thread->random_state);
+		v = pg_erand48(thread->random_state);
+
+		x = floor(pow(u, -1.0 / (s - 1.0)));
+
+		t = pow(1.0 + 1.0 / x, s - 1.0);
+		/* reject if too large or out of bound */
+		if (v * x * (t - 1.0) / (b - 1.0) <= t / b && x <= n)
+			break;
+	}
+	return (int64) x;
+}
+
+/*
+ * Computing zipfian using harmonic numbers, based on algorithm described in
+ * "Quickly Generating Billion-Record Synthetic Databases",
+ * Jim Gray et al, SIGMOD 1994
+ */
+static int64
+computeHarmonicZipfian(TState *thread, int64 n, double s)
+{
+	ZipfCell   *cell = zipfFindOrCreateCacheCell(&thread->zipf_cache, n, s);
+	double		uniform = pg_erand48(thread->random_state);
+	double		uz = uniform * cell->harmonicn;
+
+	if (uz < 1.0)
+		return 1;
+	if (uz < 1.0 + cell->beta)
+		return 2;
+	return 1 + (int64) (cell->n * pow(cell->eta * uniform - cell->eta + 1.0, cell->alpha));
+}
+
+/* random number generator: zipfian distribution from min to max inclusive */
+static int64
+getZipfianRand(TState *thread, int64 min, int64 max, double s)
+{
+	int64		n = max - min + 1;
+
+	return min - 1 + ((s > 1)
+					  ? computeIterativeZipfian(thread, n, s)
+					  : computeHarmonicZipfian(thread, n, s));
+}
+
 /*
  * Initialize the given SimpleStats struct to all zeroes
  */
@@ -1567,6 +1736,7 @@ evalFunc(TState *thread, CState *st,
 		case PGBENCH_RANDOM:
 		case PGBENCH_RANDOM_EXPONENTIAL:
 		case PGBENCH_RANDOM_GAUSSIAN:
+		case PGBENCH_RANDOM_ZIPFIAN:
 			{
 				int64		imin,
 							imax;
@@ -1617,6 +1787,18 @@ evalFunc(TState *thread, CState *st,
 						setIntValue(retval,
 									getGaussianRand(thread, imin, imax, param));
 					}
+					else if (func == PGBENCH_RANDOM_ZIPFIAN)
+					{
+						if (param <= 0.0 || param == 1.0 || param > MAX_ZIPFIAN_PARAM)
+						{
+							fprintf(stderr,
+									"zipfian parameter must be in range (0, 1) U (1, %d]"
+									" (got %f)\n", MAX_ZIPFIAN_PARAM, param);
+							return false;
+						}
+						setIntValue(retval,
+									getZipfianRand(thread, imin, imax, param));
+					}
 					else		/* exponential */
 					{
 						if (param <= 0.0)
@@ -4273,6 +4455,8 @@ main(int argc, char **argv)
 		thread->random_state[2] = random();
 		thread->logfile = NULL; /* filled in later */
 		thread->latency_late = 0;
+		thread->zipf_cache.nb_cells = 0;
+		thread->zipf_cache.current = 0;
 		initStats(&thread->stats, 0);
 
 		nclients_dealt += thread->nstate;
diff --git a/src/bin/pgbench/pgbench.h b/src/bin/pgbench/pgbench.h
index abc13e9463..6eff7016ea 100644
--- a/src/bin/pgbench/pgbench.h
+++ b/src/bin/pgbench/pgbench.h
@@ -75,7 +75,8 @@ typedef enum PgBenchFunction
 	PGBENCH_SQRT,
 	PGBENCH_RANDOM,
 	PGBENCH_RANDOM_GAUSSIAN,
-	PGBENCH_RANDOM_EXPONENTIAL
+	PGBENCH_RANDOM_EXPONENTIAL,
+	PGBENCH_RANDOM_ZIPFIAN
 } PgBenchFunction;
 
 typedef struct PgBenchExpr PgBenchExpr;
 
#46Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Alik Khilazhev (#45)
Re: [WIP] Zipfian distribution in pgbench

Hello Alik,

Applies, compiles, works for me.

Some minor comments and suggestions.

Two typos:
- "usinng" -> "using"
- "a rejection method used" -> "a rejection method is used"

I'm not sure of "least_recently_used_i", this naming style is not used in
pgbench. "least_recently_used" would be ok.

"..nb_cells.. != ZIPF_CACHE_SIZE", ISTM that "<" is more logical,
even if the result is the same?

I would put the parameter value check in getZipfianRand, so that if
someone reuse the function elsewhere the check is also performed. That
would also simplify a bit the already very large expression evaluation
function.

When/if the pgbench tap test patch get through, coverage tests should
be added.

Maybe the cache overflow could be counted, to allow for a possible warning
message in the final report?

--
Fabien.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#47Daniel Gustafsson
daniel@yesql.se
In reply to: Fabien COELHO (#46)
Re: [WIP] Zipfian distribution in pgbench

On 06 Sep 2017, at 08:42, Fabien COELHO <coelho@cri.ensmp.fr> wrote:

Hello Alik,

Applies, compiles, works for me.

Some minor comments and suggestions.

Two typos:
- "usinng" -> "using"
- "a rejection method used" -> "a rejection method is used"

I'm not sure of "least_recently_used_i", this naming style is not used in pgbench. "least_recently_used" would be ok.

"..nb_cells.. != ZIPF_CACHE_SIZE", ISTM that "<" is more logical,
even if the result is the same?

I would put the parameter value check in getZipfianRand, so that if someone reuse the function elsewhere the check is also performed. That would also simplify a bit the already very large expression evaluation function.

When/if the pgbench tap test patch get through, coverage tests should
be added.

Maybe the cache overflow could be counted, to allow for a possible warning message in the final report?

Since this patch has been Waiting for author and no update on this patch has
been posted during the commitfest, it is Returned with feedback. When you have
a new version of the patch, please re-submit to a future commitfest.

cheers ./daniel

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#48Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Fabien COELHO (#46)
1 attachment(s)
Re: [HACKERS] [WIP] Zipfian distribution in pgbench

Hello Fabien,

Sorry for not responding for long time.

Two typos:
- "usinng" -> "using"
- "a rejection method used" -> "a rejection method is used"

I'm not sure of "least_recently_used_i", this naming style is not used in pgbench. "least_recently_used" would be ok.

"..nb_cells.. != ZIPF_CACHE_SIZE", ISTM that "<" is more logical,
even if the result is the same?

Fixed.

Maybe the cache overflow could be counted, to allow for a possible warning message in the final report?

Also done. But one question: Should it be done by printing to stdout or stderr ?

When/if the pgbench tap test patch get through, coverage tests should
be added.

I have added few tests and I am not sure if they OK or not. It was my first experience with perl and tests in PostgreSQL.

Attachments:

pgbench-zipf-09v.patchapplication/octet-stream; name=pgbench-zipf-09v.patch; x-unix-mode=0644Download
diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml
index c48a69713a..3d628a1d16 100644
--- a/doc/src/sgml/ref/pgbench.sgml
+++ b/doc/src/sgml/ref/pgbench.sgml
@@ -1092,6 +1092,14 @@ pgbench <optional> <replaceable>options</replaceable> </optional> <replaceable>d
        <entry><literal>random_gaussian(1, 10, 2.5)</literal></entry>
        <entry>an integer between <literal>1</literal> and <literal>10</literal></entry>
       </row>
+      <row>
+       <entry><literal><function>random_zipfian(<replaceable>lb</>, <replaceable>ub</>, <replaceable>parameter</>)</></></>
+       <entry>integer</>
+       <entry>Zipfian-distributed random integer in <literal>[lb, ub]</>,
+              see below</>
+       <entry><literal>random_zipfian(1, 10, 1.5)</></>
+       <entry>an integer between <literal>1</> and <literal>10</></>
+      </row>
       <row>
        <entry><literal><function>sqrt(<replaceable>x</replaceable>)</function></literal></entry>
        <entry>double</entry>
@@ -1173,6 +1181,27 @@ f(x) = PHI(2.0 * parameter * (x - mu) / (max - min + 1)) /
       of the Box-Muller transform.
      </para>
     </listitem>
+    <listitem>
+     <para>
+      <literal>random_zipfian</> generates an approximated bounded zipfian
+      distribution. For <replaceable>parameter</> in (0, 1), an
+      approximated algorithm is taken from
+      "Quickly Generating Billion-Record Synthetic Databases",
+      Jim Gray et al, SIGMOD 1994. For <replaceable>parameter</>
+      in (1, 1000), a rejection method is used, based on
+      "Non-Uniform Random Variate Generation", Luc Devroye, p. 550-551,
+      Springer 1986. The distribution is not defined when the parameter's
+      value is 1.0. The drawing performance is poor for parameter values
+      close and above 1.0 and on a small range.
+     </para>
+     <para>
+      <replaceable>parameter</>
+      defines how skewed the distribution is. The larger the <replaceable>parameter</>, the more
+      frequently values to the beginning of the interval are drawn.
+      The closer to 0 <replaceable>parameter</> is,
+      the flatter (more uniform) the access distribution.
+     </para>
+    </listitem>
    </itemizedlist>
 
   <para>
diff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.y
index b3a2d9bfd3..25d5ad48e5 100644
--- a/src/bin/pgbench/exprparse.y
+++ b/src/bin/pgbench/exprparse.y
@@ -191,6 +191,9 @@ static const struct
 	{
 		"random_exponential", 3, PGBENCH_RANDOM_EXPONENTIAL
 	},
+	{
+		"random_zipfian", 3, PGBENCH_RANDOM_ZIPFIAN
+	},
 	/* keep as last array element */
 	{
 		NULL, 0, 0
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index ceb2fcc3ad..832ec59b38 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -95,7 +95,10 @@ static int	pthread_join(pthread_t th, void **thread_return);
 #define LOG_STEP_SECONDS	5	/* seconds between log messages */
 #define DEFAULT_NXACTS	10		/* default nxacts */
 
+#define ZIPF_CACHE_SIZE	15		/* cache cells number */
+
 #define MIN_GAUSSIAN_PARAM		2.0 /* minimum parameter for gauss */
+#define MAX_ZIPFIAN_PARAM		1000	/* maximum parameter for zipfian */
 
 int			nxacts = 0;			/* number of transactions per client */
 int			duration = 0;		/* duration in seconds */
@@ -330,6 +333,35 @@ typedef struct
 	int			ecnt;			/* error count */
 } CState;
 
+/*
+ * Cache cell for zipfian_random call
+ */
+typedef struct
+{
+	/* cell keys */
+	double		s;				/* s - parameter of zipfan_random function */
+	int64		n;				/* number of elements in range (max - min + 1) */
+
+	double		harmonicn;		/* generalizedHarmonicNumber(n, s) */
+	double		alpha;
+	double		beta;
+	double		eta;
+
+	uint64		last_used;		/* last used logical time */
+} ZipfCell;
+
+/*
+ * Zipf cache for zeta values
+ */
+typedef struct
+{
+	uint64		current;		/* counter for LRU cache replacement algorithm */
+
+	int			nb_cells;		/* number of filled cells */
+	int			overflowCount;	/* number of cache overflows */
+	ZipfCell	cells[ZIPF_CACHE_SIZE];
+} ZipfCache;
+
 /*
  * Thread state
  */
@@ -342,6 +374,8 @@ typedef struct
 	unsigned short random_state[3]; /* separate randomness for each thread */
 	int64		throttle_trigger;	/* previous/next throttling (us) */
 	FILE	   *logfile;		/* where to log, or NULL */
+	ZipfCache	zipf_cache;		/* for thread-safe  zipfian random number
+								 * generation */
 
 	/* per thread collected stats */
 	instr_time	start_time;		/* thread start time */
@@ -746,6 +780,135 @@ getPoissonRand(TState *thread, int64 center)
 	return (int64) (-log(uniform) * ((double) center) + 0.5);
 }
 
+/* helper function for getZipfianRand */
+static double
+generalizedHarmonicNumber(int64 n, double s)
+{
+	int			i;
+	double		ans = 0.0;
+
+	for (i = n; i > 1; i--)
+		ans += pow(i, -s);
+	return ans + 1.0;
+}
+
+/* set harmonicn and other parameters to cache cell */
+static void
+zipfSetCacheCell(ZipfCell * cell, int64 n, double s)
+{
+	double		harmonic2;
+
+	cell->n = n;
+	cell->s = s;
+
+	harmonic2 = generalizedHarmonicNumber(2, s);
+	cell->harmonicn = generalizedHarmonicNumber(n, s);
+
+	cell->alpha = 1.0 / (1.0 - s);
+	cell->beta = pow(0.5, s);
+	cell->eta = (1.0 - pow(2.0 / n, 1.0 - s)) / (1.0 - harmonic2 / cell->harmonicn);
+}
+
+/*
+ * search for cache cell with keys (n, s)
+ * and create new cell if it does not exist
+ */
+static ZipfCell *
+zipfFindOrCreateCacheCell(ZipfCache * cache, int64 n, double s)
+{
+	int			i,
+				least_recently_used = 0;
+	ZipfCell   *cell;
+
+	/* search cached cell for given parameters */
+	for (i = 0; i < cache->nb_cells; i++)
+	{
+		cell = &cache->cells[i];
+		if (cell->n == n && cell->s == s)
+			return &cache->cells[i];
+
+		if (cell->last_used < cache->cells[least_recently_used].last_used)
+			least_recently_used = i;
+	}
+
+	/* create new one if it does not exist */
+	if (cache->nb_cells < ZIPF_CACHE_SIZE)
+		i = cache->nb_cells++;
+	else
+	{
+		/* replace LRU cell if cache is full */
+		i = least_recently_used;
+		cache->overflowCount++;
+	}
+
+	zipfSetCacheCell(&cache->cells[i], n, s);
+
+	cache->cells[i].last_used = cache->current++;
+	return &cache->cells[i];
+}
+/*
+ * Computing zipfian using rejection method, based on
+ * "Non-Uniform Random Variate Generation",
+ * Luc Devroye, p. 550-551, Springer 1986.
+ */
+static int64
+computeIterativeZipfian(TState *thread, int64 n, double s)
+{
+	double		b = pow(2.0, s - 1.0);
+	double		x,
+				t,
+				u,
+				v;
+
+	while (true)
+	{
+		/* random variates */
+		u = pg_erand48(thread->random_state);
+		v = pg_erand48(thread->random_state);
+
+		x = floor(pow(u, -1.0 / (s - 1.0)));
+
+		t = pow(1.0 + 1.0 / x, s - 1.0);
+		/* reject if too large or out of bound */
+		if (v * x * (t - 1.0) / (b - 1.0) <= t / b && x <= n)
+			break;
+	}
+	return (int64) x;
+}
+
+/*
+ * Computing zipfian using harmonic numbers, based on algorithm described in
+ * "Quickly Generating Billion-Record Synthetic Databases",
+ * Jim Gray et al, SIGMOD 1994
+ */
+static int64
+computeHarmonicZipfian(TState *thread, int64 n, double s)
+{
+	ZipfCell   *cell = zipfFindOrCreateCacheCell(&thread->zipf_cache, n, s);
+	double		uniform = pg_erand48(thread->random_state);
+	double		uz = uniform * cell->harmonicn;
+
+	if (uz < 1.0)
+		return 1;
+	if (uz < 1.0 + cell->beta)
+		return 2;
+	return 1 + (int64) (cell->n * pow(cell->eta * uniform - cell->eta + 1.0, cell->alpha));
+}
+
+/* random number generator: zipfian distribution from min to max inclusive */
+static int64
+getZipfianRand(TState *thread, int64 min, int64 max, double s)
+{
+	/* abort if parameter is invalid */
+	Assert(s > 0.0 && s != 1.0 && s <= MAX_ZIPFIAN_PARAM);
+
+	int64		n = max - min + 1;
+
+	return min - 1 + ((s > 1)
+					  ? computeIterativeZipfian(thread, n, s)
+					  : computeHarmonicZipfian(thread, n, s));
+}
+
 /*
  * Initialize the given SimpleStats struct to all zeroes
  */
@@ -1303,7 +1466,6 @@ coerceToDouble(PgBenchValue *pval, double *dval)
 		return true;
 	}
 }
-
 /* assign an integer value */
 static void
 setIntValue(PgBenchValue *pv, int64 ival)
@@ -1605,6 +1767,7 @@ evalFunc(TState *thread, CState *st,
 		case PGBENCH_RANDOM:
 		case PGBENCH_RANDOM_EXPONENTIAL:
 		case PGBENCH_RANDOM_GAUSSIAN:
+		case PGBENCH_RANDOM_ZIPFIAN:
 			{
 				int64		imin,
 							imax;
@@ -1655,6 +1818,18 @@ evalFunc(TState *thread, CState *st,
 						setIntValue(retval,
 									getGaussianRand(thread, imin, imax, param));
 					}
+					else if (func == PGBENCH_RANDOM_ZIPFIAN)
+					{
+						if (param <= 0.0 || param == 1.0 || param > MAX_ZIPFIAN_PARAM)
+						{
+							fprintf(stderr,
+									"zipfian parameter must be in range (0, 1) U (1, %d]"
+									" (got %f)\n", MAX_ZIPFIAN_PARAM, param);
+							return false;
+						}
+						setIntValue(retval,
+									getZipfianRand(thread, imin, imax, param));
+					}
 					else		/* exponential */
 					{
 						if (param <= 0.0)
@@ -3680,6 +3855,8 @@ printResults(TState *threads, StatsData *total, instr_time total_time,
 	double		time_include,
 				tps_include,
 				tps_exclude;
+	int			i,
+				totalCacheOverflows = 0;
 
 	time_include = INSTR_TIME_GET_DOUBLE(total_time);
 	tps_include = total->cnt / time_include;
@@ -3705,6 +3882,15 @@ printResults(TState *threads, StatsData *total, instr_time total_time,
 		printf("number of transactions actually processed: " INT64_FORMAT "\n",
 			   total->cnt);
 	}
+	/* Report zipfian cache overflow */
+	for (i = 0; i < nthreads; i++)
+	{
+		totalCacheOverflows += threads[i].zipf_cache.overflowCount;
+	}
+	if (totalCacheOverflows > 0)
+	{
+		printf("zipfian cache array overflowed %d times\n", totalCacheOverflows);
+	}
 
 	/* Remaining stats are nonsensical if we failed to execute any xacts */
 	if (total->cnt <= 0)
@@ -4501,6 +4687,9 @@ main(int argc, char **argv)
 		thread->random_state[2] = random();
 		thread->logfile = NULL; /* filled in later */
 		thread->latency_late = 0;
+		thread->zipf_cache.nb_cells = 0;
+		thread->zipf_cache.current = 0;
+		thread->zipf_cache.overflowCount = 0;
 		initStats(&thread->stats, 0);
 
 		nclients_dealt += thread->nstate;
diff --git a/src/bin/pgbench/pgbench.h b/src/bin/pgbench/pgbench.h
index fd428af274..83fee1ae74 100644
--- a/src/bin/pgbench/pgbench.h
+++ b/src/bin/pgbench/pgbench.h
@@ -75,7 +75,8 @@ typedef enum PgBenchFunction
 	PGBENCH_SQRT,
 	PGBENCH_RANDOM,
 	PGBENCH_RANDOM_GAUSSIAN,
-	PGBENCH_RANDOM_EXPONENTIAL
+	PGBENCH_RANDOM_EXPONENTIAL,
+	PGBENCH_RANDOM_ZIPFIAN
 } PgBenchFunction;
 
 typedef struct PgBenchExpr PgBenchExpr;
diff --git a/src/bin/pgbench/t/001_pgbench_with_server.pl b/src/bin/pgbench/t/001_pgbench_with_server.pl
index 864d580c64..3c7f3582be 100644
--- a/src/bin/pgbench/t/001_pgbench_with_server.pl
+++ b/src/bin/pgbench/t/001_pgbench_with_server.pl
@@ -261,6 +261,8 @@ pgbench(
 \set maxint debug(:minint - 1)
 -- reset a variable
 \set i1 0
+-- yet another integer function
+\set id debug(random_zipfian(1, 100, 2))
 } });
 
 # backslash commands
@@ -371,6 +373,14 @@ SELECT LEAST(:i, :i, :i, :i, :i, :i, :i, :i, :i, :i, :i);
 		0,
 		[qr{exponential parameter must be greater }],
 		q{\set i random_exponential(0, 10, 0.0)} ],
+	[	'set zipfian param to 1',
+		0,
+		[qr{zipfian parameter must be in range \(0, 1\) U \(1, \d+\]}],
+		q{\set i random_zipfian(0, 10, 1)} ],
+	[	'set zipfian param too large',
+		0,
+		[qr{zipfian parameter must be in range \(0, 1\) U \(1, \d+\]}],
+		q{\set i random_zipfian(0, 10, 1000000)} ],
 	[   'set non numeric value',                     0,
 		[qr{malformed variable "foo" value: "bla"}], q{\set i :foo + 1} ],
 	[ 'set no expression',    1, [qr{syntax error}],      q{\set i} ],
@@ -412,6 +422,31 @@ for my $e (@errors)
 		{ $n => $script });
 }
 
+# zipfian cache array overflow
+pgbench(
+	'-t 1', 0,
+	[ qr{processed: 1/1}, qr{zipfian cache array overflowed 1 times} ],
+	[ qr{^} ],
+	'pgbench zipfian array overflow on \random_zipfian',
+	{   '001_pgbench_random_zipfian' => q{
+\set i random_zipfian(1, 100, 0.5)
+\set i random_zipfian(2, 100, 0.5)
+\set i random_zipfian(3, 100, 0.5)
+\set i random_zipfian(4, 100, 0.5)
+\set i random_zipfian(5, 100, 0.5)
+\set i random_zipfian(6, 100, 0.5)
+\set i random_zipfian(7, 100, 0.5)
+\set i random_zipfian(8, 100, 0.5)
+\set i random_zipfian(9, 100, 0.5)
+\set i random_zipfian(10, 100, 0.5)
+\set i random_zipfian(11, 100, 0.5)
+\set i random_zipfian(12, 100, 0.5)
+\set i random_zipfian(13, 100, 0.5)
+\set i random_zipfian(14, 100, 0.5)
+\set i random_zipfian(15, 100, 0.5)
+\set i random_zipfian(16, 100, 0.5)
+} });
+
 # throttling
 pgbench(
 	'-t 100 -S --rate=100000 --latency-limit=1000000 -c 2 -n -r',
#49Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Alik Khilazhev (#48)
Re: [HACKERS] [WIP] Zipfian distribution in pgbench

Hello Alik,

Patch applies with "patch", but not with "git apply", probably because it
is in CR-NL eol format. No big deal.

Documentation has been switched from SGML to XML, so now tags must be
explicitely closed, i.e. use <foo>x</foo> instead of <foo>x</>.

Patch compiles with a warning:

pgbench.c: In function οΏ½getZipfianRandοΏ½:
pgbench.c:905:2: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]

Declaration must move before assertion.

The random_zipfian(1,100,2) call result is not tested.

"1 times" -> "once".

"array overflow on \random_zipfian" remove spurious "\".

--
Fabien.

#50Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Fabien COELHO (#49)
1 attachment(s)
Re: [HACKERS] [WIP] Zipfian distribution in pgbench

Patch applies with "patch", but not with "git apply", probably because it is in CR-NL eol format. No big deal.

Documentation has been switched from SGML to XML, so now tags must be explicitely closed, i.e. use <foo>x</foo> instead of <foo>x</>.

Patch compiles with a warning:

Declaration must move before assertion.

"array overflow on \random_zipfian" remove spurious "\".

Fixed.

The random_zipfian(1,100,2) call result is not tested.

I have reduced range to [1, 10] and updated the test.

"1 times" -> "once”.

I am not sure that it need to be fixed in this way, because there can be another number instead of 1.
So, I updated it to “%d time(s)”

Attachments:

pgbench-zipf-10v.patchapplication/octet-stream; name=pgbench-zipf-10v.patch; x-unix-mode=0644Download
diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml
index c48a69713a..d20f4aa498 100644
--- a/doc/src/sgml/ref/pgbench.sgml
+++ b/doc/src/sgml/ref/pgbench.sgml
@@ -1092,6 +1092,14 @@ pgbench <optional> <replaceable>options</replaceable> </optional> <replaceable>d
        <entry><literal>random_gaussian(1, 10, 2.5)</literal></entry>
        <entry>an integer between <literal>1</literal> and <literal>10</literal></entry>
       </row>
+      <row>
+       <entry><literal><function>random_zipfian(<replaceable>lb</replaceable>, <replaceable>ub</replaceable>, <replaceable>parameter</replaceable>)</function></literal></entry>
+       <entry>integer</entry>
+       <entry>Zipfian-distributed random integer in <literal>[lb, ub]</literal>,
+              see below</entry>
+       <entry><literal>random_zipfian(1, 10, 1.5)</literal></entry>
+       <entry>an integer between <literal>1</literal> and <literal>10</literal></entry>
+      </row>
       <row>
        <entry><literal><function>sqrt(<replaceable>x</replaceable>)</function></literal></entry>
        <entry>double</entry>
@@ -1173,6 +1181,27 @@ f(x) = PHI(2.0 * parameter * (x - mu) / (max - min + 1)) /
       of the Box-Muller transform.
      </para>
     </listitem>
+    <listitem>
+     <para>
+      <literal>random_zipfian</literal> generates an approximated bounded zipfian
+      distribution. For <replaceable>parameter</replaceable> in (0, 1), an
+      approximated algorithm is taken from
+      "Quickly Generating Billion-Record Synthetic Databases",
+      Jim Gray et al, SIGMOD 1994. For <replaceable>parameter</replaceable>
+      in (1, 1000), a rejection method is used, based on
+      "Non-Uniform Random Variate Generation", Luc Devroye, p. 550-551,
+      Springer 1986. The distribution is not defined when the parameter's
+      value is 1.0. The drawing performance is poor for parameter values
+      close and above 1.0 and on a small range.
+     </para>
+     <para>
+      <replaceable>parameter</replaceable>
+      defines how skewed the distribution is. The larger the <replaceable>parameter</replaceable>, the more
+      frequently values to the beginning of the interval are drawn.
+      The closer to 0 <replaceable>parameter</replaceable> is,
+      the flatter (more uniform) the access distribution.
+     </para>
+    </listitem>
    </itemizedlist>
 
   <para>
diff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.y
index b3a2d9bfd3..25d5ad48e5 100644
--- a/src/bin/pgbench/exprparse.y
+++ b/src/bin/pgbench/exprparse.y
@@ -191,6 +191,9 @@ static const struct
 	{
 		"random_exponential", 3, PGBENCH_RANDOM_EXPONENTIAL
 	},
+	{
+		"random_zipfian", 3, PGBENCH_RANDOM_ZIPFIAN
+	},
 	/* keep as last array element */
 	{
 		NULL, 0, 0
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index ceb2fcc3ad..f794ff5669 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -95,7 +95,10 @@ static int	pthread_join(pthread_t th, void **thread_return);
 #define LOG_STEP_SECONDS	5	/* seconds between log messages */
 #define DEFAULT_NXACTS	10		/* default nxacts */
 
+#define ZIPF_CACHE_SIZE	15		/* cache cells number */
+
 #define MIN_GAUSSIAN_PARAM		2.0 /* minimum parameter for gauss */
+#define MAX_ZIPFIAN_PARAM		1000	/* maximum parameter for zipfian */
 
 int			nxacts = 0;			/* number of transactions per client */
 int			duration = 0;		/* duration in seconds */
@@ -330,6 +333,35 @@ typedef struct
 	int			ecnt;			/* error count */
 } CState;
 
+/*
+ * Cache cell for zipfian_random call
+ */
+typedef struct
+{
+	/* cell keys */
+	double		s;				/* s - parameter of zipfan_random function */
+	int64		n;				/* number of elements in range (max - min + 1) */
+
+	double		harmonicn;		/* generalizedHarmonicNumber(n, s) */
+	double		alpha;
+	double		beta;
+	double		eta;
+
+	uint64		last_used;		/* last used logical time */
+} ZipfCell;
+
+/*
+ * Zipf cache for zeta values
+ */
+typedef struct
+{
+	uint64		current;		/* counter for LRU cache replacement algorithm */
+
+	int			nb_cells;		/* number of filled cells */
+	int			overflowCount;	/* number of cache overflows */
+	ZipfCell	cells[ZIPF_CACHE_SIZE];
+} ZipfCache;
+
 /*
  * Thread state
  */
@@ -342,6 +374,8 @@ typedef struct
 	unsigned short random_state[3]; /* separate randomness for each thread */
 	int64		throttle_trigger;	/* previous/next throttling (us) */
 	FILE	   *logfile;		/* where to log, or NULL */
+	ZipfCache	zipf_cache;		/* for thread-safe  zipfian random number
+								 * generation */
 
 	/* per thread collected stats */
 	instr_time	start_time;		/* thread start time */
@@ -746,6 +780,136 @@ getPoissonRand(TState *thread, int64 center)
 	return (int64) (-log(uniform) * ((double) center) + 0.5);
 }
 
+/* helper function for getZipfianRand */
+static double
+generalizedHarmonicNumber(int64 n, double s)
+{
+	int			i;
+	double		ans = 0.0;
+
+	for (i = n; i > 1; i--)
+		ans += pow(i, -s);
+	return ans + 1.0;
+}
+
+/* set harmonicn and other parameters to cache cell */
+static void
+zipfSetCacheCell(ZipfCell * cell, int64 n, double s)
+{
+	double		harmonic2;
+
+	cell->n = n;
+	cell->s = s;
+
+	harmonic2 = generalizedHarmonicNumber(2, s);
+	cell->harmonicn = generalizedHarmonicNumber(n, s);
+
+	cell->alpha = 1.0 / (1.0 - s);
+	cell->beta = pow(0.5, s);
+	cell->eta = (1.0 - pow(2.0 / n, 1.0 - s)) / (1.0 - harmonic2 / cell->harmonicn);
+}
+
+/*
+ * search for cache cell with keys (n, s)
+ * and create new cell if it does not exist
+ */
+static ZipfCell *
+zipfFindOrCreateCacheCell(ZipfCache * cache, int64 n, double s)
+{
+	int			i,
+				least_recently_used = 0;
+	ZipfCell   *cell;
+
+	/* search cached cell for given parameters */
+	for (i = 0; i < cache->nb_cells; i++)
+	{
+		cell = &cache->cells[i];
+		if (cell->n == n && cell->s == s)
+			return &cache->cells[i];
+
+		if (cell->last_used < cache->cells[least_recently_used].last_used)
+			least_recently_used = i;
+	}
+
+	/* create new one if it does not exist */
+	if (cache->nb_cells < ZIPF_CACHE_SIZE)
+		i = cache->nb_cells++;
+	else
+	{
+		/* replace LRU cell if cache is full */
+		i = least_recently_used;
+		cache->overflowCount++;
+	}
+
+	zipfSetCacheCell(&cache->cells[i], n, s);
+
+	cache->cells[i].last_used = cache->current++;
+	return &cache->cells[i];
+}
+/*
+ * Computing zipfian using rejection method, based on
+ * "Non-Uniform Random Variate Generation",
+ * Luc Devroye, p. 550-551, Springer 1986.
+ */
+static int64
+computeIterativeZipfian(TState *thread, int64 n, double s)
+{
+	double		b = pow(2.0, s - 1.0);
+	double		x,
+				t,
+				u,
+				v;
+
+	while (true)
+	{
+		/* random variates */
+		u = pg_erand48(thread->random_state);
+		v = pg_erand48(thread->random_state);
+
+		x = floor(pow(u, -1.0 / (s - 1.0)));
+
+		t = pow(1.0 + 1.0 / x, s - 1.0);
+		/* reject if too large or out of bound */
+		if (v * x * (t - 1.0) / (b - 1.0) <= t / b && x <= n)
+			break;
+	}
+	return (int64) x;
+}
+
+/*
+ * Computing zipfian using harmonic numbers, based on algorithm described in
+ * "Quickly Generating Billion-Record Synthetic Databases",
+ * Jim Gray et al, SIGMOD 1994
+ */
+static int64
+computeHarmonicZipfian(TState *thread, int64 n, double s)
+{
+	ZipfCell   *cell = zipfFindOrCreateCacheCell(&thread->zipf_cache, n, s);
+	double		uniform = pg_erand48(thread->random_state);
+	double		uz = uniform * cell->harmonicn;
+
+	if (uz < 1.0)
+		return 1;
+	if (uz < 1.0 + cell->beta)
+		return 2;
+	return 1 + (int64) (cell->n * pow(cell->eta * uniform - cell->eta + 1.0, cell->alpha));
+}
+
+/* random number generator: zipfian distribution from min to max inclusive */
+static int64
+getZipfianRand(TState *thread, int64 min, int64 max, double s)
+{
+	int64		n = max - min + 1;
+
+	/* abort if parameter is invalid */
+	Assert(s > 0.0 && s != 1.0 && s <= MAX_ZIPFIAN_PARAM);
+
+
+	return min - 1 + ((s > 1)
+					  ? computeIterativeZipfian(thread, n, s)
+					  : computeHarmonicZipfian(thread, n, s));
+}
+
 /*
  * Initialize the given SimpleStats struct to all zeroes
  */
@@ -1303,7 +1467,6 @@ coerceToDouble(PgBenchValue *pval, double *dval)
 		return true;
 	}
 }
-
 /* assign an integer value */
 static void
 setIntValue(PgBenchValue *pv, int64 ival)
@@ -1605,6 +1768,7 @@ evalFunc(TState *thread, CState *st,
 		case PGBENCH_RANDOM:
 		case PGBENCH_RANDOM_EXPONENTIAL:
 		case PGBENCH_RANDOM_GAUSSIAN:
+		case PGBENCH_RANDOM_ZIPFIAN:
 			{
 				int64		imin,
 							imax;
@@ -1655,6 +1819,18 @@ evalFunc(TState *thread, CState *st,
 						setIntValue(retval,
 									getGaussianRand(thread, imin, imax, param));
 					}
+					else if (func == PGBENCH_RANDOM_ZIPFIAN)
+					{
+						if (param <= 0.0 || param == 1.0 || param > MAX_ZIPFIAN_PARAM)
+						{
+							fprintf(stderr,
+									"zipfian parameter must be in range (0, 1) U (1, %d]"
+									" (got %f)\n", MAX_ZIPFIAN_PARAM, param);
+							return false;
+						}
+						setIntValue(retval,
+									getZipfianRand(thread, imin, imax, param));
+					}
 					else		/* exponential */
 					{
 						if (param <= 0.0)
@@ -3680,6 +3856,8 @@ printResults(TState *threads, StatsData *total, instr_time total_time,
 	double		time_include,
 				tps_include,
 				tps_exclude;
+	int			i,
+				totalCacheOverflows = 0;
 
 	time_include = INSTR_TIME_GET_DOUBLE(total_time);
 	tps_include = total->cnt / time_include;
@@ -3705,6 +3883,15 @@ printResults(TState *threads, StatsData *total, instr_time total_time,
 		printf("number of transactions actually processed: " INT64_FORMAT "\n",
 			   total->cnt);
 	}
+	/* Report zipfian cache overflow */
+	for (i = 0; i < nthreads; i++)
+	{
+		totalCacheOverflows += threads[i].zipf_cache.overflowCount;
+	}
+	if (totalCacheOverflows > 0)
+	{
+		printf("zipfian cache array overflowed %d time(s)\n", totalCacheOverflows);
+	}
 
 	/* Remaining stats are nonsensical if we failed to execute any xacts */
 	if (total->cnt <= 0)
@@ -4501,6 +4688,9 @@ main(int argc, char **argv)
 		thread->random_state[2] = random();
 		thread->logfile = NULL; /* filled in later */
 		thread->latency_late = 0;
+		thread->zipf_cache.nb_cells = 0;
+		thread->zipf_cache.current = 0;
+		thread->zipf_cache.overflowCount = 0;
 		initStats(&thread->stats, 0);
 
 		nclients_dealt += thread->nstate;
diff --git a/src/bin/pgbench/pgbench.h b/src/bin/pgbench/pgbench.h
index fd428af274..83fee1ae74 100644
--- a/src/bin/pgbench/pgbench.h
+++ b/src/bin/pgbench/pgbench.h
@@ -75,7 +75,8 @@ typedef enum PgBenchFunction
 	PGBENCH_SQRT,
 	PGBENCH_RANDOM,
 	PGBENCH_RANDOM_GAUSSIAN,
-	PGBENCH_RANDOM_EXPONENTIAL
+	PGBENCH_RANDOM_EXPONENTIAL,
+	PGBENCH_RANDOM_ZIPFIAN
 } PgBenchFunction;
 
 typedef struct PgBenchExpr PgBenchExpr;
diff --git a/src/bin/pgbench/t/001_pgbench_with_server.pl b/src/bin/pgbench/t/001_pgbench_with_server.pl
index 864d580c64..908da9fb60 100644
--- a/src/bin/pgbench/t/001_pgbench_with_server.pl
+++ b/src/bin/pgbench/t/001_pgbench_with_server.pl
@@ -231,7 +231,8 @@ pgbench(
 		qr{command=18.: double 18\b},
 		qr{command=19.: double 19\b},
 		qr{command=20.: double 20\b},
-		qr{command=21.: int 9223372036854775807\b}, ],
+		qr{command=21.: int 9223372036854775807\b},
+		qr{command=23.: int (1|2|3|4|5|6|7|8|9|10)$}, ],
 	'pgbench expressions',
 	{   '001_pgbench_expressions' => q{-- integer functions
 \set i1 debug(random(1, 100))
@@ -261,6 +262,8 @@ pgbench(
 \set maxint debug(:minint - 1)
 -- reset a variable
 \set i1 0
+-- yet another integer function
+\set id debug(random_zipfian(1, 10, 1.3))
 } });
 
 # backslash commands
@@ -371,6 +374,14 @@ SELECT LEAST(:i, :i, :i, :i, :i, :i, :i, :i, :i, :i, :i);
 		0,
 		[qr{exponential parameter must be greater }],
 		q{\set i random_exponential(0, 10, 0.0)} ],
+	[	'set zipfian param to 1',
+		0,
+		[qr{zipfian parameter must be in range \(0, 1\) U \(1, \d+\]}],
+		q{\set i random_zipfian(0, 10, 1)} ],
+	[	'set zipfian param too large',
+		0,
+		[qr{zipfian parameter must be in range \(0, 1\) U \(1, \d+\]}],
+		q{\set i random_zipfian(0, 10, 1000000)} ],
 	[   'set non numeric value',                     0,
 		[qr{malformed variable "foo" value: "bla"}], q{\set i :foo + 1} ],
 	[ 'set no expression',    1, [qr{syntax error}],      q{\set i} ],
@@ -412,6 +423,31 @@ for my $e (@errors)
 		{ $n => $script });
 }
 
+# zipfian cache array overflow
+pgbench(
+	'-t 1', 0,
+	[ qr{processed: 1/1}, qr{zipfian cache array overflowed 1 time\(s\)} ],
+	[ qr{^} ],
+	'pgbench zipfian array overflow on random_zipfian',
+	{   '001_pgbench_random_zipfian' => q{
+\set i random_zipfian(1, 100, 0.5)
+\set i random_zipfian(2, 100, 0.5)
+\set i random_zipfian(3, 100, 0.5)
+\set i random_zipfian(4, 100, 0.5)
+\set i random_zipfian(5, 100, 0.5)
+\set i random_zipfian(6, 100, 0.5)
+\set i random_zipfian(7, 100, 0.5)
+\set i random_zipfian(8, 100, 0.5)
+\set i random_zipfian(9, 100, 0.5)
+\set i random_zipfian(10, 100, 0.5)
+\set i random_zipfian(11, 100, 0.5)
+\set i random_zipfian(12, 100, 0.5)
+\set i random_zipfian(13, 100, 0.5)
+\set i random_zipfian(14, 100, 0.5)
+\set i random_zipfian(15, 100, 0.5)
+\set i random_zipfian(16, 100, 0.5)
+} });
+
 # throttling
 pgbench(
 	'-t 100 -S --rate=100000 --latency-limit=1000000 -c 2 -n -r',
-- 
2.14.1

#51Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Alik Khilazhev (#50)
Re: [HACKERS] [WIP] Zipfian distribution in pgbench

Hello Alik,

Patch applies, compiles, works (I checked the distribution). Doc gen ok.
make check ok.

I have reduced range to [1, 10] and updated the test.

I would suggest to use [1, 9] and the simpler regex [1-9] instead of the
full list.

Also, there should be an empty line between functions. I just noticed that
one is missing after zipfFindOrCreateCacheCell().

--
Fabien.

#52Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Fabien COELHO (#51)
1 attachment(s)
Re: [HACKERS] [WIP] Zipfian distribution in pgbench

I would suggest to use [1, 9] and the simpler regex [1-9] instead of the full list.

Also, there should be an empty line between functions. I just noticed that one is missing after zipfFindOrCreateCacheCell().

Fixed.

Attachments:

pgbench-zipf-11v.patchapplication/octet-stream; name=pgbench-zipf-11v.patch; x-unix-mode=0644Download
diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml
index c48a69713a..d20f4aa498 100644
--- a/doc/src/sgml/ref/pgbench.sgml
+++ b/doc/src/sgml/ref/pgbench.sgml
@@ -1092,6 +1092,14 @@ pgbench <optional> <replaceable>options</replaceable> </optional> <replaceable>d
        <entry><literal>random_gaussian(1, 10, 2.5)</literal></entry>
        <entry>an integer between <literal>1</literal> and <literal>10</literal></entry>
       </row>
+      <row>
+       <entry><literal><function>random_zipfian(<replaceable>lb</replaceable>, <replaceable>ub</replaceable>, <replaceable>parameter</replaceable>)</function></literal></entry>
+       <entry>integer</entry>
+       <entry>Zipfian-distributed random integer in <literal>[lb, ub]</literal>,
+              see below</entry>
+       <entry><literal>random_zipfian(1, 10, 1.5)</literal></entry>
+       <entry>an integer between <literal>1</literal> and <literal>10</literal></entry>
+      </row>
       <row>
        <entry><literal><function>sqrt(<replaceable>x</replaceable>)</function></literal></entry>
        <entry>double</entry>
@@ -1173,6 +1181,27 @@ f(x) = PHI(2.0 * parameter * (x - mu) / (max - min + 1)) /
       of the Box-Muller transform.
      </para>
     </listitem>
+    <listitem>
+     <para>
+      <literal>random_zipfian</literal> generates an approximated bounded zipfian
+      distribution. For <replaceable>parameter</replaceable> in (0, 1), an
+      approximated algorithm is taken from
+      "Quickly Generating Billion-Record Synthetic Databases",
+      Jim Gray et al, SIGMOD 1994. For <replaceable>parameter</replaceable>
+      in (1, 1000), a rejection method is used, based on
+      "Non-Uniform Random Variate Generation", Luc Devroye, p. 550-551,
+      Springer 1986. The distribution is not defined when the parameter's
+      value is 1.0. The drawing performance is poor for parameter values
+      close and above 1.0 and on a small range.
+     </para>
+     <para>
+      <replaceable>parameter</replaceable>
+      defines how skewed the distribution is. The larger the <replaceable>parameter</replaceable>, the more
+      frequently values to the beginning of the interval are drawn.
+      The closer to 0 <replaceable>parameter</replaceable> is,
+      the flatter (more uniform) the access distribution.
+     </para>
+    </listitem>
    </itemizedlist>
 
   <para>
diff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.y
index b3a2d9bfd3..25d5ad48e5 100644
--- a/src/bin/pgbench/exprparse.y
+++ b/src/bin/pgbench/exprparse.y
@@ -191,6 +191,9 @@ static const struct
 	{
 		"random_exponential", 3, PGBENCH_RANDOM_EXPONENTIAL
 	},
+	{
+		"random_zipfian", 3, PGBENCH_RANDOM_ZIPFIAN
+	},
 	/* keep as last array element */
 	{
 		NULL, 0, 0
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index ceb2fcc3ad..41a38281ae 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -95,7 +95,10 @@ static int	pthread_join(pthread_t th, void **thread_return);
 #define LOG_STEP_SECONDS	5	/* seconds between log messages */
 #define DEFAULT_NXACTS	10		/* default nxacts */
 
+#define ZIPF_CACHE_SIZE	15		/* cache cells number */
+
 #define MIN_GAUSSIAN_PARAM		2.0 /* minimum parameter for gauss */
+#define MAX_ZIPFIAN_PARAM		1000	/* maximum parameter for zipfian */
 
 int			nxacts = 0;			/* number of transactions per client */
 int			duration = 0;		/* duration in seconds */
@@ -330,6 +333,35 @@ typedef struct
 	int			ecnt;			/* error count */
 } CState;
 
+/*
+ * Cache cell for zipfian_random call
+ */
+typedef struct
+{
+	/* cell keys */
+	double		s;				/* s - parameter of zipfan_random function */
+	int64		n;				/* number of elements in range (max - min + 1) */
+
+	double		harmonicn;		/* generalizedHarmonicNumber(n, s) */
+	double		alpha;
+	double		beta;
+	double		eta;
+
+	uint64		last_used;		/* last used logical time */
+} ZipfCell;
+
+/*
+ * Zipf cache for zeta values
+ */
+typedef struct
+{
+	uint64		current;		/* counter for LRU cache replacement algorithm */
+
+	int			nb_cells;		/* number of filled cells */
+	int			overflowCount;	/* number of cache overflows */
+	ZipfCell	cells[ZIPF_CACHE_SIZE];
+} ZipfCache;
+
 /*
  * Thread state
  */
@@ -342,6 +374,8 @@ typedef struct
 	unsigned short random_state[3]; /* separate randomness for each thread */
 	int64		throttle_trigger;	/* previous/next throttling (us) */
 	FILE	   *logfile;		/* where to log, or NULL */
+	ZipfCache	zipf_cache;		/* for thread-safe  zipfian random number
+								 * generation */
 
 	/* per thread collected stats */
 	instr_time	start_time;		/* thread start time */
@@ -746,6 +780,137 @@ getPoissonRand(TState *thread, int64 center)
 	return (int64) (-log(uniform) * ((double) center) + 0.5);
 }
 
+/* helper function for getZipfianRand */
+static double
+generalizedHarmonicNumber(int64 n, double s)
+{
+	int			i;
+	double		ans = 0.0;
+
+	for (i = n; i > 1; i--)
+		ans += pow(i, -s);
+	return ans + 1.0;
+}
+
+/* set harmonicn and other parameters to cache cell */
+static void
+zipfSetCacheCell(ZipfCell * cell, int64 n, double s)
+{
+	double		harmonic2;
+
+	cell->n = n;
+	cell->s = s;
+
+	harmonic2 = generalizedHarmonicNumber(2, s);
+	cell->harmonicn = generalizedHarmonicNumber(n, s);
+
+	cell->alpha = 1.0 / (1.0 - s);
+	cell->beta = pow(0.5, s);
+	cell->eta = (1.0 - pow(2.0 / n, 1.0 - s)) / (1.0 - harmonic2 / cell->harmonicn);
+}
+
+/*
+ * search for cache cell with keys (n, s)
+ * and create new cell if it does not exist
+ */
+static ZipfCell *
+zipfFindOrCreateCacheCell(ZipfCache * cache, int64 n, double s)
+{
+	int			i,
+				least_recently_used = 0;
+	ZipfCell   *cell;
+
+	/* search cached cell for given parameters */
+	for (i = 0; i < cache->nb_cells; i++)
+	{
+		cell = &cache->cells[i];
+		if (cell->n == n && cell->s == s)
+			return &cache->cells[i];
+
+		if (cell->last_used < cache->cells[least_recently_used].last_used)
+			least_recently_used = i;
+	}
+
+	/* create new one if it does not exist */
+	if (cache->nb_cells < ZIPF_CACHE_SIZE)
+		i = cache->nb_cells++;
+	else
+	{
+		/* replace LRU cell if cache is full */
+		i = least_recently_used;
+		cache->overflowCount++;
+	}
+
+	zipfSetCacheCell(&cache->cells[i], n, s);
+
+	cache->cells[i].last_used = cache->current++;
+	return &cache->cells[i];
+}
+
+/*
+ * Computing zipfian using rejection method, based on
+ * "Non-Uniform Random Variate Generation",
+ * Luc Devroye, p. 550-551, Springer 1986.
+ */
+static int64
+computeIterativeZipfian(TState *thread, int64 n, double s)
+{
+	double		b = pow(2.0, s - 1.0);
+	double		x,
+				t,
+				u,
+				v;
+
+	while (true)
+	{
+		/* random variates */
+		u = pg_erand48(thread->random_state);
+		v = pg_erand48(thread->random_state);
+
+		x = floor(pow(u, -1.0 / (s - 1.0)));
+
+		t = pow(1.0 + 1.0 / x, s - 1.0);
+		/* reject if too large or out of bound */
+		if (v * x * (t - 1.0) / (b - 1.0) <= t / b && x <= n)
+			break;
+	}
+	return (int64) x;
+}
+
+/*
+ * Computing zipfian using harmonic numbers, based on algorithm described in
+ * "Quickly Generating Billion-Record Synthetic Databases",
+ * Jim Gray et al, SIGMOD 1994
+ */
+static int64
+computeHarmonicZipfian(TState *thread, int64 n, double s)
+{
+	ZipfCell   *cell = zipfFindOrCreateCacheCell(&thread->zipf_cache, n, s);
+	double		uniform = pg_erand48(thread->random_state);
+	double		uz = uniform * cell->harmonicn;
+
+	if (uz < 1.0)
+		return 1;
+	if (uz < 1.0 + cell->beta)
+		return 2;
+	return 1 + (int64) (cell->n * pow(cell->eta * uniform - cell->eta + 1.0, cell->alpha));
+}
+
+/* random number generator: zipfian distribution from min to max inclusive */
+static int64
+getZipfianRand(TState *thread, int64 min, int64 max, double s)
+{
+	int64		n = max - min + 1;
+
+	/* abort if parameter is invalid */
+	Assert(s > 0.0 && s != 1.0 && s <= MAX_ZIPFIAN_PARAM);
+
+
+	return min - 1 + ((s > 1)
+					  ? computeIterativeZipfian(thread, n, s)
+					  : computeHarmonicZipfian(thread, n, s));
+}
+
 /*
  * Initialize the given SimpleStats struct to all zeroes
  */
@@ -1303,7 +1468,6 @@ coerceToDouble(PgBenchValue *pval, double *dval)
 		return true;
 	}
 }
-
 /* assign an integer value */
 static void
 setIntValue(PgBenchValue *pv, int64 ival)
@@ -1605,6 +1769,7 @@ evalFunc(TState *thread, CState *st,
 		case PGBENCH_RANDOM:
 		case PGBENCH_RANDOM_EXPONENTIAL:
 		case PGBENCH_RANDOM_GAUSSIAN:
+		case PGBENCH_RANDOM_ZIPFIAN:
 			{
 				int64		imin,
 							imax;
@@ -1655,6 +1820,18 @@ evalFunc(TState *thread, CState *st,
 						setIntValue(retval,
 									getGaussianRand(thread, imin, imax, param));
 					}
+					else if (func == PGBENCH_RANDOM_ZIPFIAN)
+					{
+						if (param <= 0.0 || param == 1.0 || param > MAX_ZIPFIAN_PARAM)
+						{
+							fprintf(stderr,
+									"zipfian parameter must be in range (0, 1) U (1, %d]"
+									" (got %f)\n", MAX_ZIPFIAN_PARAM, param);
+							return false;
+						}
+						setIntValue(retval,
+									getZipfianRand(thread, imin, imax, param));
+					}
 					else		/* exponential */
 					{
 						if (param <= 0.0)
@@ -3680,6 +3857,8 @@ printResults(TState *threads, StatsData *total, instr_time total_time,
 	double		time_include,
 				tps_include,
 				tps_exclude;
+	int			i,
+				totalCacheOverflows = 0;
 
 	time_include = INSTR_TIME_GET_DOUBLE(total_time);
 	tps_include = total->cnt / time_include;
@@ -3705,6 +3884,15 @@ printResults(TState *threads, StatsData *total, instr_time total_time,
 		printf("number of transactions actually processed: " INT64_FORMAT "\n",
 			   total->cnt);
 	}
+	/* Report zipfian cache overflow */
+	for (i = 0; i < nthreads; i++)
+	{
+		totalCacheOverflows += threads[i].zipf_cache.overflowCount;
+	}
+	if (totalCacheOverflows > 0)
+	{
+		printf("zipfian cache array overflowed %d time(s)\n", totalCacheOverflows);
+	}
 
 	/* Remaining stats are nonsensical if we failed to execute any xacts */
 	if (total->cnt <= 0)
@@ -4501,6 +4689,9 @@ main(int argc, char **argv)
 		thread->random_state[2] = random();
 		thread->logfile = NULL; /* filled in later */
 		thread->latency_late = 0;
+		thread->zipf_cache.nb_cells = 0;
+		thread->zipf_cache.current = 0;
+		thread->zipf_cache.overflowCount = 0;
 		initStats(&thread->stats, 0);
 
 		nclients_dealt += thread->nstate;
diff --git a/src/bin/pgbench/pgbench.h b/src/bin/pgbench/pgbench.h
index fd428af274..83fee1ae74 100644
--- a/src/bin/pgbench/pgbench.h
+++ b/src/bin/pgbench/pgbench.h
@@ -75,7 +75,8 @@ typedef enum PgBenchFunction
 	PGBENCH_SQRT,
 	PGBENCH_RANDOM,
 	PGBENCH_RANDOM_GAUSSIAN,
-	PGBENCH_RANDOM_EXPONENTIAL
+	PGBENCH_RANDOM_EXPONENTIAL,
+	PGBENCH_RANDOM_ZIPFIAN
 } PgBenchFunction;
 
 typedef struct PgBenchExpr PgBenchExpr;
diff --git a/src/bin/pgbench/t/001_pgbench_with_server.pl b/src/bin/pgbench/t/001_pgbench_with_server.pl
index 864d580c64..4e74dd26d4 100644
--- a/src/bin/pgbench/t/001_pgbench_with_server.pl
+++ b/src/bin/pgbench/t/001_pgbench_with_server.pl
@@ -231,7 +231,8 @@ pgbench(
 		qr{command=18.: double 18\b},
 		qr{command=19.: double 19\b},
 		qr{command=20.: double 20\b},
-		qr{command=21.: int 9223372036854775807\b}, ],
+		qr{command=21.: int 9223372036854775807\b},
+		qr{command=23.: int [1-9]\b}, ],
 	'pgbench expressions',
 	{   '001_pgbench_expressions' => q{-- integer functions
 \set i1 debug(random(1, 100))
@@ -261,6 +262,8 @@ pgbench(
 \set maxint debug(:minint - 1)
 -- reset a variable
 \set i1 0
+-- yet another integer function
+\set id debug(random_zipfian(1, 9, 1.3))
 } });
 
 # backslash commands
@@ -371,6 +374,14 @@ SELECT LEAST(:i, :i, :i, :i, :i, :i, :i, :i, :i, :i, :i);
 		0,
 		[qr{exponential parameter must be greater }],
 		q{\set i random_exponential(0, 10, 0.0)} ],
+	[	'set zipfian param to 1',
+		0,
+		[qr{zipfian parameter must be in range \(0, 1\) U \(1, \d+\]}],
+		q{\set i random_zipfian(0, 10, 1)} ],
+	[	'set zipfian param too large',
+		0,
+		[qr{zipfian parameter must be in range \(0, 1\) U \(1, \d+\]}],
+		q{\set i random_zipfian(0, 10, 1000000)} ],
 	[   'set non numeric value',                     0,
 		[qr{malformed variable "foo" value: "bla"}], q{\set i :foo + 1} ],
 	[ 'set no expression',    1, [qr{syntax error}],      q{\set i} ],
@@ -412,6 +423,31 @@ for my $e (@errors)
 		{ $n => $script });
 }
 
+# zipfian cache array overflow
+pgbench(
+	'-t 1', 0,
+	[ qr{processed: 1/1}, qr{zipfian cache array overflowed 1 time\(s\)} ],
+	[ qr{^} ],
+	'pgbench zipfian array overflow on random_zipfian',
+	{   '001_pgbench_random_zipfian' => q{
+\set i random_zipfian(1, 100, 0.5)
+\set i random_zipfian(2, 100, 0.5)
+\set i random_zipfian(3, 100, 0.5)
+\set i random_zipfian(4, 100, 0.5)
+\set i random_zipfian(5, 100, 0.5)
+\set i random_zipfian(6, 100, 0.5)
+\set i random_zipfian(7, 100, 0.5)
+\set i random_zipfian(8, 100, 0.5)
+\set i random_zipfian(9, 100, 0.5)
+\set i random_zipfian(10, 100, 0.5)
+\set i random_zipfian(11, 100, 0.5)
+\set i random_zipfian(12, 100, 0.5)
+\set i random_zipfian(13, 100, 0.5)
+\set i random_zipfian(14, 100, 0.5)
+\set i random_zipfian(15, 100, 0.5)
+\set i random_zipfian(16, 100, 0.5)
+} });
+
 # throttling
 pgbench(
 	'-t 100 -S --rate=100000 --latency-limit=1000000 -c 2 -n -r',
-- 
2.14.1

#53Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Alik Khilazhev (#52)
Re: [HACKERS] [WIP] Zipfian distribution in pgbench

Fixed.

Applies, compiles, make check ok, doc ok.

Note that the patch may interact with other patches which add functions to
pgbench, so might need a rebase depending on the order in which the patch
are applied. We'll see.

I have added the thread to the next CF (January 2018), and marked it as
"ready for committer".

--
Fabien.

#54Alik Khilazhev
a.khilazhev@postgrespro.ru
In reply to: Fabien COELHO (#53)
Re: [HACKERS] [WIP] Zipfian distribution in pgbench

I have added the thread to the next CF (January 2018), and marked it as "ready for committer”.

That’s cool, thank you for review!

Thanks and Regards,
Alik Khilazhev
Postgres Professional:
http://www.postgrespro.com
The Russian Postgres Company

#55Fabien COELHO
coelho@cri.ensmp.fr
In reply to: Fabien COELHO (#53)
1 attachment(s)
Re: [HACKERS] [WIP] Zipfian distribution in pgbench

Note that the patch may interact with other patches which add functions to
pgbench, so might need a rebase depending on the order in which the patch are
applied.

Attached a minor rebase after 16827d4424.

--
Fabien.

Attachments:

pgbench-zipf-11-1.patchtext/x-diff; name=pgbench-zipf-11-1.patchDownload
diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml
index f6e93c3..f119ae6 100644
--- a/doc/src/sgml/ref/pgbench.sgml
+++ b/doc/src/sgml/ref/pgbench.sgml
@@ -1093,6 +1093,14 @@ pgbench <optional> <replaceable>options</replaceable> </optional> <replaceable>d
        <entry>an integer between <literal>1</literal> and <literal>10</literal></entry>
       </row>
       <row>
+       <entry><literal><function>random_zipfian(<replaceable>lb</replaceable>, <replaceable>ub</replaceable>, <replaceable>parameter</replaceable>)</function></literal></entry>
+       <entry>integer</entry>
+       <entry>Zipfian-distributed random integer in <literal>[lb, ub]</literal>,
+              see below</entry>
+       <entry><literal>random_zipfian(1, 10, 1.5)</literal></entry>
+       <entry>an integer between <literal>1</literal> and <literal>10</literal></entry>
+      </row>
+      <row>
        <entry><literal><function>sqrt(<replaceable>x</replaceable>)</function></literal></entry>
        <entry>double</entry>
        <entry>square root</entry>
@@ -1173,6 +1181,27 @@ f(x) = PHI(2.0 * parameter * (x - mu) / (max - min + 1)) /
       of the Box-Muller transform.
      </para>
     </listitem>
+    <listitem>
+     <para>
+      <literal>random_zipfian</literal> generates an approximated bounded zipfian
+      distribution. For <replaceable>parameter</replaceable> in (0, 1), an
+      approximated algorithm is taken from
+      "Quickly Generating Billion-Record Synthetic Databases",
+      Jim Gray et al, SIGMOD 1994. For <replaceable>parameter</replaceable>
+      in (1, 1000), a rejection method is used, based on
+      "Non-Uniform Random Variate Generation", Luc Devroye, p. 550-551,
+      Springer 1986. The distribution is not defined when the parameter's
+      value is 1.0. The drawing performance is poor for parameter values
+      close and above 1.0 and on a small range.
+     </para>
+     <para>
+      <replaceable>parameter</replaceable>
+      defines how skewed the distribution is. The larger the <replaceable>parameter</replaceable>, the more
+      frequently values to the beginning of the interval are drawn.
+      The closer to 0 <replaceable>parameter</replaceable> is,
+      the flatter (more uniform) the access distribution.
+     </para>
+    </listitem>
    </itemizedlist>
 
   <para>
diff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.y
index b3a2d9b..25d5ad4 100644
--- a/src/bin/pgbench/exprparse.y
+++ b/src/bin/pgbench/exprparse.y
@@ -191,6 +191,9 @@ static const struct
 	{
 		"random_exponential", 3, PGBENCH_RANDOM_EXPONENTIAL
 	},
+	{
+		"random_zipfian", 3, PGBENCH_RANDOM_ZIPFIAN
+	},
 	/* keep as last array element */
 	{
 		NULL, 0, 0
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index bd96eae..7ce6f60 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -95,7 +95,10 @@ static int	pthread_join(pthread_t th, void **thread_return);
 #define LOG_STEP_SECONDS	5	/* seconds between log messages */
 #define DEFAULT_NXACTS	10		/* default nxacts */
 
+#define ZIPF_CACHE_SIZE	15		/* cache cells number */
+
 #define MIN_GAUSSIAN_PARAM		2.0 /* minimum parameter for gauss */
+#define MAX_ZIPFIAN_PARAM		1000	/* maximum parameter for zipfian */
 
 int			nxacts = 0;			/* number of transactions per client */
 int			duration = 0;		/* duration in seconds */
@@ -331,6 +334,35 @@ typedef struct
 } CState;
 
 /*
+ * Cache cell for zipfian_random call
+ */
+typedef struct
+{
+	/* cell keys */
+	double		s;				/* s - parameter of zipfan_random function */
+	int64		n;				/* number of elements in range (max - min + 1) */
+
+	double		harmonicn;		/* generalizedHarmonicNumber(n, s) */
+	double		alpha;
+	double		beta;
+	double		eta;
+
+	uint64		last_used;		/* last used logical time */
+} ZipfCell;
+
+/*
+ * Zipf cache for zeta values
+ */
+typedef struct
+{
+	uint64		current;		/* counter for LRU cache replacement algorithm */
+
+	int			nb_cells;		/* number of filled cells */
+	int			overflowCount;	/* number of cache overflows */
+	ZipfCell	cells[ZIPF_CACHE_SIZE];
+} ZipfCache;
+
+/*
  * Thread state
  */
 typedef struct
@@ -342,6 +374,8 @@ typedef struct
 	unsigned short random_state[3]; /* separate randomness for each thread */
 	int64		throttle_trigger;	/* previous/next throttling (us) */
 	FILE	   *logfile;		/* where to log, or NULL */
+	ZipfCache	zipf_cache;		/* for thread-safe  zipfian random number
+								 * generation */
 
 	/* per thread collected stats */
 	instr_time	start_time;		/* thread start time */
@@ -746,6 +780,137 @@ getPoissonRand(TState *thread, int64 center)
 	return (int64) (-log(uniform) * ((double) center) + 0.5);
 }
 
+/* helper function for getZipfianRand */
+static double
+generalizedHarmonicNumber(int64 n, double s)
+{
+	int			i;
+	double		ans = 0.0;
+
+	for (i = n; i > 1; i--)
+		ans += pow(i, -s);
+	return ans + 1.0;
+}
+
+/* set harmonicn and other parameters to cache cell */
+static void
+zipfSetCacheCell(ZipfCell * cell, int64 n, double s)
+{
+	double		harmonic2;
+
+	cell->n = n;
+	cell->s = s;
+
+	harmonic2 = generalizedHarmonicNumber(2, s);
+	cell->harmonicn = generalizedHarmonicNumber(n, s);
+
+	cell->alpha = 1.0 / (1.0 - s);
+	cell->beta = pow(0.5, s);
+	cell->eta = (1.0 - pow(2.0 / n, 1.0 - s)) / (1.0 - harmonic2 / cell->harmonicn);
+}
+
+/*
+ * search for cache cell with keys (n, s)
+ * and create new cell if it does not exist
+ */
+static ZipfCell *
+zipfFindOrCreateCacheCell(ZipfCache * cache, int64 n, double s)
+{
+	int			i,
+				least_recently_used = 0;
+	ZipfCell   *cell;
+
+	/* search cached cell for given parameters */
+	for (i = 0; i < cache->nb_cells; i++)
+	{
+		cell = &cache->cells[i];
+		if (cell->n == n && cell->s == s)
+			return &cache->cells[i];
+
+		if (cell->last_used < cache->cells[least_recently_used].last_used)
+			least_recently_used = i;
+	}
+
+	/* create new one if it does not exist */
+	if (cache->nb_cells < ZIPF_CACHE_SIZE)
+		i = cache->nb_cells++;
+	else
+	{
+		/* replace LRU cell if cache is full */
+		i = least_recently_used;
+		cache->overflowCount++;
+	}
+
+	zipfSetCacheCell(&cache->cells[i], n, s);
+
+	cache->cells[i].last_used = cache->current++;
+	return &cache->cells[i];
+}
+
+/*
+ * Computing zipfian using rejection method, based on
+ * "Non-Uniform Random Variate Generation",
+ * Luc Devroye, p. 550-551, Springer 1986.
+ */
+static int64
+computeIterativeZipfian(TState *thread, int64 n, double s)
+{
+	double		b = pow(2.0, s - 1.0);
+	double		x,
+				t,
+				u,
+				v;
+
+	while (true)
+	{
+		/* random variates */
+		u = pg_erand48(thread->random_state);
+		v = pg_erand48(thread->random_state);
+
+		x = floor(pow(u, -1.0 / (s - 1.0)));
+
+		t = pow(1.0 + 1.0 / x, s - 1.0);
+		/* reject if too large or out of bound */
+		if (v * x * (t - 1.0) / (b - 1.0) <= t / b && x <= n)
+			break;
+	}
+	return (int64) x;
+}
+
+/*
+ * Computing zipfian using harmonic numbers, based on algorithm described in
+ * "Quickly Generating Billion-Record Synthetic Databases",
+ * Jim Gray et al, SIGMOD 1994
+ */
+static int64
+computeHarmonicZipfian(TState *thread, int64 n, double s)
+{
+	ZipfCell   *cell = zipfFindOrCreateCacheCell(&thread->zipf_cache, n, s);
+	double		uniform = pg_erand48(thread->random_state);
+	double		uz = uniform * cell->harmonicn;
+
+	if (uz < 1.0)
+		return 1;
+	if (uz < 1.0 + cell->beta)
+		return 2;
+	return 1 + (int64) (cell->n * pow(cell->eta * uniform - cell->eta + 1.0, cell->alpha));
+}
+
+/* random number generator: zipfian distribution from min to max inclusive */
+static int64
+getZipfianRand(TState *thread, int64 min, int64 max, double s)
+{
+	int64		n = max - min + 1;
+
+	/* abort if parameter is invalid */
+	Assert(s > 0.0 && s != 1.0 && s <= MAX_ZIPFIAN_PARAM);
+
+
+	return min - 1 + ((s > 1)
+					  ? computeIterativeZipfian(thread, n, s)
+					  : computeHarmonicZipfian(thread, n, s));
+}
+
 /*
  * Initialize the given SimpleStats struct to all zeroes
  */
@@ -1303,7 +1468,6 @@ coerceToDouble(PgBenchValue *pval, double *dval)
 		return true;
 	}
 }
-
 /* assign an integer value */
 static void
 setIntValue(PgBenchValue *pv, int64 ival)
@@ -1605,6 +1769,7 @@ evalFunc(TState *thread, CState *st,
 		case PGBENCH_RANDOM:
 		case PGBENCH_RANDOM_EXPONENTIAL:
 		case PGBENCH_RANDOM_GAUSSIAN:
+		case PGBENCH_RANDOM_ZIPFIAN:
 			{
 				int64		imin,
 							imax;
@@ -1655,6 +1820,18 @@ evalFunc(TState *thread, CState *st,
 						setIntValue(retval,
 									getGaussianRand(thread, imin, imax, param));
 					}
+					else if (func == PGBENCH_RANDOM_ZIPFIAN)
+					{
+						if (param <= 0.0 || param == 1.0 || param > MAX_ZIPFIAN_PARAM)
+						{
+							fprintf(stderr,
+									"zipfian parameter must be in range (0, 1) U (1, %d]"
+									" (got %f)\n", MAX_ZIPFIAN_PARAM, param);
+							return false;
+						}
+						setIntValue(retval,
+									getZipfianRand(thread, imin, imax, param));
+					}
 					else		/* exponential */
 					{
 						if (param <= 0.0)
@@ -3683,6 +3860,8 @@ printResults(TState *threads, StatsData *total, instr_time total_time,
 				tps_include,
 				tps_exclude;
 	int64		ntx = total->cnt - total->skipped;
+	int			i,
+				totalCacheOverflows = 0;
 
 	time_include = INSTR_TIME_GET_DOUBLE(total_time);
 
@@ -3710,6 +3889,15 @@ printResults(TState *threads, StatsData *total, instr_time total_time,
 		printf("number of transactions actually processed: " INT64_FORMAT "\n",
 			   ntx);
 	}
+	/* Report zipfian cache overflow */
+	for (i = 0; i < nthreads; i++)
+	{
+		totalCacheOverflows += threads[i].zipf_cache.overflowCount;
+	}
+	if (totalCacheOverflows > 0)
+	{
+		printf("zipfian cache array overflowed %d time(s)\n", totalCacheOverflows);
+	}
 
 	/* Remaining stats are nonsensical if we failed to execute any xacts */
 	if (total->cnt <= 0)
@@ -4513,6 +4701,9 @@ main(int argc, char **argv)
 		thread->random_state[2] = random();
 		thread->logfile = NULL; /* filled in later */
 		thread->latency_late = 0;
+		thread->zipf_cache.nb_cells = 0;
+		thread->zipf_cache.current = 0;
+		thread->zipf_cache.overflowCount = 0;
 		initStats(&thread->stats, 0);
 
 		nclients_dealt += thread->nstate;
diff --git a/src/bin/pgbench/pgbench.h b/src/bin/pgbench/pgbench.h
index fd428af..83fee1a 100644
--- a/src/bin/pgbench/pgbench.h
+++ b/src/bin/pgbench/pgbench.h
@@ -75,7 +75,8 @@ typedef enum PgBenchFunction
 	PGBENCH_SQRT,
 	PGBENCH_RANDOM,
 	PGBENCH_RANDOM_GAUSSIAN,
-	PGBENCH_RANDOM_EXPONENTIAL
+	PGBENCH_RANDOM_EXPONENTIAL,
+	PGBENCH_RANDOM_ZIPFIAN
 } PgBenchFunction;
 
 typedef struct PgBenchExpr PgBenchExpr;
diff --git a/src/bin/pgbench/t/001_pgbench_with_server.pl b/src/bin/pgbench/t/001_pgbench_with_server.pl
index c095881..e3cdf28 100644
--- a/src/bin/pgbench/t/001_pgbench_with_server.pl
+++ b/src/bin/pgbench/t/001_pgbench_with_server.pl
@@ -231,7 +231,8 @@ pgbench(
 		qr{command=18.: double 18\b},
 		qr{command=19.: double 19\b},
 		qr{command=20.: double 20\b},
-		qr{command=21.: int 9223372036854775807\b}, ],
+		qr{command=21.: int 9223372036854775807\b},
+		qr{command=23.: int [1-9]\b}, ],
 	'pgbench expressions',
 	{   '001_pgbench_expressions' => q{-- integer functions
 \set i1 debug(random(1, 100))
@@ -261,6 +262,8 @@ pgbench(
 \set maxint debug(:minint - 1)
 -- reset a variable
 \set i1 0
+-- yet another integer function
+\set id debug(random_zipfian(1, 9, 1.3))
 } });
 
 # backslash commands
@@ -371,6 +374,14 @@ SELECT LEAST(:i, :i, :i, :i, :i, :i, :i, :i, :i, :i, :i);
 		0,
 		[qr{exponential parameter must be greater }],
 		q{\set i random_exponential(0, 10, 0.0)} ],
+	[	'set zipfian param to 1',
+		0,
+		[qr{zipfian parameter must be in range \(0, 1\) U \(1, \d+\]}],
+		q{\set i random_zipfian(0, 10, 1)} ],
+	[	'set zipfian param too large',
+		0,
+		[qr{zipfian parameter must be in range \(0, 1\) U \(1, \d+\]}],
+		q{\set i random_zipfian(0, 10, 1000000)} ],
 	[   'set non numeric value',                     0,
 		[qr{malformed variable "foo" value: "bla"}], q{\set i :foo + 1} ],
 	[ 'set no expression',    1, [qr{syntax error}],      q{\set i} ],
@@ -412,6 +423,31 @@ for my $e (@errors)
 		{ $n => $script });
 }
 
+# zipfian cache array overflow
+pgbench(
+	'-t 1', 0,
+	[ qr{processed: 1/1}, qr{zipfian cache array overflowed 1 time\(s\)} ],
+	[ qr{^} ],
+	'pgbench zipfian array overflow on random_zipfian',
+	{   '001_pgbench_random_zipfian' => q{
+\set i random_zipfian(1, 100, 0.5)
+\set i random_zipfian(2, 100, 0.5)
+\set i random_zipfian(3, 100, 0.5)
+\set i random_zipfian(4, 100, 0.5)
+\set i random_zipfian(5, 100, 0.5)
+\set i random_zipfian(6, 100, 0.5)
+\set i random_zipfian(7, 100, 0.5)
+\set i random_zipfian(8, 100, 0.5)
+\set i random_zipfian(9, 100, 0.5)
+\set i random_zipfian(10, 100, 0.5)
+\set i random_zipfian(11, 100, 0.5)
+\set i random_zipfian(12, 100, 0.5)
+\set i random_zipfian(13, 100, 0.5)
+\set i random_zipfian(14, 100, 0.5)
+\set i random_zipfian(15, 100, 0.5)
+\set i random_zipfian(16, 100, 0.5)
+} });
+
 # throttling
 pgbench(
 	'-t 100 -S --rate=100000 --latency-limit=1000000 -c 2 -n -r',
#56Teodor Sigaev
teodor@sigaev.ru
In reply to: Fabien COELHO (#55)
Re: [HACKERS] [WIP] Zipfian distribution in pgbench

Thank you for all, pushed

Fabien COELHO wrote:

Note that the patch may interact with other patches which add functions to
pgbench, so might need a rebase depending on the order in which the patch are
applied.

Attached a minor rebase after 16827d4424.

--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/

In reply to: Alik Khilazhev (#1)
Re: [HACKERS] [WIP] Zipfian distribution in pgbench

On Fri, Jul 7, 2017 at 12:45 AM Alik Khilazhev
<a.khilazhev@postgrespro.ru> wrote:

PostgreSQL shows very bad results in YCSB Workload A (50% SELECT and 50% UPDATE of random row by PK) on benchmarking with big number of clients using Zipfian distribution. MySQL also has decline but it is not significant as it is in PostgreSQL. MongoDB does not have decline at all. And if pgbench would have Zipfian distribution random number generator, everyone will be able to make research on this topic without using YCSB.

I've been thinking about this again, in light of my recent work to
improve the B-Tree code by making all entries have fully unique keys,
adding suffix truncation, and making better choices about split points
[1]: https://commitfest.postgresql.org/20/1787/ -- Peter Geoghegan

Somebody posted a patch that fixed the issue by making the heavyweight
lock manager lock a value rather than a heap TID within the last year.
I can't find that thread right now -- perhaps someone can point it out
now. I suspect that my patch will have a similar effect to this other
patch I'm thinking of with the Zipfian workload, though it will do so
without touching anything outside of the B-Tree code, and without
changing the user-visible semantics of locking.

Can we possibly find a way to rerun this Zipfian experiment/test case?
I bet Alexander's recent B-Tree patch will also help.

Here is why I suspect my patch will help with the Zipfian workload,
particularly on a multi-socket machine:

The logic for following downlinks in Postgres is that downlinks are a
lower bound, but we still cannot follow them when the key is equal to
our insertion scankey -- we must go left, even when we have all keys
filled out. This means that we sometimes end up holding an exclusive
buffer lock on "the first leaf page the value could be on", even
though that page doesn't have any such values (they're all in later
pages, to the left). So we accidentally lock-out insertions of the
value 1 when we insert the value 2, and of the value 2 when we insert
the value 3, and of the value 3 when we insert the value 4, and so on
down the line. This is the worse possible thing for the Zipfian test
case, I think, since most of the contention is artificial.

I think that my patch may ameliorate the problem, because:

* We make the tie-breaker heap TID attribute have DESC sort order, so
generally speaking contention starts on the leftmost page among leaf
pages full of duplicates -- for reads, and for writes.

* The patch works very hard to make sure that large groups of
duplicates are not spread across pages. There would be no mixing of
leaf pages containing the value 1 and the value 2 with the Zipfian
test case, for example. Old duplicates would be packed into full
pages.

* We can invent a fake lower-than-any-real-value heap TID in the
insertion scankey in certain cases that don't have one. This way, the
scan key as a whole is *not* equal to pivot tuples that are at least
equal on non-heap-TID attributes, so we *can* follow a downlink that
is "equal" to our scankey, provided it has a truncated-away heap TID
attribute value.

In short, the artificial "false sharing" that we see in the B-Tree for
the Zipfian test case may go away. There will be *no* contention
between an inserter (or really UPDATE-er) that inserts the value 2
with concurrent inserters of the value 1. All of the ways in which
that was likely to happen are each fixed by the patch. There is still
buffer lock contention on heap pages, and the contention of waiting
for a row lock. Maybe the busy waiting will automatically be fixed,
though, since you have one point of contention for each of the really
hot values at the far left of the leaf level.

[1]: https://commitfest.postgresql.org/20/1787/ -- Peter Geoghegan
--
Peter Geoghegan