Parallel Hash take II

Started by Thomas Munroover 8 years ago70 messages
#1Thomas Munro
thomas.munro@enterprisedb.com
5 attachment(s)

Hi hackers,

Here is a new version of my parallel-aware hash join patchset. I've
dropped 'shared' from the feature name and EXPLAIN output since that's
now implied by the word "Parallel" (that only made sense in earlier
versions that had Shared Hash and Parallel Shared Hash, but a Shared
Hash with just one participant building it didn't turn out to be very
useful so I dropped it a few versions ago). I figured for this new
round I should create a new thread, but took the liberty of copying
the CC list from the previous one[1]/messages/by-id/CAEepm=2W=cOkiZxcg6qiFQP-dHUe09aqTrEMM7yJDrHMhDv_RA@mail.gmail.com.

The main changes are:

1. Implemented the skew optimisation for parallel-aware mode. The
general approach is the same as the regular hash table: insert with a
CAS loop. The details of memory budget management are different
though. It grants chunks of budget to participants as needed even
though allocation is still per-tuple, and it has to deal with
concurrent bucket removal. I removed one level of indirection from
the skew hash table: in this version hashtable->skewBucket is an array
of HashSkewBucket instead of pointers to HashSkewBuckets allocated
separately. That makes the hash table array twice as big but avoids
one pointer hop when probing an active bucket; that refactoring was
not strictly necessary but made the changes to support parallel build
simpler.

2. Simplified costing. There is now just one control knob
"parallel_synchronization_cost", which I charge for each time the
participants will wait for each other at a barrier, to be set high
enough to dissuade the planner from using Parallel Hash for tiny hash
tables that would be faster in a parallel-oblivious hash join.
Earlier ideas about modelling the cost of shared memory access didn't
work out.

Status: I think there are probably some thinkos in the new skew
stuff. I think I need some new ideas about how to refactor things so
that there isn't quite so much "if-shared-then-this-else-that". I
think I should build some kind of test mode to control barriers so
that I can test the permutations of participant arrival phase
exhaustively. I need to propose an empirically derived default for
the GUC. There are several other details I would like to tidy up and
improve. That said, I wanted to post what I have as a checkpoint now
that I have the major remaining piece (skew optimisation) more-or-less
working and the costing at a place that I think make sense.

I attach some queries to exercise various interesting cases. I would
like to get something like these into fast-running regression test
format.

Note that this patch requires the shared record typmod patch[2]/messages/by-id/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com in
theory, since shared hash table tuples might reference bless record
types, but there is no API dependency so you can use this patch set
without applying that one. If anyone knows how to actually provoke a
parallel hash join that puts RECORD types into the hash table, I'd be
very interested to hear about it, but certainly for TPC and similar
testing that other patch set is not necessary.

Of the TPC-H queries, I find that Q3, Q5, Q7, Q8, Q9, Q10, Q12, Q14,
Q16, Q18, Q20 and Q21 make use of Parallel Hash nodes (I tested with
neqjoinsel-fix-v3.patch[3]/messages/by-id/CAEepm=3=NHHko3oOzpik+ggLy17AO+px3rGYrg3x_x05+Br9-A@mail.gmail.com also applied, which avoids some but not all
craziness in Q21). For examples that also include a
parallel-oblivious Hash see Q8 and Q10: in those queries you can see
the planner deciding that it's not worth paying
parallel_synchronization_cost = 10 to load the 25 row "nation" table.
I'll report on performance separately.

[1]: /messages/by-id/CAEepm=2W=cOkiZxcg6qiFQP-dHUe09aqTrEMM7yJDrHMhDv_RA@mail.gmail.com
[2]: /messages/by-id/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com
[3]: /messages/by-id/CAEepm=3=NHHko3oOzpik+ggLy17AO+px3rGYrg3x_x05+Br9-A@mail.gmail.com

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

parallel-hash-v16.patchset.tgzapplication/x-gzip; name=parallel-hash-v16.patchset.tgzDownload
��KxY��{[��8�����8�XXtC�=vcl�`C�L��GOKjA��n��e�f�~�����7!�g6������:]�S�^�,�����Y��K���:���K����n����i���;;�~�w������:�No������T�3�uw�M�|���Y&��P��0�V��f����2�����z�a���O���z�4��s��Mo<��a�W���w�A�_���Awg`����vw����M���|�_��������^�����R��E+�~���"2j�����W^�?U����?.g�������������z��������'�>U;;�<|��z)�0�S��&�?��E%_���������.��[�^���$���G_�A�n0�y����W7�~��C��G=�p�f0�X��7��_1&�e�]�-��C��Gi0����������x����V%s8*
5Z�*H'�������8�e�x�����y�8~�I#k�<XDw��."?���$j����Wo.��~x�����_����������}�1�'���{-�\����4�q�P���p<[N�un����I`)}J�`R���^6T��I���FA��P��2�C�8Z���@���u0�V<�ow�4�f���
V`�7��K��f����x�=�
������/�v����5V?������i�?Ofl�!!�����:4���f0���^����i�
%~�����0�nOM���_4�67ET�y��������A8�?+`���?�t�{�Vo��v�F���Hu���2���MX�5������^����:�(���r�G�����q�	v��T�T�p	S�{��"-�
i�~�\�|=�x
g�sC�=��'��r���Y�[�6�fsc��:�S��M��yO����G���(N	���g'����������hK=�P��l�z�h�:��&�G@����������z��_����-�Rx5>P�����=v��S4.Z�*�	�x9J�'9Y.p�8A�����E��-i�����4Z��e��h�(�f4������E���{�}D�����v1�i�?�h�z� O~�t rE3������Y��r������.�zBX�h���"��z�f?Wwop������f�?��O��!�&�E(�7�TC��,�$	�!��>��06����AIU�#7^P���7M���_��#�����mY������z�����M;����7[w�P�DKU����P|�	&����2�qN
�f�T}�\�[m� ��s$Q��>��L�����p��0�FQ
BE���vfaV��j!�A^$k�	q#D�Z9/�U�!*���������>�`���&�h`6O��)L��\�5�����y��������pvr4<�����G������N����]z�^9�����V���'�Ma�����
G�)����'z������ag��<�"�J��q���_�"��������'����?��2������'�z��{��q�������p���I
�����g���0��k����(���`���k���@�N��G�z�L��i�>��������'����U�P���hX�f����[��nf�U���+>���zV���?����:w�;��F�@���E��?����"���i7vU}��e)�L`��fl��~�Q�5������-�F����M�&��A��Q�0�QD��"�&��|n_sA�D��it�^�?q�h�4��.�Q�_1A^p���1h�7@���9j*�B��	�
cd$��(%����8�h��������b�}��a�i�y.��5�i��4�����S0Z�pI���N�jMD�[�&J�/eR[��F���$�`J�����+�,)F7P���p�F�F��u��)K��w�Lh���--yJ'��bg���)����+��g�N���;,��y�t'���it@�o�{����*�h��������x/q�tPP=E�N�'��4�@�f�T"����C
��C�{j+^R��+�T��K��C<@���l	C|y�b��
3_!�[p�V���:H�L��4V����t�=@�vk����D�_P��P4���)��%L��%_�Gdib=��E�
�e��H���s�M�Y����Q`!��q���*�wE$$�}F�1��(L��_C__F�>�[�4w�,v{�hT��&>��!-c�9Vd��X��o�V�b��g�GZZ��V�p������+����6��!	��c�y+i���������O�q���'^����EG
�^g�=ht�[���aL��B�l(c��Sy)b�+��P[�����mw�������T�hx�a���9m�Lf������|K�P� [Y��(	��b�viE*����f�����^�����g(�ePE��*�W�D�]K���u4CR�����?_��_wH�2�3��av�	xtW1��u��7�����V�����{{����h�]�fW�b6�!Z
�H�Ho���*���SUr6�h����$<�-t
���v���k�������v�o��8�[�o3��������[8����F��E���h��S�h�Nj���*>�hOX9���.�1Y�����E����&@�%9�L�|h-CU�l�i��gt������z�"�������"�������|(��8�Ct�h�D[M��>��U�i�h�2��y�������Q"Y��x����7�����F�8�c5�+��������j�7��'��s����.��������w�����������?��A��d���X����7.�(\;`���F��|������K7�;n�}�M�^l=��y���M��9V������������/�a�������pt���!��ql� oO0G��G���	�h��$s�Ie���������7s��9���{0���A����������
���/���������)A��X}���1* �Z�������/�#?F=��90�O�{���
l�4[���ByrO�f��5t���D�������X�h8*�>�0��d�p�!��7��gd+k3�����T�����0��+m�G"'�Z[�	M�D�����%m���`�v��z����*�nR�,�f�G��gG!~��M�U�xS��2$���mG��������5�w�v��r�������e�gR�	�UMlL�A[ �w����>�.��~�BB���<��AF��,�3j����j�S�d��
&����Q�>���f�j<�E�0�~il�����A6_�C�������re�c�(�a���^�z����O�h�Z��s���b��7�#����v������~�zo���gX��h
M`��[4��#��K�0���c�d�k���-w�X��\0�"80����f�9��=��7	��>�fj��K�*���_{�	��d��"�@"BK���J�w:{���n�8�E#�������G���	@pw��$��%f����*��k�����&Z���,�
T	9�C��~-���>��v������~��~�0{v���:��v����6���xv��T�k&����m� i��>C��E��z���lfm�o��4|}z>�M9:?�pvyQ����rx��������T���ZY"�o�����in�W��A�,�{�n��{��8(���$d"���-h)+� ���<�N-Gg�V�e�	��z���bB�t4G��@s�.��t���,���d�����N�����f��\��prB�d����[���br��v�S������p�(h�l����'@�+#�P��X:=��!&�K� ��1�l����K��|
����/�kzA��U �P��_�e�"e��L�����~sT����/2+�PM�wWh�;%���d���w����;����3h�3�
C�
�5�}�n�����$�����B��J�sZJ��[��4%�`m�����[�� �8�������
~�e�)����TP�*)i*l����O������r��3���VZ���+t����~���+�|UD��������Q�,��^N���bOH����I�v��H�DQU���������)��k0@~���]n��Uc����>,r<@HOs�J�<��/�7t0����,�Jj�(n�C��|t3
��i�d�{��6�l6aw�w�1*���pN�hN8b�;x����w�D��1��c�^��=N�����^��$f%l���V<���o�VS�8o�$�z���������#� �c�_<��{-gDy�3F@����^�K�}hy����n)�~�G�%�
	#>#<�
	A�aN�Y�/V��/��0Ho�!���	�)ghD�x���������o�y���R��f�[���~��%F4���_{1�Ji��:.%�&�V�����.��l�1�����A=�
}
�]�1�"ro4&m���9�D�la*\/�?��P���tH���2��1��X�������l��!:�J|t�7u�H�n�*d��+�����g�LW�J'L0�,��+Xs�S'eu��zQ��
s�p�����iB����~���fl��P�NG�����;�G�,��heM��h�:>!�Z�IJ�V�)�RI��{h����V���������b�Z����DqU?���Wla�2Z�~������	���O����Z�^A�j�+��������>���CF�nQ	p�������'(���:�0Z^i\�:�����f>]��L�N�"e���Uq�pZU+����������
]�$M��\N0���4+�����Cwb��+6.�o_������:�����Z�b�[;�������iwr�_��N�_�_�����u1�B
r2H��W��������\[��������:���	����qpu�nO��Ucq�5G�M����M��K�����p?�Lp�Q�j���x��&%�7SW��@�n�T�������7s�]���~h�����l��ZIPv^I��n>
���5�����Vk:������7�����tz}��Z�;�^w��Zx�4~^/�Q�B;�d�O��%T�_-cN�����n�m���0�i��&	^��F�� p��������
�����E������uJ��`����=v��<���}�/}'s�
"Kq��rBG>N
����X ��?��~��1
�D3�N�����FR3g��	�<t!��-3����d�]8�����K���e�s J������_a���A�� �U7(?
�N�Ev���A����D������_����\�!*?0w'xp�&��/����4��l��7t��p�I'��}Nz4�(a��?�O���Dk�Z�E�,{�����V�O���h+�8�^q�6��~����hF�8����kA��:���Q�D��7�[��gd��g����(�e���X���o������������q�g)���]�_���(�{$���{�0NE�7���|�.C��M���Q+��,����U@f�RB	�����G�P�[���D����zs�����'��n��I��l�%��3���NV]�,eV����;Z9�R��8vq����������a�7�x��=��D����4��K��"��/��5#Jt1�R��
~�_vm'�k�Pg�?�0��J�!.Y�s���E�H�Z��o(~Xe��yc��5An�����!����M�CA�_�Cy_�Mu:���y1�B�
�e�E7�vP3�@�?����E��K����������{�d]I`J�����E�	�j�$���/�~��7�,b�!G�Ixw���D����C+�)��	�N��	���KY�`n}��r&�nw��|;�lb���[��d_[&��=������a��Bf��7�G�����p_�s�W���*�b�wS"���������'j����e3�m~��;~+swg����N����w�t;o�t!��z�]�
z��K����JQ��N�w`���mtF08,�v�9Wz#&�#9�; ���rL��:��!��~a�&u��OB!zKZ�B����@KO��s�NB:@.8���Ou�{@Eq���f'�D���f��=-g�2S{�B�-]��23����&���cW��
gQEf���+�[�[F�r���dQ%��7I��N(f���Er8�Z-����i�%�o�+`o@�I��v�����e�(r���rir&��.R� �h��m1%/=�	_�F9��p�	���8�qfdA��q���+��CrY�
5)Q]���(,'H�b����\���������W�W���h�yq��=��������v����������?����`��A%��E)���
������4@��x��L��:�1����cE4(�
P8
|J�*�4�D�7�$�@��>A�="�<�tn�%p�,%�;���	�m��B��h'GK������@<;�8���L�'�;;nl������4����t�z=7�����,E���'���`{�-��g������,���@��+�h�W]�z^�V��|
���:����J��������������^{��p0���������l���x;;��.�8^��?M���x���y)�����]>���=�;�,G�*zn�-��e0�p�Vk[���#O����V[4����,y3�	8}��1I��`���"���{��~��!��<	�B[�����n��m�����k�
��<T�5����9���q8��|H#����G��7I[d}'#]�j5�Wo�i��T��9
A��oZ��tw����#�#�������lv��7!��x���%��f��0}���:&'��I���H�/����:�#N`��
MmHg�^�4'�+([��Md
h��W������R��?��h�����8Z.���o�bx�M{?��J�$�	�z��q�O�f��@o������J�3xb.X�^G�>�x�3�����>�l���8o�~��������R�Y�S�I��M�H�����p���#��q�gdn I=^P�W����!&v���"0��(���$#�nF���<��#@v�M�ak��Q`�5�rM�����+�g `�j��NP���GZ������/q����+�s�s���3	���, �Dxcr�)��?����H�����Ox���Wo���a����J�8!�_���D�1���I!�h9�-EF ������=Ia�f@���	(,�d��}d1�}���xq���^@��%!��d������a��F�1���e&c6���1�
o3�����
	�UsnN0��3���!�
#M�:�c; \
�^E�i��(b� ���i0�����I�z��	/!	����#T���h� `�J�������`�/��v����&�5@��@>=s.r��������nJ������� ������y�L�'L;�9'����fg�?_�9�L9q��bK��l��es���?x=
�����[�����A�9#(���M� �*i��ho9�7�������>�7pDR�SN���R�&����^y�_:{_��@o���i�;��q|��|����K����1y��2��,F@3�5g[��%b	G����"�9����}/�Qr����Zd��
`M�������#Eo��������4���a��=�c��Y��I�9�Z�*�T��>�B���]��=���:k�xL�BhX��!�-ity����V�B��,�O�3;�9��dbz���?�����@����	e.1�'1���N�V)��8s�J�����(0�]=X��P��	u�`
���������P-��*n��[`dAc�d�$T�X�l�	���I��D7�<�Y`��
%8�%�=��c���J('6�$�$<0�qS�7M���t����>�����;�A�Y�[���O�d��\
��o+i��d�����+���?���z �Z�_����_����"#��y!�@A3��B�D7������WL[]VT�:�'��Z�d�t�oi��!����_�>><�<>}/�����t���#���
���%��G�c=�������e�XL*������1����<`�~R��E,�u�(�2@��k�fr�"�c��9��
�����O46�
"�������[�����D\��|VO��T��������E�D���Kcy��|�JT��*���3�2���'[����E��f2�������'2����O������3����c`a��B}�%�����
:��X�I�"1:�G��|�^��f{@�'g,"!�����wH���@�p�������tf�Vb��0Ib[3��|�����'L*I�7��[<�$F�4
eA��H��(��?���E�9^������W8������
�Q���� �YpK���x���j6-��1�������.�)�H����Dv���"I��Ea�Q4��G �!��6�i`D,�~Ry�@�\�+i+�J����#IQ�u����d���49�t����C���f�FsH��_�~m�q������ �KO��l���s-�N�:�"�o���s����s�����<H��Tj3���SJ�L@+`[����H�2���0�����RC���u��1�'�>cu5w�7��m�/�r.k��A�����]�Z�,D����#o������������)�KEe>��v�]�4���1��YbM3���La�x�t���b����'�s�$/��p����,�����h������C�� ��	����44>��*���%02k��u�.2����54"HId_`mo��]-C!���IY2dK@�(�eL���R�����\�2�8�6T��mQG3�0?�8�u���� �=�Y��������C��P�{������8����r(�T/9�w�q��4U*;���2{�V�_WrA�2�Y����ds�x(8���{�����8+2e�
m3b����Fm�;�2�����0�@�d�s<�c�E�j6��������"����e,�&��z:l���`����j��@:)n
��ebAB�K1�h���W��A��o���r�����K��+vIR������V8�.�����f��f�1Z�*��z�H���l�5��m����@�>g�r��%���E��u���DKK7���;���^�Pp�j���R�=b�<�	+�����kne��,U�mp
����O<4�3�1g���Xg��>Nu�>��M�e�j�9�XU1+&U���O������������D������)g}���
t�Z#�,~�J������/^�c	�$��|�u����$�T��vF�V�5�;��
z�������n������@7��N9�H%���v�R�\��~�v���Nww����T��T��	O�w~�������)��2��6��%�������|��Vxs�2����}��NW!�=� J<���Z���2M�@<�h�m/���8�Fl�����$�$?l��6@nCt0^EH�cP�����\d2d{.6LMOf�\��K�E�� y�)����tZF]�;�lR`��8D���D.@Q��ZP��-�@v)�,
R�L��lUB�Cf/a��X
����]I�i�8v_)��h���@��7�Y�����3:��+�m��;w�����ts���N�_�~��_+�_���� ��F����=����o�`��/��oh4�n8��B)L��}��z�T'	�P�� ����f#?��&��`f��i�A��\�#��O6�����jA�y���q��S�Ab@$Vg�f������zDQT�
V��@���>^g�0%���TgZt#AT���.6���!��f��-1jke����f
��&:���`	�$E��Bb�Ip��LpX�-��'�� q/1�e�+\dp����H��.Cg�Y�.C�"�sA]�������������f(����"�{;��x2�����Z��t������~�]�2X����Vt��������n#P�W�T�����������j����A����-���k��������\�����ip��������7]s�����@2��*�w(�(�2��9!��2��D���S:��U<�	���v�$KPo�	]�@�=Zb����^����;T����4[[���t�;��������C�����/�Zf_���A��Q���bF�?��y:�}\v"�s���Jl�7l���7��+���YKF-�B������!c$��#��gLLN�[���N5�(F�
a����Y��<�l�Z��||��W>�
i��m�`���Je,�A���Ss+)��
h�z������!�'t�`�Q��,x�-r���3�R8f���3�W��l��� z8��mV�����1���%<Y�����n`	�#�D/�w%����~n��dS]��s�+Dyze����bw�,b@�
��=�',���:e��s������_�F�7��M���O�4�%�?��>����2��f�����Nz�����������DzJ.�"����(1&��H���2�-c)����R��}AF#p�x���|KG�UQ<�qp��85y���"w���;=w��\G�%�7�R���?�(t��X_��f�V������^��	����'&i��Lt����BQ���Kz���
�������-F\��L0���������xx��C��o�
4�/�R����M�'��+=de���jG����JN"5�tu	�=���C�t�{��V����T����T�X"�W��z�rfc3��d�s4�'��e��dQq����iO@�iS���I�%n-��n@A����������a_��p�����D#���Q�V���"�Q
o���fB�?s���b�,?a��DW����r��e���#S4	�T.y�O�z��~��m	�:U�)����4�����o�8����i5N��f�"������t�9P�
54�/e������
�<�PI�����Q�����q -3������YHL������PG|L(��4�A~6���V��.jD��K���o�?8�J���'D��E�T���dk�F
�f}�9�6�]��C��qO�l��pt�t3[��.2��k>NAs��
8��9:n���liKS�$7^J����?�_����w��>���b�cA��sS�����f,�����2���mE4k�41�]�c�������c�X���$(��Q}�1���
�;�(���r>�{�_��"�X�h��R�o����r#�����p/J��-]��`;���$:��!57��pb�!�����p��+Y]�pmT�:4�����|��w�*�[�2��}t��|a����.�D���{�L�����
i�9NX1+�Ze�P.N���?2�����^j��r$��9b)�Zgt@�.R+$�k��M'�����t;(��	����?y�����S�&��,�iY�Rf��bm�a�z��;�^��p���8f��/�.%����V�����M�spy������_D��*�:�"�
"`#\����p9�L���
�"���y��g�����ABB�R<�U5�RmrAk���h��&�+����gu����?�&�W��[����&����AV�����A.�c���W�����k� �yu���+b\ �a��~�����G����	�P�
7����blV��F��dCI�+�'��JDC�YiK�5������4��-c��^����<J�;Y��>�������@�mm^��]Y��t6u���@���"t�Wqf#���\�fe})����+���LX.]]��e���&����w���V�:�C���)�bW�;W�{�~p����w��'*w��Z��^�]k������)���U�F���5Pe*����{�l��]d���7���]����dW�z�.�rK���^�kg2����G�����������NE��l+�|�~w�����Q��x�����{k�9��[Q�,�������_���49'=�8Da�D��f����nw���%���hL�L� ��F��X?wd�e�H\`�WmK
��|ph�m��S��1����	������	���>Y��M�)���t����Y����4z���;������4��K���f��"A���!�	��AP�R)*�,_FPs��'[��^L�������,������e���C��3�S o�C�+�������S��LD�7;��������WC�83�?w���T������r�V	XEqP��zX���
�>--��Z	�j4�W��)��X�_�����a
����7�g��aE���6��l����/�����"����#��yAXs��'��HS����|�h�v4)v��\�'�������I�
��gZ�Bh�����O�0_�Iz��}��5��5����1���N(M	z�[\�#��R�K���o��h���X�e�6��X2��?���
Y6
���$�Ii�=
��z*�f����|�'Cz>�����67V2�2��h�[8�����
f��o�+��/���I�Z$(B�f�w�X
�Sw��<������}� �>\��nA�ZQ�6�VD�]�=����hw�jy��x��O�5K���Q|���"���6���=-+�QQP��)Bg���M�,�R@H�j^=�2��A��B�����8E����������3"��~�����rQ��
2��MZ�������RY�gd�����
ni`�I`�}��9��
yzb�����u2�������2������Q����6jv�V�7��������k%��#d�q	(�����>6er��L�x��b#�Y{VH[�K�����1�3U
���O��	p��|x.�'#"�2���T���x�	+]��Em3�*�'A�H6��
'�]�9-?G�Sf�|����L��Ap��X�\�b�dWQ�{���I�_V�tYa#\q�2
�0��'~�(���t���F�����9���m�������_Ku0<�E�k���%X�*�j|%��`�^�}x=�9���#��Nnb����J�^�C��F�k��d��N�x:�	����~c�V$"�XqHsMu������w�N���v<o������I&�@� ������.������Xa*BrQf&����F����Q����^��
����������
0_#(Q����3��y��<j��f`CI�t������!Z3�~LJ�}����+����:�Ngg���t����������l��9V�>���qMl��=T�W<L����!Z�#���?;���y������k����w'����^w�j�y�`:���L�<5�g�6$1��G�~��R��i�z�~��R��*@�9�GwsF�qB����s>������#���cr
s��)�9�9�2��1Z�\����c<�`Y�i2�����=��wR1���Q4������������ ��1����q\�|D�|k)�qHk�~���q��~)����D��>a�����a��+��=C�lf����SH��������K�r�E�TJ/k+�1���N:���n���N��������5���B]Q���=�L����������%i0:���]�}}�y�~e-��{�����%�7�����WZ�������������]H�����B��G�FS���h<��v���;��:�������"��1�������-�kF3XSaR]�u������V�o�)��\\/�It:6��Tc������w���l�����4���t������~���i��G����J�:��d�6&:��G��`�c�1(We�[�y-k���
�f�f?��h�3��n����\3V��������������l�����k�#�������v����wv��no�-Q'�z������3F��+L\+�m(+R�;����+���H�]�y%^�u������<����F�;��n����jy���d���+w\�\jF>��X�K����Y���B�v������k�3��8��g�5lS�P�k@]�����5��51|�	�.�����;���[���c�� +e�*s`1���
<_����>�=���#8��v{o���e��G�6�0�P�����Qb��%�~b#����/���������2������-i}�~�}��b,ZC�(��G������V������?�>���r��� i��|�~.��������n���N���������u����H�\N_a���HMgf0�<�7:_Dbr�Q��1��\$��p.JV��
;o^n��S��I�`�*up�#��g�*��ra���x��s�A9���Qa����9}���m�����
���,�N�`3ww���T1����	��^����������������[�*�WtT4$�Ja�/�����)������$4���2�`b<lp��9�W��n�����D��N�uo2��_��wy����su�c���q�{S��h��R�>������p�M�8��7N"J���JM�@[0��������1�kOhA�Ex=[&�5��Op���P��M�P�)��`��xvK)�o�������T�A.��fRu%��O �����k�-qT���Z��{e.x��mR���F���i����t(��#����j�8�������+��3�4�Na�X����.L�5�Z\;���[[�.�}�3�@��$wQ�{����1����MH����Q�7�O�k������[]8���"@�f��0�F���p���/��9S�EZE{�N����p��[Pa5����n/�Sy7������ve�eB���FN�������
�rf!��}�I�|������8�;~���F�
2������Z�/�7�)�n��,�D}'K�nS?�6�F�pqc���3zBJ��8E�.qP�������<�>��`K����clq�m4.��F~�x���J�;�ax��/��'���'G�=R����`�f�~�;�2����,��F�p9�/i=�'����l!����z����Xo���f�T��X�4���?i����/Z�4���3���
w�Yv��4q�h�)���Z����#�x��BiXB�Tf&/���6��Lk��[������Zh}�*����[�;1�XB��\�|��m��Y����s�%��nsu$�F
�d��:?�Bc9����:E,��)Wgh�Z|��JMN��Yt)�r��0:1,,�K��k&S }�6�D����f	qqK�AA>�,�#.�M���p��X���
����S	��y�?[x���[,�������/�Cq�7�3#���g�O���r6
Y�=^TQ"�2f��ZB~�x�?:}�Ss��������
�o[.�w����s��7���lg9�vA�O������g��g�?��_>��J����v'_�������k��Z����
s��h����Za]��3�.rzJ��|'��5��R:^]���)V�2]�����	5�U����������r����|��@�"
!��N�h���F���IgS}����t�����
1�������6f���<pU�zjBw�#l�U�
�qa�4��lr0O���66��XX�� $�&����LJE��+�d�MX�9f��c�DU�1?��YO����SLC��3g��76�
Z�<I����M)����m�����)�9� ��:���c���6���,12��	Ny������r�0��&�uQ�����	�
��^�<���4*
m���D�K7a�9�{�k�p.A�'FqL�N)�(c���I���������*x��6�Rw�[��x�^�m�M���]��4F�^{��������d���re��<V�^�fR�\�-j�C�3E���ym����h�l��������������0�������F�
p��Q�TTlFT	-�TO��1�����S4��Y0RVCz�MEr��4��,�g9�[�x1��~!z����2�	q�f~��FQ������7GBMT�����k���U_g��+�zq���|�M|�
���-����������~_NUx��~)������&��bQe)�����$���>����K���M{�T;B�eM1	�ihR����
Xf`��s�xzU�z$���*�bVb�q.BsN��'L��yfYV�����v��v�MU^�F��������j[~�������a���h��m�9,�$���p^n"lj��<��<�B>�d����9]�%?��K\K*]�tH���+����9_��M���`gY��N���H��n��O��?��������!���r���zyr����n	P*�H|B��s6Q=)�vk�<]�$_s��$�\�!��`2L7��F��zF����W:�7=s
b��/��69��T D5�+M8S,�M�a����!�/��<s����$K�B�#bwN;��{�sK���SL1]��������8Lug^�t�|���d�
T"����o�����kV�����,��Y����T��gX���9�H���3%�N
��4�����qH�������������;z�����g�����QRX{�����>��	9�=qf��?����	�RT���@�Q��XiU�rj�_�4@5��� ��L��,�j�[l�%�P���C���i�k�q��
�7�l����4���k3���!������n>���.���S��1�A�1P]X�`,8�Nt�� 6��Y�����Z�����b������r!�t��2�E������S��c���f%	
�%-;���_������l��x�Ki���c���Zqo6b8r�o@����\���A�+"q��NH�����z��l���u��4��\O)fE6��=WE��������7�4�r�>��4��'�7)�`�7�tx��w��!}���I/3j�a�����9�N/�L��k�n�zBr���1������k�v���<a�~���nH@���O}�x���?���jg����N�cD�p�a�]�m���~8�=���}���4mns��Tbp���	����Y09&��M�T���
�ck �����6<���c�fg|����c����������j�;����t���0�}�.���e~���e�`��h��8�+U����_U�;h���w��X}Pi�8����_b�x���$��Dk-�Jz�d��.��Y���~�����|ACe)�T�u���g���������^|8��<�����hx
bvY�7'�/N�
	)���0��x\��5�#�)����QC��^�;cMd����������,Y���p��������_���^���E�/��l���^���vuy^-_�����m��{�-������
�%�@�a���5H�+�1�G�E0��]�G�-���~7y�Yd�I/q��������2/�4?;?��&����I
����|���9���Z�!T��S�-���!P*�C���%�j��T�p�b������ndp����
~T�U�Q�w	n{C���H���^��P':E�s�V��Q�*������}X�
kc��-�K��m)�����5���#�{hc�-kv@g�����W�X�/������7�@Gu*�T�w����&�����ws!�*��9[%�����5)IA��#��"=�t ��P5�������S��VL�����t\�
J�q�|<U7b"��2��"Gl
�'E�nT�T���V�&o"*oT�'HI�����'�=�Q�p��r5������M���c���R�����h�������K�s��,�na��
�����r>a9��~�������jXc���1"��E,�}��/���6F�f�}�\�kf���d�jd�U���v�r�o|���7��8[����9�,���Q*&'yC�"f�<L5�-�i�x�H
,�ys0�HWuM��o��A��?H5p�����*�^
S��
?
�B����Et��P�G�����:O��P�a�����<@4�u�
�)��|�:|�EF�mw�l��*���3���M��z=�X0���GJ��b��Y1�I�l5l��k(g0
-�c

��2Y0�w��� ����T���n�G�8�����*�]�
3Z����1��W'�o@m��6�1<>eR����e�i�,$Wq-����&o��r�[3��a< R���e���;�����B���>/l��J���r��e��K�u ���E�5�+�Wr<^.n
����RLs�����$�Fafl�������y[��rx.�[!��y��]��	�WG�:]%DE��90-��h��������
�'/���tO�B��4�Z7}g�����*N��JX/.��43j~SH#�m�|anJ�%F�\�@�.����f���]?���z�8����������j��Y�f��1��u�2�P9
��1w�j�AfY�1�C�G��,@�Mw��5�a��n������xu��b!��5u-�"{Y�+m���z	Hx2�hzJ��%y�\�+��C���'l�+�o,�#p#_{�(�iF4�w���m��&g�-���=��,�����4�
���`�'o��Q����a�\���m����4��;�������e8i��(��Z�5�8�"l!�I�}�b��p"XD��a��+0N�]i� N^pnh�uE����s���9���:���L)F��8�P��+EZ"�d-klXk?{F��V��3��N��������S�����#�{��R�q^D�H��c���0g�~w{Gc1AgD�W3pR$P���p;�{���G��q
7a0z�8��<���'�+H�I�;QF/8�j@�=�6aX�0[�����f�2��+����(
F����b��m�/�����0�=�e�y20��B�������Xy��5p.�<�L<}Kt[q�^�FZ�0��\��� L�e1�t�3��}��/�������� 2�Y�/�L"�����F8��8��9:/A�#@�C�����S3�|V3������,~�^-��!~o�JH���8�7;N�_�Kj����*��w�������x��E��i�h�`�3�c�s2�G17d��$Z�0�9H�����q�@��\�����������&xS3#p���A�%q��5�����(�a��C5�0������ ��f���s�
���k��1:b[�'
�����u��V��S	/2�������#��n��e������3����k!J�$��g�!~]f�(���u��N�g����e-���7��������������(h��������/�xtx9|z9<~�F���?8�8�<^\\mi�0t�'W5�b�9{V0Cb�Y��[�e�n������1�*�cpUN'�UK���j���	��Z�d
=�hk����&�i�- [����^ b�c��9�#NA�'���F��:�!�������4�
�W��'6*�5�a^7C�hqZ��U�1�i�&Xs�>��Z8�*.��&��y��gP��0����.'X���b��H�����������y�BW���Q~xuhT�7�`-�	���]����e8mE��o��U�N5����w�J���~tS]�Z���v�����#?G�*���*��~Wt�1<���zEO=�S/���,���h!�	N�,
��c_X?���������v�� uSC��m��� Z&F!m!G�����h�lH�\�`%�#H�vJw�c�������1`�!�"�[HZ��#�6_u�X3�[�w���&�,���)|�B!QS@(�G@O�����4o
z�������d�]�%B�4�B�������&�~
���DWF���<�.��Q�p�S������1@��D�����.�V	Z����5��;��e��I�`�^6�=S�/a2Vw'���3��QabJsc�(������h�Z���u����<MJ�b'e=#f��<W]vx��Am����0�h�ns6�%;�g�Gw��E[�������W<W��{�o~t��q}�m��l??�kq���O�*���N�	���������-����I���oR��I	���O}��x���prbq?#Y�Af�����|��e��n���c����C�2lC�8��
��Ey<)`8
vQr�:����E�%s��:�F.�|�7��1��p���.���gA^s�p/�O?��F��3�����Q�ZL�Lv�2�/Y2(QpDH0�E�n�,Y�$�q�F�@f�N�Y��M�i����x�w.��
��6��w'�������������r}�Y���N���y -4!��3?Y�((f�
#NIGb:Z��!7k�^�����IXR�-�vBM�����M�\���frrr����������{H��as����]{�;�������7�eH���|����+#)�G��gl^1�l�U�H��a�e�����0��ATw+�j��j���e���
��s�q����%�{d�a6��p��f��"��G���(4aA�����U)=<.���	:�l��T}�
�R��5�	���,������c�;F�9
����t��
%�$��H�(�+���&�@�u���R����D��f�v�)�**}vz�}�zO��6x8d���'��O�s9����~{oC��un������3��e�'�4f9%��\�OPu�"��d�������[A+�����`�:��r��4dL�q^/��?V$����lno�t�vo�m�v{���7���k�� Hw���6��������d����p�j����Q
(<����p����'�F�t�EQ&������f$W(o�Fs�����F�u��L���U�=nT+�y� ~$
�\Ho-KK�^`su[���\LJ��@�o�������(��c��B��:Ca�dh'q���LG���6�Y�[�����}��;1����k�CB��"f��8:*����H��nDg��+�bK����)����)3!�����<���$��Qm��M3`�N�
�����I9�4�'�'�4/l&Zl�zD�T�`�m�_��h��g��/RX�9u't���0-���&� �ub�SD�Z=��2���|k�+�.|���!��+� ~��Ig�C�b>�sz���o���,����9U��*ncY��t�������<��������$�z�e���N-D�u�	��Y�'Xr���R5��Qh���GL\{�l�#���$Th���SJ�i���������ec��Q�!XR(a�"������������t�0�ODz:2l����������������h���8:��F�4�r������c��k��p<!"���_����4�`1W.|KW{�^&�;��o��UI8� ���ar��S�3��,�r��d����B�F��=����vh:�>�{B_e�CV2���sgT2��
#X$ �<�d�	�����agz�I��qv��~�����i< _w������!�_����yQff��8N��
]��N��M�I|���"�Hc�9G���k�8	x�t�����o�n:�cY ��������6i���������>�O�� k=,�������n���-C�c(�m������@�7�UG�Kv�{m�,�9���S�N�"�m0����	.�^���N����7:mJ4�J]�E�?�M����)i�Ub�|����Hc\Y�Y&�d�����~�zW�������6�#�k\���Y_�!���Ni��(b�2�8'��%��8VDH�FF�{��2�m
�����W�j���jv8�7��B�R�`�����������m�CH����;�fN&S������C���/Y�u�����cz�����F�hg
wl�A�!�f��q�2�I>WG�O��_������i�YW�0B��~��&�<.�?(�����@�/Zu���[qr�ul/�)��WRL�1aZI:��0$�q#��2[��k�1��b_\+w���D����6��r���!����?4{�$rCh�����>H���
!eh�"�U�\d���s���n�Y2{�>R��	Nn����l�,	�&?�i���k

��r�����ec�( 0����h �Z�c�h.���s��Z��R���
q�<Z[Y����GC-eo�|c������$h�4������>d��e}
����&�OBA�%u~@�{H���6�R����K�Sr����S1j���&HC�����1�����8��
D�33���S�W����eXX�)��W���������V,�����^1�d9*��Q��j�'�s|���eFn������v�Ml��_{��Z��l�5n
y�8(RC����I��8/{f�`PJ�$�
����C���i���6��E�j���wc��X7*6����U��Yr1Rh�C<�.������R��k;3{qH�
_TD#D�v�=�:�|�%�I���i�h���P�wp&H���0YRo|�y����c�dH���L�m�S�'l��j����sMuP���D�s.���D�Hq����d�����<e���~B#��q#�r�V�e�����
O�r��e��'��$H�z��8��b�5I���J���;�3����1�Uw�)���c'VB��f;����*�`��m=��v��D4@,2b,rP��g���������������g�Ex~V���v�Lv�\03�S�=��`s"�b$[fvT�e���v):�+���HR�(rR�h�je�#T�,�	��F�T���Dms�����9��6����8���8
%n:��=���e�P �'�l0S66_f�D��	d�/�u�G2���4}�r�o
,�*t�3g2��t�Vi�OKL�
�B����H��"�j���CB�	'&a���c��s�t�����hK����-�r��5:��!��nP���xj��8�����'
#�Y6�E��!aZ:�d�fA�#�u��A&��1Y�������������0�����K�ty���!|:{3|y���$�����m�����U����:��,�* �X���NB�X~+�+0P�H����Y��7b��$��'��f����{wo�S(�),����!/��/��L�&u��������o�$0��C�3b�4n���c�������u�*�&���<�N:��4���8Y�j��&�/K]��&�Zv��.��S�r�V*��mx\��}l�=��B�s���/R��@$>]e(|r�g5�3�d
a�0Ui���'��LUB���^�)����|e��h�������������:&k���EMeH-���K�����$K�"	z;�>����v�b�������:t�*�	�}�N��/��~_��2Q��P���6�@�Kr��&Q���K�iKV7�Dd<�
�2�oq��f��q�_(�H��v�7�l����5l��GJ-SB���Y�&�lU�[����
g��(�f��v��nl��a]U���	�� �?���%"2�D�M����_�;��6���G��k#�Z�� 
f[���k]��-��[�����+{�MJ���7������,�7�#+/!kZ.��%�|�C��/�����N�������6V�Zy)�-e�?�~5j��Y����c�*�>�2F��--u��C��n��U>^����6E����i��%w������nmq��/��j��'��e��6�d*Q���� yW��w,p�����x��b���TC-��^�"����be�4�}���as4���)sxXf��kl
�:7������?A�I��$B����?��T�zd�
���":�&�������������s%�����^�W8r|�T� �=�-�&GO*+�X�aK_�3E&��H�R�y/.F������
�e��4}�����w8�ngg�b�e��^�������R�!�B{.[���4M}! �6!��R"�)��<&�E(C��U�"Jr���[P�>k�pW��t��bS�H��=w}���v�hk��5�?d3��!��6��UG����|C��k�zR� 3�L	���la��&�h=xio�x^r �r�^���	�[�.��~����������c}b}�nD)����3a���!�>^��>�:�^w��,+�	O7,[��oP������3WUY�����dMX�������7
m�W9��~���2�Aut���2�������K%g
^���U.1��=��2�I�����Y9���<�����z���V���H9�l����ES�L��#��"+�6QB�9���a���n>�sC���8�����V��7�L��_���^p,.���6�n6��t�[6�Ef�Q��f��U<��_n���"D����W���s9t����yAU�u���#K�%V��xj�GI	�N�k���.V!���V���=�!��-
'����L��W���pz����}����:x�!f�X�"jBM�������������`�MC��X�*��x�����u�:�Ur��M�@O��%rfG��kI���G���kJ}��u�v;���V�����I���L��)UC,^W�nK%*����=`��1�Q��c)/Y��!8r��4'�}�{��b���8������c������	�*Hq3pna��Dd"��E��~��f�x���m��5�d���j�H���2���s���w�l����a4�#�o��xu�������|��-q��/��z�����������d���'���`��o0���;�j����������o��z�}���V�u�f�����}���[��G"�`��jM��w8v����
�Ieq��������o�5�`���?�V��������`�H�����.1����� ��h[nPA��+e���g^	_�At��.�M�]��0���7e��E\4]F��s_��xKu��]L�G��/�|�����7S���EB�7q�\�����70������T|�����o�Y��M��
�����������<��k\��u�)��o�i���N�������[{W)��qn�0�&K�m���|E�&����� �x<1����u��g������xk]g��9�.���J
��2�N��=k3np������4��w03$�W^LF��c:f.9��l,�97i�i0�����N0�:�d����|��
t\�i-K&*��W ��M0��	��jOx��-����9�mi)3q7�O:�z����;�Q���7j_?��;:�n�����Q&�.�b�����uh�D���W?�����0}�nS��T�������O�*�������V��<#y�=�={R�kk��x�����H).��\7�.L�,��Z��
���#���/�O�{��7�
F3\���#W+�s��|�q�'�z���7m���l?L��I{�e�jHE6]l���������I����r�o�����o����a���c}�����Eczh��G�e� {���|!V��hx� ;���!U�z�����z��������"����X�=�Q���.��s��n�����^l���/|�c>����FySz@��(T4����?���{���D�HnH/��}���>�TLm�����;��<3��?��D%��3_[f��*%o��V$���W��,�W����-L�e�TAS��z���2�����D���w�&=PM@����wv���	"�����(���;���������R���={ll3�m�F;���m&�N�-H��L�$4�!^��|J$��V�51�94�U���7Q���u.����J��/;F����g�K���wh����(N����(~3��W���H'AD��>\���/������[��F=q��������<b�>�d`�S�G�GAzm��t�,c����"+�,�2����dz������9�M$���cyR	l����p�
�kSH��^:%�m��E���QH�&�b"<�<'���X�:��l��������N�zi��CE=��2�P������n����?�g������:��M�kX��v����������.F��m��;��:�����v�����n��V�^�7h�w��������T���:�,�F$e��hE;h6��x��Q������q4�.�����w�����j�������E���D��Z�?U�K���32����������z�������T���������`21yt�Bd������g��KK	y7m���
giO@��A����8��&�������a��ONFZ@��(4��C?������L��@��������W����y�!����O�s}���F�mL�I�b���]��W�.��g�G�������3�����:*�E���������E;")I����{XI�gkn����)�{�I���yw���
��������^g<����G�����?���v5�����v���c�Z����/C�|��v0���fvKn
4�d��M:��<c��Z�����>9>�t�{�*{��[ O8Z���# �S��W?�j��"����PtC��������-�v:�9��B2j�H��]��?|b�������������2|uq`�k@x�>����o���{D����^�|�P������������������3�o.O��0�/��=/��%]sE���T&���B5��'��9�X��xA��"Mg�)�EgZ�����G�����Vk�7n�y��������U��������)�F����F��h�&9A�t��y�pQ1�A�|�����������j����A���S<z����P�z���C���~&�m����qmXh:�������������P����k;-���t0����A��r�7:Ld^��������4��2w�HR(I!��.�fg����q��Q�%��n�8=��y������>�����0 zV�d(�\�y����<s����O�T�0Y�V�A��~��MG4��$������<xi@����b��p�:'=�^�i��Pu��5�y�v47���)�t�c�k���+
���L��b'�&]����u5=2uSO����R)C�������U3&�y�5���I�T�n.�21--�����u��X��i���q�$�p�_a�#��u���Y��UAd��4xV?���IKq`�^%fw���]IA)��~����3`'b�>R
y�yP�:0�0T]���~�%�������.�|�����Ld�+�.��o�O�~8~yr4������w/��|V6�l�y�H��O����� �����r�UG��5Q(���*�kqh�q^!��V#�7s$�����D��H�Hm2����G�*�[{�x0��~�z�d�|�7V*�D0 ��(�N����=�2O/�c��l�r'��e dwW���{��&�l_�o�)�9�`#��m�&x���`8���s:e�l��T�*	��L�����2+�T2���{711
���|�\����/b[�)��)���w�brP�_<s�x�;���U������S����'�L��C����W#��H�C}�<��.
-��;D5���h�����z�h5k���]�s����uH��KP5�
���@��)�\���H��X}=��Z��1��9"�������E����~]�e������%;{������I��z�Z6��aU���i��~q]��a�4m�R��Q��q�^�!-�����
��9���,���f~��7J00������5b����"�j�� J��j����5�������1������U�P��QF��i�I!�M��1c��-��a�(���5|v|�3�'^����M���AE*�K0����A�r!��v�"(4G�������oL�sbN5���A��H��.`�
�
�"Q�dW�n���W��4P�%jF}9z5��{����+1����	��o��!�����J:��FP����ndX���%�M�e[:�#%�`t���..�FT���#��_���LD��z)�������������������q'O��L��3+���'�
@>�!��T������y+9�=:B�X���gl��z�"�������S����W�����/)�@������SFU��VD��zf!����M���6*_j>Up�7�����}��eGm��*Z$�ui���;Fr~��� �q?���T��=�����������`�J��ll���������'_��8����E������la<��H�	�����K����)���%.+�����R�
F$����IL���3�2�K}���P��1���C��&g�Q��E�����H� [H�k>'_I��fp��ZRL�<\dW��wW	�e�q��'�r7m��J�t����BCf�U����s����\J��nG�I5�>�z�!bqt��?K���
*�p�YX�M���H��������O9I�h��k��.2��J����Ao!����f������)���'�J�D����,4�Dh���`&�����V�6�����QIT�Y!�k3��6��P.�
G�
@���I!y8�f�S��u���������nahcu�L��|$�_e3���Z�<�e��v�S�$^��\5W�-?��P���J�J��tp��r��#M��t��bx�a^�_yO���h'0�E����q�+(<�E�>�&�&H7wDf�/��/��$�����`�QA��A����oG!Wj,C7iH�����8�1a�����Ah�zO�2�R��
�[�8A��#�����%0(����N�����Q�G��),���!���=>��z3�g=�g)u5p�����s�e�L"��H�t]*^��ldn-C���C�8m�M�7'�F��F�=+�E��H+K����SK�����X5��g�
�|��z`�YB_f�S�����&�$�6w}�!����7����cD`�����PrVl�
(� �Qz+D&kX*��c`�Aw���2��,
����&\��SmvbXZk�5A�Y����-�4�`K��'��{:^�M�W��z9t���
{��!��w����%���{�r��(p]w(�?K�O���%���2�.��SE�0�S����y�9
��j�}�h��)���W�k����X��v��/�,�NBk�l�n�V3+	���-F$`X���g[_A<7y���mYV�@R
�0|�:[C5�}���(��@�h�fbm��u3�_��S^�����s�\��S2���B������'.=�B�ZA:�����<��v���
s��
FF@������]2�p��@��G��/
Y�X��9�uR�`
K�#� H��� K9���}�3)6D�}������p��F��)No��g��o�v�h�4a:�nD[�*�rc_0�4yx5M��a��c�1�-����0��J|_��D�J]a�X\�T�����@1�x(5�c��|P�dC�@@&�~����/D�t��X�oq��kE��@�
��P�U{���Sj��j{$��Kw?<[�9���_g(������&�O�iu�"�	N���b�f����?���Z��ud���[|���y>�5[�BM�{�7Wd��x�)j�Dja��J��B�XP�	`r
�>O���X���CB@4���9�t9}���k�2,Z��d`�L36Ip)/1��m�dhqI��I����M�����8�����qK�$�k��9b�8+�})�k&��~�!�5����Ji [M�Q$�Y�GQ-�|l�;g�,�9�Vl���C��a=
!�|`�X��u���A��xu�2�S��{�*������,��B�^����w7<o�c��n��h�,fP��j�	V2H�yQhL���������d�c��Ae6�1+��=c�`v�`Qz�f�_P
6�_K/�zR����Teu��\_���X��2O�y��l�%?�{���{�U��
M����$����F�J���H
�Y��TQ��q����h�.�"]���H�|��9���ox*��Y�G��+��/4�?J��8������������\��_�vB�Vc��C�b�&7�%�E���P�
/eUXdW�EPJ��a9�k�
��YZQMH�0C��}������"���w
�5��f�&b~�FW��VfT��Tkk�^�_���Zd������U�
�&[�U
��4��"]@��G�~���?�>J���`��#@0�R������KC�l�R��"�X�����������g�_�{)�!�6����j�	����3��he HF���V1���&Rt�_�Jy��X� _�k���q���R`O4�P�����^h0b�`z����B��^	`o��Cxj�~���B�2�(7<z���'����W�c�gGG���fX��zY��(�<�<��v��3d�rp5�rW�0t��}��jBE"�*'9���y4�dk��aO��cZ=7�U]�RK����n�R|�M�@&lo��3WpG-��;��LU���<u����m �>��Xa�9�� �!�w#��e�H�\d�z"1�f�x�AN(��k�����
 �	���1����2�N
�Di�{"0q�Z'�)b-Y�hL]�6�S��i��h)���qBXl}y���v<#MQ�2w��g��,�q,��v��r�#2A��45�_P������O�����7�o�*_J��b-��P����q������z�^���6a����V��/��4�M��� �*-��8�-y�S�Z�����4��Jq\BQ�pf��t�JB��;������9r�Lv����������=�Vf���\?.�/��R��`�o���V�Y�e��`I�Q��1t�j��Zi���l�M�����|����L�����qi1�jo;������ie���*�;��#l+�:�8s�]%T��<+��i�=���"�>T��]�N!��2�,�����B*����X4g��>X$�G^�p��;J�w�0������<8]�����.���q��,�E���\�!]�!8H(zqX��
���>B����k�lZ�2�e 3EJ�)��L���&:h��\���A���	�����^���������������f�o�B*��0����7�6��zO�\��_\�/66��t1����{���0��PJ��ApA!��.���}��2���q$��[t��8y���i�{���<�6Q�N���R���5��7��g��OgG���Go�O�^�K���?��a�fU�Y]4z�1^��V�����{��E*?��R�uH)�e����g�R-���"�D�JcP� b�*�
b���,D?X�����t��*�v���Mx���Oh>T
�=���E�-	���G��5)��f���H��T2����j>��a�Ne�xP�X:5�)�~�t}�wHD�vx���2M�}��q�����,�_�%�T��E�UW"��A$rc����lH�Z�08����0���Q����xR��������%a��Z��������wkwg����o��Z�Jo ���C���w0_�y/�~�����L�����d�hF��{�?�Z�%�`���/��1����1��|M�������y|C�_}���Y���u�^
3�8�|�%��=�{���'���j�i���x��V���[��bh����[T��a`�M�AhSp��t�����	��`N��90��R�k�_@��.��'�Rcz�bM���#����
��f���YoN�H3���e�&��/�%m��4�Q���N�{�`=������&�w!Ro�aG��2���=��=R#\K��7���{���l;�y����P
�Z:����E��Q�3���ri?ze7��O��x��8�n/��)���=T����19����I�:�
���A��S�������v;%4j��4��,5���v�>��
����~��F
��? �p�v����o����BDlQ
�c3��3�P�������8�GFw�>�������#��o�6*h�	�1�UCs��`�M�4jF�H����6p���0��|f��+�FH#IvJ4\�n?��#��C_�{U[X��j'X��e��9��f8[(6M����������%�k!���?�����T���-���da��gqi���uCk�[�����][�n�4P9NT�8�.�{i��9jU�������A�W����I:�%oI�S����_��63*�Y K�
�������f�
��7S�54$]jc~s�B��u�o����H����uX���E�#6�^��^�_�_�\��o��\�����t^�	`+c�'O��=�����8���S`�����0-�g�1o��lz��x>����c���d5��o���C��eF��_��S��H����P��~�?'������9e
�|�������w���^d�g�=��:/�?��m ������S���_KV������0'
�y�\�}�������5F��h�O>��E$��x�X���{\����O��nD�G�O��,�Z��H�2)fg��&%�HQ����Pn��(��R��a�Ep-8�������P��.S�r1"�����{���7Mc�(�����46���X�CSd��Fd�����"�����-��V��L`�C���f��>�FW��W��M�b�����O.6��;�����.���~�W]���2p~���������������{�����&o�t��d��+�|57W����7FO\���V�kNP���,��E�9��m�����2��4;�M�'��������=C��G�h�_��W��I�(��(+��[X������g;_K\��P
�r���I�w�	
��T����5+�=�D�L�������|S���[���u�?yCI V���m�����$�����������o�����F������o��f��w\�'(A<���]����K6��Ny�^�t9���/2���������P�w&���.���r��n���������������������z�����������^��"N��D���F�1�9�9�������
�*b�fH	�;?� Q���WoP��#=�7VW��aw�\�������}����s�-o���?7�VeG�XX�����.������������f����������J�^K�x+���8G���v�c��%���L	�E�Kg�0�2*g���e�_�G�t�/�i��U�-���]������jo��;��f�B��+�B\�*�"
t��K:5R�4��,5��qz]�q��%�pX�x�����E�CXw"�@�|?���'Y�M_S�+8�3 ]�%$EP�:��q:G�����t_��������
�.�O^��83?��������9Tx=m�*��l��8�Q�������M�I�W	��S����`
�AA.��O�|�K�/������4:�>���9���s���W��w�?�ifpr��������R6]K:�X�K����s��z2��U�
�~�Q�2@�7ha8��Z�6
a#=4���s|y����iFF���b���XGc�?3{
������V��{_���SQ��5�v������&�r,���!�!��g��Z����������Q4�����i��'FV�q������������<����e/U��l���p����Y�j]������#����`nVO��&���?�,.�)�Fo�l����=������P/u�������� ^�r�*fi����8��|��v�1���{���}+��LB�Qi����i�����|����t���k���W��*�'v����Gfy����P���f}t-�����s�k{j]*�(�8o 3m��T����. �`��g�g�>{_��>�Q=�>c����`�xT�Y��,Md�7�b���	c���n�{����I����l�}gsk�����PF^��*k������3ud=�<�{ye�^��N�$����3,�9_[z�:����i6�C`+9$l	��bB�r���Xwo����q�;����F����nA;�`�2Y��|qRt���<ZnZ�Eo��N����{��f[�Q�����E`�@�A�q.`Q`|�Q`�dH�:7�r�2(��O��VC�N��~p7�,�\�B����--�5�EnAgf��k3�����M��z)fj�R�h�S����.a�9M�����[��J���>����ip,�����~|k4���||rr|�3�� �T�$��h�L~�������	�+\I��c������t@�x�)�v�e��/fB�O�p`�p-W��A'2�v����y��oGF�4}k?*���P����_~:~���K��8,�u% wCnT>pztv���^a]��l8!2�kt�]��m�k������S�E��9��<��������b=F�:��_�r'�Yk*�o���p6F����]��`k�J�����R� u�S
�H��v����
oB��Ig��C��KXz���3�F�*����HN��	�^bo�b�O������1Fj�?���'��i�4�*;��aWF*��p����J}A�QP��4�-�<� zq�#��$a����B��ooa��F�`���B��Z9�������xw���-,'���VkV��
6�i�oB�x��'C7�D��J�V�!j�Z�vj�J�K�oz������%�)����w�d�}��P=�o�@�
���]/��p����;��S��q��f�n�?���#Is.����r�RJy��1��s��P:������z���]z�[<�����V���<����d�a9N��B�P���
�>���	U�9����,ww���k�Mg��f�Dd�h����~��Km,����v)���WJ�o+N�X�����@������g=����
b"|���Mq���)�����0��=YA��Ei�����w������j�����������S��V�JN`q�u��"R'����]YDUCG�,��D��+)r�w�%��� �	�^=9�������/�Y�'ktM��!�e�������*�W�1�{g�V<����������Z���6���&�|���^�R�O��;#:�v�3��������b�el�9=�QO�1����������5�|�
�4��j�E3N|�Uk�GA���x����E���do�&V9��] �P8�
�]X&���\���U�������z��X�E���ir[�A�g�p�E���a:C�I�w�&}d�7��t��C)�e0.������0�������Sj�R����J4�Vk(�p���\�s��w���m���?��������Y>����[m�d�EC^��'���I���`�a"[]����\YX��`#� ����
��P���v��c�}����'0G�~���|k�Yj��v�1���������|��0�������_�lr2��LU/<	�j�W����"p��6�����W���d��O�<���r��
��t1�-��\0��\��`����M�++��F?��x��t��~�W[�Hod�P�m�k�lm����$�yo�r����#N3�	�s��(H�������!�em��z����L�R:1�|2��(A�6l`����4���+2����%�c^�����J��a� �������@���mh�3M��#�$��f��]�C��:��\*�`3i���IS0�`��K��(�J=�P2��g�����S_����}(��;<#�����9�ekX��6sN�,�g�b�>�St�$s������d�jb�S���a�?.�f��>3�B
��4 .�����/���x������6��l��xL������v�`������$���T3$fz�������>�&������15a��>bv���T�Iz���c�w0w�#���wpI���_q��eTV���=����6d�������OYP��n>\
���rQD_����e�kFhS+%@�����Bh3R��2���-��]o4l�����a�T�J�g��=�����3f�K��F(�.X����5I���+zNx��QH����t����+�5%�!�6���+��x��I��-zF�V]���}����bx_����~�oY�E��n�H�EGl�/��EJ���Tj�=�w�dV���4����E�R�N�
��/���__�����(�L�������OV7��i:��(�c������6
o�`k��m����(�C--�)#Y�'5N�!��}X�Es���CUi0�J�dPqfF�m#��$VH��!��-����&������+���l�!�j���9*��93F�������m���m�������=�7�3����������:.���ds�� ������O���:����$4d`6cW�$�(<|�1��������@����?}O}c�K��z���o�k��������h����e!�1���Wx��d}�fB3�c���U
e0&�?��"�:�rP�o��c[0�.�H��W������:*5�x%��<B�AXo�Z��H0����l���"%����"���E������h��c=��r:K�,�g SZ/�B�W���b�I��:(����<�}t��M~P�f>�H������Y���i�^dT���T���I�1��T�z�1 �_:t��
�m�\��Ptb}��b�$����N�sl�*%�������:�?�ST0���W�-&����[��$	 f�P��\Tp]�3��X�U������N
�z����xPr�%X���\�����O� �\b�R���Z�&��]��L�������f\�f�c]
B��B��\��A	�����(�U*�J5]�����	�-sq;�������Xc�R�,����/@(������2�r���x�+�$��J�%��������r	���xn�kz�/���%������8}n�
ci�b{��Ww�hQ���}���U6���>�`�W��H|���
���v-�Vk �"*t�k�����0�����~]�E�����u�~����YN3��ph ��	F����>I�8��g��U����ODt$��|=c�qD�)�4p(��&_u�������Pp�C.$�n++�%��XA������������E��9<rj���{��eT������1�T����I"q�L�N��w����fD

�2��6���"��si�H����$�RM����{�BI
s/Q4\7t����
E��D`Z*��V�1�UM���"���P0�3�$vw��Adw�XJh���MGGm�h�n
"�;K�����u�
8a���Q��n�*K�"���F��|BFwg����������a?(�aoo�R8��Y�{$U�&c�@�)�wr�Pw�"�����V������&��?��'��#�'����;�J��e�����l$�}�K������&6h���s�?��a����N� �e0���fC���&���4��F�M����^���v"���Wb��y�j��b@�xf�<��9�(���OU��|�����,�b��Y:��kl�Y7fC�3��y��D[�ze��G��/���C�_�%��Hw��S&35��ga.���T����s���J�/�fw������`F��f�D;��;�_�5���z�a@tSo3�6��(L3��H�����1�����XQ�-����v����?���a���O�fUR�n��s�QOV��j����_����������z�)b�p��T���N��cO��������z�p8(@��c�=�[PW�0O5E������(��'S�|�����X��}�C���a��_Pf`P���H@G��(Z-[��}#�a����v���B�UXM�~����t2�E���&�Wg��lx)rsmw4F�������l�S��k��Z:�c%Y}�p�������/��/=��*�vt����k��?���%,�3E���D8z��d�G�;�0rRko��K������r���""2��%D�RA�	.��KF}���9�����_2{
`�	UG��]������kf�+��Ac@��?� g���w�o������
������t�a�:(�M�@�|0K�n@<S��yy�g�]��)��+�70�����%����M����v���(
�U�W�!���T����L3������]	.��:#��� 91��FY�p�;*M��3<l���Y�,J1�#��U�Dm����<�FN������yISN��3K�NH��v��`0�����~-������t�6����%���U	D6��������Pw7�f������J�X{G����a��KxxW����������������]Q�S�������M=��r�mdi��c����X0���J���-��EVB��Oq�m"P��f{w������u�
n�\�%��s�(p�H����JI�i)s�a�������LQ)�r��j%�U!m���H%���8��F��;.fp�}@jU.=U5eb���l�e� �f4H!�l��PA��1���Wp��|���P�,�|l�hW���p7$�@F�6"�A���u����Aw%Zq����K�������a������@$eg�-��f;0��"��s"^�����u��+Yj�GL�� ����o	B�%�M�V�Y�@�A� 	�c�t��|7x�]8�5l�k�#rF��*+'�,�b���/���u�NP��9;��Y�_��"�:m
xprNN%�i����M\)�9��-Q��DVBjT�������G!�G�|�J���K��v�#[r=V�aN�uX���
%��`��;X�&�R�e�l.��x���4��I��v���kv�\`��yK@�P����9���(��-]������zQ�%�9�Z}k$���.�CIb��-?�M��r�7��N��*��C�����s x7��9f�_�tfb���3��=��V$��*�u$��\��_|����.���
��0<es�mY��=s�X�	A�3�
�S�"���$w~��Z-�;+a�������kr�T�ds*�=_��0�+X��F����W/�G�/�S����K�"���%h�u{��HU�0���t,=h�]�0X���i�1�B{W!��|4�Z���x_o67_�2uv]����T �f:��B�R��us��K��:���p��!�F�	���kf�"o���=����fr��{�||e��i��O(���a�5�9DEz��n���l����'wJ�e��E��x���J1�v�]7�wX%��P�_�~[&��*o�p��<��MB0F��]
G���2vk?s�6��b��t��4������2����.������>�1v�\��]_��h���/�i����Y��%?�����.wNn��$�Q����A{g������OZX��'���#iUz��8s(!�y�5������u0`H��a����`��;�8�lI�Y(C�H^{�aX��Pp
��	�@4�9�3X %���l/���7��.�3�)'�v��������8Dp�B����e=
���R���7md�8�P[�N��2s�����"[����H���@$�u2?�#�gW��+2�DP#=�����	:�t��0&���e��!�
�+s���������9L��&�^.�S�cO��7�s�0$;���������Vp��[SJ�A+�ZuZv����h�F�t`@4G�JF9"���.��u�~F:�4�%G��:L�l���+G����j��C�z���p��|����yt�V�`���tw�
�O���A�-���u�Z~���W9��"�P��������ASt0
�p:���	�{�p�mMK`]��P�T*���A�����B�������_(�sq�h�*E/�=�@L� ��Q��a >}�t�)�|
o����e��<-�q�+���,F!*�]�����6Jp.�
��t�����������88+�o
�f���Y�u���\�-��Rs�_�tnn��\{����dh���*:?�+U�����c����}K�1����I&"���k��Rg��|ME��1���rF�e����W{GV�I���'��dV,&����|�h��I����[�B#g�S2��k�\��-�L1�!gK�����F���]+Ji�=F�w4OE�i�(���~#"�@S4A_�N&�]�%�����L���U/c0Hn���Y]C�.-��E��i���a����c;����BF>�un�|j(����tV�&[�d�%���B�,������k;p����>YZ_A,��R?@��������$/P��[�����a ����%��8�J�����k�#+;jQk������Y��rH���s<d��i�����T&+��+a�������5EH��(���1��%��qx]���\7$������������j��Vdd���o z�O����v�����N{{��:�s
��4��aR��_N��[�������C:�o��(F�5��T#����+��SK;�z��*�g��X����m��7��(Z���"^#���m���+q��cM4�}�����v��}��3��9����x�J����}@����1�5J|���)Y�iEF�����E&���J�R�����]z���{}��~@��P�L����$lZ��
l�N
!R`�Y�������C�+V���2RG+~�;��8��_��J����:�/�~<��f>��Qd�	G�0
�<��-����S;X�b�+���������p���Le����mo�~W����/�7N���9:=������9X&[��iF����%���O9^����:�
�Td�u<�g��>4~+��o?�"��N�g��y��;A��mD4��O+L�M#yC���(��C�����"�G�8zG8X�/�����&|��	�ja"�L����w����//z�����Pk'�>w�����Og�;���Mf^F�s
<������[�/�4����E+����F*GD�UW<q���m!�Z��2��b��jN�aVe�r��f�b�z���V���~Z���`�����V��>�n�����������5�W��v���������1�h����_���P��sfr��]Q9^��'�E�Mm���Z"�O_�w1����DiH0��?�OK��v��D�yoR�ilmpB��EK�"�����k���"��y�c���������!��)����C��3p�BP��
�"ZqY�R�Ai;�(v���O�P�3������Z�7E��x+�U������5y��Y���
N���_�?(LS 
�5�_�K�d�S`8���u�G%_���jn�
A��#.�~�i�h���,(���&|�}`���m��u��Y6�%q�=0�T�>h������ItQ�!��|+]v"d��{�����D�/��P�7����N_��.WvL�'�l�����c�C������
@uH��2@�b�%d���/E{0��
����Nq����������]b�M�����$K���V���1�6����9����8=�cG!"u�N&�����	��%u���+V�O���o��^%��5��zY������C��������X���
�;���h-v�|p� B��FkD��4�w
C�Z�T������P�*#a����O9����=�0.���d\(5�5u���D"/YQ�P�%��@`��%U���gk}���~Eq(����5���������*�z)6�~��9�s�b,y�D|��R�a�����<�/�R�s�G���h��Ge�\U^��@Q������'^�P��kUf�V�����f_����P/}=G��k�gT�v�[[P�zo��d�J�^tQM�+��g�,�p����)�����/�pm��#z�Tx����q�u�;����
�p���G(�L.�lnT�%++���i��8�a<2@�k�`�<V5w���UY��Yb�;��%���A`M#|�
,;4�� �zV����3�2N�aG�W�n��b������+�����T�YY ��"XQ�N����a
`'��]������W� <oQ6�]BaSo� [Y�y�d���U7T�.�����
������tF���U��������������!��;E�1A`�^�@��SY$�*���4��5����n@��0����07���	]��X(+�j��0nlI�h��}��N
�`��RG�9�j�v��!	�_K^l&	w����lUl�-[eS�U�p2�P�����hXw]�}u�z�J��_u�X���S���r�#��2��+�N���������z��������B��y >�As�3M#������������"|V�KX�`�'�O���F&�r&[[w��U(BE4VBN���gX��,������l�M�:���:gb���]��X����~���LP�
	i���_���<����	"�����( _�Y��/��|�*q��G�|��&������!W�@Go��
��>��k�]��4�G`o�g7w66�(��������
������H1O�����v[�S��tWI���<L����v2Kh{6zf�0B��f�>M4��o4,T��{[��� B�)q>Y��]g$�}������xj@�P��`�2"��A��l/2�����Jf�^8���E����v�gG�2�	+\�~�� ��c�p�O�qsg{#e�`�F^���E�:��x���5|�Al%�F0�x	���7�lL�+�<5"h�����X��h����r�������k���*c�q��B�T�{���`���<��"b���1��H~���>�u��&+<'S�=QU�I9�d�|j���H<(����<2��f�p2K1w�M+D:x���g�J���������]��v��|;%������X�4m{u����0���0��=��]"����s��^�a�#x�%�,z�����a��S67 i������43)�����F�������-�7�b M1������D�>V��.�.���v���>1�s����`���	~�7.�y�[��|�\�'9����oFt}����'�#,���^���F2C��r2o ��g�ZV������s��q�e%j�#WL���G��/�9�.��}��p�P^|s��2�������~9a�C��d��e#h�;O�ss�`��ce!�Uc��|�IJ�VLb�z3`�/U�x�hx��.1��"��d�mN@E5`b���1�jL��7k�Jx��3x�R9�tdR95k��\C�E�v���&%S�d*�ge>��Sm�����R�}����5�]�|��%��s{>�X]�"�T��oU�A��~��">`�I~����������N<�i����v��
�^�&���bf)���9�L��`+������%��C���^����>T�W�����n����$��*j$:O�#����(�����J^��&lq�6�`�7rsop����)�m�V����=����+��s���Mf�a�7�P������C�[�1=��������v�Y)�].���+�G�� �<���t#O7v�G�M4��^3��=5RI��gu/!�����2�/2V[U��"��J����w��~����E���/�N�A��I����� �������b3|��?H$5L;��X1�u>���66��54�\3�t�67P}b��'����F���h2�������9vG7��*���,��l0�ge�ukk�����]e;��L����+�A��X/�wV�i|����I�-�M��|fw��e����oJ��u�C��P)����
�Ed��tln��F��e8�f���\R�������W���J�)�lB�b*�fK\b"���*�O���{eC�)���FW��&6vQ�T3���w�Gr�F�T�S�\=p��c :��<����"<~K(�hb."���{��}M���v9:����>.J����n���2�p\�������_*g`q}��S���������DH���UAm�!7������<Q����+U�ep�+W����%s�FN�������[���OF	��.��o����cX����=(G��B�v!,� ��Dt1�6vA]h��.�d�>q0
��w�UV��wd+`���f��}3�f�XC��8���>cX����z}PUe��xe]��C�4����.|��C������By�����N�	E�B]�be�T�;s�6���i[b����+`��D��#��
m�����r����c����+/���)�mc}������Y���W��Tu����bt��L�G�E{��gY;R��$Du�������Wn2�p(
�����nv��/��\+$bAq�%��s����Z!����V3�����AD �"w���W��������[�T%@5���tS�u����18E(���O�H9������w�"$�
V�� L���0Q�������-����w��&8��>`����[%7i����p8k;���	k������9� �*U��n���f�^A����������a����e��&}`T���vrvt�K�pP�<fF�����~8�{~5�Nv�k�����Kz����T�+��j�>W�MiA`0*�gK=@?%?����(��h�4��>��&/���"JC������2��=	�@x?.��b<�gju�����(n�-�������V���a�������u�j�\����� �����~v�������M5�{q�T����~8.>�SHX~F�n���rg-�b���n����3���vt��P�v/immlv_!V4"84<B�
��2��,���FF/Y��
��sT.���td�B^pC�� ���"�k�\���%����Ik�T������8������1z6�F^�9$����2Nc	*B�������as�a����<�wfc})�
���6������fcx	�J0#�Ge�#�$��Xy���2�yn4��x��x5-�z���+]�������me��O)�3D�4s(F����wQ%�	������vV�����c�Y�Io=9�qG��R��6:�1��%2:XP�gF����w��g9$���##E�	�!�M���S�<g-�
�-�!�o�3�F�)?!=�
x�.ka�w%����R�	��^2x���_��W����������({ye��
�h�W�����Wp�g���j�-��Y����:2�����a}�tD�\
3[Y�K`n A�w�_}���0��j�5K/R�$��A_&bF���� qH	7�8K���=�]���m^(����[���Zi�8���V�(��%��tbyn��7��e����/f��S[{�<�Z�A�d%�N����V�WX���?hoo��@7�_~��F����L��]z��B��v� q6����+���Vz�WU��n� ���@c�������a�=���T3_l5��z�QE����k��GDge��1����w�%`�����(�'w.�!R�c!a-����c$����F�U����N�l���r�\a�&����b�$���(����Q���������	7�3��:���X���:R��J?,hisx�����E7X��blIU�0\�����o����s�\R'����t\�ji��0p1����q~1�`u�;`����Fi_��73�D��0�?���	
�+~7�� ��(��l���f��u���Q����'~F�w+T_6ss�lo��2��0C98*�Z%���!��\dK ���9a��t�vAfR�8#��C����?��m��XM�#V���9��&w� ��,��f{���[�~V�_���m����.h�L�|$�c�A��	�������0r��X�q�dw����B_�:�����*�4���]��o�B'x?�+��{��k�S�a�b'�chK&��*1�������)��QG��N!���5z#����p#�c�NN(��q�����q
Aa��c��	2�~��*�-H������Q������Rr�2{��S������S���>�N�
�B����.���m�u�4��sRU�cE%�A�"�@������������]X� IN]�)'���)�=J�'�w>�p�a�������F�(������Y�Sr���:9�:��P��CZ����Z�?h���g�bJ��d�{e�Ns!�E6���X�:����>|@���W���<v�e������\Bj�R�M���9_�X����	q
����R��0�����z\�+��Y����#��N~�5l�,\�?�Wj	���&�u�%mFQ�m����E�EM��R��t(�4�%^�RPp�V��lU}v����#�TW�FcnHBfHA��|V"�8gX����f����a�@��19������,�^e����G��>�c�o��_[���P_(k���������_^x����`=��d4�]��W�t�r!�f}�w*�����@�*��,I5�`��_#��7�����|M���"���A!P�������y���h��w����:�b���TM(=�v�
P��
�5���I�hDk�p,��o/Vuj�U]������)0N��d����W����g�����\G��jE�i:..3�5;w)�(iSk	2��(z|�6���
q�g������
aB����V���q��u��/�+������AXF��[c�@
:�|F
����X�*�\����%�NAQ�*��7%�O���X�7c�G��������CY!z�
�m���[�j�b#�t}�"��Wj����X�B#���4�C.o�����H�(+K�Jx�I�B�P
����|zh�C�E���)��gY�$�
�yY���=<�����x[�����A����
���Y�?^��������!�fc�����N�@�I|!��F��y�`r���koy`d�Q2�Q��:2��a��O8)K��2�Jn��7i�*p��H���������{���w��������{)i^���w��� ��q�y�y�|9^
�AS��n����W��H��N�h
�[�)_�����2���d��}���`�nr���Dn�����W���j�a�6z�;6gx�9�x�A8���H��$E�)�F���a���������Cs
��g�O�sC\|~��)���
s~���:8��"+b�4���+��GP���6��^�����R�khk���e��!+reaU������y��]S��w�VU��<{G�l����vb�8E����T
�U70�<P��i�+�
4%[mp|���z��/&�{�/��z�[�e<��?
�~����R�T4�O-�<�im
� 2����jI\�o����d���j��`����������a5��1��7�0�����N|���<�g����/i����N��nX�.HT��j��o�#M�B�V1�&�Q�^�~X����h{���gWR���PIr@�w���=���5�����_�o>�v�O��NO?}<?[�a-���-8��f1��{ ����S5�]��]�9������'��~��Q�^�Y:%���`�)�;��1��������HE��k��`v�hT.2?�4R��|b�g%PWH%E�'�R���"MU|B8A��4��0�����D��o(��;j@\�r0�������m~�}��@5F5p��K�������@����]�]��t0��e�R�E�8\�����*�N����k��@������3U�u~"g�W�w�4���[�����Bm��U]�U�B�K�
�^���w	%�4b���tJ���$6�w4�*����N(�&���U��zI�g��U6�M�0����
��W �?D����D�����������%���.�xV0lA�� 6�2yh�hL��8�Ff�vUa�����E�?`�����/fp�Y4�u��{�IAm0��L���L���� �����`.����B1b!����q����ce������P��5j@8�������8_�U�M��7n��&��[�������{�x����Xv�������Ze+N����{cM�p��I���-�1@W|�D�C�Om����T�M�j��fcZx2�Hj�B��8P9��N���T*H+�c(�qQ����R���;������`��q�M����U���.t�s{m�.4���m�/��:�:���#����|��@�����z�]U$,�<����N�"�/Q�<���mo��	�������[$q�Ck�*e61S5RX�e����V�W��]�}�F��-�����p.���C-�[ �p0��;Pikww�}�or*�8V��}�����������K��Z�%�7����3�V�0���0��lP�����wwBpd���r�i��QV��5��P,{&	^H�(B���:�R1�r
^�B�����lC(��9P;���Q�1����^��@�~�0/�'�J$XW_2/��$�������8>\Q2���Jm�f��HN�u�H,�U2[�
PL(i�����o�q������]�o��Zw����e�"[&���'WR���.Q��eM�(��]�����@������`2��9H#���h�<	��i8�	g�T�
�oO���~x���`��Q��s"(z��8����QN���NQ�0,�C�!��h���j����J��P���c���wa�T��N�>��6����X��`i�U���������
����y��D�,8dBwAr������R���a���q:��>��~�#>�(q6�����eq�F�����Y�X��?������T\u���N�^�2���1�]���D�\�>N�g�yAh�
>�B8�&P�TS	����(/ZS���4���|�(<�.i�����/�t��������I!��L�0�p�#���� �w3'���bZ}�����u_��pz�=>yuz����pv��#w%\0x���5���v�������w�#"�L��x���.8+<����a�A�m}7���?��d�\l|��Q
��tn�]������������sN��FI6(Zt�
&�Jw i�X�%��HE�BJ�m�d!(�nc�E�U8.�M.>�o�pz9�c� ia3�K��@u���Rf�i��J�Z�����-s�L1�=��!�^s]T}��:��=��|��oq�S@����`ZR,;��[qS�����e���A�����������t�V(�YyLH��)����e����;��l�wqS_U�}y�5A+B)�3�� �E������=4�x}��
OR^���(���9�%��������Q��bk	��:�,�C�P�7�Br��Q��E%A��`P���Z��sK&�s{(�Tj�TkJv�U3��O.E����^��E�v"�4`6X���g��5�%)����|�����
pA{LLu+.���[
���n���:�� ����s@	�5�\��@�dp2������.��,'D��tpu�9�$���#5h���������ke��-�qU+��Wx���P+�����2vp������@u�f�����81���0?�����.���~����\�P����~� ������)e"rw�������c(4G���}�M�,k~��g_��v��
�;�j�j�qo�$�o-[[<�Cq��R����]����j>�7'�
~M����_�r�\�����FAg+l�@������s�;Z5��p�9�����Aie���n�e�����=�`cT$�W�����\�$����Cb�nfq����R>����t�������6)V�|
`��\{f"����h��0�����H�G��,! Z�9xiA�8��ej�	��S(*## H����F�+f�&������;�6u{�\o
n�L���/���&��i�J��djW��y>uy�^��s�L)O�3s���I���`����{�qI�����N�7K*�=k��i�����d�j���$*J9I����o��
����bJC���U�
�~�������s
���q��yWK��j`�k'���`@(����o�%[Y�3��I��������a���K�h��JOTk��������|��"N +,c��M�^��1��,��������j��>��E��yuu�����^Q�,��/R�)��,��:�_�~�����6cK�n�c�w�$Z�����!��1��}�fA}���~�]�{�v�p%��bs������jn���#'�1�z>�z�t0����������?�y��8\��*atYh~�9;��fm�[i<-F��8��wP<�Lc��('+���V�TqV��r!��'�9������m����[O �3�~���y$c�����[�^8�3��v#wB�E2-�M)���7����&��gk�h1�T �F.�w����,@��Sp�|���5"���H_������x�����Y(M,����|d$�|p����S������Nz�b�����z0���#��� �nV8$%��=�0D�z%���W(������<����D�sq���]���������"���{�|����+{_|��Y<������+��BB�#��[-*5�
-���'tPD��]=m|(�Cf��]�[�B<-<�����;�	,m:����&�i��"0$p�����A�Hr���jsU��f|�x3(0���vq�d���C�r���7�zI���`	+=F�1��
9c )����WYBz��r3���h���
K�.����xLR��FDhA�TB��(Y�O�F��T�
0�e��@F�)N�����Hr$m��`�S4�D�)���n�l8�|T�}�Q���v�A��������������Y�>��U����g|F��N�_�\s��&� ��,8���VI
������`�l���Q����a>0.�+�kE&�h<�����+o����J�����V^�}��Q�#����`��~|��%� �,;<tQL�V�Z���K�L+����0���G�pr/�>���Nz������'�@�v7��`�N����4���������B��{�[����x'���MjX�K�l��r@�Qdz&Q�l�� e>V:dy��y� �1.��`�4��E:��33��-|�:%k*�0OkY�Yv5�-',R ��������L��F����J�?�T%g��)z�H�'��[N��|����/��D�`����|�T�����U�Z��Y~�w��3�BzH"rj��������7`��[��z���!:����}��W/��fS�d`�rS�Z0�������N�8
6
�Ym����Q	��<������""�xF.����qxvz�'�SYbO��9��,�RQ+a�8Q��
]sIq�1�j��4��JX>�>���-�����T�V����j��m���^,#j��Dr\���L���	�����&���s�nS��l�UF�����mv�8Y{���d���X��Q�|kx�r�0��,�9������rAY�E��C#��^�����@b�t��DHXm�����tZ/�%�H�;wX�����y���E��@��mV1�n���������=��>��l��"�O��kT�z3����������@�����q���|O��6������u�q5
_������^`���&���&n8���|g2�F��}=e�A����Q������e��\�Q�q1�=f����h>����&$�4�u#�7y�����{����of������ts;��zY������s���4���V���������������;�)U���)�@Z��%�{������y��]M����.�y�_���u�5��X~������v���^����}0�� �^�sm��� Eh��:~}tr~�������'�U�$Mf��~	�gg?Yu�������;>{o{kUWTm���]��2tz���xC`�H��0� ����n��v��b8�:��E2��7xq��TL���x9���z��)��5�1(�R��c��z6$�U�Jo12����X���C�yV�� ���`�<>�d+���u���C�(XP��
��B:����ZP�{�w���lt
,z,����o�����K��g��0�i
6��|<�lh���p����*l�9
[�=��������9�.\����b��������|j�h�Z����s�zG8,��Xn�-'���!;){y���+�����8�<Q��8U�B��yr���4Ip1�����etzd���!q(	bq��+��{x����ar��Mr
�����vh}����� U>5�������K��s:���y~� hpw��w�Q(���t�\j�K��
!���@}	�
�w�E�h��@2��U]�N���;�HMb(��+Hi��5���M�C's��(�;�Fo����@��4� 2h!d�n�	V��]�dA�0�%��6�2BvnV�K��,b�qe����_Hz�?K~��p����6M�L�Oe�w���J�]�Rr�P�)���fA��tN�"����m5�E3��xqK��J��Tab�P�&y�R����LR�����h)��0}��t�(�}e>@^���"�gI�A��b�\w����v����
7���[���-��	0|�]Y�4>
f[�;���M'bU�0
�d�zp���f����y��]�C#�����n/s���O��o�������Q�����''�'?�_��%�N��x; �C����gH�������	����J�dw�7�O��^��J�#U.��NN�����z@��7�;C�r���'Xn�,Q��O��Na��QM3����lJK����^��%�E3�t��_WtR���P+V��t�<�L ��E��m���Y�DS���^]�m6;�m��RLC�k�_������q��[x<�q�1|�d�+���3`��!vix�4��xCG������:^�Qc@-$�w^��������;'�[#B�zqD����7�d��+@ �~B"4�����y�����F4�^��T��4$����lBH�(�r(I_J���3�!�oF.(������G)-u$�X!�\H���:t^�h�-"��hrA� �4�+0�7qD�����0y���1�4�"���b�H$H�"�$����$G������<I�������E�a�I�	-$��c'7�;z{��O&�N�q�f�r��!N�����V���Z
Ef������Rq����1�:�k�j������n%�TR|X
.t�	T)n���.�t���#P�����H�OI���R�*�1�^��B�t��������^�UM^�;zqj����XE5#�������)�:b�U}�>zu��zX�&����Q���17z<|HOa%_w)����>PDh?�����yF��$.�/f'�C��@�'2)&���)�$!X)�<������H�G����t�n��	�~��z/�_����/������>G��gd�P���-���
�`��a���9c���=$3M�)�$���:g�m���>Ltsk�`�~����Reg]�B�����!�~t��q[�9�����<�B�I�38�3����@�D�$��B[�O�����5�T��qm��AL~�
>�:h�%���'���zo�c
Z��2C�X���lPi�H�R�deZw�_��E��������$8|�*o�d�6�lM�����x?xNM)���+\����;d+���RA�H�d�j����(F|��z6�\�&�x�3L����w����NWq������*�fB�Y����n/7"���-��X�70���3j+R�v����,��Q��D���D���[W�\��,�v[O�rl���J������ZyZ�v_�����z��x�	�i�$[a�c�
{������X����z�v������������Z�����#tE�����G�<��[�5."�L�{~�k�����*/�Fk��g��m�R�h���7�:x����\���XO�rS6�H���A�5��@�F��%��5��N1KP�����@�����!A:��d��"��9^�6���
�=�$�
��i��cq-<���$+�P����b�2�k?/��
�����|k)c���v�5�r�X�W7���@XD��������J� s���w�V?��J��x��=��v���dR������@��J���'��A

��(�+�����-z��-���YaF�����*4R�j�A?'���:#�9P��8�x��	�a�K&<
�e�����0h������H��eM�����R#}����
����n�D(���Z��?D�P�S�
b�l�����%��Q�u�_	-�}�u�,1#=9:zm��3����<Z��DA}����P'����,��:���������4�������9W����y������F�ON�	������W=f�A���M�%$y���WA���QJF$�T%�@�9�6~�o_t�����C��kO�c*�S���2��F>��qkC����W&�������-���V���g�^��Wq��4�r���Eg�������8���E��p���,�����N>�8>������H �p|�>�^��DtU�������&�a����c@�C�>������[{K�K�KA��F��?B���W*��_��cR��	�f�����}f����r@�Qt�v9<�RS��(�G����*OW5P�[���D��_��)�I�H�����|�f���������l�����.�I�U��%��c�����Bk���LDO��x�.9{��L��,k��O��5{�} '�K�$- ���t����N�e�A�8�' ����C:���Q$?�#���A{ow	�������|8$C�5���g�$���f�/2���?�
%�5�~w�Y������t��6:rR�R��0���!R�)����vY�h8�v���%?��Ra����kf�b�XDt��eK[��=P������$a�RZ��mz������j�T�z���^�"	��LE�W��+#y��������������M��A�F!��j,�-�d����M#p4�&���
(I�4R&��)H������o���r�nt[~@���J��
��"��V��Y��"�-Et�Z���pN�]n�NXP�m��=��lfI���)O1'&���QB0K��"�I
�]�LxU�9�y/���z��:�\��)tJ���g1B�6^�{��.�>�i����T�'U���i�,���ai�����SQ�����L)C�r������]LQ~�zo����_�~q����R��PX���[�b���l�_���`T���6v]~�������3��QY��cE���Jx�KWW���"ammo�������n^H���I!~6����T9UG�
�v���e�W�v=��h�d�i��]����o2�v��\(�qe`����~���=�����<�_��,#"���g�����	�MC�1���W��[iK���Z�����l]�<�.�'�&�2m�Gn1�z\����eI���.$���J���FP���Y��h`G�C�Z���,�,5�p6���%�>��,Cq��|Q�c��E�6���<�,�m���6�������A�%�<�9��l������h	*!���O;�����/��@q��8����������i��tg�K_���HJq�������m�"(���kp�4*�N�(�>�u�7�R�`i~�Y��p��b>��5�#��>V���� ��3������pR$��M��f�M�"z������k�]����^A�#}��+�������xp��2��i6M�F����k�v:L?8sHAF�Nl�X��.�9m�8y�L�z']b�2����L\��"r�X�(��A����a�Gc�C�i�GR���F{k+i�l��JA��M�3r�T�&cm������
�D�o!U M�(+���A�H���P�$����dX�L��w-�k����W8��h^+�{�8���a5!������:�^����8�(�U��H��Io�S�)�gf�B�'����b(���#+'�*����d���G�A�'����u���Ap7�!L5�`F�Z!Oj����l\Y��u��!�����s�CI-�y��|FE6��#0���FI��`���$��k�������N�.b�$0V��B�,^J��J2���7�Y�
/5���gZq�QS��}�/P|�K^�T������#PW���?%o�HE�����K5�?�^w�-��|����^{s�;bs{=9�C������V���0xz�����tG��|��i��N����J� ��*�%��E���bp5���!Ew�17����-�	>{8�u���L3��hj"$�m��lz�f_tof�8���i���0��Rx�0h��.�bc��hG	>j�D��Pm����&kk����n�w��������/fS��)r����h;��$�7i�����s(�F�#�fU`��.��cW �Am	��c�Y~��:���}
,)�9��t*Dg��,�dQ���m�������u���h	�F�A���"[+���V�|���g@`~���� 9rs3*v��#�wn�������N�nB����#��?�c���r��hJ���,���Mj'�������/���s�n�����ztz��TT��l:���q��n����Uk#L���j�>�7xY��
��A�>�e�
B?%?������R�gV�z'm���7x�M���e9s������*�6��,��IB�P���hKX%�������������L_U����u%6�����������g�3��9nY��������S&���y���!���j���
��X��r�g�-S����;����4�US�o�S���ECX�P4�l\	[�j,H���]_��-QK�*e�s�Gs��*[������j(wC�����d��U�a!M�������F���*e9VI������/dN~�h��6;��u~�a7r1�.���8��Bk�g.��Xk�����R	���eCZ~�fZq]7����"����0��}����L�Z��U3JA��������e�W<=wd� \h�'\��~�&������?T����6��6�6��[��fg�ry�H�*��hZ�M�`T-��k;����H	@����t�������$��HFP�+`H3���<���������opg���q!�1�lU<�������YGX���-�+�uP�J�w3kEr�I.�m��u��xx���SGW����[�>)��h;�f����`=G���"��.��v{s9�]�Qf��1��"F����<��+d���z�eZ�:�D��M�1�����m�c����6i�������,~���M(�1�x������(���K4j#�8�"�Q���T�@A>	��i�If08$�.�tN���y�fx���<�L�����6���F{swg9�v'������$�q��5":����|��o-2byI�1����q_��X�+`�bG�f�@����i~�v�YnC���*F?�m�3���'~�������jF����yW��L��C����}�~�G���3^7�:��%\s��}��Y��,�/���U�7z��R���z�����``v���x�9���&p��r�/�mi�����l���re�����{����p=S���p���I�@m��<L�]������E��x!��7��*f�������%�>�O����������H�\y&��+^,M�������7�l�i}�{�F����BMZ����O��Y����|�n��(�fW�f���]�p��*�:��7�?�	dI���gvDY�{/.�V�W9�H�k������6�#�_�_1��X 	��1r�E�y-H9v�Y<`@�HV�������������Dj������������A���=�A��7TLk��4�n����������4���.%���
P��L��-%E�Z���R�wxd-��R���;���tU:����y���,�1;�w����n�����[��c-Nf���H� wQ�M�Y�8�+]��v}�������=B�{�N��7�W�
YdV>�
�4���hs�-��/T�
�q��������O/@������I?�xw��";)�h�/"3��`�q�}��R�C:��5F��?�U(��sl���;Ls���@��p>��&be�~�P�S��MF���"#@.������"�=�$C�oh8Cd�2V
1;xs����)S�;���h��("�������A$H��:r�{M2��KO��G���v�s&��7�N�Ni��7wwyO�9{���������w��w@�}s��_�_���\R
��'B��$�W�!uo��!�9��si��D��xo�0L[(J )���.�x��LG��$�E6m��D��l(iZ�v*k��G��'�k�j��8<�OV�l�P�N�dY�M�r��6o������c-C��*c��#��>�9�Oa�p}���3'���E�&�x5�(R��G
�m:O���E-������>�`�I�_z��1 Y��N:���>tp��-����z����6��}�3N�|��pgs.�(3/�_�������,�T�Y��F�u�o4�]]:��Lf7��[<����	��""�S������?V���E��ws�F�X#�n�8�;�j��
���W�}9���������/����i���h8_|���a"_Z�\2�Uk~�:(��Z�0���R�>(
*������9��u��X� :�Mw}���/z+�:����lh�=����w��l������YOA�."��t>]?�2*tg��M8,#YN a�`��B�hg���mo�,��gM�)�9�4Z�V�/��B��l�a���6��E(m
�*|�T�	�_<�PN��B���l�xp^��CY�����w�7�3�dpi���=\|rss�2\����}�������Bq���zO�e���a2�)!k6�r��f}�,�FA���1�5��/
2�&)���]�"��EZ������ /]��>Kv*Lu*�t*w��p?�8������Y_ h���t�#��F�?1���d���?4���P�4��j��,6����N19�uY@5Y>5y.F�5����p��S���W����xg���������;����w|�������`�o�����m���}(��BtToe�o�wA�W`��w�:���{�|W�a�5_:���O/z7��?S&I����1�p�]@�������y����-���v����;*���*��%\�r9_�(_9�Rk��+�A�"|gh�	����Sjt�P	�XN}��Q����=�u�i�M+��]N�����h
	M�-p���������z����X����dd��<��~9���\(-��lH����m��.��g���d��>�y}N�'�_�#�����m���i�Y�D���H<���o
3T����9�a���J�"���3�L��l�d<}~����m�=��+N�"(n���n>�C�z	D�?�E�� :|v��02�<M��Z���$�)�V$e���}/6��*�@�HzX���H%�o��JGi+%�'W�$K��	��*��BY�,��d���Y��0��C�hG�eAN�$Dy�������)�����P �����
;H���7������|�3����b�0��uM�����T��(D�Tc�58����K]r���>`G%�N����=u��p�4`�XL8�>��0������J;u083�"��A�]���1���vQ����;M�i��|�S�z~����'I�A�"���'c~Fr�h����jw�t�D���W��>l����{B��e
�!X=2)
�Q��	��F;/�z	����FS�#��B�a�n�u���#_�����
��?�4��P���5�3��s`�+#/��K�:�,�*������0��w�T�Y7�-�����^���b����'#� U5�����E��aLEV�9��VW��QS����7��l����n?4�������Z��������������\L~G��h�v���}d�N~���|%�
;�%Fz�(z!�����}���g�Yao��9��^Ym}�@|�9��E��O��L�J�ET��v�f��zI�XUjl���D�j)��J%x%������/`�\�1�����C��-��l���H��������99=\�_P	 6�v������jOG>#�$
,�|��;E�r�W���j����lg�>�!`�l�(��&GpJ)F�$������l���H�_���PS�&�-c���q{\��-_�����s-7��&����FEr0k�U\5_���Tv�8�'��50��o!c��=����������W����?��|4^�4�PM
]�'�	��&!��Y8�0M�N:�7�Et=Bz��%�2oV����a=����������?�D����-2f���( G|�QdoOh]��P��jT�m�X��S�1�a�������Y��T�{�K��D��� ��[@���*�}O�������x�5K����v�����SD����OWzK�%${���V����_l3��a�����F���?��4VK�����9G�����q4��R,5x(�g�D���@��-l��v�;:b��K�=n�?�� A����m!c��ch���D�F�O��z���tZ[�s,sT##|�T��[�h�`	=����oJk���}i'��u�F�����`�c������rp���J42��w�=�p�om���#FKyZ(g���s���UD�E1���gVc��g�Z���O�lk�V����m����V*
�j�Y�
���Q�V���������m��L�hN-�t�����HW�&e������
����t����w���|�KyI/����u,�k��?��,�SR<G#�%��>�vLs{$��W]���gj���5�J�N�I��6;���[Y�7�3l��`:����I��j�Rr�b��Q�^��������PL8f����P���!����yR%`?����1J����wl�{_�-���Cu��-�������3o���
3LQoT����%�	�Yi~us�&�����yA�c&m4�������s�
l@Q4���V�������#la���"��qSGl��d���Q,��*T�m{��83�m2�65[��d-�����]
N��(��r|��x|_��B�0>�1��/li�b4����.���@����#�^��u�]wK���������f�Z��!�@G2��s���>���O%�������j��j���z�A�_@OXi�w�r�����������<�\+��
��H��a����sp�d:Y}B5�����0� ��[�='��������q�{2`,�K�f�_c��T�HmOl��F��;�Z�K�\B��;��f��������8�i��P��&�t��j�2A��F���Q�����>�F8�p�Ky�� r1�F��/>�����Z������Qm}��(I�Tv�����8�d�������:������Z+_�?��&���h'���b�d��<-��{4�BC��u,h����[��������ZbB3U ����V}$o,�d����
�W@T�L��`v��5�h�
0����u� �`@LcmB�$K�4�[;�AI�~k�2���t*4`�����AC��w���t�"!���]5������V��L�w�3fk^p2��A��d}����Mrj��aj�s��k��\>gD�����C�IWo'���?	���i�AaxiI'X%�����-�9�dVb�	��Q�R�"Uu�{��=�'�=o**s.}��4?�>f�#'
_/Rn�
����_���D2���������8�Ys$���Aq��w:a�>�j$��N���}��h��y�TI6�T��:�$����
)��0@+��%���,{�p���I���!-Vd�l*,��/��
�8���0��7EE_�R.5}��>����YD��WeW���E�������=o�F�����Wf�
-'�U�U�^������d{f�����]�|<����$YI{��C�>����6��]��W�ON��<���/�N�rQ�{$�a�YX������T�^\�;�i:�q��{��l	��e.���r�������
�l��1��\t����|���S�Ma��S����7z@��~^ �,p����(sc�265e��i��8Jc�h��������<�4`;�}X�Gl��?�.7�5����U������/��F���+C����F�f�G�6J�*����~3fZ�����
]���bw�o�&��
��C@���5����o���Z��a����x�j�O.��s$7nj]e&fn��RU������P��d��-T��nD�����^X�v��h���XD���|V�������E�����I����4c�7>:�/��U?�b��W���K_5�z�f��Q����K�J��7�������ax��VL��� �$�����S�|�
����w�P�!W���U����5K��"f|�'FR��L)�f?d�W�����>�>�e����E�����������|��������������#������r
�w�� ���F�m���������wZ�=a��O��D��8�S�$��C����
3�����A�9U��#}�����}���X����	�GJ�}�����y��Fg5�f��8j�'��P��F:]T!��Noiz����
���^��a��'O�:��$�m��B���S%���'?\�v��q�	���!R�/�A0�jA�0���j�Zn�R�E+)/]�8	h��5�7�}s9������?Ys�����{{�}$�����>/o�vO�:���o�w����������o���A�w�r�0���O�!��>���`���� �
�1:���^_w����5%����]r��r�5�
O{��}�S\�[�;y��<�l�8�nG�����9�l0��=�\������o/`H����Lq�$�)���;����F��o��E�{�
,�R��7J�$Y�L��L���$+�$Y�L��I�?�&:]r����
q&�[�����FV"��c�(������ l�zG��G_a��1����d�P�a���,k��8�����ay4��pp���YG6���,
�q=�V��;�k�4�q������~��p�����HG����m�������5r�h8�z��q,?�G�OP
�����9�wr��vW����'���e����v�0��T��g�Q����X��{*�U����6B��L��zx���&�M�?x�X)2�=P��!��������I8<zXl/V@��V��A�P�����`0��2%	BI�IT���*��7�DBC��@sO��zJ���G��G�+_~d�����e���������q9����f�\!�y���i�l������:����P���`�"P����|���$��������}Ds�j.���|/����
i���6-�h��]���n�����2�LU�BV�{�j���8�?������b��|)hc�]��g���^o���2?�I��iy��q������0��R04Z�`X(4��Z�R�����3WJ�;���"*�����X(�����^�{Z�:�,(�qcy�-^�@x���R^:x��c�z����'�.&�$�3�Rx���x%�D�+�����`~T��$�I����HQ/<�|��+�Yu4h6����
�f�Zk���q9�-��w���
���-�P��'(d��� N��^���e2vv��S�n���J��-����,�l����
��,��>bNs�u�$�Ryb�U�(���(7#Z��-�a0I�:��G�`~�a�9���W�c8�$
�`���r����`�`�� ��0�s�U��R]����(�,��e(7��i ����� 
s��w�m����`|��^��>i���A	�����x��1y��;4OG� �X �0���N��4^���%�E+��p�Q�����')��A�T�B0|7�}����.h��S���8Rv��2u�Ps�����?
��hP/V���C�g�J.UW-v8$4�:��F�m�0�I�BR�>X�a9��K9�6�T�h!�9��;�[X��r^�F�6q2������|��9�l��A!%�������&�2��s�|�N�h>�I+��1���1��5�	
}�,����8lK���&��Li�0���[���Z�L���SxJ&���)5����@�I���R�:������&�*��1V@4�J�������e=a^9$�3�O����������K���q>vr9� �r,�����+{��w��|S�m�w�i����39Ji�(��B^tk���8a[����Os�B�Q�>!�>�y�H>q�����RFHf�������O�!�c�as�sjm�
���q��6��#Oq���@kV��5��Xzd9m��Z����Y��������]��[h4��b4�����`B/D�5DxPf��_��K���d�+�=U7��-	MVb�5���!p���b>)��=�@-t��N��v,��5�&"�F&)�j�(_
�[�SYA�, ���i�8���>�`�:�AMmg�'����Jt]9	Cq�y��,u�k����a@���r�����+���`��Jr�&�:�����*(�'��;��s�����NN����$"��3S����=��/^u����k��y��sSI��!
LFFN�����d�D)������<XLDk��<���?�p+G(2��R�2~��d��E�k�&�eIkn�b�Y-$r����B"me*�_QJ�S���}�f����J] ��yU�=�?M���:�����9�bI�>"����KE�|Z!����,���k���&'"�I9 �����q��gw�Sl|���L�"�f�n�H�B���_v���������#��I��W������'���T��b�em�T&c`�V�W��M��{xl7_�u����7Bx����q�D����IB��A��������<��YO��w�U�1���6��D;�d��:���n%Y|���[�P8|�����u�Fv'�Z���C�������VI�"���{�dl�N�|J��xL��yJ���*��eR�@��h��L�JB�a#��Z�����h@F����F>��O��~�!gG�3$R<�D��q�������tl������B�E�����(��;
�{�0d"�K?�7��&��r7�?8e��G=��$4J��r7�rqo�==;�����������i�IN{�z��0G^u&�"�&N*3hr,y�QX���'�=�\~�����x�V��G�����B���w��z��Uj��2��C�#8��LY9/y�bE��Z�x��/Of,?��>u���u�@���%����v����t1;�iy&k���X�W��"�,��GbIq�1��d#������%u97	+9s�����x	�S&O��8+��HQ^��F�$���R�-!��Q�P&�8Y���2�f3����JN	�I�R��*�A�XLW�b��F�p�~��OP����Y8]R���_���N��������j�1�d����q�1�vS@^"�A�8#��,bx����!q�:(���� X\��S�
�AP�=�D�N!�6�C��r��V
w�{�,�	X�?��9A�CsBC"fG��u8�_���%���K�j�����G������Tf�[n)�&:��]������t����<R�����\Rh)W��vl��)>������ �'�@B3�
~�6�{s �xc>X��L�V<�0��F�g@�"��y~+�.�E�IXF��f%��6���Rx
��+WK���,:z`�]�,�\�.A��A|k]mn������T�nw��{��9tV6
�������M;V��=��TJ�PB����n��:��%D������25��M��M��������	"G]%����-�O�����%�X+��1mw�C|E�5�.g�2
h&y�cB������>�G��]�����AaC#�38�	1X���n/��k�&>�83�&l
��q&��Tsj��sCc%����
4���8�����
	�zxR���������M$�:���*5�
�����.8i�X�~U�D���;�E�;Y1F'(�y'b.B�2�e,�yO��jD�g�:O�h@�O���\��h[���!����T�a��k��U�*��I+���l�F����eo���@�{P�,`j�EQ����%����[�
L����Pl���cv_����-A�~��<qwP0j)e���M���
�Ki������fj:nW�5WB���4�t���YP��hB����R �q-Q�?:���0Z�����U��+>PD���MV���0�5fe���d����[@!�y��P�8v���*���N
�{��#�����2`p$m�	��������q��Q-�j(�y�Pr�ox�PvUP��X����=��J�UE��D�Px�%��O��R:���D=JB�E���
�]\i*�a�Cw�Nt��!99����W'U�0].����Tq��f�`������������lo'�/8|�O��W%n%�����P�I(:�[*9	�g{���J���U����5i�I ��`����FO_n�g�����������Ew������M��������*���H�~i��L��l�R�r\�c%QHCRD��]�����Q����ka�_Sv���i�g%�2Gn�F�{	��d���4:���I�pB��t�$SC�bk�q�1*J�B�^���R�K[�&Y�2\��Z�HIf���a�)����i���R"��c���F�V�w����~q(��[�j��}��m~1o����Q�C�"���&����h�������u�?��-��4�!d�K�\�_M�l������A�'�����35���u�i�I;SEo������}O�jb���|����	����}����=�y�>G4���p%b[����{$��	����4HG���u;����/�a����
�7�y���X� �<�-H�M_j��+c<Ke��Nv}��E�2�I��.�t��L�������@��a���
��z����Q��?C�n�����l]��WY!��gx�U}�
��?�����	_��/�J*����3&:e�����z��\���TD�"K�c�b\�(>�P�n��x%��w���h�q��O�
�9_S�:O(��9����F��/NT�t��"��h����Gq�vh������O��$��Ixz��O���\�X-�z�a�',!�E�D����Y�@���=��0���A����Na���B&����/W�
�hU�x�B�M����L�+}�6��M�*
�j0h��
DjJ�IJI���#)'Z�rB�����N T'�zQ��r����BU��!D��4'{/�����e��>L��9��*-��j9yxHx�g8�'*���r��|��\,/�w���
���Zl��G�`�[.gc<o������x��e���<:�|5n������s�������+4j	i��#{K.>�����^�^��}�Ro�zr=_u�4�L�A���������q�[UB9��U9E��q��)�I_����
���b��0��rj���K��8.M�=��d�����.aYa��}r	��}�����58*��BG�q�?`2WN�z�q6�$�%����*W��%�4(W�z�Q(�J�A}T��`�d�)�If��P�F0���ds�����L�}�� �JG� �BN#�r�.�O|I*w���H1���i=��N����������0��^�	xd"��H���������{Z2�4��i0�N���w�	�����l�����P0�L~�7�0�HT�'_�f,���U��|�M���t[[��y�Pag4Z��L�B#�F����x�{��������(b�5l��}C��D.�y��;zr����3'������b*7��x��x$1drM"+W�(�����;Mv2�WY0�W+���1<�B���{I��,���������x`@x_���
���Z��L��Z`�������=���������f�q!c1����5q!�,gQQ�S����)���AeX��Y(���?V���&�eJ�ZVv�cxO���,�-��S���*G���I�p@i�V����Z�F�Yn�j��2H^=����d�������N3e��~���\.	hc�=
t���
1�(�t���R�X!������yf6��Na��wuz�KF��>������$^,<Y����_���Ph������l2���<��B����h*��))<���w���N�������	`D���i��K�\,�2*O�!�E�V�"���%?�9?���w��f�	�#0�Z�Zn6�m��K>/p�m��"��)�0k����g��������'�]��`	�a�@)/D�72k;`7��o07��?��3���Kl�K�c���j0����(!\�O�O��I��`_�G<{���IyVj���6^l�W�~H��LSE���T�6;�W�����3��J��9��3�Z�����V�'!�b��U��~l!�Ln�T�c���+��jk4h���0��Q�5,;r�f��sV5f�����c�y�����jV�����mk��r�a^��#=]�����a�o+)B�T���8/��j��,�W�Q}�E?(6����2L�M 9���T�R�����4@�Y(w�s����w7��X"��:������}gn�T��m�����X��B���|Q��\e�#�]���\���#�.�
�

@gfi9���YZu����i�Y��"A�U7'T"m$��	&}��1Xb|/��4�@��~�/��q��j�A�ZJJ��D6�K�RTw���!��
v\� ��B�^T��'�@+������4�v����7Pr�~��,<�^��}����6����#������I�H1BJ&�mB�y���|�9b<v��0^r���2���RZ�auP)9��Z�1,���nO�L������dty/��v��o��v^8o���������C9)��=��s	������n��U0{_�?�_{�U(���T���:��!�F����2������^x�*=�kD��������0d<>��$�R~
���0�8`.�ZT��%gz��{����?��'.��'�<�Y��������?���� ��a������x��i����]���K�9��hdk?�`�+5'���V;�$���_�^{U�P��lxQZ�O�9��U������������x�����X���s������I����C��,����A�T�ep������H����Z�~�K�M_$^�X�x�7O{��|�����]���>k%]��o�����/g�/��tD��^[0:������h�>w�����������]�^�^�~�L�����V�������e��M�c��Z/�)h� �Bv�>���/�gQ�B-j4>��%���=�
�UT�;��M�5:���#�^��%�4[�?���0�ko���?��B����A��)U�4*�����Q���TkU��Z��J0����OZuF��K�Un��0)PqJ������e�
�N�{��k>�%|����x��?t�������_���������/(�~	���~�������~����7?<�`
hj-test-queries.sqlapplication/octet-stream; name=hj-test-queries.sqlDownload
hj-skew.sqlapplication/octet-stream; name=hj-skew.sqlDownload
hj-skew-unmatched.sqlapplication/octet-stream; name=hj-skew-unmatched.sqlDownload
hj-skew-overflow.sqlapplication/octet-stream; name=hj-skew-overflow.sqlDownload
#2Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#1)
Re: Parallel Hash take II

Hi,

On 2017-07-26 20:12:56 +1200, Thomas Munro wrote:

Here is a new version of my parallel-aware hash join patchset.

Yay!

Working on reviewing this. Will send separate emails for individual
patch reviews.

2. Simplified costing. There is now just one control knob
"parallel_synchronization_cost", which I charge for each time the
participants will wait for each other at a barrier, to be set high
enough to dissuade the planner from using Parallel Hash for tiny hash
tables that would be faster in a parallel-oblivious hash join.
Earlier ideas about modelling the cost of shared memory access didn't
work out.

Hm. You say, "didn't work out" - could you expand a bit on that? I'm
quite doubtful that justaccounting for barriers will be good enough.

I'll report on performance separately.

Looking forward to that ;)

Greetings,

Andres Freund

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#2)
Re: Parallel Hash take II

On Tue, Aug 1, 2017 at 9:28 AM, Andres Freund <andres@anarazel.de> wrote:

On 2017-07-26 20:12:56 +1200, Thomas Munro wrote:

2. Simplified costing. There is now just one control knob
"parallel_synchronization_cost", which I charge for each time the
participants will wait for each other at a barrier, to be set high
enough to dissuade the planner from using Parallel Hash for tiny hash
tables that would be faster in a parallel-oblivious hash join.
Earlier ideas about modelling the cost of shared memory access didn't
work out.

Hm. You say, "didn't work out" - could you expand a bit on that? I'm
quite doubtful that justaccounting for barriers will be good enough.

The earlier approach and some variants I played with were based on the
idea that we should try to estimate the cost of using shared memory.
But there's no precedent for costing the cache hierarchy beyond disk
vs memory, and it depends so much on your hardware (NUMA vs UMA) and
the data distribution. I have no doubt that variations in memory
access costs are important (for example, it was data distribution that
determined whether big-cache-oblivious-shared-hash-table or
MonetDB-style cache-aware approach won in that paper I've mentioned
here before[1]http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.225.3495), but it seems like a hard problem and I didn't feel
like it was necessary. Or do you have a different idea here?

Another point is that in the earlier versions I was trying to teach
the planner how to choose among Hash, Shared Hash and Parallel Shared
Hash. The difference in costing between Hash and Shared Hash (one
worker builds, all workers probe) was important and sensitive, because
the only difference between them would be the cost of memory sharing.
When I dropped Shared Hash from the patch set, it no longer seemed
necessary to try to deal with such subtle costing, because Hash and
Parallel Hash (now without the word 'Shared') already have wildly
different costs: the latter is divided over N CPUs. So I felt I could
get away with a much blunter instrument: just something to avoid
parallel build overheads for tiny tables like TPC-H "nation".

I still wanted something that makes intuitive sense and that could be
set using experimental evidence though. Parallel_synchronization_cost
is an estimate of how long the average backend will have to wait for
the last backend to complete the phase and arrive at each barrier.
The most interesting case is the build phase: how long will the the
last backend make us wait before probing can begin? Well, that
depends on the parallel grain. Currently, the ultimate source of all
parallelism in our executor is Parallel Seq Scan and Parallel Index
Scan, and they hand out a page at a time. Of course, any number of
nodes may sit between the hash join and the scan, and one of them
might include a function that sleeps for 100 years in one backend or
performs a join that generates wildly different numbers of tuples in
each backend. I don't know what to do about that, other than to
assume we have perfectly spherical cows and reason on the basis of an
expected parallel grain reaching us from the scans.

One thing to note about parallel_synchronization_cost is that the cost
units, where 1 is traditionally the cost of scanning a page, actually
make *some* kind of sense here, though it's a bit tenuous: the last
worker to complete is the one that scans the final pages, while the
others see the scan finished. What's really wanted here is not simply
page scanning cost but rather a percentage of the total cost that
represents how much extra work the lanterne rouge of backends has to
do.

Two relevant projects here are:

1. David Rowley proposes changing the seq scan grain[2]/messages/by-id/CAKJS1f-XhfQ2-=85wgYo5b3WtEs=ys=2Rsq=NuvnmaV4ZsM1XQ@mail.gmail.com, perhaps
adaptively. I suppose as this number increases the time at which two
workers finish can vary more greatly.
2. The parallel-append project introduces a completely different type
of granularity based on unrelated and separately costed subplans
rather than pages. Perhaps there are things that could be done here
to model the fact that some workers might finish a long time before
others, but I don't know.

Perhaps what parallel hash really needs is not a user-controlled
parallel_synchronization_cost, but some number produced by the planner
to describe the expected distribution of tuple counts over workers.
Armed with something like that and the cost per tuple you might be
able to estimate how long we expect hash join barriers to make you
wait without introducing any new GUCs at all. I thought about some of
these things a bit but it seemed like a big research project of its
own and I was persuaded in an off-list discussion by Robert to try to
find the simplest thing that would avoid parallel-aware hash for
little tables that are already built very cheaply.

[1]: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.225.3495
[2]: /messages/by-id/CAKJS1f-XhfQ2-=85wgYo5b3WtEs=ys=2Rsq=NuvnmaV4ZsM1XQ@mail.gmail.com

--
Thomas Munro
http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#1)
Re: Parallel Hash take II

From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Wed 26 Jul 2017 19:58:20 NZST
Subject: [PATCH] Add support for parallel-aware hash joins.

Hi,

WRT the main patch:

- Echoing concerns from other threads (Robert: ping): I'm doubtful that
it makes sense to size the number of parallel workers solely based on
the parallel scan node's size. I don't think it's this patch's job to
change that, but to me it seriously amplifys that - I'd bet there's a
lot of cases with nontrivial joins where the benefit from parallelism
on the join level is bigger than on the scan level itself. And the
number of rows in the upper nodes might also be bigger than on the
scan node level, making it more important to have higher number of
nodes.

- If I understand the code in initial_cost_hashjoin() correctly, we
count the synchronization overhead once, independent of the number of
workers. But on the other hand we calculate the throughput by
dividing by the number of workers. Do you think that's right?

- I haven't really grokked the deadlock issue you address. Could you
expand the comments on that? Possibly somewhere central referenced by
the various parts.

- maybe I'm overly paranoid, but it might not be bad to add some extra
checks for ExecReScanHashJoin ensuring that it doesn't get called when
workers are still doing something.

- seems like you're dereffing tuple unnecessarily here:

+	/*
+	 * If we detached a chain of tuples, transfer them to the main hash table
+	 * or batch storage.
+	 */
+	if (regainable_space > 0)
+	{
+		HashJoinTuple tuple;
+
+		tuple = (HashJoinTuple)
+			dsa_get_address(hashtable->area, detached_chain_shared);
+		ExecHashTransferSkewTuples(hashtable, detached_chain,
+								   detached_chain_shared);
+
+		/* Remove from the total space used. */
+		LWLockAcquire(&hashtable->shared->chunk_lock, LW_EXCLUSIVE);
+		Assert(hashtable->shared->size >= regainable_space);
+		hashtable->shared->size -= regainable_space;
+		LWLockRelease(&hashtable->shared->chunk_lock);
+
+		/*
+		 * If the bucket we removed is the same as the bucket the caller just
+		 * overflowed, then we can forget about the overflowing part of the
+		 * tuple.  It's been moved out of the skew hash table.  Otherwise, the
+		 * caller will call again; eventually we'll either succeed in
+		 * allocating space for the overflow or reach this case.
+		 */
+		if (bucket_to_remove == bucketno)
+		{
+			hashtable->spaceUsedSkew = 0;
+			hashtable->spaceAllowedSkew = 0;
+		}
+	}
- The names here could probably improved some:
+		case WAIT_EVENT_HASH_SHRINKING1:
+			event_name = "Hash/Shrinking1";
+			break;
+		case WAIT_EVENT_HASH_SHRINKING2:
+			event_name = "Hash/Shrinking2";
+			break;
+		case WAIT_EVENT_HASH_SHRINKING3:
+			event_name = "Hash/Shrinking3";
+			break;
+		case WAIT_EVENT_HASH_SHRINKING4:
+			event_name = "Hash/Shrinking4";

- why are we restricting rows_total bit to parallel aware?

+	/*
+	 * If parallel-aware, the executor will also need an estimate of the total
+	 * number of rows expected from all participants so that it can size the
+	 * shared hash table.
+	 */
+	if (best_path->jpath.path.parallel_aware)
+	{
+		hash_plan->plan.parallel_aware = true;
+		hash_plan->rows_total = best_path->inner_rows_total;
+	}
+

- seems we need a few more test - I don't think the existing tests are
properly going to exercise the skew stuff, multiple batches, etc?
This is nontrivial code, I'd really like to see a high test coverage
of the new code.

- might not hurt to reindent before the final submission

- Unsurprisingly, please implement the FIXME ;)

Regards,

Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#5Robert Haas
robertmhaas@gmail.com
In reply to: Andres Freund (#4)
Re: Parallel Hash take II

On Mon, Jul 31, 2017 at 9:11 PM, Andres Freund <andres@anarazel.de> wrote:

- Echoing concerns from other threads (Robert: ping): I'm doubtful that
it makes sense to size the number of parallel workers solely based on
the parallel scan node's size. I don't think it's this patch's job to
change that, but to me it seriously amplifys that - I'd bet there's a
lot of cases with nontrivial joins where the benefit from parallelism
on the join level is bigger than on the scan level itself. And the
number of rows in the upper nodes might also be bigger than on the
scan node level, making it more important to have higher number of
nodes.

Well, I feel like a broken record here but ... yeah, I agree we need
to improve that. It's probably generally true that the more parallel
operators we add, the more potential benefit there is in doing
something about that problem. But, like you say, not in this patch.

/messages/by-id/CA+TgmoYL-SQZ2gRL2DpenAzOBd5+SW30QB=A4CseWtOgejz4aQ@mail.gmail.com

I think we could improve things significantly by generating multiple
partial paths with different number of parallel workers, instead of
just picking a number of workers based on the table size and going
with it. For that to work, though, you'd need something built into
the costing to discourage picking paths with too many workers. And
you'd need to be OK with planning taking a lot longer when parallelism
is involved, because you'd be carrying around more paths for longer.
There are other problems to solve, too.

I still think, though, that it's highly worthwhile to get at least a
few more parallel operators - and this one in particular - done before
we attack that problem in earnest. Even with a dumb calculation of
the number of workers, this helps a lot.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#4)
1 attachment(s)
Re: Parallel Hash take II

On Tue, Aug 1, 2017 at 1:11 PM, Andres Freund <andres@anarazel.de> wrote:

WRT the main patch:

Thanks for the review. I will respond soon, but for now I just wanted
to post a rebased version (no changes) because v16 no longer applies.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

parallel-hash-v17.patchset.tgzapplication/x-gzip; name=parallel-hash-v17.patchset.tgzDownload
#7Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Thomas Munro (#6)
1 attachment(s)
Re: Parallel Hash take II

On Wed, Aug 2, 2017 at 10:06 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Tue, Aug 1, 2017 at 1:11 PM, Andres Freund <andres@anarazel.de> wrote:

WRT the main patch:

Thanks for the review. I will respond soon, but for now I just wanted
to post a rebased version (no changes) because v16 no longer applies.

Rebased with better commit messages. Sorry for the changed patch
names, I switched to using git-format properly... (I'll be posting a
new version with some bigger changes to the 0010 patch and some
answers to good questions you've asked soon.)

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

parallel-hash-v18.patchset.tgzapplication/x-gzip; name=parallel-hash-v18.patchset.tgzDownload
����Y�}iw�F��|�~Eu�%��E�;v�,���������{rx@�(!&%�;~�����P�(Jv<=���X\�r����	��\��WNtU�n
K'�^E2���/��l6����}��lw����N���u{�VG4[�N�����F��g�NCY��0Xs�6���^������,��F�~�7?�_�	���\���4X���_����j�����w�~�[���A��J����h~���|�_��B��i��Nz9v�iOG�vw6q;�^w�Lf�^g&��8�K��f�	�'��Y�����
N$��R��c��A����e��H���4X<�>pb���jnou	��Ck��3z�m�j o��&?�i�D������k�
���O� ���Rh�	Z
���r{�M�z�[_I!?z}�<=
\)����c0�E$��z^�\I[�S��t|W�a���Cg�/����V�"�R6�8����������g2��Tn{|L���VD8y51Y���E�	/W��8�U�xW����zp�d�M#�OXL{��.E��{�(����A��{��������y|~��C�}[~�q������x�)p��+��J:.���Y�2��<:_�p��o�W���RJJ�`R���_����IT�#&^�S��2���y1�#5	�oz%n��V��sp����5�"X1�v�
}!l�#:}�F�c��[v�i��>�����n����]�������=�
a0/C����/����=Wq�����.�(�2�]������Y��g����?N/��x/�����~'�#�����������7W�����\8��qI�������?���}�z�=��rv��M'����r�i���t�������n������"
��@d���������]�k8"���U�����n�0<�	<<nx�WOtDnl��7�%�^9��tk���
�d�{U����p�\�����6%Q�_�8�w�|r�������m5�Qw:����F{�tC�3qz�E�����m�Jl����OQou:�V����f5��C�o��?�1���������f��K}����do�#��g���8X"����[��?
%P������@�y�-��[���S}(�~��h�L�;$9�P.�0��/��=�;:{'{�����x���Q%�UG+�f4�I.VKS=;��s���V:v������,w?��W���  a����P7>�b�����E�%N'�;��S������G��y,O����j�h� v�4�����t/����6B��[x���.������&������t)C$��� ��R.OV�y��������H/�xLX�lV��"��j�f?wop����U�������jIx}�7�P��'�TM����<�@�:F�,V���O!�eP��|����p�gV��;x�NWHL�~��� �{qc�����Z=Q��������� ����sX���pT~L;^C%�����Wj�����j��V��3�l4i���gH���>��L����p�%��(<��0��,��Y��n4����z�r�F����s�[�"�k�l@B�1����9m�Pf�40�Q,��d ��7��]������Q��3����wo���=<{}�w ������;>zuR9'�|��H5	9_�����O��&���8����6�����XQ63<�,'1�D����d%h���z��m�������������_�?�?�Q
��+��^A� 0A\;`����.\��3���~���<������z�SuA�9J4�!�/X��5�A����})%�^�,������d�Q�E<���d[/�����F�B4�� ��z�3=����`�*��o�_on�<-^���'��,,:w�;�������
��D����zC�h����
Du�VRF�$�"vB�	p" ��A��p����kg�����Yt�&M�_�����~��5��|��F�Y��s�&���B���*��+D��U�.����G����@�p��9"2��Q]����O@T�"#DE)��c� ��e3�.os����8�~�i�y.����]4�����#&`�$ 3��[�V�lM���nM�H.������F���$`oF�����+=5#X
R I��H���������L�,.�c��"�Y���[:��;6�NI�H^+�?M?4.���'S_Y����Z4�QsP���#R��p������VVg������AAUx�%28]�jk� ��9S��nk)�;��S#X��j\�JD]z.?�=�������;�W1�����n
�L������Icm�G�KZ�n��j���LF�i~C��D��[�\l��0��w|�|��E��P�����U$��HO��u���,XL�pX�n�Et�
��}		k�GzC?�c������W�����>�����������\	gxL�Xz��9����'��>���,��4�����+~�H|����K����6��"	��c�Y	+i��������P��P$3w��QgB-��M�^k�����������aL��B�l(c�]X81b�%HT�-����^R�w�-�GJ!^�),�����������2Y�d��:�X����IK/FIH��o ��,��r"1A���RQ�iZ��j����P�K���E
�,��-<7W�II�o�r��o?���e�w89��:���;�av�v��;�&N��v����9����u�����������[�P��K���@��4Ga{�{���E��?�C������T:m`M�f�{[Up���[�
�N���!a+�?�����`n����i�]
��EK?$�^..�#�����c��������)�y|X-i$Y�������e�����<�i��N�62|$^�@YFqXp���"��[[	���J��C9]�������m5������fa�P�-�j	C2����u�1<��f�H���c<� o�
d8�%��<��X
����l��hu����n���?_�g}�G��%�
C@�:�!u2�����I��F����h��l�G������5~(�����=9�4��ak�k���h����h:i��3���W���u�t�e�m�� �,aq�=4:�c;���,��1�8�,?^9�,�	t���%|�D@'LB����O�,��0�A^��v[�.��[�z,T�����7eb�\ 	Da���:&b2����c�:�^�@c[:>��@�{B���!��@��7$�
�~6����`E44�>���kM'�UxCw�� �K����Sqx������q��Gl�9Y-&2D5���]���%��s�Z�H��<��
j�z<m���9����]�,V�,
$[��q[)kR��!�����
�sb���7Gh���[�����R�mPE�('�+�D�����(vO�?]9�\v� "��e���p���,��;���K&W�XM�G��$)��;00����3�m��������C����������a���dE!<�eBd������������(���F���U��h+�o�J�3�����s"U�*<�#�����g������C��'�7Fe>0�yzq��� ����\����0�e?H]c���l���Z:��)��+��NDkV�N�NJ��|��k��9$�4{3�����o}�D'(����-{��f�����=/���}2�Ss�`�m*V�0�L��i���p�m�'��7�	��>�fb��#��w��� �8��+@�!��r  �!�wHY��wZ{*�n��[O�mx����?�����] '���/2h�?���5���b�k�b�����������G�Z��\K�v���b3��)F�����q?����s�k�=.m�9k���=�[@��|jBy��;{�$M������� X"RO��:�T����gkk������/O���)�gg��^�WT����G����I/���R���(kd�8�}�mLM3{����7��k��C���A���$!��6*�����	�7����r��?g��H�tp�L���@Lh��i|�6������y8Y���lB��p������3q�������AI���G)B�����S�_����zp�(w�l�b����8�V���Dt���������:H�U���?[�d%5��T1������^����8n�c���	0�!���oN�P1�������Pu�w�h�;����68��6BAZ���&W���%FL�l�_����S;j�I���i�s%�:-��'�k����U���Q��0Q�E=�{���N���N/.s%v���VIISa������oQI���Rk�5:��D�]�{<���w�e��)��$��h��S�7��X�QV���bOH�����I�����8�����s	
�/�A�5�`�����2��M��5�JC=�"���8#��������`�|K����H�,�f����=�ms���Z��dd�{��'6�l�aw�t��#T�	����Uq$q�a>����wA������V�Q�^	�{�q��K�4!H�J���������[�"�
g�Q�8#���3��s��!GGH��zS�_��}��.������m,zJ?�3�<hZ���c1�~�I�%�
	>#<�	A���V�X�b��X�����3��p�Q��U��54�3<���@{�n���]�o�N��o��%�~e�m��#ZmR��^9!���q�4�K����Z������D���s����n������!�"r*��hL�.��/�-�*p��p�x�H��B�L��G:#DOc�!��
����n�8���Z���H��$Va����1����0����6�C�df��l����@���8s	k.���/Z�'�>jG�nYdF���m"�k�l��0�]�-�ti
�|��P�{k��%_�%�w.X���q��$���)�TI��{h��$�-��->Z������^L��W����z�;L<aJ������#���4��FGl}t������'��*�j���V�?�{�|T)�T{����h�G��~@���~���*�2~�1B���K�]��s'����Y��*9y8�����lq'�ock���
z'�I�\F0�@�8-�(���~��������}������_������� ��|eR����8V`�K�����:m��2�_�~������C�_�aw�����5:�i����9��Is�:�~��i������=�����:��hI1N-uT��*3�>���M*
��m{�K���_�(�px@o���U�;[�>��('S}r��oWM�+�R������#}�����7�-W�I�u��2�<2~����8��e�x4��Lh�cwm���o(�pm(W:�k��6=�WP�f��F}�Z_#��
�j4$|���������iu��s\�=m�{��q%��w�T��j�\��v��v�1J���B�p�ifxzn�)��3����0�fP(���(�~��B6�O�k����	u�J��`���<�z���=�����+i[@�N��E�&������X�|~L�+�x��fE]������E��)�_M���{q�l�Qr�j��GR� 3����,�x�X�Y���+�T20h������b��?Oo�I��^��(���Tg(����L�?Dv`�N��r�h���I�~�6������t��P�M'y�}Nz0��R��c9�O���@K*[�=f,y�Y�-o�4O�Wy����e,z��Q\�
QTx��9�UQl����H�|}�T��:���.4��/P[F�����]d��
�Q!�R��q�Sqbv1
\�@��%BW����$tv-�[[���X������]���c���7���61n�OF�����E��2{�r
��>�d�P~{�9=���'�dF=��+��.�=����������^`�E��L������J�8yGK�V������zn���g<�{�
2Eh?2;�B�?������e�`)�Pd�������������	-���b�+Y�G��
F�FD���7�0��"Al���R��y�H�Y���Q�]�	z�}5����\N?�c
���)��~��8��Q�$��t�.�q9w|���1���
;�n��=����|�S���
���{��y�d]�@�":�"-k(���kk���?�j2L���[ca�HYMT�RTS&�LY,J�NX�YJ`t�XM��f_"a&�_q�A�;�;�$����g�����&��i���X$����-s��Kd�}3eq����'�2N��?q@C���]W�AeL�N�A&_EV�<��1�����h�w�����~�����������	���!��r�\�r���l��)����b��<o�����`0X���:��F��Gr4!�G\Ux��_T�H�����+Vj*�i�$����(���Th�����f'>� �\I/�u.��8*[��KEg��f��=.f�j�I�������ff42fg9�����FV<v��p�g��b��v�}K�Z���A������fW���	�B��v�I���C�!�������YA��*�0��k��	:w�1��2����$��������?�j�;L��v���h�$G��. ��I��ga@�2��<��"�=7$�5'�%!��[[[���zTN�+j={��8=:�������z�O����u�>q���a}b9
�Z�K(���W�8z��F�@X�����������w����������[M��G��`�w�aGZ�v�o��]�5���_���n=iu���e��.�-��?��0h�t�FKa�r{�����B�D�o}g�MU���>8�'�|�22�
)��qT!$�o�g��&�����P*���C����w���0@G������������=&A�s~����
��y8���yS���]*��pF�f�sOY$~p�\(���$t�[�����X��n������� �������7�����f:�����c�����N�������C�]�����]o9�}�|��`�����e�*D!�O�����C�[r��4�+�?��nK��;Y�O�9����g���p��;o����^�3j���4#w�N���h�&��X��Sx�~al��3�����0����m���x���.��h����u���?���~e�rL�9Fd����@��b�-��F��_x����99E�\D����]x�a/�[��_�UFoo����m��fHax��� �q��E}������G����
����@[[���h��F{6a�)�BF%�MW���
z)G�!�u�������@��M�+��y����������-iP�����_�~����
X�4��`������6��Kq����h�F�:����dO���X�"��Z��%6{��`��R�j�%�i��~FPn����=��9fX�I?e������<�B0��"����a/!x���cv�S@7��4�(zFE�'<mpm<Pq�}L"E\PS$J�Qp1����
�pt�����/����a�*X_r����S��y�^|�Pq�[��K�C�(�rS�;��(��X`/�:_0��	{}���LP�5����3�-��
�0��M��kl��%����$p�s��*J���������q~#�����^c2���� z�'�9� �F� �bV����>&$��@$���|�P���`z�9+�Qe��(b�}8 ���1;��,�k���g�Q�n�&^b��JM84��=m,���0��"i��f���=Ac��\=^��S�A
��Yy+�x��j�h9��8�7yl{�K>e�D����)v.s��{sZ�n�j	�[����
X�#B����?"�5�5��d
�B0�a��<��.}aa�^$0������"���
���������B���3����0�?�����i���J��@2=������N��� ���g)Z�P:t"D�����jR�sD
t��&A�8P�fj��'�z�-��
�q��-�Y��'J�9��cxG$F:e��h,�`r��m�`�}����w
z
���=���C���A>T�L�Xj�TnR�m�L�5�Q������M�
tB�"������h�:����>Q�Qr�MU�U��U�5����J�T/���(H�8F�%�M��Z�����:p���E�kE�A�"h��JM	��S�h�K���3�����5i<�Z-4,�z��	i�y7���OD��,\F&� �����vz�8�������@)�C3�cXNb"��L�V�)�
!p&�MMQx:,�w^`P������������0���P.�Ck�h��h����x����)�	>�`�k-�v�-9Q>���<,�$B	~��`E�<�X��@DTp��$��x`��jf�o�J&�����g����j��oQ��
=�c.��|"&������x���ZJ�6%S��`'�	v���!`]9s@HM ~���Y��t�yJ6��B���F�)��M�nlp���(.0m�YQ14.���Wc�=�Wx��F&C�O.�^��]��(}�nER=�EUG���Q �a{H�W�������]-j����f����t�>��7�4R����
���"^�:�0@=�kT^x�"�c��:�x�`���zNil^�ED����?�w7�:�_�� ��U�g��,�G���g�&��|�����=����|��U����Z�g)X����u�9$y^�.�
��K�����?R��������'3~L��30���=k����#�^r{J*�4$b]$�*��J��H��tV������H�qDBb���1���J"����+�cK�1h������I���R�W�~
����xqM���U�����TI��(������h���X��x�����Jfv��7	�-��/Yp���sL{���JR�0 ��/_&����S���6
"��I��[�,E��ir��s���H#b���z�yJ���y�����$��>�S\�yP�H��Lv��MS����<�a�7�����(b�i������u������P�nLb�Ug�O`�����9|��j5{�u��<=x&Z�A���� �������$^L��	�R���L��;4j�fdzi��������������}��j�8�����!�{%1��$��)j�D��j!h�9�"$��	�	V4�)5�SD�T��c��4�`A�,�s@*Z��iF�I&�;����q���\�=�l�9 �B�7��\�`e�!l *��AM���5]��m
J� 8�jdH�J
�B��O��R0Rk��u<��CNc5��hD�� yAb{S����W���8�F��P���*��P��Y�����\�2�8�
V
��b��:����{���^m������&N��*�7${��42��^�������w�$�@9�Z�
������c�*���SqL��l������ J����������Pp��|{9�qVd���M��	/4jk�������e��IzZ$���Y++�������NKWO�k��|��������R��.59��~{E���#�E�2� ����4�`�[��^��o�(!�����^
8JK.�%U��&�f6�����1���\��mm��%��������,���fX�M1X3���4S.�*krG�k)���"�&��������O���Pp��c�3����)��OX��������Xs+�?d`)Cl�S�n�VLn��h�9�oI�:��Q��'��m�^E���V�*����
?9����>��"rS�mJT�P�c�����1 ���J����?�O�@vk����$���|g	���^��k��]�r��S��/I���.i��U���t�p���e&9��	u��FJ���H��O����p����7Q���A���z�!*�o��S���<:��kt��E��L��f>����;;;:<�N�'Z�����>U{P��!�=� J|�xkb9W-�iZ���u�`�M#��MV����|��5��zH���WQ$�2�kZPdp�3�=�o����@f�L*��E�!y�)���tZD]��e6��F�"R�Z"��H�T��n0�]��K��F=L��lU	�~��^��2��t	;.�w-����L
!0����C�(�����*f��"���i��uX��a�Z�����Y�i�ke�?�[��������O]9Lg�d8u���k������?��Y���z�v�����;�'�^Y����u(�FKa�R����F�m�?o1@��s�q8�&�����i�Z/� ,�@�����o�!Ld|��L����5g�l[�=����O�Y�n��^����R���'!r������"�#�657�/<���X��[�:����B�w'�`S+UA9���d���r������s�)�#�K���4���)I?�5T����h��]��@��&0���1|F��z6���le1�l��e2X���/ak(���om����d`x6�+��N�Q���
��P4��]w(���ufSw�G��t6��F������E���EwQ2(w��r����#P�W��i�����{����g�~����O��g��v�JaP��@;��\aN���/� U��K�2��~F*&g�6�e�@2��*�w�����D��9�W|&*+�k�T2�R8�/":BcC�V���"��@�=X��?�j��u	�G�t�S�m�S�z�io0i����i�/�TK<���n�~)��{E�2M�M��"���o�d��{eC�IJ&S��Ub;�����	��r@{����d��"����.rg��:$AqUb��)�xKsP
���%	�%,I�#f|Jq2�Ge\�_)�B}|#�K��
��f'I���-�Uc2�B���S�L��B[���BkQ9
�c��)`�U:1Z��'��6~.(L�����7�}E��}*���C� ��iJ;p9pk���P�'�$��c���"�Pa����m1��pG���\"�%�����<Q��Z�Hf���2�����Gi9�-9��,-��9i��s���Xo�_�F�;��M���O�T�&�P��Wd�����Y�0g��I��dC��]}���x�����AU�,Of(��@�`z����R��U���h��k� %8M��HqN�Q�� tu b&NM�@�N�?*��;=����#�����1q��~�[{x�3;&�i��rD�v�$�xp�21>�SU��`t����Q���jy��
n����y_YAp)vR��&�>_���x,�D��s�����z)a]����S�k^d	e�����Y�������D�5�tu
�4.��c�tI�{������TN��*c,�KbTk9����u��Yj�cZ��h�K*����o���V>���lyc�(�b��n5��(*.��;98��
<X0`�O{����j&9����Q&�Q�o��������+)�~��w����Y�)��A�q�E�2ES���U�W�T��8���V��e�l3�!�c;��(���=���G�F���m����Y2(��T)C

�S��*3�tt1 T*T��X�2c�D��j�	���M��d��-D&��b��Svl�%>F�Ea�K� �a(&�%;�{JQ��B���[�/�A����4�BT~\	2
�u]-��w�o���zR�2*��so��S �'���l���^$���c:���'����gx�4G�-�`����,j:S�dL���xz����s3�����L�\�V�p�J1����������k������R���m�4k�42�]Y`��(YE
�C<� 5�\��A�Z*�Om�h�nC���<r$>��O����t�H9�ZZ����K���sg�W����B��*N��r�F�1Yg�y���H���NL�C��T��?e���f�J-�n�1����rJ�[s>�;y���"�R�
%zou��U|[���W��=A��$p�$E�l�W�
�VY4���7��'�fv��'����c��$`6���
����s��I�yj�d|��m�����n�41Z��'"1Sk6}�7d���69�bQ�������v���ud��t��G)z�e6��v�R����l5�k�������.������t$�_ITg�!I�k���������4��.d���%xv��Q@H2�\L�]������T�L���s.�"�I�+�V������8Y���7�H^����AK��>�$��P��/���iw�������:?���N������V4�ak4��Z�No$g�Y�mO�����~��=�>����T�|@K��%���6".����!��l�����Vl�&.ECb�*��{�G��K@#O���J6G+����"K�
��:>�}6q,�=[`m����a�L.!�� A��D��i5^��W��vg(��A����
�A"���5����0����zc*t�C;0�p���a/�x�,��7�X��n����}�,�����+B�s�s/�Pz��z��������~r�Q�S��6|�\�b?�J�W9�����(�PY���m-)�Y��Z�+�Mw<b/��f��W�j�G�wp�
,��g�b_Y�Q�����n?�����\���4�K��4���f-��'�F����f��v6h;�����\�NT�����/�zN�rFz���������;���������bQ��1���K�Mn
�x���QM)����s@��4���)����=�,y-��*�?��?B1x�����_h+0O$��D�U��>-��_������~��DbNC��p.���XM=��s1�3�mU��,B�����*���������W&����u�v�\a
�}�p�����Y?����d��@�J���,yo��
������"��D�~��U�����o���?�����b.U4�����2O�����0lo��~[4������/�@*G�{=W6S+��'��$i��a��P���3�XoXG��qhf6��Eu�v���G���#��8�_�����!OSU+ ������h��$ZOp�;�Rc�_��n�����gZ�Bh������z�2���T��&�����f�.�I�4%xj���(�zX5`��~%
�
M�w�������Am���RSX�)lN���@
�����6����S�?7������2��cX`�/m}k-�,���g��%�`�X�F^����;������z�E���Q�&�Q���'t��x!"�D��Fv}�Y������q�^%�F�^�+�r�n4����������sP������"��U��>t��TDEA�/�|�%��6E�@J!�;�y�P��2��"�����JeT[�*c{K�n?%����'5�1�]-+��Fv8�Qc	�z�9���*U��,C��?�4��w�fX�_'�?��m��PO��T�dy��mc����h��d����4�����5������O;�V��lx��;�������4:�5���H.�c���O5�$���Dl69k�����@������{#�����P�)�S#"�2����Rm7�X��LX���6j���>	bF�H*����J����sO���gg�L��� 8��X�\�b�d�Q�{���I���e����9Q�uk�nS�r���F�:=9�nk���y��N�����P��e��P�s�������E�S��W@&��6�/����-���{���P&�4a�tH:��6�^s�7#
�������oJ�7�/E"���5�4s���r0q:����5�I{�t&Cw���Ld��A(��#��Cj�L�����T����L�
���V��)��P�{	��*�*�Xz��JP���/(���A��hl��)
�c�y[��X��Q�����-l�5���I�����x��V��m�no�i6���;5����fj��X��������!}�J�W<L����!��%���?;���y��I��kp���NMo���)H��F�9i�&��+�kZ�����$7����C��y�y����=D��Y)�Q�����qsF�qB����u>������#���crs��)�9�9�4�81Y��:�#�S9�������N�����Xb�!����� �w{����31��B��#��������)�q��X�!�Y�=���mT����r
1����G0T���5]����p��������@&ws�����V���\J����c/���;��� ��Cg�4G��~3�y<Y?��g�_���M[���[�Yk�.|s�y�~�-O7{�����%�7�����G�_7VO�>'��/�.���}}���|��#�7�*�f�=��Nk:�w��u@��:�I��������v���������bw�[������T�<������r������H��V������p�X�������}�+3�f�S7��I����������n���[k8��w [�f�S�/�0�2��+3��{x#k����f�f?��h�3��n����l3V�������������R�l����������A���K�����
�d4(�T�����ppD�rT�+SF��kL\kll(�S�;����+���H�]�y)^��?=9���0kC�JaT���Nw %P�Q����v������M6&Q���*XW�?�G�Q�-�hW�$o��{���}i��������R��
�nno���}��iC�}L���+aQ2r'�����e71��;�RE�"Gv��� $��kP�����l�F��+��F��J�luz�+���8������sFee�(0s��>N"�����q����{�`x��ec�����9n?�>T��1m I�����$��|n����������\���b5{��e=�{��\,���s���,�Y��	�Z�t�g��jw~���?�����z3w������u��)���=����ik����l#
����'�����,�s@����B�%��$�����Xa���+uD���d5���S����4 ��tT&kJ�^��V&G,(\/������^S���
�Y�T!g~��r��U�=*�En
3&���J��V�� �a���V���R�J�W���wR���okb�wC�^�r�����������s�m���l��U��{��8
_��W��W�^�8n��|���w�|%O�����+�R+���M��W���*U��;Z\W#);��:��m��V2��7io��iJ���i4�f�3u�=P��N%��~��)�����_Ru�FS����hU3�.�;VY�����U�t�.9`�R�����)3��.���J%�]�����w���=�c����O�ij,�OE��?r8?�O�s�����n@����i=�����hF#\���E:�C��x�/������c|JT������K�Z�I�^�%��-�����<X/Z��v]�8rq�$U�-�Z�LT()0Y\3g� p�*�V
���T����6�RV���&�N�V�"e[MF�u��-�]N{�����o��o���1��8�)��r"+����/���a��c���/���JA�\OV"�%aL�2f�������n�+��9��!�^��M+��$�09���a�����p������eYS�EZE{&N���E8��-
(�t����=F��
����j����t�<R�W
�We#�R���qY���3/c����\%�����thu��z��j�|A� �1LA�p����^-�?��e��PM���-��}C�>��%���n<pOY����I������%\A�l���h���X~�t���J��{?~��o�G�{���G���G?�}'R�]�+�U��<IkT�\#��Z�EZ�I���Mr�4;��R�4[��mu��,���b5���N���fy��{���R�$�w�iz���q�h	���([����#� ����I��<	�6XffZ��6��U��
�V7�)R����5���ET�=��p�{�v�
���8��%��ns�-��5�d��:��[��`
g���Y���N����vv��hJv�J�X���x�O���!��-���}����Kh�"I�Y;Z����JE)��� ���|��`��z�7t�9�E�JM�S5-�����
_�c��D����>���P�����������O}_�	P�Y7d%y�y%2/�aV�z���������/�c��T55��F��b���E1�s&��D����B�S�YL�Sk��k�Zo���\�nHFLM���'�k1�N�d]���O�U���jw��������}������i����N���������'���8�Y�5���d�i��4K�?C��Z&��[�(�-�>��r[wD��U����C�)�J5?��R%d�\(1v23��	���4�������5fjS`r~}����^>� ~!�Ev�s;�(
��_�g�j�B��7m���>s��cY��7������\035��������:
�.�sx/v��=���1"}�9�%�+���{>��Oa�i���k����G�z������R����<c9�[������h����SM�k��v�t�*�Vfu�N� $MS�V��'����w|u��#��*�~z�4_X����A���A�jr�8��6�uP<���G��*�Q5�&)(�4j���'�Q��p�"^�3�#�/b5w� 
�5���:yT%n����j*���UD=\M9����\Y�������8B{'���D�����T����Yg������
��64��?��]�>>�|q������6�~�r��?;���p�z����pxs�W�,����//N�{����W�����_t������������=������6Y*%��� �Q��4Y{�.�W�Q#�nr���Z3WDm�Mv	��A����S3�.�:���*���^a&�f����<�~�����������g�����Fk8�Sw�u[kR"��i3!��
�:����Z���M��pU��&?N�P���l�A���{/&`��@����j{����,p��A"�X��G7���:D�H8�� �@9��}u�;�D�Y �%~H-����9�7C���+�~q�\����(`�������N���m���9�q��hF���,�Y&�&� ��%FtLg
�382vn��;�����b=���Z����J�.44�W����`�,��\8���B�#��� �T��4�������Ux��Q��'���T,[FI�/�	l������]���3.w��Lo'���r#�����,`!�h7}��rN������R����
�q\����%z02m�Z����e�����;�q���g{`a�VZ��4����zm'�j�8]���������_�Q����[w����?��}G��T���5��0����Ia��D^��~2�1�I�%��Yz�8��s�c��)���������UN��0K�����,����������1������|W����S����]���m!��;���n����U���S������������7S?��������Lz�@%�i�-�(�4��z����(L�{��yK=��X]W���;l�����b������C
���sa_��E��<>|�pw��l�o�7�o^���4��z9��`<D�QB	
DD��r�J������RT���@-PYJY�#Q����)�4@���bw3���,Jfv��M��j�9D���Y����U��o|��M�����g�(�n)�l�O���?�.�O�9E�c�E�	�.,�7_E��������c[X��x�9��i�A^-��q��t��2�����h��~��.g����
�KZv|-���K���o��y+�c^��o5ul��Z+~���@�������]���A�+��B����0�f�L�8�=���+8���X�������FT~���#
Tmu��������Q�t;u���8p�B�]�������;,��F����I�3*�a���9�\�*��]����'��g}�^(���k����V������Zk�i(D.r�!
U��g���F��U5:�>���-����.�&%�"������Y�����f�-��Q-��>�
K��G��3�����4k"U�0�
��5NmtRi�W�{�S�T��c<�\N�|� ��sT-�e�r��I����@������evj����R�d���e�W�+e%%���_��Z�$��u[�\�>��s%nI�/@�wU�d Y�%Z�0jG����GI�a���J�
�g��
�|AM�)�j��;j������_q-�_��{w|q�"�������)��Ew�:>}�w����"�A��Y��e8ZM|�!���F����W���7oY�����;��>9�G����+G-l�.���������_t�qz�7}�8>�[�f�R�
O�Q����;$�������������;����o)\��a���5H�+�1����`T��|�h[�u����t�\��9���yD�%����&��T���I5u)!�����s����K�pYB�VS,�t[R4+�3�@����wK��E�ta�/,�����Srw-�K�5wo�7E[%�i|����
��tN,J��	���r�b���"=*�C���Q�k�amm��E{uc�����z"�5���#%�	��D[&������{
��bk���rxvvzVC5{
b��)m���_�<�.�$������\�-O)��(t�82I�#H���T�02��V�V�)M��&a/&g��e�1���2���]+����IQ��2v�
������#F��wy1��hpz������� �Y�t��qDNf��M��~e���R%?�h�����v����PV�7{o�hB��k9��_�/�0�W@a�VnIGo�`7�V�]e��xNB�i�n�L�kj��-k��j�U���v�t�o$�����������,[l�I+��zC#f�<�"c�A=-S��Iw��(���#g&��
/��?H5p��51�d���h�~D.�������\!��r����Ce�����	u�{���m�y�4��7 ����1�pq��6�?���J���T<���'�O��TT���!�x?Q'�����O�t�U��ZXMX��	�c<��x�S��i����4��n��q�8Z��*�'j
�d�v�K�c�����@m��P�1<:eR���w�V� �YHn�\ (6�V�2�|b�T�4��i�"���������cuK�0���E"��2'�����{6y7�9,_����B^��t��5d�N�M1M�4�y��L3cs�w�PM�1oGTE����>�<q�^�c-���\�NWx���0-��%h���G��G�����T�nj�B�AB�L��uSZ+���V���*X`��������eL!q���9���)��)r�=�f�n��C�3��3��DOKL����V<PoM����k��\�<\e��JQF*g��*���^m8(�4�8��6`�h|�{���`�&u���,�"9m���p�KUjAS�	hIQT%N�'�p�^��7����`_��O7�-�0�lx�-�`�������Dj�Be�!M	����������Yd�h�����l+���i��7W�`��3_�(�5�B���q��*��d��}i6�w4�	�s+������u��8$+D�>AS�������"
/��U��$���(������J�t$p�
Ha9�r����6�������k"��(OB$�� �e������Qc�UR����E�@sq|Km������5,h��}��R��wVDMI��e.%�0c�~s�6���P^1��H�@%���������)�y����p�W��K0�w�?]A�M����etb�����!i��5��e��m��*�����j���`�LK,T�����,�&=�u(���.���9�����p`un��c���s�E��t`��[q����Uu�V3�=G'c�	�,���;!����7��[���q��,��d���\��D�u���
��F��%H������
_�Vw"?�fL��]��S�S
R���Rx�����k�+58@��&����qj��]R����Tj\���8�(������@w�$P����:��0f0'cT��4���+�v!{qr��)�FB� q��[NlE�S�	�fs1C5%p����Kd�#��^��p�����$/O�2���7��0F��K����T[ct�B��#?d[7�[ejB�J�x�����5��Ax��p�L[&#����9������hER�f���E��62���j�z�i�m�����G��0�!�\V�v+�7����O��/~8����^��N��}v��wG�G�������m��e���:�������v�m�@��[�~�����_�:��i�j	>&��N!b�R�����*��2���	|�h��!>?�(���c��!� ��hb�F��:�!���G�"�6�`Px��{b���R�k�us�����SDU��i��5Xs��>��Z8�*��iR`C��b�A%��P�6����pu�5`�J�,�)�x]S����X�?�X�r�v:���F����k��H�����b��2�c%v����T)��B�s�F�;T&�����zq�VY���1M�u��KR-���������sGO�(��Z�S���Tbx���q7Z���S�Co�������b����w��0`H���9d��t���( �-d�@��Pp��
��m,�t��.C�.,c�a5 C�zb0�����$-��!%��%�.k�����(k�qoY�c"S�\j(QS@(�G���$��{��
�n��v��!v���p3v#�����x���5x:
H���s;���5�9��-�l.�T7@��L�E[%h	�It�1<�����u@&��A�{�H���_�d�����O�F���T�M�G��K�Z@��Ykq�������!�4�lM2��3b���3�f��^��ii���Fw��H�P/��<��)��z�0J�f,��_�\����#_��
���%n6�<g�tC{���D�J{]]�%��G��:������H*?R��)��LJXK��}�s����������S�E2��x��\���i�������v����Q���(0�CcSO
��]�\�N*.��"���qn���2`Md����NS@�	���� �����,����F��;������l��j������d��@�QB�a��(Bu3�FN"oP���g~k�+�������V����	4�sY�pf��7�|��;Y}ov�T�J���T��������4������&��2�F��'+*
��|��S�Ab:Z�Q�Ce�r^���a�IXR�-�rBM�����M�\���fr��r���������{H��as�����=q���p8�F��I��t��2�z0��������J�[���1V�J��Jb(������Z���&F�3��0_�O7��S�l��n���%��&!��p��)��dU�
��Fo���%�:8��*�Sxxl���	:�l��T���n)U��6�)���Y����T������V��[�4�D�3
	�eR�x�c��i^���r�hV�l7Yy�#�J_2=�F��=X���U���q����4�I�����^�6�
�p����~D����G�\��q�r0J�\�OPuT"�N
2�q|�����f�L�M
z�+�	�x#J3@�t��f.�����~��Z;�f��4;�v������3u'�T�!�Q����j���ok��Yp�:"��%���Ggk���G�,���=Bm?Z�(5����,�2	(�T.jJ5#�B8�8X�>���nT\'���"=B��5���r��0�5��DA���Ee)�lR��u��j%��B���O	�Vr��2GIa�H�Vs�����
'C��r�
[WR��]��c������K��\��\��I���	��*1��`��Q�mz�F"�f�D@$pfn�"k�a��Q��2���2�L.�����-#�RG��-v��p��W���T�0�i���,z�L,�H*q�b�ZV$��q�C����V�u�EC�}��<��\�������Q����$�nC�y��V���z�)c!}^����Ri�U<�R	�
�V���O:��y��s����Q��r:��\L�hn1����S*K���C��3r�w��x��Y��8E�pi!J@�e'p�o��(z��C��������BS>b:�����O0��KD�Z�o�<����
�orz�w����JK|�`I���*���:`�����_X@+����#��?8<>�8�]���O�7�
��C��8���� I�9>zs�o���p<!"�����g�U\�\��L��M���N'U�����3���j�$h��
��qt�O�3�Sw�X��z�����I�����'mx���gj�B�To6fE�!�L��[���]�����"9Xda� c�����[���z�Nr{��%��N���=���G��T��'+Iu�\��dtq�<�(3�U�9�)������3�'�$�aPz}T�f��F���k�8	x�t������z����,��������Xm�����WZI�h�}z
�p� k=,��o��o��gSY����@���������VUW��b��<V3X����Q�0��n��Go��#������n���V�
��Ri��^�	������\F��o��ui��+�=K�������P��NJ|��)u�s��-��� mW�i�s��GZ�8����
5�)Z����|�,+
"$<�F�{%c5^m
�T���W��^{4~QI���y�j�*����~�I�O}�L�V��$
��%ofd�)�R�A���F��u^x�}@�`-0X��b,/c�sP�����[o�m��l)?�^&=�g���������g�VL���*	�"�)Pl������ +�x"�`��E��
��'3^����"�|)�L0� L#������4n��U��d�w����'����6X�BIp��$��r�]�!�����>�)$�Ch���\������)
!eh�"M*E&2��P�q��v�`2{����'3cY�L�|�
�&?�i�g�ZC�{�r�U�1����1�
���+@4�=���S4�O�s��J$��2��q�<Z;i����GC-�����b�e��X�$h�������������e}	�K�����P�������?u���5m�QR�
�N�#�X)����ql�4t��q-)8f[��e�Z������*�������8��s��[z%�,AZ)����eW�i���'ZM�F$*�����S�s��L��1#O������������2�r�QbQr����5�a��H
e.gq���8/}f�dPJ�I:�[fG7�CL&��!N�m�/��j���wc���7J6����U��]��h�E<��	����)�������rH�
_��F*��{u���������h�h���P��p���3�f���H�y����j2$�7�L��:�)K��}3M��|�VT��^"�Y��$<Q)N��q+���%�����g4�>%nD����k���MU��5Ov2���O�,:���z��8��z�hk�(�v�"v>`�h��GS�\"�n'��F����.6m��f�NK�����%?��z����7N���K�+19(M����u�����8���z��������;3�3��Y	�S�}����lN�Z��cK�����}y�.Eg}��3�j���
9��j4�������~	��F�T�K����H����a�^�I��a��M�X��� ��7���q��3cc�EJJ�v�@68	X��9��<@�WBS��M�%���r83&3J����R�}\`������/t�8�D_m�<$��5;�]@�1�A�<�-GKn�2a����/��R,WR�F�X$���W�,�$�v������q�Hq	��V�+��PaZ��d*�[9P��V���@����NL/�������0������tq��d>�}5~qt�g�}�O��L�����T����Zi	��!�T�PQ���������W`�T����#�]o�����,J;Np�T9��X�����O���'�XwG�����3�<qd�6�[��������pu%�1>?68�,��m�6~�04}S���n��hB��c����O����������t��������R�������Y*YL��B���
�K:�'	c�H|NO���"5���C�U��'w1P���|f�C�!Le�g	��I�1S)���E
�� ,�GY�F���p�� ��0�ro��	�I��2DFhFSR\Tw��ps�dIW$A���R_����������r.mm�J�C�����������\�������_M�BLo����"�H�
2F$X\:M;ju3KD�S����0���5}I�M~�P#]_�����1�<�����D�9���p��1#��n�?h����S�n�ch"�J�b������Z-��n������/"�� �?fU�Iq��:����/+r'`�����>C��Tpm�������%w&q���J���8�(����*{��hy�/�o����^�o��z	Y��r������8���sn%�o���n����Am��U/E���w���_�Z�jG�����j����Q7�Z]��%�d�n�:/=�����po?o�'2Kn}���������f?�6���������b���]�R����y+���4XPS�y���_�7U+������1y"��AuX�>jO�as4���)sxXfT����}4nT5K��-�E�I��$B����?��T��HwY���������j�z��m�e(��
���������x��]^y����EQ�am�75zb����4,���4�/��,5�KC*a�	s
��b3�g�l��
M����=�Q���o�z��.��?u���n�]�p8N�j���3�UYJ���
��&�r.! B���n�cB]�*D'�
^D�\"�e����O��Kg����M���H������mM�����5��g3��!*+m���?z�����EW�'���Azf����%��B��M��z0io�x^2 �rf�����;�.������U���L��#}b�q7��Q��3a����2����G���{�U��,4'�}���uOq�
��;{9�vf��*�5�r�*�&����wvrt���
�"�R�����R��������G��M-gr^�#� �S��M�$���%Su3+�\���XX[���'����Xc�Ul�.��ES�,��#�(��X�RB�9���a���2�3CUU�8�������o���_���&8�JT����������$a-j�[%�h���
�}qk�UUB��z��|����}��7+_~\R�g�$c�=���=�����Z���X������3�Z�N&�T����h�Y����"4?;6	��R1W���<���`�)#A(��3s@>e�0:�k�G�V�4d����	�|G��X;�S(�JfcD����mn	����4-����d��T�R��:�����fg6q������iJ���J��R�Jl!'�����rT��H�����u�b
��x��t>�����q0b�	(
��2'4�?��	r������Kp"��M�"}*�L�i�E����[��Bc�-�0�A�i��R�S��B.X�����o���������Y���7�e�/U���Ol�KX��
�@���O�����@���|���s�q���!�������x���.�`�}�V���[�3��������'�V]�o��}��}�H�cw�Q!��B����hl���W GNJ��O�~�����6�iS�f��ht��h���~�H��+�w�F�bQ���f
m��>�!h��r��u�~��p
�s�u��s�R����)��������`yb�kQ���v�5��z���/���<�8sq�-X��
��r��h��a��T}�Q�x�{X�GW�����{NR���������������5�~�>�T��������������|�:�UJ_5�M���lC'6�[(�6�j4���C�dmc��:���=����Q1k]g��9��hGU
��6�V��=�T���^R�i��9
X���S���P����U
� ��F�Mo�MT������XK�e�r��k��N�.gZ�%S*����\��\4�R\�1���Q2��5'��-�a%���Y��u=zg�C�/E�e����_��;��n�&Eu�M���\���Z��V���j>B_�Y_��)��5`�}O��s�A?�s�����2zJ�W����S��ca$��C�gCG�l��%8��Z#���T�F�K	���F��7�?6G�������\�O��f����G�8�2�\����}�8���M'�Ys8�������m��i^�/��g��{�|U�*�K���F,@����@����.�[:T���\�.�C�(�<����#i8�	�j�X���	�S��>��4�=|>QV^�~���{�I�0�!�{x@2�T���s��v�+����/6��v��R~�����z~�5�{Dj��B��f�Z���y��ix��!=��lv�D�)&�>5�U���;w��yj�������6�|�@0��R%y����M�X>�\f�H>�f�la"�\�H4�X���z(n)U��A���H�nT�;�Q�7��M	H9-����r�5���d�(���g�
���|��c��F����i����Si�Uv�h����lI$��Vn�X�94�u���i�2��W�p>�o��d��89�2r����/��	3������Q\��(~+�����C��P��w�������b��x���������$��������E�P��L���PE�N�+�X��C����^R�`\�s��dz��e���]�`OG�$<�6����?=��4� ��NI}�{t���2a�����#_�OD�v��\�hlI�2���7�����������
���E�,�|y�t�e����{����lOx�M>�63bl�8��'@�t�$'	MO����b+�6���lBN���S�V7�d9���w����,�EU�V��o�v6V��������\��}Y�_;��J9����y�����>������V{�&�����9�A�6�c
/�������][��?[k[���lnmmnm(������������� �QMe8M����k����c"�����yf��~�Y�������ds���������N�I:k�^omok{#�����duv��������Zg�y]�f�8�>�RF/���U\����1w<I�-w�����e�Xh2kAw�4�E�����oA���y����G;;�|�!Z�{�Y�%Rd���nb�,����3���'����(�8���I�N����q���A�.cRx:m��]�m��~�0�j���)�1��jQ}*�A��+dN��@���Dfj�&��-�'_��N������3�L�O����&��4�G���A���*QK�!����i������G���8������I�l����a2�\���w�%��������w�q���������~��s����o�~�������Wo���������������,JR�?�T��5�6����3�E�x��Y�jC[nvS���hks-jV���|��|9`��k�S�G��M��"h��A��jE�kP/-!��J��R�(���J��<J�]�0����@*�����Vgk�juUI��^r�{@|���V�}u�=�k���E)������A�q*�&�!����&�C��s��������)����N��(��<�����{�)��m�}�5�d+���y��t0��l��\OC4X,�� >���
���%~�V������%�w���������801DJDb����,��w�y�=��z�|��=�^
}x����v/?�))�������t�yydR�u"0)US�h�����w��"�s�U��fi���<��|z6v��:��$�����w{k�qo}+����w�*���oQ
0�������X�f���,�!>{�f�7�'�Et�	\|o����������G������jmU_��^���/"����d5C����_�q����e��bC�3�����s6��O��D�v6p+��x����������^?���.u��������{9j#������b�h-��X�����������'7*��[����M`����ISL������~v�&���O�lv��*�/����{��|*"P�a>.�Z&���,��-YZDt�"����d��]�����"��4y�L���#�����X��Xr������J��IB�9StD���d��g�*[��p�
	�_j�A[rT8��z \��nO
���}^�K��=����[����'rg����u5�D��:�]�&8}����'�D/���H+k�F���_m
��
�@�Ze�{����-���FU)���_�-�<��Dt������P������~Q*�;�"�M���8����z��9������_�^v1���������_�C�p������dj��qo�1����)U��i��_��AR!�S�8"$c
���^��u���_�4Ga"+�"'T�4-�6��I��+Li?�S��^�Ny���@|���.���^.-L�k�� K�_���ol�2����.xw�:��-���(�m�w��G��1�9z���V�M����Ph�
�b��Tx]PB9NZ���$��*)g���h��l5��!v5�\'�+�*��t�iK@f�
p>S�T��!��
IA���7@��2@���4z���?GBS���p��0O��Z��������+��ow���lc;C�u��	��Q��O������8/Jy�Uq#L'	�4�u����gS-���������P�Ol���2��p4	�����[n��OY��`�
��k]y�.��wz�b����N\\(�.(���c�H���6��S�{s����[�B�����"�,\�G'��	spqAVQ�ufu#��1�������dE!.��jW��X��������}c�8�:�p��������
.`�
�
m#�TrHS������%#
� ��Y_�N%.�� 
pn
i�E�}�z�k�B��l��J*�)z� !����3��R�I�l'4g\���;�K�K�����H/�?��/�a+�����o�a�]H�t_��HT������������T���t��:��qZ������z�9�=:B/Y��C��������a{�S���RZ�U���5E�s_��Y��3]��2����8��3
U�u����x��T�J�@��j��]�JW��$�h�@����e��!�s�H=9��Y���:���������4����hIR0���f��)0l�}�uhG�B�X.
������t�?��H�������I4w���^P��W��M���L`	#����~'&VF��Z�?��L��(������"nu���(���gF\I�p�~�-��7���$MI�6�BZ���r������6�2�����R���6=��J75����BC���U���b#	���s)���eaU��R�
����@x>(r-�����N��ja�7Y[����*����2�D^c���&���J������B��-20�m��h��BS��!�����fP�~���W�'W�T�������l���7�Q�
T�Y!�0r5@)WC������k�p�?�����g����:}��!!t9 ��+�Xi��A��o�����d����(��L��|*�����k��j����bo��HiBSo��;�]*�j����z��T�#�e~�a���,e�\��0�<I���C�<��W��GC�Q>���;"3��\�����eH�u��������T��tq��C�+����4���9M|q^���\���
Ih�z��2GR��
�[s8A�������%0(y%���N���!�:���r�px
-��z���BS��$�
v�Ob�*5!����3�eB�"��x�t]Z�-��P�Z��e�l�����0��,)����|���4�;���X��1��l��'�4����`��Y���i�S=B@T�#���m��>IQH��K�r����<,,x,9+��5(���Q�:��LI�[h��HF)�nN;��b�8��4 p�*�PV�jm���jc��|*�7��o���[�����lE�xaO����Nn��q@���o��x�����/!���AvgA�C��ettr�U�+z���5��v�����j��Go������oS���M��E�
�Xc`j��'��'(�fAwZCg[r���YA��3��?X��x���%D���O_��EE$%O
�G�Ag������a�\	4D��%�F��X7����k�?|z�L3�������3���#.�b��,���.W���-���)�]|���p�%#�`�����]2�p��1��G���/
Y�P�{9TuR�`
K��� H��oA�r��Q�6��RR���9���	u%�b��d�Q���]O��o�2�v�4�:O�D[�*�c_0�8zr3�G�A��c�2���Z�'0�;�=�z�_�qtb�J]a�XB�T����6G1��^�s��P�dC�@@&�~����/D�t��\�oq��k�b�����m�tS����@V)���*{$������^�i��B�n��3��[p�t�b�)����xn��k����E>@���G��ih�E���_���_M`N���>�y�7W�#�~��>�L���P�D:q���u�('��x�~�bwtn�r�IE����� ���h20�����$�F�TWe�$l���eb�;"�F����m�&�+��g2�|_vcd.-�
��=�Ir���z6� ,qV��R�W}�=�Vk(���<#��@�"�-F���
uD5O,sE������Dg�[�m��kQ���S�4`$�i��qqq��3!�t�	����(W%��2��#������]�*vBU�'P�QP������WN���O���M�"4H�Y�4N������%��d�#����lr�V�5:��t����J�n&�U��*����#�@��
r�P�����&��*��yR�+����~��*m]�#V5^�&�@p�f���D	yCu%��KLP
��b\*�����Ktc\4f-�.ZZd�M���I��7<����Y�G��KW��/���H���!�����+������L���P�A��t'�9
�
z����M�mK��t�M�L^��0���2��{&�2(JEuD�����;a�"i�v����E�{)�Jk(���
5LDPR����m�Q]�R���|�~����"k=o�
�_E�j�h�I���� �^e�+�jD"�0��i�L�WgP**��;�N3���G�.6X���(��.�K	>t��q�p4v��f��������t^�5�UF���|�2$#aXX���N")-��/c���N�/s�������x���B��N)�X�:G�oq/41��>�n��Y.�q���3C�C8j�������*6d�Ql(��zKm��� ���M��F:�.�
s�a�@�,�l� �<����v��3��:��M8��f(�@XB��jB�0�*���:4�Os������'}�1-�����.o)������
n�������L�4�KSVg-e�,�zz���f�5x�"L�V�����&�:��1oS�R��lN�M%]s)%���p�9��9�D��U�FG�6@YF4�G	T
�!J�;)��A�� ��6w� ��s�;�6u=6�PKL������R��+�������ii�xJ��`e�.�<BQ��X((njrenG:d���hj`��jVE]� �P��+5o��[,�x)��C<~@a���c�m,�%3��9x�^ZK��S���J7_X5�������Z�K:1*[�1�0	�f%E��~g)�E�N��(��bI(�0\]r��;C�b�I��n8���[8`�S-�l��Dv(��	�����23Xa�����:����-"	2�iwI]+���QX�}�.�p�����.����� S��4j:\Z������������Z���2���*�;��ClY�tlq��J��yV����#F Z��E�}$��!x�8��B��e�#������B*����Xl����}<O���AI���$��a�uy�6u9�������]�S���;X��v�W9gC�m�� mx���A��7�S��S^K���L����x��)���;2�+�AC��J�SJ���h�5�t�2lE��?�W�~�?r<��0����R)��)�`����l��vw�WW�����Wkk�^<]TwT�0�_A��Li�\EP�Kjk����Y���#<���@	���������-����ym���8Os�
7�i(<����/.��.��o���]�~��������*����Zm������s�����sKY�fC�����Rn�H)��5H)��wD�s"|Kc��]�������	5��~0�[�
������4��okx����Nh���{jk��[�	'�:�+RH�����:=Y���O0�����[�S�����Ss�B�FKW���D�`�M]�V���?n�j���!��UY LU�_ \u)��"Wq2z:���dXk�g���j]�=
p��g@���3�����������>j��0�����;k��u�Y��_����?���Y�ol���������������x;����W�{��������
�y���v	�sg-���������|�i�Y/)����I����Q4B<4�#��2$E�n�
;�B���OH�I:�:��A<����q:���y�?������)��Sr��3
��>f�aE��K$�e�{�����o��q���c�����Y��j���f�Y�[,c���`B�X�a=_^�D\�<"��S���{\Jx
�'r���P0��)������5����q��
���`�oIoF�H3HI�e�&��9/�5m�d��Y���N��eXO5+��:?L�GAD
T6P�(.Z���#��2t��!��Z��D�
��JFJU����^L-(#�Z:N=_���	������K������M�B��I6J�'��e��V����W���bc�������i�/����?�\_|���m�n����������(�|��C��]���������S�
��|K`�Q��.��_�[��n�i�6�d=`;����^��N���v����0:�����q���������#�T�7�����0�����N�Y��v`C^l<�*t���?�G��W��V�(�!,��oX��*&���Vg��n��Q1S����<��/�)gD��)�xF�iD�f���Q����~;2����Y�l�h�@����]���&�m��H�y�Skn��E{~;M��B�~���#�.i�[��:����aqi���uCk�[�'�����Q�n�0Q9NT�;���{ig����v[�����(�`����L����%��gZ]�A��m��j� @��|��3�@U���=<�s��5T]h�~3�#g_���E�omm��"6����E��cW���X�~���K�{[��j������_�W�}i'd�/}�����w���w��rp������E�� ���X�`O���(��xf��*Z	�[�m|�l���Di���x?*5dr�
������Q��H�/�����FRj�?��Ae��*�|��T��L����T�d2|����Ue+�*�������AF0&�����}���S�B)Z�X}h�!_��4�[��x&]���Tsr�i���H�h�	Q���=�#�n��f��	����������[�Wr�:}X��SF)pT��r�����u�RJ�������/�9�`�X3HcM�� �
���3���`�l�(����L��{�"Z5�����&y���	��7��LY����������:o	�����������~���d��o��(��X|��M���#x��PuW���K�u�d���$c2����7���3�z��R����t�,y��	�]]$�7�#X�Z/x���+���K����@M���Z=
`{�����?E���
���r��	��Y�����W�o=�y#2��b;�H����oX=~���Pc���J�Q�$�����0��nFv�����f��5"�����3�����[���.k��5��o/>��5�ln���NM7�t�E���
�m�?p
w���@lg�r
?g�/���:�uzI���&T���������{D�?�HS���Kz��]��;E��oI��v}������[]���l�;��R�jrz��<No�����2[��=!��������N��E,�)�w�}��dr��E�?�������:���+�:�_��v��8���k��@gc��u��2#�,�y��3��E���~u�Q�������v���i�U���[x-m����_�J��S����z���)X���O�Av�4�.��������
����?#{,X��n[�K?����-���Z�Tw�Fk�R!{C�^".���9\#(�(�x�uE����(�-���8��|8�A����k�G�C�y��0A�E?���'�T_�9�hM�.$���`rJ�_M���_����tx�}}���G'o�.���gX\��:��S"�����<���|��'��h� �Q>Nz�BL}
�0b�/��t��=����p���K��z����*EM0,���������Oo�tx��<��Z}��a�����.�J���;X��.� ��/��l�V9�5��D��)^1��=���zh~�(v��P�7����	��Xt^�	e>��5:�BA�f�B�)�����	���pb���_f���-����#�o��31h��,a
>�d��W?d�
h�����ES���o9O ��D�j0/��*����w0~F�;�b
�)_�b`NJ��Wk����v���U,��h>�;B����)s�jb�5ee�Apo~�Lp�0�b{�Y��.�.����(������&e�'s{�@�]�>�|����������Y6��c���}+|��{jO�5��X�';0Z���t���*���.����9����L����A���-?����k�9q�=k]J�(�8� �q��������!Qo��a�g�>{_���7���y������<.X>+|��t��&�
�1m���������}�'�;(>����
-���l����

�8Oe
����~SG������������kR����:?���S���������^5d����G�CAb���3&�)�_��U�f�yo�ys���Q��a�_�{D��N��@�X�P�8�����w4�g���>�9e���	QO�*"�_hW�	;���0�1��J�AEJ���� ��9�~����B�}BW�v������'�$��Yw��gi1�K��3
$9U0��U�����5��-X/ M��N)�5�����HH/a�}P���/x7W=(�n�)�Y�W6�~�����JAS
��������l;��d��`8N��h(��k��I�K\�����@)�t�������n�]��g�
�x�'c���$�
:�OmE��.��>T����T,y7 }A��9��tt����k	rK
�^��2����3�9�^���
�6�Mc"{-�n!���x���.t~trqx�.�h]�
�4O2��G1V����'���\��'�	
�Me2���1R�Z�k�t�`��~�o�>�N�W��
�GXG��=S�V(57nO3
I���@�28Sl��(G�*����|C��Fi��D-~;IG�J��;^/hR�&��2Q��S�]���������=&�G��..m!�b{!�+�`���\G���V�tvZ��������u�P����B�������vm��)���j�����!Y��������e��j���oK?D�����Z��������Z]��/#����|���d���Gd��s��� "��9�^��L#�b��)\r���f�n�/N���%I�3.����r�RL���9�����T���M�)����Y/p�y�r����#Z��P*r��%�BK��-7G ����t����L]O�" ��_u���;�O]�2oBH�73'"�DMN��B��^���Y~i������XD���\oP�3��{SIq�~�y�l��C'�W�-������	�	�
��`��zQPE�-n7|]�Tq����2�Ms���<��v��H,n���xYD�l$��)m�����J���tX�K��9���"^F�XCkL��\~:����P���A�L�"������?w�Y��p�
��s+���^�-�;���KC\�z2��d���8j �5>_6������{�-�t-4]F�������zR�A���
F��}���igcS<�����.�q���Z�=�Z�������E���do�&Z9��\ �P8��]�Xj�J�\���V�����f�!z����F}��{r[�@�'� �B��� �"p)��x
�#��>2H iBv��)��0.�����C�"j�E�	�����(��R���o�|(���P�1`a�?��
��u��We�XM���(�!����l�q!��Z6��(��b���!&5R�>�-������/��D]YXZ�
�#$%���
��J,�p���l���:�s8���>���f��vli���Y�g�ft���|
y:N&���l �_�d|2�xQY/�tU��8�Ek'LD�S/l0Y���t���%(�>15�;��SKK'��0���sAIre��+���l��^Z�dw�s��WI����Av��
�FfQ
�� ���z���!$��{�����R�$���4���+�iB��$�H�7��,�U��M6Tj3	K�X����c�e�a����%��������v�Do�z
�[�"�Z�������<��q+���N5����"����?�m������T�C�C|�����2�R!���(�J=$�2�g�������YJ�E�����t�43���F����a��t�9������]�yp�$s���H����d�
jB�S��y����?����L:��qX����F���Wh>�����jc�8k����03,��=�0����"��j E����p���6�����-��q���`��>�v����62s#u���c�w0wy�C���70���������	�fv�[��SV�n����&��eAi���$�IG��rQ_�M�j�kJh�VJ�xK�K����H������o��zgC�:��Q���O9~Aa��,����v���1�X\��5B�co!���xhB@$e����8�ec=�������~<���7Jk��A���s�W��q>@k'�����BXu
�G�Y�+���}�������%y�%��<�Y�U���~)���S���4��Yi�S��>Z�KI<XP��|
�q�������j�v��t�����nH�Su��R�G������G-
��D�-%���8�b���1s��=��q��d�C�`,��#m�l�a�P�(��ES��H�&�B�5�5�ro��W�3��s}����L9�W�
Q3G��#7gF����Z��-�;�-[?x[�A���v�u����l��},���K��*�4�
b~�?�~r��@�\;�	A�@C�A�v�Ib�US�����|hy�;���T����7����R�+v�|3_i���
�Gb�7���ea�1��!g�#!�z�|1�����&W�E�tD>:F<~~� ��?Z��`�����Go8o'�6QouPj2�J��y��
�A_[�����p;o7t���"e���n"���E����Cw4���]�9�����3�)��M �+��w��(C��i��C��>z@X�M~P�h6�H��Q�������I�^dT��hT|k�I�q��T���6�_�t��
�t�L��P�d��I�g����N���b-�W8
N���t��j��ST0���7�-&�9���[e�$	 �����<�n4���yPS�PQ,oY�:�;){�eG�`��a�P4sd����>��23���9�K[ke>��wk���H.����Xv���k��ql��;�1,�����������*+��j:��;580[��~�p)Fmo�c��VH!
�|��r�a�(w>�����X|�x�����IP��F��������r	_Y���77��X�����"w~6�����+0�-����{uG@��\f(w��d������(�T$�k���Ei�B��~�l�*b��v�]?L{U���_�[��j(��h��8o�����*�r���A��|`�)YB��%���E�FY3v��'"8�

���"��lS�i��������h�,z�������mr!�vkYQ,���r�-��u�7���,b�w���S���-��xUQ��
9�*.u`��M�e��p���;'M6#P�������X��/�\�_��lJ".�E1���w1�5Q�E#�uC�hp��P���c����h�A�Z����y�Y��
�zf��n�����+�����MGGm�hn�Dd��m����0��+�}�����ExvE��R�iK�����^�B��V���W���C����
J��Qf,�y@�W�(�2����B���>D_��#_�bNw$�>�7� ���{�>�(
��������p�����������l��}��F:h���K�a��f����NNsc��2���P���Qta$Mv����t��B�)��J��������u��cE[T1n3s�^�� ��G)�tZ%������G��D��.������iVd��
�O���m�
�����,���:�p�K����H��Lfjv�n���\BR�9T�xN�����s���-�����������>-�m�/��q�V=S1 ��������(L�J���'��.���XQ�-�G�ml_E����b�j�]�{��1�>�8��'-���7��7�a���-&� ������<
?��rv�t}���o�hY���
)�1���SW����'R_�N����d�#�/���n��wl���3��m�5�Ri������uQ�Z6��:�J��<!y����~�:��������y�d6����zMd?���N�dp-rsew4G;�i��u�������3,��4�D��he��=)�����/��/<�SUN�|x�����������%CQ�P�������Go�,�Hr{%'5w�;F\�!�$��t����,(!x�r:Lp�_�CN��dJKs0��&8��#6��o]�������
���fo�1 ��? g�����k������
������4�0g;(�E�@�<�F���7 ���X�����9�^���M`eq&��Q:c��2��$`nu6�a���iv?��(
�U�W� ���T��f�L3�d����\	&��:#���g)HN��nQ�����J��Dkz�fZ<RL��1oU7�6�iG�N�v%�p�f���� �YN��s����$\ik}�Yw��~���'^H��[���a���n/P��.� �)�g=T�4�uws~`2�����[���wDq��
����wi`���+p�g�5l�1p�g+*�%�|J����/�i����N9���2��n�0��q�K,f�S�H�~��K|�EV)!Y�D��X�twc�����h���� ��,�5rq���1���H6"1�#�*%�����1������2E��J�|���T���#,9���)�P�B$0&�Q>�C�R�G�
������t�e��G&���
R� ��k(�����}�+8I�l6��0Rg6���K�JE��~ ��L� �T�A���#q����8������q�H�mh�0;^C���@$Eg�.��+j;0��*��s"^�����m��+I��G|X�[A�+�F���k�������	�x��@j�z��=�*����2�k�(�gd�J�MVN�YR�(E_��g�����H�7��T�f_��"�:m
x�������$g�����CS"r�L�Dq�Y	���m�=�>j���,d�l8�F����hp���M(�o���r7��@
���FDo(A���B�`	���K��t�\|��2F?���!1��+5�`N�}������[:������J���������.DQ�L��j���Cg�����&>6mk~��F)�o,7�����0�T����#f@(�n�
2�������t���S��=��V$��6�Hx�8Q��
3����d�b�V]	��i�-���c�K4!q&uNa}��H��;��/�����$����'+���Ki�eS����P�|uO`px=����%�x?0=7��dz^<zxz����W|]��T�|��RZ7�p*�T-F[���-��,��!!�`�'EF��������|6s����R��7���6���AW.taE��H���{�
V {���u3�������p�6C��<P�����"��_�{&��q�F������F���h���P���7�8����1�!���dz/PPfGh��U� �d�Qv:��������3�W�t�z�U�8����o���j������e �$c� ���d�� c���S���z�,
�gH����	q�P*=���Ax���*���\��������e��9
x`��r�j1~���"�������arC�5�����B1	
����Zkm�(�����u��E�x4?�:�Ve/�ge <����`@��R���+&�	���-���,�#��A����R0������E*��zF������)Y���(d{�����$��M���V�R���B���r�TR.�2U���KE,����l#�9�����m<��/�����'#���}��6�)�d�N�g|��#��&AxE����@O9P��:A'��s��YX{3ZAu�.6�r�[5XR3c8�Q<D�$���l�b�I����bN�d'�5��R�;��]|����}����f��]�+�y��&���Q�H�@#�Ise�����4Ip���
G'���q����AXlUS�)A�~R	�D[�]���*8D+�X��Q+���Z���e���^�-���u�R|����9��D���_wJ������(d�a�,�w#����,[6-�uNB	R�4
�����</>�����=��z�bA�R����d?+0����T���������S(�I^�;_R<$O�!
�����"�����|Gj5��<~x����Y�;����m������XY���^�OY� ���>��/V��!~�&)�V������*t������
�mt�i����`���D4&��Hhl���#��ic%_S%�xD!h���mY4�������6����z�����9���;*�?�1��~R�gv���������L���$��#G.S�j�����t'�R�8sW�R��#��6OE�i�Q&���JD���h����LZ��^�%��zz1k�N�����&Y���
\Xl���U/�IR�;~�i��,,�`dc�P�6�O
�^����
�dK�L��x_��_X_�t�'��!��aK�*��8`e����Z4��R��g9J�x+k���Vd�����B��GW	�R��&�q���X��Zs�(t����C�N�
�x���������TFK��Ka��������EH�W(�7#�1����y8]���X7$�?^~:;>�~<:����������������[[k�oE����Vkc��:�q
�l�����w��r���J��+M��
�t�� �Q�j
����o>�31V\Q�����<��������6v����F���N��G-���0%����X�>��*z��v�}��S���s�(N����Gz�e�1`|�_c�q�����"#�~����"�tY�d)
��R��.��z����h�Y	7 ��c(������M�:]�����"���]9-��I�wq��b��+h/u����]�u����Eh�0�����\����t�R^E��8�
�d��c^�%{��P�`�����J�%��C�u�)d�\}+?7;���6������p��O�w��]u���:�� t�����d��7�us}I��r�S�W;��:��cC�%qmG�����oFTX��'�AD��y�����H�ct��FD�����)�iD�`�4_���qjZ��e���:�����|�t)��	���b���0�P&���������������A��A
�Vt��{��7��.��|��MfNF�+
<����j�[�/�4����E-�����T����.9�.�T)��Br��{iFg���[�p��2[������5���[��2��iu6r��[���|��#m�N+��p��=�R���_!���km�����b��`	|9��
Y�L��D+�����r �|���"��\-��D��|41����D�H0��?#�OM�����D�yob�il�����-���[O_o9W��N��c���OUeY�H�'�2IO�E
A�S3��h�E-J���`���I���)��8�11�������sSxL��r�,���V���������M|d��w*�d����4WM�(dwt�Hv�.Qt����2����������J�:mWC03����`����1-m����x����/�������S�<����Cd���F%�s4�lQ}��$:�O�]���.���]�=2��wQ���`L(��PtS�/�e�+;��i��sg��1���_zs�p���vH�r u����As��=�B���*�Bn�0�������;�Nw	y6��]\����,���i�Ubm����9����8=�cG!"u���9�*Z*�K��������`�)@�1��Jj�+��/t�l����C��������\���)������-j�N�.�E(h�h�(��f��60D���Hea��)_/��2�����c����5�Bi��B����S�%�#y��J��.����T�cd�����\��������&�?�	_�)�U"�Rl����s�b(y�@|��Ra���Y�<�/�R����z!�%���q�����>� �@Q������'N�P��k�f���������2�]��^�z �W^��r�m�������Vkw�L��wQM�#��'6�X
��	�]����_$�Z;���&���;�'�<i�8����
� �t��Y&�jV7�����������h��1	 ���?0T���;|�T�*��e,��D�^p� ��>�6�� �:V�S�o�3�N�aG�W�n��b���h�V9{)��;����e�x�`I=;���D�v��>4;�Y.\}+��[�
Ea�P����(��V��{A�7N{_ ���*�%��5�A7���p�e����1���1Y�����k�U7����h8&��)����S�I��Cjn���Za��0 �RW���07��9�	M��X(K�j�N?nlA�h��}6N
�`�RG�9�����	�&	�_^l&�w�����Z���l�N-�*��a�Rd��H��������hV�WJD���{�rX4��2��(77��Z&��B��I^��_9a]�O���B|w�[U��}���c��L������91as4n|�����V3�k����o(g��\L&�BJ��"r�@��>���g����H=�E�o����}��s&T,��%�K������C�z	��BBZ�r�+�;��~�-#��^{���U������?Y�8AA�#L>�SB�~�fI��t&`Go����>a�k����	&*��A����������
���������f�S�uCdP�=l��|\T��@�,��+�9z�)�FKT?���H�C���F��H$"�kh�Q��|�z�����<�z���=E����������,����E�:���@��t�:)��#+��N0��k��*��K![����E�����1�6\��Y����c��$�B����=@�V���@����J��@<L�������tEg3%�@��H�JdIB'_7P�A�]z��,t������e�^��������3d`Gx/4n%�z�������63V>{�n������I��:�����a��1���Z�Q1��`�|����"�?.D��<U��f��6a���������_���Z��j�Zh���������o�h<�E�0���N){�R����CL���
/4\��yXl0��
�����%T/��\�������Y���:E
�P~SH�`"���������T_���h�K����\k�>h�v1�p������'?x�W�>���D=d�Y��������p�_������7�����d0���?+��m��N>9
���I�[E��(�����yyc�5�eY�?�l��+��-+Q�9r�x����5�yi���w��X�� �w:�����m�A���s8
�w&=]y�&��8�����������]6�$�zK>��F�K$E���P������N<�)�@��Y�c��"��XV�,&����p)�����`�m2)�4m1\C�E��5����%��h�<K����
h���E�BhX�����x���v\k��-Q������,�������y!�}���M���d7+���[��y��>����G��W+�{af������xL{�0F��V&e�7�yD��.�����u�X�b�w��w�!��/&E�Zu�D�����[x��h��`�4�[�kT��������Hvv��:���r��j5������5X�|��x��J�d���x=
������}��3�Nr�X�u���B�����p���-�d�{�8�����&}�3��%�+�R0�����l��H��xo�L��l���Y����%qV|�s��N�z����E���^j�<*�Ri�.��b�{��q��iZl�O���	��&��Q�� &����x��>�EMZ�d:�TS�����;����J���h2�������96G7��*���D��4�����Y��Tw�����T5���e2(�����R=���R�>��ir�������w�/��UC��R7�E��P.8�J�"��d���`�.�^�1D&09o��^+�(TE}8��pn�u�A���~�S?�J�����.�����Atww��]����Pes�l�j�Ee�����9�~ �Q�1H�@�N���$L��i6�5������fb�>�|D��������u�H9/���
���2�q��v�0�=���Y�	�+1��������PQ��%@::��KO�X���������X���j��T(}\^P���X���>R��UBQ�4d0e�N�$�~'v0�e�2��Y9�4�VR�(��z�h����,�x�yu�����y�#������_(�������C]��0@(�b~��$�N��UvWH#����a�X9��kJ���#��b7����d���05�&��k�.���x��\p��;!OLP�E�X�&�:�%����������������}$�_oX��6���\�r����a��3�K/�����}s=>=x[:S/���V�<��)h��h����t1����l���Cx��Q�M��*}�`7����Y���&X���M���2��XPFm�z�zn�c������I�����/����u��[y ��"�����QY�U��p7�F�k�&�� 	L)�����cPq(m�O|/B��`��
,�E1QS�����X�m���wo{����0��W���L��.��H�(g��J���j���I�0:QWTT+����|�
��6�1�|ueaC����"�S��$�R��_;nE���*��D��������s��W��~��3�p�tIOYiIU�2�oVD6���&��"�����<�y��������y�����o����h��4�@�^!��ASM?�
����p�1fTS�c�����Co�$��s�N%��*�����&D�&�hu���J��5v�a���d6��p�������\�/��J9r�OG�Y<����g��V�i3���#�rV-\T��pA�x��N\~��]�k��K��Z��I��fxT���������������T.U��x��BNDl� �����r�U�*����Z
mT8�e6`F���M��h^�b���<����*q�~�-K����`�o5��S������a�SL�px|67��4�;�[V4��5�E��?J�"1�%z:����<�}�/3%���������+���Z(
��\S��
�b%�p-k�xA��>�����K�f� %��U���7�	��I�g5@��<���%+��V��)w$<����o�?#�X"����F`����a&pG�l�A�1�\��=TRd�@&����*�,�Vb�����1L���$�O���am��i�Q�� 7�3Mqf&P={�Wm>�$�vf�Q�
?�^�<���++����"�O���Wp�0�t5��#
��2��9k�t���X�,1cW�T��F��R�O
�����i�>t�O�I�&#�E�$�d�6��D��t��K����p��2���1�R5Po����
���x����]��JE����N4�-[�:����l���n��V����k�W6h�i����u��Y6
���������_���Z����7��u��+"�d���ly�`�G�f����psB]��J��b������t-4��������a�=���)�S#6���Z6 �Hz5V�oM����]Y�#L���(�QF���b���6�d�N��1�C�x,��e_�{�b�$����)
�_b����CA�n�)�Qgyyy�P#	I<�%�z�fJA�v��6�����r�v����J�����MXE&�����Fk�E���oP2����/n�����o��5=�����v�X �[@���7�����91�g���Xw�����[
B���v��z���x�'xq8�E��)�����S���A6]y���	,x�lml@ ���������
���L�0�.��@��tR��
g`	b?f�|�w:uFN�� ���]X�IN�����
b����-�&%�����b���;��[�����uk����q��Or�K�4D#Z�
�I�
�:��>_��:��2�0n��W��\��_��s��^���s��3�^�$h�����'zKb|4�V�H�y�bq�h6�����+���A�����S��0��Eo��x���}���o��	{@8��J�!(3x���"A��s(���j@�6���=&�y�-I,$G�)�w�}��w[�p�v7[{?�T�W���!��d@k��n��	)��rI�*z���4hZ$U\���f�!�}�4�j��������r.��3��G�����G�T��?�v;r��$�)�����m2�{���X%�W�A����q�A+k�����y�������p=���^���\��r�L$|5Tf�.Q��o��U�&���z��l����A��
D��m��}�#z�o;��^\C���k�@z��xj�e�	�8J�����{���C'�6��.�o��j�����U('������4e.f��"�����)nN���VX�uT�`nEnjW���4��R�@S]1���!	o��7���1�9�r�{�{a�(������~�����v�M�.�{t5:T8�������k�h���Mv_0����u��k��y�����Y
g��j�l��.P�������A�*�
(Se�Y%�:���k@�v�U���7l��s$W7�����w?(�P���;e����\����M����)| ���=�V������e���E��5V8���[q�U����{]k�m`D�����1�//����G���yU���x�_'��vi2�Q����.�Q:t��}4V7"�Os�t:�����-���)������_ �W
��B3�����2��F:�t.��V������r=GK��D�;ET�`_�t,q|<P�Z��s6�R\��VU�����uQ��ry���FO.VRH�E5R�8���

�3l���>a�i�f]cs$}�����P�j�$��%R(��D����z���S���X��QC�/K��"#!(�3q*��g������E�/Or]zd�\O���3�����4�B���M<#�4a�x�6���i,�f_H:����oo��:`r����`i�d��B�u���A�eV8)���2�rJn��7�ifj}��H��������D�e����Wwee�y����RT�>:���wA�/"���&�r�5q�`e?5��'Z�*fO� ;�}�S� >�C���T�-��<$s�>���	����j>.`U���M��m���B-*��	r�{�������E�Tz�� �Z��9$@A�"�a�����0�}����$���P]C�����z������6����:�{�s~
j�G�
��v�����v��+��DX�G�R=��g^Gx��?��aD�R���9��P�*�\�g��^�t����:c��k���F�N����,�^K�l
����)����f�:i��ia��s�����9`��4����f�J_�����3���������cC�K�c*��'�u�4��W��Y�C]&.����Tmsx2�Sb5;TUj}ss����cX���h�2��P�����A���r����/u��KZ�ew_}7y��	��UmL���#u��6���-cu����������k�v��}�nI97�G��Z�U�ti+5��d��;O���jC�U��������=�~rzy�\�GAz%����<�7���:�L�*>�\"�VXH��A�Ld�������X��J<���J��?�����E[�	�J��z�L���'�6wv,8�j@\��:�Acy���Cv�|��@5f��*�O�ja�MX pbay0��kJ�n�G�,CW��Z�T%
q�
�DQ���=��RYTP��*i�:?���W�w�4+���[�����B����z�J��1��:�p=*j��
0�J�i��Ji
y�I$l�o�l��O�y/�PxE"�F+m_����e�V�7���v=n�����	r�'n�(R%���|/7G���(y�ea���gC�>�C�	b3��'����D���d�f�c_`W��a|/�a
Z$�)���(7V�0��D���b_���Mrj�1f2��f�N
^�2���� z���x�Y�g,;��\1�}��hT�����j]��F�7�-�hS1��[%���}�VY���}�VH7vSw���wW���������PTP�l�Q�b��po�~�/�e�.hn
"��Q0H:���9���j��PU{�lLOfIM�+��-gC%������
��Ja\>�"{�#�2��:�unu:6�����,_;�J��%���^���Jc�>��aa;���S��c@��@Y����z��(U$,�<����N�"�/�(�~�	m��N���r���}+A.��j5��|M��A�,H�"�nf�V�����.������3`����9���j!^������]�@�������������X��v����������m,I5g$�7,��F�gX��3"���o�l�q�'C���(W�M�:���Y�5��gb������"��n�+���=������p#�~�-�Vjs��0J5'�r}���2��"�K�H�����K������M�[���-J�P��^E�l���v�<�~��V�f�KZ�<.����l����a��i�ZrW�{����.����Ad�8��>7�W��5K)Z�t�B����8^,
�o�(@�&(_���4��
�'v"
8�W�������N����2��
0'��N�^���<�v7�)P��STc��
v�a�s?]����������6t���>3�A�����R���������uv��2�K��Z��,���,�:nZa��8r� �9���%�Lh.H9���:4 	*���l��[�������0J������UQ�Q�f�]F�D���5���z���b$�+C��������#��q�y"gK�4.���Y�� 
4GF!L'6������ /jX���4���|�,�.i��������xe�')���������W�v������ �w#���bZ}����^t_�;=?���9?<�P?\\��q'A"����U�P�����4���oGOE<t3��Q���p��X��i�4v�y��I�A�	�.��2�-������R�?���$����9-��J�=�9�/%�����K��+%����#� ["�)��-������	�!��qYn�p�Qv����t�������/����J	�����PU�ef.x�����3�� s��{�u�	P�	W�A�x����J�|�P���f��3���b����M{�J�����d��
���*�����
�G'n������R����Gx\�\I��97�U������P�"b9=��	R��Z�8K��C�����o�$e����I.��=���\�W$�
�/[K0F����`y�0�z��Y)T ��uO�PT��A%*�{��,?��dB�me�*�JR���j�����(~���:���N��Af�5��}�)^Y�e����aPR+�.��i���A����{����
RY'9�p�w(A�F�����Ns�q����u��f��<��7P.�CI@"����&��]������m�V����W�i�
�}�Z�ke���������'u�\jv.P�[��E�N_�	���X����l�T�;�s�F?tI�}n�P�\��}B?2h���f��2�;u���P���!���W�>��H�57V���Qz��B��������,/�D���e+k�9(N�RJY�z��{PV���������)��a"�����?�:��#a����A��r]Y��y��qNcbG+J?�����[�  �T`��8�P#��t�e�I��"�����N\���N�4%���tC?���`#|���mF�3&n��g��I�j\(���r��	$~�Z�
��(��D##�#��d�`�g��9Q����OsL���BQ�A
$?T�\>�-0�����f�	��Y�G��V#�V����/@���Z7<�S)9���U�Q?��8y'��e�?�3s�/[$
t��.��o�����:����1*�=�Zs�.4t�N�����m��XQ�QT�n��������l�%�0�YR�Up�!�2-9�$���-�h��#s���[�r����bM���-Y���1Mb>�dM����$P^*FVz������,;m(��5(�3�x��"6��T���sY�2*)����om���e��4�]����L�doQ�4s�/P����,��*�_������7k6c����c�w�$Z����� ��Q��|M{S�>FH����.���{�K�X�bs��E�]6�n��k��X=�$�A�A��0g1>��d��Oi�E=o�J\�>����Y�:��0�&�[����;(�E>c��('+���Z���2�0���\2�O�+85�������b[Ml=�l�x���w�#��,�0�\�{�������	9Q
Z��Bo���?�&qO���b��@1�\0`����X�(���29�w#j8D�mM���	
����4S��Z&��4��WO�����1��~z�U�t�hw����3�\%���V�������in��dFb�����H_���2��b��o�Y��;�����~P��<��&�p%	�����m�'���q`�������}�[�y	��������^�J�B��"�	A��PO
��i�p���Vj��s����@o��DhK�dtM��4Yy�y�x�� Y	$9b�p����*�X3N�'��9&Y�.N�,���@�o��&e�
�^@�]P��-=�2c�+r�4@RP�o�8\E	��m����B]+D+$PSX��w�d%�a `��]4"B"��bvG���xB7��m�A,A�$d��B�d�p"����$G���	6N1����8Et���,�1�D�xTn8���Ah��=:��A���GuV�����pi���3>#�B�������v~��n�+8�����hes��Q54@Cm�a�(WIH�P�r�
�ZQ�	.������+����iJ����HW^�M��4��z��-�_#`�%�5���M�����X��%i����E_@e��c�pr/���M'�<�������2�����M�,����7f���]�W(U�����N�k�wp���k��������F��)EG���De�Y[)(����!������k�����)�J���#���,�����)XSyP�<�e�����il9a��f����L;�����vWj���[E�Y�G~��5����m���S���H�`V"@�i���=\*�������f-A�,?�;T��Bz�rj����)a�{-l�f����uD[g :�����=�N�}M)��/��B���O�� /`p�����pa#�U�Y���3*��'���.=�\Dd��e�i���<;�����,�'h�9�{p���0�Z��VZ�]sAq�6��t?�yy�|V]R�����q����{���&.�)�Y��{��xh3�(������z���
F�0n|=7���j�sv���d��2j�N��_o��������H���%E����.��5
0@o�.)��W��6�J�<(�XG*I���
��D�Q�%B�jcDU%h���\�T�q��X8�����o/|�
 -n��i6����i:�����������c���}��B��6�4L��OFn<���d���G�;����e;d�F:�5�+��V�ZU���^Hu�dl��H�i���3����&��� ��WS�5Ik�`,,q?�����0�?�'�g�x��	������������D�:o-S	��
�{{��Fuuwo+N���z}+���mon.���z�.7���#C��6�W4�[����w��	U���	�@Z�5����v{}���Ro���Ir����Av����d���`yx?�f�9=m��Qo�b�4J1�� ���s���Q#�y���=<�<zw������������F�_���s�n��]|��}��+��:�f����A�:��h�o,	4���d���J������A�5�fy4��wxq��TL���x9���z��Y�+k�#P���:-(G`��d@n%�d�cd���������YY�&��M��YN�B�[�m0�s
��!�"�����v\��qbo����!��N�E���b��
5���t) ^��])�9��8��R*��kP�rcx8+�W�C�Q��	^��m:bT�P�p��b�i�:?�/�d�EC���`F�"�Q���l�'�r�my2V�$O�I����`�P4������"�2�
R�����5q3��\L�-u��j��v�L��Q0$N%B,�Q|#9|O��?�<�����GN��:���/w\}5���Pe�
���k�]��>?��t��t�����1e��C�\G�7�vp��[��WQ����Ho���,���@��9�1������=V��(R���uz)
�����Q��B|�/�d.v�}����;������FD-�����|'�j�K`�,��j���5���\����J~I����A�Nb�m���/$=��&?Mf�	-��~�
�Z&�&������k%��K)�A(�L��fA��dF�"����-���}�p����[`�T��g&�m�')%[l�$�?��-ZEJ�)L�1$
<
r_� /bSx�`��$� �P1a�;t�v��
;�tK����JQ�=75K�p��&70������L&��A�DL�J)����L�^\p]���rYcRo����l�d��Z���u�����Qt�����������NN�N��_��%�N��x; ��>�q�����_����'����V�5 ��C�Y}::~k�*��T�L�:9?�8�k���I/�;E�r��oc,7A��@�G'�����YM��e�tJK������_����/�t����t���]Q+V��t6y��@h��"BuK��`����`K���lvt���%��k�O�/�aY�_����-<�8��O
2��|���0���T�s��P�"��K�C�n_5B��Ps�������Wx' ��9��b���!�p7��O�A�B���$B�i6H���X��d@���U�H�TC���@jL��d��-����t��95������
����P+
Ji	�#	��
��B��J�w�����E+n���
����0 �|�`��p��R��k��
{?��K����BZ��U3� �x ��"�9S0do�C�y"/��Sw!���2#��ZHW�Nns�$�u�D

V!"D��ct;�x�$!tB��_�gG�1,?#-k)0�
�;�sL��"N������������J����SI	pa5��-'Pa���:�P�1%�@1"��"I>%)���������gL��N�T����������s}�n^���������?QOa�_(���������71]7���O��q����xr-��K	�h��t�"|��5������ qq|1;���u<�I>�S�H����D��C#�B<"����F�E��z���v|u9����=*��VoU>G��}g��P���M�������b���:c��=$3M�)�$�L�:g�m������:���)h����v�V7�Tw��������u��Hjv?�pAG�ap�T�9�"{����
m���s'��i��sZ��D?����|�������EY�6�!��r�n-�����k�&4������������P_����]t�����RVW����Twc)���p�`k��.�����qiJ�o]�*W���&K���-!����IF���\���"��t�g��k��E��)�����{����|�iD������4}=�j
@�����L	�/"�8�|����fz���H�?�I�R��CX�F��`�����ZI\�
�mM�r���d��[{!,�6�r��G z�(~���u ���^rw�?Ea6���d#�U,�@�n�K_�Ej��ga}k��Q����f��o��hD���By>h�b�k����������+Y(>�-��T��^��x��F���
�R���������Q(��U�`h*7e��D�K���B	nd�W�-M�����s��$�
p�8��=�i�4$0';,kB:&�~����GA/%3�i������}�;&/�TLG%�HK*��#�|�����b��r#y�N�����mdl��/�1���'����M�)���y ��TST�^��t�k�^}��x���(�#�j������%���WNh_�X��L*d!���}I�\�Q�E�?r�V2�*,�� �5%"��X�;�g�=c>]ed5g�����t`1�r������7��������Z]I���	YxZ���1>�P������j��H(���WX�?�O�Q�
"�g��on�m�v�k�h=��j��"Zj����^Fj�'��o��|���?�C����(�/�
��c;q^F�1�N���E��1>��h�c����A��j��������_��$�3����:�|�P��6e\��=��OI�SB)l�0R����<�;m��^t�����C��iO���S��$��q;Fd�'��lf�n��
�6��^�pC��%��[Oh��qy��������\�I�b�f�/��sR���DT���02x��^5����<�V�����{�����. ������e^�?��Ui������	L>���������of^\n�,�(��9G65�r'�B����0"u��Z�n[n]\�W�	9�-Egl���� 3���t����O���XE[���D�%;"�"��S\�����M@ar9�$c����&��g�l�>�(�]D��(�R�������m��,s�3�;zwp|]~8�D��H����������A���i}��>Is��1]`����h�<(���#�\�@A�ptH#�5��6���������-��)5�(?�
d ��6��a��$��lqX�U��a����a�r57�\lZ��e9�91�L�����\��T��j�+IV�)�<��vI��n��v��@%7��`����k��b����'�"�Y�@��;D����JiE���qT=�B���Rk���u��.�>�Xd�)t��������Qq���$�1��FC�$���,Ds�`��=������Z�c��	�$)�x��
$Ha��8��&�t
q�:�57 �A�$MC�qi{aS��,�y��-Ep�
���`F�]��MXL�����#��!M�2E��LB��G�@-Mf��c$'`|w�#��"�����8��������\H���9��|$�����w�q���#�06����mb��dUh�v6_�o��l
�b@Gr)(r��nta�B�^�����g������^/3��~�����>�P��ca�M�z�$��f��T)��dm�����:�Y��	����$-P��T��i"���B6O����f��57w7r�BR�L
�����8��������n'�^^T|Em��i���f
��~}���@�����W��������b��S��O��;�H�����)��"�c���hj<g�y����4��_U[�u��}Po]�4~�ecu�Ci��"�^=*�cl��$etC��Fb�����g�G	+�Q�H(�-��K��h��d��<�Yp��`�(�.=?�<-^fE����7�Z����������7�&����wG9�O"$tx�L�#�����]�A�S�Gnz�G���/U����}Y,v�^%%��eK����r�����S��i�[��8�v�W�W���� ��b�EB"_����������0��$��g�2���(�y�Q*�������������uCNa�t �#�.��o����R������GS�|�)���	&��Y����H�Jo���f���Q�A�,�cw&s}�EM�"��)��}���7�m9
8wGzs�[4��y�������n
��"�M���{���&c���3�W����7p��eB�E���.�fR{&M(�S�e�q��26�u��1�^���D��_
�����Xs����hV����o���J�
h���$�#����Vg�U�0&�"�h�V�
�b�8���$���[�#I�8,�T�E&-���ZV'������4�l�<W%w�p){�A�SA
7�9��k�yic��EDQ��n�|y$�e���o�!Dw��B�2Zdf�V
t�%C�@>p�ad:IW
�X,QW���t#�\s=��)�IT�="7�%��z��"��.�sO��/��B���G�VC�7fBRB�AR��*�w�=z�Rf�A(�O�������~\�B�K�y�;�hy3^��dd�����N,j�_2��m�^o��������#�X��
3�$�O�_�7��<Pvhb���.��h.�t�)iV���n��A�3fDf�$n���	X��leF�W2�0�"��)�2����hS�-|�U�	����W%Z�l���� �O�yB0�h����t�a��LQNg/�vf>����uM)	E���\|b�/^l��%l�������(a8{�����2\�Vv�s����m"�B����GpO�ydwW/�f�^�K!(W��f�(���$%��
�9�F{@��}���	1�did9"2������8���I����r#K4���vDR�����`?li���,�F�&����:)s��&��(6#�`jS�m?��e)L����O?������3g	���Qz[N���J&���������P�H_��1��_m3���z����N;�m�7�Te���(n;��z��;>����
k���^w���-4��-��2�'Z�B!k<���bo�a]D!�4��CL6[@JB �J��qg�
l��E��:
�f��&
l��[��,c�r����e����U�[�8�3p��%����������x3��#{��X&:���|_(��l!����y�1��q5��GV��e����N���-q��`Q���y6����ls�a�]�\��b����
L�����e��r����S��`7v6Bv���a��%��cAr������>R8��x��%`�����P&C9��r	5B��KV���hN�e���0�5������J���U!s^�
'�C4�~��5�xt~�fot�g�G��]*im���H0�n1)
��^��5���r�3�m�E�i�v�������}�I�sWw��$��E�"(S��0V���[�9������Pd?C��M��r��o7Q.�.l"X�bEy��
��*A�X)�SP7������Z7����"p�'0S�� �Q�"�~��3{�u~�����p@���r���?@70�g&5#HC�9/�c:~O�9S}�k��^�wK_��
��{�)+�{aX@��u��QX~����g�=��{��"�$�����������M��I�B����qi��h�\D ��a<�=�T��A����(��1�:�����D�0	�c��P��������X�a�9~���=��M�:�����Nh�����c�����<��J���{�S����7�x#��#��_���C7�z�G��{S��0j�#�y ����0	��$3���1	7����f��,/��0���Q+V����bP�=������q�9+��>|�N��*s��Zg�4-2���7�P��t��I��W�'+;��a/�O�e'ng��a!D� ag��D��j�f��,Lu ���1�����9[E�����C��F~=�{�����%����-g�%{��C�6��T��c6*��q��i�������r6�)�+B�.|
/?�����$pb����O���`��T��c_Tk�'W����@�}"�/o�c���knb!zu���6�H���pwo<����I���L�GlmUf����k�	��el��i4��geC��r�sL.�*�/���1�`f�5>����8��g,Yw:��N��y����'���xk�k4>�3@pTU�{0��O[J�X.��L�`�@������+���$y�/D���:r�^�
~�,�hm�4�����"Z�X���t`#q~�Z�����>��)�U�n��f
���d��I�����j������(�����_IW���}�j������_f��������M�2x}}+]s���\�����>
��I4M�z������^�9�2�G�x��4}�}%��Efe0^sNS�Jp�6���NJ���R���0��	��{|ur&�zNX��x����wW�)r��b������ta'���L|cT�c<���4F���?F�e����j��zLs�B���`6�M�le��}��3R���#CFH1�X
�6�����
�,�e�	���j�����@����Z��FhmF����x�?����	c�($�E|4�p�"�.=�g�<��w;�[|w���n�)�����(���������������s�N��]]�vzWwg�����T��,q��/3����;�	����!�XC�����M�YEi�eX�t�����K���w����JG�`�(�3-T;������0�e��~��lqp/���b�;��%�7
lH�>t�d�}8Z��:����MyS���?�y���o,�9U/),�5����E����=j�m�{��V�ZB����9}���$��?�d�c@��g�t`}i3|��x#[&��������m�	��0N�t=����]V.�2-������
����9T�y��F�v�n4�]�:���f7�� �x����	��",�3������?V���E
�w�F&X�]7W.�;�j��
���7�}%���������/���<i���p0��<�q*CZ��s�5+�Z�hW��a�����5��Zvn�4�tV�t��+�+x�?�.�U`���*w�U�AnO�o���=9�8����w.Np
�w�'���q�S�|*��1�e�k�H�C�?���+�i�q�b[��9�y�`�y���:h��v�TT���V��Z�us`���
�&�U(�����8��K�?���N����PV����w�����w2�����?T����"c&��������L���J��Y������{�d�SB�lX�<�������J�a���Fa���M�M)k>�:5])����4%P��x��u;����������9����1g����W��n>�3�L�8�K� �A�O�zf=�r=[�Z�_����V���d�����]�]��8��J����`��a�z�I�����7��.�z'7���v~��L����N������}�^�t���%�S��������N����������|���O?��0��/����N���W�����-��\�����|�]������7����?�>����/�[iW�b��]����(�����*�z
VH)��3p
��DV�-���}T�*@n���>�	�i)1Ec�_T������Fr�������
�=����7@i�%��e3'����Fs�Dzb�IR2@�tN�P;��P�<:��I6M�����$���[���m��j�oT����$O�gz�O�z�Y�,�N���9�~�P�Jv�g���~�NB6m4�>>��~F��m�=�4KJ���m�+��H�W�A���O�3�����eG��R�i���%�XO�lEZ����������`�}���a	�R�"��?p�A�+-���H���N��*q�'` R�(�	���F����g!Zt����
Y�{���8��mtS���t8cO)�d�Mm��+l?�����������f���m�����k������PS(�R����g4u�
���;�}��r���y�����%Z��c��b!�������K��A��8��i�v5?�CF�K�����4�����H�+�8^=r�1N�I���"���HN����
q�S2\��~���	�.;�=�}�t��E�����5\���H�P0��LM�H���h�60��n�8>��-T��L'�+�0�e_R�.j(���LK
��IX�Nn�#�2�p0<������"���a6�#��qkL���a��*k�:���I��R,�es����6HU
�y� lA<CX�R��4�c[��*|7jj}��x�������R���/�����y�
 G�I��b��Js�	wv�yx�{"�D�����Nvm��v��w���<,��G���e]bd���;k�����~f������Z��/*j��/����T�q��������XDu+`m�d���{U%����O��s(,UJ����}�}���An�0����}�~���4�Q18���5:'����*�����4u|]��@`�5N�`���-��`d.%yUN������J�v��#���!g7��(�H���H3
��Y��Nb_D��PS�%�[F�����"�[�6U�����oj�M*$5G�
g]����j��S����%P
)J��1ja�Q�L�]{��ES0�"��������O:4�Vr�l�&��S�$��&!���9�0N�N6��0JEx=�z��$�<oV����a<����������?E�������'
��0BG|9,CgoOi]��P��jT�m�X��cb1�a���!�#���*U/"I����v�{}0����)���_���IT>\��e<����+��0����I����U��������6c��AZ�z��0J� �0��D-}g`X��Z��uz����P�j9h�P4�4(���ud� ������t���{���%,�����m!g��e`����f���'dc=I�^Wz�-�9�9��A���Us$��c�x��Fd	b-�l�/�����������P�
�;�����
Q�G8�����P�����#�ksN��Z��\9C�w�|�1';Y�e���.���Y�b�]kY�vhkF`S�b�u&nS�M��j�Z����=*��a�\-���&n��Z�U�s�b
N�v�U��
nV�&���UHfc5�K0�e?+��1������l6{�[�|��Q����b6A�s8D^>�Ao����A���k�Qm[����Z/2|�����c���a�5�`#+�Z#p�
�M��<�)�7]M�TJ�oZ,�0j40�o�Q��b����p����>��gCx����J�e��gB�G��A����$��6[�V����:a�yk��tf���T,&����7*f����+0�Y�}usK�&Q��7YS���GL�l?�I��)���7r�����[�e�C�#l��'�\����#�	�Vd���r`z�P-6�u������Y��\�lpS��Ls�`$�
��H���t_!���1���PF�����1R�F�����!+���h�������s���q <��?4k������E(k�#�Z����W�w�����������B5v��j	���
���eOHi�g�r���_�V��o���ZI�W�8��H,���c�������DuN$�Y��|��9u�wu��w����#�4Z���CR�"�=���u��j�.��R���h<8mu�F�����v8���;��M��1�\e���f�����kz��#x8NA�$��Q�m��~
�ZH-'�PS#	���?s`��M+;����oD<C��������N��Uj��F�Xm<��&�3�h��{�� O2^�1���G��1C��N�:�@jg����[�������ZbB3U ?���V��h,�h���
�W@T�L��Af�����tA���O�@ ���&����I_iL�v)���Sx�p5�x����i�Y��h��zlI�;���d71,��U��My�ne������A��1[�����.Jl��@��[�W���jd�|�[@v
�������2Pqp?9��I�1���b|�n�AX>���������h�|n�s+������K)Y��_)���^�������\�-7
��IY��I#���Z�q���,�E�m�"����k���{4i���&kY���M������$z������������u�x#����Nj;�U�Y!c;�Th�j�)�������u�;zqp�2�2�h���i�(I/lF L��MQ����GM_8��	z|5u���w�VP�rWk�������<�E���`Z*��U�U�j�f�~e�(����W3�+��g��R���Bc!�a�y��g���&���R��)9��q����=_�����b����A$08�����i:�q�Bf���aV{�z-j
�yI�N6 �����u�o�c�����)4�OQ�"J�����z���a��Vv���M�lj���9�h3q��Bn���������N����\��d��_�.7�5u8K�ZU*�aX*5j�r;Uak������Tm�T5y��K�I�i)����T����=@��=7��U�w<$�K�[�d��J��6�x��y�Qn��Q-���|���M��L���^��~���o����b@}���E�'�7����#�D;%���fC�Y5��C�u���}�;��,�Vc�L���p���`�W���f�O��X��I��5Si��A�9j�K�Z��F95����$����K1HA�b��O{�<F��(�*���{���tE��q^�������Kta���l$��`���hc�@~hH�=���B �,~1���\O]��:���w�KR�&�p���0���b�]x�<����m���o�h���U�w�BF�M��������HvZ�=a��O��D��x�S�$��C�� o�)Eo�Z� ��*���>p���nj�6([��=�q,���d����������9B����:�Z�	��#����*�l�Z���[��t}r��#����� #9����Z��}R���y��A��t.���0^�v~q�)����R35�Q%j7��Ri�l�Z�FT���R�G+-/}��u���hM�I~�T(�=pj����`�v��oy~s">�Q��Xx�{���=�;�t_!?B�eo*O_rNw:�B�";d�����iz�Y����������+\�.=Xq�����'wH;\G����i.=�����I�s|�G����������N7�b7"��'����!)7�'<��z��oL���{~���d�y1�'���'Yy"��z��'���'Y���N��I���0���
��E�/(1��Ds8��
yJoNt-�#�����a��0���{�������c�3��������~<'�����p%f�����}t8y�_N>&\j����(j���T��ZM�u��F����e�;��Zz���]��&?PQ�J�{��-�)x2j�Vsv�.��*Z|�
��'O�����eQ�,���UO�z��z��/�A��:RY�|�����1�]�iwvUT��2�TC�
���1�G������J�Pv��(������l^��U|�8����A������Wk����hW��R�>,��p��XKR�����������]�q4�O����}QZO���"���������n@r@����E�������E���UT������M������;���7��u��|
�
��I��7�������#��n�2NITD����+Z+��p�&��`\���V�qI�I�vh�-�~o}��e���BQ�q6���}x������@��l��#`�/��k�PsL ���|/�S?MJ�K�8���l�����1���QuT��J�Z���V� ����R2�A��,$U6�l�>P�r�\����^�6�XnGh���,����\
6x����# ������\��3�/)���� �J�v�._'+0�]I�L��Z�G���47�7�!��*�����r��Qm�o�*�Z�J�V�Vo�*��C)�����:�c���?��w���l!<P�����h4�F[[�*x��������fb����`��M��QE��)����)����i�d1�������
��`-CH{?��$��c�H�!��a���H.�>�Y��Q8���!�`1#�i� Dp���
,�������Fl�BRm�<�� �4��tW�_)
�w	m)�������H~v��� �����%����Z���2�V��B���k���q� Q�M�}H�*�������JE��(p�c���~��S�W"��>�h�������T]I*�\j�zj(��Ri��F#I}������9����J/U_-�:DH�$��mG�&<��%�R��_�f9��K)�6��w�����{������h�.m�e~��l�t�B��l���Bs\
F,0F4.)��T��:�!�%���0|��U*� ���1��x�L�*eSp���-�x��R���J�������@�����T�_��tV\�F�F�TW 0��fTl���N�W�8	�E��*hE�F��_�����j:��r@�9?�F�+;pz�Z��}�|l
v3��+�8��=�(��+�����u���:������.{�����" �Y��	���cR�9XPv���qN|�>
�'��#�Q@N*��>At�W
c	�,S�p9h���x��H7�z��E����bB�1eg��P���9�Z����G
-2�Y�&��P-N�Q�,�s�CX<�����_��
u30�^�p��_L,b�Z����t6�f�W.!c������A���&K���d�Xr�B
��l:T05&�<"[�&]��I�c
����6��������)�wX%x��A��d��3.O����GL�A�[��mm����e��,��b(*�-����m��54���N,u�H,�w9H
�4hBO�#~I��VA	C�����:wzm-t
������2�L��93��;o�<�"7�����	�g��* T #z���^�����C8G6J���r1!�9�@`HM��k[9��)�#���~(��.�^��-I�D�`��8�Oj!�����"�����y�N�*��a�E���X��1��G5z�c��$�����o��03�)K��f��6�*�l��P����9�C�i�?P�G��r�6AQ���x���O��)�>m��f�j�W�/$��$(����E����r>�v���g�;��aGy���Bf[WH���������^)�j�M��K|yd7]��u��D�o���*�F�8Z�$I�+�:
����Y�s{v}q�U����P��1������U���S���Q�m�[rj���!�)GkF��A�r�;���|�8��@�0Rz�"�V"�pN)ta1i[��8���4�P>x�2��
���:,8\�	��Pf���m�V`&L 50U� ���U������}����R��iRy�"����`%�l�]:��� sY������x�Qmj$�F��(?������&���Q������6g�;����*=T>��V!����gb��@����)i;4*�y��&�U����0���7{p��(_(��I�$��03�=fP��
T~H� ��/8m+B���\dG�% {�g]|�:�f='q�ZE���{���LF�<D��Cq+��E�[�H8^�������������\
�
�a�U�o'��?�C+��0����b���`a0dkqq��%!SOsJ��,2�YZ����2�G��zd���u����f�������}�(��3��Y��E���#����k�L/��6���I��c��Z1���J����%^�ou�p~��N���,����������iI���G������|c��3��Y���c���H�y�1��������o��M�����G��j5��`d��"�'w���)0�k��@�S�#��La������H�=6'4 b��+^�����S��H�g.X��U`Jxt��U>���|�kS��fl�����j^t�����31�zdP�5PGd%P�>kW��vl��)?���G����<��f�n�w�@�B����O����>a_��O"�E��yz+$X8�
f���/�J��m8�R�M����5vGK�y�'^
/\�.A���i|p]mnBW�N�|*T�[t�=R�<+�V��
��%��|q��'��1�8(�c��1<��)�	 �0�8�G�t�:zSpc'z�SeS�HSWY��T��6�ck�uxv�!�J�w�����C���5�g�)0��L�:F� ���x
�)�����1e1� �������<���:��psq|~���GlF����5J�5���Je�
V�n`����>]!J
x.wx�i���W
�zpRK�1�A�C@s�u��1���*�0~O�����c��U�b������xI@�R�/�8as �I]���J�W#J<��i�Y�|��g�z���u.X�wk"��7��|�<�H���%j��n�1eA�Y�o.c	��t9?Z�cJ�L�+����u�����K���z|����{�|M�:;��%�������JF-���{�����^�{l�AM����I���������
�z"�+�������*>#�0��d�|XKX�-�F�w������t*���{�e�h���t�����EKI���M������<x�P(�G����
G������~PCi0(��qB8������m������}5�<T�$��*T|�K9T��+h�xY�����@��P����k��.�w3:����D�%��\"zh�U�GV��p�z���������S]������k�.W2j�}�zYA3�u��}(Lw��B��Gd��U���<|�O���!
�����YO��%���F�/��lo���Zi70���p�-��7*h�e�u���]��+�w�]}}
)�v�������W�L�������i������_��C�*�-U�Z����$�H���b���\t��X+���Z��+%��]��srN���]��0U#/^H�m<��f]p�
'K��GI�k����I�0���~cP/�*#��Zk�G�
\�,Z9�IV-��m#oP��1���%��i���b6��c��L��V�F�R1���}����W��"_7|�,ZA��|�����H9	��)��m��uQ�?��������LCB����5�����S���GU?Ps�l:��r&&����v:��3��L���?���&���-tS2������:���Y��'/��o��)4|Y��V����)�,B�w��#C�j���LD�r������'�<�����|A�{'e��#���0����x��&��^v}7�E�1�i�
.�x��D������RI�k�v�5 �g5hT�� ��7��L��@����5L�$Tw>��	V��6s������L���|iWR�`��	���f\���z���p���(�d��:9��C��R�v��Zg�k���u��6U��Q<����Y.����5x@q������$���>��M��/e�f��r�;N�d\IK����j�ff�=`��r��#
�|Mb.[N��e�t�GMn�S���1Zh�Q��h�{��Dn�o�������2��9����(�)��s�r���l��m���R�n7��V����5M)-U�u\�5Hd����R2�!�8�[y�#"�%%#�W���Y��"��a�y�0"/�~/fS���UZD��b|�r����OU�,���9��5�S1?�(�E,�TN��z	�<Z���?9�R9Y�i?C�BO`��,W���������m�	��hSg%��)8�	*��k�e��?��K<���?���G�rP~{I��J������� ��^!��������G������v��GA�G-7>C0��_�]�]��GW"
��P��t�/���*
.S��6�_��%D�J�� ����uC���J�1��q�Cr�����'�wO��:�l�(������;�Qs�V�R�T�A�2*W�Jf�R�d������?P6'9�`dPt���I���0h@
���������t�E�qh�T�H��aZ����d�8�_����rL�C�c�T���,�N�|�H�o�.I`^��$��<�E�K�p���Q�����	�<�4�p�P��)�xU|�|1�0�P��vIW���J�E�:3��I*�Q��U��q�g�X������ �<@*T���
��A�f���qiEr��P}��%���8����F;SK�r=�
�GC&k$���
e���E�2=�JMv<���`:^.9Hv��\%��	�=Z�,�-�M��{�"@�����r��"u#�~
!��L5�'���@`L0%a�/������FbVo�k�;���\Q������)VV��`��r�)����_n6���:�e��ZV�c#�N���,��M�s��G�mP�e��}0�8u
�O_HC-��o��(��!Ocg:��4n������4SvC��YY��p��Q����7�XF���e�����;�]����wfC��4:�~�Q��� x��cN2�\�O������(���F��J�f{�(��-'�?E(���*8-����y`���8�y����t�!��8�>����t�����R�S���x0�%�����3"����7����?�f#H�Q�z�Vi���$�W:/P�lt�B����
~{zqq~ys��l�@��0��
������[����Y}�aN����)|}o"�Z����U������`>�?��v������A6L����)��b�<��������Le&MS���|]i��
�j*���j�j	��g�L�OJ�%���:��@��,��2�TQ��fEJ�F��*�F�Q�Y�5��&2�"�'��j�C[!��8�e���V������Am[So3������\���Z}=_r��g�c�&yA}��/�����o@@7Z�~X-7C�\�@z�u��eB�y��I-@v���{�.�!z���+w�b����\���D��g
��	N�-d�'_�����Y�N�b���(D/*_�������Jni5���)��%����'�8o<\�������%�HN89;�J���{�Ww����E�zI��$x;Y����o���r9l��R�
�����?�^?)B�u��BfpN���:,'��K��v��0F�Di����������eSvq}��,�������t;�r|��u��6K�Z�vz{�����b��L��R
vy��4�3��!��k��������-G���A�Y.���z��A���9e3��?�Hp5< ���R�m�Pj$��I�n+�����(���#@���Q~������=��{})���} �'�9�VX���'���u�[����T��N������(��k�|
��Me��"}�FGN�������cl�.�C
������T
�����+�O����]���>�����**X���4e%�R|+�����?��BN�UN%����K�o@
�����N���$���j���L:����Y�<�h�si��9��
��������T�����74������X���7RF�Wz&q�6z��H�/���BP�e�����I������6�����^$�`,�E������^>��db�+�W�g���h�|^N���~=~���C������Sp�\��A �~}���������U���./���^A��=8�''�y���L��
^D��(��VK�Z#����p	;{�h���)��m�E
���*@����@�0]����^w_��^4��syf�?M���x����J��>�����P��$u����Uu���9jW�zU���(E�f�Y���mD4���U]aN�U]�Xi��0-PaJ�IA��L#����BpE��������ot����������������/(��	/�����j�����?���������y�<�?���������y�<�?���������y�<�?���������y�<�?_����Y,�
#8Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#4)
2 attachment(s)
Re: Parallel Hash take II

Here's a new rebased and debugged patch set.

On Tue, Aug 1, 2017 at 1:11 PM, Andres Freund <andres@anarazel.de> wrote:

- Echoing concerns from other threads (Robert: ping): I'm doubtful that
it makes sense to size the number of parallel workers solely based on
the parallel scan node's size. I don't think it's this patch's job to
change that, but to me it seriously amplifys that - I'd bet there's a
lot of cases with nontrivial joins where the benefit from parallelism
on the join level is bigger than on the scan level itself. And the
number of rows in the upper nodes might also be bigger than on the
scan node level, making it more important to have higher number of
nodes.

Agreed that this is bogus. The number of workers is really determined
by the outer path (the probe side), except that if the inner path (the
build side) is not big enough to warrant parallel workers at all then
parallelism is inhibited on that side. That prevents small tables
from being loaded by Parallel Hash. That is something we want, but
it's probably not doing it for the right reasons with the right
threshold -- about which more below.

- If I understand the code in initial_cost_hashjoin() correctly, we
count the synchronization overhead once, independent of the number of
workers. But on the other hand we calculate the throughput by
dividing by the number of workers. Do you think that's right?

It's how long you think the average participant will have to wait for
the last participant to arrive, and I think that's mainly determined
by the parallel grain, not the number of workers. If you're a work
that has reached the end of a scan, the best case is that every other
worker has already reached the end too and the worst case is that
another worker read the last granule (currently page) just before you
hit the end, so you'll have to wait for it to process a granule's
worth of work.

To show this I used dtrace to measure the number of microseconds spent
waiting at the barrier before probing while running a 5 million row
self-join 100 times, and got the following histograms:

1 worker:

value ------------- Distribution ------------- count
< 0 | 0
0 |@@@@@@@@@@@@@@@@@@@@@@ 110
20 | 1
40 |@ 5
60 |@@@@@ 24
80 |@@@ 14
100 |@ 5
120 |@@@ 16
140 |@@@ 17
160 |@@ 8
180 | 0

2 workers:

value ------------- Distribution ------------- count
< 0 | 0
0 |@@@@@@@@@@@@@@ 107
20 | 1
40 |@@@ 21
60 |@@@ 25
80 |@@ 16
100 |@@ 14
120 |@@@@@ 38
140 |@@@@@@@ 51
160 |@@@ 20
180 | 3
200 | 1
220 | 3
240 | 0

3 workers:

value ------------- Distribution ------------- count
< 0 | 0
0 |@@@@@@@@@@@ 113
20 |@@ 15
40 |@@@ 29
60 |@@@@ 35
80 |@@@@ 37
100 |@@@@@@ 56
120 |@@@@@ 51
140 |@@@ 31
160 |@@ 21
180 |@ 6

4 workers:

value ------------- Distribution ------------- count
< 0 | 0
0 |@@@@@@@@@@ 121
20 | 4
40 |@@@ 39
60 |@@ 29
80 |@@ 24
100 |@@@@@@@ 88
120 |@@@@@@@ 82
140 |@@@@@ 58
160 |@@ 26
180 |@ 15
200 |@ 9
220 | 4
240 | 1
260 | 0

I didn't know what to expect above my machine's core count of 4, but
this is for 8:

value ------------- Distribution ------------- count
< 0 | 0
0 |@@@@@ 116
20 | 2
40 |@@ 36
60 |@@@ 69
80 |@@@@ 95
100 |@@@@@ 113
120 |@@@ 74
140 |@@@ 71
160 |@@ 44
180 |@@ 36
200 |@ 30
220 |@ 14
240 |@ 18
260 | 8
280 | 3
300 | 4

It's true that the fraction of waits that go into the 0-20us bucket
(because the last to arrive at a barrier doesn't have to wait at all)
decreases as you add more workers, but above 1 worker the main story
is the bell curve (?) we see clustered around 100-120us, and it
doesn't seem to be moving.

If we call the fraction of samples outside the 0-20us bucket
"wait_probability" and call their average wait time
"expected_wait_cost", then one way to estimate this is something like:

wait_probability * expected_wait_cost
= (1 - 1 / participants) * (tuples_per_grain * cost_per_tuple * 0.5)

I don't think we can do that today, because we don't have access to
tuples_per_grain from the subplan. That would in theory come
ultimately from the scan, adjusted as appropriate by selectivity
estimates. The grain could in future be more than one page at a time
as proposed by David Rowley and others, or "it's complicated" for a
Parallel Append. But I'm not sure if that's correct, doable or worth
doing, hence my attempt to provide a single knob to model this for
now.

I did some experiments to find a value of
parallel_synchronization_cost that avoids Parallel Hash when it won't
pay off, like this:

* a "big" table with 1 million rows to be the outer relation
* a "small" table with a range of sizes from 5k to 100k rows to hash
* both tables have a unique integer key "a" and a 60 byte text column "b"
* query (a): SELECT COUNT(*) FROM big JOIN small USING (a)
* query (b): ... WHERE length(small.b) * 2 - len(small.b) = length(small.b)
* work_mem set high enough that we never have multiple batches
* one warmup run and then the median of 3 measurements
* all default except min_parallel_table_scan_size = 0
* 4 core developer machine
* -O2, no asserts

Just to be clear: The following number aren't supposed to be
impressive and are way shorter than the queries that Parallel Hash
feature is really intended to help with. That's because we're
searching for the threshold below which Parallel Hash *doesn't* help,
and that involves running queries where there isn't much to hash. The
times are for the complete query (ie include probing too, not just the
hash table build), and show "parallel-oblivious-hash-join-time ->
parallel-aware-hash-join-time" for queries "a" and "b" on patched
master. I also compared with unpatched master to confirm that the
parallel-oblivious times on the left of the arrows match unpatched
master's, modulo a bit of noise.

1 worker:

5,000 rows hashed: (a) 157ms -> 166ms, (b) 166ms -> 183ms
7,500 rows hashed: (a) 162ms -> 174ms, (b) 176ms -> 182ms
10,000 rows hashed: (a) 161ms -> 170ms, (b) 181ms -> 210ms
12,500 rows hashed: (a) 169ms -> 175ms, (b) 194ms -> 188ms
15,000 rows hashed: (a) 175ms -> 181ms, (b) 199ms -> 195ms
17,500 rows hashed: (a) 173ms -> 175ms, (b) 201ms -> 202ms
20,000 rows hashed: (a) 179ms -> 179ms, (b) 210ms -> 195ms <== a & b threshold
30,000 rows hashed: (a) 196ms -> 192ms, (b) 244ms -> 218ms
40,000 rows hashed: (a) 201ms -> 197ms, (b) 265ms -> 228ms
50,000 rows hashed: (a) 217ms -> 251ms, (b) 294ms -> 249ms
60,000 rows hashed: (a) 228ms -> 222ms, (b) 324ms -> 268ms
70,000 rows hashed: (a) 230ms -> 214ms, (b) 338ms -> 275ms
80,000 rows hashed: (a) 243ms -> 229ms, (b) 366ms -> 291ms
90,000 rows hashed: (a) 256ms -> 239ms, (b) 391ms -> 311ms
100,000 rows hashed: (a) 266ms -> 248ms, (b) 420ms -> 326ms

2 workers:

5,000 rows hashed: (a) 110ms -> 115ms, (b) 118ms -> 127ms
7,500 rows hashed: (a) 115ms -> 128ms, (b) 131ms -> 128ms
10,000 rows hashed: (a) 114ms -> 116ms, (b) 135ms -> 148ms
12,500 rows hashed: (a) 126ms -> 126ms, (b) 145ms -> 131ms
15,000 rows hashed: (a) 134ms -> 142ms, (b) 151ms -> 134ms
17,500 rows hashed: (a) 125ms -> 122ms, (b) 153ms -> 147ms <== a & b threshold
20,000 rows hashed: (a) 126ms -> 124ms, (b) 160ms -> 136ms
30,000 rows hashed: (a) 144ms -> 132ms, (b) 191ms -> 152ms
40,000 rows hashed: (a) 165ms -> 151ms, (b) 213ms -> 158ms
50,000 rows hashed: (a) 161ms -> 143ms, (b) 240ms -> 171ms
60,000 rows hashed: (a) 171ms -> 150ms, (b) 266ms -> 186ms
70,000 rows hashed: (a) 176ms -> 151ms, (b) 283ms -> 190ms
80,000 rows hashed: (a) 181ms -> 156ms, (b) 315ms -> 204ms
90,000 rows hashed: (a) 189ms -> 164ms, (b) 338ms -> 214ms
100,000 rows hashed: (a) 207ms -> 177ms, (b) 362ms -> 232ms

3 workers:

5,000 rows hashed: (a) 90ms -> 103ms, (b) 107ms -> 118ms
7,500 rows hashed: (a) 106ms -> 104ms, (b) 115ms -> 118ms
10,000 rows hashed: (a) 100ms -> 95ms, (b) 121ms -> 110ms <== b threshold
12,500 rows hashed: (a) 103ms -> 120ms, (b) 134ms -> 113ms
15,000 rows hashed: (a) 134ms -> 110ms, (b) 142ms -> 116ms <== a threshold
17,500 rows hashed: (a) 110ms -> 104ms, (b) 146ms -> 123ms
20,000 rows hashed: (a) 107ms -> 103ms, (b) 151ms -> 120ms
30,000 rows hashed: (a) 124ms -> 110ms, (b) 183ms -> 135ms
40,000 rows hashed: (a) 125ms -> 108ms, (b) 209ms -> 137ms
50,000 rows hashed: (a) 133ms -> 115ms, (b) 238ms -> 150ms
60,000 rows hashed: (a) 143ms -> 119ms, (b) 266ms -> 159ms
70,000 rows hashed: (a) 146ms -> 120ms, (b) 288ms -> 165ms
80,000 rows hashed: (a) 150ms -> 129ms, (b) 316ms -> 176ms
90,000 rows hashed: (a) 159ms -> 126ms, (b) 343ms -> 187ms
100,000 rows hashed: (a) 176ms -> 136ms, (b) 370ms -> 195ms

4 workers:

5,000 rows hashed: (a) 93ms -> 103ms, (b) 109ms -> 117ms
7,500 rows hashed: (a) 106ms -> 102ms, (b) 121ms -> 115ms <== b threshold
10,000 rows hashed: (a) 99ms -> 100ms, (b) 126ms -> 113ms
12,500 rows hashed: (a) 107ms -> 102ms, (b) 137ms -> 117ms <== a threshold
15,000 rows hashed: (a) 111ms -> 107ms, (b) 145ms -> 115ms
17,500 rows hashed: (a) 110ms -> 10ms, (b) 151ms -> 118ms
20,000 rows hashed: (a) 108ms -> 103ms, (b) 160ms -> 120ms
30,000 rows hashed: (a) 120ms -> 108ms, (b) 196ms -> 127ms
40,000 rows hashed: (a) 129ms -> 109ms, (b) 225ms -> 134ms
50,000 rows hashed: (a) 140ms -> 121ms, (b) 262ms -> 148ms
60,000 rows hashed: (a) 152ms -> 123ms, (b) 294ms -> 154ms
70,000 rows hashed: (a) 157ms -> 122ms, (b) 322ms -> 165ms
80,000 rows hashed: (a) 154ms -> 138ms, (b) 372ms -> 201ms
90,000 rows hashed: (a) 186ms -> 122ms, (b) 408ms -> 180ms
100,000 rows hashed: (a) 170ms -> 124ms, (b) 421ms -> 186ms

I found that a good value of parallel_synchronization_cost that
enables Parallel Hash somewhere around those thresholds is 250 for
these test queries, so I have set that as the default in the new patch
set.

All of this might be considered moot, because I still needed to frob
min_parallel_table_scan_size to get a Parallel Hash below 90,0000 rows
anyway due to the policies in compute_parallel_worker(). So really
there is no danger of tables like the TPC-H "nation" and "region"
tables being loaded by Parallel Hash even if you set
parallel_synchronization_cost to 0, and probably no reason to worry to
much about its default value for now. It could probably be argued
that we shouldn't have the GUC at all, but at least it provides a
handy way to enable and disable Parallel Hash!

One hidden factor here is that it takes a while for workers to start
up and the leader can scan thousands of rows before they arrive. This
effect will presumably be exaggerated on systems with slow
fork/equivalent (Windows, some commercial Unices IIRC), and minimised
by someone writing a patch to reuse parallel workers. I haven't tried
to investigate that effect because it doesn't seem very interesting or
likely to persist but it may contribute the experimental thresholds I
observed.

- I haven't really grokked the deadlock issue you address. Could you
expand the comments on that? Possibly somewhere central referenced by
the various parts.

The central place is leader_gate.c. What's wrong with the explanation in there?

Let me restate the problem here, and the three solutions I considered:

Problem: The leader must never be allowed to wait for other
participants that have emitted tuples (it doesn't matter whether that
waiting takes the form of latches, condition variables, barriers,
shm_queues or anything else). Any participant that has emitted tuples
might currently be blocked waiting for the leader to drain the tuple
queue, so a deadlock could be created.

Concrete example: In this case, once we get past PHJ_PHASE_PROBING we
have to allow only the leader or the workers to continue. Otherwise
some worker might be trying to finish probing by emitting tuples,
while the leader might be in BarrierWait() waiting for everyone to
finish probing. This problems affects only outer joins (they have
wait to start PHJ_PHASE_UNMATCHED after probing) and multibatch joins
(they wait to be able to load the next batch).

Solution 1: LeaderGate is a simple mechanism for reaching consensus
on whether the leader or a set of workers will be allowed to run after
a certain point, in this case the end of probing. Concretely this
means that either the leader or any workers will drop out early at
that point, leaving nothing left to do. This is made slightly more
complicated by the fact that we don't know up front if there are any
workers yet.

Solution 2: Teach tuple queues to spill to disk instead of blocking
when full. I think this behaviour should probably only be activated
while the leader is running the plan rather than draining tuple
queues; the current block-when-full behaviour would still be
appropriate if the leader is simply unable to drain queues fast
enough. Then the deadlock risk would go away.

Solution 3: An asynchronous executor model where you don't actually
wait synchronously at barriers -- instead you detach and go and do
something else, but come back and reattach when there is progress to
be made. I have some ideas about that but they are dependent on the
async execution project reaching a fairly advanced state first.

When I wrote it, I figured that leader_gate.c was cheap and would do
for now, but I have to admit that it's quite confusing and it sucks
that later batches lose a core. I'm now thinking that 2 may be a
better idea. My first thought is that Gather needs a way to advertise
that it's busy while running the plan, shm_mq needs a slightly
different all-or-nothing nowait mode, and TupleQueue needs to write to
a shared tuplestore or other temp file-backed mechanism when
appropriate. Thoughts?

- maybe I'm overly paranoid, but it might not be bad to add some extra
checks for ExecReScanHashJoin ensuring that it doesn't get called when
workers are still doing something.

Check out ExecReScanGather(): it shuts down and waits for all workers
to complete, which makes the assumptions in ExecReScanHashJoin() true.
If a node below Gather but above Hash Join could initiate a rescan
then the assumptions would not hold. I am not sure what it would mean
though and we don't generate any such plans today to my knowledge. It
doesn't seem to make sense for the inner side of Nested Loop to be
partial. Have I missed something here?

It looks like some details may have changed here due to 41b0dd98 and
nearby commits, and I may need to implement at least ReInitializeDSM.

I also need a regression test to hit the rescan but I'm not sure how
to write one currently. In an earlier version of this patch set I
could do it by setting shared_tuple_cost (a GUC I no longer have) to a
negative number, which essentially turned our optimiser into a
pessimiser capable of producing a nested loop that rescans a gather
node, forking workers for every row...

- seems like you're dereffing tuple unnecessarily here:

+               tuple = (HashJoinTuple)
+                       dsa_get_address(hashtable->area, detached_chain_shared);
+               ExecHashTransferSkewTuples(hashtable, detached_chain,

Yes, here lurked a bug, fixed.

- The names here could probably improved some:
+               case WAIT_EVENT_HASH_SHRINKING1:
+                       event_name = "Hash/Shrinking1";
+                       break;
+               case WAIT_EVENT_HASH_SHRINKING2:
+                       event_name = "Hash/Shrinking2";
+                       break;
+               case WAIT_EVENT_HASH_SHRINKING3:
+                       event_name = "Hash/Shrinking3";
+                       break;
+               case WAIT_EVENT_HASH_SHRINKING4:
+                       event_name = "Hash/Shrinking4";

Fixed.

- why are we restricting rows_total bit to parallel aware?

+       /*
+        * If parallel-aware, the executor will also need an estimate of the total
+        * number of rows expected from all participants so that it can size the
+        * shared hash table.
+        */
+       if (best_path->jpath.path.parallel_aware)
+       {
+               hash_plan->plan.parallel_aware = true;
+               hash_plan->rows_total = best_path->inner_rows_total;
+       }
+

I could set it unconditionally and then skip this bit that receives the number:

rows = node->plan.parallel_aware ? node->rows_total : outerNode->plan_rows;

Do you think it would be better to push plan_rows_total into Plan instead?

- seems we need a few more test - I don't think the existing tests are
properly going to exercise the skew stuff, multiple batches, etc?
This is nontrivial code, I'd really like to see a high test coverage
of the new code.

I've added some regression tests in a patch to apply before making any
changes. You would have to change the "explain (costs off)" to
"explain analyze" to verify the claims I put in comments about the
number of batches and peak memory usage in the work_mem management
tests. I chose to put them into join.sql, and then a later patch adds
parallel-aware versions. (An alternative would be to put them into
select.sql and select_parallel.sql, but it seemed better to keep the
non-parallel, parallel with parallel-oblivious join and parallel-aware
cases next to each other.)

While testing I found a timing bug that could produce incorrect query
results because of the empty hash table optimisation, because it had
an incorrect value hashtable->totalTuples == 0. Fixed (see code in
the "finish:" case in MultiExecHash()).

Last week I finally figured out a way to test different startup
timings, considering the complexity created by the "flash mob" problem
I described when I first proposed dynamic barriers[1]/messages/by-id/CAEepm=3yJ65sQZUAhfF3S7UfEv83X_rnH5a4-JXmqxGQRQ+7qQ@mail.gmail.com. If you build
with -DBARRIER_DEBUG in the attached patch set you get a new GUC
"barrier_attach_sequence" which you can set like this:

SET barrier_attach_phases = 'HashJoin.barrier:2,7,0';

That list of number tells it which phase each participant should
simulate attaching at. In that example the leader will attach at
phase 2 (PHJ_PHASE_BUILDING), worker 0 will attach at 7
(PHJ_PHASE_RESETTING_BATCH(1)) and worker 1 will attach at 0
(PHJ_PHASE_BEGINNING). Note that *someone* has to start at 0 or bad
things will happen.

Using this technique I can now use simple scripts to test every case
in the switch statements that appear in three places in the patch.
See attached file parallel-hash-attach-phases.sql.

I'm not sure whether, and if so how, to package any such tests for the
regression suite, since they require a special debug build.

Ideally I would also like to find a way to tell Gather not to run the
plan in the leader (a bit like single_copy mode, except allowing
multiple workers to run the plan, and raising an error out if no
workers could be launched).

- might not hurt to reindent before the final submission

Will do.

- Unsurprisingly, please implement the FIXME ;)

This must refer to a note about cleaning up skew buckets after they're
not needed, which I've now done.

Some other things:

Previously I failed to initialise the atomics in the shared skew hash
table correctly, and also I used memset to overwrite atomics when
loading a new batch. This worked on modern systems but would of
course fail when using emulated atomics. Fixed in the attached.

In the process I discovered that initialising and clearing large hash
tables this way is quite a lot slower than memset on my machine under
simple test conditions. I think it might be worth experimenting with
a array-oriented atomic operations that have a specialisation for 0
that just uses memset if it can (something like
pg_atomic_init_u64_array(base, stride, n, 0)). I also think it may be
interesting to parallelise the initialisation and reset of the hash
table, since I've seen cases where I have 7 backends waiting on a
barrier while one initialises a couple of GB of memory for several
seconds. Those are just small optimisations though and I'm not
planning to investigate them until after the basic patch is in
committable form.

[1]: /messages/by-id/CAEepm=3yJ65sQZUAhfF3S7UfEv83X_rnH5a4-JXmqxGQRQ+7qQ@mail.gmail.com

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

parallel-hash-v19.patchset.tgzapplication/x-gzip; name=parallel-hash-v19.patchset.tgzDownload
��Y�}kWG���
����f,�$�z����d��>���,���Kj��f���U��/!�c2�f%F��v�c���]7t�S9�]��e���n<��d��>�O���u:�~w�w�����G8-����fC4�n�i���l#X���b7�����0X��M&+��y���!?����i8�}��E�L�By�(��y-�Q��K7��2����?��6D��m������4���o9���D�k,�����<f�=����3i���7��F�����N�q�Ovn�~���\�'�����]s6�cq~��H������W��/r�p�x��8�=�|����8s��hv��� �f�q�����@�<[�~������������v�'h)-E����R Z
���R Z�q����������L�D�
�p��t�"=1�3w~#~Y���Q�JG���n�����1��
@��!6�`
��Ld��2�A\K1v�0Rw|)`y��bL��f�������"Z�\������������0�0����������=�y�Y��6E�wpx;jEv�&��d=X��7�j6��:?��_�
>��M8�n�M�
S_���UE����2�a������&����.�X�;k�~�F�M��b0i��}��e����Q�m5[�'�[���0�������Z��lT[b�7;���Tt�5+��B?��	R���MQqZ"���M�)�0X��M%<����m��
2G�$�gi��������5C�a�7��I�'���4�?���8�b����}� "��"yM]�2q������)�7 �?Z"���7@>s��0��
�8�}�h�o�on��c���@����jI.�\"%����*N�J��%��������!L���H��{p�3e$,F0����������H�~D�f��|��,����f]����H>"(q "�1�A;\�d��Kb1!r@�&�R#��B�C`i���A�>Zw��U�s�
�J��.���/����c[�����F����P����)���6
{31���~O8�%t�������1s$`L�YEMV/4
��}+~ji�"�"�k ��`�F�2$z/����%��/�"��R�r��P��8���'
����a@+�:��v%�9E.�e`Ydv��_��7���u�8"��xg$.���z�@�l[q�H�}l-h�y����4�R<g�=�
~,;�@��@�3��l���h�%ov�S�Px9�������	����H�U��`�!��{�m]6(��9�AT�z �w!c�~&��"%����
@�.��%
<@�����Q ��.���rl2���y,�q������,���	 5V|WQ���������������z?s�4��$��'B���!��2 ���`�=���#�G�k�j�]��/�U���!�4AH����2�t�����|���.���7�"���	:���i�Y������,�D}Z;��6��
I��,y�y��A��G����~T�
P�\53�����f`e��9��55����[��h(e�n�`��+b5q�� �4�%.
FA(Y�!Y(�5�E�����"f�0���A0�]�����������������2~����64�;�|������p��k���� �K6���KP����s����c��-'��wQ_.�����e����c��k��I�����Os���4rnr;�>;<_�{c�O����'�������u�CA�=;z���h����{���+����w��h2�d�kH��p��0�����Nr��.�&5�B�������F����F\u?+e��h�0�r���(��������Ach��e��:���HF�:�HK!8M30����s~�5���K��a���� ����/M�k��K|R�W"#�rS���������w!��LV3��0����������[��&���Q�������X��Nb���&�>
�'v���P3�W,��b�� ����Y1���o(m��U�'K-1�B�Td�J�Q�q�y &��q������2;����9��w�5����#k�� E,A�k*c���{1�o��C�����1�w�`%�n5D�w�w�L30d�JE��R��A����#W����������4���%�EA�������N�*?�4*�L�F�����:�3�SZ�hi?W���V���:g����F�uN{��o��LA*I�L���Ng��bF'�v:�u2:������r������^8�E�w���LY���<�k�-��[���������-��?7��?#��?!������?m���o����.N�N'�����O���L`���{�<���[���U���������}��,�����=�����G���?���-�=�e�3�)q�fp,�Z���j��o6�N�����wWp�������g�U�G/@�^��a�����}����������]'S���V��k�P��������ov��s�n���'W������;N������s~��"8S��<n
��e��T��Y`�G��`�M�����������v�<^�*Y2�!gU�Q��7��{�2�zh��;~�_�Vv]���X�����\W�M��0�����	�a��C,Ct��L�\�!�����qa^c��6id��5�=g|�l�v���FA8�{)^����|�������-%?�893��cR1��+>��R����`��8����w<]z=�d,��4(Ao�I��}Z%�����pA�[Uc���\���HM6�_�h\��yv����U���p7�e8���\���D����STI�G��8$y�.�F���) ��r	��3?/���cq��������"��hg�s�(����{����?>�/���������|+�������������.�]
O��2s�i���Y��������~�}wr���p19{��t��^�����|2���g�_n������G�����aK=w�@�6Q��D���hT:���)�H�!�Qh��������4�5����h�/jEE�V�|t[�R�9
o����n���5�^�k���rQ�`%I�ZQZA�Uu`p���9����?�������a�0�;nm�
"*�������GKx��_U�{nh�G��z9{���[��P��y$���$�
�"T��X�E�S	�"c"|q������k��z����8��v�5���v�h�#���&��@�h��j�y'�����o��~��^��^�=��W���g� c��A�_?��#.��Mk�x�r���	������,@�5-i�����8X��E���h� v�4�������/�[�A�RN��c����Z1~T�1�4�A�,0���
��x/��5��X_�^��{"���M`A��Q�m�o��7{O�����+7k�6��g��/��'���E;U�;�
��:����^(d�!`�H�����w��}CjB���"���h��N��t�v��V�f�.� �f��`*^��Q�c�q�P�>_�^|�v
�������b0�Qo����(��������uF�"���B��� %l��i�af���:JhI^U���Z�����rX�1D`�C�
HI����KNu �l���T�0B���uj����)�`��������9>��������gQi��������+g����9rMB����le��}b�4�����8��i/��<�&�7=$v��XF��F�V�4}�x=�'��v��������30�7~���1��R��Sv���Rd� ��y���.\��S��Z?��rI�f��n[�vW}��$���`�N�k��>C�W��lt�+m��J+3�(�"�xXF���MB�X[�a!���^z����G���,VE=�.�d3eR-|v�����E�;��E^SSF�Ar�EQb������N<��8����5��Q�	���0f�����G��D�'����.e5�����I������8�<�I�y9MR����������'!���m�W�&H3����0��O�1�Z>;�CY@�;.�n�A��Ua�b���b� ��e3�os����0��Rj�y.���i��4"��k�	�-	��$}��4��D���D���j��V��Q<a@��?!�V����&��K����7�(��� v���)��w�Hi���--xJ{g��`�o�d�T����n��4�h�{�Y�	E��A�l��U�9����2��j�c��mdm�K�>NB@S����u]����tn�L*�m-X�Z�� ��E��%�Yp9�*Qu�_~���SIoK	���,/���/�y��
�2i�I����lP�8�v��4rXs���U��%�&�>�.a�u��Q�~D9��8\��9�bC)�������T	��J���~DU8���HHXKg6^��?�1��3H�������xKS���7��;�A�
��'������t,��U��=!����di1���#�-�S�
?\$�}5��L��q�nd��U�q{�D��4��}����F(�{(��{n�*�P�>h�����V���������*��eC����t�#�Q���������{�4I)����D
��ix61"�|����%?����mX��RT.�VZ{1FB�'���F��J�����f���nZ��j���g���PE��"�W�F�]���2�"+���MM�����Y��Y���\���Z��^Mw�z�~g���N��n������w����8fW�b1�/m:lP�7���@��8���F�(�%��A'�����TZMMN��>��*hg��Fe
B�*|�q$l�����	
�f�����&���)]�t'��|v�'X��8���,�c��"�`V�rA ��v�	s>����B���8���.QP9>�(_�<�8,h��Xr������2��(��o����D{M�P��?1�i���,0d`�3?�Z��!���)�wEK��{yMo �)-��a��j `�0���t�o?���������/L�k����^&�R#]�n�?5�^3���tZ�o�_���?&n��ue{L|0n���4��x��,��c�3�������i=nw��?Z��AhY���2Ght�f:��^>�'c��3���K�#�n������
�WI�,�*���O��,�N0���)7u�B�u��Le>��\�cf�(�Dv�P��a`�%!Cq>FNd���0�t~��D�;%B��!2%/�(�O��Jo�v��
��}��+��^owA����3}���v�A����x�*�=�3B
dmo�G��i�����Y���+��%��s�z����<��	�Y��=����9\��)f�Y��Z4h�V��
5+b�(p��_)-��s����6����:�O�M�������1N�P��<�3�Qv������8��L�|�ap+�1��,V|v������������$�[��O�����m����#�Op�*G�������Y��-s?���`GT��1h����Vb.sFGw����9��"��F���<�R�+�D�����J��c�*�����D��}`\���\��A�������Wrg��y������1��lEa�o;��~!c&m�D�f���R��W�#��xS{�>a������4|���$;A�5P07�W��9|��~���n0v�O&vj�)�r��-��.�c/n*��,�c;��<�����LP��-��������+@�!��b� �#�wHy��wZ{*�
�Z���y�r��S��O���&��
�q���"=��I!hE�����
^y��A��W;�����_����+��N9)+1S�O	�/���}�3>-p�#��l������=G7�������!w��,M��������6����NW�����������aSOO��9?����~����~��^�5U�<o�q�m�%���1<[��f�z5�w9���V���D�A�b�YB*��}T����Dr7/��j9������V&f:9e&CS'���70�L���my2������o������p�z�@���'^�=>�p�{aR��-�Q����dd����~G�Z��e��P*&��` �����hs�'�=�1a�f�	WSF.��O��YI
��6�_,1�rc����'�����&pss�WM���?/�o>�r���+%�jF��b�fw
8%��mp��X��O������b2u/0c2�c�����
Q-���g�4�J��F�E-��':j����T���YwK�0Y�{e=`x���V���V'�.s#v���6I�Ra��5��s��J �fXU���It��S���%���L�2e|�<(ZsT�(��z����d��1qP^?���<Q'qT���@h���xQ��3�{�1��]TWh+u��U�{(�qFA_����������5�M=��Y���=��;h�*����:����et�;��Gv�l�`w�u��C4�
������#I���]�l|4*�VD�����J���(�k�\���AbQ��]nw����oY��F<h8��!AD�P�WY���
%:�&��1�/�����SQ^~�W:�����qZt-� ��[����v����h��i9���B<�(�s��E�N0���:+h
�8�I�������q�m����S�����n�2��������d��6��nH��E��u�WeN�����x�~�%J��$���}�^zV�/��2�2z &��W�I�E9�e��Q%.1�.�?R��P���t���>��1��P��M���6�!jSM2�\I��*������.�S������W��0��*�,�,�(�\"��Z�E�s�����v���f���w���x+�g�7�����p8���+����wN�K�2YK$��\��6>!���I
���)��H�����I��b:LZ��V�~��ki/���S����7\�|�O����34?���V���4"������^�k��$�ew��,���&�|V)�T�����h�{g�~#�G���$:��j��	��:G���*��jWx�%�I7E�(w���J(�U��#=�A���������M�I��1���(��H�������X?�������O��[�������I��s�R�����Vb�V9`w��rZM��2�_�v�[������/����x<����q:�v����n��o�z��;�z�Q�k�uw����6���-��d�Rg�b�E;�wn��������������C���"��T�s���ad��n�-��)���aPj������fDw��h����k���Qk�N�E�����E)\1w���a��`����.�N�Z/����T���T�tJ�Z5m:���M����r����e%z��>M:��h�|��6N��g���CG����raB	�����*#�U������B:��#;G	��g\I���!�\KS*
g����w��P*�c��(.,Cv�s�8��������
 h��<�z�|�?��~Y���
���t'��X1����;����xk�!m�c�t���{t�3^6�o�5s��5����!��]FI��,�������f"kg]�x��-�"D��W��d`��&w�����b��=Io�9j=^�����L�3�5�7F�)�����	\����+qu�>�V�U�m��p�(e�](#�P]���'��X`%��
�3��$�,����G��(���q���aW�C�e0��*Jl��:!0��T��"n�Buz��2���]�������w-sX���� FAH��U/�&���$tv-�tK����=o0/EW����8|����S�7�'#V�e7����J s)���i'������;���!N��z�����]�{����7����=���f����U�)J��.	�������]
�J�\c���xXwp�d<��dv"��z)����0�R���$����+�A���M���ZL���nQ2�y��`ThDD�U�������	b�%� �:��g�����*�*L�����y�f��'r�!R2�O��8�Cu�s�h�������e��t����l�wQ�KT�4P������D��c��^�Z)<N�<��.T"J�I���E�0���M��U5�����2�u�,�&�b)�)���)1�S)0�5K	������K$���WR��pa�f�Ii>�|6�o�D�?i��=�f���Eb�Z���4�_�3���)��]5�O�d��%`�������]S�AeL�N�A&�"+K����aS�mv��;~R^�f��^����6;w�}�a���.���O��UI��*e����Oi-*�����M2��h��"�����������>IU�c<��F\�$�_�RS	M�'��%.G!��]��UO���'s�`g\I/Qtu*]N��8*[��Keg����=.�j��%m��FZ���X�����o*Y��Uqp#YD^�30K�*���-�jY�DU3��w�5��(A��}��V��z]6��������Ck���C�r������/�9��)>�ASS�2�����������s�B���,?�.)��7����$�Y��0 B���H�}]����-�����������Rq���������x1����
������Nm��jnm���/��"�r����Pt3_����/�P��{ V���f��;��|;��U~(�����3��xM����x��;^{�m5[�3r���h�������i?n���?��Zb�FKa�����������K����k����?T����"b_R�b�d���6�]U�	���~��&�����P&���C���*v<�0�@�������?�����;<_����Qu��@�d��c���/��Fzn8�3���<?�Wx�d(7��(t����@w��|{�h�p����
����������?�����|�������?�?������r��@��/�;x�6����h��em�P���j���h�
o�S��z�b9P����m������{��v�U
���rCZ��u'QB;T21��!����D�r;���	z�[���7nv����R���Q��ft�3��|.Z�<�JJzn5-���?��]����������V�V[4����4�S��8y���������������>��R�5�Y�_�]l��v���������hD>�����C]��+L���o�>g��q���%\Q{��^\�����n�)�|M�M�����*$aTj$?��O�7�v���"�����|9���*��WU<=RE��_6��(��J��z�t�L����)��q�5���u�M��K��
��xK8�A����
-�������������`�WH�a�\��]��ZGZ�����#�|s�N}��w��X>��j������i��ayIOro(����s<���4`p�{Z"�Q���dE��p�g,l��Iyt��Qp1���G&.�[����*Sg#���
���H|w���K�N/���/g���{�0���R�%J���O"<
�WI����a@��RK"`�jpvI2A��<?�=d���hv�R��9Lo��R��]��w�k����OQ����)������*\�F�\�cZ�w���y|�j

�qr�#��)��FZy����Y�)>s`��r[>
T��Z\^0��|��U������c'�v�S�M�s��b0����3?�����W�SB�U�	�C\p��B����\��8='h��xBE�W���`���j�iF�vj��4�Va�0���c���
�xV&���G�s�>4�4���z����E��/!����?"T�"���`Z%`�J��(��a�>e���3�O-L�k���L=3���w0c�"��n�;I(�;uG�A�FcK��2sz1�pg\��X��0;E�|���
��pI.��N����
����T>����I�7N]��3��I���m:������ho9y;%�����X��5�H�|����X�? ��V���e����W���W��h��G��8�CG&�I�LZj��������Ik�@3�5���
t��"�(����x�"[W��~�x?�����s\4�S�W�����(s Rg�1��Fy"u�1��A����H��Y����W���9����Yj�W<
I����69�aJ��D��c�C�����0a��,�k�x�e)�����IXH	C�����L�^F?Nj
L@	E~M�����	U�1�'1�r�L�F�9�J1!p������u�'�w^aP�/a%N���H�]f0�����C����������2$o�-0:�Bc�d��T��f��f��*b���tu����D)��/1���=�9���L��2���L����������)��Z�wh��4���j��8�zg�:l��5�H��y�v8�l�-�m��)��b��O{Pv|������&?fY�3Y����,�� ��j�|\S:�D>,D7vc
U�!��������������N�:�����v]f���������������^���T�?��H������%���A�A�m��O0�t�����t�����v/V�w�ZK���] �e��N�=_�S�*�}��������1�����FfvC%L����6`��b�����G�.*�o�*��F��j�f�Lu=_M�������5b�` �c�}x%��9�T|������jl���;�^%"'
������4~NG@���r���7#EV?e�bR���#!o���]WxH�zp�
V�������31�}������=��\��d��Uf��*��{)X�v��*��%��X{�[�wjN)T��:����t".���X?^�8�����r�g� k�6J)
j�O������p���!I�"�@��V���4j�.�`�9�n?R�b(�8��!"���T�w�����H������J� �?U��/e��`RQD�LU)K	��4�x4�����R�p������{J�r��0�2R�4{Jfv�d�	u��
G~��k��ZhXH������"��|7��ol���C��=���R�LMS�z���8_yD�s���-2vZD���0�	��������t��N]��F~�h��Vb�+�G6g�C��q��H�f���HM����X�e�N�L>~�����������/gin�?�e����amo�,|�YP�'��I��x&)}M�-������*��mq1���!��CJ>�����r��	�_�`� h�2M�$F};��%���7J9�PX�\EYl��V�!R&�����J�Y�_}/�r~_�J��Y��a��J��g��K�y�O|�OVn�N�'�����
�%K�J���&c��v�>�5�����i��,P��'��Pl�!_QthY����(�?3	�?y]�n���#)-�^�J��K����3T�n�V�������$���M��)��'jU�i�
�r�:D���Ud�v�~��69
.*�'/t*�xP���yz�q=3��^��<0\HE��"��������,��1����i7�l���:�j,pC9YR1U��H�
H�;�:�E�VJ�X�����Os�K#�*@di2��#!
���2�~�[3���������G���w������JJ��5��=�c�$�a�q�Xro��E�������b��Ir���ga������\��tAVx�(KGlo�Xm.�����UY��w��h����Fw#�M
1#�R���n���������r��m��IMpW��P����:r���D�9��D����������d�Y�5��m�����@t�'B�U��U����d�$i�E�~�^rf���J�?������VV�5f�n�C?�oi�Jf@�jwm���<��8U�|�7��j&�������j�u3��bK�H�evXB1v+N������6J�S��N�Z&����a���`���^F��J���e�����m�����\�����|�5P�jb����t�CL����
�&��9\,�
`�
�'�
�h�.�5����u�9�5��y-�s������2\4�Y���KV[&9��k�8RJ5����t���:v>�/��/�c"���#���gK��x���D���������A;�p_�=
���Vb��b���l�A��%	)���1U�����C�+���O�U
mn���X�:����A�:�(���X�����+053��4G��+L���0��>|����Z��X�a�����LVzr�W5K\E���s�YR���}���b]<V����"�|����������Ss�7I��X���#�qqMZXT�x�E���L�y`�JU���k q��s���>?���
}��2*+�Z��:?nT�n��@DXe[>��H�u���t�(�q���x5���e�����9�tk&��J	��l&=�H�t(j�{�Tf�	����Q�B7�ex�`�q�)������"�]�%G������!�~�(Z�W���g�L���=����C��8��g���J�����l�:%��?�?�4�����ZZ��%�s�n�(AG1�j~aLa�(T<��3a�Jbt%#�������o�.M�*��5]�qx�'��ccM��6�OOON���*��S�j����x�NxS\�?�����&q\�yO,
��^:�LKX�����{�	x�jK����zyP�Mo������
P_DK�������Xzj��9kR���i���p���*�vJ�W���{��`"%�L���K������T���
:'�L�T���!��3�����	��9�tnM5q�r���-�J���4���r�>uC&��������"�1�������/���BW�������]06�T�P	JJ�Hb��V[�wU�Dw��3��F��>M���5��������O���:���C�,�K������[%M����o�������&dv�.��"Fk/wr�\��������o��� V�0�5��\�,��
J�W.w�INe3*��T�%�R[�0�Cm��"�I�����#X3���E�-��~c�}�������;d���5MNQX��IR#i�$��kPn�]e��)�S�O����`}���}(�N��-k�S���vgJ.z�Z���WQ�1��3�,W�u�����j��U����x!��|�B��
��-m���3�p��cjZ'���1g�ZrZ�&y}v:���i��E��dF�	`Y;�_���T�������"zV/���,`����r�	�����-#iXU_����-�(�&J�\� �P3�[����Z3����d:���A�,���3���x�`y��+�k���jK�NQP&���C��y@��e��Z�j��SPR�Y?�]uF�����g&�4�����n�fI#C�����C[A�*����+���h�K)���)����c�i��E�����Z�V>����������^o�F��N&�A�����i�\�����b���Hn��t�a,g�!;r�YLl&M��I��vFW�(�)0C��p��)3���%�����!�;E�����"L�3tK]����|�8�F�������3�nuV�1�� *[��_���3����&b5��R�/�(
�,�+9$'0b���j-H6��x��%�AI����Wa)�����
�=y�	�g[�)
�A��<8`���xE�����]�%����4i&����������������7��d���?�=�H��,�/V��^.i������8��j�0�S�������������������C��������h�!�,�1���G��i���]�g�G���}�5qG��d"'n�i���X'A;�/~�R%+M��st�"�,i���M�/�+N��2��s���������e=����3���5X5-�k"���?��5b��$(���.~X��_J�!j��h�
�Y����+����Ln�n-�8e�gj)\c0���Sc���J�c�R��1V�s!)l�"~f"fIwm�m��U�bP�V�!�H��5�l�J���C����w�J.�K�*_�������%U?��&0�?%h0tk��O���f*����LZ��Y�`�@��dD��,�tI|���� ��=(X�;�vS���Kx��f++���7FeO��;�^�k:��S�����`${�V��"�
#/���(��X�Yl�/V*���l�w:\���()��&����_q�#�\&[^�a�h������������Y]��[S5�j =�45������e��>Uchw����q���4����������_����'�N�!{����F�v��j{����5��N�!��T���{������R�/]�K��0h)�hI��F�M�?�`)0���5�����/���<�>D?�uX��S������M3����1c]�Q7�j�Tk�B���*d�}���N�4Y:�Ke���3\����'�LT?2�|��J����S><L��l1^�E��	�*��������X����)N7I����&�Y?����9F%I��F���t'�����$����C�\�|�#-�Q�>�S��)��Y��}��7�l3���T1�%�Z�JfVs^��l
3s��:�������.��[�tje�9��}������);�vZ��7q��A�>�LF��3pr-����cE��l���l���w+F��y���7���_���{�vT���o�`_���nY�����!���z�v�y;]�����.�������3���e�@2�������p����#:����3qY9���`�>
�*�������P��B������;�����pui�R�(��T��]R��3��F�����@Y��,����rM�_����"I���&IO��`���Q���>,�[f&E(�L��V��F�s��:H�%�������u�rp�P:����_�T%����n�$�;�jW�]K�4���'	3�SJ�<��d���)��W2����la�%"L���)����p��E;4�E���^;.1���6�s���8�:���_�'/~�\�l��v�Oq_M�,_���C� gX;$��%���2L���������@�!�z7������^\��V�L��(s2FfV.Q�bs�|=�k^(�Z��-o[��5=5D�
��4���Y[=zz�t�t�S���A@~U������4�O�fDx�^Zb�8:����b�����s�X3��I��`?��]M@ZI�t��
;�n96��j�2&�3��D/�����U^�>��%�|.[�_�R	���N\PO��
BO�d�<id���'����R�D����g�r /S�(���$�9��0��t
6�"d��V��$�����:1v���[��b|�a�9U����-+Wi�x�X�O�}�Q���J�-2E%��UK*]����y����@C����6���H����@
#���q��5�w~���Q������[.'���
�J�;��)���j1���(������r�������m���h����\�u*I���-`���_|hb
V6�V�.o@����������>[Q����>V���W�W�VU���I.T�>��Q��?�3�;#������	1�����C�1�<���@�h��scU�1K��������zG)A�m�"�[����dw�J�����
'�����\�_�4�?�?�������(.3���[��T>�9����$�'��0#�2�	0�������gb	F��\����Y����bX������{����r����
��~�����:�[m��������Y������g@�����O1����������t��d�1��+�����B��iiHAZ��k0��J'����
C�����~<9�����_vx�b"I/W+V8F����������x������C3�2;�T�!�G�D�X��Ff�+3�E���p2����p2�ZK������i���������r��>�����r���
o9��FUu���F\��������P��
s2�*��G����j����$4�x��J�TG���8�Wj!��k�Z��S��
�����J�[�O�6�����oZ�7��yG ���IlO�)#	�>)fl6�J>f�P�,��������fvZ�'����d:�G(�bF��J��[t�G���s+d�k��M����W�T �$��B�RDb���4U�)���S#�
�MW9�����k�����t�V��t���Xn�V��x
uR��kx�z��6��/�������w�K���0>n��	�FIG��|9�B���U�E����Y�������t.��6�s��y�i���ZS}��f�6�qo���������=;{U���f-jx���w�O�	����?Z�������n}���?��1n{���7r����M������}����^��9�qg4r�_%�����y�4��?z��d���B�%j^%h�iT\�
�@���fs?,� S���������S�=nx��K@#O��,���V>�M}C�,�2DQ�������``����h��.����y4�C�^����J������,�~z|�:$���Z��gn���?������o���%LX��b�S?�����Y��A�+{{	f���b���+�=B�_�����w��'Zx��Z��^����u&����0�Z��I'�����B�;�}Lw[�g��-�(�����K�M�t���r�.�m�\�K�N��u[�G���5�[���d�!��9��=��l6P�e�����@et:Z��N��9�Q����z]�V�7n
:�����gq���K	�� ���}���)9�����T����;?#���I	�n
.����V���7;�n�x�U����<���f�a!AWJ��j�����xL��@�������by�hq=�K��d��-��o���k�]yh��j����.��f������|Tr6V��q���jO�#U�7)et/���������Tl��������P~������;���$�q�O����T����P?b���3���j(%�M�d�y�\j��-�@������`�G�}Af�5{��:����g��IcO@����b� ��c�,}�a|�%`�3(x�j��$`w�=�
��w86�I�����e����HF����W:���-��a�;J��#f��B��0�^�S|�{� JF�y��y���#���N��
:VDt��a��+��Wl��|��B>���e,��J�m�z��p,��r��k/r���~�3-��4r^u��KV}K��;��<bc������� x�\T�#�����������}x�������������gI�qZ�~U�AP�sEx�g���q����i��z���P��3���P�d4��CXE�1�_mc�bS0A2Z�:�b�K��R����/��A<^�7�!��t5^�K9��`9�x�������S����Z{��^C�.�*�����Z(4�K4����r��d�#�vgT��zN�-{����a9B]a?��S
�]u���F�M��d����*���M�OT�A�B�[$I�P�,�{2�X�6�nbTf�
9��3U,zCW��%v����"��C������8�;����g_yP�	phb�[�fx���y�'N%���Xe����h��
�����e<V+�Y�����7vGr��8�zg��ZM���k��v�vtZ�i ��_}M8���%�����Q&�'F'�	��/K�������h<������e��0��{
����J��X��f�p!�7a�����mP����m#����t�����R�)	z���dg�L��� 8��X�\�b������o�K
�4���NB�p���TC]G��{�	�Fo�O���]���yM�S�)�e����G���Rv�4]M�_BL�����W�!�&����������B��w���,��4khu��Z�����\g�!�v�>�	}Qq�1|.&��� �LS�(:��3�}G��n��������z��6G��zU�����q4�\�2�"��T�W�KS��	*�wR��w�4��q���L�?]].�)��D��z�y
�c�g���P[����aN�����o����#���x��VoC�qu[���5zc��m��=�3h��w!�;�m���QH�U)I}Er*����h}�����/E<���$�`��F�n��;����RoG�Qs4�Fe�nw�$
IUj� ���bT�.K/�������b��(��
���F���I����C ��>R��&�K��]����.�5	�sgVa%�P�0����u��N��k�n��z�������En6�t������( 	�|}s�0~�Q=��q
PXq����������[�s����
���KW���VEy��h�s���%c9�8���'m�����������G(�m7��u[�3����YL�w���/
�F��Z,���N�����~
�oh�r�c1�]�"�U���"0��4�w/�L\�_�+/���� 0��s�y�y�����mn8�~���x��������(P��������5�����oR����4n����K�'���������%�R3/Js�T��m�i
Y�
�c��x}��������7/�������Pkq���
�m�O����F]�����G�~K:�W�jh���iG:a��=�g����^��o�]��A���,>�dGrv��1�ly�GP�no��O^���y�+v=�P��Z���9n���[�F�w��~oM�K������s�)8�/F�B�
gz~;�����o}���G��t���j#e���������������������+� �T���Q�MdgP�F]+�������������<�)�%^�[\�+����kX��8y_/���U��6'S)v�~v��������_k�V��u{
�����K9�m�j7�w��;����(��������Q��B�^��b��(�g����x7D)A��u-'l)���]���9��)^�5i`���/e�5)v�.����:`�K�������Q/�th=RIN��@V�H���A�7�]O��r��8�f�?j�I&	�[�$i�H�K$R(��D����W(��!�"��H�s,�xE�q
O��$qo�i9������R,��vW1�N}�B0j��sw���E�t��x�u��7���n��P�[)K�G��*|��Yo\!�@���sD����S��=]�Ex{w�f�~���t�q��+�=#W�w%�}rz�)��o�Y]��_;���J��.'����E5?:��Em2u/�u�~V����N��C��4���?|����s=`�`z�k��V���t�^s��;��t�~�%{_��C�q���=(�������B�%��&�����Xa��FW��L���rBe��X�{i@S��L4��v^<�Tnz,,<?z_���kF�N-�!�Mo��u>��N����X����O�@��]a���Z�v��2��%��F�)�2����|+UZ!��*��^*��P.A������f��}�z�]���t��e��w��8_�y���/^�~8v�
��{�?���|!O~��������.V����,�o��ZQx!�s��Vr�D���fg�s�%���H);����n{�t[������n�37���%��������F.<�h��[���M7U�#���c���}�*���L��\E�Nmw(�NW�m0]���� ��y�t?�k�����o�X��z�d���ta �+,��.��w����A�����a|���T�y2����~��g!��s7>�5��\���c��R��|��.+�p��S������\�x8��L�*��������m_�D>�-A}�&P�8?|����W�*�B�����5s�0��R���{����mR-A�g������9a�q������������'���l�����]�4�L"�$vOT��0�3�T������	����� ��[����J��|��D�!aL�k��s=�������Z9'�5tZ]Z�V;�8�=� i����[
Sl}Og���Wt�Lk��H�h{����oh@����
w{+�����^���S��&����+��
�Rh��0(a��r��2
w�����"�S��QfH��Rx��g���)1���0��*� ���j����XF\��e���
�
�=�9U�\oW�"��.���\�9F�i���I�%\I��s:������exB��+�j������������P���"���w@Y�T�B����5��D#�|9�/#u">��j7���2�n5����&��!o�X�|�_�o7z/�z�,��u�n���'Q�{��U�+S���s<���luk"���C�_G:=@�R,M�f�D$`�y��L+��[����*\�&��u`�>�ro���"�]���b���v�
�v����5��n����iT�}f��O��a
w���E����
�r�1��GU�3W��+�� �����aa�L�Sv��B�-i"�����L�������5�,h?�4"�.�0��p��@������|�1��Q���ei)*��CJ�����/VDq�7�Q�j�]��nWL
9k��$=��q"�2���h�!�P����yvL����NMg{��p1�������:���D����Img1�N��������3��{^����.y��
Pr-�jL�������6�S�O�i���L���7����!�O�3���~k,[��x,��I��l�-���5{���l4�����<n�����L$_e��R�T��B�M}#"����������*��ltC7!(3����	6���jlLsMP���&��*�1�ym�����>��~��t6��c����g�j��lT�$�dg	
|�q\��ra0TUE�������i�7��U�[p�4@W�M;`[��6��}c���S�Kp0��|��I�c@�����&��1:x����f2�\��>a5<�)o�kH��L���\_���/������|���\Uf�Lj@w��Mri��f\}����a�����6q�x����� =E�/������B���AO5�H,�g��:(�z������*��%����
I38Uk=Y����'�.�W@�?��"V�-\��a���k�N.U��<B���K���r�t���b��!�� ���_��8B{7���F��[���{��*N�*U/]�*���2��0|{q���7�����o����j�O����^��u�e��k�����~�����Y|�z���=?���]�^�x��H�U��uy�����7���?{���w�^��O��{�{��o^��MR,��o�+�Y#�NB|Tn��+�:�G;�l���O��`��]� �|S%]�V�fR^��,��!�*��[��k��|=���������6�^�Y���9]�-���2�Zt�-z[�)I76*QS U����0�B�R����J�M���
JN`Zt��r�Xd��,p��Ab�X�{�k1�
|���r@�A��z��^�	�m��%��E.�C�����z�(�f�k'�
�^�1_�3s=i+0p>.D���eU��	�?C?���bDK�A�)fO��5v�Z`n���v��+�(O����4�krw�ue(�����%����U�� m�&�"����6�*�=�K�hU0��@�&|��9Z�Wx��Y�Dq�+���F]F������o����}��Yw\���������0FqM��m�`!�h'��z�I�Ij�e`d�n�!���z3�����F02_�������eO��c���q�nG����7���e`�E�����ib�\�Wm���KZ��}�������M���J��qG<=>���?�P���������{���
��D_���e��%��'O�,������m��]rI�9z�/��g������5X����G�@5�aW M ��/Y�s)v������hv��7us�$��,F�abz����n���[mTB��S���[u�n���fO��Xw����l���'=G��8�X��b��O�`ML���Hb�BF���������G���3���D>s@����[N|�����B��������������}���������x��1N����0�!�k!��WEl%��v�5��M����@��2J��#Q���,&�O�&�+?��Lj��(��Q�o�G��^�N��:HAd�k��U7������7:r�gJx)R��?e�';���pL�d�B9M$`�����m|�����\��7�������(���x�8�q�R�`O-�e!L��d0&���]1�'5��S�*A��{I�N�����^����B��yy)�����N��Zqo�,B�3��O��o^��?���W*�AwB�����X��
L�D�}��t�7���X���r��FT`'(��M��n�x�
�S������f��zx�^a���yiT/i~k6��jH_�r�KF�3��
l��=v��F���G������H���a�Ia�Q�62�q�\=���:�u�ri
�IG�6�L��HT�
uDW[���c�x��\�;tq�`*]�9��������&����_��i��[�?�_�S�{�{����
���
,�5NlD��
����C���*:�	t��3��_5��n��Vx�x9���H�]_�/����t�����szD��`�vT��*������E����q�E�T�N;���	�v���-��)���) !�Z������UrWb�U�0��m\)\�g���Za��/��4GS�g����l����7\���g���������u�f��px:uQ��'O���
	)�9�xR�q�V0_t���AU�y1<?|�������S?~���i��|�p�����������������]�/���}���Tj���x��_p���������oy�������[
�3" �A���s�8f���F�?�����;���� ��`���:�� }�� R/I5sz�4�����������o���;�a��Y��m���)������bP"�C���5�r��P���b��f�)i]M�a����h����wn{U���I����^��W+G�!��V��Q�*������}X�
kc����������'�G~k�WS�E"���&I�LD�:%��������`������*z��f� ��2�������c��L1a�yAr��Z���� e�#�n%QK�����l��L�+�SLj�����1��8�.����h"��?8�E�YA���O�*��(h�����`�F��n��c2}���0��w�c, �0���C��yDe|��e���i]���������s7�p��37^�����)$�<\��_n��r>b=�b���^�	�EXa�������/KX����u�����[�w�'��1��j��=�����;�s���_e�/h�Kw�ZrP,��������c`�6�u+��zC#f�>�$��A=�S���Gw�F �DF�D&�~���vj�R%kbr�0T5���6��(79�#���B�A�������s_%3(�r�������Am�m�S�u��|�r��6�?����K��T|���/�O7���>'�!��?�m11^U��%�<�_0�V���4���S�b8��w��S��i�
���<������8z-�P�D�YUf�d����cJ��'/�l��i<:aVQ��;E�i�,$�I��`��Zi0�h���SU����i���r�#P|�7���+�����H����rt�`���M.��yO5��Pr<^.n�$��9�9fMr^�$�F%����t��<�m�m��s��%���'>�W�@Opy�Va�Ue)�iB�nA�=�7�!����{�'/�jM7f�a ��V�Y��)�g��T���q,�^\x�Ij��2��8F��������K�����83B7�!��^��A!��5����	�J����ub�����Lu��S��g�r�_,C"wa�������@���y����r ,�1\�:2|Iz��Wg�.����N#�������vOh�
��<��3�#%��/.c��k7���
�0��l�uJ,Ta������H��Be4��@�~��}�C�Ob�����Z
nK�h	���������p����.a��pP9lb���Jv;Yd�U�M�M����C�R�#���.9�!Y!����:I�$X�v�),�\��^�+0N�]i��#N^�@74VJ�#�Co@
�9���-|�qxdV�TN��:X�K��,�@I)"Z��H��5�Z��h��^8��;�������kZS�G
�AFK�"95��
�|��������0+tJe@}�K!E�~R�n��^����_b��rK�^1N%�s���	�
2mR5Tp�8�\5`�.i�0l���-��,XphS�K������(�E��$�B��y��b��2����d�pY`���p�9�P3V� z
@r�E�L`"�-9�S������63������CP�	�,��T8!����W��[��a��-,��t�7�\M"�:OxI�pp-u�R�sztV�^G���`���b3�3|V1������.~�^����~��J
�<e�q�vz�:�t������J���:�pYG���'�1
����6��N-'�������!�<p&����C�~�P��r�V�!D
gPr���V�;����5����NtW`�|�l{d
��K6nq`�b�������W& ������Sy\X��jo�N�W���(��P!i���T�r�y�����u��Ex��p�\[��EVb��s��<�=D�����|;$���7
��d�A��V��t������G�'���<�
L�5v�V�nFE�<8yv8<y�����������5:�O�������pxv�~��}��q]T,��k�Y�
	�x
;r��L`b���lm���+Wt�69�,V���������X��*$[��J����ll��6�������J-~H�X#Fu�U0��$�X������5��A��`�n�
J�2yO�T�����y�un��i%�SU2FY�v
��}����r@�lf6��&�4�G�)�T�1E������+
���l�����j�u
��oZcy�������tj���q�X��D����]��-��N+������*�p�@�9�l��Ce*��]��*+]t0�a��EH���{�We�l���sK�vQ�vI�v6�g� ��<�(�w��L+#u�0���}a�0/n*v�{k�	V��M�C��'��`���HN1P���!;������ �e8����a>�d8T�O,��4����E}sH�=�+�.�k�	�	��=��aM����s�@��B�?z	��*�����w�Y������8�[���Ac��]�J��������A�(`Mt>��m)��=r��/^o.�L/���{<7��J�n��\�x�rp�!����A�e'F�&���d���;��3Iuq�;J~X��z�Z�����CTT���I5j�I%�����D�za�Z���/�5Z��\2�z�A�i��� }>:1����HYOd��+�����x�o~���y}I�F�
$�L������.+��41��QW��(����t}��Oi)/�<0*��@Y����r)a�D;���H�G���o���Oi� ��E��r��HvLeO���	r���a�a�*��L���/*���Qp�����*)B!�k��:)rN�,`��Q���i
2!�c5W���S��7�����Bx3v�m��P-�����8JI0�Ehn&��(HD���8���Lo��D:�@��8�fG�
�\ 4�(���\!�N^�k�CG����Q���h����4��R*��.��"�F��'+*��|��S�#	��(������S������[�������1��5�R���	-���Q�	��-6����;h,�as����{������^An��9)B��������L����������V�*���jP�P�SI���������{�}b�8#���%��pU�����wk�=��	v�N��QCV������0v�&,(�����z8��cC��� ��`���Cl�RULlG��!����'��:�`<tG�6G�v��N^��3�����2���}L��_���,UPSg�hV�b7Yy*�F_2=���?�a�'c6	�<���3#����h�'|z���i�~�����~D��-�G��-���`P��p�?��Q�� �=�/�g�O�X+S_������v������1}�y�r�XQQ��"����u�F����;�������h�f�tK�
jC�+h��_��+Wp�:"��%���O�5U��Sz���`����J��)�$��LJ7���2�H��2f�O� `��3�"�P�rE�q���0�{�#U��BzkQ
��s��;K5����B��*O��Vz��2GI�H�V�s�����
�'C��q�
{WRU�.���`.�h�����\��B��9<d%^S��UjV������7�9�\�����������m��F����ft�TJ3�$&���n�:�
��5r�Q��#���Ja��$�=Y���Xx�����V��H�Cr�G����V���������Y�9��������Q��\��$�mC�y��Via�z�G�B��P���J�	�"�J�d���Q���}�sJP��L9�s;$�[��5t��\�Kdn3�v���1T�g��!�'no5U�����R�o���:BtY�����	���-k��[����'������k7R[?�Lfb,���r��1U5oxF�MNN�O���Qq���J��[� �e:��?8� 0'+�d�����?;<><?���O���
��!��Kn��K��9>zu��7Mw�2�O��?d.���359�&^�M���7�Z���$����{DT�$�D�T3P����|<��^�j=aM��{GZ3�R�D�>=i���vh:���}Be�'C6�3���skT2�
��c�M����4��
�"���FH�H��v6��&��tVe��H�ZR(�1U��������D(�\�o�WU^[q�a�=>��~=J�")r0�����?�ig��e����x����.���h�7`�6����T�i����+*]b��A�LO6O�pLW0���8��pJ�j&��a�C�t`(0d�,��Y:�|g��l��C�\�`��|v+��$�����EH[���/(p���g�_�'F�����FL��!���j�/���K����
�Vt\[��b�,~�)4g'����<������-�n���l��h���P����p�'�D]��_g�����=4Z��Z���&�RL����J���^O`�PC�HC,g
.�r�LB�
oJ��t�?��D��B8x�)��d���:J���>��xG�S���pexx����j���a������%;����u���u���|!+�e�g���d]�|v���P�t����T�PX�_1����j�U�p��Q��������qd�d��������[�)����ue�"����H������jA��!���:*�{�g��8�x-��;E��B�iN�>0k��#��q�����W�-L�V���^2������6X�B�����C$��VF7�e����O�5��.4��"����H�#!8^��+R�Rx�������J}���}��7�V�]3����O�4��&"�A��b��)����f��L4���E�#������)��J�$�R��5�!�h�\�Y������7�H���eaI��_�y����_���O���S�+��&�M�����������6m���v�.�N�6Z�8���H��T;i���6-q;z[��u:Q/��vD53��`����p�A��"��d%L� �����+Yv�O���`8��24�h}�9��){�Q�3��GnN��i���k�����&gF�����5ha"�Hie�\M�<E�x����>R|'��.���h&���N�l4![F��nMv��vT<��yK�.#�&Y�3h$��cS�����{6H��ED��;G�C�:��FZ����
����hr�uf�S6�54$�m��'�d0�����$ID�FK'?��G��+!J��������g�#��Q�0M��*�/'�&/�rS[���M�������="\�7
$U�����<�?��y������3�A)�K$"�}V�jENFM�o
���q�1���Y�"���##Ak24���F�!��N$�,_�����.�O��1 wfc��)`��\����7��k�=��+G[�������#�n�}+��Kg���������66gv����sW/�w����F�,K�o��!��X	9�`�6��b$y
w)T��D�9l�������i����xd�����N���$�$G����	
������Q�� `�'#^U��L���F����1�����=�Pw?
���g����������J<�!���4`'����MM���=�f�"f���/ro������3�E�Om�x��xa��0����_?64g�y�[1�?v��I'��6����q��<�p�������s���Y�5�����������C����;o�O4������n�D����q�
�3�+w���/���������W��T��a�A�,:��� p�������5U���P�������~_��[����!o��Qo�1d�4)�^�yU���/'���}fXS�����cA����5�a��B�s��SCH����>�������*�?G,��5�X�<l�;m��S�0u�3�i\7����/@�" >�'��!5��!`*��R�?�&���Q�i�T$y�*�T��*@�t�C�c��Q���A[�O\�E]�G^�/U�����L$,�Pz �"y�L&��EY���&&�m����������������i��}��}n���=
<j���RQ���6�@�FVz��DR�������-*O!
"F�Q���O�7�]�_�Bo"�"���5��[�B�.h:���'='h���P+^7�j��cF��f�����l�f������/2��@�?DU�L*qR�M���%��9�9������X�p�d�R0S����C�vNcoQ^E����ER^���[��yk��'��NP�K��8]�6(��9�'W�����jw�v%��;�s��wl���A��
��7���S��pC����4��Gj����&�l�X���6YX�����	o��/��7?�Z�����9c�-?���V/Z"�N�2q��6����) �M}
l���d���R�h����F��Z":���)����n��
����<�32�i� ��-�Yb;��0��Z	��;d!P�G��G���z�)�}v�5�N��U�����KS^N>Jn?���w��E+��I+�^���������<�=S^�g�
'����$�(K�["hHf��I.���g�����3�H�d������
�o��������F��=|n����$����^�V�<�������&�r=<���3E����"D]-"�!�j����S����x����)6y�X���+��O�����.n��z��?$5��!��6���������e7�N���A��T�<��la����%�@�^M[^�&|>�W�]��HT�a�$����K:f����D�����y�
�?6X�G���G������t�#y	s_�����W���;{9\=�6U���+G]����������}C����F�P\�#�A[K�W�+�����=��6���Y��Go=7�)���mJs&�Qw8&������v�����I�$��@	uj4��`�+�a��h��0��������Bg�\��=zCe��N�{��B	��6db�t��nF.8�*��6�j����f�Zx�K����JM�=���
��L��H�kg~�l�1���{2%?���Y��w�R�Pyc�x���8�hk�k�;�6�}�_+���\O�%����ay����xh>�����7������s[����(������J_���BgIi���`�M���NA�;����"�k�uF�T�6&j6�I���%�f������T����q��f��wwZ���kk�W�qo#����)��W*.��(!�����m"��R���9����������P���fL�o_�	X�;���(
���K��I��: o�S���;�K�F����s��>�/�1Z��dz0�[y��cSD3��I
�o�
��%;�i,r~����fl�����!��M�o�?��A:e@0��O����~|�E����O��u�/u�^����>'��o{{�nI����������7��.�o���~��o�6�V/��zh��z���#8���P
����e3:���9rR�����������DhQ�u�gmm3����w6w����<�%���f����Ea'p���[nl�*���[,��N��P�����I	K����)]�sJ���t|7��h�[������,����1z?H/�A��t����$���i��m6��$�$� T|��F�~9�}u�G��>�U�����������N��9)8�x�K�j�����������������F�D�S��=j3Bc��7�<>���mHq���A�#����t��~����2�QF�@�,Jcj�= ��A� ���\c:i��@c��D��!�_�$W��3�h��F\7��M���	<M��x�,u�F�p�k��J����Hk^2
�����;M��J+�����6s��U.���o��
�il��)�-[���|`�A�m�F
��.���3�u������<�H~�M~��)����-R�^�-_�j���Lg��g���z^��G���:���1������r�� x
j��7��7����%sj;��Jg
.z>�	�g�S�����s���������9�M�{8s��%�����W��{���m�������z7������eH}����������	������?�������q����L��&�t���@y$�M���=Lvy
��a
�S�Nk7O��F������ee��������_���1��iJB�	�.02��F���7��|������<I��x�7J}UzI
b�����\��`jn�mo���oHo ���BS\��0X�\��Lw��������;�Das���Mz���\	�M�X���1���� �L�*�(�[4J��*�'M����dg���D�`o�����6�Dl���g��i�����]QX%����J�������h�
� �-����8U�����YZ�!1���[��01srp�e���6R���*��������rp��������b��=4D��6�@�!<
��1-��
 =���O-���Q����Ctx�d<|�dy���Vm|�������?X��jRc|�G`A6�	����M����j�T�Az�u����zVK=IrJ�t�cO{:��y�m������ow�I ������rt��<a:Bn.G�nDA�$�4RRgt����p1��*Xt
*��+�X	�x�{KtS����Zkc����?��s"��
~o��)y�{�����zk�y��59�����IT�9����&�0,�P��:��wg{�����?[�[�]�hm���6���z{{kc�D����A(��p6��%�T�����4�H����?�@s������}�y���\��n_��������V����������w��S:���q�����_��Ej�Z������&�Y�	�2z9���������I_=|�k�"�^V�&�*�4w0�Fm`��x��z��o���s�-z�������<o�~�����
v��u,��?b?}����.c�"�'q0��/���~�?�R3%4�e��O&������(3,g=�f5(e�"�I�;Sc�@�@fj�&��-�%�����ko�^D�!Ya�>I�o��x��PO�7hI5���D-��8������,��;�"��N�������b]K�����G�x����U��[��t���������_�������O�v���v����������5T/��5���<`�����`x=y>�1F1O����7��W�L^��y���$��:��:��f7U���6��z�j�gKy�7������*<��]ek/����Ed��F���)�Rc��k
5E	��V�������
�1�2����[���������b�[��J��G_�Z1��[��=������EQ��B�!�z����g�}tN�G4;��M��$��.`[�7��S\���E�X����_V�M���SE�~L��O&��E:v�h�t�(�yU����s�[-���TX!���j�.�N?(�����������rt�o�����HqML�:�%��N/����p^�o>5�'��!�-~�����m���g��]���1^���/OLt��
&9k��&5R.s�=gy��I>�e�^/�'
����)��gcg����&�vwm���]�����dc���m�������`ty�{#����2M���Y��lI��]��&��~�����������y������:PW�%_��5O���<B�i<N�R4����Eg��^�\�W�)c��@gh��`M=��JTig�B����V�������n/���.��V��2����{�n=������b�h�������������-�n��#
��[y�M`k����SD����e�~~����_:���J������{��*,P�c6��Z���s2iZ�����N���I/z���l*�9��~�0~�9��4���~}]
b	����)J�^�IBI�9xtD)�:���g-++�E��	�_���k��8��!����O��|�W���cO'j)�-��	���8���a3����cU������\F1y��~�hx��e�����x�&'�+P	j�UV�W��P��-JT9���������d�nD����O�@�}�:�xB�5t�/�.~�W���O(��g���93���l�S�o�>������:ut>}zst��~h.�>�bz�L-K�0�NR><�E�mJ	f�3���	L���Q!�SHB"G���7������~{�j��Y�$�qB�I��F5�Tv�)m�Y����0(��
�Oa�	}�����J��*���O/�k�W6C���^�P��e������J�
����u�`$���}^sn+�&W��d�,���	���W��]�C�uj�g�Y}W\�:�9��8���j��C8'�j�N�W�h��f�����I;�~��Sqz��2�-�`��O>f�E�L�?�Vu�CV2�A��j�Wl-z����Z��9���,�
���20x�H}��R3�}�OM�~Q$���t:I0�1��M:���M��9�W4
�+����w�#?���va��a:�J2��n��"�He)����09���a����i�s�:�d;pp���@�u���}��D2�������R(��59��(��*Ji�`�0U���3|~|��2���E]�!v�J�r>��G7i8�MV�����v����-H��N�����>�����[
o���tb�~�#$�V��~d�*�AXi_�&=�M|mp�H5 �j�W�����'����Qn^�u�����!�8�u�[�������	b�������J�L�c���C-E�axg�9�\�D��r��y�������.[�<b���y��v�aH���G�G�����;O�4��7�+���!����bg���
���rJ9�B������X������a}�R�]�H++�aA1u����Y�����mh)U
eZy�
���
�J��:�wZu���S�>K\A��j��]������K�E���z��`�CG����������uFZJmk>�����d�����7F��������&�l�&����u�������Z�scG�
I"i�KOoR*��L���X����8	&nRUbH�!��T�w"bEG{>A��'7�����/�����5x��z�Y~����K\J6q<��[H�o�'�����m��4Sbk�>\&��%�oWy�Q��7�p7���xw+���/s
���V�M����O���(��z�U��+I4�C5w� ��4��c/�:!f����dm)�ftgZ(Y����pY���y
T�|�+���{������lS�;��1Q�Qef���^I��,ou��bf��
"����?<n y��7���Mg��p���\���UW��L�p���x�+��K�6=�Q��;�N�nx:������X��A��m���u2��������o&nk�����Ei��Rp�oP�r��Q�$[�I�j�������IRa9�����R?�0&�\���L��|�,I� �C�<�d\��K]�Q6�	��
��[&����
oH�5����p�QU*b�~9?���O�*��U�x���*>;/Fw��45?i����Gp2G���
b[s(A��n�}�7j�@��e]��;9s��HFq�`2e��mO����s}Ro'�Tp���$�������^�g�����1�>=��D
56T��:������u��L�U'�D��F�����|[��>��V��>����I	X4���
Bu�����,���4�)�!�*��x���m��>IdH���S2y�0"x�Y�X�X|Vh�(]��Q�:�L�Rx\��GOF��nn;��bK9��4�r����iCmm���je�N�*�/��_���k�����lD��zaK4M^u'�g� ���/�y<o��M����v��� ��P���|�*:>��(�=��yDa�T^�o������[�(`f����ang�{���-����E���n����������-��h�,���Fm�b`<���WY_B ;���J��z�)yb~�.:k}1��@�`�3�I�.���h����������/�����j&�U�)�yg�!H�]���\����], �/[���<R����y��p&	�# c�0���]R����`3��G���
Y�Ph��8�5R*`
I��s���\���Yl��
��dY��Fj#2wB�	xX`#��p�wj��1�[5L
VM���[���B���"��]O��l�<c_����2H ��	�� ���8:�Z���i�*x���zz�!�||7��BVB����!= �q;��I�:����X�m1��g�"����m$u������RL����H':����jN
�Kt�)�������`O�����x�\��*�4�1s>1INC��(JvT�X���|=�9������_��$���^S�I:�/�N��@�cA*+��H�<���c�;�7)�$�<X�XHg��eT��W��h� �Q_'j�$���f��;&�FN�����&����G2I}[6cx.��
���I|���j6@',1V��R�WM�����5k��ri�k�f��U�
�*�����,�bo�R��3���6��5�(s�Z�!�!	t`�\�F��� =�E���2�U���L�����y6/`g��W����(@�p��z�WNQ��M!��u?�h���,��;�����KH�G����V���t����>�:�'�����|�R�;���)��� a5<�k5�MX�e�-���"��U�~�r���eA+��@�vP9y3�mx����z�A�%*��qa1.�xT�j�&�2.���p
�2�&��y��[���]�d����������g�(2�b����������\�u*s�L7�}!�	�P�j��������u	����)I�i����X
��wO���B
y�(����3�Vb��P8mW�:2�"�s+�]Cn
5�Y����J�����-5��P*�5/����Rkd����a��Hr�tRZkP@� �;p�����>����p&���)��xd��l��L9��|R��5�������E��d�C#;('	{c�6���%O���>��4PsTPE�=:g��V�d�
�4C��I$�F3�e,�7��f.������x����L(�.E%��{
n�#�@�'�
�1�D?��>vFhSG��?���P��L:��_o�
g�D�����`�#)p~t�g�U�C�:*'�E�I��^����!u�����	��q*
u.����Pv:��svL�3��g�k��aK�����[n*��������f�ZK�5/Y	�@���W&�2�Z2_Yp�T���&�h�"D��sgs�K����b����y�6��9i6w����%b�u��s^��	9r��j3:���2�Qp;����������-�3�-�:����Q�$��P�t=5�PCT������R���R��B����]��)n����xx�?�y>�} ��I���
i�	�#����\��u]�0�g���&��E�������D�������G���\2=��������X=D����d��E�*��<��[���#�E�Rr�@mV�S~�w��8w�(��j�h(��2�s��#��3�Q�2�m�u���;��y�a�mxN��2��?�m}Z�����["wX^c��N9o���T�K��Z��_���������^����t^�+�L�����PiQ�
��6�����O��,�������*��ydY>����Kl*��cdYA�O������f������x�b6	F�q�`v�:B��DF^��bc��l��y����)������?��W��Z����z�6iN�\��`^3�^���������������#�N)(�^
��XWq7����S�^�w�rWGll;
5�K�Oa<(s4�<
W�����=��[���������l��jH�0�|��^k������^��]^���.����x>��n�tTA���P��\EP�Kjk���1��q/Cx	a���O'�����������2p��Rn�SPx����_t���u��;>9z�������~�f�Y]V*V/e���v����eonv���l���R�M)�����J�;��9|Kb�R`������	U��~0�Ym
-i��SQ�����vQ�����
���J��F�������R���Ca�NKV0�����>��a�Ne�xP�X84����ty��D	6x��ke���~�������,��^��T���U���C�s���X�%�Z;�9c���P��|5@i�;�"�w�3.�n���u��V�>���%0����[;��Y[�����?���Z�������Uk����{���D���~S��Jv��������]����y��y��*�n���ci�m�-/jZo�M�K�������A}y�������A��������+�i�A��� �����?�/�x�?�����8�}p���hG�0�4� �5���o���u����/���VV�����QpF�c���@���2�
��XF?g��H�`�z���.���YDbh���/T���P�'2��DW0�����S��k���g����Fp*�?T�H~$�5"� $���:f��W��!fF�����a=������0
)��@xFq���������Q6�
q-��&�oX�2)Qa��f�1%���Hk�8�|�*D�J�%~.i��9����<�7�t��e��BZrt���3^������������~��������O�����89mn�|;�����>�����{iU��p��L�k��(�a��\ E�����\a��m����] ;�z��^P��p�P��n{#���bxy{p��ql������H-������:�������A���/�VZt���^#C%�N�C���p��l�����������<��b�������.j����*����"\-t��
��N�C�"�
O��kP*�Fm��p9������#���>��u��0?A����n��j-�0oEQE��sZs�/����#~#������6w���*����n����^qe��H�pCm�[�w����(�����&J���m��3g_��z[�����:�}�J!�Dzr�t�iy���>lS%U��C	;�7Io�8��������D��:�TK�%�eH������E�www�|�;e#
���O��o_n������V�����l�wvv��]�S���?��~c��9�����^N�-z9Z�M����,{��
�������l����V���J��2/����G�&J��_���Q	"������~����WO3R��*5��?�~@���2�zR�T=y��(a(�_Cg`o{]Z�������x~}5H	�]�^>�k�m�@�@~�^*Qk��V����P#�U��������y�C5&��f���T���������,��/`d�����~ ���inLbh�%q�I����K�@e�G��S�?�A&?��!VJq�����u���sg�!�Y]_��
�#s���o���	��s���5������D5������G��R���)�6����i�ta��"���J)�>I�������j_������{�8��h��(j/���$�w����`��+�����������$c<������3���J�S�y�z$
��^���cX�y#�B�3��y�w)�+���K���h!�>�VOc�~V����WQ���6�E9[N��Pv�%(dV��b~;���k�i����@�|=�����Kv���X�?�u�v^h�@������4�UC��C���&��Y30��G���CRZ���*���0�0�/y�*L�CC��f8�w(V��������h��`z�=������������?i��AV�<u�z���y�*�����{m���6Z�{�5�|K&WS�S!})���M�&s��3��������V��%=��w�eXE�-����MQ�s�K�C�m���zR�{�kk��W{�^�q�Y��5��{hMI8�;[����'G��f6������lW�V�6�}��dr�X�`�`�XY���2�8}o�h�-���[�;����vg�R�a�L�W�����Mq�y�����j)�������^�^����V[%[i�"����_R��?��n^c����t9����� �VB�#�������a<�e�I#6�"6��K.g�������	�bc��$�O1�T��%dF8KS<QR��k�Te��Q|����!�%�q������}����=�����`�^J�U�O2���&���q8�1�im"�	����,?�����v���������8 �����/N?cz�O�Z�[
�
�$��a�<�`�vD�v�f�XG�8����������4�!"~�#!F��D�z��y�*Q
PD������k����?]t.N?�Q389�VVzY�AY5���fs��I7�E;�E=�
��*'�&?���c�'^��-���Z�?l��<��P�5���	&UYt\g	)�>H�

E���S�w���Q<74���$�
��w�g���p�il��[���&Fd-\��-�����������q����0:����h.g	�P�(n�E;Xv$u+�z0Is��\���=����������$p��0�v	,:4��VF����Y�a�5�E��{uv�Lp���d{�Q��F�O�)f
�P�<�A��dlo�z,���Y�&�4�������f?�c��_g�����L�$����^��/#P��nv��J���!h��xT�V���=���B���/w2M+vc:�
���z#�b�=qEYk]
NQxq�A|���)�/��sD�
�������ss�M�D��
o�?����{VN3��B�-1�����>�8���	��W�h�J�Pb����h��P@�E�K�������t�8mOd
�� ��eK����z�����������8?���-��������p�4J�H������#�r����L5���q�fJ�H���s��hlmT�4��[r�8����������q���Q7�R��t�q��s7�	�P��c��kS�q�������p�����
�/�����}�?����9z�o�%$lD
Q�73N�I�����O����;�����s�&0��Q�����5��
h/ P����,���|��I/a�5M����/(���H��'�3t��Ej6�t����:�?(M	l��ON�O���#��O��8�R�q /�c�#�C��/q>�d����N__�>���� �a�
����	v<����A�]����T�������_����:��;-�u
Xk�`p�o�|������
�����D�wu"���tpvt~��N��]�&���k�5�6EFu�z������3w�E��9��<I��!�16�YU����<�F�[�*�>.\L�������]!����,���eO�����z}�����$���)}.$�7�)@�$���	d�M��Ol��,J�T�!��e����������?Ba�g���xn�:8����B���H%`^�?�o3! C�e���i)�s�]��d8�h<B"�J���hc��z�������)��NF���/��vl��)��Ok���	d�
����S
:�]��!�*�������-L�t���7{����_E
���O�6w�t���=8��R�XAS�b��F5\F>#����5\r���	5���F��r��D�R��������?Sxz~Lb�Y|D4���x�t����zV���A&���k8���IlQ�h4�]�:���r��7L+$���LHGP�2�8����Di\�To�2������(+g`\��;��zRT��t��%3 �	3�j
4����t!2B��L��������J&�m�����rP��vt���)��0d�����o������Z�/�p���7�~f�Z�
�gIg7�@T_I�[��&�e��
a�����/�;�����x)~���[y$;�^��������D]�����uI+Z7����s'�\�j<�[e��4{h@��	��H2�I��pt�o�!�Zg�cVb��+���RK�2����A���X��f:6��*mX�-���y�U��G^��~>;}�n�2^���V@�5���^y��&��d?�P>��.�_�o���G�G��0� ���x������ �"�+��x�����2l"I&�_�3IB�	0.�����GW����F#g��Q��%`k��$>Q\�-1c@1�8�������
\����w���2\e���x,�a\u�,#��3�'�$�1��p��3~��N)*�g��zfL.N�c^@���w{���"f`�8����Y���hYk��k��������r���������ppNu�q.j���?Q�%qK���|�su�,ZV�t�����J�
����q��CQ�tL.�nlK���IJ@��~�O��2KL�f(�n���tub�fm`rng���|6-�>���Aa��i��Z�f���p7��ng���1q}|E��L�����t��Q�R�(|
��X��#��D�L�����GXj.��S��""&���1�m�R��q���n@��fk���G��Q�@=�V_-��S^n���=�����E�0��q2��	LA���$���`i��Z�I�c_��"x��.j�a 	XV`��
���v�r�8�a��}�
K��%����EF���V���T��3����D�}�;��(��^��Az��Z#��8������\o�7�H<xo��>�9?�S�O#�S;��]�\�&�'�>�t�����t8�1Z`<V�|<�L=(�XuX�������4�Z$^���~����Q����e+���H,��M�f�^�wZ��I�;���_o���M���	o:o���C�Z`n��E�����d��Q��@agL��V�	7���B�e	������������5�>�&��u���?�7(���^�Oo��0�=n���a"�?��!N����q�����Hg����Vsb�f~����Eh<��A���-�8�����0#*��=1����!��H.�A�S��V�EG���@���X �Xv��������UGF.���G�����Z_O�z�9{�4�K��5����1�$��w��"����YX��mP��.J��I'����R�`���23�V7�����jY�����WK�����9��A��2x������%���>'�7���};�;�>�w�"�����Go!���g<4 8$E��������V�������;��>�R����"z.�^��'���L@k:e��� Ye.�H{�"����������dD�����_dqH(#���<IVY�j;_�
�1+l���5�C��*������N�W�`��Oo��|y�6���|`F'�e�B�d�z�~Q�����_z�{��A��@�����x���;�Y�w3�{����t7���64����B����k�^����6�\�Hb+�h�1q�N<i1yU#>����Ct�!��d��0�j�t��;�>��������-�;�-��~�mI�.��9*.�����{E�����Uc���%z�4��#�\~��A� T
.=a��DYY�] c6����g��v��m�8�?�����`4/�y���nf+�}��5a�x���d�����W<I{�73���1�����F���@�cf.��hO��.�M�n$��!���K����rM�=��������+���JF�tMg��@X���U�s.
<��@��>�F��j������$}��Raw�	�1���3"��x�@E567�O2�����>~��m6&���-]��d
�I��(�a�"�=�$U8gH���H�@N���d�������Jk5}}"aY��(
=�K����s~�=\�>b��������P�Z���=��;~�)��I ��q��8Qm����7��0������X4��8�����6�2���' �O0��L�1;9j+��P�`�n����h�KOp�pV��O���� �d�.LmJH��)�������rC�������N6 �����RI��l�{-��,��X�����U��}�3�i���/x�����hn4wb�7�p=�W��a�����f���zu\�<6����)?N����X��N������5J�r���.��i'�4a��xC7#��G(�G/�Z�G�z_<�,/��z���.���<�����rN?}%��,���}%��$mcPy��7%�	�G2D�l}F^�QT��Y�	�9����8���^x,�a&�gm?�E��7"'S��I�[mnc^Q4�D�2z-/�t����,"��H��R�'��.�{y�����L�%�:Hn&
�G��g���6s�S6#������6�����g��(�]��K��3��m	���D^@h��[4��f���3��j)C�=j�G�8�zhJu�A?	��t="�c7O�=Xv3���vo���~����g�����[���u����0����=���f�t`3���M)��������\/Y��B��+��'��!���n�B�K-����"�
#���HA�w2�Ps�,�����V�)��>G�$�FY
�'��*���F���h���e����l7���N{�An��1�H;
�u���4Y�e�!|���Q�R|�YQu��Qtn8M6����d��B�;x�J��������b��cnm1n2q�^�@ ��GP�~Z��T����{��D��w������jVx�����������s�����G�}x_�$��Hv���L&jv����z\LR�)T�XN���`)2:����������y���h���nj{9g�$�2����+<�txj�
�I��$���S�'�fQ��#m�-����6����;A�[}��.f`�����
�7�2j�x[e^������q|���Z�)�3��P�S��,c�K�7KM~�x�SS�t0���\���oFM=`����������b��|�H���t�����C�v�a�o����4����_��#fY��u������0^N�������1VS�o:��B'�a�
h���~�k�?���������m��%^�%�e���&p�1]/vP������]��3�I��5�4T+��dGl��������[���w�]����8=�L��GK���/i������T��]7�����T��.S�Q�E������H�]��2�CK�
����)-�����c���!b��0����M���{XS�^���*���Sz�X}�a�������W�E=VN�}/��R�D�����`��;5��m5�����KHZg`�J��?���.1�[-���w�7���Q�E_�(�������
at������7���'��{Rc��B���P��u2������>�N��<R5��OL������X���E'K��S�z=rZ{�c�,#c�5yV4!qW�j�sT�g�G~���'�y{��6�����k���Ia�7����������n�M��5y9��k�������^B��y�F�����Y��*�6���]�����
��em����)��;��"��8D2r����h��V�����`�>��@t��1M��V���z��mI���Ov�L�9��14��$��uEY�����)�@��		��d�B%��T��(�v4z2X|���Sh����aT��l
����(�Q��[qCa����E��h�B��^C�i�G���V�I���IWg���\3E!\w�C�tj�1?IF���}�w*H�Mj/�p���b��������1���mh�2;VC��BA�.gl��U�^�0j;0P�2e�s:�"qan��}W�X�!��X�[A�+'i���l���,��{�0p$��u��{
�y�1�����2�qDF���8
�Yr(QSV�M<FA�ll��n�~�k�a@�)`��A���A�2c71���t�i&��C���J�[�m�6�QH�p��0������CE�����������3W����� /qFS��JY��hH
��RF?���
�+[0���g3�u�X�~��1����*��wA�C�����;��T�[WL�^N��E���?uPL���H:
�&����as����Y;K�h�?�&�|B����(j��6���O�!�A�(��VVX�#^xd	E1���p��W��b�W�������+�����52�AF��JI[�[d��[��~�N�3G������kM���B:�B��-�^���
�}K^��~`x�Gz��<����-�
/_\��V�����fc�}�[G��t����Z:�:��aV\
&=�wD�
�%�xg���S�~����L�(�D
JZ|�	����g5�#����
��i�+M��`����'z�M�R�6��z4��Z�@)���3!�7"�-���kU�?���~B��
s�ac�2�vj��W��N�dr������ut/\qqev�c������Z�W�4�Z�Ubw�B��1���"yL�w`�����O�a<�w�i�i8���]��O�������'��R�o�������q����e�� �<J�����}.p���g"T��2�L���0py�M�����Hq�*��?f���m������p��Zol�(�����s��"w<��������uG?G���$��g����>#��d|�;���Q��e%��O>��W_rI6���Z�� "������5�rl/A?�S���/gS��'�6�����~��8@(�L�TA5��_���!��[�u���
u7��Jf��ml�-KF��=��fS`�x����	=Lz�u2/��!K���8s�^su�N ���9�I�la����>B8(���	��4�=��!B��V�fP(cK�Q����9�b��$	���r�������+]r�A�{�e�x�����Uv�t+A����a,��
�e�hX�����jm���#<fd��I
CW��ca�0��|��Y�{I)�u�u)�����#�Z�-in�����}��{-������J���4������H��8��*Yd�F�id�������h�
C��������(a�&�����g�;�wQ�{���������)t�O����[�a�����;���}}��w����P���-�Q����=��N�9���Ap��C������]�9��Ky�����w$�����
o������0"2�����Aa��E��`�d,2��&Q�.��L=��zA�=���&����tN����;x�Z+x��1,"�����D\>������C�ew�	
x����@���7�H|�%�����W`�=���N�����f�4DP"��S���~,�W�g>�����4���%K�#�<��L��fJOb%r��f�lC:�7m���J����m��������N
�[��>E�%-�zr6K�N��S�$�����[l3rD�+�	�3)�'��a3"K�������e�C6�y>g9$�%��_P��/��"���4���3X���8sxp�����y�rHt�j0N3�����G��JQ�r����j��% N����[d���E��{t������h=<��i��C24�2�;��S����c�D���l\V��\�R�5��H5��l�fxN���g��O_><�|:>������)�$������f�\_o���zk}w���}��r~]�����8{7�R��:I��+
��T�� �R�t5m�>�y>�:V\Q#�6�=��p�I���Vv�*����I����0V���tu�l�c������]"@�JZ����>
2{����3����8�~���}@��C?50q�[��S�>[���k�1�������K*�����k�>WD���{y�,��hb�u7~a����j���y�v�	V���,����I�nL��
��@��}n���
q�����
+t���}����l:uO^��q?�
�`��1t�+�.�.���E+���d|����� U�X��V�����zU?�pt�������zi����|�8'P��Iw�l��u�e}�H7�q�[�O;�
����6���3���_�(���/J�G�����������^W�3m����bIF{/�Rq�84���<_���`�����|�pI����|\��0>��������B*8'�5��r��RM*UX�������5em��\��]r�]���m�yK���@����[��^/�X��<��-�K�~Z��\�������}���m��j�^��T����0�w���������<�#xJ��Vr�mC6	�!I��D�h���d<��~��#��1��8��|2��:��R�����?����}�(j��K(� ��Z'�|�i�D��Z�{�!�6�:I������&��-A��	��L���S0��/����IrQ�R�V�uqA'K��z9Dx:�v�51�������3}x���s��w�+�a��f�  Mv�����]![q����p0d��'Jz�fV4���2�
���c��B�N��	P�-L+����g��v�a�K�e7�#N�����Oc%�?e6���C�v�j�-5�T�D�9���O��=�����]N|dN<?B�����/�!�T��I��3���	$�hm�w6<������O�v��Jh��,[��cE�EpF"q�c�D�W��u�����
� �lV�� �`}�������)/�������N�.A�-���,c��g8��H������1�6c���@���`��@�����?�j���I�a�Iy|�+��w�	������.+���#:���P��P�$�����!��g
����]��P�
��G�����Q����=����E�>����/����R<�hOy�M)6�\�.0���s#�9���������?��F��
�pi����Ld��l~W�o'���L]�����iu��Z�>��b���VM��"�j<�n
]���\V�V�	��������~.�v���r���/����/U�)�����x����$/Z���x��+J�22O�
�x�J����(<uY:�<�}|='3��$�g�b
t_��H�?G��P�n��y�K�%���(��������h`���B�����.e�2I�xA��@F6r������W�=�d�����f_�^�,�X����>�����j��Q����i��i<�]DTS�	��
4�p���������$�^��T���%�,iJ?��`*��78.HD�dr�z�	&�x��o;��F�-�����5����7�����{����&���$������g_0��<�t���4��*��o&{
,��a��)�/�	�;f�-G?�$c;�q�%��4@��,Hj��op�P4�
�t�W�r��y��g'��e8���
��q_�6�1��#�����@!]�"v8k�0���[T���^h�]7D�W����r <['W+c�����e-������\4��
{a�l}���:#�����p�aZ4`���[PBZ*%���������%�kUr#���~JK����������X>=[+�B�(�q�l�5Z,
t2�"5t�/�(�U\��*`��c'�c�	���}]?Bk�Pb�4��*r	x��]�"��N�������;�9
����H����U��+s������D�<av6�W��(�w]�v[{�������r@.��[ >p���WJ�a��`��8J5Y)���SY	��j_��|?��.����(M�O����jW*���;V�bG$�������+�������*y���_>�?Y�L�	$F�S���|�*J�\Sw`����tGy�0NM��.��A���z��B�2?"rY����zksc���'6�cPnaE\|\TI�*	�_9���~���M�=Z�����F�
L��'&j��E�?!�`M�����}T��������9���Rd���y_�yQ�2x0v0#Xkss/����@I�P�#+4�n0\h��2���^[Q��
�����VQ��\������>c��$�������@y�6������X�1.��;���O��]B�$[�y�xpt�P��#	���������������BoN����k�<��A��m���h8Cju���J+��%���n��o�a�\���M���Y<�i�*�Z���I��k�GDy'��r���gP"z�4�������j�pUs�-�k���l����_<�[,�Y��;�o����i�u��)�l��)�FB�� ����Ab��V�3�Y�s��������v�w�~�4�5�v���P5�kg�������BkN@K�m��(����P�@��1Ab�h��7��$(��
��(	�	�F=��F����b���R�c���^���_���)g�n�wf�a��q*���:������0����(��m���>9
�0gU�[I��*�������h=���Yp�*�����q�e%J�#O����G�����.��{��t��}����7h{�z�*�_�4a������4#4�-um�l�����F��K'_$5s�$j����>C�!��>P���7a��st(3�)!� &Y�1��u���Al��@h�����>�,�9���NMk��l�uv����Ur�	�d���h��T��F���G!���Q=4�:�~=N�m�����e@q[�pd98�VD�0�A����_���^�,����SyJ;R3*��Zk��Y��|��}����<,���k[�:-���$�f+��H,������"��@�UF�1)F)lX�
u�<�l!�gkgo�=�^�>6T��D��V�Wcq�m�cb+0����_��/��#�7I��~/�e�w2�������K����t$��b��5En��~O�������|yJQo2Q�Xr
��`����Wt~;3�
���<�`�Z�i���p�/����Hs]��~��"�l���Vn
�SX�C�`2d���j��^���-2�7�\��q��*M��� "���2�R�&���LZ�$:mJ�����j�$���^I������������F����E 8|4��fI��lC$-���uK�L���_�,��X�+���K����G�a|{���y'�����_;����%,b�2���q(��������=::$�~�3#R�2o�Z�U�	z�rn���k�%��8��u�Z}�C0��>��1�]�E��0����"[{�9=��2�:��F%x�+U5��9���BB%s��z����`�����4�{��Og��|����[�a���'�~K~�+z9�Q�ft����$�.��t�vh�����k����s&eI��W���8����c�N�6�6@�����$�8����lTduZ��������>;:?����������?J�bX������C�*������"HW!\7���H���sX�_���M��Z(F/����sq��G�{� ���1��}<�TS� ��J]c,��uR�����K����:,c���2�$g��a=@4S���`Hx<��>�da=�����Th�Oh������N������" ��i2����
iHd���8,c�@a���)�8���+����qU�NF=�n��������B9��� �x�~��x[�s-�������8�la@��qrt�,��!'���8Nu6ab���<�>Ywh��p���T����?l�O����s�=�1�H�rd�|ud�?�.T�v<B�p���.g�d����������J����g�5��5t��\7j�W�N���__"�2����l����is��
�h��6�7S<3QMd���iMZ��G�x����0���J����fs���v���(@�Z����Z�k��x�/�H�8�M�L1!��{�� 7@�2��)�z�o�@:��hm��������
!r�
y4	��p��Iy��'�c(9����&5��M�<����#q0�i���������E9����[�60,a�c#:?:�sG��yN���������3�aV�� �;jx8;$����*2��W��<��	-�I�>u�h�E���IM�Q�I*u����_m���M���)}���F�������@t|7�����aI�$�"7}h-�y���8�Z��"#y�v��
q=��[[cr^
�������"k<��t�|?��!�)�c�����n9<�����
����J>Z�����-+o��E�!"����$tF7�J�^��m�Tp��l�����Q4y��o]��{�'x�S�rIY��C��s\O�����+��X�+�`TM��'�^���� �)��6e��)�U��A��������q�vq
�/]����**�����aI������=^��
�5�vk{�����p4����]����Q���a�{v�*�J,�(�����]��j� �v2sB�n: \��14���%y����j`L��'��|���af�)���T����P�	l�sN���E�SnHh8��1�FGt�b��.d��B%��HP�
��i
���(��a
 V�$E�Z�$�kT3�z[
�C��+9�w��'������x���;��=d ����@��]��3���J���XV���|�DMY)D}i�:^oy��T���������k�\���O����Js�Z]u�xk�+9G\���>���������**�'(�?�4�]cO�����Na�XS���� �M�D��{�����5�s�������d��-@x�K5��ci���!�����`������N[�I��f���H6kq�Dj[���/�4�HxB���-x�{�	�����)���I)�c'���^��r�	��zQ/,�l�n76Z�+ko*�l���2�o�����O,�����l���42
�&f���8�7m(��J�Mo��6��`'���5��m�,ma-�N!���^����Q��������D��e��1�����F9!�K����2T���#�.i�����;�����/���A�i��0���
��d�����<���x��~���]�Y&����f�'��0�T������B�3w ��1V��95u���+�e�x�W([TT�>��wXsj7��]<����T���Jq
h�����x~��|�<�%���r�O�[oP={cM���d�T>���&���l=���MA��S�+��A�B������fkc������E��
���L�0m�TF :��-�����N��1D�������7�#��lOB.oO��'�u%���%�a����&B�[0���w\}�6�������h�>������t�u(��I�%��=��	���1�w�S,��Y�{<�y����>��J$�����~����+D<Y��k7�H����$��ld�7f{]z��@(�!�P���4� t�YOP��H��o��#
f�P���
���:A���C��R��>U�$�#������U����
�U���#�*���{��7����"V_��(��� Q�l��$-��q��n����i���=����f��3������d�.AC�t�;A�t�b������(����w����4�Z|�Jm���������O;eV�RQP{��������?�{�?�Ls)�e2g�PnK��D�>t`�X����F5�=�<�Y.�O���/�
%�<�['2�7��I�0�t\�='�ha^���0���v�rUb������|����Yc
�Ks������A(���]d�A8�N�mA���c����T�'G��R:�3�z�q��[�Fu��P���>u���w��$�jF�d���'(����in�	0I;8R��?"7�Ir
�/"��I��s���Pcaol�b������
��:�}AMJo�ZM���'?�P�i
g��r�u��.�r���-o�^P�
(e�Q�9�*�Bc
r����=Y����t���
i�����S�'�g��:V��8e�)��<����I �����*+���aC���M��$�T:�,�,���m�2a9{2.�?�u��7�SB�Fm��:9���A^����a� ���$eW	�W^X����8H��	�T�-�{��;������NSjE�u�l��}=Z�q\8�����-K����t'v�H�R��o���Ab�n�a5�k0��3�V)����N@��,��3���b�s:�To�(Y+K������l%�M��~]?�W�x�����8����$��-6��EJS��L.�R�h,I���9�<*���["����}�YQ��	�;��h0�^���=����;�"�<��$%�+<!ok����e:�� ��R�O�OX���l�m�a<�4��<l���XH���o��0I���n�@a����!$/v�	>
��b���� '��O�8�(X	3y��I6�o+5�CY���m��v�h
��u��Y_�P�d�=h���s�������<�6�;o�+X7`j�W3j�����e��>Mu����c�������M��*����>0�m�6�;�7�53��Nb_�YQ��u'r^����^i��&4���q!�]r��g�A��R�UN_�|�<�(�����G���'?���:\|����n������kL?	W@��e[����{< ����^\��dN�g�����|$N��HD6�lX
�������/�T��FI�"����]��96i��h�������P��)R�Z�l������P�?��������v�^Lv1=�Z��R������_w��/��~/Y�)?`�Z�y�N�<A�\��I��8_�B���.�DO���PZ�6 �n�?��� ���:�n)��o�7�}6|������������&�C�5����
��@:Ru/l�ae��"�Qe;
���FZ�\}���;�t���_ gN�0�Wc�0��M'�.��Y2��8[���d���YE
&���������2J�	q'Ys��Y�u��$Q|6�"2D"�-����w��f�|�N�5����<����L�8�7b�kly7�oOU��D�v����[������)Q�,`�`Z�7j���;�� ����0���&PL�����C������Gt)���7�y��Vb�@ n6�#�(�\�m�]*�)*�Su�@xK��*>il/$��,���Q��e�
���H+�����Z���N��l��Xdc�����4I������	�i���>��8!g���
�WX�����E�VX	7���v�+X���������D�g������-�%�hX��p�Y��Yxn�e������w 1Qx�0�d��U�
 ��9t�A�D��J'�z���_�I�2��#��-���*�A�pR��oY'	t.U6�
|OA1��\R�'��l6����Y/a����6��A�)���
	"J]�>!_v�#�`�T�*�&l��[em6���"�xl��^��n�`'�n�S1{!/�������!�����_rK%����D����	4h����p(a����H��)h�I�#��s�cR�@��t�c��,W�R�ra���$�9�a�<�V�B��v-Gr��z��e�bCW��8���u�*������Y���
�� �WM��A�Z�g���er�D��x����]��w�u������m3h���6��Q�����^��2��*�*Y��y@]���z�U�^����<���a����<;b5�J�A]������� �W{�����/r+�:��]`������~�j�����Ip7�z{8���#�1{�A�E^���
�k���[�~���>��G����/��wb����$$d�oo�`De��g]���,g%�~�
p$���ml�?����[�_{I�,4�:�8�����v��-q<�s�M���W������IFG��Zr_m��6���F��g+C���sZ:S3z�riVn�O��0�U7�W%�����~���
|�-�xz��$B��#M�Y�"���R������i �Cu@�l�2�\,!I#!���������������6\|8;:�p��-�`��a�c�V"��5Qv���;AJ�aEu���N�bQ���
[�`J�v����� �[�3cF9����_�%X�'�_j�Zg�\,C��4���@En��d��r�r*��QeF��N����Y4$��<,�uHN��[AR6��
�������GOG����:�hU��.!t<�*�n�5g=��$��`����t2T��(O���w���u��P����^�*(�0�bA�]����H:���E[�N���7���/�Z���U�!����)�zG
yR����v����_R^H`�;�����������Ps��;o����u�O����������!����S\S����������Ua���h��:6�s���xT�}�(�r�$� n:�������g}��Q��G�����W��4"���[A���=��E�$+5�x��v��:�s,��Z��� ���A��z��O������|����[V�^�G��� pZXM�Vn�Q�)q��d�s���T+:a��w�|M\���a�~���<�U�e]d����W.������
 ��Z�1�-���K�����x���*K��`�����r_��pd����i~L�b.�
�w�����2@����pS[E�my�q5A-B���c�'�y����x�w������x�R�i��I�����L����
�o�[K H@�����<c
������1���(�
7�
s6�@�^|OC��zn� k�h��j��,�1�t%��=���.�����E��]�|��_I������
�SR#�*��i���A����{-�/��������:�� Y#�����%s�q����U����b=���!s=��G4�1��4vv!��|�����RJ{�J\�B��*�{�X�Ke�����:��=����z9|��^i�yVf���9$�
��L�Dhc�.����r����������A�Z
)�H�_���o:�26%T�%��3��X�K�o��c}�}�������<
�������L��(��3#?���}�U	�����^�9>�h�Lyd��MH�##�<�����s�������;��3�����
B=��BIEj���ZL��1v���s�9��8���������!��rw��U��-��es<|^S�R~�C�����w�JZ����F�t
t�fA�9��"mWF����8[:�,:����u�������w�� �g9O�"�����e��H7	���p��	��[���G��kPp�?V<5H#rz���;K-~=M�5�i���y�J�]�g=r���U����-�Gg_�%-*U�}2Z��^�s����� ����Q��d7�������[mEr ��_#o����C�>��j���|�]�u������K��M}$8M�h}%R_��&�E�C!r�E���x�_4�n,�������?�Q��;��C�.�U�D���{���7��g�Bp�m9���Y�pW�/��q�\CZ>�S7
R?������
��S��&�X ���W|�9vY�����Z@��x����:�L����~�f��� z/�|s��}��=/V�V8�/�`e�Y��[���.g�'c|���q4��:���	 �Ae � �/%����p(���?�37#����K����2�n�S��,�����}J0��1`�,�<��:C�3iUC����/����c�����T������D�9�T��P�v�)M�"�B��sT��_�-���������41�<|����,�:���=�V�K�LA�m���n�kY�& ���{:������V;Y5���QxC����
��A$�1��������Q������~t���� IC��[U����Z�<S��i��2�K�\
P��.
a�4K���Yvw'�'.qD+@�� @��xC8������`$$��R�U����������w�m}��J�qNcJ��w�Q�Y�m6��Jr��y���$(1�� ��m���s��AY�����d`w��������p`�]�~c�R�'!��F�tN&�I�%��$�=�	f��oA��Y+�'�b[X��M���������_����`�"Z[�<����zPC��{bi{S��	\����<���H���A���j���$����E�nl����p���m����@9�:��w��P��H-�xj1��,�;Ry���W1~���X6"��/� ����/����z:a������'
���;�x��=�pw����r}����\��dO*���s��h���l�1;�D��'�?390���pdF�y8/(N�iE��s))'��^`{2��i��;0-h��F�PW3`�������>�����V�2;+i����(��������|:�"�M]�
��}.pfG�TQ2r�9>���J����Vx��<�$@&d����Gb"�pun�a]�;9F�9��l"o�T���n��se�}:��JcdT��u@������O��j�yA�)0��x���~NR�i��_I�����_]e����l�����a�<E)���3?�/1-\o������R��C~:���<-H���>��t#6�H����!�}�"7����L����9b�Q[W�a|%.���BpV��<��L(4C)KHp��E#�0h&�	z�^�z���9�Ba��������o�����?dw\)+��K0�AI���pj�\D$�b�X����C*C��A�0�E<�.Y@�&%l<ncW�R'����6��!���p�,4�'����F,���y.���2{(�M;1�����U��dL�G	��dg�E����J�D�w�1�4#���m��HL	��](M����L(	�d�8Q�@3#���hnz5�~�SqI�GUlR��$YX[a��/-�=���"n�(�q�����4`2���?��<f��)�Rg�������\��~nXmr��������a�!%�-v��$��1�-�3e
�jS����%�k|N06y������=��M]����w�
�� /=d��2��#��<�%���>�x��G��}r��D�P����0���F�=�:��z��rr�
�`��V��ws�w�5J��K�7��]���g8e�����{�a��\�~<&�"Gv��5��s��O���+��d�h�H�t��\!��$��\����"�'&�����s�M�F'��~��Cb�����+�o���E���;����z._@S�h>!��)�;����<P��Q�1�TF�V'h���N��U��fs�\.���n�X��e-o^J�]�0&�����p��xUW�y�n�]�Wk%Q"Z�,����z�g���y'��`.����z�Zj���#y�n>E�z�;�Z�	���`�e�c�?��]�_������g���yyF��~	p{W_+u�������w��z���Q�+��*�k����%t�����xB`bS�����`�}�j{�^9����:��0��=h��W�t��^�(W�[3tW�����e6e����3�3����k���_h�z8����S�����$�r�7���[p63CS#
�|+���?��8����� ���x�+t
L�--�!�
K�����N!�elJ0���?w���R�w�D�6<\��+R�(l������t��7�IF��
p|��%���0�?�	�b��@RsP��"&%�|K��w�t� x�,��L���Fa�} �7�B.S���B�������`���tn���k�������i��G�������+����5�|�"��s�qL�\B�t���Z_�9[�=��q'\�Y#f��cl�%���?]����0:pY�G�E������vJt!�c����?R�X[@�x��5eQ��m�J��������h��;��	��Lo ��Hn�{�U$���9|������q�c��K�{�����AARop!_IG]w �Qt#����lc@.w���L�8���NiNb\|�����t{����"�4��q}�P2q�\w%�)t	��],���[}�V5l��5!��C��_2��i��j|��&X#�����A�o����(%�f��.�`��"�Ft��������(�[������d����q���&G��I%��?��Sjq��Y7
����X�����5DL�J	�!~!S7''�.���Hc����a82�\�j�v�CH��}��x���������e�u����Z<2�L�����j��G���� �Z4�C-A��:4��B�y��z��z��J����d5r����=�C�`4HEA�r�?-0�	Y�
���z�0Hg������U��;}�B6��<�1B�l6����[�8Ii$B����s�v�b6M��3���GIL��":���Q�c�,ut9W�	��#o[��t�y����=sX���.b^R�-��8�~Ld��x������JfS�s�P� ��5�!�)�{.�`���Dvz~tb��^	���_�b���&
w3���� �]��
��
i=���4,)d@���Q�H	FDE�����,;[��T�b�DH�[�TX��P�H��>J�	�$C�����;V������E+�1��*�&C�@�a��\A�V<��#�����5{�c�/h��%��=�����n����k�C���N�x�8�C�&��iF�n-��,.��<!��[1��>�L����!V�5��a�5GV���h$�^�$����0}���7Y��7�=���tj4`���Y�x�*���	&�zI	��\813G�b��HI�8h
��<���>��d�3I�f���W��8c�zs�?�.y"���*����Ku�'N^@����������QKn����Hk��w�?I�\�q3(-����=��|NO�$_}(��.�64I�~0�d��u@��
$.���K��x\�Od-f��V$IBPR"��C%��H<����O����h���P�{@p������=K��*��2���j��{�����~��.+,��,z!���A/�L�|E�;I%+@�����<����m�d�-^��U�2��y? �C�H1h�+d���?"W��=�c~��N\�%>����L�������?�)�����oyZ��qCm��)����BP��N���5AYMgJ.����YPwx�D��!����NT�2�L����W����^�����Z��
����e'`��p�am���<������j��y����2Y
�D2	&�#q��F����!� ���wpE�g
N��W��������e�����E��4�D1����Q(l���Q�J x�g�`��/T'���U�eo�������l����d�������9��i*[���n�c�\�-���%�W��������W����0\F�Br!�Q,�������}���
������?�z������T�������,$��Z�]=m{��p�,Oj��}%��i�G,�y�~#�!�����oK)������U>9
y���w��U�(�B����Pd��8�*�e_�������A��NAl��1jl�1L��"���!)�Bn� ���Z�����>+#����2yi��kX�(�GJR� B��I;�]t�u�%o�	�Q�������GD+������b���$�L����@k[PMRf�v��������??�)r�H�:��=%}�%-�p(�'�C���QF��Q�*s~�v���3����#��y�oL�Va�O8|K��))b>����qH�3�l�QFVs6�
mq���A�&]N���)�L>����R�p���z$4-����G�lL�	����HU4���>/<IQ)�@�$@?��q~J�Z ��b��F�Q��s�h���S=�jULKs�x��g���������Gshi�����7^eK�c;q���mj���|z>������A;o}����<"?V�
^��f,���3�#�:���W-Vw��?�4e&���\6��TJ�b(���	#UI]��cw?�g/^���]�%hI�������t?IW����#�<��k�v��i�0%	�%�b]r��X�C]��|p��%�]�uN�$J1h3���2��r�Uy�(z�,ad���]�Pt��������Q�z��k��z�G>���Oi���w�9+{���~��&��fO�/�"��^�<9\���7�P���>21��G�;!7&��El�����_jF�t���p����we���t@:[���[�d&Gb[��Tb[=T�Q�H,Km%
���Q�=�9q\���iC�lC.�`�T�W���l9N6i��
�qn�RF�r�Ei<V9���'�o�����w����R=9��J_��lm��8;�/�;ia���[|0k��4���-�O��j<��J�F�v�	��F�Sj�7>�	5���g32�)c������$��,�[�0��Dx���0d����6%������.Wp���F�C��o���$+���yr�]���T���;����`D�|���C��b����#�"���@�E���5�����^�R�L���)��__{��dp'�K����r�z�P�<9}Fh�����9��C/��dT��R�b����?N�4j\!�i�HB���H��	���(/�����
"��;��t	��+f�M��
[�|�������)w���MfkR�Bl�b�Y��	~��2 a
���IB��G�@L�z���$'�w�FOZ)$�����X����#�����K�3�9{�|/
���G��������4���qi�$�OFN�g�5`��)�ej �$G��"O\����UHgH�f�����%'������1��V>:��f:Az=�70��'�$���x2r�{Xe����������F��#�$�+��Y+u�b�����B�`�����!��o���
�UO^���b�����h��M�.����W�(����0.�qVi�"�e���\Sl��W�������Z�md;����
g~������k�g�}�P�6����US���~�o^��}�q:C���
�4=OjYl�R$�5>��o�Q-h�j�$����(794R��%F�������
�,���$���2����#�u����&�
�zN�G�f�T�y�N�Qjt���!���6WG�.�M;�=(��9�I���(!����
����A�1��-�C����y:�IS�.G�>eyK��c�"JZ����RX������43�v M(�U������I_���C*��E@)�������M�c��N����^�o��J�Ar���/����2��xl�5h��s��cpb�/������i(�%\vc$�>f���<��3���C+s8L_hGDf��k�X8
��4�w���$Rs%u��6�g�4I\)��&rH�@'��L�I�_0���J1S�$��}�(��^��l��	��M��q�iJ�4�l����o�x���$U ��QN����#���C��$d����T�Y����xu<��,�����by�Hn��0S�	���	�d�$5V��������bS������B�R�Q���}���k)����
�e(����Gss�\`������uL��N���+4u���n�C�j�aD�j!O�Q��s��s��u��&~����c���Z�����G
Qe�E��GB���kt���u������g��_b�
eB&"�Y�|XI���/�H%�'S��y��X�������F
S�N�?��g���?#
�C��A?1$f�V]�-�9�t���+��%���x$������4�?$jPVP>,Q�����mL�#�qn^���Qoz+;������	m@)u������x���7*������Jl�F�Z<W6������jG��Z�h-�'@��I$��������6��"����S�c��fC�I�b����n�Wf�ID����%��]J�n�|`�2��OW���`+��1�����t�o������GW��<|��eI9K��U�������8�A�Uj	��Vm�J	[�f�-9}��1��2�I�����^?����E�$j�L����H����W��^Y�ulx�j��aFZ�w�$e<i��:��`���=b��
�����&�S���K:4��J�k:�dZ�#|J��	O�E���cp�<^����4��>��_����.A���!A�����?���6{ ����n��U��%����np��f�K��s����K����"�b(���R�p��w�M��7\�.�{H��"5
"���/NUh�k�w�����j�`VTx��*���{a�+�7	��6�������43;����d��$EN0�N�0K2�tj��!��s;��Q��STD�@:�� 5Fa�����B�����k.*M�'�h�z�gS:�����,y���$���$�-����e�����`����y�(f��t��������u������-���F�����x�7��&��cM�l��;��JFH��v9�������.��C�p�+�A--�}��A���-
Oq��*�.
i5�WdN�;j�e��E�r}�n���-�l	qxJ�� ���x���W��B����@�.�Z�T���M�)�u��~Cy��}���s[vD�5A��2-�R>7?���=O��Tr��#�G�e�.����<�����������n�
���z�S�W�t\	�U��1q�
��T��U4�qn�TA4Y�4i3�y�Dc�����C5�Xv�i�a`�
O($#!��0������}k������[�����H"~�ly]��(|���V��iy���:7��~(�����u�?lz���'8���w`D����UQm��e�����N_��H��[��6����v���%�<�"c��7��de\��t.o1-)I���O�hU�@����_%:;���������}#s5�,�S�Y�`�t��[���C���U����a$�B�=A��)�]��E%����i�(����l�e���,��1�M�1�v�>0���pb8G�yv�D��f�t���j��v��wm%���I�ag-�k��hs���z5�7tM5�@��7B^A��XV����Wb5K��-�RK� Z-��x_��g1#!M�n�C�����z�������TH�V3���&�:*z�b����1_�/F?�x� q�`�q�o���}C1�QpN{�N
�o,&t��|����n�
G	�E�������o�i� �r�x<D�
�=WW

J������#jn.�J�����]�����nl#:e�GP�?����{0�`Q�$^@�)f�C6�J��O����A|�N?c���<L]Y7d��[&w���U�5�pk�5�p�]�j��c���id�4�#�6]?1�\88@�x:�����@4�����{EJ�4C�07d�6�,��_��Q�h�rW!U��;��J�k�t�-���y4	J�J�PQ[�H���r{@F^�Y�\G���;��	����u���La��[3G�!w%����]<2��%�����2n!]���+P�<mNR���X�w�������F��X���>�a���:�l!�f/���Z�|c�v��H��=CD{����+Qo2��]H��$�v�I�&�63`��dL��G����������kM�t��7�~&�N�=3)�Y7N��	�
��e0�������7�����������>8�jT���;�����>���Z)�m$F���SI9Bk�L����p�c0ZE�)�4�B���I ������`>��JdF��y��)�5S8�`���he��A��"�����q�	n(A����@�!U�b�x\G�&6i�K&EV�)L15����b�7�6������l�[�PT��]�N8�����7���)���U�S'�^��RT�]�o����w�����Iopv��`������~"N��$���R������G��1�ucJa������(m�Hyh6 ���?@�'<�v���#�X��Z$r:b����'S7P;�f���-���L����A$%<��$�'�)P�tbN:
�t��I���Ja-���6U�BAz}���~cH��rqf�.o'��7E���Q!b�S'R�&���j�^�Hz�\�NRK���PV�%����R����5o��������s�
�[OG(��������V6"5m�_����
���������Y%��A���y��:F0�h4�����ls���$��9u�4����1L6nR8�]pu2F��9#�t�����(�1�"�_h����'_=zW'�b�������Q�x���Q"i\�=���t��I3�[����n���N��NO�l)�(.Ya�-�?��x�2�"��
�:���i��������
���N�.������j��$��q8[��3
���X�RLK�����""��_$���9�mo�*���Z�������8h�F�Ju�u;����q
�����(b�y�/����j9��	�
%�\y7��e��78��8������KA�_�����xuz~t�X	����R�<��O!����a1�u"k5�����a7�u����R��`��[�x�r�-���YC����@\�&Sw�;:I�v|�:��6c���{�������Y���M�:,_,��P�A�O�zj9��j��wZ���*6Cg\�t�~��J������^�]	Wy{��������b=��
(�:B����^�;�_�Kp���T�������,�v��������#8�b�$�����(s~S����4w�����\�����A�22������<?���i���z�9L&�������%���z�;���U���0W���d�t�[��j�?��z������bop*�(���`+l��
Ht�����q�^�)1�=,u�� ����C�:�����>��5$��.�H(��OH��'��	�o{�A�,��LR�������FP]�b�`�����h�o��i=����>�=��$xR�����di�D�.��&~b��p0��n`L�(:SF���m�x�g�a���LjBf�h��}��-��3��|��V���!��#��p"���RP����dMy��>��!r�^������t��X�"�k�����{��?�����7��b�������@�A�0�����(�?(�e���W@@$TQBL�dX�8��qDm�=r6d���r��� �4��T2_����q��\�
0�0������wX1��������*�����.��B��z���F�[��<�D��*�!U[{5���B���(�w$0PT���|�����(�z.d]z��a�%�h�\O�W���.�=���#�p�?���j�.��Hy�=?��w<�d�%GR+��!I%:lE�:
�~W�����J����q|���1���N�5\L�����k
����a�k�s�Fmc�`�{�a��'��<->�R����QC���/��2��?�\��Q����t+wHlxi�%,e�����/����3�Q��Q���\�<B9��������_�E�m���O�p@�bpUf�	����v�V�Mn���
�.��������G�������������Q�H��R�i�9�[�����3~��T��o+���k�L;��[Gu���J�v�K���a�E�d���;|��`��y�������<��{�S��'q�>����;9bR�
�!��G�%�����!����bZ���rq�y.��?]2��s	�@�q��O���S�"�$����D�F
��h� sal8���TE����W�>�A/9=��'�S�[�zJiz����jgj�@5;�)09��l+����,%�7���1e-u��u� ��P���B�v8�A�FK��2�2x��M��I�����Y�D�
���g�WUv<�j����E��9��������k���i
��A��W�{����n�/&k�\7���W�	b_��Kj���	�E�_�%]��'�A�ot��^7c�@�R�?&���~s���?^��+��cN�5g[MU`�����u�'�.y[(�\���6$,8�1��0�V�a��oI�*G��^�
��P J3'�J���4���C�iL�����0p�8*+W�D���Kz=��`6�H�RGb,/�7�K�����*t��'=��n�����.��
��C���R���TX�f)�G�i�����iTk4-�
dA�'�:�`��������*�.��pt��*���o�2�>��mf����!k�I��R��imq��!�Q���Vk~���Hl�;�����7�� if#�}E��M��`�c�k��ox���?9f�z��h��0�[�)�H#���������T��?{&��J�gC�{;\N������-A:4���k�������������4��u'�J�;�V��(��'���&n�,�����������
vfZ�M���*OP������^W���It19��0|�G�b��Q�>h�e8C�s<FZ:�h�������~����:���N�6Vs�4�=u�t�l����\�e�h��_��8?l�lNW������bK���mL�n7���9=���9E�pxp6�*�PBQBl�;�q����5f���&���>���z�����iZ���#�L��o(X�M3)� 7Jb�
3����\�����(�u"�� �t�%R�d�F�9�F@�4����$c��x����WzEl�����	�
���������;����yo��A�\����������c��:N�L����xt_!�����0R�2Q��8��=F��@���
���@�>�
H3����}���	�KE�����p��b����7�(�d\v��K�}��gj5���w�F���K�]���A:I!FBB�5��v�x��mr^��������@�&��b([:��Lg�����E�':/!�c�p���P�������G��{<a��k�N��1a�*{$�'���C�Jm�^,6$|��������3y�i6�������6��	u�hrJ7���*
-P;���Z����'���Q�D\�\����B�;�_����2��4������i���{>�������Q�����y�bp�&��]-5�[�S9��D�(�)~����2�Q�d<�����
���X!�	�Aj7�?9%���0[��� �plg HX�HI[��J{�
�S�pY�MHM@�����|�z>��:(�@�P��>rC�$����[3��S�9|k�����l�m�U�r0FW��"t Hr�0sb�	�O��lE�sXi_�f2���]�6[����V.�mGB!����o�K%��%_��^�,��9�L�D<O��8�E�i4@$R=4�JK�`%��W�G���X�-f"�	*�Q�R� u����{�4�=o�"s!}��@D�>$D'
_mR�F7��0F�m�l".�kI�5a��g��~�6����No������%c']�&D)����^�M�� M��

'q�����rP�
s.�E���)���{�t��^�c��M�D����ao/l&�L��M��.E1���p����zn��*�j�Z�V�Z��`��o����2�]h�8U�v�-�������%?1Y
W1�����m	J�]����G�A���XN?@h�������$��XgU��P���:�C6.���5��	Ne��%hG�B�
���G�=��6�����W�%�P^W�35��pB >ozG�����j��&T����O��:P2
�y>2 lde#5T���ej���n�����]$77� �H��2.�����k�����xI���w��v����f��h5��z�[���7mf^p$J�j
-.9�7!�%�����Q�k#���B��Ie��{_���_k���oS����������@{�*��.��s�7nr_`����5�KUu��v��o!����|[4��K�����D��^�����#Q�E������FI����\�����lj�Epl������C�����9��z5�����]+'�L��>
&����&�v5�����x�Y;&^�b�jU�A���r,o�l� ����s�j�-@�y�V#�ET��r���*]��?�!S������
�!dR�T�ne������masy����n�o1��$���N9�a�[!A�{�m�����]���k�h���U�w�NF���9�$���/>��V���0��'yk��S�)�&��C��G�
s����A��~OE��m�yS���������xo����x�U|��CVv:���0;����ej��T��Fz� B�^l�
8��������k�<$�k2�
>y,���'��mz�W*���A�����7������5k��VP��j��jw:�Q�1�e8R��J�KW)t�iu����������������=������=50���JJ�������7����!��j0��X�g=�!@�gd�����d{�Y����v�?{�5�O�=�q/^��-�O>#�pS����'����l���/{G�Y������e�v���l�2 ��-��]ohR0[5���������������w9�Z���v��������x�Y�m��w��$,n��q�m�r�
����V�S[}'��s5h�%Hk�^�R����2�rf��T�������ng�������|���u���B�'G�G.�z�LG���a>���T���/fw7��{��KU�2|X��H��AS������j���X�a�n-y\�������y�P����5���^�spzTH,=S�?���a0�|�^`H=�W/h���oK����._����,�v�nj*
����yf	Wh"��g{2V���2c�\a�zp���I�M�?zgP���K!f�nG�}���NK��E/������z� ��k	������?�t�`X�?��x�NA�����(�Q����������($���f������z�|D������.K2��������(�����&��4�[/�+��Y��x}<x'������w���	�����d	���0������*��+�[>����LQ�3#���b}��N������qK�Y��1&[�����*s�X$C�+�y�^tLx>��r���Ad[�d��	�:��gN�9����@����?g��J�C�Z:m����OZ���1��'�Js����Z�1i��)-g���:�#�1����]�K�G�R��[;;_zJ�8�����[����Z���bA~��J�T�2���zj]��;����Dd4�����/��C�,$N��P,#���e���q�A����Wny�R�K���x����M��TF��f�3�7'�$�����#���!�_k�����&
gP�
��/��d:vv�a�.���7��{�eN=��c2���:RL�	<m��)0������o�o��.N��J�qB��!��,�&�JywIY+��
���}�CI�����
�1wl
)�F��|���� ���Bu��Cl�����O�n�C}��<������|�����-��P�(������+y�3m7�<�F�����{��d#��eW.w>�� ��!� r���M���
EV��Eo=
w(��Gb�����|�R~
 ��`����F�<�S7���x!�x�m�(!%�z�`����dX�T��a���������[�U���]���s�A*�5��X�����	 2��_�[��My������!�;������i�,��j��C�w���Lm&�	��B �8�N:��!�����)42�!�w��z�}��	E��JS�M�<�(�K%<�YR0�p.�N1����X��Z�u���S�%����2v������JX\�{5���2���*��j�,�5
������L,����AS?��It���7��;�R���7��N�`vCp��#����b({^�{�g��?�k��M��do����KC��[`�,[��@&�)��W(�K���&.$ 	����,�7gI��������a�x�)���Ux7as$��}N������2>�����l�I���90[����GN-��YJT�FL�6��g-�g������	\����RJ�b<�����c"/�5�x!��9�k2CL)W����ITnL���b����@2�U/X���D������.�=����"MABi�@��nW�[�RD�"\Kp(/�j-
��L��A����4����; ����(��m��7��$(z�'��)[����15�c/,
�*�6^�����+I��-��~���A	n�����z�jo{����n`�#(��2�v���Sn}�m��<V�.��A��@o�1S)S�k���@�{Oh&K�2�:aMp�}�(��'���m-@��!$��/=8��L�����C%O���n���:�����������#{����L%������b��UD�a��h3$V��
h���i�y��b�G
��
5��A��> �qm���u�+@cX��gI����@�,����m��o���L���Y~K�"�g�lK��hP���W���/�|tyy�7�F{�����KR��cOTHB`�
����P�ls�^j6�-��{x�77��uv�D�o���)��I�����+�8����7���7��';�)4�����FL����ZnMx�qN<�G�B��A�V����r�"������7 4�]���<��������#�,�&
'�Bo�����IKgm<&|�:��|!�S�H !�p�_c&N����!u�-qO����(B���BJ����� ��r����b,���{��l��:4��xti�Oi�_�����)�<4���0�FHOl};�F�3R��8�
m�=w
���t�m��{j%v�T5����C�_���N��l����VUg:.l��z����:'�g��Lx7�l)�|4h��D�X��m��$~;���{��A���5h h��2q�]��x����`��m0[Xm ��|���A�����pw�*O��?���03��-�00�	/T���A�>�.i/��[���%[Y,���X��������R�UM��k��3�YR�s7a�\#���D���w���6���Q������������P;K���t���(��nt�'-��e�f:/�l��O1�I
�[7�b�ZM�b��Z\x���?#�B�)����^����mIj�IU�|ukt_b��?�����~�k�+B~^B�$e"6����M�qT�A�� X��g3^�Q�Ib����!�fen�ZO�*`�'I�-0�������#��A#z���P������NuN���]����|!�����g�2���y�pMp �:�������31�����{���zMBm�j�����&`t�R��T0�B���K�2<|��B����p��z���?ba���[5�E`7�}/D#�pd+�x�0R|�w�il�%�����8@��z����h��Ww7������j�%[90�a����SEh���
�}�9ue-���Z�����c�i\��I�p�8d
]��v��!�;�F*��H��QzJ�n�D�u
�`LmJ(	��2����o�fx�
�u�b�T&���I(�+|�Idq��@(B2)A�51����O���`��5�(~�	9�l0�7&�q��������#Q\Q���(s��T/��;,���X	��}�B�B�����;�D_K�XR�@S�
��C��@���6�sD"5�
&�����/8q�9�e�X��=jgw�"LO!������@��,c��{R��3J4����4{P���z��#C/X/�k&�����v�q$���b%qgw����}Pj�����M����������E>��2�����	L8��r�}��V������Jd�m	*�5�����K1��>�_'�K��}�����i�4~5S���R��&j$���20 ��x(���SD#����{	��!�[�h���V��V���P��XZ�F��w�O���x�R2|�i,�P��������@�>^���y���u��k-n��1$�1�"o-'�7\�9��in�����t��5g������
4���(�ta�bh�����7���]���T9OCm�	��nDMm����N���z�R
�<��s����urRU	=�zJ	sL
')(B0K5��|/�,�xF�{;�|	�|R��&v�Z�q�v67Tw6d�s��j�����glU��.`j��P�H������f��=���j�
�=m�G_�BAp�=�\|��#���y������3z�sqvnu�c{�B��&K���� ;��(,*���b2V$���R���Sf!�`Wf) ��\��GI�w��FE�qF�%�����$�5I�`����Z���=jU*����h�[�F�$���$���v��V5�1'��[��j�����1�.(�������`�Vz�����W��E�nX�*��x�sV�@��)+��A
g@)FYo\CT��@h�P�{��i�C����_����O	�S��q��F�wf!�rf:���<�t�ij�M��f�
��)t@.,6�*\��x���9�u�0��Q���o����c�6\	��V!���)���C4vN2�"C�j��O�'D�b�
UJ�C�}G^!�<�ZH>���1��KN��`��g�d�9��I��c��R�9�P�EoV�d����v)f��vw8N����Qk7����������������M��$~����>��+�N���?7�/��������]�Cg�u���`�X����=.0��'���Z����]��W��~��^�����d�G����(�\�S��n������W��]�y������_�g�m7��6����r�$�J�s�,�/��WS0#�).V��Q��Kbs�|j*/��aVk� �c[��\j4�\
 Y8���#��4��"\��;����CpM0��K����G�Z��L�u!��A����v�yk��$WM���Z,��l:�q
1���d�B���
�:$�L�����o	���#�����2������2��������~��}�� �h��q��P�^seT&�,VN�]���b*��O����,�t��[�#0��4W������p���Q�uV��/3�
x�
������s�b�������zx�.���$1��|�>9W=�;M��z�\���v?px�������
�<j��)�I�������%=�b� ���L�K|��tTQp��]����
�/>b\�m�?��:�b����@p�+����`����j��2�8������x���ot��Fm��u+�fw��5������l)�3�e��:mH
V�_���D�3�:�Q��$PPt4 YD��e���b�@7�|��(k46�����/���p������������n�12��MC���.���K��,��/���"��L8����$�mK�����v�B�K�vTI��Z��2�b�?L�4Uc��5br&rlQ��J���}��(����84P�������GP{j(C�P�?,�
�������%�mCq�)On�TGs�D�i���b�T��cvC!��lH'�RTA���R���\H����J��h�� �Ms)-=x_bC�
���@�c��!{
�/��b��\-�@����X�=�KM�Ik�/>LrX�� x 3����UG���gj3�U�|�n��_+��
Z�q�	����[A�^�������P�2�������4b@�]���h�?����,Kd��	�����BJ�/�g����"��������	�J���y�-�N�d��������Tl�CD2���.���>F����?���5uP�������e���)���xF���_K%�1	�~��J��'�����DCI�L�� ����y`���8�x78��]A�9qx���,��)~W+���H�c�}[��������F�����u���G��q^���w:�|�G�%)�Y������8���~}rz�{q~y��y1B�[h3PJB#��<#_y���U����t��/�i-�q�K����L�w��`��?
�����<�oA� #&fV���*��"oN\e�AQ���&_�&X�i���^I����g�5��es�5g���E���U�'��b�e��q�`n:�f*��E$2B��>�t��a�R�L�k6��a5�3�btF1r��i�-������U������Ms��2�`-����.��Mc��������T�`��8-�����a�5�;��U���Q���t���W
$�\�B��J�*���0!�X{/_��>��]]���{}}�`������!����}�\���������~�n7)/����N9]�^��:3�H'��"�9"�;Y8J���i���'�u��p��%O�2f��$�����Y�t��e�?��]��f��8�J����yO
���xX��]�R�4Z���z�
%�V���9	s��a�y��'/
��)J�����tv9x{tq������'���wi����������)��AP�\������~��t�^@dr�V�����"�� L\]���a�B2���:��zC�������������9��
�W�]�!�����&9����
�5����uW���^�Z��`���M����X���f��{�H��������JB@-�\���
#�)%Z��UU���gw p�1a����w�%H���*�e:�dT��{q@�g��A�n�*����UB��/���
]�ZItM���������������
�z~�Gk��s4��������j!.o��u�9CY����=N�;���W����
��DY��M����xeRP��n��}b���?G)�/��O�t�������N������_��oC9P���C�-�Ol���{0�j���BxDe�vS0�/�+�]`��,�o!���k� b��g1��y��F�g������cy��RD�*f]��Q���?�����? f��^���A8e��x�C����2�D�5�4;
�F����g	cf
�{�N�qA�����8l�0�����?����p8�~���H{���tnP�ar�����<��/�)�����U���(�F0������?x${�Gg)h�;U���:�j�Q���'���j�%h#}-������{��U��$�����/�Lo��E(~�^�{���9�R	�`���z|wg��{f�������K�q�7@v[��2R����9:|���
���������V�W�?=��L;�y��=Q��l�����.2ut�<]`���	m��� E��o�p,y��>�V��L�GA�6rzs+Y/0kbz�a�e����ll���f��,����h�?�]�w���6��;������Z��l�3��M������x�D�=t�%A�F�
���P2u�!����K,�[����U+>�q�@��O��t�H�f��R9�wf*�Sounv������B�	9mbB�]��^�W0��	����'��\�[���������:��?rLC��b�h�>�|�|Zl�����M[���i��d}3{�G	3z���t,��5��8"L��%���V��1my��s��l�
����� Z�9�4��h��:&�d^�	O��6r�yXU>fS�8������$��q����qu�i��yv��/���?�X|���jU�;��M�����-n��\v�`�nr��l����jq��y�{��'X��Xpf�^b�,
���Rkf��w�S��~��4�V}���m^�@@���Cr!A/uH\������[:������z�=/��������O����y��k���d�X�[(2���8�����q���V�H?9�p�S�B�%g�����h*Da��U�K�~H���%���	v(������vy���m�W��4�����,w[��[���[�J�=i��S7_��&�Ln"�������#�qRkJ] ��&r�5qD�����-x����t��Hj��M�����M��=�=1�=���j�;I���l%=m����@���E���{��������[6S�S�9�o�������~����T��]���ImJ��0)ml�Cnx�P�J�{�HR�F�r�M9�L��y��2������&������������aM�_-�\�fc�x�K#�{�	����Z��&-��A�Q}�U��>�c�y'9��>��~�|�)��W0K�2��#	%���<'�$�zf�^\�����t�5�i������b�:W�2��)N���O67nW-4����R��������e�?s�-8tAM����G��4���������4�Ir�f�c��p3N�����K$�q��[G���V�a��p�R���h	Q�����B�P!'&Av�\�����?\�k�{���W�_�5��Q�2����� �6��Y-���Ye��qP�A9�Qo|���%�9J�����>�8��')W�������U�'w��P�'w�t��d(���<�"�%������'C��vP����m}�c���"�F��H���mp�/�4�D����8�7�S4� 2�
��j�]jU�K\���m�I������w�L~�����o�5��������p~{t��F�����_����<�~�����������U[�c���_���ix�F~�k�_�u�)!Zm� ]� ��#���]��zW�O��'K�)���������[�����������������?�>�<������(
parallel-hash-attach-phases.sqlapplication/octet-stream; name=parallel-hash-attach-phases.sqlDownload
#9Robert Haas
robertmhaas@gmail.com
In reply to: Thomas Munro (#8)
Re: Parallel Hash take II

On Thu, Aug 31, 2017 at 8:53 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

Solution 2: Teach tuple queues to spill to disk instead of blocking
when full. I think this behaviour should probably only be activated
while the leader is running the plan rather than draining tuple
queues; the current block-when-full behaviour would still be
appropriate if the leader is simply unable to drain queues fast
enough. Then the deadlock risk would go away.

When I wrote it, I figured that leader_gate.c was cheap and would do
for now, but I have to admit that it's quite confusing and it sucks
that later batches lose a core. I'm now thinking that 2 may be a
better idea. My first thought is that Gather needs a way to advertise
that it's busy while running the plan, shm_mq needs a slightly
different all-or-nothing nowait mode, and TupleQueue needs to write to
a shared tuplestore or other temp file-backed mechanism when
appropriate. Thoughts?

The problem with solution 2 is that it might lead to either (a)
unbounded amounts of stuff getting spooled to files or (b) small spool
files being repeatedly created and deleted depending on how the leader
is spending its time. If you could spill only when the leader is
actually waiting for the worker, that might be OK.

Check out ExecReScanGather(): it shuts down and waits for all workers
to complete, which makes the assumptions in ExecReScanHashJoin() true.
If a node below Gather but above Hash Join could initiate a rescan
then the assumptions would not hold. I am not sure what it would mean
though and we don't generate any such plans today to my knowledge. It
doesn't seem to make sense for the inner side of Nested Loop to be
partial. Have I missed something here?

I bet this could happen, although recent commits have demonstrated
that my knowledge of how PostgreSQL handles rescans is less than
compendious. Suppose there's a Nested Loop below the Gather and above
the Hash Join, implementing a join condition that can't give rise to a
parameterized path, like a.x + b.x = 0.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#10Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Robert Haas (#9)
Re: Parallel Hash take II

On Sat, Sep 2, 2017 at 5:13 AM, Robert Haas <robertmhaas@gmail.com> wrote:

On Thu, Aug 31, 2017 at 8:53 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

Check out ExecReScanGather(): it shuts down and waits for all workers
to complete, which makes the assumptions in ExecReScanHashJoin() true.
If a node below Gather but above Hash Join could initiate a rescan
then the assumptions would not hold. I am not sure what it would mean
though and we don't generate any such plans today to my knowledge. It
doesn't seem to make sense for the inner side of Nested Loop to be
partial. Have I missed something here?

I bet this could happen, although recent commits have demonstrated
that my knowledge of how PostgreSQL handles rescans is less than
compendious. Suppose there's a Nested Loop below the Gather and above
the Hash Join, implementing a join condition that can't give rise to a
parameterized path, like a.x + b.x = 0.

Hmm. I still don't see how that could produce a rescan of a partial
path without an intervening Gather, and I would really like to get to
the bottom of this.

At the risk of mansplaining the code that you wrote and turning out to
be wrong: A Nested Loop can't ever have a partial path on the inner
side. Under certain circumstances it can have a partial path on the
outer side, because its own results are partial, but for each outer
row it needs to do a total (non-partial) scan of the inner side so
that it can reliably find or not find matches. Therefore we'll never
rescan partial paths directly, we'll only ever rescan partial paths
indirectly via a Gatheroid node that will synchronise the rescan of
all children to produce a non-partial result.

There may be more reasons to rescan that I'm not thinking of. But the
whole idea of a rescan seems to make sense only for non-partial paths.
What would it even mean for a worker process to decide to rescan (say)
a Seq Scan without any kind of consensus?

Thought experiment: I suppose we could consider replacing Gather's
clunky shut-down-and-relaunch-workers synchronisation technique with a
new protocol where the Gather node sends a 'rescan!' message to each
worker and then discards their tuples until it receives 'OK, rescan
starts here', and then each parallel-aware node type supplies its own
rescan synchronisation logic as appropriate. For example, Seq Scan
would somehow need to elect one participant to run
heap_parallelscan_reinitialize and others would wait until it has
done. This might not be worth the effort, but thinking about this
problem helped me see that rescan of a partial plan without a Gather
node to coordinate doesn't make any sense.

Am I wrong?

--
Thomas Munro
http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#11Robert Haas
robertmhaas@gmail.com
In reply to: Thomas Munro (#10)
Re: Parallel Hash take II

On Fri, Sep 1, 2017 at 6:32 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Sat, Sep 2, 2017 at 5:13 AM, Robert Haas <robertmhaas@gmail.com> wrote:

On Thu, Aug 31, 2017 at 8:53 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

Check out ExecReScanGather(): it shuts down and waits for all workers
to complete, which makes the assumptions in ExecReScanHashJoin() true.
If a node below Gather but above Hash Join could initiate a rescan
then the assumptions would not hold. I am not sure what it would mean
though and we don't generate any such plans today to my knowledge. It
doesn't seem to make sense for the inner side of Nested Loop to be
partial. Have I missed something here?

I bet this could happen, although recent commits have demonstrated
that my knowledge of how PostgreSQL handles rescans is less than
compendious. Suppose there's a Nested Loop below the Gather and above
the Hash Join, implementing a join condition that can't give rise to a
parameterized path, like a.x + b.x = 0.

Hmm. I still don't see how that could produce a rescan of a partial
path without an intervening Gather, and I would really like to get to
the bottom of this.

I'm thinking about something like this:

Gather
-> Nested Loop
-> Parallel Seq Scan
-> Hash Join
-> Seq Scan
-> Parallel Hash
-> Parallel Seq Scan

The hash join has to be rescanned for every iteration of the nested loop.

Maybe I'm confused.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#12Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Robert Haas (#11)
Re: Parallel Hash take II

On Sat, Sep 2, 2017 at 10:45 AM, Robert Haas <robertmhaas@gmail.com> wrote:

On Fri, Sep 1, 2017 at 6:32 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Sat, Sep 2, 2017 at 5:13 AM, Robert Haas <robertmhaas@gmail.com> wrote:

On Thu, Aug 31, 2017 at 8:53 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

Check out ExecReScanGather(): it shuts down and waits for all workers
to complete, which makes the assumptions in ExecReScanHashJoin() true.
If a node below Gather but above Hash Join could initiate a rescan
then the assumptions would not hold. I am not sure what it would mean
though and we don't generate any such plans today to my knowledge. It
doesn't seem to make sense for the inner side of Nested Loop to be
partial. Have I missed something here?

I bet this could happen, although recent commits have demonstrated
that my knowledge of how PostgreSQL handles rescans is less than
compendious. Suppose there's a Nested Loop below the Gather and above
the Hash Join, implementing a join condition that can't give rise to a
parameterized path, like a.x + b.x = 0.

Hmm. I still don't see how that could produce a rescan of a partial
path without an intervening Gather, and I would really like to get to
the bottom of this.

I'm thinking about something like this:

Gather
-> Nested Loop
-> Parallel Seq Scan
-> Hash Join
-> Seq Scan
-> Parallel Hash
-> Parallel Seq Scan

The hash join has to be rescanned for every iteration of the nested loop.

I think you mean:

Gather
-> Nested Loop
-> Parallel Seq Scan
-> Parallel Hash Join
-> Parallel Seq Scan
-> Parallel Hash
-> Parallel Seq Scan

... but we can't make plans like that and they would produce nonsense
output. The Nested Loop's inner plan is partial, but
consider_parallel_nestloop only makes plans with parallel-safe but
non-partial ("complete") inner paths.

/*
* consider_parallel_nestloop
* Try to build partial paths for a joinrel by joining a
partial path for the
* outer relation to a complete path for the inner relation.
*

--
Thomas Munro
http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#13Robert Haas
robertmhaas@gmail.com
In reply to: Thomas Munro (#12)
Re: Parallel Hash take II

On Fri, Sep 1, 2017 at 7:42 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

I'm thinking about something like this:

Gather
-> Nested Loop
-> Parallel Seq Scan
-> Hash Join
-> Seq Scan
-> Parallel Hash
-> Parallel Seq Scan

The hash join has to be rescanned for every iteration of the nested loop.

I think you mean:

Gather
-> Nested Loop
-> Parallel Seq Scan
-> Parallel Hash Join
-> Parallel Seq Scan
-> Parallel Hash
-> Parallel Seq Scan

I don't, though, because that's nonsense. Maybe what I wrote is also
nonsense, but it is at least different nonsense.

Let's try it again with some table names:

Gather
-> Nested Loop
-> Parallel Seq Scan on a
-> (Parallel?) Hash Join
-> Seq Scan on b (NOT A PARALLEL SEQ SCAN)
-> Parallel Hash
-> Parallel Seq Scan on c

I argue that this is a potentially valid plan. b, of course, has to
be scanned in its entirety by every worker every time through, which
is why it's not a Parallel Seq Scan, but that requirement does not
apply to c. If we take all the rows in c and stick them into a
DSM-based hash table, we can reuse them every time the hash join is
rescanned and, AFAICS, that should work just fine, and it's probably a
win over letting each worker build a separate copy of the hash table
on c, too.

Of course, there's the "small" problem that I have no idea what to do
if the b-c join is (or becomes) multi-batch. When I was thinking
about this before, I was imagining that this case might Just Work with
your patch provided that you could generate a plan shaped like this,
but now I see that that's not actually true, because of multiple
batches.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#14Prabhat Sahu
prabhat.sahu@enterprisedb.com
In reply to: Thomas Munro (#8)
Re: Parallel Hash take II

On Thu, Aug 31, 2017 at 6:23 PM, Thomas Munro <thomas.munro@enterprisedb.com

wrote:

Here's a new rebased and debugged patch set.

Hi Thomas,

I have applied the recent patch (v19) and started testing on this feature
and i got a crash with below testcase.

with default setting on "postgres.conf" file

create table tab1 (a int, b text);
create table tab2 (a int, b text);
insert into tab1 (select x, x||'_b' from generate_series(1,200000) x);
insert into tab2 (select x%20000, x%20000||'_b' from
generate_series(1,200000) x);
ANALYZE;
select * from tab1 t1, tab2 t2, tab1 t3 where t1.a = t2.a and t2.b = t3.b
order by 1;

WARNING: terminating connection because of crash of another server process
DETAIL: The postmaster has commanded this server process to roll back the
current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and
repeat your command.
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!>

Kindly check, if you can reproduce this at your end.

*Thanks & Regards,*

*Prabhat Kumar Sahu*
Mob: 7758988455
Skype ID: prabhat.sahu1984

www.enterprisedb.co <http://www.enterprisedb.com/&gt;m
<http://www.enterprisedb.com/&gt;

#15Prabhat Sahu
prabhat.sahu@enterprisedb.com
In reply to: Prabhat Sahu (#14)
Re: Parallel Hash take II

Hi Thomas,

Setting with lower "shared_buffers" and "work_mem" as below, query getting
crash but able to see explain plan.

shared_buffers = 1MB
work_mem = 1MB
max_parallel_workers_per_gather = 4
max_parallel_workers = 8
enable_mergejoin = off
enable_nestloop = off
enable_hashjoin = on
force_parallel_mode = on
seq_page_cost = 0.1
random_page_cost = 0.1
effective_cache_size = 128MB
parallel_tuple_cost = 0
parallel_setup_cost = 0
parallel_synchronization_cost = 0

CREATE TABLE t1 (a int, b text);
INSERT INTO t1 (SELECT x%20000, x%20000||'_b' FROM
generate_series(1,200000) x);
ANALYZE;

postgres=# explain select * from t1, t1 t2 where t1.a = t2.a;
QUERY PLAN

------------------------------------------------------------
-----------------------------
Gather (cost=2852.86..16362.74 rows=2069147 width=22)
Workers Planned: 1
-> Parallel Hash Join (cost=2852.86..16362.74 rows=1217145 width=22)
Hash Cond: (t1.a = t2.a)
-> Parallel Seq Scan on t1 (cost=0.00..1284.57 rows=117647
width=11)
-> Parallel Hash (cost=1284.57..1284.57 rows=117647 width=11)
-> Parallel Seq Scan on t1 t2 (cost=0.00..1284.57
rows=117647 width=11)
(7 rows)

postgres=# select * from t1, t1 t2 where t1.a = t2.a;
WARNING: terminating connection because of crash of another server process
DETAIL: The postmaster has commanded this server process to roll back the
current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and
repeat your command.
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!>

-- After assigning more "shared_buffers(10MB)" and "work_mem(10MB)" query
execute successfully.

Kindly check, if you can reproduce this at your end.

*Thanks & Regards,*

*Prabhat Kumar Sahu*
Mob: 7758988455
Skype ID: prabhat.sahu1984

www.enterprisedb.co <http://www.enterprisedb.com/&gt;m
<http://www.enterprisedb.com/&gt;

On Wed, Sep 13, 2017 at 12:34 PM, Prabhat Sahu <
prabhat.sahu@enterprisedb.com> wrote:

Show quoted text

On Thu, Aug 31, 2017 at 6:23 PM, Thomas Munro <
thomas.munro@enterprisedb.com> wrote:

Here's a new rebased and debugged patch set.

Hi Thomas,

I have applied the recent patch (v19) and started testing on this feature
and i got a crash with below testcase.

with default setting on "postgres.conf" file

create table tab1 (a int, b text);
create table tab2 (a int, b text);
insert into tab1 (select x, x||'_b' from generate_series(1,200000) x);
insert into tab2 (select x%20000, x%20000||'_b' from
generate_series(1,200000) x);
ANALYZE;
select * from tab1 t1, tab2 t2, tab1 t3 where t1.a = t2.a and t2.b = t3.b
order by 1;

WARNING: terminating connection because of crash of another server process
DETAIL: The postmaster has commanded this server process to roll back the
current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and
repeat your command.
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!>

Kindly check, if you can reproduce this at your end.

*Thanks & Regards,*

*Prabhat Kumar Sahu*
Mob: 7758988455
Skype ID: prabhat.sahu1984

www.enterprisedb.co <http://www.enterprisedb.com/&gt;m
<http://www.enterprisedb.com/&gt;

#16Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Prabhat Sahu (#15)
Re: Parallel Hash take II

On Thu, Sep 14, 2017 at 12:51 AM, Prabhat Sahu
<prabhat.sahu@enterprisedb.com> wrote:

Setting with lower "shared_buffers" and "work_mem" as below, query getting crash but able to see explain plan.

Thanks Prabhat. A small thinko in the batch reset code means that it
sometimes thinks the shared skew hash table is present and tries to
probe it after batch 1. I have a fix for that and I will post a new
patch set just as soon as I have a good regression test figured out.

--
Thomas Munro
http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#17Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Thomas Munro (#16)
1 attachment(s)
Re: Parallel Hash take II

On Thu, Sep 14, 2017 at 11:57 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Thu, Sep 14, 2017 at 12:51 AM, Prabhat Sahu
<prabhat.sahu@enterprisedb.com> wrote:

Setting with lower "shared_buffers" and "work_mem" as below, query getting crash but able to see explain plan.

Thanks Prabhat. A small thinko in the batch reset code means that it
sometimes thinks the shared skew hash table is present and tries to
probe it after batch 1. I have a fix for that and I will post a new
patch set just as soon as I have a good regression test figured out.

Fixed in the attached version, by adding a missing
"hashtable->shared->num_skew_buckets = 0;" to ExecHashFreeSkewTable().
I did some incidental tidying of the regression tests, but didn't
manage to find a version of your example small enough to put in a
regression tests. I also discovered some other things:

1. Multi-batch Parallel Hash Join could occasionally produce a
resowner warning about a leaked temporary File associated with
SharedTupleStore objects. Fixed by making sure we call routines that
close all files handles in ExecHashTableDetach().

2. Since last time I tested, a lot fewer TPCH queries choose a
Parallel Hash plan. Not sure why yet. Possibly because Gather Merge
and other things got better. Will investigate.

3. Gather Merge and Parallel Hash Join may have a deadlock problem.
Since Gather Merge needs to block waiting for tuples, but workers wait
for all participants (including the leader) to reach barriers. TPCH
Q18 (with a certain set of indexes and settings, YMMV) has Gather
Merge over Sort over Parallel Hash Join, and although it usually runs
successfully I have observed one deadlock. Ouch. This seems to be a
more fundamental problem than the blocked TupleQueue scenario. Not
sure what to do about that.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

parallel-hash-v20.patchset.tgzapplication/x-gzip; name=parallel-hash-v20.patchset.tgzDownload
�h��Y�}y_����>E��M,�$�/f�
��f����������.A�%���&3���,U���x�d���u��Z�~N�Z8�3��i���.kW�F}���e$�����~�F��������V?��nv{����f������l#X���b'�����0X��M&+��y���!?����i4k��W������"�Q��Z,�8���N\�d�����?���"y�u:e��iu������F�����X�����y�D��H�n��p�u�
9t������f���{����wz-�*��3���h4��h��`���`�D�.��>�������X��P���`�d�������������p[�����nGl7���r��t����o��^o�i6���R$h)-���h)-��pO�77��0���e�$N(�3
��������9���R����V8z�ftmfN��@:7�J��@L�e(<2�!�W�@q-���a��{)`y��bL��f�������"Z�\������������0�0����������=�y�Y��6E�;8��";���N�,c��h�Zb{��x�/S��&�f�m����S��{��/�W��@�R�0���*�[���N�V��c���1���6��'?�����3��z����iwZC��&q"\����	�Y�����u��F�-��w�+���K@@GQ��)�M1A����)*����hkSl
/"v�S	�v7�a[���#B��4VsC��b��G����(�E�����]nf�����
�X=���r�5���-�H^S�L\��%`�t
�
H���H&��
����q�!Lmz�#�b������. ,��h'��B-���K��[�YE��hl��3�=tn�yp�1����q�L	K����������_�mj��_��� )_��K@����Y�Gx�(��J���A�k�W,�p�xIL#&�@����dWj�_\�p,m>���4�G����{�[�[i�e\9�r9�9��`N�i$n�
�l/<�L�h��7�����Ds��N<������<f��	6������!�u�o�/B-�YD[$�`��{�����P�D��57������R$r/�,��
5	N�z���8�2h`T�����$1����!���C�h�7#4��G�������A�\���mk!n�#i�������r���]��������b���5�pF"��M|@1M����.��b
�
�&!���@s�=�#��|9��JX�lV �2r.�����?'0��ZO��.dL���D�[����a�������A<�D��(�y�1
�����TPN�M&[�2��rWm��i���Y �Rc��pE��ooO�.����<����0�/@3��H�_{"�K��\+��;��cQ	�>"~���&��L�"�Pe
��L���JI_ �JOi9����]8���|C,�7<A���;�:�1p������e��O�bG����!���%�;�}`6(}�H��V��������K��V1��q�;��,�!����fA���~�
�l�M��~E�&�{�f��E�(�/K?D#�����C��RsU��L��4����c��Xr���87n�����o���-��
M��.�8r~(�1y���-C�"?Hw���s	�?0T���y�T����.���E#�������\�@qp�8tm��3Iv;i��Z�F�M@n��g��krol�)�����U�Y?��.q(��gG�_|���T�|�x�Vs��.�M�Lz
���}�}����IN������Z(r����h!�X<�B����g������RNU��1���}2(b��T�RGX�(^Gi)D�e0�7p����2��������
rmNi����4�& l���!��p%2�)7~�^��y���d5����������8�u��h"�>�^������P�$�
i�n"�s��~bW���8	5sz�rNq+�jb��9�39�����F�]x����c+NE��$%�t����0�>�����0�U���1��F�qD�c��'��Y��)b����XS��7��aC�?�����=�Y�S�+�w�!j�c�3d���� ��P*��
?J��l�:<��
�~�he���-Y-
D��&��u�U���QAv``5������Y�����DK������^���6]�^oKg���M�u[ng
RIZg�
ft6��ft��fw�NF'�P�V�����,eS|/��E�w���LY���<�k�-��[���������-��?7��?#��?!������?m���o����.N�N'�����O���L`���{�<���[���U���������}��,�����=�����G���?���-�=�e�3�)q�fp,�Z���j��o���v��������i?����j����^���^��a�����}����������7z�L�,��������?�����w�����`���N��kwd�q����h���/P���rYEp��Os��=|�i���iQ��g��%��AK4
Znn�
<Bnn����txM�TdQ�p��U�G�7�TNz���<�a����������Gs��G����\Wr�&��E��I��hF�����!�!�X�&q.��CN�h�80/�_`E�MYmv�g���#[��}��QP�^���g/G/��������dK�0N�G��a<�K*��{��\J����G2|t����'�C�I��a�H���T����U���Il�@�U5f�X�%�~0��$`|�RE�"���}�� �*����/�y��sG�{�D����STI�G��8$y�,�F���) ��r	��3?r�Tv�����E�xg�~|�e�3�9�?���^��=������E��������oec��r�z����Y���������_f�?�_��@ �������:o{�N���.&g/_�����G��z��Ofsy��������W�t��1 ��{;l�����&���M�+�F�3=������/�h��ZA�(`��QU�j
�����Q�F>���J)��
o�q]����[���x��������$�jEi�v���{�S�j9��Ch��*����������'��KH�M��|��7����=Q��H�l���{�<!�����,����	���Gb��I��P/Bo���-��J(A���wo���^���������x���Q%��G+�&4qL:GC�T��;�Dd�~#��[�%�j| �r����=-�������q1�hZ3����6N�ww��d:�iIK~�����54(jU�G�3�)G0g��v�x���
��r���;P������*�A��G�d�� �oL�{)�AU��b������E�h����m[Dx�]��{��
�w]�Y��Y�@<}|� <��8���/�����1V��A@��e�<�B!����E�wdf�����R��<� X����E[�i�����4{������ ����3X��,qT��v=������_�])�s�j4��Ln��A��=�:�74��?0�C�Q�H$���o4�A	�s�A�q��Y�z��Z�W�a����6��5��EdX�uRR$:����SH�2����/�9L��>.p��f�|�<�Fu�=�����o�G'�sx��p��@T�x��n��������+�\���%)<[�vz��7��`"�`2�ob��+�����M����$��������M7^�������<=<{yr���_���tG)��);�QP)2DW��k�~.��)i�D���rI�f��^G�vW}��$���`�N�k��>C�W��lt�+m��J+3�(�"�xXF���MB�X[�a!���^z����G���,VE=�.�d3eR-|v�����E�;��E^SSF�Ar�EQb������N<��lV�b��RZF�&�*v��	p� ��A_q����+g���T�?[|�&M�����O����'�jl��4I��^��3�&�?�������O\!� �l����<�?A^p��k��e��8�D�Y8�s0TW�x)�%?�$AE�fL=������a*���F�\�+��}i�D���*0Z�pI�j�U�&J�[�&B$_T����/��	�/���	���NF�41m�Xr�����D	H5x��S�f�NY�\�G�EJ�$li�S��;���~�&S�
�T�w����EkT�3���N(2�M����~u�S/��+����?���F�f�����$4��K�P����8@�f�T������
��}�;Z+^�����U������:����@|z�b��*a��B�������)s�6�d=Y���eI��iU�����|C�B�G���A�O�K�i8s�|��G��0��@��P������`5U)����A���<��Y��W$��y|��{y�<�oijn�
������N���@�#Z�R:g��������S|����n����i�
?\$�}5��L��q�Nd��U�q{�D��4��}����F(�{(��{N�(�P�>l����7��V�~����:��^��
u�c�	g�\�F��W�S2�w�-�$���),��������!��N���V�a|KQ� [i��	��|���*E(��������h��R���^�B���^Yw-$���`��$��759[�7�F$f�~d�s��r�ok��z���&�������N��4&�^�1�
V��]���\�xi��E|��������( ��5��G�/	��:��\Tv��n�hj6Z�c���v�oT� ����G�F�~����`n����i��R
���EKw�g��|�����c���r���h	���i��3M����M�}������Du@m��H�|�������`E��76T����f�t�!��1�1V��� ���b��,,JU�X.`���g~V��uC��7S����8������@�SZ"��H��@��aT��0��~2?��?���K��/L�k����^&�R#]�n�?���V6����4��|��������w������v�qk�h��^������������m?����?���AhY���2Ght�f:��^>�'c��3���K�#�N������
�WI�,�*���O��,�L0���)7u�B�u��Le>��\�.3s�I"
;X(���00����8
#����D�]:?�["��!��U���X�'CX�7�z��P��Bz���P�wz�L�N����TzCg�� �K�E<�|�������q��G��yMgL��,�{
z��|��x��K�Rd��P����,Y���c���G���C��VV-4[��{�F���u����e�9q	���Ct�m���O�M�������1N�P��<�3�Qv�����{fA&D���0����
�?Y+>;�s��re��dyD�D��-���C��q�	��M^n���'�C�����W���,j��9��Bx�#*�����[kkk+1�9��7�R������"��F���<�R�+�D�����J��c�*�����D��}`\���\��A�������Wrg��y������1��lEa�o;��~!c&m�D�f���R��W�#��xS��a������4|���$;A�5P07�W��9|��~���n0v�O&vj�)�r��-��.�cr7�j����}s��Y�cn&�����Urwb]	�����Zg�P��;�<a�;�={�[���V��O������'�iU�`��8�Nq�~��I!hE�����
^y��A��W;�����_����+��N9)+1S�O	�/���}�3>-p�#��l������=G7�������!w��,M��������6r��+������q�������'�#������o��**�o����q�m��eMU)�[s�ut�8�y������^�@=���z�V�>�e���@��J�b���,����(�ZN����l�����N@G�����"��
4�41m[@��L��>�g��i(���8\��:��z���o��)\@�^����`�~�"�h(���b�_A����@|�b1����4�s�`b%��N�I�j��@L��)B�����K���%hVRC��M�K�5����"��	0*���	����U`�����G���d���JI���������C�aj��5Vu�j�jj�`��L���������sCTK��@�6[i�}#�����5N�Gj*}d�����������0<K��n�M��n��N�����@M��d��{���ra��b\%�Z3�����$��
�)�I��s&L�2�J��	�w��[K=�j�w2�	��8(���m[�(��8*V� 4�vT�(
�\������t�.�+�����*�=��8��������ro�|C����H�,�f�����m�s��T�����gt�;��Gv�l�`w�u��C4�
������#I���]�l|4*�VD�����J���(�k�\���AbQ��]nw����oY��F<h8��!AD�P�WY���
%:�&�|���rr_���(/?`�+��X�Kw��-��C���-f�or;����J4u������|!�}��?��"�'EBX���F`�$�������8�6kw��)��mn�_��oS�Z�DK�Mj��R(|��u�U�S����5���q�%,I,�c����������������b�~Q�|��"oAT�K�������,���0�1�O��iL1�>T�{�������&X��$Lb����sBH����K�J��+ss����?YE�VhY�d.�O����9x�IaW;�u�b3�o���pK���^���hNti49��9P��=R-���%
�|e��H\��d�m|B�5���-�SF������f�l��l2i��@Z��-&����:O����p��-L>a��������[y�_����Vg�k�{Ox�������}g�����t�Y��R�>���u�����@����s�i�&�r~�1B���J�]�y�p'����Y�*�<�V�
d���q+w��~j'�4�&����V.��c"Q��rT���Cwb��6.�o?%?ou�W���c�&AX[��K���#,��X�ax\Z���-���n�5����t[������5m��m&�����as�Mw�5{���ve����^�?�Z�_���fY�W������q2h��r}4�I�q���Eq�I5s�)`I[����7�E�7���U?�.*�T�0�[�US���
��&�/�/�����	��q�7��
W�
�����������R8b�������73�]����^j����\���\����j�t^AQ�A+���l~��/+��^��i������+j�l���6�j���P.L(�y��Ted��us�3�UH�yd�(�Q��+	�13��ki�A��L|9��� J�|L������e����c���b9uB�u�4�-w0�}A����G|�/K��V������+�4�|]��xo�?��~��N|sw��{����M��f����&\q0<�8C��(i��E[�����L�c��� �/��EV��=�
3����.�pV��rC�;�'�
6A��+�����Iu�����=�����;���5�|��"�^�:@����=�-tn��,��ac*�K<�.���VR����1��K��hny������Q}����Fq�+0DQ!^S����6����h�AU�\-���.4P�h-�nQ�����<Lm�2���w���b���]Q��pi�iN�Aga�����A�)[���Rt�
��s����]h=��q�|2b�Xv�}���2G�r
��&prX������t�q�Hg�S���|���S��~����Lm�9V^4���N��NQJ�tI��w�th�&p~�*P�V����������� �Q��#�)��K�,h\�	��e&�-^^�
jF��l�t��b�.u�������B#"���
5G�L�-���Y�<{��,��6TqWa��E_��5C�=���xD�tz<E������=�����kN�����s:�e�E�,qP1�@�{*g���ir{k��8e� ��P�(E|&�ZVp?��2�6��T�d�{c���� ���4�����������N�� ,�,%0�J�f���/�s�_I�~�;���$����g���N���f��i���\$����
Ns��%:S����8��WS��~A&�_�'Nh(y�I�5�T6����d�(���9	���6��fGk��'�Un�Z�Un7��V���O0��Y��X���s�*��W�,�a_S�)�E�;y��I&���`2X���3"Wz#��#=��'�*�bL�������D�+Qj*�i�$����(�����������dN����+�%����N��	S�Ge�bs�����B�X����Y�49c��-�H3����U�Me#+�*n$��sf�^�����T-�����jf����fW���b!HX�/R����P��V�?qz=gRph�3�~P�5��:��1��2'39�'9hjJSFT^0S��T3�bN^H���G�%��p>��8�yf�A(�q���+��sC�Es�ZR�������[*�wA�������/���Va����:����{^����0�eX[�X�����n��%����"� ,�bu���i�����o������N�;���N���]�O�����I���w������|��O����y�������B�����%�A-�A���7*��y���;3�T����"b_R�b�d���6�U�	���~��&�����P&���C���*v<�0�@�������?�����;<_����Qu��@�d��.pq���a#=7������W��+�Q2��S:�
��o�;Va�=x�v�ht��F����������_����w�����zr���OG�8�s��\,G%���w����}-�R���B��t��uB?���-c
��Y�\,]E?�n@Z�Vk������mW]��n��*74�E+Pw�%trA%SZR*\���MtA.��m[��:���{n���+J	�G�
���2���y�h9�|L*)��	�������v����oSg��[�[m�0�+��`�L1����g��f#�n��G������kJ��@,f�1w�r�u�>�����!�T�b�6Et���0�k���8���>�B�pE���xq�",so��J�����
4m6�KN6C~���Q�����?�������&�����|9���*��WU<=RE��_6��(��J��z�t�L����)�7��c�������&`���|�h�����WCAX��L�����BK�L^�i��+$��0X.n������#-V��AJ����9p�>|�;fX,��~���J�W�������$�'��7����9���K0�@�=-������D��g�s8B��3�����<�j���(���^�#����glN���x�^�KA$���I�%u�A���3Uf��As�cs(��%[nJ�'���$fx�K��0��j)�%0k58�$��Zk���g�G4;B)i��7�j)x��.p����5	�Ew�S�D��8.������\���H�+sLk���t3���S����'G>r(A��*l���(H�A�%��3&,����@%������g
U��]=x9vBn@ >%��1��-�+�>�����~U<%�8P��ph0���X!���9
�����q��s��z�'T�x�����U�N�0�uR�]�!��
����F�~�o@��2��e?�����9�������.��-�x	I�����8��.�*�U��G�x
��)`�Y}ja�^d0�e����$��(����dwS�IBQ��3����0�X�����y�3�����2}��)��T�W� ��Kr)^�p:MDT@@�<�����&�N�0�q�����OZ��h��F0�}>�F{���)�����]M��@"1�)+�Fc)����[�F�1�_Z{_���^E���Fw���{��LT�������OY�3��"F�f^kh[;L���E�Q�/	1�E��:��Z�~D�����s4�S�W�����(s Rg�1��Dy"u�1��A����H��Y����W���9����Yj�W<
I����69�aJ��D��c�C�����0a��,�k�x�e)�����IXH	C�����L�^F?Nj
L@	E~M�����	U�1�'1�r�L�F�9�J1!p������u�'�w^aP�/a%N���H�]f0�����C����������2$o�-0:�Bc�d��T��f��f��*b���tt����D)��/1���=�9���L��2���L����������)��Z�wh��4���j��8�zg�:l��5�H��y�v8�l�-�m��)��b��O{P��XONPR�����,Xajt��M�^H5P>�)�J"���F����g�[mQT]V�g�UG�I�U�}]���v�����������ke��nH����t�X�i����vz� � �6R�'�z�zztx:zv������;Z��^��.��2_Z'����)}������i�������c�Ho#3��&{���]��^1iq�B�#`��VKP#��G5e�	�`&����&���pxJc�1w
0���><����X*>	�R��Ui5����H����@��bu?�#�r�R9������?���h1���H���7���+<�l��d����|I_���>[����}O�4�=�-c�Yf�J-�^
V���J�kI�+�������S
>���������������+�G��B�Sn����dM�F)�A��I0�r�rN����+2
��mU*K�6�B	����#e+ �����!r��JzwK���K���<�8z�|�4��SU��Rf	&E��T��� ��J#a�A�\/�]+U
�����w�T.gX
/# �H��df�I��Px�p��!k�����T�1�� m)R+�q#/P���N�?�~����\@+���4%.�������G?��m�"c�E���s�pm.��Q� �KM���e�l�W���j%6���}ds�<t\�:��tQh������d;�m��%[V�������i
/�Nj=(���r��V��/K?��k{��e�#���<i�\H���3I�k�lI�H��f�~T��'|h����U�qm2(P�����wG�#�O����)@;��i�$1��976(�uO�Q�����2�*�b��R�*0I��$�Vb�
��{1����2�T��x��V��8��^����;~�r�uB<�]N�^U.Y�T�p��4;_X��S����u�TM�Lf�
�� ?�>��b#���C����F���I0��;�hgp�n�Ii9�U�X\������2vk�R������%�}EXo��O�=Q���(HCU���@�!R.��z �����4o����r|�B�B�����w�3��%������T�*��i��l�k���S[��v�(A���������'��%US����������3[l�t�P�����X}"��\���W"3H��	i�\n��X������.�0o^�~�?:�;?|}6VRRl�q��Q[0'�c�����{��.��~��|��'���$�O�+p�Vz=�MN�kJwd����q����������X����a~G����Z�kt7r��3�,��8�v
:
(~Q��(�A���+��w��u})n�#YO���MKtqNk�[]/i�Jv�5�Z3,�&J���
Dw|"�Q^�ZU���J�J���X��w�%g6�����c�<���}m�aYc����8�c����d��v�&(.�C-�S�,��a�x.��Vs0!O�x6u��T;��	�[rE�,���0���[q�W�d�e�Qr�b\t��2)W7X5c}���2UZ��N,��UGns�4t�&H�(�������XT�}G��KbZ]�U4!����b�6P�V�?�W�Fsg��	��v�S��!�i��k!�S&�vv-��"��!���m�^��2�1�uX[��R���M�;\�������D|1(@���|�i�X�p8[�H,�sT�&�E_NUn�Xf<�Q������xP�\|@��'�#g����/IH����1�������b\�A���jhs+�����A�?�
*��D���@�����]������9�L^ab���1>����%���*
��$�h?Ud�������Y�*�}���fIaC>R�����uM�X�j&�@��K3*���'rBO���0�IJw�zM����k���*��x/�4�db�U�MD_�c%��V����>PW���v�PaX���'��q���U[D�U�u�S���_�	O���WO�W��6{����r��:�Z�"�RG��?�I/��4�����-�Y`���p��Ml��^*�l�r��(�g�HxWz��wa��pH��7��V�d?���) �iO<���C����"��lQYT	�8��M P������O�'����|@#����dtn�	%��(�R�/�)��jc��p&�[I��d�8��x��M��I��AE]���:�_�D}tl���������i�0U��y*Tm_�@o�i��`�+��9�5�$���=��E���B�i	�P������oo�7�Pm	{upY/���������uU��h	�P}�t]KOm>�"g�C���5��r���Z��N)�
�9=wnRL����iU�Ay�w_rYB�J���A����I�jP�4�@�y&\���8�X1����&�WN�^�%�@)U��1�Y���n�D�\��x;V$>&}��}:����\�
�Z�"�A�\��"��*AI�IL��jk��*����s�h7��IW������{�����Q��{(���b����`q�u���u���M����V��������W�h��N.��6�/|<������:�
��f������VA�����0��lFeq�!��*��^j�B�af~�
�TD0I�T�pkZ�~����=�o��OT�7}���|��)b�)
�X0Ij$��$�|
�
����?�}
�����������E����em�rY��L�E�S����*j<��z�������8]4QMP����/�R�OZH"bsUA�����"~�sF�6�LM�Y�5�L^KNK�$��Ng�t6-��t��q��(7,k��<8�J��u�zRD���Z����PUN5�����e,m����t��e5�D�"B�+�j�{��R0Rk�t���LG�3H��Qt&A��4��,/�s�~�9�~@m	�)
�d�{��?H��l7�C�]��a\c
J*:�G�����u�b�����f����������!id�q�t�]{h+�^�CPV"tE����s)[�5����w2�{�H�c����Q�����D��NC�1�
�u9n���w{����9@��f�&t�qo��"�����A�,�X�#v�x���L�8��=B����Q�7R`FJ��;�Sf<�K����3E�����"L�3tK]9���|�8�F��c��F�9�nuV�1\r��-i���av�}�OP�
Vs�
�S�FN����
���X�$�dA���� �$Fgo������sx|������3	�����F� XT�Icy����g�����Q����
�4�������������l����C�D����K�p��^�	�jo/�4��g����
����2���&}p|t��|tp������`���k��g}&u6A{H'�C�Es��:����K���x���O��'�x��L��KO����:	�m|����*�Xi������eI��m�	_q����1/�+�N]���,#���h���V���i�Xa����F������`����a���)�������*@f��k��(�;/X3���a���D���E�p�M��#LI��O*�]4J�R�HXA����������%���qKW�AE[��#m���I�*	��{(nd��%*��/��|=�r�z3���,��Q�4��)A��[�lz��G��0S���Tg��mg��B�%#���e���K���mWAW�A���1���:�@�@_�[E5�XY��.�1.{b
������zN���A��w&��ly�e?E���uP��_�T������t���
QR*SMV)����GJ�L��D�\� Q3�)]�[������W��j&�@z(ijR+�w5�K��"	�}����x�K������jv���}�����7���v��x������y�a�;[w������������[���_=���Kyi�-E-I�H�����,������\R��:��������c�v
x��[�i�0��5f�� >��S��j�_�}�\e�����6�	�&�Bg~��P�y���r�������G�O�R���t������-��#�H�1�zA%0x�1�QY�T�:5��&�1����7��X}�3��$	����3���D�QT���<�w~���oc�v!J���pJ�2%>k�O����-cf9��*f��W;W��j�k0����bf�^s�bfE��lP�����H:�������}������);�t���4�a��N&���9tr-����cE��l�������w+F��y���7���_���{�vT����
���K?�����q�B�V�B�0�v���
��]d?�g*�����4d�O���!�G�2c;Gtq�g��r~���}�U8�&r!���D��0T
C�%v��EI#���d��Q���fk��&�����[gpk����Y�)��/��J���S�E�2��M��b���w��:�[�}XT��L�P2�ZU���l���u�dK�K�#yC	��,���t&����*�J�92���Ig���.�� �i�I�Of ��$#xt���K�S���dx!Y_���K*D��u�S��g��	�vhT�Lu#�v\bf�Smt�-��qLu���$>O^����27*������|Y��A��A��vHrIK:���e�:E��Q=��I�.NCv�n�
�����N��F�bKQ�d���\�����!�z���P�P�H%3Z��:bkzj�>(�il	��z������H���}�����%���i0�@�������Rqt����<��S��.�f,��^s�~l��������6yv8�3�rl���P	dL�gf�^`m)Kg��n}�K��\�N� �8."r�5����:G��.�@y��0�!O<=O�����Sk�L�@^2��q��I�s��a�c�lXE�V��h�&H���ub��S�\��V-�T7r�ry)u[V���
W�O�}�Q���J�-2E%��UK*]����y����@C����6���H����@
#���q��5�g~���Q������['���
�J�;��)���j1���(����8�r�������m���h����\�u*I���-`����_|db
V6�V�.o@����������>[Q����>V���W�W�VU���I.T�>��Q��?�3�;#������	1�����C�1�<���@�h��sbU�1K��������zG)A�m�"�[����dw�J�����
'�����\�_�4�?�?�������(.3���[��T>�9����$�'��0#�2�	0�������gb	F��\����Y����bX������{����r����
��~�����:�[m��������Y������g@�����O1����������t��d�1��+�����B��iiHAZ��k0��J'����
C�����~<9�����_vx�b"I/W+V8F����������x������C3�2;�T�!�G�D�X��Ff�+3�E���p2�Z�U8t��r���P�4e���y�X|v9wo���t�l9�ZZ����s��:��z#�_wLa�U�o�J��9�`���#]��b5�V�f\�gC�U�#u�����Z���Z�VF4�����>�w��R�VG�S�
%&�����M5u��?2�w�t�H��O������� �&��rv�����������9��&-�N�
��Q$��(������}��
��|w�:��&�n�0��������k2M�k
�$�����|�U��`}��������-�^�sk:[�{,�a��U��:����5�m��pn�B����v~[���e�%�_IR7X�L���#E}��i!j����"K���,�@�eq��h
:�_u���[�����dD��>�f��R�����nu�G��e�����E�i�5�����w�'�����n��~������m���?��������<�sa�����o�7p��F���e��U�?Z�����FY�G?����R(�D��-7���W��z0�l��ed�VX^Y�4��2ua���
ow�c	h������������oh�%X�(j�9�}6v,�=[m���a��^�B�7�@���}��K���^i5:7:���B����@�DvSk����0��=u}[��M�z��	KT�B�x��3g����=��|!(Wbe�`/����R,�B�tE��A(���z����P�DO�\k��+y�����/v�u{�[�A�:"��5��w[�t'���n����%vQvi���.�^]��%�M�kwI���:�n+����&sk���5�<4���{��l�v��E�����t�N+}a��w��+'0�Zc�����������gq��Dkl��k	���^79%���5s��Py3u�gd?�?)��M��������f���O�*�U�g�P=8�l7,�!�Ji��@�X���I_(�?[����P,o-�]��]�I6*��{��V}?����W����v@����r�kV�89��G@%g��������W{��:�qD)�[x���E����b���5��{��'�1�\$���[|�������W���`��k������WC)l�%k����R��m������#?��2c��c��A��<�O{"?��#Q���e�
���/��A�+V��'���QW�����iN�Ve��,{��h�G22��4��	��o��s�Qjf1���PFq(������,��Q2�����(vq�r���Cc������7�w|����m�>�ya���}����{[����@�.�eQ�~�E���Op�������8�Bb��`#bq?vR�GlLty��h���
|�b�t����x�������=?9=?zw�,�;�k�R�v�����&<�3U������	���������t(����\f��_2���#��W���R�)� -sg���%_��\�~u��� ���xt����<�q��{<m�h�s��)~]�}�=�d�_�XE��w�n1��%�{Q[e=L�����q�;����f�#����\�z(����(l����T�-�V�`�����L6����b������`[�2T�E������ #����aW�&@e
����8S�7t��]b��,�=8������X���#�z�{��e`h�@w�&6���i&������{�Tb�H��UZ���F���n�M\�k��u��{�3��n�Y�w'�~�����&1h��iG'��
d��k�	'������S5�$���Db6!�:f	��3�;�?�gP��W2�<F3�aO!��Q�r+��j.��&,tm]������4�o$Wq1�nXwsSj1%AO5����l��Tx'y+��^l�v_���mp�a�v�q�i�.�4�j��������d�����;qz�����yM�S��.ip����e�f);o��&�/!&W��e�+���Xlkp����i�gr�����gM��W�5�{d
�^�V�s��3��L;O����(���>����V�i��b�v����AS��N�9�\w�]W����Ud�#����j�)��wK�h���eE��/�B�����?�T�#��F-Zib���+u�>��\�S�e?���������������+6tQW���)We�"��X?F���n
����)����f��� �;�xNw��B\w ,�.�7�DU�!MV�$�����&��AXj��z����v�$��k)��:�;l����o��qk<��+*�����j���*uX�oC1*����t`�RKYh�`E�r���J�@#���$bQ��!��O)tO�%���t�sf�����3���Z(R��#����[������.{�����n�nE��[
,_��~k"�9
H6_�\+�!�FTDku\�V\:�����=4E����f�\�� �������@F�UQ��5^�\��l��+�IS:����z�v���rH�c�6��]�����L}�,&��x�{��W#���-��L���gCPt���74o9�����U��*@^B�`j����;�
&��/���n\KZ���9�����^��x��
7��A��t]x���t����(P��������5�����oR����4n����K�'���������%�R3/Js�T��m�i
Y�9
�c��x}��������7/�������Xkq���
�m�O�����=����q;���-��W�jh���iG:a��}�g����^��o�]��A���,>�dGrv��1�ly�GP�no��O^�����+v=�P������r;����v��v�^k2���pi��`]�1q.�Qm
�Q����������;�-�[���7i]����HY�}�/����������������l��
4�4�~�ac�������q�w:�A� ���[P1���`�.i����-���P���5,im������*�x�����^?;|w�������5q+��z���e{�����lwZ����8e3�.1�.���s*yc�����W����0��4-�(�
QJ��3@]�	[
���h�w�f��~��uM�yD���Ka�F���K%?��X���od�R?tGtA�/Z�T�5+��4�6�x������z];�f���;k�I�"I*���&��+���@ne�����6��%^�;C\���>I��{Z@�q���������U��_��Zh��&`Q���z�u{Mg��{�u�[�6��V���%�
��y�W�*�$A���%`���,�uOi����Y�����,�i��Jf���]	~����AJ0�[V��N�,������s*kQ����lQ�L��l��������f7]���m6��?|���0n9�nc8ny��x
S����;l�;���{�f���Q�����~���P�FK��KZ
B��M�0f}�+uE���x9���.V�^D��4i*�5��/�n*=���q��5#X�����7t�:��u���
W{�Ru���B�X���b�I�J;D�������x#���\\�g��*��}Z�s/��y(����^y�e�����k?���tx:�������~�/�������/k?7�
��{�?���|!O~��������.V����,�o��^Qx!�kr]�����������K�3^�����u��LZN�^w���t�C�UB-����4��6r��E�t�*���h�)�ITU;��;VY�gR��*�uj�CAu�ro��B�
���������T��kb���=�#=����;����@�W:X�9]zH��#����������F�����d������c��B,=�n|��r������c�J-���2��h�}��O�����nr���D��2����K����}y��^����@%���q��+^���
��O��Y,���J��(��:�o&�I������O�d�f��5�,���
�]{B{���OX�O?�5O��$�Kb�D>�b:SIe
*xf/���[[�.�q�y��@-��(���J������8�s
J�L
��s�_���-G��x��^�4��`���)6��������+�X�5uZ�q�=q��_��74��jP{��=�����~���W��&����+��
�Rh��0(a��r���
w�����"�S��SVHk�V�w��g���)1���0��*� ���j����XF\�,e���
�
�=��8U�\oW�"��.���\�9F�i���	�%\I��s:������2<!d��|��n��������G������q(��g�������y�Q��oz�m��|�p�����:�Fw��|�wOn�M����&��!o�X�|�t/����^�n�~���7�K���	����*����D�1��EQ��5�YG����#��a)�&R3y"���~f���-a�^M.]��:0E
�_97Fvb���g�J1��n�Um�u�@�k�QCH�6�[���AU0�g��O�T��S8cT�'(��M,T��k��,�8����R4^�@�x��n��N+`Z������zkIA���Vdj8��^w�,6�!(gA���y�Ep���������/�N���K�,��d/K[H��P��rP�>��O}�"�;�����U���?u�b
P�Y3l%�������0���@������'��c���vj:�k����?U����lg&���<��Mj;��tj��������am��j�e�t�+FT��k�Vcj�L��������6�����������:?����^�u����:�����'}9i�&�N�?�y�v{0�t�W��4�����!�-I�W�)�-�=��rS����+�����l/�J�<��M
�c,rc��&��\{��I���B�!7�-�\�����/��.����������z6�����FEM�Mv���'�4-/BUuQ�� �=.���v}s�^U�EGNt�d���[hs�qa�7F����^����,�SN���x���G 5A�����d=��7���B��	��aM!x�^C�Ge��]`���"�x�,�t_��k��M��2��`R��n�K��6���cZg�A�;�:�����j�����i����S�
u#=!��"�8�M�,��_��#F���GX�����7$��T��d=��"����_����X��p�������:�T%v������Jc�2r�fC}���C�AzKu�<iq��N����6��.�S�fU�cT�^:�+T�]�e:�a���x�ao������:7������������������O�_�
�w��������������z������@b�z����g����������{6���{�j�|�T����~��*x�m�b��A�^�
��w��r��\)��?�!d��v}R��f������*��l�^a&�ej�R{R�������;���������+{N���ZE)���5���k)��M����U��tc�5RK��.�����=0Tl�����UPr���&����S�g�[F����s�V�	�m�C����s��O�nS/Quf(rI������GI4C_;�Wh���|������P�����J��U-Tl&\^t��8�Rt#��-�E��I<���Yj�����������<}3[�������!���<��r��(��W�������`�*G�����.��G�U��"����{4�h^�
;�gM���V"u�o����)��v�� f�ip��~g�v�O��5�^�M0�����t��&�&���������x����J{N��|i3���K�=��y�����%{&��JK���A���b���s�~�5$�/iy��9Z����#4�76*��������g��R@�FJ@z�
cRxJ�����+xo}]_��9N�P�:�sd��D�A8��v�%�������Y��*w�_34��_����]�4�8&�d�����cB3�������U�4o�Y���i�����f�o�Q�@�N-�Ko�m�k��1<q����s'����3�Oz��"7�X��b��O�`ML���Hb�BF���������G���3���D>s@����[N|�����B��������������}���������h��1N����0�!�k!��GEl%��v�5��M����@��2J��#Q���,&�O�&�+?��Lj��(��Q�o�G��
�P)���`����������FG�4��L	/E�����L��d�������^(���,��wA�����]��5 l|��)	
�M�rN��g��.����!X��@cR_����YR3�:5+��*����t��x]��	���-�H���2�[M�DL���":h���������?1zE��t'�?>��e����M���ML�}c���� �)�HmDv��	 \��T�M��)ocUTzw3�T=<�0LW��4��4�5�b[5�/_9�%#�cF6���L
W"���#�
d[���G��Y�0��0P�(�Y'���-����T��u�ri
�IG�6�L��HT�
uDW�Z���c�x��\�;t�q�`*�9��������&����_��i��[�?�_9S�{�{����
���
,�5NlD��
����C���*8�	�x������q���m+<Y���zR���/�l�~y���@U�9=�TA�d;��`�JY�H�"��W�8�&�C�V;�N.aE*�'z[*�SP��S@BJ)��kBwO�����1��a����R���t�7���_Pi���������g��o��������=>?���������t���/�O���RDs0�����2�*`���w���x�bt~��
��gG�6~�z4��h��|�p�����������������]�/���}���Tj���x��_p���������oy�������[
�3" �A���s�8f���FU?�����;���� ��`���:��� }�� R/I5sz�4�����������o���;�a��Y��m���)������bP"�C���5�r��P���b��f�)i]M�a����h����wn{U���I����^��W+G�!��V��Q�*������}X�
kc����������'�G~k�WS�E"���&I�LD�:%��������`������*z��f� ��2�������c��L1a�yAr��Z���� e�#�n'QK�����l��L�+�SLj�����1��8�.����h"��?8�E�YA���O�*��(h�����`�F��n��c2}���0��w�c, �0���C��yDe|��e���i]���������s��p��3'^�����)$�<\���_n��r>b=�b���^�	�EXa�����}��%,W�����\�k�-���R�Mk5������@���{���������}-9(��d�����_�10����:���I�!��A^������)@�G����;o#
z"#g"�\
?�T�G;5p��51�c���z~D��������\��������Cc������t9�YJ������6�
�)��lB>|�EN��lT��aU*>Yg�������m���?���������}��/M�`�jXUX��
h1r)������_x�w��+��"�*���$|�4�GK9�%�A`�DU�/�)}!��"���0��r��N�U���N�jD0IhR�(1�V�2�|��T�@���i�*��t�a���F��>y��P������;�W���������a�:����
A��rqc�&Q��1�k��
'�7*a�����SU�1oKl��+�-�}=����(z���
#�*�H)O3Z�ub�����U�S�8y�Tk�/�Q��r���Mi�8{w�ZEO��`�����OR���1��1����%��4_�(E�u���� a��B��
<�q0���MpV�po������u�`�����_�8#(��b�se�G����S%6�a����2����K����:3g�PED4wS6�E%�{B+W% ����q,)p�q�>\;!�tnP��4f�Sb�
+�o��G��RG�(��4�����c:�g����Rp�JGK��@��D��<��%W�t	�hTE��&�1�d��E�X��T��T��:���ja0��������"l#�������=�E�K��v�I�+-Q|��K���J�s�p�
Ha9�rz���6������	�Q�"A]��h")ED�������U���3��f����-�%+'&�������� ��b�����@E�|��������0p�:�2��b����"�J?)E�SE/SA��/1�M�%F������U�|�6�*�N�������
B�DR���B,8�)��L���^Ukl���uZ`�J�����e�To�c��F�G�,0O�\8��m������U�= ��"`�0���������m�H�������(��\	�z*��@���+g�����0J�	wP:��P.�&�g�'��B8��:m)�9=:�A��@�C��^���>�s�nFAJ�U/W��P�w�~�H���8Q;=N_�Mk��XN����[�s������x���
��`Q��X�����d�����f8�`I��!d?N�^S9e+�"�3(��Hb+���K`����QSJ'�+0h�D�=���Y�%
�80o1MJ\b���+�����	c���.,yM�7F'�+�	�#���K��4�{e�B�J���y����:G��"��F�V�-s�"+���9��]���hIZ�z���E��v2����m��n��v�|\���#�����A&��v+`7���_�<;�<����|���|t�����{{tvt~8:;�??��>a�8�.*�����,��a<��A_&0��Fz���m��+:a��V����cb}��B�_j�-�P�V[nV66ANk��C�����?$x���:�*dxO,�a�hm�F�� xt0D7��W��'v*�Sft��n�:7����s��*�,M��[�>A�Dj9�u63���N�j�#��*����\����k�X�_y�SZ5�����7��<��x�r�v:�O��
���V�Ut"�{��.����eX������N�W�b8Y ��S6���2�����.ik��.:��^w�"$�e�=���{�o���%�g�(�g�$�g;���]��S�X����J����Y�zs���}��������v��`u��Pl�	3?XF�!k!��S�h�l��l�`)�#H�wNwn9c��������4
@�m!kQ��iO�
����Ai�atD��nX�c"s��9P���P����F$��;��5�n�g�v��!�(�g�4{��'*l��k
�� u�&:������9����7����`��=��E_%X	7Iv�q<U9��O��
L��������}	�Ilw��vw�z����8��%?,Hk=�f��Y�U�!*�}���5�������G{���0h�����-�m.C=� �4p�@�>��G��c��'����U��H<�7?Y����$l���~&`ct�~��w~��R����{���l�>O����w�T~��}S_E���p���t���������	��4�d�i�"�u�\o�;��'�����A�0��U�E�	�vj���d��(8D�e��Rq�����sc�9'�0@��|���4��^�1���B	��)��Ma���A�P��e�� T�t�DG��%k�R���@��I)0
�8x#�N}�?��8��,��&6����QE<@�;���
Fs� WH��������D����c���&Z�8�E2
B(�T��h�i���������b1_5���HBG�8�s�c�|������$�)��|>�&�la�d��T/aeB9E�|�g�nf��|i%���f����*@���@������W��rN���r����+�)`D�|.n�z1�t�U�J����T"T�T�Ca����0+j��j�9�v"|	>5\����l9���g�,q������msT���U�5$?2?��@�	
u��.�N!�����'�,0��C����T��dH��e���?�N>���Qa�������yF�L"#�*��j�ky���)-K���&�U��MV������L����}X���&a��C�8fd�1���a��O/vC<
14�v5�Q�g���a�G�8d=%��\�O0u�AD}(�|������SA+�����d�'����+5�6lLh^���?VT�������=n��N��k��],?5i��x�f�tK�
jC�+h���f_W���uDn�KVS�hk�H����b#���-P��S�Ie��n*gUe��^!�e�(�A�0*�g�`E�D����j�aa��
�G� �����6���w�jh%���WU����Te��*B���8�H/��9&O��������:��]l�9�\Z�:�K_�\��B��9<d%^S��UjV������7�9�\�����������m��F����ft�TJ3�$&���n�:�
��5r�Q��#���Ja��$�=Y���Xx�����V��H�Cr�G����V���������Y�9��������Q��\��$�mC�y��Vic�z�G�B��P���J��"�J�d���Q���}�sJP��L9�s;$�[��5t��\�Kdn3�nO�c��9��#L1M(��n��g9����U�u����1�3DG�[�*��D�}O��)�	�N��~����X"�G���c�4j����������}c���#X2(a�Jo!�T���n������YX�' ;���������h�|tp|rv��l���Xr#`\:�������#��e�+��|BD�s	>&���q5��Bo.-����N��'9~�L�#��&	'���7�n��h�za���5].�yHDh�HJe�����
S���hrO��Y}������� Sg��Q��6H��b�";�ea� c��<-�lYv�7Qrk��%Z�����v7�ei�����T_�,�H���lc8l�6IVe���B�.�B��0�J���gOg�!���U�V�������>{|�f��	{��m�N��O+H�Mx^Q��oRfz
�y2��`��A'�X�y$��Sz��0P3q���;m��
C�A �gY������;���U����kmU��[��&��<{/B�����5xA��$�?�2�=1��fm6b:Y��U�}�_�VDWH�����$fg���L�9;���/�\g�L�rq��#��	�aso��l ���u�	�?�O�����_g�����=4Z��Z���&�RL����J���^O`�PC�HC,g
.�j�LB�
oJ��t�?��D��B8x�)��d���:J���>��xG�S���pexx����j���a������%;����u������V����2�3���x�f=�
��F��Q:���s�\
(����el}5�*u8��(�H{fC�fs�8�L2�W�����������VD���2�aj�NS����B��� ����@L��=��FZo�����"�|!�4'F��l�r�8I��p�+��d�w�Y/��O[��i,o!�\���!�RN+�k�2��E�'�j|�l�U�@��S$��/C��C)<�H�CQ�o���
�j�Q��O����?+^�.�����h��w�Y���gY�~��
NVJQ3�m��b��"��Q�s�����k-KN)���Z�*�����QSK��Z�����$��	�/���}r���_���}Y����ta�&����dF����@�A�6�cP��F�L��
-EM@r$�x��4�_\����-��:��jh;����U��y�iI8� t
s��+�&JW������,;����^0�lvQ��>����=�(�U�#7������f��o��dr�3�Q�]�k��0�S��2Hz�:O�/�{g����I��+zG�C4��iD�`6��-
�j@�&���v,x6
6C���*�\
F2M��g(�<H�W����;=�l�>���+Hv�,��u>�3���B�Sa-���~����l��iH���#O��``i5�%I��
&��N~�I��}=!J��������g�#��Q�0M��*�/'�&/�rS[���M�������="\��
$U��k��T<�?��y������3�A)�K$"�}V�jENDM�o
���q�1���Y�"���##As24���f�!��N$�,_����.�O��1 wfc��)`����G`u����
�����-��SZ����h7��`����x{��A�D�bD�3;LBS������^Z�S#q�%�7�x������G0Y�[G1�����C\����vQ��i����{}22�K���Mg���`�@��c`�}����#e���%BnTE6(�������GU��3�������rt`��2�@:pO+��O����YA���� e� ���OhHprF]
��z1ctR<�c�������-%���[��2�5��b��3�)��,^s;���}����5���g�V�o�����g��,��
�i�bz<4�=h�z9k���z�C
������vy���H��������C
���4vM�z,�gQ��?C�r�#�R���K�,�o�{8J�iv����z�z'�(J�OX�MPS�N�u���-M��%?�����N�F���C&J�������������P���k*����,����r�rX�Cq�������gi���ed$n����E��K��@M!�"[�N�f�T2L��L���
���c��� B��������a�0��I)��U�O��(�4a*�<hM�DM R���!�1��(�Ht��-�'.��.�#/K��E�����L$,�P� �"y�L&��EY���&�m���b���qie�DJq]���|�������e~O����?�R�c�M��{����n�)����m���zK��SH���Q�A�����M�DW#������>��#@�uG5�����������$-X�*���]C��T`��V����~�����j�[W����E�h�����	"B%N�6�������34'��1��R�>����k����RC
fJ�_tH���i�-������^�H@��?�y�/~������r������GNW�
J�~�������0u�����Z���N��"��y�k�rg�i�
��c�T�6����s�M����$�n��2/Vs��u������r�[r��o����R���?u��b�������V���m�L@�u�� �Ms
�oS��[�)c�)�T-��l�������6dF�t�b���6yF�2�/���F�0��q�h��N8L��VBn�Y���������nJe�]}��S9y��lm�����S�����if���m���m���yr��� �=$-�4F��W�s�	���8�A-�R�����x���>��Y�}����o$�
�H�������������]Nf�N�>7gw���qWsXh�x�|Jh�knRoP�.@B���lNcY�������x5oA��Y����x����)�y�X���+��O�����n��z��?"5��!��6���������e7�N���A��T�<��la����%�@�^E[^�&|>�W�]��HT�a�"����+:f����D�����y�
�?�X�G���O���]t��.G�����g����

w�r�zfm�
�W���5!	���������(�#�I�~��4G��<�����W\��]:����-L9��j���zn�S��!���L&��pLT���m���1�����9HFk���h.����&�6I���`~&?d����2�
��0��{���U	���n�<}m��8X�����\p�/TT5#�[�}������W
�Q������+�*3Q�"u���i�Q�|����l��c�gI>�}K�B	��M����|��-r�m��Z�r��~��?�s=�P*���Y������`_X����+�G�O�m�������
�&��� ?=(���������#��2wBm�D�%����X��mLT���~uKj�
��aI����+�aJW��~/���&���fwgo����[�T�b>\��,���b��w��;J9��:�\�knBl�/�g�C�����1��}�'`�S�`F���4�K/���'	�����Oa3`��.��"k�Eb�8�L�th�����n���M�lO&���6@��0�`�M���-�������~v������Z����_�����?Ig���
h/�>?�8�r~t�>W���z�?����f����=L(�-Y�p��{g8k�D����U�o��~�����[�|���}��}#O���(�CE4L�3xw�w������
��Ia&�����bw���DQ����V���n��l#)y�M�?���E��N��5j�[���*���[,��N��P�����N	K����)]�sJ���t|7��h�S�Z��.�a�����W� zZ:�c�~���e�4���j�a�po*������Q����:��~l��G���������)M�t����Q<���A�&g�ry������*��j�&1�Qw���T�oA����p�54�O�:jR\��/}��>@G��q0����<�0�w�Q �1���Z`�=k76�C'��N�D%F��!!���hD��'Io�s�0
������������8���w����N���nt
�W�T �i�K�"CB!������e�	����i��
�|��`N��*��w-�M���u���>�
Q:�e������#h�M�����P]�b���Q�UUz���O��OV8��z�E�T���+ZmQ����7����P������0['�v8�������C�b9�OA���Frr�c��q!�dNm[5�����������3���s���o�����p�c����&�=������^�j�����S��Nwg��mt�<o_�R����!�Um?���E�N�?�'@r��������D��1��C2�����E<��!��t6����0��) ��	44N�;m�<�K����{4�����J��[=`_��"��)
]&����N0��VT�����NFv���$���mH��(�U�}$5`���[�s-_&�A��u��uV��F�!���~CMq5�6�`�j��2���n�@��|����-�|mZ��|,H���J�n���@���H�Y��-D�d
TA�D����P��T>i���6������M�������m�'��H1-��pE)�Z�os����J&<�o!��ecw��)�����0\�S:[�T�hB�gi���Rjo�C�������9"��HeX�����+��
����-%g.8� �4��CC�k���������CL�����\G_O.?DG�_@��gI�G]�h��'Y�_<�����5I�&5�g�p6d�A� V��l��mV���k�{TF��Z�I�S:��.{"��In��hw��n�y��k�H�^�%����E5�	�r�p9� Dp#
*�8'�����:���=�������V�j��kP��@^�'�J����[������6�������������o 0�{�������Z���F�Q?�v��^W�]'*X����L�BV(���oF������6Z�o�g{���������vC�o#j�v��6�G��g,�B!�P���$-)���z%�i2����%���h�*���F�j6{��^����Lz�I����4�[�N������>���"G����x����9Y�f^D�7�0��O����)�k�%��O�����P����5	T������5j�������m�~S-�^Pl���o�/�>(��y��k����W�Cx��c�����8��[�]�u��<��i_}�����;�q���)���b�xZ3����N~��`D	�a9���(7�F)�QM��!�;���z8Sk5q�o�<��On�]{s�"��
��Iz}�\��Zt�z�H�A3H�q�^%j)>�q����ufY�����t:�^<>F�F�=>?:<N��W���:g�����s������\�~��4���_N�?}�������O7o�����P�����j���[�Bh{��������<�������^2y�6�9����<r��������T��#��jD��?������o"���yUx~�7���~Dm3��(_�h��	)�Rc�U+55E	����������
�1�2����������W��ov��N����[+FZt���{U�Z�(
��WH1Xo�|���������f'���C�����l��Z|�4[{h�q��:��|B��Q�(��)����$]�L�����nE=��=uQaNq���a��V�G-���Z�������?:<?����c�_���[����q+R\S���g������'h$�����O����j���s�_|8<?~����Y1v�g��r���m����c�I����I����A�Y�sp��@}��VK�IC�&d�wJ����m\5[�$��llt�:�����N6�������n)�
F�W�7����k�t����]p�����5��`Q�a�����>���^���'������uU[��^����."����d#EC���_�q�������|E�2.��������A�����P����[k[�Vo��M6�v
����h�n����l�����	�
���[1v�fm|��yv����N���q7���[��<�&��WT���)"G�v��2r?�Q�o�/m��{%���������=g�~(�1�Z-�����9��-^ZXtF'�������M}6��FT?�a
?��D�GLK?��z�X���p�3EI��>I(I3��(%S����ee����"��KA��cMv���"�����������j^Rr��DM#e��^ a0@g�
6l���a����q�����(&�����
�T#����2���Dw*A-@��j�*��*]��E�*�R�����R]����h�q}�i��P'Oht��.�E��������	E2����3g&<���s����������x�F�����Oo��=��E��TL���e)��I������M)��}F����),rT����������t'��u���^����C�'	t�Pl�gA��FM"��tJ�n�u={=�;�v�����G�nB�{n�����i�W%������}���f(s[��j�w����RU�_���x���������m������L��v9����G��8�]����e�w���H���s�C�����;�s���0�}���:a6�	�����g9�w�.��R�6���cf �\Pd��D��oQw;a-���x���V�������E��C�~��2�p�}�&�g�����'5S9p����EA���N��������nn�����|E����}y�{<��nf�v0�C�$s���[)B�T��K�~��ZO�X���=��J�'��	�Z�L���1I$#������)���]�3���+����f
�Se���=�'���.sxqAZQ�ub�$(�1���t����d�I!*�xkW�`�X��$�����]�c�^�������I'!�<2@�`��GF������l�������$P��F��
��\`=A`����@�r�
�#��������c�*
�L<&�N��@���$��}}Prg��
�j)��;��i���%���S��������Lt������;��C2=?8b�8"48��e��)xr�%�q8]A���!���;+��UP�<�S��z���u�,�u���;����EZYI
���>�����LunCK�j(���`U�4VXT��H��������
�Y�
:�W-��r�t�U�]��,���sf�:rn�OOF����3�Rj[�A~��g%�]��f�1Z�t�hm��������&?��u�$r�"/rk�.�Mg4$��/<�I�8x�3�b!>`c�$��I�C�X@�H��z�+:��	Z�>�9����)��5���;u��K���Of
8X�J�����B�~�=�N��$o�'�&��[+��*�,9~�r������m�)��i����[��U|�[h=���o���6��Z|.E������v_I�a�9���y����`{��	1S-,�&kK4�;�B�Z�UW���X���@A%�q�w���l;�a���o���6��3�EU`��.�����fPg�8f�Y� bO��S�����z�A>�tF�,Pg�<��x�\uu��dW�
���1�0]Z��1�J��xu��u���f��v���Z�xx��l�����)�g��?�3q[���4/ZH3N���}������z&�
M
V��t����|��\H�
��x�^����1q��d�g:����gI2�a���� ���*��!L�^n����2I���lxC�P���������bP{����_�}��0�H���S�g������M�O��:*8����������J�?���r���%P(yY������S$�8��
0��Px�6�������>���t*���nSS}�elH7��L`QD�B��K�V���WK�fYm�_G�:��k&���E"�W#XN�gA�-PK��z3�UV�jT��,i��Q
��:��\hf�`gy���`�H<�����[�$2�Z��)��c<�,�,y,>+4�(]��Q�:�L�Rx\��GOF��nn;��bK9��4�r����iCmm���je�N�*�/��_���k������E��zaK4M^u'��� ���/�E<o��M����v�~o��Z���R�~��^����G�<�0	P*/�7�bSM���-R03���c�0��	���V�ktL� [�L7������Nr�������d�c���C10�U��+��� ��|\�k�=��<1?Y����hz�{0L���$P��L����h7�������_>�p��	`�{Jj�BygW1��9��`gH���+�����~^a.�IB���#L����g��6�+-����|-��BV �>e��
�B~���4$i�G`�o�x*Y�(�������j�H�!������G�V5���U���V�%-���9����g��x4$����1/��*�q��$HA��?�N�V�)l�
���5���d�� �Ix���2��BoH�D��t��N<�50Vh[�2�Y�H���z�uI�����@���d=�-������?��S)���]g
��`�i��&�Sji��!� ��r�J3�|��_�OL���:����-���_OsNg����|��+0��s��m�D����K�'5�X��
��2�>�'���������@
4I (��t�B����(!��h���&	�$8����NH��S4�d��I�����LR�����K��}qG}�5������%�J1_����=��<c��bMX��#B.
�a���l�jZ�����e.�������+���M!����=AHcH��*���<�2HO`o=�|U���Rj<b? C�����b�UE����:
�9�ci�^��3�qfS��x�O$��4�-��N�df�,����e�Aa6�U+��eiz���I�f&�%����T���z��f
9:3HX
��F�ke�`�4��H5jV�_�\��kU����+��TN�Lb�)&o��dPq�J��`\X�K%���������&�EM����i�"����lW0�
�D+�r�~�F3��;���2b}�l�:z��Wr��\0�
e�FHv�4����2�/�l���yH�������
��
�^,J���BQd!�<U�Z���M+�	��f(���W�?�����!������P�DP%EP��~���U(����WKm�5�������U�
9M:)�5( R��8��tx���E��uZ8������L<2cg���n��V>)��
sC�l�P��"�X�����������g���w��!�6���b�9*���]0�G+N2���I�!O�$�l���2V���d3����5. �u=J��KQ�B	;�n�[���2��veL3��;�����3���.T�"��|E��[j�:�4�hp%�H
\�`�����IdQ�ARy.��y��wH�^�z6a7<N���"r#V��A�r�b���s&���tm~6lI�p||����.o)��������|�K��L����+�iG-��,8{*���&���e��-.�����S3�������mDgs�l*���+QK��������r��Y�ft4o�eD��v�A��C�7'Z�g�["uL�>����It�7`����HC5Q���*;K���K �/o��vA���)rV�F��1��#0 ��8�E��&MW�6�]&h����Jp������=(�v��7I�,����
�$"= ����>�6<�����|�
-]���C���L6_Z4YD6�'��~+�`4bD��SJn���s��/��\��%�Z�Q
�PF;`.�z��>w�8�Z&�m��|��a�=0O�,�
���P������O��`�>rK��k,\�)�-"12�jwE]k����[�]�u��+_y�(����� S��4�:TZ������������j���2���"�8;��CvlY�4lq��J(�YV����#l j��D�m$(�!����@��e#����P�B"��������%[�t�(�qJ��������!���~��rp�q=��4��B�Cw0���r��t[
�N���@����o���l�M���L�^�I<$j���W�����NC��J�S
��!�B�_:���'�_�W�t�8�����\�Q
�����m�47[���������~�����������b�Q]����C�
�ApA!H�.����{��2'��q$��z�?��?>�9_>+��&J��y�+e�9Nu@�q~xwr~q��rq|�~{������j�_~o��Ufu�P���R������;W���-
;��R��yH)7y����o�RZ��	E����[��#����`.�H��\����jkhI����h?M(��|�4X�-���]�	�:�KBH�����:-Y���O0k����[v*s������9F�t���[�G J����.X+�����?n:�^�e]t�����������J�cUx��`�p���dXk�;g�|j]��(�g\��.�������.���G�����y{��hnz���;���������;[�{qk?���������*V}�7w����^�����N�w��
��j��U�n�h��hm�?7�������66VWO5�;�$������g	����<�F����v��\�����@WaSp���	�4�� u�x����{��Wl<����U����>8���d4�#g�}������H@��������D++��w��(	8���Y��j��Kg�Y�[�����`B�X�a�X]ED\�,"��S����y\J(��xC�+k����)���U���o\�V#8����$?����j�
��Uh3^�m�D��Q���F��UXO5*��:>L�G�G
$;P�Q\�88�5G0�Uh�
|C\K�������JFJT����NLI.("�Z:=_���
������K���_����!���$��G�����
�:���_������������}����u�����t�M�}���wn��
G���o�_n�P����*��z8xN��
�{������.��t��	�E�����v��M���\=�O/���R8u�^s���CG1��=���8v���W��Z��xUg���4��4��:�E�J��
y���id����s�o��>Z�]���Fz����+�q����=��������()��Bg�����:��n�z��r�dD�V�����[~=�����Y�h���	���R�zF�x[(z*o�����`��_O��9�~MU����4WQ_�@Eu9�W���+��GJ�j���f����EY���@�6Q��@=o���9������u��&<���V���':������H�3=��a�*�Z-J�	p�I�x�7��������Lr	��:�TK�%���������E��vv�t�;������U�'��o������V��������u�v:�j�Ka#��/�����^���&�^O�-z9Z�I����,{��
�������l�����V���Z��*/���@G�&J��_���Q�!������~����WO3B��)���?�~@���*�zR�S=y��(Y(�_Cg`n{]Z�������w~���c��a/����6n v ?E/��5��D���vP#�U��������y�C5&��f���T���������,���.`d�����~ ���inLbh�%wq�H����K�>e�G��S�?�A?��*���vc�a>�|���>d��!����adr�uPW��q������}X���E�Z�y���U��0�D_�f���6��.L�P��^-RJl�����4��vbE}������n7���E���/�D[������x��y���;5���cG�yRX�h���aY�wJ<�_���jA1�+7uK7oDX�y��H�.Epe��pi��kM�����i����^ ��:����Z�('���J��������o�x%2��t;`H�����Rw��rB���
��'������[S��_��<�f�a������74�b�?��"=��IK���?_O��F�%OW��phHP�G�����������vH����{�3��R����'��7��6O����}���������^�����4-~M'��I����TH_��-tS������o��?w�e��������2����Q,��EQ�s�K�C�i���zR;�������~��l^m�}hM{�ZS��z�������^�����vt
i���
�*�������L�k��kk~qX�g�]��m���y�V���a����Y���+��,�)�����Vo{�u�knl4�vv����\`;��J��*E\1>���a�m����K�,��r�O�Az���6�F�W��I��x����BLEl���\����Q�ak^��N)H*�b���[+��$*�p��x��������w��&?��%C�K�� �s��
�|��w����.�����d�QmM�#7�hc6���D���1Y~:�O�m���E���;�?89=:?>�P?\\�}����=�@�j�9���y$�����N�J�F���q�&5 O]��i8:3B@��K�#��o���L�r�U2�>�����������/G>�l_�}z�fpz��u����j2�D�&���sn��v��z:��UN2M~.Q��O�"@s[���P}��y��a��h<'�S������R�}��4�G����S�G���
����6�"���]�_��-����oaz���tmf��?^RpNS��?2�	����?�D�DS�����'A}��E�`�����2��$�}Lc�\*^��gT
r��Wk��������V$���|Po�Z5�3�f����d����Y/��.�V����v�l�m��S�"V�69X������&��X�O��MBi�p{3}��~N��������o���I�C[�
|�^��@Y������/�@�/'���r��������t}��i�`7������7Q�/��W����������_���2��9G��`��[��;7��$I4m]���g�bb��P��i��]���%�����X��g�h{��(��9�X����(Z7:� �4[[�=�Z?AF')���D�������Q�$�|�wyeE?� N�!�:��3�^�������������1��4���;�:�,��	��TC��g�d��L�8������B���K��`�2Y�Q�8�����<�fZ��s�N2��v��*A�Zy��m*2~�{�}a�B��W�`��%p�v���g���?G��������!���f�I2�8o8�)��YZ��S5�T#yN�f�7j�_QV�4����Z���eC;��O 9�%���	�_��e3�����~�N��H�F���?�S��%�)��������{R�`���I��Yj8���v}���N��Q��[�:�����k�'/PUd;l�f@T�>���G��36��+r���j-�zxr�>����5U�t��%�Nj
������o�1�/��r��w���4���N�����/N�����K�$�1{-�f������Z�5~rzq|�.�H]<�')�8D1���1�*��X��'�u�Ue������=3�U�Z������:�,A��K��(����x�Nm���B��q}�RK��i�?���������O"�!_��z��Q�x:Q���L�#�&{�-/����c�i�*t�����T���EQ��6�1$Wf������=��5�pN���q�#$B��4�����6j���B�l�PNN�r���m�V���Q��$�V-�	�@��0(���1e ������]�r��m	�(Z����KW|a~��_��Q�U���8���ksGN�l ��3`��� ��4.!v�o�P�e�04����\������P���m��8 w/O+��8�yJ{��c1E���$v��GDC���7M�����g`��\�`r����#Z����!��%�=M
-g�qq��1�\Z���tR&C^�\�(M�}�
N4���3je�K�zG��BO��t����c`>a�R��&ow��.DF]���4ct_��[x�YY��p������������l� $����1z�[he�uf��u��"\ �E��M�F��`��Y��
2����-�{�2z�A�0�����������L���T�j����<��W�E��6��xP�K^�.w^���������w{
���{.�b<�[e��4{h0��	��H.�I��lt�o�&�Z'�cVb��+����RK�2�������X��z:6����b[�b���Wm�=�j-�����7�q.����ha�\�� N������a�m��s�S-���%�6
�z$p4C���Y7�}���

�)"�������J�+(�&�db��<�tja���^��x�aU5;8A4rB��Y���M��e��0p#�3_��y]�����r������]��0��� ��N:�e$�p����� :�7n�y�O�)Ee �U��I��s�(1��no:]�L�o4B�F���<���-kM�qm���I�\��Wn4�XQ�^����5�o�!�E��8�'���$n���r�p���@�J�����A)S���<�xx(����U���b�V~99	H�����I#P�@f�	�E0�m����N,���K�
��a�8T�������
�4(�R,-�Y��������f�������3&����U��	��b���?*P��O!=�Uw�����I����J�����u@D��D��P>�
\�;N�C�;5�v�j���;�v<��������vt����z�g;�Q�h&y6N&�B )�]��d|: �ZQ+<	v�[��^`��ZH���`�n:�m�\��u�&�h�b��oe���x���F6E"�U�2U��L5��2Qw��q:J0o�W��y�^�����< ����:[�ZkS����O���<���0�>�s���H�%�o}��OWe��:J����cu��L����U��p����@_��I3�D���m��',��U�[�&.������i��e�5]��d��#���������|�����f
�;�q����MQd���H��L	��v���j%�p�|�-�_���mQ�ax8J��a�Z��cam`��Z�]8�#��m��Ep���������
�&B�S��$}���dQ|o��t&�?k57 fh����XX��3J�aH{�T��0M?u���S����������I�Er����j
��%:��.�T�v��x����C�D�v7G��:2rAEv>���g�w��z��cL�5x��_�u��^���	��w�[�}��&���z�����wYP��v:�_�G���P����������1f��P��_���Z��G|����������<�^�/� 7��9����z��!�1�Y�il6Z[���J�������
���"p���Z'���g�n:�J������9��{-#�l.�3�h���o��d�Q��#��(�:��_�[������b��!�������$�Ye���|
7�������@���PO�?:����������^m��>���N��������T
�����!��t���I���77�Zo+�m����%w:��f!�(AM�n�mh�Yu�j�N)�����%�>mJ����VH�Bc��w�b�i��@�^�_��%�|��)�����if�H����zb��J��l���(��q�%t;8hg\�|�L���c�e\:N��f����������p��g
�x8Pi4@���M�ee%w����/<Z��&���w���xLyc�3�h^���M!���*l�����0rI9�Z#��<0
���x��p`fL3�c�������?"�@\�� \�	���l`]0���H��#����S�[���{ ��Q�A�k���KB�tE'��@X���U�s.
<��@��>�F��j�������|��Raw�	�1���3"��x�@E567�O2�����>~��m6&���-]��$
�I��(�a�"H=Y'U8'H���HO?N�R�d�������J}}"aY��(
=�K����s~�=\V>b��������P�Z���=��;~�)��I ��q��8Qm����7��0������X4��8�����6�2���' �O0��L�1;9j+��@�`�n����h�KOp�pR��O�y�� �d�.LmJ���)��������hU�cy���d@5vu7e������>�Z*$OX>�{�
��G3������f� +�^�GS����h��ro��z<��5�,K)��s�������yl�u�gS~�>W_���h���O��;�j���2]�]\'�v�d��{��"nF�4��"P��^��8����xLY^�m����L_{�����u#��~2�J��Y����Jf9I�����o& J>r�d������~��;��7"8�
r�=c�q����4�Xt�L>�0�~\�>2 �oDN��g�L������hB��e�Z^��\R
�+�YD\o��#�&dOxt]F���3���
��Ku ��L���S�n�sm��7�lF ;	C��1�i?�B5���Q�T%�4�g���+�w�������hp����Y'��R�v{�b� o�����<@-�~��zD<�n��{��fK1���L����U���[/E���/�]�����+`�1�{ �������fv���R�{���	���^�B��V�W��OF�C�����2=�ZsUEE�F�A����d��fY��)���dS�7�}�tI���6O@�u�'��"�;)�r�[�������G���^���c��v 1��)�A����<C�&�3�����B���~u���p�l�C��&���tw�.��p�Xim����}���(b(:\g��#���@�1��$���r�����Q��y���N���_c����!:�9������S1�Z[�/\����I����H�=�L��V]?����LS������o�Rdtf��������y���h����k{9'�$�2����+<�lxj�5�I��$���S�'�&Q��#m�-����6����;A�[}��&`�����
�7�2j�x[e^�������qk|���Z�)�3��P�S��,c�K�7KM~�x�SS�t0���\���oFM=`����������b��|�H���t�[���C�v�a�o����3����_��#fY��U������0^N�������0VS��;��B��a�
h���~�j�?��AO����h����
/�����M	�n����;���L+��������X��_����}�#���r|��������;���]Q��o�H���E���4�x����O���5��(�48��da���0/(.x�2�L���jCn��dJKs8����1dr�X�:��5qS<��V��������d���^*V��c�.��cq��GQ���h�l���Q��4,8�x�N
��pK
��1z�r����������"�K�v,u����1�=�Q��E�\9�XrS!�n���x��F�Z�$�xOj��V�{�
u��Nf^w���'�i_�G���������Vykc�z��diW|
W�FNk�s��ed��&��&$�J�-|��{�������$5ooT���!���x�s:'�����uP��x0���a����!/G]t���\��!VKH�Z�/���h1������#��\e�"��Z���+���^��u���M{�;��z�w!���u"9�q�C�	J[����r�O��U�@Y�:�;�����m?��kI���Ov�L�9��1���$��uEY�����)�@��	���d�B���T��(��"6h�d��t�'9L����y�Gg��Z��K�~�t�����%[p������K�=�f�OW�<��@���4�Z;��P�2������b�;�_�dj���Q2��:I�t6��4\��l�V���i)<T�#���1t��	XFK���,����
�X�&�����1f+qF��������M���9��W�/t����R[�a;W);��U���������
1�������_9c����U�`�d{f	N
�=�c�vQG�����#JY��g�����
��jd����$��:���x�b5��X�$��0�.D��hS�V�������e��d��M���L��~��LC�`[�mr�����0ah�g��}���A!�k��)6*��4gH-�D�
���\`8���4�����u�.���j���tX�`N5�3�Vg��c7���Q�Q@�`U���F?��t�����q]��f������h�S���.�jb����'>>�n����;kg�]���ob�C���^7�P���E��2^u��5���O"��Q((��\�~�4�P�	��4+������E}O�����/��`�>M������~�J�P=U�:����kE���B:E1G�-������>�%/�� 0<�?�`x�xx�����/.N�x+�	�hs��������j�}���t-�a�0�
��.I�s���k����p���J�K���t�z����
�������������Xa
B���Z��&ds	hl�����&H�bJ<_�\L-R���j������
���*��z[?!���9� �P�_;���^2�\���v�4�{�e]��������"�59Z�6���a��9���h��y�Y�����g������0A�I;�4��A�����[C�]���I��"�@G���%�C�
h�,�9�Q*6����s�c�^?����w
����T��KWO�����_�c�!z!�~
r���6;����.d����;tq�;��L���K�:�_ �!o�p�3���{K�����2Z�#�{�(����J�B> ���/9�$�Hb��g�Sg�F �[I%��Z9�W����qO�W�)&9 U6�����~�;@`�L\-bB5��_�A�!��[�53���
u7��'3�P?6��%#
>��I@�)�d�N�g���6��:�OJ�.��@�9T���A��q���$A���fL-�d����H��F�RT+���������O6�WC�h	�P��V��rT�=�z�A�L���u�.��B�ta�5GP�� �5�&6�`����h�$��Z��#t�@^��XHF��>_�!A
�nR�GZn~]���:8Lu��`-��4w���=v�>Y��L]�U��z-��M�Pd�	O�q�C���n�,2�g#Dx�=xH��8���!�������g	�0p
`��T�3����(h��{�������";����@B�-�������X��~���1h�Nd(�Z���/8s��P��dy���|� 8��!oy�/;��9��W�`Pq'G�9��<��V}<><�C�����H����^f�
�L�"�Yj"c�B0������#���k�Xy+:Bg8+h������N�����.�D���h�(M� Y����P>��Qv~������<zx���G^r��,��~������T�	oQOn�LCU�2�9u��a��G�~eKz�3���	�RN���]�^P;�����%)k�t��$V!�Ok��6`�B���w/���z����Xnh��������K��ST_���'g����mzu�] �n�@���6� �|��z��W��pb�6#����)L��X�Md����s��e\D�%��r��!��?0�K�z�j�1��m`i]�3���H�y�W+��'��4CN_y}${=EE����/�o����@Z��?�7n��'kQ
������=�@��fy����\��NF���
���s�qY�o~|t�:H��(BT#�0��%��q8M�3��8�?]~������������Z����;�o���O��������j����m��_�7F����}��0	u��m�T��N��JCe�����76%]E�E��e��9O�W���u�F��,�y��h����m�
�7+�T�:-��U4�]�"&���x��c��P��m�|�O���b2���|a/N����Gz�dw�kL���d��������:FL����E&���Jax������+�Z���<��Y	�+6
������S�f5~?j���+)Xz������7���U|�e �_�>�f�Y���_�l����e��V����=yaG�Z��X+L�����)���P�`���Sd�%��C�u�*���f�I�������C������?������Ks|~����A�0'L��g���C,��G�.�3�r|�IW���A�^�Xw�a�c��jDY~>|Q����/N��������i�����H"0��xq����i����G�{����$?�s�K*>��������	x �q\�_�?�z%��s��_��/.*U�R����
_Z\P��P�����������J�v����}/�-����
K���EH�N�J��! �������l��o���W�,�V�����{M%��oCz��_��z��+���G��{���1��0��CC�������wM��u�&3 H2�s�iq�??�d�1���:��?����P��fQ���5(��2����NH�,T����	�2��6m�y0�����+�/M�-A��	��L���S0��/����qrY�R�V�uqA'K��zUx:��8>1��{=���L���\-�]3��h���(�����(�z7oW�V��g�*�!�������FbR�Yfb���6@rL�YH�i�8���Pa�%B|X��S��0�������f�`����5��c�������2c{p`2X�d5�B��}Z�D�9���O��=�����]N|dN<?B�����/�"��"��I*��'�y"�	$�lm�w6<�Y
�0��D��k�j����}��l�q��Q^a�o���":<'*H
arOXY'.�����s�W/GW'���nR�nc8!���8#�k���a����g#����"X6FD@�����);�3H��	�`_�r ��	�J&�5'��9���r�e&d����cc������h(�C	�a@a_��
��>��p;�+d��wa�C}Jlp�og;���v[1�r���;�"=���^��K<��cK��b_��6�H=s����V��'���tc��Z���d�NP�#t+�C��\\�2��~��]1��0B�uyFn��g��wUB��EX�<N�j�D�T��v+��VE��"�b��X����j���������8�������J:�[����OZ��EK�o]����6#���a����t�����S����A����s�:N�x�)�@�����s��	E0���De^��x��(~Q��k
�����Q/�����9�P�2�R��/xld#��1���;o�~��jaxz�������a���e�B���|���*���Z�U���fmw�H�1�"���O����w���k�?�����m<�&��(?���'���h7eI]���k�P���9p!3�$���i��������n��2()��[��y/��I?X�\�hnBO>H"9�����f�GC�1#6����:Z�3������#��"��<��1ap����G��dl'�"n0Cw�yY�@i$S�U��7���(���E:���o����E��S��2[Yni���d��x�A����~E�.&l�?�6p�j-�[T���^h�a7D���]q(�@x��N�ZFf�S���Z������h.�f�����$
iuF�Y�]#z����l��������RJ4��?(/IM�;�Z���^�)a,�nvK>���w�b��l���M���i�h�(��p��X�9�,d�8ss�.���/��@�e#$(���-t��eB��
��|?T�����E�3���1�m?v.,s��������\Z�O���o�'�G������_���8�-�B���G����^����'n9���Ed�_)}����v�(Ug���^Oe%l�m��p�d��B�4�>�{_�e\�,G�Z�X���q���n�##��^{�G�����O�=�x��tM�34��(�O��f�5�(�sM���3�/V��!t�85�s "@p�x�i:f�]F<E�����eio�o���Mu��i����[��UR�J��W��n'#t�{����hq�NZ#I��v����2���g���t����},�ju��GQ�s�?�����������e�`�b~����f A��%QB�����X��p�y���~�nE�J+�J�"��j��R4�ux��t&��������7������_`����@�G���n�%���E�g��G��8��\~n�������y^��Y��Y�iX�5p�	�}'�V}M4�!�:�AQ������D`7�;t��p��,W#z?�']mV��Q���Q0i�!�-��(���Sn�T��JDO����sU�U9Um!���c��Z�e*�����WO���������u������g6��F��N|OQK� 1�Q[�3����6�w�Y{	W�����yd�H\�h��Z?B��];��~]��Z<>� 4��Z�����n�~>
Ba"��i�����0_�`��*���$l&`Pu�B�����n�K-�y��G{����O�G��y�i��=,v�����mo~k�&�|��xNN�����m5��,8��U�o%m��83d�
'���x6�fQ��l�������(��<1^�z�^��?ti��+��E���ME��
�mXo����#M���w>z:�

|!n�����-�A�'M����/���`�}�]����U��'�� ������9:���E�����:Vg� �3b 4�Zk]�y�����MsL
��5f�g��:;z��*9m���Z2��l4ck�C~#uQ�������`�y����L���2��-r8�+"^��a����_���^�,����SyJ;R1*��Zk��Y�%�|��}����<,������uZ���I��VB/�X|���9�E�7�)�6X��cR�����0y��F,�������}Q�XS��A�[O8�^���������X&kd~��/�&�'K���<��^T�`�d�'�k�M�Wrm�eiK|���=������MGy9��	�����d�������FE-[���v>f�.��y��b�����D��_,��s���|����E���/���6��J���d���������W�j���B�#5ThBtpwH�
����24��%g��� �iQ2����~�w 9<��JR��.�����f���=6W7*=U/���i4K�<g"i���lY��cU���a	���^9�]_YL^+���WM:�w2��������#��E�Q�93�|�0�!��U�7X�GG�����qfD��[��]��w�s����;|9�����c]�:E���!�J|���.a������js��W�o��x��T�x\��i��y�T�����W?*�4$���������x,�q���~:�4��C<�D�2{_N?!��[��_��Q���c��'�w�x���@�F|P>�X���v����JK�k�*�t�O�i�O��Ct�����&w�e'�I�
�mF�p`�"���~�]?0������%����>�5��Q��:/h�x��-�V1���] /�t�u��,�t�8���uz]�T z��b�2�/
?'�x��K�H`�C���L5eBn��e0�b�N��VTx|)1Z�eL���@���<���f
��KeKr��D�m:y�J���y�Y��%{��d`@�6��H�`���d��B"1�*�X�0PX��}��xn��m'jr@\��U�M>��Uv�����/���{�%��:��k)�������	e*/���;g�-�;��%@� p��	���A���C����>��]'v�ac�xv��p�^�s��x��D�sD��#sX��� �&��u���������� w8�&C�'�.EF���S:^��<s��>��c�]���Q���u:B���
q��1���fC�>�L�K�T�E�i�~3�@�s�DVo���%=�Fq���;
^�c�]4���)o6�?�m�Y���Kf�tU�w�B}��� ������)� �R�c��\��6eC/��-HG�����	[�fB���h6��(5��.��3Nh�Pr�	�M*<�y����G�`��>	����o!�r$]$��E������Ztq|���zX��(VG�9k���g����%�	@^��4��p�I�\��
Ud�]�=!�y89ZL�����-�����������T�^�/j�����)�~s����Vs����*�.�n_�����I�En��Z �#���Sq��2�EF�����
q=����`r^
�����r�"k<��t�|?��!�)�c�����n9<�N���
����Z>Z�����-+o��E�!"���$tF7�J�j��j��Bu6�����Q4y���]���{�'x�S�rIY��C��s\Oj�����+��X�+�`��"�O�0i#�A)R�3$2�m��aS@����R�OI���1�.8�,� ._(�����-�����+�%-��=���D���lm������;�V�o�G��?
}Q��(ZaVy��v�g��������mJ���;����'�1'�������C��/_�����f0��� x��(�f��R.8H��
����v�1�$�Ft2����3J�mt@/���bA��(T2L�D�P<���
�"�/�b�JRd�5I��A5s���0;$m��zG�{2	�A.�@��wp|*�3~!��C�O��j�R�/Db�3���K���XV���|�DM�P����u���^�i����s^7��Us��C?�2*�9kt	���u�Q��yp�C�,"���Wh�X�>Aq������{rev
c���Nqv�h
&RL���w!��`����d$�H�u ��m�;]�A��+�.O�-<��~?��p��O2�6�,��F�Y��= R�zG�}A���C��"����h�����p�*1�1	Q���|���R?vj��Ez-w��k���B���Nm������j��+��F/
����rP1!���+����O#�@�c&^x�~���P�_�t���{n��
vB�?Xc����6��R�����Q�!
~z5��oU5���H��,�	F����l�C���(n�N!C�}�0B
��F��-�#�L�����BN��F�SM�p�L�E�����|��o���C1�����!�B��7�$f�
��w>YHu�$�2��3���7��BgY$��
�U��l�����5`O8�6���D�R\.��4>�{:!�q��<������B��X�c;��b*�{�G�x^a����{�����)�e� U!�GGC�s��	������q���W�����f����D�[��e��p8��ivS �H0Z�����Scd�
�IM���I��DT��d\q�$�!�Bx���D�r��`2��������6������k7��~J���Q>Q��N��d�PP������^����Cg>����%�����}���!�y�S^�D2O;>���:aQ���G4����-���v���I�QL��FV~c���=��BI
�����2�_��i���z��5D�g|��Q0�������o�����@���6���0�B���iaNb)>�o�~k�[��W���[��]��[aRs�&���S���W�$��������E\5 .z@�u�8�j�BU�g��p���Qtn\�90���l�!h�q�#�1��X,v;|��%t���Nw�6���Z�lRs��b�8�1�i��E�T��^���0E��<���^� �\	�r�L��:��F�.Q��%��)t�Q�=D=�C�������B	<O���L�Mca����v��"Z���D =��q`��\����t2s1��6EwDag��X����E�i�p�dJcg|W�c����|�Cx��X$�A��&���Qs��N���}�d������E��*���G�O]>6��z!	���0������	�9Er�w�[a�@��8���
a�\�����u���{u5�X��������h�����Nz_P�����V�^�x��
u���p��)[�(�)g|\��f��e9Y��"Q�U��ZD�]jLAN�V�'K_^�������!m8Y�|���������<�o�6�;��y|o�ko 3��J?gwX��@Av�d�>� �TW�o��6[���=�����.�7�SB�G-��:9���A^����a� ���$e��+/
,��T[���o*��������C�if@w�)����P6f����8.`�i�
��%s�
fy�;]�D)��7�A� �I7��j�5���+��Xt~' ��@l�������@�9^�7c�l��Z	^���J����bn��V��V<L��Re�^]Bc�b�N�2�)p&[)l4�$M����X����-���Nq��>�,��
������h0�^�5�=����;�"�<��$%�+<!ok����e:�� �M)���'���	Y6	k���(�
o�p�6q@i,$qb~���^����l����������;�����P�_����p�f��'d�f����<��$����������z���U;����N���S���S(Z��4��Q��O�Y�W�
RA|���7����05��5|��Y�2[^�����Y�1xA��`w��&�R��@Bs��C������\@'q ���������	9/B�
��4_zk
���������3h��S|)�*��	g�V������I���z�B��c�].���;|w�����5���+ R������M�=��GSt/�[}2'���~�u�B>�NO$"�b6,���ZTi�J��Ww��u�$Y�k���^��96i��h�������P��)R�Z�l������P�?��������v�^Lv1=�Z��R������_wX�/��~/Y�)?`�Z�y�N�<A�\��I��8_�B���.�DO���RZ� �n6��|^xW�8�n)����7�}6|������������&�C�5����
��@:��^���������"�QX�`?4�j������Q�[\��r9sr4�X����Y�l:�u����A��lmHz���Sd)�x�������
��(�'��d�!�9FgI�����D���Gd�Dt[8`	�?�O���D+����k���Uy"�5���qBo�Z���n�'(����!(����6�-�-�n�Y�����(P0W0-�5��m�D@h���TU�H(�W����	��!ZhS���#:�K��<��r��V ����nU����.�����:V �%W�m�4���Ql��(�����[i��T�aHu-�yT��7`,�����.-u���7�5l��d*�O�yN��$8EC��/*d�g��V�ME'���
��#`��!}��1#Q�Y����qsDl�pj�3�x0t�[d����=S�$&
�&C5���*;@$7��7����Y�Yo���8)\%:��E�\%�:��N*�m`"�$��������)(�s�KJ�������x0�&��b���F^9��!e�5�!AD�K�'���nq�AL���J�	�{�VY���=d�/�Yt���7W����������P�t�Q^cRgqL��/����InKu"
��^���c��q8�0s��d���������p���1)p �w:T�1�a��%�T�;�\�o-�h�y0���B����V��HN��BOv�lYl�*0�gw���Ve���0��a]a;�����1�@B C���]�J��(o�xW@�+�"�N�N���)�m;������<���p{���UfReP%_6h����U���#�������r:�X\r�g�Ol���\�5��?���Fx�v�[���"��c��f/~�^x����]2;���w������O0r��'t������u�������Gy���}��I���X�N���������M:��l:���+�����d�os���;���n4c��x��k/��E�f]c^�;?0����%�Gp�������j�]n�dt4�K�����9���u���le?pb|NKgjF�Y.��-�I�f����Z���>�������a��g�8���0I�2�H�i���,�(���/���?h���P� �� CH��C=6��xx9(=����~i�
���/>�}|�%|f�`D�� zM���t�v�R�F�FQ��`�S���C�{}�D��4��������<�����Q���.��r	V�	s������1��X�ukU��"�ej��g�j9�����2�P'�ag��,��]N���:$'Au�� )w�����w��#���#����g���Y}�:�u�n�g=��$��`����t2T��(O���w���u��P����^�*(�0�bA�]����H:���E[�N���7���/�Z��I/���A}����#�<)���O;��_��/)/$0���p��R|V_��D����7������'�G����������H�����W�)���^��\������h]��@3�A<%��
���?�Uc1J�\7�6�[�N�d7ej�U�Y_�v���������~-��)��V���q�iQ)�
E�6^���])������������/�;�,�Cj��2����������H�N�	��
:�#�1%.P���S�c�*E'�<������W�1�@�����c������,�|�[�����7�@��Z�1f0�%���^�������zY2e�klooVC�����#W}�L�crsqV(���]H^���������*�pl�K��	jr��s�8)�#-e��c�{x]�~�7)����p�t������M��yE��0�F���t�,�3�P��7+��#j���p3���0g�T���4����	�&�����&�����G=�h��4t���P/�N�B��K��H�,����|l��j�-PA}M#u
>��k	��P;xIe��v>FD�9t��i.��@�fp(�#����zqg���t������J���k����\������Je(�-+q�V�P�b�/������0����O.>�������.���"!!rH����N1����]^������5�[��A;�Z� �
$R����]��t
elJ(�6�K��g85��Wr�"u��d�t�	c���y8
�/	s+�����Q��gF~������\��	��Bs|d�������:<�FF�y�����Mu
�����i{�;�]	�4g�
%�}*jj1����y���Y�\�$�v�f��S��,���-���XLm��/�����"V���H\&�����V�����5��k��������i{at-�
���c���mY�]/�<��~��2x���&,�?,^�J��t�P�
w.pA�0���>z���w�c�S��"�7�����������Y������W�T���~V#g�/_I�9���xt�5�Q�2�RU�'����V���\��}D��Hg@9o�D��e��t]��RP�]lEr ��_#o����C�>��j���|�]�u������K��M}$8M�h�B���uM��8�B� �2�#��:D�hz�X���

Z	]��&Ig���]�?�H�g�\ko�A���<�rP����>_N�<?���|$�n�~�s����0.�'��M��D�y��s��N�m��������3��u@��	�K���|]�A�^<������/z^���p0_����z%�*��]�$O���a%�hwtN?-
@��@~A�_J01��P4�A~�gnF�`����/d&7��#�0
jY��q1��`��c��X�y�]
t��g�����_%7�Q)��%��B�������r>�^���S�$
8E*���=���G��[�+�<�5-ib(y.���!!Y�u���{ ���0����,51)��-���M@����t�]���w�v�j�]��&��bs& ')��H�c����[�/���c	�C��2�
��A����=���Cm�Z�<S��4Et��%u�(sb�0J�%�S��Q�,����8��JJ��F�!q���`sf0�E`)r�*��nf���iCOc�R�'���J�bN)��~K�gI��y��������m����+}
�9�)�����������*�I��^�%�����w�;�}��4N�����b��3�3�=��Nf��T�ma�x4����7����9�8
�(��5�3��H�5���#��;al��u.��"j
��Z�-����H�;}��P^D���QlP���]]��(8������o���R0������K���#��z���,�e���!���b�>{tu}~��	s]���?Q0�������Q���stt��s;�6��5&{R�����Dg����%�\8I��Q��A�#3����������$�<W��BpbX��'�|j�&����f��at
u5�����@>��C�z�w`�J��YI�N�E�����//���������V�
�s)�3�8��������T�T*���������&2!�}�$=��s+P
�R��1��9��d9x��2~tk��+c���v�#�lN��'M�\/�T#�"��H��������s�JM��J�L�F�����*�U��fx�����)J�����8����}�ia��@��q}���B�H�1�#��A�?]
�{H��	E�2�O+������'zf��(o��k����#+q��B����Q�fM!JQZ���En-q��W;7qQ8(4�Z��D�t l
�2P��$Xd� �=�f���J��j��oK�,����Xg(���EP�/R����C�9�|�54)a���>��:���O������5Qf�I�jf
t-0b��v�3l5����C��h����9�;���%��=J@L%;/bO�T�&�����������n6�G�K���nir��{B	�$C���r���k7T[s��I���JNi�l���:)��
�D.j9�i�4q�E1�����&f��XLf�q��1��O)���6���};���g�s�j�k�\�������?�o��W-���q�l��)S�T�����.q^�s�`����&H�=�=F�m���|��o��T�~9�!k����}�6���t���{���(���`$R�����F���5����a�����lx���2����mL�Q��^����/(O�u�A��g��m�;����\v�x��E����m:t�hz��M.W.����;=�"�.m�M�50I�*��9;��x�E.OL&y�!�>���N������&����ww�/����o�D�V�p���B�@�)?O���l�������Q�<l4�A��j�;^�Ri5�{{{���],s~��[��Rl������.�eDL�^��;^��i��*���(-o��_N���p�O���C0
�w`�~�W�5-5J�������-y7��>��}-����G��=�y�;��]�^���{�g���yyF��~	�|W_+u�������{��z���Q�+��*.l����Qt����&xB`T������}��;��^8�N>L�U��ar�{<8�(����=�Q��%�f����@y���p�5��hH
L�������z,~���84J'O�ft����:��3\��l���c�(���`Dp����\G�j[�x����)0���8���j,!�#��9�k���)�8>���fJ/��X�E��p���HF��������_�@��
7��y*$���p|��&\�%�I�A
������-�#�-��@���<��2�s��{�/1�!ws�GzF�?d��{��s���(����c�NL�do����(b
��!R�������Y"X<�z�4�dZ�������3O�+��|�������9�&��������c5�=���s���',
�,@�����S���?�A��@���f�)�:`�w#�J�dL�fnFc����L(f��xr7�rr�S^#I|x����t�|���!���]0���D<�Z�z��J:�BhA��������`r�Cv.f�}��I���Ps���$�p����S��������������`��(�L�3��bO7�����Xp/�X�
:��%c��F��On�5RZ���t�X��Iy�Rbk&�����h�*RjD���p��{�����;�r"����@��9����7�8�jrt
�T���sK=�'�(�ep���x+�e�W����C���`�2usp���b��4&Q:z��S!����F�n�!8�D�����7�_�9������{gg�������d_�5^�f3w�?�=d������-j	:o����8���cg���w���V�|d� ����U�o��	�/
���>�i��P��h�wv���A:{�8���
��u������Q�Q�Y7�wgo>�$���h��[�03���4�s�Bk%��K��(�
KD��!@��1��La-�{��m���)��M���aY����YI]�p��3 �]h�Q/�-D�� �gK�yI��E�B��,������P��������������{% X��s|yj8�a]+�(��(z����,,v��<�I��+��4�^Z��������GL"%#��kRc0'�ml9w$�2���!�oIWPQ`��B5"1�V�(1���VH'� X�C��#���DJ� �([��8�s�_��#�H�r����������)G(��#V���#�k�G�$G�����;8y�1�XA"���q���B���v���[o��S��2Yf^�X)��n���������Wx��<����=bz�`3����g�R�y���.(huT�-�`+CN:�@�%%�}�$���M�#%A��)T$���"�`*���$���2�~��^�������e�����?����v/��8y�"JgdVS��_~K-�Y�$����q�$�[pML��(��K:c,n�<��9	<��|��R4���0&q�����V��K��8Z�/���q<�I4�N�Z�$	AI�t����#����{��wF�U�
���}����d4i�Y�T�}��UV����C�������
p�Xa�$������X
zIf�+R�I*Yz>4�8�0��k���l��/R�*KY��!("N����������D���G��a'9���n^Q�r�pd��������i��<	�����6����d�V!X�z�����������	%
������;<W�C��Q$(�L]����������>^�����Z��
���f+`��p�am��>���������lH��9��p�=��Id�LRrG�$c�Pe��C@/[*�#���48��`��z����w��������?�"�i=�b@�_���P��+���2�@�
z���H�_�N�S��.L���A�]����m3��S�!j{�s����g9������r�����_��k@�S;z�^(��p�
��G�\�!��_��U_d��6�,���R=����c3��R`;�[0;B��<j�v���pr�M2B�����,_����T��}�R�C85z����R�	!��|&r�(c��0����Q����'����s �(��	PW��/0����;�����c�uE�3�W���2A����9
z)�t}VF �.��e��LE���Q����B���/�v,!����HK��L���
e#c���VDW�ih�g7�I(?��)���6���2�~����)��k1�cv�S�l�f�{J�KZ���P�Pf�d��
[�tV>���l���gJj�G��07������p��SR�|N��U���g������l:��R��,M���c7vS��|���A�����k� I�[�5!��]��H�	
5��h*e}^x��RZ�lJ�������2��@A��T
�J�������?�T�z4�����^z��g���������Gshi�����K/��%m����8/=k�;�3���Of|&�s��[��)9�����x��6�)��&����+-�U��� �4M���G�=�
�=��/6%�`D�HU�FW�����������dpe�Z��i|�/��O����v��$O�D���]�u6De>��I�u�z�X��"�PA��>�_�;����I�D)mF\P[��V.���*/E��%�����z������^��?�]���u������������&x9~� ���#��.M`r8l�4���M���u�������x%�*A�#�~$��c"9�Y�6:�H]��f$N7���N��}W�9�M�����>��Afr$�E�H%��C�������F�����u�����	�6�&�r��J�}.����d�6L�P��*e�,7���c�8�|zR��^��z�o.��)�����������������������V�m���6�K�Q!����<�$����S��7�$j4jW`|����V~�#,�Pz��j:%�2�I����NIb)1���^��?�C�����`S2L�����b	��il��2�Z�F�HH�b}H�'�9�e�K�I�}����P9*�F��7k�?J*6*K]?R,bc+�q
d\t��Z��x�\���U-U������B��������L�w�����K�!w������g�&1�B<���=�BjNFe	?�*�k
�����M���J�v�$T-���q�*�������~1Y.!"����Hw�`��"�[�������G|)j<
��Rp����x�"�.T�&,&�uL�����!-�4
���!��{������5Hr�w'o$���BBL����%j�<B������9C��7���I�x$Z*x����M�����Mb��d��z&_��bX���AKrt)(��un]��P�t6Ek�-	��Zr�K�z����i����m����c�{O@�z�@�k��'#���U&A(1)}�=�}O-m�y<�u"����R�+6����l!T
�9�)�h��&������+\�T,"�������	���yx�*�@X3��8�@-W������k�-�Z�Jt�������D�C�=_-q�7�
=�l���}6����oC���Y5�WkZ���U���a8�3�S)����N������)ERZ��� �z��V�K���L�r��A#�z ]b��?�
�h���"OKRz�.�	�=�9"]g��^����0���{�jTJ��Wlw��z'� x�:��jsu�������rz�����d�>����@o�����Z��l�
S-���)�L��w9B�����9v+���J.��}�L/*J3SnR���Q���~�Y��e��rHe�(E|0c���	|,*"wX�+���X���3H������b5��xC&9�
�f�2|:pN�����p��6
�����nL��"��L9��g|�~�rhe�	�m�������8���B�!M��--a'��\I`����2IW����R&�Iz?k���;c0S�j�yI�j��K��W�4�Z��g���h�bF��*��([+0a���"���-I��q��_�"��H���"�G�qb�P*�,I^�U�:�
X����r��X�<S$��p�)��A�e�n2^��!��`�{�E@���i�ty�
�!e4��nf�>�l��`�Ap��2�s�k����V.�d~Yt��:�An'��A~�e�:Arp7�!L5�0�\��'�(h�9�����:�p?bPo�1�lH-�y^�S��#����"f�#!���5:-��:��RBh��3`�/1H��&���[>�$Q�$���)?�,�f,J��n�hfN#]��?o���c�JU���������3��.����o����X����x�`�F<�sL�	L��:3$uWP>,Q�����mL�#�r_���goz+;�Z�����	m@)�5��;��_���������)��Jl�F�Z<�6������jG��Z�h-�'���I$��������6��"����S�cd�FC�I�bd���n��f�JD����%��]J�n�x`�2��O�e���`+��1B����t�o�]����GW��<|��eI9K��U�������8�~�Yj
��Z�7K�	[�z�-9}��1��=���])_�~:�R#�A��)H��h���)�:!k(��/;��L+������������R��-���:4��tG����%�+0cb��VL��.��D�(���(�)0���
(�3'Ge�'SH���x)��R(�$/��Sh~
����+#��:���O�d���I��
�����W�����n���U���.���-t//�/�&W�P����AF��CJ-(��'J�E7�gD�p���!)t��4�\�8U����?�=�a�_�M�YQ�QJ���o��i���$HN�(�N��cF���POg�����9-:u�,�@���g�Lh��|�G�NQ�}�h����B��>�
�.�z����4i���e�]��M9��B�*����#�xr���L+�&�2���b����n�Qc�0[�0h�S�m]�,8������-6�l��<6Re��������7A-k�{���38�d�$hm��������p����+8�57�r�������G�1toL�����J�����V�~E�d��f];]T-��'�v��l�r�,!O��D��c�Q7vJqVH�2m��{��9U{�<GSn�q�����A�G�>�R��-;������H�M(��I�
����v*�����#��2k	��eh�bsfx���FHy�����j��.�*R����k��8�W�JW*tD�*��8K� ��l����<�u�1N��~�!��`,;��4��00��'���T���IVS������=U����}�l��^$�G����f�M�J���
+����<�@V��Ex���G{@i���6=���Zn��;0��@�~��������"~|{�y��WP�B����x��Ru3�]���do�1L���E�2�xs:�7����$l���~���@����_%:;���������}#s5�,�S�Y�`�t��[���C���U����a$�B�=A��)�]��E%����i�(����l�e���,��1�M�1�vB?0���pl8G�yv�D��F�t������f��wm%���I�ag-�k��hs���j9
��tM5�@rD�!����l$+�����+���Q�����Q-�Z��B�3�����a7F!������l5�����iG*$�	���_Td]F���I
f��/
v����<o�d���8;8������(0)�K,��o,&t�/}�������}	�E������{���A����>x��;���
���`9G��LF���_���)�7���Ft�$��6���>Pw2����I���S�l���Y�������<�~�&l��Y0����n��	��L.�"�����!j0�"�k6�"����Y��u'���i�Gm�~b�-�p$p���t@GQ5��hV�����ri�an�,mYR3p�`3;�,����B��=�+v���X�d��[Ri��h�������V��O�������	���GWw�E��%���;Q���V6f��B�J�]-��xdl]K(O��e�B�|;�������$e�������(����*od����x�S��x��\x�)�r����7�l����3D�g�����U�&���U�=M�*a7��jrl3��G��;:���u/�'��w�5��m�I0X�8���;e~���tg�8��&�7����lX�2S�>�t�vr������7B���1�Qe8~���[>���l�^�m����8��>TN%{Zd�}����2bH	�A��H��m�>�	�P"�0�d�tOA�����^��K�������|���N8�pC�Bg`�����
i�S���82��IK\r0)���Oa���&�E���q���Mf�tg�����z����{����������%O�n��J�:w�����o��w��m���O����7{�Ue���p��%Q<�������<�?�)�S�46��EiE�+(�@��I%��?�!�#�������"��;�D
�?�����0#����n�,�`2�,��
")���%�<��H�R�s�P�'KH����U�k�����
��C�p}��!�'���E������b�Z��-�N�H��<�^�U�{�#��r�;I-�G�CYI�|�>�K]D���e��W��%*�n!<�@�ro'�s[�����~��+Tgk|V+���fdI�I�.�����\�q�4w�#����	Zs����%�<w���0Y�I�| t���i�������V+�����������G�8|��]�|��g���^G/���R[G��q���.n\�j#�
����7��j�1����K��L�,�0��&��o�q�C�}p�^R��]�������_�W������W���It5uc���8���fz3q�M(�%YB�IQ���/�g�����v��g-�~�k0h�G�j'hT��r�n�AgX
����P�E�;/�E�/���_-!Ao64@�d��!����]w����b�����{)�����i�JP�W������J����3�O�b8�k\'�V�x/��`���+���X�a0l��A�-a���f9g��MKEpM�XL�e��$u��e�����I���9g��'�g��N6�s�D�x12�@��?1���x�+���n6���\n���vm��������br����+�
o�Q����5�����>%WG���I���������;�'�H���_]��2k����z~��<�s(VO�)�?��2�7U]!�KLsG�����u���
/#�[����������0�����G���xyl>�!��g��Z����W����wb�`�����+�k�Z�T�����JU��d
{��S�G�>[��`��\W@�;���������O�Q�a���9!V-����D����!�~u!�G��@I%~Br>�^N�x���hd��d��<���\07�Z���{����G��|#�O��\d��>�y�'���o��YZK��Od�bZi�'V���c����4��S1�`��Y��7;V6����&d��&����H�"�93m��yaIYh�N<���B�^-����K����3�0!7k�6�q�!�OI�j�E/��f�����[�,�=L-�����
	�tS1M�J�����2�P�)NDB%���K���S��A���!�`CzH(w�J�LL%�U|{5�����������-{��hy�%D~�Uql-\ti�b���`i%4"�2��$2�V	
����!�=�u��X�@A�#��rx�����'�������s!���n�.�K�zBH����|�v�>��FT�����TSw!�F�S�c��Q����'�.9��Z�I*�a�0B�i������=�W���������{v���w�hu"��b
����XS��^���6jCk��6��<�F�i�9���tN$�z�/)u����!����B�'�d�[��Cb�K#/a)C%tp��|��G>��a��J����!��(K�*��yH��[�f����HY.�J�l�=Ax[������mU������x���W���=����P;���w!JI�Sj>-7��z����7x&�o�
1�m%\Qv��i'_��N~�AV����q���3�����Q��>~��3���0��x�����W�G>`}
��� ��'�<:u'GL�[�8$��p��t�2?8Dx?`�^L+��Y�"�� �E���Kq|.��>n�C#�i��x�X��b������B�-�d.�5����(0������g8�E"��^����a�{+]O)M��_�L�#�f�<&gv��m�x���D��V>"���[��d�b�Q����"�h��T&X�p��U7�8��9+��Za�������'S-��@Y�(�1G���c��sv�Y77MA3>�w��������B�����J�������Ux����"��Z3ei��eQ�WbIW)�	�GP��g����;��T������{��\{����x�J����f��FS�. �ed��	�K�J'W�!!�
	Ny���7���p�'�[��Q*�cC�i)���	��1�o�C�x���~����!�,�����8Q&C��^M�#��"��������s�V�&�]W��������
C���%�P�}h�4VJ��
k�,���4����9�Ju��b�IS�,h�D�XG�]x[��-W�������W_@�����m!c��c`����f-������h/�*����5i�����<z�����9��x��:��f6b�W4`j�D���;��F���W*��c�A�W��v������4"8�@^�����>L����g2k��y6������n,����C�����&nI��L�����v���ag\.�:�J�^[��&n���&n�,�����������
vfZ�M���*OP��.���^W���It19��0|�G�|��Q�>h�E8E�s4BZ:��h�������~�Uo�����N�6Vs�4�=u�t�l�������
���a���Q~�������+%�7����Z������q�sz0rR�s�����lU ������w���z?��k� ��M|�+}$�i������i�7��3�/��`)6�����(��/�7��V`�3r����C�d<��dK�HD����H��	��8qA��d���.`���O��3l@��_Y��y�*�<'�6�^+�W�z�{#��S7g��M��sQ���&k�fnF���j852I��S��}�PR��	ja��d��_q��{�`��P����}H�f�����;����V���a������oBQ�����������������������	d
��t��\���6�k�w�:�8����c�'�k%�_�`
L,��P�t
��L'���RO$t^B0��
+;���;���]���wx��1W[���c��U�HOd�a�F��<�XlH����'�g�n�lv9�k�3m>��������SEUZ��_!���67#�O�<��j�8�`�Q�}��wp�q-$�e��i�oV91��&��|�'�5#�)���h���?� ��j
e�R��6��rZ���Q�S������<F���
z��6�f�b	�T�&������T&��l�':�T�
��� a�#%m�F+�=�6�N]�eq7!5m��nVbD�ex������ AC����
)�,~R�n�XW�N�g���j�
�V�)�!VY(��]@���� �-���ab$X>i���u�a�}����Cwy�lM��[� H�	�����/^&�J���|�G@z	����40�2q�<���dp�'Q�H�0� ,*-P���wP^�M��`���Y��'�`GY�J��X��I�%��i;��E�B�$(�����}H�N�����n^�a���|�D28\�����m�
)h��5b�&m�&�����/ �U+K�V��M
�R�-i��N�
�A�*QN�8E�Y!�8�<
�,\�?��e7J��:���8���2�2�(���i� �^�������lE]�bj9u�$?�Q����?�U>���R�"N�v��.l���1e	
te��q,���"[z���3�K~b��4br�O�����B!6�a�p���g��|��&��6�R��I����:�e�v�u��l\,11k>��!����K��8�T*�q��{�m0��%*@��^K���8�gjg��@|�t/�c���U�M���E��(�u�&d���|d@���Fj�����8����1�(;J#!�Hnn�F<%�*Oe\���/��2���Y���|�Qk���V����r���w:�V�6r]o$����H�N�Z\r�oBvK�Ei7��F|������q��\%�F% ���5���V)y��o3���O���Ul]Z���o������1}k�����_�.��B��%��h�C��a{
����0���G����sa3��������Eo�A��"�
���
�1�)��&�U�s���r2��c��&VN��V��?��[�r�Q����J0n���3��vL�,� U+�$~���.X��PA��;�� ��+Z�����F��������3T�d#)~C��G�rhC������D>�e��Y���������"|�b��I4����r����B��w��B%������-���|�����Z�� yks�I��_|��^�=a��O��D��8�S�M�]����f�
{�h�������[4)��v�I�2,�����z�����|����tS3avr�Y�����<`��vA��XkpzKce�M�'�xy@��*d,V|�X�S�O���</�1�?����9x�GK8�o �3�;%�k����^�
���r�S�k�V�2��sMW[I~�*��;�v
]w�7�}�{����Sc2��+�e����G�&�1PI)���}x��{�5�#�\�gB�k��Kx (��l�<��l1����{gg���_��������o��I��|�;=�l��d�5w|�=��j��|��.�����ef����l��U�zM����������7�W����\������*�����_8>�^������i�'��^&aq�'���i���m�����J����;����A3/AZ��/���2=�o�A�3� ��������f;�e^�0�H4������{����?9�>t����f2��W�!�p��5���H�&x1��Y���|\��r���*�oW�B�B-n���f#h7��R+wk���]�f�+��2��`p�6�������B`���:���
�Q�#�C��F�zAC�]�-y����|��_���QHG���4�>�/��%\�	�v���X5?���a3������W�`$�7����A!k*/��)�����N:-����D�7�a����%
������$V��zc�_m�3��DCI�I�(�����L(����B�On�;{���Gt)��tY��7��p'G����?�8�>0��?��J�l�o�x��g%���q����J�_�u/�*�
H��$d*B���������n�<J�Sd��3EE���.����#:1$C �zJ"�-eg�z��t8l��{���!c�\H%oO�����c���ao�C��"���O�!�I|<s��A�����"�����9--�#"��i�]�7F���b��M���-���v%��z}��;%��dL��-e�����Q)W*�����^�R$�#4q`���������9�X�b��R*���1��Z����r�q?
��$�G��Q�a��������`�4^%���_���z���+����i��v����ry��o4��Zc\KRKzK�:��������&0Q�<X��l�p;����8�r�'�`k+��b�a������E�(s���!m��� `�H�i���M�y8�t���~�~�vqr�VB��9po�5	V����H�Z1ClXl��J��,t�e���cHi7\���?qN��.b[��Wm��[��F�E35O�:�7�6������B|})��?���i,~vJ^�L[�O�����k�<�$g������G=7�=f4�\�v��d�B�U�q�@��J������ ��'1��F����7�����)O��M*o/^H;^b�:JHy�V����?yg0�*��������JnUW)rPD��V�ZE�m�|���b!,R����Qo)F7�-��K:s��|n`O;�����������2��-
ej3�MPFa�!8��p���q&���H����
yo+�����0��P��M`�4��]�����T���%�	gR����,��U����L�?jP2�/:.c'��[
`��d���PC��=+c-������
���"X��O{������� [4��X�D'9/z(�]/U��}�zl
f7�+;��(����7�������o��';;�|\����g�2YNH���B1\���5q!I �O�f�Y8K:�.��,;��L����-����#�`v�srM/�����}�#OR����2]�=rj����R��7b: ���>�� 8k9<��~v��^H����H��Rb�	\xVyp�!`��`��]C��`�KH�����H�M�rc��%�m�o�y$�z!��<��$����gt���w��`8	�	I��]o�K}�p-���X�6)�v==�q�z��d*n������h*�-�N���D�����h��lu��g�����4D��xE��dXI
P
m���sl��JpKG����^���)�S`|w��������Z�+cN�����3�X���<<}����Le�
�XC'�9b�{B3Y�!�	k�#��F�??Al&lkj0!��?��(�2)�b�3|�y�<E�k�B�%N��
��bs����!6b�2���b��c���sW���j���Xy�*��;��i0����4�6�H+I�������W�<,�aQ4v�%=����%��x8
h�
�u,��]rP2���f�
%�D���8,���AQ���N�?���|xyy�W�F;�^���%�p��'*$!0e��gIZ����Y+5�����=<�����:�N"�7@~n��8
��Ib��P�����������������^x
S#&�x�o-�&<�8'��%@!���uK���k9k�@���f��.WiV	qzp������#�,�&
'�Bo�����IKgm<&|�:��|!�S�H !�p�_c&N����!u�-qO����(B���BJ�O��Z?��r����b,���{��l��:0��xti�Oi�_�����)�<
4���0�FHOl};�F�3R��8�
m�=w
���t�m��;j%��T5����C�_�-��N��l����VUg:.l��z����:'�g��Lx7�l)�|4h��D�H��m��$~;���{��A���5h h��2q�]��x���`��m0�[m ��l���A�����pw�*Of�?6��03��-�00�	/T���B�>�.i/��[���%[�/�%��X��������R�UM��k��1�YR�s7a�\#���X���w���6���R������������P;K���t���(��nt�'-��e�f2���n%�b��*$��{�Z��.��>�_��p?JZ�xS���1p����'�mIj�IU�|ykt_b��?�����~�k�+B~^B�${D,lx+��� t�2�h�4��A0?[M�������	�C�����k]����S�$��L���Ik�Z?c�����ml�WC94�oR�3X:�9e�.w�~��[���rO�������Lw#���5��T�Vg�����
��& kU	�v�e�f.�d���
Hy�R	��

$�k.������
���6���G�et�7����i(~0�o���������v�u��`�	�H������
���Cbq�j������^�����B���� l���?��.7O�A�zJ�+��=�����@+�j=r&���q��'��1��)ty��1�B�hV��#efG�)��1��)`�1�)�$����r~"X�e���7�;�]"��R� �N'�t����&��Yg����Qf��|���<���Y c�$��-&D�PH��L�s�����.N{g
�tDqiDM�o��Y#R�dnv����c%����
QH�sq��J���}-�cI�Mm"�Fx��#�p�x�!���+�?zO�����c�0��b�"��-����0=�@_v��"5���J�I�^�(�,�N���A����&
�`5���8rZ\o��x��'�������`v��~�����7�rn`W�3�R}��3��@�[�'0�&���1d[��zL�*���%���x�'��Z,������/�w������)��I�LM2�J��J ���H~���(��f�u�RM���W2�>�%,�'�nM�1:�;nX��[��Ba�ci!�:�1`?A�k���iK��A��`[@���*�"����x�C�}3"��p�5�����X�������������}�aV+��a
T�M��>��
H�s(PwPN��H�U�=���&�������7�P�I�r��&Zv������h5����&�!&>t��H
�<��s����urRU	=�ZJ	sLu')(B0K5��l'�,�xFf;[�|	�|Rv�v�Z�q�v�7Ts6d�s��������gl��.`j����
��N7
h�e�v��^���+�w�]}}
��v�s�Ew������L���r@�������������+�,e�[��p�$
�H���f2��X���2Kq\FrO��ddH�]���8
39�%}�Rf]��y�p���t�$]�]��~�l�����Y.���W��j�9���pM2J1loa{���s���@!��f~\L�s��r��(o���Jfi�����}�������
kS��Oz���C5e�bv7H�(�(��k����

v�0
qu1�kz����(�yJU9��(��4S�T�������8MM�4�h����=����F_��Ox}�#��NF�<j�X�-|��l��+!�*���#���p���I�Td[�A�������BZ���J�Pb����+���R��Vv!9F0}�iz,��,�L<8���|$7Q�<'*������5���.��u�����q�����?V��<>���r0W�������Z����3�`���@�������e���S��r�kv���N�����,�5�|��f�@���d�VK�y������r�����8u6��,��(�x=e�sj{��<���Q��w�+9���>��K������������[N�d\Ir���h5~53�.��|��i�K�SyY7�Z��J?
�R�Q��h����T�_������b����^e���k����\�=��N�oU���1�/����q�3�4+`����\5Y�X��D��@p�	����9"]�3�+���|3�S��?�%�;�;��f&�p��3�B�`�\Lnn�����B�d��(��~Muz��P��X9iv}>����?9��{���y�n������\c��������F
xt��Y��u�8�*��+`f����������������O���|.�m���,\v�4����r�s0?����Q�!+)�����*������0&u���+�K����eL���3�.�\�QE�e2vA�F�+\���q)��:��G�n���P�=F\�#�1�pDn���[zB�yw�����r�J�����k�zu��v��Fg��6F�����l)�3�e���-H
V�_���D�3�:�Q��$PPt4 YD��e���|�@7�|��(k46�����/���p������������n�12��MC��!�.���K��4��/�G��B&F�vu��%w��kBb;O �%\;�$�O-�|_1_�&#����1��199�(@R%�B�>�
y��nUe�(�������#�=5��!d���_�����pi������8��'7J��9Q"�4���y�`*����{D6�L)� Vv�;)�io&$��HT%�t�\r���������/����p �2�������Y�z[.A o����,������5�&9,�e������������pup�3���*T�q7����������jUj��]�VG�ul�h(�kE��m���w���6��A��ebBu�%2�����I�@u!
���3|c�A�F�dz
f�Z�N�v��<�D�^���J����-Ft�*6f�!"�WN��F~#�P
��������:(�t��NU��{J��jN<#]�N���&[�q0�+�`?(�4n��WA�e>�P�&Ep5P��%��$(������E������c��dQ�N��X�R�f�@*C<���\� ^�	���7��^u��{��~"J�Qx�Z��n�����0Pfm��B�B�\r2�����i��������Y@� ,
o��@),�����|�5r�V�3�;����D��$��.
�"3Q���mx������s�x������Y�*������9q��Eq;S�|��`M�e3s{%�J��/8����,���9���,��N�
?	��,���#s�5S��."����q���
���x,x\��j
*y���X�3��m�H�f�����(��"�� ��Wm�S/!k�
n�t����>�/� ������iA>��G���9jT�F�\n�������WXr��k�^�L^!h��AJ�������p��{����8*�^__v��u7���>�eH2u�d�'�;�0�zgx:�_��M�Kt�r�SN��q���2��"��t��(�N�&�d�kv�I{�5G6�cl���Y�9	�8~sx� ��������g��{(�����k�S�A�3T*~�/����~3h6G�a��J4��[�"d�$�U[��-���R��(�E�(q��	��������gY�;=?�V�9;��~sq�=�{/S������J������>	�h�����-�
��E�A���!����B�c���uZ�r�3�����`����iN������l�W���<_��
�<��x���-A�������`�F
L�G��_��*�Kf	Y �
��W�B1���{0�h>�"*�������xX�.�y'�
���k�E�U�7��I�^�!>�O���l��q��{����r���������=����X>!�Q`�@��
Z`TUluY�6��{���@�<��C7w��_Eb��!bGCl&\-t�01��Dm6k�,a,@aw��)� �!��_��&��v\�X����pi/�m��5jL��T}�=o�=z.������	���R  j������AV�]����yoW���lW+�z��&�����B���Y��:s�C�#���jr�����b���XX�����W������Fp[��.�}��������Y��=���/��[����.e���3.�st��8��/e�T���2�&��[m_�������2������EY���d��
�x��*z��� E�A��=���H�fi��ZI&	�L�p�������&��vK6Al9/��V�ok������*��V�Sa��Z����i����n|�L{�����?���$���k�VD��~P��o����j%S{?#�"��`��R�V�Z���s �v�dO��e�k��?[*��L��w��N}���_]���S��7@��>�'�>��GbB�]	5����L@VO?y�����
���f'?�������c���kE�q����bM���nB����EO��$����<J��Kt���D���fFaa�`:�\��2�������Gl�7 �
��\y2��D�<���n-�T�D����x��*�zn� n�����
>�'��J��������j��D���:=Z�_��X3k������g�w`$��b-��8(`O]�L�&6,����D0k�uP���;D�G������m
�KH�Nvy���m��*��<g{r��������������[������o6�fr�h<]CU��yD�"NjM���ZG��&������Ig[�78�sI�#Y�y��T���6'�'���Gq�_�q'������:e7�e���n �����_���O�*�8��������V���~5�P(9�Bx���g�LA�Q����������x�:x�q��r����p�U"�gMZ���[�&�����9��It� ��0�&�Q�v.R��QS�L������O��E��i�"�����'[�l�s3��w�3���'���B}|����!3��>�Prg9Y'A�2���_NvU���V���<[�Z��8��D��o�����LE�'��	�
�����_`e����/���Co��j��$.>�OG�
�^<
�N#�$�n$8�hw7�;�q�t`�H�K����|�}<�BvVNjU������sJ>���c��^BD���V��Z�<j���`?T���[-���[e��� 
N���s3�>qG��'�j^��j���\��i8.>������]�7���^���HtIX�N����������(sSg��mv:|W��Sh�w��{�=Q3�?*����4�`ti��l8A���5/���}�~A��'h���K��y=A�
_�O�[���� �f�>�;�������&oG���DB�	���37���m��4����K��h�h�Y�� ��>����]�z�O��'K�)���������2n���z����>�|�����������?�>�|�����������?�>�|���������s�(
#18Robert Haas
robertmhaas@gmail.com
In reply to: Thomas Munro (#17)
Re: Parallel Hash take II

On Thu, Sep 14, 2017 at 10:01 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

3. Gather Merge and Parallel Hash Join may have a deadlock problem.
Since Gather Merge needs to block waiting for tuples, but workers wait
for all participants (including the leader) to reach barriers. TPCH
Q18 (with a certain set of indexes and settings, YMMV) has Gather
Merge over Sort over Parallel Hash Join, and although it usually runs
successfully I have observed one deadlock. Ouch. This seems to be a
more fundamental problem than the blocked TupleQueue scenario. Not
sure what to do about that.

Thomas and I spent about an hour and a half brainstorming about this
just now. Parallel query doesn't really have a documented deadlock
avoidance strategy, yet all committed and proposed patches other than
this one manage to avoid deadlock. This one has had a number of
problems crop up in this area, so it struck me that it might be
violating a rule which every other patch was following. I struggled
for a bit and finally managed to articulate what I think the
deadlock-avoidance rule is that is generally followed by other
committed and proposed patches:

<rule>
Once you enter a state in which other participants might wait for you,
you must exit that state before doing anything that might wait for
another participant.
</rule>

From this, it's easy to see that the waits-for graph can't contain any
cycles: if every parallel query node obeys the above rule, then a
given node can have in-arcs or out-arcs, but not both. I also believe
it to be the case that every existing node follows this rule. For
instance, Gather and Gather Merge wait for workers, but they aren't at
that point doing anything that can make the workers wait for them.
Parallel Bitmap Heap Scan waits for the leader to finish building the
bitmap, but that leader never waits for anyone else while building the
bitmap. Parallel Index(-Only) Scan waits for the process advancing
the scan to reach the next page, but that process never waits for any
other while so doing. Other types of parallel nodes -- including the
proposed Parallel Append node, which is an interesting case because
like Parallel Hash it appears in the "middle" of the parallel portion
of the plan tree rather than the root like Gather or the leaves like a
parallel scan -- don't wait at all, except for short
spinlock-protected or LWLock-protected critical sections during which
they surely don't go into any sort of long-term wait (which would be
unacceptable for other reasons anyway).

Parallel hash violates this rule only in the case of a multi-batch
hash join, and for only one reason: to avoid blowing out work_mem.
Since, consistent with resource management decisions elsewhere, each
participant is entitled to an amount of memory equal to work_mem, the
shared hash table can and does use up to (participants * work_mem),
which means that we must wait for everybody to be done with the hash
table for batch N before building the hash table for batch N+1. More
properly, if the hash table for the current batch happens to be
smaller than the absolute maximum amount of memory we can use, we can
build the hash table for the next batch up to the point where all the
memory is used, but must then pause and wait for the old hash table to
go away before continuing. But that means that the process for which
we are waiting violated the rule mentioned above: by not being done
with the memory, it's making other processes wait, and by returning a
tuple, it's allowing other parts of the executor to do arbitrary
computations which can themselves wait. So, kaboom.

One simple and stupid way to avoid this deadlock is to reduce the
memory budget for the shared hash table to work_mem and remove the
barriers that prevent more than one such hash table from existing at a
time. In the worst case, we still use (participants * work_mem),
frequently we'll use less, but there are no longer any waits for
processes that might not even be running the parallel has node
(ignoring the moment the problem of right and full parallel hash
joins, which might need more thought). So no deadlock.

We can do better. First, as long as nbatches == 1, we can use a hash
table of up to size (participants * work_mem); if we have to switch to
multiple batches, then just increase the number of batches enough that
the current memory usage drops below work_mem. Second, following an
idea originally by Ashutosh Bapat whose relevance to this issue Thomas
Munro realized during our discussion, we can make all the batches
small enough to fit in work_mem (rather than participants * work_mem
as the current patch does) and spread them across the workers (in the
style of Parallel Append, including potentially deploying multiple
workers against the same batch if there are fewer batches than
workers). Then, single-batch parallel hash joins use the maximum
allowable memory always, and multi-batch parallel hash joins use the
maximum allowable memory after the first batch. Not perfect, but not
bad, and definitely better than deadlocking. Further refinements
might be possible.

If we don't adopt some approach along these lines, then I think we've
got to articulate some alternative deadlock-avoidance rule and make
sure every parallel query facility follows it. I welcome ideas on
that front, but I don't think the rule mentioned above is a bad one,
and I'd obviously like to minimize the amount of rework that we need
to do. Assuming we do settle on the above rule, it clearly needs to
be documented someplace -- not sure of the place. I think that it
doesn't belong in README.parallel because it's an executor-specific
rule, not necessarily a general rule to which other users of
parallelism must adhere; they can choose their own strategies.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

In reply to: Robert Haas (#18)
Re: Parallel Hash take II

On Mon, Sep 18, 2017 at 1:06 PM, Robert Haas <robertmhaas@gmail.com> wrote:

If we don't adopt some approach along these lines, then I think we've
got to articulate some alternative deadlock-avoidance rule and make
sure every parallel query facility follows it. I welcome ideas on
that front, but I don't think the rule mentioned above is a bad one,
and I'd obviously like to minimize the amount of rework that we need
to do. Assuming we do settle on the above rule, it clearly needs to
be documented someplace -- not sure of the place. I think that it
doesn't belong in README.parallel because it's an executor-specific
rule, not necessarily a general rule to which other users of
parallelism must adhere; they can choose their own strategies.

+1

Graefe's "Query Evaluation Techniques for Large Databases" has several
pages on deadlock avoidance strategies. It was written almost 25 years
ago, but still has some good insights IMV (you'll recall that Graefe
is the author of the Volcano paper; this reference paper seems like
his follow-up). Apparently, deadlock avoidance strategy becomes
important for parallel sort with partitioning. You may be able to get
some ideas from there. And even if you don't, his handling of the
topic is very deliberate and high level, which suggests that ours
should be, too.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#20Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Peter Geoghegan (#19)
Re: Parallel Hash take II

On Thu, Sep 21, 2017 at 5:49 AM, Peter Geoghegan <pg@bowt.ie> wrote:

Graefe's "Query Evaluation Techniques for Large Databases" has several
pages on deadlock avoidance strategies. It was written almost 25 years
ago, but still has some good insights IMV (you'll recall that Graefe
is the author of the Volcano paper; this reference paper seems like
his follow-up). Apparently, deadlock avoidance strategy becomes
important for parallel sort with partitioning. You may be able to get
some ideas from there. And even if you don't, his handling of the
topic is very deliberate and high level, which suggests that ours
should be, too.

Very interesting and certainly relevant (the parts I've read so far),
though we don't have multiple consumers. Multiplexing one thread so
that it is both a consumer and a producer is an extra twist though.

--
Thomas Munro
http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#21Rushabh Lathia
rushabh.lathia@gmail.com
In reply to: Thomas Munro (#20)
Re: Parallel Hash take II

v20 patch set (I was trying 0008, 0009 patch) not getting cleanly apply on
latest commit also getting compilation error due to refactor in below
commit.

commit 0c5803b450e0cc29b3527df3f352e6f18a038cc6
Author: Peter Eisentraut <peter_e@gmx.net>
Date: Sat Sep 23 09:49:22 2017 -0400

Refactor new file permission handling

On Mon, Sep 25, 2017 at 11:38 AM, Thomas Munro <
thomas.munro@enterprisedb.com> wrote:

On Thu, Sep 21, 2017 at 5:49 AM, Peter Geoghegan <pg@bowt.ie> wrote:

Graefe's "Query Evaluation Techniques for Large Databases" has several
pages on deadlock avoidance strategies. It was written almost 25 years
ago, but still has some good insights IMV (you'll recall that Graefe
is the author of the Volcano paper; this reference paper seems like
his follow-up). Apparently, deadlock avoidance strategy becomes
important for parallel sort with partitioning. You may be able to get
some ideas from there. And even if you don't, his handling of the
topic is very deliberate and high level, which suggests that ours
should be, too.

Very interesting and certainly relevant (the parts I've read so far),
though we don't have multiple consumers. Multiplexing one thread so
that it is both a consumer and a producer is an extra twist though.

--
Thomas Munro
http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

--
Rushabh Lathia

#22Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Rushabh Lathia (#21)
2 attachment(s)
Re: Parallel Hash take II

On Thu, Oct 5, 2017 at 7:07 PM, Rushabh Lathia <rushabh.lathia@gmail.com> wrote:

v20 patch set (I was trying 0008, 0009 patch) not getting cleanly apply on
latest commit also getting compilation error due to refactor in below
commit.

commit 0c5803b450e0cc29b3527df3f352e6f18a038cc6

Hi Rushabh

I am about to post a new patch set that fixes the deadlock problem,
but in the meantime here is a rebase of those two patches (numbers
changed to 0006 + 0007). In the next version I think I should
probably remove that 'stripe' concept. The idea was to spread
temporary files over the available temporary tablespaces, but it's a
terrible API, since you have to promise to use the same stripe number
when opening the same name later... Maybe I should use a hash of the
name for that instead. Thoughts?

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

0006-Remove-BufFile-s-isTemp-flag.patchapplication/octet-stream; name=0006-Remove-BufFile-s-isTemp-flag.patchDownload
From 1902f6534b8be678ea74c9ef80239f8ad58feace Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Thu, 17 Aug 2017 23:03:49 +1200
Subject: [PATCH 6/9] Remove BufFile's isTemp flag.

The isTemp flag controls whether buffile.c chops BufFile data up into 1GB
segments on disk.  Since it was badly named and always true, get rid of it.

Author: Thomas Munro (based on suggestion by Peter Geoghegan)
Reviewed-By: Peter Geoghegan, Andres Freund
Discussion: https://postgr.es/m/CAH2-Wz%3D%2B9Rfqh5UdvdW9rGezdhrMGGH-JL1X9FXXVZdeeGeOJA%40mail.gmail.com
---
 src/backend/storage/file/buffile.c | 43 ++++++++++++++++----------------------
 1 file changed, 18 insertions(+), 25 deletions(-)

diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 4ca0ea4f2a3..a053ca459c2 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -68,7 +68,6 @@ struct BufFile
 	 * avoid making redundant FileSeek calls.
 	 */
 
-	bool		isTemp;			/* can only add files if this is TRUE */
 	bool		isInterXact;	/* keep open over transactions? */
 	bool		dirty;			/* does buffer need to be written? */
 
@@ -99,7 +98,7 @@ static int	BufFileFlush(BufFile *file);
 
 /*
  * Create a BufFile given the first underlying physical file.
- * NOTE: caller must set isTemp and isInterXact if appropriate.
+ * NOTE: caller must set isInterXact if appropriate.
  */
 static BufFile *
 makeBufFile(File firstfile)
@@ -111,7 +110,6 @@ makeBufFile(File firstfile)
 	file->files[0] = firstfile;
 	file->offsets = (off_t *) palloc(sizeof(off_t));
 	file->offsets[0] = 0L;
-	file->isTemp = false;
 	file->isInterXact = false;
 	file->dirty = false;
 	file->resowner = CurrentResourceOwner;
@@ -136,7 +134,6 @@ extendBufFile(BufFile *file)
 	oldowner = CurrentResourceOwner;
 	CurrentResourceOwner = file->resowner;
 
-	Assert(file->isTemp);
 	pfile = OpenTemporaryFile(file->isInterXact);
 	Assert(pfile >= 0);
 
@@ -173,7 +170,6 @@ BufFileCreateTemp(bool interXact)
 	Assert(pfile >= 0);
 
 	file = makeBufFile(pfile);
-	file->isTemp = true;
 	file->isInterXact = interXact;
 
 	return file;
@@ -287,10 +283,12 @@ BufFileDumpBuffer(BufFile *file)
 	 */
 	while (wpos < file->nbytes)
 	{
+		off_t		availbytes;
+
 		/*
 		 * Advance to next component file if necessary and possible.
 		 */
-		if (file->curOffset >= MAX_PHYSICAL_FILESIZE && file->isTemp)
+		if (file->curOffset >= MAX_PHYSICAL_FILESIZE)
 		{
 			while (file->curFile + 1 >= file->numFiles)
 				extendBufFile(file);
@@ -303,13 +301,10 @@ BufFileDumpBuffer(BufFile *file)
 		 * write as much as asked...
 		 */
 		bytestowrite = file->nbytes - wpos;
-		if (file->isTemp)
-		{
-			off_t		availbytes = MAX_PHYSICAL_FILESIZE - file->curOffset;
+		availbytes = MAX_PHYSICAL_FILESIZE - file->curOffset;
 
-			if ((off_t) bytestowrite > availbytes)
-				bytestowrite = (int) availbytes;
-		}
+		if ((off_t) bytestowrite > availbytes)
+			bytestowrite = (int) availbytes;
 
 		/*
 		 * May need to reposition physical file.
@@ -542,20 +537,18 @@ BufFileSeek(BufFile *file, int fileno, off_t offset, int whence)
 	 * above flush could have created a new segment, so checking sooner would
 	 * not work (at least not with this code).
 	 */
-	if (file->isTemp)
+
+	/* convert seek to "start of next seg" to "end of last seg" */
+	if (newFile == file->numFiles && newOffset == 0)
 	{
-		/* convert seek to "start of next seg" to "end of last seg" */
-		if (newFile == file->numFiles && newOffset == 0)
-		{
-			newFile--;
-			newOffset = MAX_PHYSICAL_FILESIZE;
-		}
-		while (newOffset > MAX_PHYSICAL_FILESIZE)
-		{
-			if (++newFile >= file->numFiles)
-				return EOF;
-			newOffset -= MAX_PHYSICAL_FILESIZE;
-		}
+		newFile--;
+		newOffset = MAX_PHYSICAL_FILESIZE;
+	}
+	while (newOffset > MAX_PHYSICAL_FILESIZE)
+	{
+		if (++newFile >= file->numFiles)
+			return EOF;
+		newOffset -= MAX_PHYSICAL_FILESIZE;
 	}
 	if (newFile >= file->numFiles)
 		return EOF;
-- 
2.14.1

0007-Add-BufFileSet-for-sharing-temporary-files-between-b.patchapplication/octet-stream; name=0007-Add-BufFileSet-for-sharing-temporary-files-between-b.patchDownload
From 601065cf3a2f54fefba9472e53207a181afb168b Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Thu, 17 Aug 2017 23:15:09 +1200
Subject: [PATCH 7/9] Add BufFileSet for sharing temporary files between
 backends.

BufFileSet allows temporary files to be created by one backend and
then exported for read-only access by other backends, with clean-up
managed by reference counting associated with a DSM segment.  This includes
changes to fd.c and buffile.c to support new kinds of temporary file.
A BufFileSet is backed by a directory full of temporary files.

Since File closing and unlinking are now disconnected, with the former
performed in every backends and the latter only in the last to detach,
there was an order-of-operations problem in resowner.c: DSM detach callbacks
are invoked before closing File handles.  Move DSM detach handling to the very
last moment in resowner's cleanup.  Without this reordering, unlinking would
risk failing on Windows where open files prevent that.

Introduce a new reusable function pair for deduplicating sorted arrays as
infrastructure.

Author: Thomas Munro
Reviewed-By: Peter Geoghegan, Andres Freund, Robert Haas
Discussion: https://postgr.es/m/CAEepm=2W=cOkiZxcg6qiFQP-dHUe09aqTrEMM7yJDrHMhDv_RA@mail.gmail.com
Discussion: https://postgr.es/m/CAH2-WznJ_UgLux=_jTgCQ4yFz0iBntudsNKa1we3kN1BAG=88w@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm%3D2vmFTNpAmwbGGD2WaryM6T3hSDVKQPfUwjdD_5XY6vAA%40mail.gmail.com
---
 src/backend/storage/file/buffile.c    | 478 +++++++++++++++++++++++++++++++++-
 src/backend/storage/file/fd.c         | 281 ++++++++++++++++++--
 src/backend/utils/resowner/resowner.c |  20 +-
 src/include/lib/arrayutils.h          |  68 +++++
 src/include/storage/buffile.h         |  13 +-
 src/include/storage/fd.h              |  10 +-
 6 files changed, 829 insertions(+), 41 deletions(-)

diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index a053ca459c2..ce6a24d2221 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -31,16 +31,31 @@
  * BufFile also supports temporary files that exceed the OS file size limit
  * (by opening multiple fd.c temporary files).  This is an essential feature
  * for sorts and hashjoins on large amounts of data.
- *-------------------------------------------------------------------------
+ *
+ * BufFile supports temporary files that can be made read-only and shared with
+ * other backends, as infrastructure for parallel execution.  Such files need
+ * to be created as a member of a BufFileSet that all participants are
+ * attached to.  The BufFileSet mechanism provides a namespace so that files
+ * can be discovered by name, and a shared ownership semantics so that shared
+ * files survive until the last user detaches.
+ *
+ * -------------------------------------------------------------------------
  */
 
 #include "postgres.h"
 
+#include "catalog/catalog.h"
+#include "catalog/pg_tablespace.h"
+#include "commands/tablespace.h"
 #include "executor/instrument.h"
+#include "lib/arrayutils.h"
+#include "miscadmin.h"
 #include "pgstat.h"
 #include "storage/fd.h"
 #include "storage/buffile.h"
 #include "storage/buf_internals.h"
+#include "storage/spin.h"
+#include "utils/builtins.h"
 #include "utils/resowner.h"
 
 /*
@@ -51,6 +66,29 @@
 #define MAX_PHYSICAL_FILESIZE	0x40000000
 #define BUFFILE_SEG_SIZE		(MAX_PHYSICAL_FILESIZE / BLCKSZ)
 
+/*
+ * A private type used internally by buffile.c.
+ */
+typedef struct BufFileSetDesc
+{
+	pid_t	creator_pid;		/* PID of the creating process */
+	int		set_number;			/* per-PID identifier for a set of files */
+} BufFileSetDesc;
+
+/*
+ * A set of BufFiles that can be shared with other backends.
+ */
+struct BufFileSet
+{
+	slock_t	mutex;				/* mutex protecting the reference count */
+	int		refcnt;				/* number of attached backends */
+	BufFileSetDesc descriptor;	/* descriptor identifying directory */
+	int		nstripes;			/* number of stripes */
+
+	/* an array of tablespaces indexed by stripe number */
+	Oid		stripe_tablespaces[FLEXIBLE_ARRAY_MEMBER];
+};
+
 /*
  * This data structure represents a buffered file that consists of one or
  * more physical files (each accessed through a virtual file descriptor
@@ -70,6 +108,16 @@ struct BufFile
 
 	bool		isInterXact;	/* keep open over transactions? */
 	bool		dirty;			/* does buffer need to be written? */
+	bool		readOnly;		/* has the file been set to read only? */
+
+	/*
+	 * Shared BufFiles use deterministic paths for their segment files.  We
+	 * need to record enough information to construct path names for new
+	 * segments when extending the file.
+	 */
+	BufFileSetDesc set_desc;	/* info used to construct directory name */
+	Oid			tablespace;		/* tablespace for this shared BufFile */
+	char		name[MAXPGPATH];		/* name of this BufFile within set */
 
 	/*
 	 * resowner is the ResourceOwner to use for underlying temp files.  (We
@@ -94,6 +142,10 @@ static void extendBufFile(BufFile *file);
 static void BufFileLoadBuffer(BufFile *file);
 static void BufFileDumpBuffer(BufFile *file);
 static int	BufFileFlush(BufFile *file);
+static File MakeSharedSegment(Oid tablespace,
+							  const BufFileSetDesc *desc,
+							  const char *name,
+							  int segment_number);
 
 
 /*
@@ -112,12 +164,19 @@ makeBufFile(File firstfile)
 	file->offsets[0] = 0L;
 	file->isInterXact = false;
 	file->dirty = false;
+	file->readOnly = false;
 	file->resowner = CurrentResourceOwner;
 	file->curFile = 0;
 	file->curOffset = 0L;
 	file->pos = 0;
 	file->nbytes = 0;
 
+	/* Clear the members used only for shared BufFiles. */
+	file->set_desc.creator_pid = InvalidPid;
+	file->set_desc.set_number = 0;
+	file->tablespace = InvalidOid;
+	file->name[0] = '\0';
+
 	return file;
 }
 
@@ -134,7 +193,12 @@ extendBufFile(BufFile *file)
 	oldowner = CurrentResourceOwner;
 	CurrentResourceOwner = file->resowner;
 
-	pfile = OpenTemporaryFile(file->isInterXact);
+	if (file->set_desc.creator_pid == InvalidPid)
+		pfile = OpenTemporaryFile(file->isInterXact);
+	else
+		pfile = MakeSharedSegment(file->tablespace, &file->set_desc,
+								  file->name, file->numFiles);
+
 	Assert(pfile >= 0);
 
 	CurrentResourceOwner = oldowner;
@@ -190,6 +254,414 @@ BufFileCreate(File file)
 }
 #endif
 
+/*
+ * Build the path of the temp directory in a given tablespace.
+ */
+static void
+MakeTempDirectoryPath(char *path, Oid tablespace)
+{
+	if (tablespace == InvalidOid ||
+		tablespace == DEFAULTTABLESPACE_OID ||
+		tablespace == GLOBALTABLESPACE_OID)
+		snprintf(path, MAXPGPATH, "base/%s", PG_TEMP_FILES_DIR);
+	else
+	{
+		snprintf(path, MAXPGPATH, "pg_tblspc/%u/%s/%s",
+				 tablespace, TABLESPACE_VERSION_DIRECTORY,
+				 PG_TEMP_FILES_DIR);
+	}
+}
+
+/*
+ * Build the path for the directory holding the files backing a BufFileSet in
+ * a given tablespace.
+ */
+static void
+MakeBufFileSetPath(char *path, Oid tablespace, const BufFileSetDesc *desc)
+{
+	char tempdirpath[MAXPGPATH];
+
+	MakeTempDirectoryPath(tempdirpath, tablespace);
+	snprintf(path, MAXPGPATH, "%s/%s%d.%d" PG_TEMP_FILE_SET_DIR_SUFFIX,
+			 tempdirpath, PG_TEMP_FILE_PREFIX,
+			 desc->creator_pid, desc->set_number);
+}
+
+/*
+ * Build the path of a single segment file for a shared BufFile.
+ */
+static void
+MakeSharedSegmentPath(char *path,
+					  Oid tablespace,
+					  const BufFileSetDesc *desc,
+					  const char *name,
+					  int segment_number)
+{
+	char setdirpath[MAXPGPATH];
+
+	MakeBufFileSetPath(setdirpath, tablespace, desc);
+	snprintf(path, MAXPGPATH, "%s/" PG_TEMP_FILE_PREFIX ".%s.%d", setdirpath, name, segment_number);
+}
+
+/*
+ * Create a new segment file backing a shared BufFile.
+ */
+static File
+MakeSharedSegment(Oid tablespace,
+				  const BufFileSetDesc *desc,
+				  const char *name,
+				  int segment_number)
+{
+	File		file;
+	char		segmentpath[MAXPGPATH];
+
+	MakeSharedSegmentPath(segmentpath, tablespace, desc, name, segment_number);
+	file = PathNameCreateTemporaryFile(segmentpath);
+	if (file <= 0)
+		elog(ERROR, "could not create temporary file \"%s\": %m",
+			 segmentpath);
+
+	return file;
+}
+
+/*
+ * Callback function that will be invoked when this backend detaches from a
+ * DSM segment holding a BufFileSet that it has creaed or attached to.  If we
+ * are the last to detach, then try to remove the directory holding this set,
+ * and everything in it.  We can't raise an error, because this runs in error
+ * cleanup paths.
+ */
+static void
+BufFileSetOnDetach(dsm_segment *segment, Datum datum)
+{
+	bool unlink_all = false;
+	BufFileSet *set = (BufFileSet *) DatumGetPointer(datum);
+
+	SpinLockAcquire(&set->mutex);
+	Assert(set->refcnt > 0);
+	if (--set->refcnt == 0)
+		unlink_all = true;
+	SpinLockRelease(&set->mutex);
+
+	/*
+	 * If we are the last to detach, we delete the directory in all referenced
+	 * tablespaces.  Note that we are still actually attached for the rest of
+	 * this function so we can safely access its data.
+	 */
+	if (unlink_all)
+	{
+		size_t num_oids;
+		Oid *oids;
+		char dirpath[MAXPGPATH];
+		int i;
+
+		/* Find the set of unique tablespace OIDs. */
+		oids = (Oid *) palloc(sizeof(Oid) * set->nstripes);
+		for (i = 0; i < set->nstripes; ++i)
+			oids[i] = set->stripe_tablespaces[i];
+		qsort(oids, set->nstripes, sizeof(Oid), oid_cmp);
+		num_oids = qunique(oids, set->nstripes, sizeof(Oid), oid_cmp);
+
+		/*
+		 * Delete the directory we created in each tablespace.  Can't fail,
+		 * but can generate LOG message on IO error.
+		 */
+		for (i = 0; i < num_oids; ++i)
+		{
+			MakeBufFileSetPath(dirpath, oids[i], &set->descriptor);
+			PathNameDeleteTemporaryDirRecursively(dirpath);
+		}
+
+		pfree(oids);
+	}
+}
+
+/*
+ * Determine the size that a BufFileSet with the given number of stripes would
+ * occupy.
+ */
+size_t
+BufFileSetEstimate(int stripes)
+{
+	return offsetof(BufFileSet, stripe_tablespaces) + sizeof(Oid) * stripes;
+}
+
+/*
+ * Create a set of named BufFiles that can be opened for read-only access by
+ * other backends, in the shared memory pointed to by 'set'.  The provided
+ * memory area must have enough space for the number of bytes estimated by
+ * BufFileSetEstimate(stripes).  Other backends must attach to it before
+ * accessing it.  Associate this set of BufFiles with 'seg'.  The set of files
+ * will be deleted when no backends are attached.
+ *
+ * Files will be physically striped over the tablespace configured in
+ * temp_tablespaces.  It is up to client code to determine how files should be
+ * mapped to stripes, but the ParallelWorkerNumber of the backend writing a
+ * file is one straightforward way.  While creating the set, 'stripes' should
+ * be set to the number of stripes that will be used to created and open
+ * files.  If it is set to 3, then BufFileSetCreateTemporaryFile can be called
+ * with values 0, 1 or 2.  Stripe numbers will be mapped to the configured
+ * tablespaces in a round-robin fashion.  'stripes' must be at least 1.
+ *
+ * Under the covers the set is one or more directories which will eventually
+ * be deleted when there are no backends attached.
+ */
+void
+BufFileSetCreate(BufFileSet *set, dsm_segment *seg, int stripes)
+{
+	size_t num_oids;
+	Oid *oids;
+	int i;
+	static int counter = 0;
+
+	if (stripes < 1)
+		elog(ERROR, "cannot create BufFileSet with fewer than 1 stripe");
+
+	SpinLockInit(&set->mutex);
+	set->refcnt = 1;
+	set->descriptor.creator_pid = MyProcPid;
+	set->nstripes = stripes;
+
+	/*
+	 * The name of the directory will consist of the creator's PID and a
+	 * counter that starts at startup for each process.  That's not ideal,
+	 * because a BufFileSet could in theory live longer than the creator, and
+	 * the operating system will eventually create a process with the same PID
+	 * again.  We may want to use a shmem counter instead in future.
+	 */
+	++counter;
+	if (counter == INT_MAX)
+		counter = 1;
+	set->descriptor.set_number = counter;
+
+	/* Map each stripe to a tablespace round-robin. */
+	PrepareTempTablespaces();
+	oids = (Oid *) palloc(sizeof(Oid) * stripes);
+	for (i = 0; i < stripes; ++i)
+		oids[i] = set->stripe_tablespaces[i] = GetNextTempTableSpace();
+
+	/* Find the set of unique tablespace OIDs. */
+	qsort(oids, stripes, sizeof(Oid), oid_cmp);
+	num_oids = qunique(oids, stripes, sizeof(Oid), oid_cmp);
+
+	/* Create the set's directory in every tablespace OID. */
+	for (i = 0; i < num_oids; ++i)
+	{
+		char tempdirpath[MAXPGPATH];
+		char setdirpath[MAXPGPATH];
+
+		/*
+		 * We need to compute both the top-level temporary directory's path,
+		 * and the directory for our set within it.  That's because fd.c needs
+		 * to be able to create the former on demand if creating the latter
+		 * fails.
+		 */
+		MakeTempDirectoryPath(tempdirpath, oids[i]);
+		MakeBufFileSetPath(setdirpath, oids[i], &set->descriptor);
+		PathNameCreateTemporaryDir(tempdirpath, setdirpath);
+	}
+
+	pfree(oids);
+
+	/* Register our cleanup callback. */
+	on_dsm_detach(seg, BufFileSetOnDetach, PointerGetDatum(set));
+}
+
+/*
+ * Attach to a set of named BufFiles that was created with BufFileSetCreate.
+ */
+void
+BufFileSetAttach(BufFileSet *set, dsm_segment *seg)
+{
+	bool success;
+
+	SpinLockAcquire(&set->mutex);
+	if (set->refcnt == 0)
+		success = false;
+	else
+	{
+		++set->refcnt;
+		success = true;
+	}
+	SpinLockRelease(&set->mutex);
+
+	if (!success)
+		ereport(ERROR,
+				(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+				 errmsg("could not attach to a BufFileSet that is alreayd destroyed")));
+
+	/* Register our cleanup callback. */
+	on_dsm_detach(seg, BufFileSetOnDetach, PointerGetDatum(set));
+}
+
+/*
+ * Create a BufFile that can be discovered and opened read-only by other
+ * backends that are also attached to 'set' using the same 'name' and 'stripe'
+ * values.  Even though both 'name' and 'stripe' must be provided and the same
+ * pair must be used to open the file from another backend, 'name' alone must
+ * be unique within the set.  The caller must have either created or attached
+ * to 'set'.
+ */
+BufFile *
+BufFileCreateShared(const BufFileSet *set, const char *name, int stripe)
+{
+	BufFile    *file;
+
+	if (stripe < 0 || stripe >= set->nstripes)
+		elog(ERROR, "stripe number out of range");
+
+	file = (BufFile *) palloc(sizeof(BufFile));
+	file->numFiles = 1;
+	file->files = (File *) palloc(sizeof(File));
+	file->files[0] = MakeSharedSegment(set->stripe_tablespaces[stripe],
+									   &set->descriptor, name, 0);
+	file->offsets = (off_t *) palloc(sizeof(off_t));
+	file->offsets[0] = 0L;
+	file->isInterXact = false;
+	file->dirty = false;
+	file->resowner = CurrentResourceOwner;
+	file->curFile = 0;
+	file->curOffset = 0L;
+	file->pos = 0;
+	file->nbytes = 0;
+	file->readOnly = false;
+	file->set_desc = set->descriptor;
+	file->tablespace = set->stripe_tablespaces[stripe];
+	strcpy(file->name, name);
+
+	return file;
+}
+
+/*
+ * Open a file that was previously created in another backend with
+ * BufFileCreateShared in the same BufFileSet, using the same 'name' and
+ * 'stripe' values.  The backend that created the file must have called
+ * BufFileClose() or BufFileExport() to make sure that it is ready to be
+ * opened by other backends and render it read-only.
+ *
+ * The caller must have either created or attached to 'set'.
+ */
+BufFile *
+BufFileOpenShared(const BufFileSet *set, const char *name, int stripe)
+{
+	BufFile    *file = (BufFile *) palloc(sizeof(BufFile));
+	char		path[MAXPGPATH];
+	Size		capacity = 16;
+	File	   *files = palloc(sizeof(File) * capacity);
+	int			nfiles = 0;
+
+	if (stripe < 0 || stripe >= set->nstripes)
+		elog(ERROR, "stripe number out of range");
+
+	file = (BufFile *) palloc(sizeof(BufFile));
+	files = palloc(sizeof(File) * capacity);
+
+	/*
+	 * We don't know how many segments there are, so we'll probe the
+	 * filesystem to find out.
+	 */
+	for (;;)
+	{
+		/* See if we need to expand our file segment array. */
+		if (nfiles + 1 > capacity)
+		{
+			capacity *= 2;
+			files = repalloc(files, sizeof(File) * capacity);
+		}
+		/* Try to load a segment. */
+		MakeSharedSegmentPath(path, set->stripe_tablespaces[stripe],
+							  &set->descriptor, name, nfiles);
+		files[nfiles] = PathNameOpenTemporaryFile(path);
+		if (files[nfiles] <= 0)
+			break;
+		++nfiles;
+	}
+
+	/*
+	 * If we didn't find any files at all, then no BufFile exists with this
+	 * tag.
+	 */
+	if (nfiles == 0)
+		return NULL;
+
+	file->numFiles = nfiles;
+	file->files = files;
+	file->offsets = (off_t *) palloc0(sizeof(off_t) * nfiles);
+	file->isInterXact = false;
+	file->dirty = false;
+	file->resowner = CurrentResourceOwner; /* Unused, can't extend */
+	file->curFile = 0;
+	file->curOffset = 0L;
+	file->pos = 0;
+	file->nbytes = 0;
+	file->readOnly = true; /* Can't write to files opened this way */
+
+	/* The following values aren't ever used but are set for consistency. */
+	file->set_desc = set->descriptor;
+	file->tablespace = set->stripe_tablespaces[stripe];
+	strcpy(file->name, name);
+
+	return file;
+}
+
+/*
+ * Delete a BufFile that was created by BufFileCreateShared using the given
+ * 'name' and 'stripe' values.
+ *
+ * It is not necessary to delete files explicitly with this function.  It is
+ * provided only as a way to delete files proactively, rather than waiting for
+ * the BufFileSet to be cleaned up.
+ *
+ * Only one backend should attempt to delete a given (name, stripe), and
+ * should know that it exists and has been exported with BufFileExportShared.
+ *
+ * The caller must have either created or attached to 'set'.
+ */
+void
+BufFileDeleteShared(const BufFileSet *set, const char *name, int stripe)
+{
+	char		path[MAXPGPATH];
+	int			segment = 0;
+	bool		found = false;
+
+	if (stripe < 0 || stripe >= set->nstripes)
+		elog(ERROR, "stripe number out of range");
+
+	/*
+	 * We don't know how many segments the file has.  We'll keep deleting
+	 * until we run out.  If we don't manage to find even an initial segment,
+	 * raise an error.
+	 */
+	for (;;)
+	{
+		MakeSharedSegmentPath(path, set->stripe_tablespaces[stripe],
+							  &set->descriptor, name, segment);
+		if (!PathNameDeleteTemporaryFile(path, true))
+			break;
+		found = true;
+		++segment;
+	}
+
+	if (!found)
+		elog(ERROR, "could not delete unknown shared BufFile \"%s\"", name);
+}
+
+/*
+ * BufFileExportShared --- flush and make read-only, in preparation for sharing.
+ */
+void
+BufFileExportShared(BufFile *file)
+{
+	/* Must be a file belonging to a BufFileSet. */
+	Assert(file->set_desc.creator_pid != InvalidPid);
+
+	/* It's probably a bug if someone calls this twice. */
+	Assert(!file->readOnly);
+
+	BufFileFlush(file);
+	file->readOnly = true;
+}
+
 /*
  * Close a BufFile
  *
@@ -409,6 +881,8 @@ BufFileWrite(BufFile *file, void *ptr, size_t size)
 	size_t		nwritten = 0;
 	size_t		nthistime;
 
+	Assert(!file->readOnly);
+
 	while (size > 0)
 	{
 		if (file->pos >= BLCKSZ)
diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c
index b0c174284b4..54a9061a6fb 100644
--- a/src/backend/storage/file/fd.c
+++ b/src/backend/storage/file/fd.c
@@ -39,6 +39,17 @@
  * for a long time, like relation files. It is the caller's responsibility
  * to close them, there is no automatic mechanism in fd.c for that.
  *
+ * PathNameCreateTemporaryFile, PathNameOpenTemporaryFile and
+ * PathNameDeleteTemporaryFile are used for temporary files that may be shared
+ * between backends.  A File created or opened with these functions is not
+ * automatically deleted when the file is closed, but it is automatically
+ * closed and end of transaction and counts agains the temporary file limit of
+ * the backend that created it.  Any File created this way must be explicitly
+ * deleted with PathNameDeleteTemporaryFile.  Automatic file deletion is not
+ * provided because this interface is designed for use by buffile.c and
+ * indirectly by sharedbuffile.c to implement temporary files with shared
+ * ownership and cleanup.
+ *
  * AllocateFile, AllocateDir, OpenPipeStream and OpenTransientFile are
  * wrappers around fopen(3), opendir(3), popen(3) and open(2), respectively.
  * They behave like the corresponding native functions, except that the handle
@@ -175,8 +186,9 @@ int			max_safe_fds = 32;	/* default if not changed */
 #define FilePosIsUnknown(pos) ((pos) < 0)
 
 /* these are the assigned bits in fdstate below: */
-#define FD_TEMPORARY		(1 << 0)	/* T = delete when closed */
-#define FD_XACT_TEMPORARY	(1 << 1)	/* T = delete at eoXact */
+#define FD_DELETE_AT_CLOSE	(1 << 0)	/* T = delete when closed */
+#define FD_CLOSE_AT_EOXACT	(1 << 1)	/* T = close at eoXact */
+#define FD_TEMP_FILE_LIMIT	(1 << 2)	/* T = respect temp_file_limit */
 
 typedef struct vfd
 {
@@ -326,6 +338,7 @@ static void walkdir(const char *path,
 static void pre_sync_fname(const char *fname, bool isdir, int elevel);
 #endif
 static void datadir_fsync_fname(const char *fname, bool isdir, int elevel);
+static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel);
 
 static int	fsync_fname_ext(const char *fname, bool isdir, bool ignore_perm, int elevel);
 static int	fsync_parent_path(const char *fname, int elevel);
@@ -1294,6 +1307,23 @@ FileAccess(File file)
 	return 0;
 }
 
+/*
+ * Called whenever a temporary file is deleted to report its size.
+ */
+static void
+ReportTemporaryFileUsage(const char *path, off_t size)
+{
+	pgstat_report_tempfile(size);
+
+	if (log_temp_files >= 0)
+	{
+		if ((size / 1024) >= log_temp_files)
+			ereport(LOG,
+					(errmsg("temporary file: path \"%s\", size %lu",
+							path, (unsigned long) size)));
+	}
+}
+
 /*
  *	Called when we get a shared invalidation message on some relation.
  */
@@ -1378,6 +1408,106 @@ PathNameOpenFilePerm(const char *fileName, int fileFlags, mode_t fileMode)
 	return file;
 }
 
+/*
+ * Delete a file that was created with PathNameCreateTemporaryFile.
+ * Return true if the file existed, false if didn't.
+ */
+bool
+PathNameDeleteTemporaryFile(const char *fileName, bool error_on_failure)
+{
+	struct stat filestats;
+	int			stat_errno;
+
+	/* Get the final size for pgstat reporting. */
+	if (stat(fileName, &filestats) != 0)
+		stat_errno = errno;
+	else
+		stat_errno = 0;
+
+	/*
+	 * Unlike FileClose's automatic file deletion code, we tolerate
+	 * non-existence to support BufFileDeleteShared which doesn't know how
+	 * many segments it has to delete until it runs out.
+	 */
+	if (stat_errno == ENOENT)
+		return false;
+
+	if (unlink(fileName) < 0)
+	{
+		if (errno != ENOENT)
+			elog(error_on_failure ? ERROR : LOG,
+				 "cannot unlink temporary file \"%s\": %m", fileName);
+		return false;
+	}
+
+	if (stat_errno == 0)
+		ReportTemporaryFileUsage(fileName, filestats.st_size);
+	else
+	{
+		errno = stat_errno;
+		elog(LOG, "could not stat file \"%s\": %m", fileName);
+	}
+
+	return true;
+}
+
+/*
+ * Create directory 'directory'.  If necessary, create 'basedir', which must
+ * be the directory above it.  This is designed for creating the top-level
+ * temporary directory on demand before creating a directory underneath it.
+ * If necessary, delete a pre-existing directory of the same name, on the
+ * theory that crash-restarts can leave junk behind that we'll eventually
+ * crash into (see RemovePgTempFiles).
+ */
+void
+PathNameCreateTemporaryDir(const char *basedir, const char *directory)
+{
+	if (mkdir(directory, S_IRWXU) < 0)
+	{
+		/*
+		 * Failed.  Try to create basedir first in case it's missing.
+		 * Tolerate ENOENT to close a race against another process following
+		 * the same algorithm.
+		 */
+		if (mkdir(basedir, S_IRWXU) < 0 && errno != ENOENT)
+			elog(ERROR, "cannot create temporary directory \"%s\": %m",
+				 basedir);
+
+		/* Try again. */
+		if (mkdir(directory, S_IRWXU) < 0)
+		{
+			elog(LOG, "cannot create temporary subdirectory \"%s\": %m (will try to unlink and then try again)",
+				 directory);
+			/*
+			 * Failed again.  Perhaps there is a directory in the way that is
+			 * left-over from a crash-restart?  Try to delete it.
+			 */
+			PathNameDeleteTemporaryDirRecursively(directory);
+
+			/* Now try one more time. */
+			if (mkdir(directory, S_IRWXU) < 0)
+				elog(ERROR, "cannot create temporary subdirectory \"%s\": %m",
+					 directory);
+		}
+	}
+}
+
+/*
+ * Delete a directory and everything in it.
+ */
+void
+PathNameDeleteTemporaryDirRecursively(const char *dirname)
+{
+	/*
+	 * Currently, walkdir doesn't offer a way for our passed in function to
+	 * maintain state.  Perhaps it should, so that we could tell the caller
+	 * whether this operation succeeded or failed.  Since this operation is
+	 * used in a cleanup path, we wouldn't actually behave differently: we'll
+	 * just log failures.
+	 */
+	walkdir(dirname, unlink_if_exists_fname, false, LOG);
+}
+
 /*
  * Open a temporary file that will disappear when we close it.
  *
@@ -1426,12 +1556,12 @@ OpenTemporaryFile(bool interXact)
 											 true);
 
 	/* Mark it for deletion at close */
-	VfdCache[file].fdstate |= FD_TEMPORARY;
+	VfdCache[file].fdstate |= FD_DELETE_AT_CLOSE;
 
 	/* Register it with the current resource owner */
 	if (!interXact)
 	{
-		VfdCache[file].fdstate |= FD_XACT_TEMPORARY;
+		VfdCache[file].fdstate |= FD_CLOSE_AT_EOXACT;
 
 		ResourceOwnerEnlargeFiles(CurrentResourceOwner);
 		ResourceOwnerRememberFile(CurrentResourceOwner, file);
@@ -1509,6 +1639,81 @@ OpenTemporaryFileInTablespace(Oid tblspcOid, bool rejectError)
 	return file;
 }
 
+/*
+ * Create a new file.  The directory containing it must already exist.  Files
+ * created this way are subject to temp_file_limit and are automatically
+ * closed at end of transaction, but are not automatically deleted on close
+ * because they are intended to be shared between cooperating backends.
+ */
+File
+PathNameCreateTemporaryFile(const char *tempfilepath)
+{
+	File file;
+
+	/*
+	 * Open the file.  Note: we don't use O_EXCL, in case there is an orphaned
+	 * temp file that can be reused.
+	 */
+	file = PathNameOpenFile(tempfilepath,
+							O_RDWR | O_CREAT | O_TRUNC | PG_BINARY);
+	if (file <= 0)
+		elog(ERROR, "could not create temporary file \"%s\": %m",
+			 tempfilepath);
+
+	/* Mark it for temp_file_limit accounting. */
+	VfdCache[file].fdstate |= FD_TEMP_FILE_LIMIT;
+
+	/*
+	 * We don't set FD_DELETE_AT_CLOSE for files opened this way, but we
+	 * still want to make sure they get closed at end of xact.
+	 */
+	ResourceOwnerEnlargeFiles(CurrentResourceOwner);
+	ResourceOwnerRememberFile(CurrentResourceOwner, file);
+	VfdCache[file].resowner = CurrentResourceOwner;
+
+	/* Backup mechanism for closing at end of xact. */
+	VfdCache[file].fdstate |= FD_CLOSE_AT_EOXACT;
+	have_xact_temporary_files = true;
+
+	return file;
+}
+
+/*
+ * Open a file that was created with PathNameCreateTemporaryFile in another
+ * backend.  Files opened this way don't count agains the temp_file_limit of
+ * the caller, are read-only and are automatically closed at the end of the
+ * transaction but are not deleted on close.
+ */
+File
+PathNameOpenTemporaryFile(const char *tempfilepath)
+{
+	File file;
+
+	/* We open the file read-only. */
+	file = PathNameOpenFile(tempfilepath, O_RDONLY | PG_BINARY);
+
+	/* If no such file, then we don't raise an error. */
+	if (file <= 0 && errno != ENOENT)
+		elog(ERROR, "could not open temporary file \"%s\": %m", tempfilepath);
+
+	if (file > 0)
+	{
+		/*
+		 * We don't set FD_DELETE_AT_CLOSE for files opened this way, but we
+		 * still want to make sure they get closed at end of xact.
+		 */
+		ResourceOwnerEnlargeFiles(CurrentResourceOwner);
+		ResourceOwnerRememberFile(CurrentResourceOwner, file);
+		VfdCache[file].resowner = CurrentResourceOwner;
+
+		/* Backup mechanism for closing at end of xact. */
+		VfdCache[file].fdstate |= FD_CLOSE_AT_EOXACT;
+		have_xact_temporary_files = true;
+	}
+
+	return file;
+}
+
 /*
  * close a file when done with it
  */
@@ -1537,10 +1742,17 @@ FileClose(File file)
 		Delete(file);
 	}
 
+	if (vfdP->fdstate & FD_TEMP_FILE_LIMIT)
+	{
+		/* Subtract its size from current usage (do first in case of error) */
+		temporary_files_size -= vfdP->fileSize;
+		vfdP->fileSize = 0;
+	}
+
 	/*
 	 * Delete the file if it was temporary, and make a log entry if wanted
 	 */
-	if (vfdP->fdstate & FD_TEMPORARY)
+	if (vfdP->fdstate & FD_DELETE_AT_CLOSE)
 	{
 		struct stat filestats;
 		int			stat_errno;
@@ -1552,11 +1764,8 @@ FileClose(File file)
 		 * is arranged to ensure that the worst-case consequence is failing to
 		 * emit log message(s), not failing to attempt the unlink.
 		 */
-		vfdP->fdstate &= ~FD_TEMPORARY;
+		vfdP->fdstate &= ~FD_DELETE_AT_CLOSE;
 
-		/* Subtract its size from current usage (do first in case of error) */
-		temporary_files_size -= vfdP->fileSize;
-		vfdP->fileSize = 0;
 
 		/* first try the stat() */
 		if (stat(vfdP->fileName, &filestats))
@@ -1570,18 +1779,7 @@ FileClose(File file)
 
 		/* and last report the stat results */
 		if (stat_errno == 0)
-		{
-			pgstat_report_tempfile(filestats.st_size);
-
-			if (log_temp_files >= 0)
-			{
-				if ((filestats.st_size / 1024) >= log_temp_files)
-					ereport(LOG,
-							(errmsg("temporary file: path \"%s\", size %lu",
-									vfdP->fileName,
-									(unsigned long) filestats.st_size)));
-			}
-		}
+			ReportTemporaryFileUsage(vfdP->fileName, filestats.st_size);
 		else
 		{
 			errno = stat_errno;
@@ -1755,7 +1953,7 @@ FileWrite(File file, char *buffer, int amount, uint32 wait_event_info)
 	 * message if we do that.  All current callers would just throw error
 	 * immediately anyway, so this is safe at present.
 	 */
-	if (temp_file_limit >= 0 && (vfdP->fdstate & FD_TEMPORARY))
+	if (temp_file_limit >= 0 && (vfdP->fdstate & FD_TEMP_FILE_LIMIT))
 	{
 		off_t		newPos;
 
@@ -1808,7 +2006,7 @@ retry:
 		 * get here in that state if we're not enforcing temporary_files_size,
 		 * so we don't care.
 		 */
-		if (vfdP->fdstate & FD_TEMPORARY)
+		if (vfdP->fdstate & FD_TEMP_FILE_LIMIT)
 		{
 			off_t		newPos = vfdP->seekPos;
 
@@ -1979,7 +2177,7 @@ FileTruncate(File file, off_t offset, uint32 wait_event_info)
 	if (returnCode == 0 && VfdCache[file].fileSize > offset)
 	{
 		/* adjust our state for truncation of a temp file */
-		Assert(VfdCache[file].fdstate & FD_TEMPORARY);
+		Assert(VfdCache[file].fdstate & FD_TEMP_FILE_LIMIT);
 		temporary_files_size -= VfdCache[file].fileSize - offset;
 		VfdCache[file].fileSize = offset;
 	}
@@ -2690,7 +2888,8 @@ CleanupTempFiles(bool isProcExit)
 		{
 			unsigned short fdstate = VfdCache[i].fdstate;
 
-			if ((fdstate & FD_TEMPORARY) && VfdCache[i].fileName != NULL)
+			if (((fdstate & FD_DELETE_AT_CLOSE) || (fdstate & FD_CLOSE_AT_EOXACT)) &&
+				VfdCache[i].fileName != NULL)
 			{
 				/*
 				 * If we're in the process of exiting a backend process, close
@@ -2701,7 +2900,7 @@ CleanupTempFiles(bool isProcExit)
 				 */
 				if (isProcExit)
 					FileClose(i);
-				else if (fdstate & FD_XACT_TEMPORARY)
+				else if (fdstate & FD_CLOSE_AT_EOXACT)
 				{
 					elog(WARNING,
 						 "temporary file %s not closed at end-of-transaction",
@@ -2810,7 +3009,20 @@ RemovePgTempFilesInDir(const char *tmpdirname)
 		if (strncmp(temp_de->d_name,
 					PG_TEMP_FILE_PREFIX,
 					strlen(PG_TEMP_FILE_PREFIX)) == 0)
-			unlink(rm_path);	/* note we ignore any error */
+		{
+			size_t name_len = strlen(temp_de->d_name);
+			size_t suffix_len = strlen(PG_TEMP_FILE_SET_DIR_SUFFIX);
+
+			if (name_len > suffix_len &&
+				strcmp(temp_de->d_name + name_len - suffix_len,
+					   PG_TEMP_FILE_SET_DIR_SUFFIX) == 0)
+			{
+				RemovePgTempFilesInDir(rm_path);
+				rmdir(rm_path);		/* note we ignore any error */
+			}
+			else
+				unlink(rm_path);	/* note we ignore any error */
+		}
 		else
 			elog(LOG,
 				 "unexpected file found in temporary-files directory: \"%s\"",
@@ -3146,6 +3358,21 @@ datadir_fsync_fname(const char *fname, bool isdir, int elevel)
 	fsync_fname_ext(fname, isdir, true, elevel);
 }
 
+static void
+unlink_if_exists_fname(const char *fname, bool isdir, int elevel)
+{
+	if (isdir)
+	{
+		if (rmdir(fname) != 0 && errno != ENOENT)
+			elog(elevel, "could not rmdir directory \"%s\": %m", fname);
+	}
+	else
+	{
+		/* Use PathNameDeleteTemporaryFile to report filesize */
+		PathNameDeleteTemporaryFile(fname, false);
+	}
+}
+
 /*
  * fsync_fname_ext -- Try to fsync a file or directory
  *
diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c
index bd19fad77e9..db1d688bc97 100644
--- a/src/backend/utils/resowner/resowner.c
+++ b/src/backend/utils/resowner/resowner.c
@@ -542,16 +542,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner,
 				PrintRelCacheLeakWarning(res);
 			RelationClose(res);
 		}
-
-		/* Ditto for dynamic shared memory segments */
-		while (ResourceArrayGetAny(&(owner->dsmarr), &foundres))
-		{
-			dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres);
-
-			if (isCommit)
-				PrintDSMLeakWarning(res);
-			dsm_detach(res);
-		}
 	}
 	else if (phase == RESOURCE_RELEASE_LOCKS)
 	{
@@ -668,6 +658,16 @@ ResourceOwnerReleaseInternal(ResourceOwner owner,
 				PrintFileLeakWarning(res);
 			FileClose(res);
 		}
+
+		/* Ditto for dynamic shared memory segments */
+		while (ResourceArrayGetAny(&(owner->dsmarr), &foundres))
+		{
+			dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres);
+
+			if (isCommit)
+				PrintDSMLeakWarning(res);
+			dsm_detach(res);
+		}
 	}
 
 	/* Let add-on modules get a chance too */
diff --git a/src/include/lib/arrayutils.h b/src/include/lib/arrayutils.h
new file mode 100644
index 00000000000..4a39ba74865
--- /dev/null
+++ b/src/include/lib/arrayutils.h
@@ -0,0 +1,68 @@
+/*-------------------------------------------------------------------------
+ *
+ * arrayutils.h
+ *		inline C-array utility functions
+ *
+ * Portions Copyright (c) 2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *		src/include/lib/arrayutils.h
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef ARRAYUTILS_H
+#define ARRAYUTILS_H
+
+/*
+ * Remove duplicates from a pre-sorted array, according to a user-supplied
+ * comparator.  Usually the array should have been sorted with qsort using the
+ * same arguments.  Return the new size.
+ *
+ * The name reflects the fact that the arguments are compatible with the
+ * standard qsort function.
+ */
+static inline size_t
+qunique(void *array, size_t elements, size_t width,
+		int (*compare)(const void *, const void *))
+{
+	int i, j;
+	char *bytes = array;
+
+	if (elements <= 1)
+		return elements;
+
+	for (i = 1, j = 0; i < elements; ++i)
+	{
+		if (compare(bytes + i * width, bytes + j * width) != 0)
+			memcpy(bytes + ++j * width, bytes + i * width, width);
+	}
+
+	return j + 1;
+}
+
+/*
+ * Like qunique, but takes a comparator with an extra user data argument which
+ * is passed through, for compatibility with qsort_arg.
+ */
+static inline size_t
+qunique_arg(void *array, size_t elements, size_t width,
+			int (*compare)(const void *, const void *, void *),
+			void *arg)
+{
+	int i, j;
+	char *bytes = array;
+
+	if (elements <= 1)
+		return elements;
+
+	for (i = 1, j = 0; i < elements; ++i)
+	{
+		if (compare(bytes + i * width, bytes + j * width, arg) != 0)
+			memcpy(bytes + ++j * width, bytes + i * width, width);
+	}
+
+	return j + 1;
+}
+
+#endif
diff --git a/src/include/storage/buffile.h b/src/include/storage/buffile.h
index fafcb3f0898..d6d69cd0d0c 100644
--- a/src/include/storage/buffile.h
+++ b/src/include/storage/buffile.h
@@ -26,9 +26,12 @@
 #ifndef BUFFILE_H
 #define BUFFILE_H
 
-/* BufFile is an opaque type whose details are not known outside buffile.c. */
+#include "storage/dsm.h"
+
+/* Opaque types whose details are not known outside buffile.c. */
 
 typedef struct BufFile BufFile;
+typedef struct BufFileSet BufFileSet;
 
 /*
  * prototypes for functions in buffile.c
@@ -42,4 +45,12 @@ extern int	BufFileSeek(BufFile *file, int fileno, off_t offset, int whence);
 extern void BufFileTell(BufFile *file, int *fileno, off_t *offset);
 extern int	BufFileSeekBlock(BufFile *file, long blknum);
 
+extern size_t BufFileSetEstimate(int stripes);
+extern void BufFileSetCreate(BufFileSet *set, dsm_segment *seg, int stripes);
+extern void BufFileSetAttach(BufFileSet *set, dsm_segment *seg);
+extern BufFile *BufFileCreateShared(const BufFileSet *set, const char *name, int stripe);
+extern void BufFileExportShared(BufFile *file);
+extern BufFile *BufFileOpenShared(const BufFileSet *set, const char *name, int stripe);
+extern void BufFileDeleteShared(const BufFileSet *set, const char *name, int stripe);
+
 #endif							/* BUFFILE_H */
diff --git a/src/include/storage/fd.h b/src/include/storage/fd.h
index 6ea26e63b84..26d65e97e35 100644
--- a/src/include/storage/fd.h
+++ b/src/include/storage/fd.h
@@ -79,6 +79,13 @@ extern int	FileGetRawDesc(File file);
 extern int	FileGetRawFlags(File file);
 extern mode_t FileGetRawMode(File file);
 
+/* Operations used by buffile.c for named shared temporary files */
+extern void PathNameCreateTemporaryDir(const char *basedir, const char *directory);
+extern File PathNameCreateTemporaryFile(const char *filename);
+extern File PathNameOpenTemporaryFile(const char *filename);
+extern bool PathNameDeleteTemporaryFile(const char *filename, bool error_on_failure);
+extern void PathNameDeleteTemporaryDirRecursively(const char *filename);
+
 /* Operations that allow use of regular stdio --- USE WITH CAUTION */
 extern FILE *AllocateFile(const char *name, const char *mode);
 extern int	FreeFile(FILE *file);
@@ -124,8 +131,9 @@ extern int	durable_unlink(const char *fname, int loglevel);
 extern int	durable_link_or_rename(const char *oldfile, const char *newfile, int loglevel);
 extern void SyncDataDirectory(void);
 
-/* Filename components for OpenTemporaryFile */
+/* Filename components */
 #define PG_TEMP_FILES_DIR "pgsql_tmp"
 #define PG_TEMP_FILE_PREFIX "pgsql_tmp"
+#define PG_TEMP_FILE_SET_DIR_SUFFIX ".set"
 
 #endif							/* FD_H */
-- 
2.14.1

#23Prabhat Sahu
prabhat.sahu@enterprisedb.com
In reply to: Thomas Munro (#22)
Re: Parallel Hash take II

Hi Thomas,

I was testing this feature with v20 patch, and I got a crash while doing
large joins with small work_mem, and lots of workers as below.

-- Machine Configuration: (d1.xlarge) CUPs : 8 , RAM : 16GB , SIze : 640GB

-- postgres.conf setting as below:
work_mem = *64kB*
max_parallel_workers_per_gather = *128*
max_parallel_workers = *64*
enable_mergejoin = off
enable_nestloop = off
enable_hashjoin = on
force_parallel_mode = on
seq_page_cost = 0.1
random_page_cost = 0.1
effective_cache_size = 128MB
parallel_tuple_cost = 0
parallel_setup_cost = 0
parallel_synchronization_cost = 0

-- created only one table "lineitem" of size 93GB.
postgres=# select pg_size_pretty(pg_total_relation_size('lineitem'));
pg_size_pretty
----------------
93 GB
(1 row)

[centos@centos-prabhat bin]$ vi test10.sql
explain (analyze, costs off)
select count(*) from lineitem t1 join lineitem t2 using(l_suppkey) join
lineitem t3 using(l_suppkey);
select count(*) from lineitem t1 join lineitem t2 using(l_suppkey) join
lineitem t3 using(l_suppkey);

[centos@centos-prabhat bin]$ ./psql postgres -a -f test10.sql > test10.out

[centos@centos-prabhat bin]$ vi test10.out
explain (analyze, costs off)
select count(*) from lineitem t1 join lineitem t2 using(l_suppkey) join
lineitem t3 using(l_suppkey);
psql:test10.sql:2: WARNING: terminating connection because of crash of
another server process
DETAIL: The postmaster has commanded this server process to roll back the
current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and
repeat your command.
psql:test10.sql:2: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
psql:test10.sql:2: connection to server was lost

Kindly check, if you can reproduce the same at your end.

*Thanks & Regards,*

*Prabhat Kumar Sahu*
Mob: 7758988455
Skype ID: prabhat.sahu1984

www.enterprisedb.co <http://www.enterprisedb.com/&gt;m
<http://www.enterprisedb.com/&gt;

On Thu, Oct 5, 2017 at 1:15 PM, Thomas Munro <thomas.munro@enterprisedb.com>
wrote:

Show quoted text

On Thu, Oct 5, 2017 at 7:07 PM, Rushabh Lathia <rushabh.lathia@gmail.com>
wrote:

v20 patch set (I was trying 0008, 0009 patch) not getting cleanly apply

on

latest commit also getting compilation error due to refactor in below
commit.

commit 0c5803b450e0cc29b3527df3f352e6f18a038cc6

Hi Rushabh

I am about to post a new patch set that fixes the deadlock problem,
but in the meantime here is a rebase of those two patches (numbers
changed to 0006 + 0007). In the next version I think I should
probably remove that 'stripe' concept. The idea was to spread
temporary files over the available temporary tablespaces, but it's a
terrible API, since you have to promise to use the same stripe number
when opening the same name later... Maybe I should use a hash of the
name for that instead. Thoughts?

--
Thomas Munro
http://www.enterprisedb.com

#24Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Prabhat Sahu (#23)
2 attachment(s)
Re: Parallel Hash take II

On Tue, Sep 19, 2017 at 8:06 AM, Robert Haas <robertmhaas@gmail.com> wrote:

On Thu, Sep 14, 2017 at 10:01 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

3. Gather Merge and Parallel Hash Join may have a deadlock problem.

[...]

Thomas and I spent about an hour and a half brainstorming about this
just now.

[...]

First, as long as nbatches == 1, we can use a hash
table of up to size (participants * work_mem); if we have to switch to
multiple batches, then just increase the number of batches enough that
the current memory usage drops below work_mem. Second, following an
idea originally by Ashutosh Bapat whose relevance to this issue Thomas
Munro realized during our discussion, we can make all the batches
small enough to fit in work_mem (rather than participants * work_mem
as the current patch does) and spread them across the workers (in the
style of Parallel Append, including potentially deploying multiple
workers against the same batch if there are fewer batches than
workers).

Here is an updated patch set that does that ^.

Assorted details:

1. To avoid deadlocks, we can only wait at barriers when we know that
all other attached backends have either arrived already or are
actively executing the code preceding the barrier wait, so that
progress is guaranteed. The rules is that executor nodes can remain
attached to a barrier after they've emitted a tuple, which is useful
for resource management (ie avoids inventing a separate reference
counting scheme), but must never again wait for it. With that
programming rule there can be no deadlock between executor nodes.

2. Multiple batches are processed at the same time in parallel,
rather than being processed serially. Participants try to spread
themselves out over different batches to reduce contention.

3. I no longer try to handle outer joins. I have an idea for how to
do that while adhering to the above deadlock-avoidance programming
rule, but I would like to consider that for a later patch.

4. I moved most of the parallel-aware code into ExecParallelHash*()
functions that exist alongside the private hash table versions. This
avoids uglifying the existing hash join code and introducing
conditional branches all over the place, as requested by Andres. This
made some of the earlier refactoring patches unnecessary.

5. Inner batch repartitioning, if required, is now completed up front
for Parallel Hash. Rather than waiting until we try to load hash
tables back into memory to discover that they don't fit, this version
tracks the size of hash table contents while writing the batches out.
That change has various pros and cons, but its main purpose is to
remove dependencies between batches.

6. Outer batch repartitioning is now done up front too, if it's
necessary. This removes the dependencies that exist between batch 0
and later batches, but means that outer batch 0 is now written to disk
if for multi-batch joins. I don't see any way to avoid that while
adhering to the deadlock avoidance rule stated above. If we've
already started emitting tuples for batch 0 (by returning control to
higher nodes) then we have no deadlock-free way to wait for the scan
of the outer relation to finish, so we can't safely process later
batches. Therefore we must buffer batch 0's tuples. This renders the
skew optimisation useless.

7. There is now some book-keeping state for each batch. For typical
cases the total space used is negligible but obviously you can
contrive degenerate cases where it eats a lot of memory (say by
creating a million batches, which is unlikely to work well for other
reasons). I have some ideas on reducing their size, but on the other
hand I also have some ideas on increasing it profitably... (this is
the perfect place to put the Bloom filters discussed elsewhere that
would make up for the loss of the skew optimisation, for selective
joins; a subject for another day).

8. Barrier API extended slightly. (1) BarrierWait() is renamed to
BarrierArriveAndWait(). (2) BarrierArriveAndDetach() is new. The new
function is the mechanism required to get from PHJ_BATCH_PROBING to
PHJ_BATCH_DONE without waiting, and corresponds to the operation known
as Phaser.arriveAndDeregister() in Java (and maybe also
barrier::arrive_and_drop() in the C++ concurrency TS and Clock.drop()
in X10, not sure).

9. I got rid of PARALLEL_KEY_EXECUTOR_NODE_NTH(). Previously I
wanted more than one reserved smh_toc key per executor node. Robert
didn't like that.

10. I got rid of "LeaderGate". That earlier attempt at deadlock
avoidance clearly missed the mark. (I think it probably defended
against the Gather + full TupleQueue deadlock but not the
GatherMerge-induced deadlock so it was a useless non-general
solution.)

11. The previous incarnation of SharedTuplestore had a built-in
concept of partitions, which allowed the number of partitions to be
expanded any time but only allowed one partition to be read back in at
a time. That worked for the earlier kind of sequential partition
processing but doesn't work for parallel partition processing. I
chose to get rid of the built-in partitioning concept and create
separate SharedTuplestores for each batch, since I now have a place to
store per-batch information in memory.

12. The previous incarnation of BufFileSet had a concept of "stripe"
used for distributing temporary files over tablespaces; this is now
gone from the API. Files are distributed over your configured
temp_tablespaces without help from client code.

13. For now the rescan optimisation (ie reusing the hash table) is
not enabled for Parallel Hash. I had to provide
ExecHashJoinReInitializeDSM (for 41b0dd98), but haven't figured out
how to reach it yet and didn't want to wait any longer before posting
a new patch so that function is effectively blind code and will
probably require adjustments.

I've also attached a test that shows a worker starting up and
attaching at every phase.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

parallel-hash-v21.patchset.tgzapplication/x-gzip; name=parallel-hash-v21.patchset.tgzDownload
����Y��{[G�8|�E����$#	�8�.��fx�lN=#M&�f���=����u����	�o�ox4�5}��{W����N��~�DW��v�1w��U$����R?�fs��	���g���O�#Z��������h���N��D���`��"���2[�a��4�L�<W�0����y���4Z�=��G�L�Cy�(���(������G��Hr��������������{�����������?���a0Ng0;�Q���'M���tG�Aw��-�N���v��`,��8�s��fs��m@�
��W����\J�]L5ha�"�X��P�5���Y���]q��5�����%@���n������@��/F?�q�+~8���-Z[;?
@J�H)����Rh����R�W6*��)jqy�<��J�LC���D�bt+f�+~^���Q
�JG��D��f����-@��!6�`
��Ld��0�A�H1v|�3��8p�bL�G%�Chsy]Ch�a��)n���p&g8��8L/m�L^{�F������z�^Q8���m����t�p��`�_D�������~�2(�E��/��������xS�����/�[������a;U77*\'Q�_z�p�V�h�F�w�G1����Qk��6�^����M <-�C�`+���Y��������ZGl�g�'���+@@GQ��)�-1�3�S�"�����h�"*�
�����T��'�M�V�� sDH�x��jn�Z�<����
#@�a��K�������&��b>Q�����8�4�����x��n����+������-��|��p||��0��-�8�=8����l��c���@;QeS�$���x`Hx���R�fsC��h���g&�Lc<��{p�U3e<X2�`��
l�;�����H�~D�f�G���5X��u=�������|DP�@D�-b\�v�&`��3�D4bB�<�fM���������H�?���4>D�����[�[i�E\;��b6�>�ua�'�4��+C)��S��Q����������:�p
�[�[��)&���&���p�����4f��"�k�����yH�A�p� �r�
_�E������S�&�i����4g:C��`�����$�9���0d�Y~�C��v�����9��{#q1�U8�s	�f�Z�[��F��ckA����v�q��9��l��`��9}bM��H�fPXH/Ax�qz>��W��~������#�����dX#,�6+b9���f�|��	����y�K��>((z�_t>��6��i�t�-�h�Jt�<G�f?u`>U��p�&�
k�������7x�4�d��LOc�sqE���o���G{��8��~��%H&0I����x�;�������������!�G��a5����gq�"k`�f� ��TJ��1�����������.���7D"|�t���,��6M�k/XD���*v��m�q��Y����b��'�D�h�Q�S�N�/���8��s�XY,)BN�gD��<��~�
�l�M�����XM��2\Mv���R��^�J2pMm�(������H��tZ�� ���J�	�"�	E���q����z�=.��_?�B����9?��
E9� �����r�`�N
_����Q5�����~��2P�� �%f--#6P#]k`�L�^�F��4e_.K#�& wS�����7��������M�Y=9<.1(������>��>���V�|�J-��]��4P2�8�j��y����^|��\��k��"��!����f"��#(�
�0�~��7����+9��D�GWBxKp�� �1g��U0�����HF�*�Hs!b8m30����>����H��:�P�p=WV�isJc�����
a�%�
�W�+��N�)����oM����&&���C��o�U�W����������Q�����������R��K�Rg`/�+D@X���9�b�������3��b&��o(m��O�0Z�o��)����$�� �&@�L����%GK��U���1��
F/qD�a��g��=I��)b1��C_S��7?�`��<�a����H{�~��VR�VS�y�~c�T5�A���T�(~�*[�<ru��(�h�[D++�$n�jQ b=5am,�-�4�
���I^�[���)-�����Nw�;p{�~��i4dk[N�������LA*	�L�����N����������?��j,|���l��D�IQ��=c<�P��q��5�#���H�/��G������X��G���!�������7��e��o������������~~�q?_���W�C�B`��U��b~k��.G���:d_kA�������@��o��Q����?�z�+-��e�1�)1�ap�X���?���^i7Z�F�R�w�o������������0�dX������kYY���c�{|�_@E������(�����?�A������������p���3��[n��w���h�5;;���L���;h������\\-j.����muw��%��:��#R
��$`��R��TN���s�~,�� ���{�;3o,�_�)}(=�'���`,#�*7��]�n?Q���`�*�$�H���c�_�``���@�����I����r��pb ����@<=9?|��t\6��9=�U�LF��7���X��Fzn8�[3������u�������Nx��S�.��������2/���8�G�[[�����!���������������k�N~��]?~�f�3t�9�������N������g���_f�7m\���!Ib#��[��� �ny���V0��!��m.i��1��@mV����������\����[���9]���[��J��f���4B��"pZ]M{�lJ�n;�M	� ��]�%WZ���;�(�����uv��+�Fc���v�3���K�%������]�����h1r=��JzV�4����7u�]���������v�
�~e�r��iET�'��z �h6Dq����/�����c��>�����l��v���������hD>�X���C]a��?^���_����/q���$\S{�8n�b #
S^	g��T���M��7��b��P!	�������;����-{��ph���[>��w�J�&��f�	<���/�ln=^5<i������H^�	����
�8��^��
�hH��^��o��To���N��l�-������W����`�`���x����5q&/�u���[�dy���}g���}�1�b���5����d�S�����I�/APdf�|s���"�?���b������m�����!5��H�B�7��vF#�K�����n5\4�@0@.h�g$A�%�	��C@&|��������e��6cc���E�������-W�1�EH��x0MG�����4��><�SB��������Z�������fG@<�|���{��do�h���&��8�z8VX@q��z1�S�~#G���v���9��14���8�$)���6P%�EAj5Sv�",F��#Lh-./�^`���l�e�Hr�������1�z�,�k��+�$VS�2/ p{5��c_Gph0Dh��Y ����DJsJ�8H�rPp�9Ac���d�j����U�|�����x��j�h9�B
�tt�^�-���2��l���{��^�5��d�)���M���AxoU	�����+t�!.���`ZS�f
W�c~�G�%���7m�Ld��p��@)�s7Z��p������H���Qa���
���er�3Qg��! ��G0��R��5,�7�4��<'��q" ��c�P ����c	�"���s��R��S�v(^Qg��~{��B���i�i'��'�B�3c}2o���H��C�5��?���gC�JA��eT�U������k���0~>���[+����er1�t��h���p����<�2���K��LIrBL��c�)]A�s��;�T�h��)��YjIy<�d�@9&�qW������(���l��K8��x$����R�f�5#�)��I���x,/=�iuTc	�#��Mqh4,�26�	!�9�l�z���^=>�2��!�Y�3�����m5����]���`����zb�Hp#)�d��"O�(�/���r�a=EAr*<�y��a*+q�DHQ
�<
3���|F���A(�h��:C���6��m�F�Qh�T��D�RE��)�-g��{�D���R"����<���
;%���`�)�2	�8 �1����1�TW\��$�~=5�V�}�D�wV�������o��O_���v���d���������K��/�?�,������,Xkjt��i���$�lc�����_C�x�����6�/��.��|4����U'��M^���88�8|y��wqxr����P�����[�8�om�x��5�j��_��M������|�����l�����Wv/���0��9��C���2S�+��F:Z\^�i}�v�A#F����z���
�x*=�oXIoBW@G� ~i���
��X�\�i��2D�=�*�k�C0�s�n �c��3>����!:����<��m91�e��Z��J���R�U��{r�����h!�"@zj	V$Y�w�x`9��	��)�g����X��������$	�O��
����@D\��F+��h,�������K�����"�q'��W�e}���j���*��_�����6�����!�A�p?�2bTfpc�|x�0���5��L0��vkk��U�#<���J�2�
&�d|z�������m������ �[8��:~h����mbP����~�P��<�6�� ,W�%�e����Z����Gv�qH<=*T�H�@��X7R���
�~w��EIo��!����5Z���.�M0�S'b/J����H����F�\i2��"�jJL��#����o4r�����������,���%�|��&3;��q� SBr���,�|��N�"�n�R�h�"���J�KY����e�lNz3�I��-/��mS���3*2p�n�
�O�V�K��]C6�R��@Q5�,I���7hy���r���Z�9(C4������M��1��ad��%eX�A	O�e(�������������)�D&��N!�57!�,9���y���4����%�Hp9O��yxn�}�:1���Lx5���U�B���Y�r��f!i����C\�5z%��F-)M����.�i�]B�?�h*��u��uj^�r���}��@��\�� �2��^[iNM����p>�c"6��*�����Uj��-�L��Z�q���7g���eh���P`!�k��1W�������3O���}+�����x�����!U�"�������TFB0�%������`��2�u���[���4�Z���w9H����y���n��\��K�~4�����a�^8��x��V�����U���"����BU=����En+��e���������B�����L2�!d�D�
~��B��*�S)��&rRV�
f/�5	L�|�����N�J?T����i�pB9YL����Se����J����)��)i)�cI���!�>�.�1�t+���s�!����������w�UT[P��O_
��^�_���]c�E���q��Z�%�i�7�u1�H_
���<�Y~�?��q�C-?�
�gsy�����������Y�/b69����cFO����J\L�2�$���������	!��X`�=�����r��3{����e<")�$��ddF���j������o*����d�D�p"k�~=f0l�7������n��x�\�T�=�����k�;�����Kk|�%������q�aH���<�F�P;��
��L���I���|g
�1��XZ��a�
Kedj�����/1"����p�;0�p���!h�����f��Q���G��?���[�Q>�.�0H�X�31���(�����!�����		��Tr�dF��v3��Wo��Ne���	�F���^��dZ�&���h�0h�}����ut8(�M��5
��H��H�'K���D�`
J�co�Y���@���8������(�x�w?~�=1��w�����&A����3�c�����Y�*���n��fN�0����\��vM�0��O2Y"�@z��vr�N�����;B�51{�\��H���x/��h21<��)T3b���d��~���*���0���j(�yD;��0��
����������#��Mh��W��B�XSM��@12E��T	i/wk����,��q�Zx��t=��19������&����vd�/��Z*M��6��
�������_���^r�]x'c$�E������Ud?U�Q|<��&*%KY�/���MBP��
��:7����j����:��`,��o�����O8Q9�N��":�+K���i�(F\�]Hk���Bl>�+
;���A�h�_b=��N�21f�7���qEF���z�����k�������f���<���i]���<�����(j������'X�y��X��S��E�c^Xn���D��
{�!���d�������v�����/�4lCm=�:7���~� ��:]�uM;��G�d���&�N��s�
�#	O�]��#��� �����Fs�[bE��a�]�����6�)�Q�r�3�:���x�q�P�Lc�x��f�r�@��9(�,���X
+SFO����{���s c$	J��7�n��pj1d�d)�"q� Zmm��\@�p`�:}���lW9����>����l�h�Rw�)����#K�������
>���kD��*�m�]�����DE�������S$-�
�����=F��K�1L�l�KmyJ�'��u��]a���5: 
�4;�\��+��#.��p������~��
�kO�CqX��Bq?����uP����I,$i��e�W�\�?�=�pR�]+��Z�"� <9������]y�r�zf$
�`$/(uJ����U6&��D6�������535�l5��&��{���	������kI�{ed�t3k.� ���|��4�+���"������q��d�1����)5���7���v���R3-����Ev�dF�	���3�Ep>�r~�,�?ybV���z����}��pT�M�D�)e�I��a �K!�����K�/�cL\�mp3�[/���Z4%��&B����J�\�I�-6
�f���yiy��
6�P��{��2Y�������m�\����������v^d[�C_�,���:�i��/Y#�S�>0$�9��n����S��d
��3�t�O
O��x���oh������2���:�����^�3B�U������5=z���"�3a�AA��N�[���L�����2s���+Z�j�:+1�kt������v�B�z���r��-X�)����������gJ�,�o�YVb��]��K�L�i�f�fX�qq�k���5
F�BL��Bb������Ni����F��������U��R�"�h � wI1z��2v����^�q����i���T
k59WR�c��E��*�W��Q>Z�X]�ud����MF���h��iO��q������\�O��o��b?ZM��m�L�6��l>d6&����377T��[�S�y �k�t��"��`a��H<�Q8q�h9z.f�)y��kgZ��"(��t;�~���Z����+�����]#Ce	3��c�]tN_�JnE���=:�	�#�N8���
�<��X�T��0R��J\"�7���=?���R��������b��9{�/e#���`^}����%��DS��F�G�fB��z�?<>��������}[8\z�������x�k����N����?\XV5e��Mz�����b�r|<<:��;:�a�����.%h��@!t�h�B��N����T���W��>�l�lOv��V���{�[�����~��J��Pl�?����E��.u����c���X`�+#���%!�5%���o�x����������x��_��8�y����w%0T����5���M)�Im6��W��U-=]F�s��[+phu�$��K\En��xa����o'F)�5���T@u�e��?[.�0�VmzZ���rD_��+v�%%���DTe��*���(z��[���� �k��Ba���>�W�'i��
t�?Z����
DV�������1�(�m�	(g0�H��}%'m�]��y�����E�>��q����K���.
2Y��[rOW�-h�����HX�,�DF��'�v��V��4w\��lo7�� x9��[#�"�����.Js����Z!�1��L�s�"�&�5�d�[����n)���bQvI�D�-o�����.������iu�{��o���D�\����G9~��}r7��:��l����:������7h����k�P�?�o�wv��M9�H����A����nGW:;������������%��:��O��CfH)R��R��H���u�`M�?�u������l$�������W	sT?�������f���%�����}�m�����(X�ci%w������iVbxe�}�l��Q9�*��db�PD+���*���}�����/x�!���5�r)�=�|���q|���{�������l�8?_�o�n��"|���������z;�=u��O�"��\�?K>/�����g���t��\�2	����Kz�q	f������.�������V�w������J��^��V�u.����0�Z��I'��s���Hw�����l���[�(�����+�Mwt���j�.�m�Z�K����$���;teM��>�?�4�<�V?����MC���<���D�e�t�J+%B�F��`���h������-��^�@��^�"Z�E��n�R�|q`59�H�}TbM
�*���lR<�"���V��9�����^��^�[����$eX���<��s�b|�@�\����=�,-/�N)�?��@(�7��7���FE���"�t4��5pmx�u���;�n��}v����Y);#eB��)�"��i����������_������J5#���I�,ok<8a�P��Oh��C�����g��@�*
|��&1��
e���1���j(%�M�d�y?\j�w-�b@�����`�G�}Af�u{��:����g��YcO@����d����cw,}"a|�%`�3(x�r��,`��=�
���s>8�2�����@�E�T��xd��ix����	���8�i����'L�l�Ni(�8�rx�L�M�e�(���f��C[��@5:N@n5�ME4��/dx�������>�_��P��1"�K~�����b
m����eR�3~�F���Op�������8�Jb,�G��.��"���dN���2�b^�_A��;�;::8����������'g����^$�9yM�}��Z�^����5����o�D�N�C��(���2�4����/d4f�h������aa�������M��h�8���]���_/����[�|�.�h%p������n�(X�.O�%����_Wq_���a4�$�5Q���������%�{Q[�=L�����5�A�������ZA{(��D�(l�p��j#��/F�M��d�^��)�������P�AW��wp���id���
���T�.�Z�����M��H�\�"����6j��?�����b��$iA$���@k���4��$��.�Sq&��*���s-s\EC�l�:����2V�%�������;vFr������5����3n�x��;�iG�2�M���'���''	2�j�I�����lr���%������������}od|���f���
����J��;����v�B>l�B_������IN3��8B������|QrR�O�T3�?��&�L��Ap��������.��x�
.5�N?-+������L����nn��v{����?��'N������w��tc�����������=�y�t���5��2��l|�`�m
���91F=��7��]��+M:�m$
�~�V<p��1��L;>�	���~c�RD"�/ZrL3M��uz��h{g�%
g���i�����Bt��"�1�?�Z-�I�m��)[�`Y�~Ez�-�t0E�����x�����B�Z��{��J\��?_\.�9��D���)�����fPeP��W
l����a�e����EZ3�~�����t��������l�Z�Qs0.������v�����{,[/�zt�����*=R_�8�����hu��z~������,�^�H����w:;��Q��h���Q{4qG��M��S�4$Q����>��Q��yI;�~��R�Z,@�I�GwsJ�qF�����uB�����#����rs��9�9�Y���3���Z����=���~�k4��A�}w�����:������n��>�!���8l���V�C����<��q����
K�qr����<�u��\�� �����}s��X�(�-��Svds,��V��t�Vo�w��S������!���uF-�d��d��"R����G�yQp5�(<�f
�BK�E�R��z��O� �XQ�����S��
����s�_������%-�Aq��n^`C^-�n�z[����v�n�� ��i�x�������dy{2����M��%�2��]N�r�q���4�|��=������������i�Wd����8x~x�f����������7K��4���n�
��A0�;]�%w��'���xT`�X�2�H �s����~���������!8s�����0��1��-[y���qm��������X:�w���joX�����Ns��h���h�l�GrElK������fQ���v1�Zo�����qS��{�#�Cl�&�����&KP�6����^���:�/���/��LSm>�i�z�h"{;����f������X�zf�lQB�V1�,1f�a�]
��^c�X!j|5HYJ�2,��������'�G�?�VD�4i��r{ ;;�H��t�����Pf\�l�FA���v2�I�es+4��n*�;�boMK�������PW2�����0����O���E���(8�y$�r����A�*@W�Y�U%s��\��e�s�\�Y����VjG;�^�te�!eg�r���vw�3�����$
�����(d�|>����|W����9E6��18gI,1���
��������g���NK��������=�e�E-�.���'�B�[��f�o9;���3X��]�����= �{2C����O�I����K�<��Y���"���5�]?��Y:�8������+���>9�`�^-��?��,������Yp-�����T���]���>�:��<e?��?�w�N:�C��jw����5~(�Co�q�M�)�x��i��qWn;��`g2�4�{mx0nu�J���n����)�������B!�:&�D����
&���u"����������*�G��0i����<�W�+�D�	���EB�{������#�obz���Lo�v���z
!��`"��ma��.�R;D��K��,�#���+\^�K��H�V�>��=�
A~��wg^x�����7������M~���u��w;�+�/�*|�����_�Z�w^����R��'�����+��������,�/��Y�x!��kq�<s�����g���z��d<��DEc�)����t@�m�:c����Q�����4��62�QNJD�)�M+�����9XN���}��a��\��I6jP�-r�s�1>*e��)�����v�� �]��=�+=��!���8�4j� <�K_�?r8?���~�����n�����$��zq,}����)xGY1��v�����r������>�����}��D7���U:f���%��-�+���<X/Z�F���>O.vSU�0g�"*tO?Y\3g> ox�����(����o&V�-���*}G��	s�V�E�ZMF�e��~���^s9x���yL&��*tU�u��JJSPUu�����\�<��Z~�������d%r	c�_cf�����:��s�����`���u����w��� Sw9L�V�=-�=&\Q����"���='pZ�K8��-
(��^��n���P�y��@����;����3�J�����l������>P�{�eV'q �b����B���|W�{����1�oSU�����N-�?��e�y@q&\\[s��)�r�SaA�t"��+`���G�;-�C_b�l�]��$�����n~�x���J��{?<}��������������?�I
t^�UP�<����7=i�6�E>��/f�e�n���]�&�7��
��N��BsV���Yb���ZE\q��0���q�M���I�����*����D�B��EQ��u�YG.����ts�S�3I��<	Xu_?3S.7f�^]��]��*0E
��8��w����f��y�������:p��=G	!�������`��D������:����=A�J�S��9��d1	���kX�c|%�n-� ���N3`X
�f�:��)�+4��$���
-��pH��)���G(�`}GT���%�z�#��w���@��4��a�� O�gO4<T��Jb���}���k��93�W��D������uCV���(�yssS�����P����evL����LMgs��p����b��\:���D6W�]�Img1�N������������ns���Q}���o8�uw�R�{P��9a�Nx����^+k�����?�?_���\g4��vsg��G��x�3r�mwg�34���v�7p~�������8A��?���^k3~�:���'��IJ��qR$8�u�'1	�s�Y+�I��:�E��/����^�%����G	�M�K]���y�A���@r����J�z�����|�Y��~9�G�%��@E3���IS\�L�_��R����������|ov3z��E���o��������O'oo~r_{���_�-3�h?��m�R�eJ�8h�:�~�f�2�XqJ_?Z�x�Nw2��Z�fgtg
��K����V�!�J�����S���_�G� �&�zW��c�����/�ob����5��/$������{qxtn�`Hk�(%�s��PY��MejX^���^r�z�a��N=�j��aA���@n�F`�\��wT'�z$�����?��+�qy��*����[���`T3���e���Sm�PN�WI��*o*�`�N��a�%�����.;���10SVL�+W��l������J���j����SJw�/n<7�"�8� ���rrp��]0�����P%�T^+�\���7�RCJQ��'U����wOEk��T�����������B���.�k�M5�*�vZ?VS����WX�W����U@Wm67�w�`q�����O���];��� ��
.�;�;Y��
��hB��C�����p����GSj��0\%�9L�����U��+5�3�Q�����B5��_�T9��������,������QL
 ����kCv��	���o�]�G���0a_������t�������q��>t[��D6�3��:��n��������m����W���z��2�_+�	Rr52FJa�R�3RVL�;P�������r��+����[*��+�b�(U�4q���:n���\b����7������� ^s}��������8ic�1��c�~�j���
kU4���!�V�����(~���
��i7*{���Z�4@G�B� �M�����A���4�t���v2�rm���6�S��*qg2��eH�a6t�)oH�*��vT�i�=?)�j���*\��4Uf�L��L�4�0�v����1��uf���WG��_���L�����"8#���AO�6����Th����(��X��c1@�����PA�d=������N�;�_��Z��3Zc�8�Jr�dxXg
`0B��k���F���Z���^->>�tq������5��~�p���9���p�z��������EJ"H�4B���H�g<:�I��N�������h���/lh��I���j�
�~�*������0���e+Bz�l�N���Bu���R��@��U��%Cu:�Z��9u��M�'��F��-�2�n��c.�#��9[�(1���y1�"'�C��{���&�%���0l�H+�1�WL��-BI��Y�p�����S{�tc
j�y�,��j{�E�0�0�Z�8��:s\i3X8�N �A����A����a:G�6���.- �����c�I'�g���'�9��:�xr�h��Z�k�4CI��oJ�ihl8�� �	l�Y3v�q��^���7y�Vd�q�s��5G�c�0RV�����\�&p�SE$EA��K�P�������5SGt:
.��g����0�b����6\�9�J���e.)$9�30�v��C,���3�sj���hg���n�C���r�@���#Wr5y��d�����Qv����������G
X���Cu�U$x3)��qSmP�1O8.I�V�a������s>Gj"<2�m�"4W��{��'���(�� �I��j�QXH�cH�@�e?<$�-�e@�AZg0&����3{�y5��@B_(7Lu�eL�N�f��zi"��]���x.,{r&T�[���:p�(�V��*$�f���+��^�HC�7*�T�A*���0M�`��	��x�Ax1���b�s�����
�E��R��K���d�["���j��[3U��4�*i~gh�����o��q���
�e���X
e����c�R���$�`�4�*�r�c�L�O������;����BE�iBB��? XT7�Qu��E6H/=F�CL5���I���A����,2D|�Ht��
��4��08}\$z!�����3��S���k�����4��:���
��'m���6�9l2�II�8���Q�-��U�j�=�i��/���d ��$`��/�;P��G�J��lG-0��R�[��z�UD�q�v�]�lwsA�����}�7������MYA!����De����1�� �v�P�M\,\���)@�2���D��i_l���6J�_~��L?~q�r����������������������N���^D�<*8���+��Gxmg���QM��^�9�@����3E��0(����|����DEm����������c|��������}�8>��~��T����X�,�H���q�g�f�-O:����%LK����0p�aK$�#���fce_+�$��o��7���*�.p���o_�<|�!R/I5?=;H��T���K5�UB�7�,�<�D��j�%~��:6S�/)���E�Dv���w��F��ao0�����`T����&���w�Q�^�G�o"�����3���F��2��Xh����t�H��o�W��x����P��m�#��#�5����� �ch��qI��:������o9
.�gg'g5T�1���U}���I�/`��>����$�p�(�-W�	��o:EMU�H�#�G{���E�0���&�y���z�
Ux�S����Q��PyIy�bXRR)���#��'���a�k57��������j=���f�V�0����������o��e���V��d�'�{��_7x����.3��l`��Mu�a-�c���W����+U��p>���`�ao�����-@�?�-b���Q�m�m('c���Q�u��S��������Ug�H�^e�.h�K��F�=>��(���#�e`�iEe�8�~����(F���c��i��x��`����-��3�����#ew�
\�dMLTS�B���x�2�48�ES�I���������k5��p���Gg�G'�@����hH?<a�np{�D��B}h�|k}M�O���D���?x?��A�"M
�9�Le����n�'HFg������}>�G��������nd���z�N�Bk���*~j3���N��n�:�xv�$�����Z�=l4Qyh��D��8�i���e83�K�W]�+s�j4Q-���	ru�S�;�/U� ���Lw��8���^����D!.=�.!�#;#`���I4������C�md|��B`f��1^���x4�
��K��3c:���9�����gL"S��U`�Z���#�,1�Q�1gX��`g�l��R��Z�$d�)onO�`���p��X��Z�'��/8�\��)���~a0z��m��`�f�p8<Dc�0�N�3'8(�x�t���v0�Lq}�`���������%����F^!M{�\b�������8�e"G-	��c�'s.�gf?s`u0��2��L��,�L���,�
����M�H�C����_��5L��pkS���2�9s��e����C�^����$=f�2wJ�D�y��j�{6�A���m����B�B���/��`�k�a���7)��mY�@v�B��>3�V���TK��(���eTkDQ��i�����LA���	��1]�/^K-�
�dM�^��B��<58x))������&��Y�%��Z)���r���4�$��'F�:�q0�O������)�G�h�JlO�6�T�i���������"��o8wT���Es����{z��J����X�r�
������;����JtNTn(��d���^!yi�r���:��"K�U�C�Wr��)@UH�md�
{F�Y*��X�"���}<�2���W���pE�VS���jC
�������M���t�;}ZI��Iuf�^�����.U���b~�������_�O.���h5:;���������������6�@�YtY�L��i9�X���[4��ap+�G	��v.���"���U��QLF��h�'�M�� ��%�V�&+�TY3�B����^[�(�\�:�mU�Y�tv�zw��f�8�!U	N������<U������^��c�� �S�P���`�����H��,�=�}/���q:����.$�)*<(�6D���G���[��)�J�V�th3�!b��=�+@��Yu�l1TE��+`�d�&#R6���J����v�������,G���yc����D�7��{$X�,��o.��o��5��7�Y��,��of}����r�}�����KX����SH}����J����h�>_8�s��Ib�4^�6Ji0���[$���bLx^�3S[z���L��,j&�)9�t�
������neSU��-��;�g`��4�c
-G6����W�T
	 ����i���eD��D}���w�����"�949���yt[�'���_���=�~, ����=�%����2����h��77����o��4c%��Mwca�`$����4e�^PA�2X��)��'O����e��I��qN����;)�RI<�}��IY��I%�U����6�H����OKC_]�pC8i
���'���\}C'Qg���+���r��R���5�7?Z.�|Kb�����������p~x��4?�����Wp=���]����lFQ�$?0���B��#y�2Fa�V����Z�GQ���GG	���z2�4k�|]�h�iN������z������`5�(�H*;�We����m_�I��T�
*�O���6��%�>	���/�Wd��9�����3B�!u-OM�?�-�s���^�2����+��	�&E��Y��9.���~�d$���WH��B2������	�a4u1@E�=��,@h��������!�K���#��D�q������#Nr�L�������R��f6��h�/����"�P/eYCQu��C��JD�_�/'N����_K�(���k��N)�K&�(!)����a]�0�L�������p����p���t��jx4�����G���a�=�����q@��������i8�*l�erz�����H;a{�.#Qg`�����Xx'y��E:�������*�"����M^�9�"���u���m��]�k��o�\�W��@W�I]�E�S�m�k��
�5��C�%�T���G�K��`&��P�j&����^w�uJs15-O�*gx��&74��dz�W�m��u��N�N��Yd6[&�
���I@������n��xq�=N3�f����������t�Ia'RE����x~����n�x����K.8�su�r����vw�m4v��Qg�H9Y5�9A��:%�������/�&2��o
\���zt&��x@�<76�b��h����!X����G�^M�-�������*�;y���u�X�0-tb����Z�(��X�{�"��)zk��H����}�O�8H2��.�'LR	v���v[������A�l4�J�)N��������B��
�*��u��������7921�V4]���!%��E8��aj�F��)
��\�L
d�� l�4*}��^6#������>A��J��i��M@��34p���3�������I�����x���
,������
�������iF�	Oqg<x��A�s:�9������j]�s�#��s��1wW��-?�$������K2b�Z	��R���`�1kt9zn�%�,*:��lm�jt�����q��d���:P�M�+'�bJ)�)�����]3��
g~D��[f�U �����E������s"��#[#�QE#d�7�����������}��Vm��,�V0R%H�IS''�������������"���88:�8�]��N�V�
��!��KnL\K��V�9�����B�vB����������"���F�OlCWZ����8����`���T3�����?NPPK�����I�#FKKr��-}]������t8y P;�Ue�z�!�Od���5*����X$ �,�d�������a�z�In����f�����i�#Ol�v�����j�x���"�,'!��_%Mc�45����g�&�
�bxg�T�t�H����C~���� )��5�B14����������-���
|���:��Q���TU{��k���FX%a�U|3]<J/�M�8RQ���ym�w���f�:��X��\i�X`)���E���o��	_0lt�X+��-Iy�K#|ylpjB��s	l����7�=��^c�&�v��$�H|
��3n���%t��DG%%R��P5����l.Sc�'L���a�1�fj�S$q���%�l�����v�J�j�>����-��X�2�->�&�����J;����~��;�z�L�;��$!���%_f��1�%�A���b���u^{U�C��*���X�/`�XP����[$�/6��7�4��z��$���������m�z�(w��J��h��S
�&�� �,�Z!vEB0#���h��a��`���$=E��R��`�A�F5eLE�h�H���-N�6�\_2�T.K�*�	�H"��� :���nM���S�6/\�)$BmC�R����f�[9zR*A*D������LX�^�����v�6���K$����<�
����,��k&����7��j��(m�G�}
x�h�K`�?-�@������L�9u��{�HJ�n��q�|>i���<�`��I[v�����3���8��{�6}M8�K8d���d��B�:Ua����Dm_3�."4�EB���6��FV�cr�#�����^!���K���,�=/,.RNQ���IF��\�$�[skz%6L+��
�n��+��}�K�-FE#U�rW��SQ�|���aF�`����d�M�;$W�<J,IN:����5���L�$����B<uv�l�J�M:�kfOW�����8�����{!hS�������%�a���:^92��ED�n����!��G� �{�fw�D!�D�����*�5]s�w?-AYs0�
Z,��b�)���7R]���tj4�._J�%��RP��t��7���sqf�j�B
�I]�$9�RJ�4��>e�A���5�e"��~B��P�G��@Z�VK\+�e�t\CIe#c�W�YI%�W�zZ�`#��t�S�ZR.��>'����*�I>R"_7���An�'*5�x��p�p��$�L�^Me��1q��+��������4eB�_�2c�1#0��^���3�R�$9`C!��"wVjvT�i���':�K�gD�nY*���$����ET�0�d��C�T��=mM��������[�{}�'�s�>=��8�$�����(S/WS��'l/�H�}X|N?_BTWUd7��@�������b)82�������F_���^3a�^h��P	���-yH�H�kr�)�_;"�Ar�,I��T�ne�[N����b*HI`l"&�U�M\�8��������������W�"�t
�Tlw(�����d��h���������3�c��`��~�8{{����>?<�S��d1H��Q�m2�����D���;	�e�-	^���<M��F1���k�����v$+ %�r(��=��{�������b�o�����9QQ1�����������r�!��Q�G�)_@U-��������5��1�f��6�ZB��jD���bsT�B����Vm-_�M,��������{5<5�M��x�{P�T�����,�Qa�C���NC43!>�]��2��^���X�
�y��"M�������,B]�b���p� 
�0so�
�I[�,B��~m
��'��E��H��V��m�U���r��G|"�<Zc�P�
Q�N�_����L������M_��>�+�Z�^����;,+��
����!+(�U������h�+6H�eR�Z�M��5!���Rk���2�T6.�*&���M�-Y�6��-sM�R�WB��%xc{��-��~�����UP�)�@n}�+�'�\�
�F�i������[U6�u�$����(�U5���JZ&A�WR��v����<�/��7)���_����[��Tj�2��	A���%�s��
���&��������Z�R�[Je�|�����.��,����q�_��Q75�����a��6b�����=�u�#��w[7&���[O���|����?�Z��.�����XcO�)c_��QA<=J,���$X��j��O����E
j�/S���\
C�q���k���B�9�O`H�9<,1Fl�b[V��Q���7�I��,yK�Y�����F�:���x�'EE���3%4-'�����������*a}P1yZ���6�|�����/owD���c�$���J�+YXb�q���EYj
��R���	`J�
�Y�?dK�Wh���������lJ0�"\�c'�|��7��!����}�:�TmUVz�4��i�	��KH�8h�����P����m�Q�Hi�%��b{u�L�1��)���sY��V�O�5�������A�������S����1�8Wa�.���I����I�suu#<�O�c.��������8���i�f�"�jc��,#h��G6x
�g*d���;��B���H����Z�����C�4s>'k���U1��`{!^��[��486H�x�F�^x+��5�<���[��)3��Q
�f����N��Ns�b�>�8�+YY�#Lv���-�a8etEm���P�3E�gJ(>n)�1�#�Dmo7n�[DV���"�-2����m��_(��dYs`���&��m���(C~�T\��Vk�BL�>h�x�[J�[i��-G���%B��d�,��H�*����iv��UjM�r�n�����UM;�DF ����Q��Yv#��he�[���f{�������C?�S�!�K�6���9�.^Ar_ca�i~�r���12��;Z
gC��P���	���?��/	FJ�e������S���^���H��G��1�cI~m���+��]��F��_*�)L���-IH�,��F����V��f�U8s�/����dp*����dK�I�:u���TQ�����qU����k����g]w=�����ya�XR#��#��-��jV�11�T��g�J'A�,{��H�9�qy�CLY?	BI�����)�W���-�6�5���\�X�,��������n�z]G��m��).��V^�p�Z;u'�;����I�'G���h�������;i��w���"����c�?��31O�p)���*�UM� 8��b*zhN��H:�9!:�A��X���a�������8X���q3pna������UV�eB=�=�1An��o��Vil@(���h�A��/�(�to!�RJ}�O�m��6/�f3��u�/����A�T�03_~b�m���Wh�}vp~��l�`x���������������nh�j�}���1*��Dn��M����o���o�A��%L7������Y�o�f�8��/������2�C��	��pr�����h�sE>&�d<�L��;��Fw�v��n�7j��G9��l��am���~���+���M|���E
1^'7���*���B��O�;�}�v������a�L�����"�<W&W���/�sW���f��U�������(�Yw�I:���v�d� �R�nD�Ag���$s@f�
S�co�a�$W3��p#�O��<��a�r{�aV��L"rL�>��,���=r�J����-bI�\��@:u��-U!vW�q��nj�) ����p����k�R;��7g���D����Kj��R]ZF�q�T��
B/���7[�l�B�m#�x�(�<�b���2
�7(����H�#&�x=T�1g����{rg����s~��dE���2|
T����6�n�7��7��������db����$��q(���g#2!1��8I�{����1�����v�n��v�m�uC�KR��\������d�����TTIC�K!^�����4���8���9"W�	M?�T���WehH��0���*�����
�zF���+g�M ��}N"��-IR����+~����P2�h�|Z�5�\IH�:��)��/�v_�;�m�v��8@����@�(R.�c��K���o��9o� ��97H�,'h�$���ca;u�1i�� �-�Q���De*%���$��afS ��w����I����j�(��4���/�*�M��\�����O�Wj�K
�)��u��ApC<M(/S�H�PR�������k��������x�^'8E���5��=���B��be(�'SXF��&��Z-����	+�RW���,�@�N�P����0��5x��(�2������Q����,�������pV{��	F��������.%���99�5w"��2)�\�����V���l����������6������M�����:&���4�C/�� 
�p�4*�h	SI;�G�`&�d�y:�g�G��LY@��fa���X<�:%T9=��^�`���n���V��__���M�rZ�����v�1w���v��l�{�����uX�:��z��c�G��}���w4�y
��{��l��������������uD���7[�%�_cx��2[�a��4�L�<����;�y��dM����nv�V��q&���t���v{�����#G�A����P%��\������D��lU����
f��������W��/�J���?j��|V��.w�����V�������m�v;;T��Y9��/��������b�����4����6RV*���I�����o���������'�C������)?j�����
�Hc06�<�eq-=
���
T�,��
���k��_�������-���D2t�DX���ee��P(����JrVM���������'���A�1 Y��Q5�
�[^<Y�]����T�[�n�����kO�5x~�+N%^G{%��+y��5�G�C�`����8@}��k��*/�h��"�^v�U�����y��aCF[����9�=m�{:>�������������u��[��q~������E���������_f j5.�_�J�Q�rn:���7�>���~��[b3��=X�h�X��C����f��ce�p��_Dw���?/�@�_��Oj���B��W�KoG��zJ�W	�j�����]�Q(YV77�l����F�����-N��j��� (6';�Ao�h�:��H��v{��D�ih��������l������wX*��aUVC*b�������������������4����oN^��VB E�p~��A������9p���oj�Q<�=";��y�wvx{
�����tf|&�(��jn������%)�����Z�3����on.IBYt���o�
?:����K�7�������=������U��K��K���e��&�����|8�B��&H�[�r��).*��������<�{u.v���a���*���K?�8����������l�����g�N�:�l�;f�����l�<�T��q��Xig�^����V�EZ2b��eh&?�F�!rG:�Q��~���[��	.���7�n7kx�����/A����~L0a����5���|8s%G%�5�E6��{0����6�.�bQo���N���wB�*��~$^���~s2<�
���H�����!�[��z�XE��w����$
���������{hnP3��I��((_t(�}��by�{��\4v�
�q�YO���i1��qg��w��`�S���{��Ge����!���n��ZLVy(
<8l��0�N����a�����L���K�h����_������U�`�[�@Ng=�\R6��q�!{��M��bN�+���q�7uTA��Q�MP�
������'�����LN������,Y$��������T��>*S��=?��o����[`����rBS;6N����v���V7$R;�d�\J�KNj�������`���(���[��d#$T�N�VN��r������{\f�����9�]���KJu�V������d[R�������� �
������9x����j�=);7�����Td��F��*�6N,��l����7��P����'����Ks���W�i0��gW����r����a��}���D���:"��Q.�O
5�a�dW��n/�r(_��@O�����j��(wK�3�O���_��Z������0��Yb�R�������	�cG���a�.k���Ma]B�kWd[HE,���z��K������8U-��r�T��"8�h(m_��XEk�[�M���}u-�IP8�*���h`I��M'B845���-����2��7�:-9K�@R�=������!$.T{�
;CUh�Tg����B����9[���8�����-G��8��A9��/�Hpzk�DR��"XE[X`5-iA\~�_����y��'9���Rp[8�=r�:r�^=������!o�I~�JJ�\Q���7T��k�$�>��#4a����W�6����WI�
���}���T��~CwynSE4t�H�=@����%�r�|��<��|��aN���$������� ���yG#��aA�B}�I���h�4��ei�*�-���nm��b�L��;R��,�or,�<))�s7�L��oH\|�&6�V��,%�Q
�m�F��B�����1x�����R�x-�I�����V>9Gn��0����T�-�*t�5���*�N�S���c�o,Xe�J�������u��%
SM���R	R�f(!��o���dr?�%�9+7�F9S���+CS�����F��S����b�(�|=Rk�bc�Q"�`=�f�[�<)2Y�����u��J�[�?O\7���]�xt�-���LgX$�C_dC��R%���J�8����du�+"Z��Mw�<�Qd�t���% p}�(qB���)��X��9U���?!�����x?K��H^b2+D=���aJR��	#K7#A��R2�Di�/�%N�.����������k�m�P�B)$�����T�,Q	|�6�����0��#/�(>����������������������+O|��������
�����,2��=�0������N�>a�Py�����n$^�i���@m/eL��n�������^+�:��9�"U�����j��2sZ��9Y � ~�l��J8�*]2<��.!#���q������u} ����^X������=����T�����!�x3�w�j�&*(2:����h���	Nj�fX�l�{��d�) -I>
��{���!!���$o#��Ld�(9�SuD��3RI�K3<x�|�/CJ��)Ky_�4���@E�2K�N2}����o(�I�Es�x}x����*���J�Ay����������"�*o4/�E=U�z���n�L5������)Y�d�Y#�a!y��e�l�U��O~���T)Ym��c�(y)�G�g����JpL�����RR0��Tp���.P���z KOf�"G7��^'��3���0a�eW9��]#m�!���)}�EVR��&i�9��#��V�����`�Te8
x���a��<]GYZ�2� ��,��'O����I��b�(�e4��@U�h�2H�����"����3��q�#�=%r�T����S��_:��E#����%AJI7��f��X�O��0�
��zA�)*�1�N~t��^K2�+���-���������/�r]y�T2,��*���'�8!��o.�-��@S� L$����mr
M�j��w#�B�����
�����6��di����!}�o�_��([���{���,���%�$�k�~+ 	J(�� -�k���?�?�ynyCe�z����i�Df"�y���>�5�&��S
s����j��F�pC�&\��u*����yg�q4*_F�s�7�����v�32]-�rg`�m�} �4I	�"P=������8{�
�bg�%fvH�
Y'������������y����G���{}����" h1��$�7�������d�O��X��Z�I!&-
*a����0uSk]���Xn�*�k�j�g��q!��OI�+XdH�5v"���R�?�I��qs��{�< ^�;Y�������o<b�������-OX���[[9���$(q����������=�I�
��������<�O=9�`w�_9oi	_���VX]�e"�u�2�A�%k�9��f�U�P��5m�o(�$��@�2]�+�hX��P����o�04��<�P����������G�����V��k(-����EXW��d*&�5����#�@���o�P�����e��P�l����Z�\���v�Z���FT��D�pp�x3I^x������)R~��,a>�*f�(��m�;����o�X��"m�eV�/z����|��<��]��?���^ E��I��a;�H��
G�3�D��jB8����C��$hm����M��:U�(,�P���4��sOT�;��$2��e����z����0
��`��Z[}����-�4OJ�(AcT�� 13��e���j��������b����L���z��J���X�����_O��j�B�^�5)`5Pw�N6�@����io���ds^J��,|��pE�GN'
�\yDBS9Bv��Pzl,,���A�>��z��u�H�=���I��6>G/;��J+pa�,!XX&���&����:%?��:��K��Hw=�$�c�1`����<
MO(FI���2��,O���9D���������?p����;
���R�"����M��cbH����� -t,{Q5�,{{�{^H�ZQ_l�R��O�g�a�lE���;@��@�D�����M����n����XomZ�����W�R9�}_�����4!���'�%�MUD��9,�QjYO�
s�l�~|�a�y��qL�P���
�c�F����foR�����H�P�6W_��H(�����c��th�����"������X;��d8Ap.�D0�X�3L���R{��Q�$��!<����#��x�y��f9��8w �+��Zb#`<��E�8�8�nCR$;|���)���i�2�Um,T6#���$�R��=�+N�;�=/@�]��=?��:V�*�n���)��%.�)���R��G�}��J�1�\����G4Z%M]N��J�\`Jlm�U��f�YW�d86�����$�m7��]�%Y����FM���Q�f(��
��`��k
����,��i�-$	C�UwI,a+�}l��z�t[6�F�#��U��k�c���W�k�9L[����������s	���OU��oG�3�zvrp���������B�M�?�wQ	"^���-���*&B�JD�Q�6�
>���Eh�!5��hB\6^)<��oh���{j�R.�\u�m7s
��z�	v����[��eShw�S���(��N�)d��Sy4 �>_C��jD���7���5
�&����~��b�Q �(s�X v�;q���676z��N��������T�h�� �ms+�4w)]^�J��l�6#�+��N{9��	//��'����<����* g�Up~xuzqy�U��'�N�N^.G��]�}����*����jV;��O����N��-M���2+�}���K}�)��������Z�o��.��D��\�������KV/Mk}67�<�en���a5���F;)���1���C�}����D8��L8��(�x���������p'W���e�x[_��0O��:������������DTH��O��U�C�Q�=B�\42?8�������/����d�Eb�h�k���s/|/
�c49�?��a����h+9����{��j��&��AV�?���n��������?�!������/��������_F?���/K��[���������zC]]]�_����������o�����
7 �:��R��d��@�Wg�������N��x����������}��?�����iz�������������;{�^���v���[;�����V�{���J��x{���4�w��k����:����������K������)�0�f�������`�C���b��������s��7��/�G��'<����7l"C�qR\���:-���d���RF��[0L*��s��r�� (X=��3H�o �gY'��:0c�`�PM6��D(��x��>����h��Rv&�2D��ftA������`1���]��R�,t�JY�I�<�B�<�
�5z����9�@���p��F#��~X��5_�>���s�07�����	���v@K1����A�$t�mW$�/W�0e3��#`%Xe�t��b�T[���/�f��~���0�!�@|��{J ����X�fJ����X�����1��@8q���4���T���i��eA�el��U��p��C#��9�~y�� G�@7S
@s���EE�n����@����v#l���`��/IwF�H7�1�B�kzG�N�13+MH0h��|�HF�J��N���� ��6bHm��� �&�m��0��y���a�N2J�)Y�d�� &Q�1Uv���3X�V�B�����t�,>�M�$��S.��5�}�C���=��p��@Ck'#z��D�� o&�(�-B7���~�V����[ �0&Ns�/�2q,��-�.f��#��g�R���EI(d6%�s6�E.���d/�3+��:��� Fkk��!�-#c���L�e`o�>���C�}�q� e)f("�`�	"��]2]F�)O_���?G�e�UV=2�|�KH��1���cXNE��	���@����R���.�~G�q'�iD�'q>�2���(�5_��������|����m��/���7�����_���������(�z������ �x/���=�6)�h��C��]��v�
��f`�.���������	�E�L������n?u;�[�m����(��}�_s��U�s�#���3��c������+W�>�����0����S?u���U�yn��A��0�G2�����#!p��o������:n�UuT�}������+Rb�L�^��~aq�p���������(���_(�t�M�<<�z����tw�z�-���0�?����i�7r���P�9�Ro�Jo��Szl����S��@O<��:^�S���JT
��o��U�C���_ai�TP��@7����Z�Q�o��0��@�G(N�	3�"�_s��W]�������z ����Z����n��������]Wonw�����^����6n���_����,��h�t��X�����>b�vo{o��������n�������A��]����m��L=�Ie����=�a�������y��HMC�b���h6l�P�����E����������i2|�,�|�'����&w/���n������Z'�m���=��L��Rj���G!���3�A�|�����Pc�>�V~��+M�����U=n(?E?�'J�W�1�/t�5��zY���k6��QSq�i��LD�h���4r*D���A�%U��A\O�,���K��]Gk�7h�Ql�~���BWd5c��<ca���i�,a�0�>I}b<x���d����6����j-HUkBms�����o�<����5kC�66{�	����.��LF�������q����������ww�}�Z�{�^����v*X�?P���M�����
�?�O"T�/l�onG��������������,�r	M����w��hU�!�Wz�D�V�6�cL���U�F��l��)�Fw��d��������9��(v���B����cC�z�����������x��e�7�i/���,+�3�w��5KV7��T7&�V�8���;X����c�Q�	�y����~v����!	�l�7���������
d������Z�Bz�p�Y��Q7��&�U�����'�IP����vb+�Np�R(���b��|�g��Ty�����o��1��O/��d7+�>����5�W|��;�JE�	���o�A�����O���I��d�-����c<T��0����������$_EV�������d�C&`�G�[|�����fjuOE�������o���|�] ��oIQ��\$����L�<q���{f�O��9���b�&�<��7��?��|��_$lZ��b�~���������W�mo�k(E"U�UZ�N��S���|���3�n�#��k}�!�������|Qi��2GY�N+6+$q�y�luv�����^kkk��[Q��l�b�����au�	Vd`X�L��� �1����$#<�������|�
�Z�Rg!��@k%��O�|�SX�y3�F�4��e�wi�+��z{kw��D����z'�E�~�.���S���_������e=MV�v/42�G\1�]v����^J�RzC��b��2�z�6N�v\�a!���^����To�W�"k+�-pv>LQ0t����f�f&�?��"=��l2�YE�@/�,@@�I�5�iJ��LG��N�z?�[{h��n���]yy�.���=s�-�qKK�?h��I��<EX�o�}����������!��~s+���=w����>F����Y���%kxS*�Ek�����������%�U�S#�W�����s��.��
���U�h�]����U�U��<����	��Xt+����8�URQ�pkc���?l����N���d��'+��@���H��������T3�\����d�L�k��++~sXf�>�c��=��.{`��������@U�V�����on�w��nkocco+�'�x��Sc;��*��jE��u���4�a�T�p�G����r�<!S)!��J6��m��P�K�o�
����oR�-D+�I���^��]���q���*	�b����q����D��?B��X��i7�������(�)N����!�%�p�����I��w���w���#�h���������l��M�W���pX8W�F 
�b�5���K����)�Uh������?�:7���L�C�v�~���y��04�v����7�N�L���2��/��"��3%#�8�dU+��������	b�t�����Nqw����]'>)����+[{y�F����1����w�� $}���eU���tE����/��:W����LH�l������4+���������Y~C�c-o�x[��FiK�/��
n�?������vA�7B;��1OF�l�����~o6d��q�`����0��f��F������������\��{| ��8�b�l��{x�RI��0}���e��A���'��+���������1Q���7$B�������������j���e���|����1���@i�t`/��B�`���5l`����%jg���K@�G�>��p��A4�N����f�Z`U������7��l\����~�D��b�M�B������R/c�>[s���(9��P
x���1d�A _��i,��-�|L���f�G�V*�"��?��K5s`��44�����H���	��9���P�����T\/��CwV���5|�j���P��V��
UW��l,�Z��������j�u �Pu��(wu��%{Y��ML�_��P��t'xR���'t"\U6j���X���=���}��Ut�N�D���������]����F���[��!�(|���.'���V�	Hg;`P?��T���cc�J��7x�����?�g����s����zM���7���:���w�3���!]b���~�y���p�����?A����=���G�X�cG���������P/"���I�	�2���B{����*�[�r��������'�`��s��{��w�Y���9�����pY�:�������q2�X��3�*��d���n1p�����;Y[*���n=�~9P��\�/�^�N�B�dm�G����S��6g��C]�9�������C]��y��?������>���{���&��������I��]�����h����.��G�����3�P�P�~���8o[����R�T$�``a���!q�%�1��6�m��5��_`8��T'����	�8G�;�)W�!��������@�P�_�������P�	�$�i�����	?+Zs�U"y5��z��)3�f������6XK���b��r��k4��?���������X����:7c{��Q<��&��fh�JbV����1�����G����*��n���=Lt�i��x�����N��l�}{~|tuz�I8�M����o*��"�F��P%�$�@�S�J�T�f�J"�LG:��-�a��T*g��Hz�r����SBB���zl���$D��- q26%���O���n��l�^G8���aQ��5��G�W�����]��]�	n@`]��z�>=;;�(.��]?�#%���Jd����z��L�KT��"SSOk6�%\7�,�jS.7Or��"�CH�e��aTL�n�����7Bu���u;��~X�'������ �&���Z��S����e�dr8%���X�L&��U������5��D8h�18%��LYL2�n]"y���������}ur�~���D������
��m~�9�����ca���'�HJ��=���
s�p���U�9��W��[`��b��X��:�Is���j�bo��mq?�d�.b��������	��Tr�i~6V�����+J��|�E��2mcJ���1�Gf�:������E�n\B�;z��$���������\>�	YS����.�bx����<���;�
��]�	�|�J����$w6M4�jW)�t����M����!O
���=�H/����gQ��*�0�g��Z����[M�����������>�+?�{�EbY�����M�K�� u�m
��H^�T�ME�!�	��2������j{�k�v'�1ts��QnS�ft���j]��K,��]lQ{������e�k1�)+E���]��MP��$�(b��G�<s(���m��3�bu\�����Z�t�����D�:�(7_t�2�j��-�|�F��[ut��i���^t
�%H��]����KT���$C����v�/�k�X����	�������K�������=c���x����i�1��D� ����$���0sZ��o���)g�V��t��A�:��0�1EJ���f�#�x��k9��[m����
?�[��Z�(�=�+3
2N,|M�]��\���p?Z�u%E�}�8���"x�p��)G��Z4H�
����'b�"E����_����y�s�Oj�x7I��v����?H���N.�Ji��f�XL(�S�l5��hm���h��S�d�+��>���\���ly=�����BX"�����m�������b�e��Gp�c������G%���Pj�#��u��R>

��rF���u�d�_��W]{�L
��
�s@�Q��
�<�g�X����(�v����i����Zst1<�o~�Cd�\s��!�B?YW@fo����u�t����������?Y}8���mkK#2���F��o@�+,I�'�7����dY���d����iD���ioz���i_-��+�����dCx2l(�~kAl���%�����.�i �������,��&�f�4�um��Y�L������
������a���}�+�([��u����w�+M&��&��3�$�C��_�Y���v��u2������-3�c�	�/x,��=�������#}��"!5ne�B���K���2���iF?�9�W���M����o�<��_��42]�T�� �0�Q���r9����#�����1R_�w`��M-����
��?4f9��\S��j�NF:L�-J6jrA��-���D��^�}Iu����x��9'�KIE[@�e#����()u��7�~j_�������>>�ZT��c�6y�OH����akN7�H���mB4z��# ��Q�V�<���
��p���������}����)>�����K+���#�c�w�
�p�*��
��d��o��8<����f��v7������.@[��{E�������gGB�]�es���/��� L�&��e����I(�@��4����c����O�w'���O.���mB~�723� �t�F�&vl�]:�����:)������-���t����O�lq��	�}(���//O~��Z}��|�������n�������u�����#
��mbX�����)���	B���E�%�����NW����_�)�KB9h�/��*sg�K��q�
�G��� ����h}��M�ZpY*��L�X;U��I{���2E��j��@��^Ry��eNS�.�`(Or_L%��/q���(����K�EO�R�Un��5�1>���d���6Z���H
KK��=t��8��|e��(�����R<���m��:��s����F�3@���9���y q\�����<B���
�1s��������0Y�,7�Jm�����,|��U"�T����HW��;wIqX���m]k�7�/�1I�� �Z�i���E����r
B��e�hx]C���\p^N7V�.��d�J��9������������2S	�u���n6�z"�-�=tjD[��4��,;r�A�2���:)�d��[�?~� E*�p���=�����H���q:f&h��b������g�����y=6:���;>�F�[J�0F'R\|�+V2U���R��:��g�7h����A��n�M�j��.����)���@����t-{D�s��M���h������n�����6J�11&��4�e�@�����a(T��0��l��U������62�F���
�2	������4�����1
���&���U>��������h
�{Q�H�(��D�������26��J����������Z	�K������$�%e�~hzZ�/?�MC~�����M���E��F&G�.��R�lf��<���\�LOr{��iX�8��J��R�;:���^G	-
���<KAgA�w��,*��bO�a�y�BAU�R�ci���<O2�G����C�n��/��V0��UV�X8O_Z��L��R2	��0����Sn~s���������XH�����L����P�^(_)U�0h���X%RW.{uU���W�;a���R�x]���;�/�\����8|H]��Q����?���7JU'�7
j��Xma��J�����.;'M��7��\�N?����a;5t}
�6V~'>^������& ���x=�v"Vaa���}����E��m�v�����q���n��@=����@)AGQ�	IrN��6�k#�tN!�L�r(�H`�x�dV�mU����9H��e�����������$���\s��\���3l��I�$22a�p���"���"��k�K���F�<�
AF�y��#.����1b?�Y�UG	��0�bn&>��������'����<��*����~�O����4��H�3��>��F�eWk~goOK;F~�w�h���������
%�H�H�xq�x��^��\����h8V�[j&����?�=,[�v��8��y���v����T����k�a����n���>��9�	.�J�Q�C�<��mh+�0G7.}^�U&��lO�����2��B.����e���X���������_��}����Sx�Id��(�.����������V���z�9�	x �,$�X��������_���D�{�|���6�E�K�+'�]i���wM��H�S%�J$H�)��h��S:uJ����QJ�n~5���X(fj�^
b>b�auQ%F�������hPhrU����
m��9����<�tc��:`�Y0���������'����T��o�/h�[bs7�4T7���������+�f�����h�o���aK��w����q�0�ClU,��6�fD�w��'������Ia�V%�yow�q��5]�~�'�eb*|���^�'{���l�a�?(��uT|�@����Y�
��V+�|S��[7����������!�<��}���\��tbf&7)(kJ����l!6�
����`b��TF���~LXr��8���.v���Kp�Y�]2��ih�?/�q���=��hQ� _�8���j�8�f�D�����Qi-	�x��@�y�Q${��������l�!0��7^�R�������KI���4EB�NL�a�3�����������x�����xgkX@*6��W-9t�w+Pw�2m��^�QD}6�o�P
���d�IX���@K����4��g����A��M�;lkY��o]�5��}=�����h�����(�[����������^�N��k��P��@N>cyC+>�N{��'r^�r5����0(��#����y$n�/pvr������D�7Bx_��X�a��Y�"��b1+|Y�������� �������@V+HiPn�����d�4�b
��n1uE�i�70A���Knyg^g�R��������$I�:��[�p*i1�;�O����L��E��D�_>Q{�J��*�W���{u�B���+�@�d�y�!)��nI�%��+=�v�����d�]������3���$ =��$o�q��M��7c�!����1���T�w-���.�l�$����N�(��2{<�_������N6����?�/����Z\���t�b�_��{" �_�$����V,�6��*��u�
�t^�v���VA��������S��{���E�&P��Cc�5�&�;�'���&����\;C�L����70�A<����nF�	R��e��(�\n��	�
���d�������>`(��a��Q�D�?n�N��e�x�O��q
�
D~�+C��uU��p��9��{&#D���Ld���B4<-;�����l���Y����1���#wT!����r%4�����������da�s!m��\ �46}������t���LH@x���yz������l�6��]����)��������������~�d�x��'���:�������@S�$5���6L����t��+2S��D^���tqJ�r��"m���"k�P���
������]P�v��$����.�#\n���Rr�z&����&)A?	��{h(�+��s�N;z�����$-�P( �-<��}�n3�����M<��}<��O�\ 3�dpO�kZ����lc�T�H�
�\I:X��6����!��@V=�+��6/-�EQ�w�(���a����P�V#��F��������l��+���45?0�t
Q���^U�W�,de}�%q!��_d����������I�������{=5��E���3��X�x�l@�oCd��c�F��4���~Q(p�=�s.���!��P(5���nq�Lzo*I�R����nRo~�U	�j%� ��@�����O�
��BD`�G �(�tWVl���/���@Z6�'�B$]_\��"(
2�v ���V#��#c�|��a�?l
p�h��
��LC��;��J���&�j�@����������������]v��51���<�]��,`���Tm>�U��S������F�-00o�~%�y�+���-]��]M/O�O_����`V
�	3V���3+�1�z�����a�nx����	��W^�`\����7Y��6��|�Q�����VF���)dZ�;����N����������������e���w"~�X2��e���5Q`���K��DB��
����0��E���X �d� ���43��������_b�f�����%��x�,8��h�J�qI.��g��b�{��^���D�i-9�`C"�� J��\|�����6K����<&�ek_�@j������>��ivLT��������c��V���S�{�m~/�i/	��,E���P���"{����0�8��������q�x�*����g��d��%�%��x���Up�Yc��n�q,K�e,'�8{�~j��\��������!�������)z������G�yzN����9��z��H�+����!�#"���*�(��j8�\�^�!/�� `�m��C�F8��_�8��Y)���>$c<hJS@��4	�g�	��{�ve(r���HE2?��;��������Y�8�cE�2�IY��c{���z�3�Q(x��pB�<+�z�����0'��]r|V����EjP>���r����\}x�����������^���h
{#m�]�����?i�M(v�^{=%�g�;�����*�Q�?��	6V��R������N9���F���|X��;V+C��.g{��m=+/h}8���"e�`�r��.*M����8?�������y�
r>��l��=z{��lEH���'oN�^*���wL��d���S{�]F�W; ��������k�#33^-�I��Sw7c��h���� ��*���	�u�gDc�kkac�%B=+o�'i��+�>�b���T�kk��R�C�����0U���_�j�:�h���\\|xu��f���k�#X����������We9�,r��R6e�_i���%bS������j�����z
�vf5�Q/��$�Z_U�:W�[)y��l~��L�����W��P�H�X0������z�^�,��[�����M����	�(��?D������)�.�|��c������=�f?#�p-jzh�:���'���;�]Z!�
L������f�]��|��!T����e�s��{��J�����a��6#MR�P`��A�����UE�����%��.�*VZ���@�K�[|���������ZK��+��Q�t��T_w��������(�u9�8�vNae\����w�4�+��_\$	�~-���:�k�o
���j��~"1~T���p�s`�6��;K��g5�uO��K����o?��������+�P&��������~�<��	.[��������b��~�P�	�j��KN��,G�kV<BA�g��X�L�<y��Sdu��Kk���%�X���]�����Qp���r�����v��!G$��F+��X��J;��I�.�X��P�`����eo����-L.j���Vdqa������L�q�)��L����	�$��8	]��:A���)Yf}i��R�6��r��$��\8��eX�����[0j��R���������IB�n���aS�5
�+�T'Nj�>���
_��Q��XP�v��8��{�
�*��`kS�~�4�i���g'��%�X��,��/J�V2�q��d���3��Z���gZ���<�ZM��[��9���Q/����|j-�Zsnds3�v�����wR�j�
��!��R�u���X��;���[��-�D���E^�?*���^�P�^�[N*�seu�����$�����8�BP���:��E�b��8�g��Y+�P���Bm[�������y!��k�|el+�P���@��B��+5��Ov���CU5���,tw�#"$kEBVg16�X����7�k�<��{����G�v���<�������$�v'�g�'��/�K$��%��"�9A�_��]
�D�;��jB��������Vzld�b�PtrAY8���X�mZ�8����]�f
������+\�K��?�:��I����]a��an�����4��1�_]��` z6�0�uL���x�q�E�	�p*�JR��!�g{-O=�������I�1���K/� ��d\���0�����#�(Yo)R����-��=��v_�M������Z
��L�n*��P������.l��A��U�����wAE0>��fhB��TK����!Y�O�z����ej�
��1Ud����D����eZ������������hB�T�d��PJ��D���S<���u���/���k�V{�����k��y�K
���k�V��k��y)��xf_C(����
gI�w�����JYX�p���e�9�;���uYN?�G�&����)��p����|M��q��Q����
3-<�se�=���
�{j1#�J\U���S������*$N�`�L�U��	�-���o�1�qEI�(�M�������p}�m(�
	�D6nzH�������V7\�t�4�f�2�z��`��.���P;~N�$�_�����j���SFB�`;6�y�2���3K��� ��������4�Y��7���p��,l}��OuY���ba������e� �S3V��� 0�������B,t��qD&XC��nk>����S�PLG����
{$���#�u�b v�_�A Eh�������p��|t�F��v����;
\mv��ml�>�M>�f�a
�?'����!�E������,��l@����<���l>���pi�!G�Q���1���G��;	�/0��WoU��Q���o��V}/�[�xQ?�}0���tq��������=�}Tr�)�R���a	��as���R
���j�M��������Sh��T��������|������4P��+�x�Pl����������2�!k�Yc�jm�&����Y�9=����^���(Yv�g�n���(9z��U��������T�f�@�
��*]�*[������!���*b��fsk�94w�L=��$�Z5���;��&�m����/������&FC�ze_�z�'\�&z�p�#.�i����,'���_����]*%tM�A��o��,B�s:\�(@%O���1���s��O5��NS�.Ppkk;@�A��%�2�?����-�������}��A����VVE����7��fy��0�@5tv|�6H�����^�-T�+�,�M���E�{�
��qyQ(x5��+Cup�C|�����X��.���
�0��~����aZ~��_���������#j9s�2G�8&�-;q9:�z��x�����4���h����H�k�p6R�rDf!�x��n��K���u�%5
�$q����8�T�����>�����[���[�k�^s
FV,���1��=�!t��%�����������9�}�ey��H����}���i= D!�t5oz���q>' ��cUuX�k����\1w�7�����sq/����C�_�gX��]rU+Z��M��i�����k4�8�~�*�0cH�%li�Sx��E���������: �����z�(��)F�Q����������k�:�Q�Jg����C��[��m�����(�Hqu��1B�M�fF[h�N��D��	H��{{g?$��c	�-mg���`�F)�?1z����>L�������@N�6b�E��`�7#����9�<�����/��s>���bF��-�L("�
�������'en>���bI��:,hoB	�
"�4��i�������ypn�;_'l��s@��pkqF:�p�9���
?�E�=�"L�����T�
�0������{�M&Fh9�V���\���C�w�~3��+�&�t��`�������i�4���q�Q������p��Q_4�5v�nW�zA;��2�h��r�S�#f����x�6�]�}(���F���	Cp�8T�4��;��va^��f[���E�Y�r7�K�h�7�R2/�����6�?����(����7g��&�r%�I��Sc�3{4�KRF�����y��������V�<�W�8��7�9b!l<���o������ei�H����<�	C�������i��t0����f'7,z�X\�J���^��\�\�9��[p �0���:Z��s�W(���8�����)Y#j��h.����ch��aU4��8���Hb;8���>3�2����?�T<Z�U{d
5�����0����\�~��c�/9s�4��%������ 76�r�B����9�I�RG�������I��b�nal?����A+�v��`��k���!y���Cd���dJK�`����0;`����X�9���������W��5���F7BW�|��	]����@k$c���
�6U#$���x�����������������/�:���2��0�*o�>����
�?�.5��W�sIy��G������A:LM]�<'����F��Y�*�Y
�������!�.�:�>�Z`]�����b���1���GY���Le
!QcS�k�f��D���A�^��Z���VlTX'5�&5����:�Z�UJ��fH\g��9l����D���Gs tl���r��������b�PYI�
7\	t��W�`��r���ZD�'������-�l��A��EksRG�
H�P��O���U[����hC�V��Ma��lBc�]T�BIql0
 ����&h�,�c�Y���g&������W�{�x��hE������$x�}Q�h�q�R�\Qt���"�*%�?�X<�
T<��:�gZ�*m�����g�������@�c�
��B[���(v>��VH�O`8��/@�u,3�e��n���9D����v2b�������LmA`��������Q��lILY+s�7��#)u�J
4�"]F6|�K�7�4�W��k�~��=3_/��`_8[e��Qt�xSa�E#�C�
@��qS��w��em�J���J�#$�`+'k:�E-A���.Q�t�1+����J�c>],d'�95Hs~��E[��Y�6�3*���V*�x��4�����`������7����(���^@��]�o���H&����QS����(�����f���=��S�h�0W��+Z\����&T���|��?���!��a��_����*\���y��@�i��8�r��@���?S:�L���h:�&LK����{K���Gz
�wv�=6��u	��h<QL����p-���(�����w�G���j�����5�	����F��A��s�gU�"��w��|st�.�@��T�$��G��vC��~���l�:f�c6��g����`����0�pQ������}�"""M�X��`5'�:�*���h�r\i,
B����
�T}e��F%4I
P1 �R���}T�T^��M�n/5v�W��\x3�cx��~��
�$lB�ki		-����4[��QBK�����C��������������*��TYjm��[��g�����x���L�u���F
~�J�XX��;��Y/[�O*eu���7���Ea���%����B:A�KlJQ�/,������NJ����������WC�����pM�j.��<���3��#�4%�����+� ���Z�����2|��Dr/�����as��6�U
�(e�M�G��a�8�����'��MX^l�F��\8�-�t+mH����q��k4���CWV����b��L�55 -��4�r�e������P����������R�A�F���Q3~�>�9}C�Qo�!CR����� 7�|)y��i�x�����<y�(S^�V�6!�w�)�`��vu�V
�<�;����@y�Dq��w�B��[1v�a�J�?��e2G
ySPg���S:������?�>c4gj*������y��(�����80���/z�� 3Q�}\��G����3	�1������'TF�MM���� �G��q�zCA,��u9�����De.:��{��68��#�2K&%!��.�p^g�N�C�����:��?�U�aZ���w�	�.@���M��b�����I:���7��}�1���L�v6�tA����{���a����B
�+)��C/P��7-m.n������L_e����}f�e���k]rc�c����_g�6�,]i%���B��'}<���������M��93����������Sp��q$���t�)�P-�s��B����c#B{�Z�^-�/�����W�}�@)��`Xh��N��c28Y"����'����OQ�+��TssJVP��U�3|�Hx�%�"L
~fM$� p(<�'N��I�1$��X4�5u����G:U�
��ZfL�R� ��2�v�$w�I�We�d��������)Is���?�B���|���g�2���7�83�u��R�!#4�l��=��0_��P�/8���fyx$�P�0��R�e��
�����j6��[mk��]~Z	����,����	�
]�N�=��������PI�MZ���:�O�s�I%&���H�2�x%�Ig���<<"��jt���S��t(�o:z���sl�
�8���H����������/=�(O,��=�G���NUV���1V���@R�Z�v{j�N%4�:�#[�Ga~� ����{�Y�i���l4��kg�:us�4��M<�Iq����sN��i��I0����8����Vn�[H{�(�yHG��g%�����J���9I��z&o�X6S��j��;���U�p�MvK@�^�T@>�*�2��C��i�'��vA����l I�	Z{����7����An_�DDu�K2��)�1�*~�N�N�I$C2��~��5����~?a���ay��*��p4�>�Hz�"����S���[G�	Y�Rb������1�a"}C�	�P6����_]+y=d.ko�Y�p���
���Tw�"�&$2��Y�`����L�����M�)�F3\?���*�B�6{	�'}�0�u`T�p�;c��o�Y�|�(�"�����NmbY'�'�D����0��9IL��S�a4����:�����A�.��h��Y��hO%C������xY�0�[��{�������`����<��	�?�4�����{��<�3_�����"i�s.����'+0������$��2�Qe\1�Bg4�CO�
PD@�r�0�������i�G�;�����B����Z-
;���������j$o��j��?���T%,�^�t[H�����>2�p�1��X��T������w��,N�`+
��<(^���E|����I��������.��w�`����0W�����o�N�@x����������+����"�(�na�K,���fO#�k���zdJ��_��~�V;��'�M���D��l�M6GY/���������Z�Z]q�����v�����Nk/inu���n������Y^__��������o&\�F�0Z�?T�^��<9�:}u
e����Px�1b�������'oO/�i���#��m�9f.i�X��M/@��R���~�S�z����l�[�P�U�[�TQ>T�D���h&�T�4������1��4�J�f�>`�����f
�q��9��r�����Q3�O����x��W��������Sv��X
G^�3b��������`��,���	=c�������)�����;.�7�;���%�(J
5T6Q�!U
��dz�hI	j��x��JQ�����_5�y�N����hG���boT �d�w>�|b��$�����uQ�I�*��`O��}M��	�<P�sM�
��l��4�o�"lN�;Y����a#b��c'�I�UF�Z�*�jOt� ~f�w�����s;R����'g�\#W�����@����|*=�"��k)�K&Pt���G��|u�B\t �h_�#����i��]�@
R��8"�[������@pfP\��Xa�H�+
���������
�eM]�8�V��BI4���*vG�F=*d;�=W�
� ��V���x�1���F��+�,Wu6�Tx�?�#��\�i�)�#�����%�z��G �����g�
�o��XB����Xf)U�c�X��x9� -#���Y�;K��R��z���aH�I&���J&�*A3W�D>_`�����f@T�f�T��hr�u%��m:�
f�W��I���owj���l�v1!�!�*���/��GI����O���C�R�f��z�`�9�Q8�Z��`�:�*� �������P����������+Cu���$�rR�=$���� +*���C�����v�C���59-,�@�*"���U���o��mRHw�����a�]EM	(,`������K���8�$����M��<��r��������SA6N7�z�lD!�w��Y6�����NA��a�X�
�uxuzv���*F��b�+�# �k�����u��P����iC�����Z���9ZO��S1�l��:��X>�h�h��pf[�W[�u�-�%��������VC/u�!�m�60%��
<���O�m��\��s��ew���\I��q�e<,���������r�a0(Y����,���N��A<���f�e<����b�)��m�B��u���E�)���X��^���J��� X�I�8?A��Pj��0�V�@`��KA��D�8)4P�b�H��(�����*��E��\�$O��*���^�AU�8�������D�7� .
�F���z|��S6���T�.��w&�KQA������+��	����"����j������%����W���'��ivM.:�{r��)��D��|'L�(��I�P1S�>'^�2���M$r->g���oo3�������i��g��������?��{C��&�]��iB�6���-^�q���%U��) s��������)�x,�H6�=����p�c3��%���J��Cl����)v�!#�0@�E`]����Q%���@9�������Om�J�������P�%j)t,���dD���x�bBMhb�ofO�*�F�w�.+iC���D>�kd�z�-h&b�gun�@������A���V�  X���h<���O#�'����J����l0���ut�
T��h^�&��h���Z
'�=zp1��A7��fh�:�eJ���u�,�q�p���L��X�85��o[!����\��[���3>��A�x~��D�0	��\���eE���Sc����o���D�U�T��a����c/@���eD������X�%l�$z�Hx
����#���l��uiC�t:�6k�:
F0O�IU�R��F��%��$;�y`�a�����~�f>���+������_�����#1���l��O���4�Ka���1��z�E��w�=�T���,(��k��`����zW�+�nr����RPa;'�4T�n�*����*u>����,|/{��������XMB3��.jPa�CD/��g}(%=���������2T�T�����%�n��;%k�uW����Gh���>�$bo 8@Tb9�j&�n�������k4��0�^�R�EN��^?�0����,9��������L�a~��l�������eU���x�N�dp#N��4@J'$@������X���U��%�d����Y$7���x�����xd�k/� S���5�92�81��N��$*�'h�e���>9�M���m�g}��Q���Z?�)��M�8j����L���;�M����-6�o�vu�v5rvU���N��%�vI�U��h���]Q�����U!G���|�ZL�i<���~�'=�ssz$��*���]u�v���'�=Sc���+������B��q�)��$_;�#T�G���M���5���Q��K����0JQv��8LP�7�)*�u���u��*2`��dd�������2o��������]I�$�	���C������W�~=����0C���w������d�n���Yr��J������[3��������\�B�c@)~&$��&YV\�~$�mE�k��$���}�w�^ih���jq+�)4D���������Nc�U��������&�re�����G��1*���#�1�u���u%�G��S�Y�s��\���x�C,�|���:���A<Y=[s�.�e��96���cD���b\�0���R���������K�#~�,�W�����U���'gW�&����YfY
��W-����b�I������a����8(c1��t�
�n��R�k1��f:`�MG~ofu�S�i���������Rp�Ba��m%�6wj`����>&��F��l	�!�����T��w�e�^��Y:����`�����Q��2����B�+yf�<�5N�)Y;[��z/������V�O���`���d�w�7�cpp"�pq�w����fG.R����2��J����1�������&!�|�h�)��3��Qj����&��Y���~�.�(%�9�;�x������%���>�������<�o�#GEQQ������#�Z��4�����s�z�`����b�
i������{P6{mw���3s�+.��?Pf�EH���B��Z�!X�<����I�dNf
v!	����5k(��s	��R���Or�D��.g^�����|��z�uE�y�
4�(�������k�������+\1�LuN�*}���@��	�B���
��,%�de���[��������e,k�&~7���J8DO5���"N{)����a'S�d�Z�`�Y$nk�^i@�����B��"cN�����b��.�����@�&z�D���|s��t�i���/���be�e�B���+�����tc����R�2i��"�~����S�����QH\���W�$HF�%�B�b��O�]������_��4�;4q�;bu�����1CT����M]�����mrF
���!!���6��(r����.�������EK�S(]Lh(T�?Uo�4�Le�g}�5������+
������_y	NGp�,�������,T���w��A���`��w�����=�{��k�}s�4Akq���&L/X�8T�� pJ�\������6��-u}4[+�<���<(��.N�9�� B���5<0����q ��|����l��v�VJ���Y���������-h
���`�)}
���E��F4�30��}@n�.�]#[�Bc�,�o�B.SOV8P��)�d�Y�
�"T���KHw0%�;���
���Yr�x���sZh������b���k�Tp��Gt��
�&>�1r�)��e�h<�{�"�L�����1M�4�M�0��b�S����C7������j
`<�g���*3�����~��������0�����-j��Y)C_���HX��N;�����c!��Y+���6�1vG�]��i�MB5v��(�]��u}o�����$e���Ir�r8��
�K;xzaad<���k)z�BH�d��?o�������5���$Q[#��tR�g	`�������
��i���N��������u�/�`Z+k��Of�����13fTo�hM��H��fN�c�U��c�����B���}�0���X���k������p_���H�J������}��l���^Z�P��y��A�.&e�B?�2!�*�������u)��w������Za�u6�����F6�rk�#;�����i�[��C�6$�)� �5���'�D�s���N)O]��8 pC4��d��U���WT�bxI�`���*��2���z����c\	DE�������(�����R��-^O�d�.A�,�{�4y���30\�:�\�N]�:���wrZ-.����"����O�� 4z!�����:y2�C] R���"�����U����<I�)0=���vM��N��l0��-d�+'����l�|'����I�#P�K�U�:*]���h����k(;P��-pJi�'Ozwa0��Lk�V���g;hl?h��{;�m����3p(Z�Er��vx�-����n,�d�X��R	�a�]3`n6{����f�Z\�����v�DP�
��;��v�d��N`T%X(�j+~`R/��K����ax�eF�}^�hM\_/�8vw0�tC�nK�-�+��[����)�em)��u~h�k�0�y6�������X��������R�\�5�`-Yk�`��_XV�����8fQ��LA�`&(����������_�5�������`M��=�����������nB�����:���8�dr��-���o*3?1f�\�'���h
����F�e�N=v�4T7g5�������FT�����
�D��>��9�����'��5���������y9�RG��;����2�@�����f��f�� T���ch�q���UR���.ea�O��#5+���\}y���C�
�
,�����g�H��[��|���{%�kG~��������1�����g�Kj|E/Q]��5D�7,�BN���8�2����{r/O�5��*����������/����/un���^?z%_mXY{���t�������F�83W��(���$��O��!���}�]�"��+�W����ood��SIh�	F�RL+��P.bBST4�<����7��_*��"V�"��e�%�%�Y�Q���Z����B8{�?�-��7]�u���.c8�v���`I$�� �*��nM�R��>�p�W�@8E�{�������
��%�0������g�T���[�J�n,�������i_�u��G�'��'?~xm�9�T��]�MT>:����pZ<��0�=�qC
��}}��nC�_��p��~IN��k^DQ�*��Y�:z[���#���.6N,����!��)�
����v�2E���#P1F_i%7�Er������9���d���x���k���O��Y
��:��5Z�:���
 ����H�dB�Q�N���-�_g`	L�w"����$����Nt.�kIur����p�8-���(��-k���?�m��!��4�mP�nh���*�u�R���?%��.y$�����c�$��c��G�nbI�{��dh�M�O�Z��g9A3�\O��7�"��ZA_EX����X+b��<0���|�^+t�~e���#��~������Y"�_��R ���B��RG�s�����V.�v��2(�-V�_��n�@�j��F��^a[���S&��a|	'S�3}�u�^���s[�q��R�����O�&���g�n^�-<g��n3>����[�{��{qok���/��-�T�-�!���
�R��!�
M`}V�!N��<�	��������<>z{t�~uz���
y�i,{~�
f�QE���M2I)���0��(�R����G���A4w���j�c�����^kk���nml�Z���~k���3o�����j���F�����9.���jNG}�/�4"{;�������}~�2�����*�������%;A
^�=?�*�D#z���V��*�	zr�9��w�nX��Ht;{q��w;�{����q����{�����`�L���y��'m	4�=�,L�t'G/K���:>�Xt|^s�~lb��j5��$�L6�x���QX��v���v�s�t�ah��n�z[�8��#w��-2��0���������&����(t�������b�p���|����6�%�8�)]x�������8�U�Yw��FTv�jAV�ZN~}pl��4}=w���4���I1pc�F 0�%��V�*)H��R/GAfa�M+AA�;C]n�W����6!W�/���p����|�N���4��&�W��)oaZ�qo�H��9V�
��!O&���k/��}��<�h���5�������������EZ����c��i�H2��mk�U��Z�� ]O���OV�>���	���=���g����-h�L4!�)'��p�"J�B�pg ��Y�5w��g���w[�+w�*�A��ihY����u��v������_���Gp&��iq��H�(�3�������5Z3�9Z�`X��)��U�1_Hy�����j[Q{�hS��J+G��@��h:�������,�2�J$���=�ke���9d���e����1�0����G�,�Wl��h.������|������>��y����`��3���4=��T�sa�	l�s���"�i�#�`:��U�^���>��
��o��I���&\oK�^����6��j�9�_�m]q����K3�����>���:���������.6?����mT��3�ylx��#��:?��*��p���V�E��������4��$}��0_�����k�}��f�*���mD����IGI-�-��.-��/�I$DK�/�L["h�����8
����wu����@4��q�msN���Y��b���5�K�t������5�j������{C�H��+;E�j�U� @�(L�t�F�b���}Q+�l���k�V[�7_e� ��>%��B����X����}N'���T�$&��#DK����"�U�X*~��jV���%�?O�2�X��nn��Z��}0C���{;��s���yY#�����@}�;�D������(�n?�Me��;.�dA
,�	%�.dA���r��b�D��
6ZN��5��c�4�2���C��44���c!!�
)t<I0�nf��wQOs��
�w������|�c��7�I�f�?Y��n�vvp�f�Is�[�:a�I�.��L�SPp����"��g�
����>^�����[R��#�>$���������c
5i��O�'���Ai4��7�g2�x<�(��Or_�}N����<XY�4�%��� -�h ��0��%O)��RG��|�����D��s%^�z��O����o�p����0#���j�R]�
3su�-���e�����f"K=X|e�RC��k*�k[��=D'Ha;��B`����h��J�+a�������R��ReR)
��	�����;��,�4�	���H�.��C���T4�L�!���d�p��,Y��A.��07��N�|����B�w��������������.xC9�W-[���q����!Zsv���C�6MJ��!J�\�]YC	���z���sH�J�>KF��@���k�`����!��� �� ���JM������+;t1������7?&�f���V%��� �r-��x����	�D`���pC�+�b���A.��q��SE�����������)����I���R�m��ro<?�����%���N-$O7
_�<��2�����X.^��$�����6h�Q��#��l��Djh�`�B�K[��!�j}8;��'���Z%�t���a��� ).u�y;���l&���������N=A�iY��x���>��bJ��������m����
�i���sF�kl50��L�_�Y%����9�G\7���"��,k����R���3�"��F������S�M�/Ybf��U)�\)���0�8|7�;��l�F�mh]��t@M$% �G&�
����
0��NVO6��U,�V�"qB��C��%@"���)f��X����A��|t��/�N��>�o����Un@A������]Vo��L�N��J�3"��������m�L��v+L~Kql�z{���a����:Hvv���A-�Ea�J�E�5�h�h�6��,���\�q[���6qTZXT����ox�@���(a���q�����H��&�:
 �S�Nw�:�~�!+(�q /gu
9n�c���aJ�/qp}��p����VK�$h��h�]c��7Yu�����x�k�'�W�@��W�^��@m��a�Y������i�v�Ig�nllo��[{����v�3��Xub�����/���q���F�'��� �BuV?���LU�	9*�P���}A������u����t=1jK��k	��8Lk�1@�D�	s��|��,JO���I�������`,���1��>�����`	�,H���Y��������o�R1q{%�&do-�2+,��2�>(����C�����KL�E��*/�|��{$2,�K���
t�<)�oF�c�C����$��cZaF����VjH�z?cO�� ��Y����]����:������s��/��$$�C}�����|c�{K�3W�|
����s���^����aw�����q�=��[��N�Y��Cc�q�z�v�9
�6�J�9��AQn���
���@������UL��-\���O����~}t����{�G�hC�j���8g�u��#�(M��?^S�p5�a��nbl�&#})�������P��?v����U��o	����^����A�]|I���9�/�$���s���b8�	f�w����;M��;J�]����$����������i%9�oz�t��T,�]�=��@�nw��B����������D��.��BU���������.�����X�Ly�W]$�R^x[�y{�����m������/���Bw����t�w�.��-Rz#C�H|�l��l���
wr�
��{��ng� ���76Z���x7��I�q!<Z�^�C��.8���,���p��A�R�C	�*K�����8s@)���/�2/���G�K"��.o�OH�Iz��X��U�	�?+]�+\74�����naC�����o���V�����v���;��vEtJa��6�P��%��p�1�)�T�~z��5?���&�����G���~![7�3��	��/�QCL�=
�5��������<�jD�������W�����\�9I�
�L*U�=����#���5���=�`H��.�`�4@�|��������841|�������A_����G�����oy}�o�k{�����y1��n/���A�u��=���n��{{���ag�-��\I\%}(�n���(�g9�W^��q��ld~�*m��v&��p�H��cr~0_�������<��2E�z(��K���V%b�����������X����������^@���,vU+*+���]��o�%}(
w:���U�)MF���u���l���H8�ps�V�~:�#2^���|��D��W�E�>2o�#]������"������Q�{1�HWBA\(��!�c}��i�*:<����O���r2\��H���-g#*aG�e������H/��c�I�������y*�UDmC�b���uAN/z�����t��G9Y#�O+���(���|�������g��&�P0���9���E���-�@��;L�?|���3�����
��3;���
!h2�$��h)-��C�6~���q�G0	?AxSc��?�	�T��F��hh �9��������dno�}�&����F��Z��{���������..��]���~<��?�L�zb�8v{���0��H�����~]
��&z���_d3����g?�/O�����
8R<��P�voN�^�\`�hE�=y2��+�77��@�B`��7'�o���\Z��mF;�V��WNl��qM�Ap?I�2O��Q�+��
{�4�f�������2
X��q�:w��@�:8L=~E��@\�G��QJh�J�?��#��)�X�����KC�������%g��l��3���JZ���J��*�P1@��U��3�;��:�7<�9����p���3��S�i��jQy��:���DUCj��k:��hv�{W������U����L]A��*�Rb9��ek%e����PbL�
A�q*��;}�`.�����C4�����������m�0����$�.3Q�>p�
��� �����>_�4�ns�m�����
�SN?��4"<S���"�{�u��q�ZB���2���V`������T'E��-���/��4���������wYY�W4��r8��[��	Z-���v����
�U�����"��O�N{/�$�\v�3��v�&�{\��`_�Nc qJM�S.��\���N��
N+� a:��j�u�!o8t�c�3��
��5V����zgnL�7%��~M�&�gf���o��R1���������yVpB����Z5������|M	�m9�r�-���!B%c%����
��C�0����ln-�h<9"'��o������S W�a������3������a� T,C��)���UvX�
#��T��O��T�eY|��d���j%w#T�-eyuj��%kI�?�w:�����L}��"�b�c���S�pOO"rTI����7I�M��p�pC�IZ��)?��4!���Sp���-2�
�8��W��
�v����s
����#��zh%�+�<��GAOu#>[�3��b�*Q����Iy0�<Q0��Z�D;
�}��K�|Cb�1n�E���s��Lf��P���zW��e�nCm��8M%��1S����y����J�SHU�0����H��UVDWR\�y�$�f�d�;�e'�C�3%�g�U���i�=���X�A����A���6������<�M\?�%�]�\�JP��21�<�z5O%%���c
�W�W�d|�-��WJ�����!A���c���d�7��Q��/���/�:�����ih�1��xD�*�bK���c�^����G��EP��)�7P�����h/tP`,���&���)����'��	���k����H�|�tpq�1K�d�����O�>��#����5q�9�����C�(��x/&�T�H�"�A�A���L�71K�^
b�Ew.C�J�A+h�'oL�J���X���#(n�E��0?Q��=7Q�#2����}��WF����J>]�c�+��c�+��c��g��Y�v��O�����R����Z�}`7}s�������r�����
���[A*][N�F�%�dgb���|��
��wq%�V���8�h��VC����e|��3J��x�������'�A��!*�C�-���-O��Z��x��a��HZE-h��"0dwG��0*�c[�r���u�J�|������t�u�e�������f��$1��������'�o��WMU�[{
H�P���+��sQ����3��w.�-����l���n\t|�b.����@pB����J�h�RT�j�3)0����Q��!�O&���pD�_`(N �,���8.�������t�)��Vs*��1UC����o��L����)���R2T�����1�Y���C�\���������O�~�6Z���q�X��S[AL}{���zFi�����q��:|�a��<Gf��L�pck���J��VT�"�Ra�6[�^W�^�l�C.����^�r�����4L����3�F��3�E�U�U���Q��@,���E|_��@�5��4��<���8�x��-���'��#�����	2�T�tX7�Bp��S�<�h�:�5b�����kBZ$�Q?:�z���7����"f�c9���2�8x���Sb".�.��LYm6E�]5w�"!��>U�J2XV��]�������d7~��K�\����1Y�����C�
�K�`���G���4/|Y�;>L*�F,��Vr�_�|�Q����� ��9<UK&�"����r�M�	�B�0r�E��A����Hi��=]�~�=J��8Y�A�D������S��)H������$�����4���	����P�j��G/��"�RJ�6�^$��u>��o��-�M����|P��hR�^`���i2t&��H�Og�O��'z�������K@���g)-�V���y�v���QRc�]��#Y��-�7@�Qb���6O�XQ����������-vYM*�s�pvD�q�����M���	v����j�:�[k�8�ya��i����Qg�n�{)$�$q�|a�%�K����HA�=���6���}�g�jrc9�;<Yg�W��r�w%>�U6_�f���H�o��SR�>�-������sz��zD��^DA�|�-K�g>�q�.�#����C��:��=H�V%`�<��b���15�yH�C�.&A&f����VT�:�����@'�o����T��~1��DSf� �n%��p������pwJe��0{�D'�P����7<��RS�%{aj���Vd�00�a�C�
i+:I����YR�]8`���#&,�B�|��B�0H��;Ha&(z�D��������G%��u2�`*d���eH�{�4������h��(	=�_�K*�!:
��
���<{�P����x��S$wa��Jb
�a���/�����l���d�"�NL4�
T�|B�������k50�3Dp#�9
0�3Z��>c�.�8����V��tC��13
��@�7��t������<����M�i2���(�e�$����z������Hp��l��
���HD���%��Z�B8p�Sb�+s'�dH�K��C�)w/���1�Nq���/P1�g�~��:����d�#��(MEi7�~��ZxF��&��F;�{���'�������X�� h�='�������nw�bN&I�����B 9�s�!� �������N�d����G��Z�#?�'����Z�����b��U,�y��*�
�Z���iW,��;kp�TG`{��{c���^G�7M���������H�m��H��n�|�8K��E��
V��*_Y����L�4�
u?�q��L���#������Q<"m��c�L�+�%U�+���c)������p�ag�n�GS��q2TD���1S��3%��n��&pq=������?@�������g�����v�����D�������HG�r
vJ�X�<�4����>�d���#����*G�ZJ�G����9^��rB�9��P�����j�6f�����bb~������=���<9~���/���l-5��<}
�d����0j��l8��iR%����e����3D�N�3�WwAk��W�t�}����������`���O�b?=>;c.��o���~n�9;?y"�{i���4��b�N������U�|O
O.���'�-}q~����'W��%�������@`���;fji�b��H�Z��n������n���u8���a�dMq�p����g[�*��ae,F��l���SC>C�>����Q	�G�����)�+W`��d>#_4p�" +���^��WwB���7�*��JU�U��w��c�M�����S������6h!R,����w����
��?/!`>���a� 
���A�#�GA+�c#F�������}����e���l�/ak����
b?x
[x�9����E������ZH�9#��_��z	_�q��Y��0�DP�|��r!jj���[�
���F��/�:A\��&����|P�k����h��H���%vT#,~���/�>
%pY��z�L?�T_*S��sb����!?F�q�H�!�o�~
�pL��h3����&2�':�e�K��������or\��w��U��(��E%C�C&��D4�����P8��!q���X�s#�o��V<��9��t�@3r�'c-nP�w*�;!m����S�h"[���HV�@4���r ���D!fd�yM�K)�6�8�^���x��O�_�,�3K�����������2p����)
��7��4.J�����K�F
�te}��W	��
�����V��ns�������C4�����T,�e��{)k�,&]f�� ���|��/MN ?�\���p��S>#�z�|�D�H�����&�d5���F���pi��f�7��	q��������O�}(��vBVb0+����IJb�{�:���5���vxg�a��m�L/���`�P;���b��Vj`�.���j�����'�
H	�-�#.xz?��._�6I)�W�d�J�����M��7��Y�:��"�	e��4��Q'L{�� L�i�I�����������xCL�	��|����C}w}���3��~>{��I��1�G�)�9a5i�������R,D'����)c�/}e���GdhE�g� �tyS��zL
	K�P��H��V�����lZ*���0X[.2�������'�*{�8��	�@*����p��0����qKyx�TK6[IH.�[D�kp��x6����/s�i���1���,	G�4�O��<K&�0���������m��/��}��_������
/����o�?���wnr��C�L��(6)~4���>�/7�Zd>�9��;�~��dL�������tfk���)�fr�g���\�~|���xXE|X���hkz�"�u�a�>z����}����>�g�l���}�������=I"
parallel-hash-all-arrival-phases.sqlapplication/octet-stream; name=parallel-hash-all-arrival-phases.sqlDownload
#25Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Thomas Munro (#24)
1 attachment(s)
Re: Parallel Hash take II

On Tue, Oct 24, 2017 at 10:10 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

Here is an updated patch set that does that ^.

It's a bit hard to understand what's going on with the v21 patch set I
posted yesterday because EXPLAIN ANALYZE doesn't tell you anything
interesting. Also, if you apply the multiplex_gather patch[1]/messages/by-id/CAEepm=2U++Lp3bNTv2Bv_kkr5NE2pOyHhxU=G0YTa4ZhSYhHiw@mail.gmail.com I
posted recently and set multiplex_gather to off then it doesn't tell
you anything at all, because the leader has no hash table (I suppose
that could happen with unpatched master given sufficiently bad
timing). Here's a new version with an extra patch that adds some
basic information about load balancing to EXPLAIN ANALYZE, inspired by
what commit bf11e7ee did for Sort.

Example output:

enable_parallel_hash = on, multiplex_gather = on:

-> Parallel Hash (actual rows=1000000 loops=3)
Buckets: 131072 Batches: 16
Leader: Shared Memory Usage: 3552kB Hashed: 396120 Batches Probed: 7
Worker 0: Shared Memory Usage: 3552kB Hashed: 276640 Batches Probed: 6
Worker 1: Shared Memory Usage: 3552kB Hashed: 327240 Batches Probed: 6
-> Parallel Seq Scan on simple s (actual rows=333333 loops=3)

-> Parallel Hash (actual rows=10000000 loops=8)
Buckets: 131072 Batches: 256
Leader: Shared Memory Usage: 2688kB Hashed: 1347720
Batches Probed: 36
Worker 0: Shared Memory Usage: 2688kB Hashed: 1131360
Batches Probed: 33
Worker 1: Shared Memory Usage: 2688kB Hashed: 1123560
Batches Probed: 38
Worker 2: Shared Memory Usage: 2688kB Hashed: 1231920
Batches Probed: 38
Worker 3: Shared Memory Usage: 2688kB Hashed: 1272720
Batches Probed: 34
Worker 4: Shared Memory Usage: 2688kB Hashed: 1234800
Batches Probed: 33
Worker 5: Shared Memory Usage: 2688kB Hashed: 1294680
Batches Probed: 37
Worker 6: Shared Memory Usage: 2688kB Hashed: 1363240
Batches Probed: 35
-> Parallel Seq Scan on big s2 (actual rows=1250000 loops=8)

enable_parallel_hash = on, multiplex_gather = off (ie no leader participation):

-> Parallel Hash (actual rows=1000000 loops=2)
Buckets: 131072 Batches: 16
Worker 0: Shared Memory Usage: 3520kB Hashed: 475920 Batches Probed: 9
Worker 1: Shared Memory Usage: 3520kB Hashed: 524080 Batches Probed: 8
-> Parallel Seq Scan on simple s (actual rows=500000 loops=2)

enable_parallel_hash = off, multiplex_gather = on:

-> Hash (actual rows=1000000 loops=3)
Buckets: 131072 Batches: 16
Leader: Memory Usage: 3227kB
Worker 0: Memory Usage: 3227kB
Worker 1: Memory Usage: 3227kB
-> Seq Scan on simple s (actual rows=1000000 loops=3)

enable_parallel_hash = off, multiplex_gather = off:

-> Hash (actual rows=1000000 loops=2)
Buckets: 131072 Batches: 16
Worker 0: Memory Usage: 3227kB
Worker 1: Memory Usage: 3227kB
-> Seq Scan on simple s (actual rows=1000000 loops=2)

parallelism disabled (traditional single-line output, unchanged):

-> Hash (actual rows=1000000 loops=1)
Buckets: 131072 Batches: 16 Memory Usage: 3227kB
-> Seq Scan on simple s (actual rows=1000000 loops=1)

(It actually says "Tuples Hashed", not "Hashed" but I edited the above
to fit on a standard punchcard.) Thoughts?

[1]: /messages/by-id/CAEepm=2U++Lp3bNTv2Bv_kkr5NE2pOyHhxU=G0YTa4ZhSYhHiw@mail.gmail.com

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

parallel-hash-v22.patchset.tgzapplication/x-gzip; name=parallel-hash-v22.patchset.tgzDownload
�}%�Y��{[G�8|�E����$#	�8�.��fx�lN=#M&�f���=����u����	�o�ox4�5}��{W����N��~�DW��v�1w��U$����R?�fs��	���g���O�#Z����l�{��h���v��D���`��"���2[�a��4�L�<W�0����y���4Z�=��G�L�Cy�(���(������G��Hr��������������{����]���[������?���a0���q�;�v:�fg������vK���;q������m�&������@4����h�T����
fN$��R��b��A���2���*��1f�*/�X��s'��v_�-.�m�w{��^Wl�r�|1�I��]������k�
[����R Z�-��@�-��@�����Jeo
�Z\^%O"��R8�P:�-���������2�dT���+*�-��9�7��s��e�
�$X���&�|�t7R�F���,�X�Q���\�B�ZDs���_�� �0���x#��[9�����n���n�^�WD��px[jE��G&�-�d#X���i���*?����
~���h���*�-&��>�r�K��Dw�-`�e�NE���J�I���^,��F?Z�Q��]�QL��;����m4:�^�3r��@zZD�p�Vzg&��{��Q�
�{�����vO��7W�
���f5�S�'Zb����NET[7�FET�s;���gO*�����A����,���(��y�P�!F�z����OE�	73M���|8�X=���q>�i�7�O�xM�2q��W���)�7 �7Z�1� o���.�#aj�[q{p>���Q���R�M�v���jI.�/�$����U[5�+����; ��s��:L���x�����f�x�d�|���w����mS+�L�*�$����k�4�z�
!��G�������[��&�pM��	g/�h��Hy���n�]��wy)�!�4���i|�V]7 \����*:�8�v���l(}l��:�O�i$�7V�R�.L	�Ng��7&��=��%t�������1S$ L�YEM�/4
��}'~ji���E
�8�s/��������#@X#�@#���D�+9���BM����?i8�t�Z���-0�kI�)r�-a�"���0���
���#r���F�b� p���������������v��Y9�H�.�s����G��s���D8��x�&��&�^���	��|
�2�!���@���GBa1��FX6lV �"r.������Du-'���2��}PQ�)��|f7m8��~� �[������y��~��|��'��M&�2���Wo��i���Y3 ������*������}/N���3p.����K�L`6����	�w����I����+�a�C���c�j�]����E���!�4AH�����c^�+)g���G]�u�o�D��'���z�Yf1
n0�z�^���iU�h����.$�?��q����O�^�
X�z�(�)�h_�3�q
���L3��XR����
��y�w��-J���F%X����(�e���� �?/��,d���"Q��+MU���0���A0�]�E����S�V{����z\F��~��&�G�?(r~(��<rAv!B��x��������.jU�9)��e��AJK�ZZF*l�88F�����$�����i��\�F�M@����+Rol�9������d��zrx\bPo��_}�Y}�Q������ZN����i<�d�+p��p��0�>����D����X	E��C��[�D�GP(�va��,�!0n #=Wr:��(������h�Ac�
r��`.�G�����U8��B�p�f`S[}~�5��;t��b�z�� ����/M���K|��W"��rS��7�����7!�ML�����*�������;�?�E=�V���c�l)3�{���3p?�����^�W���8q5sx��'�k5�g�j�L�1��P�H�D�,�a�@�
�S��k	JI(�A�M�����G�K��0��l-�c���^��~�V��\�{�f/�R�bD����2R�o~��y���g�_�� f�F�������������j�WC��3P*�2(U��y����Q�?����VV`I���� @�zj��X&[hD&�#�����<�3SZhi?W���vw��:�^��h����8��;�����T��j����N#:����Y%��(n��X����������{�x~�(�?�<�k�G����_>���X�?b=�����h��C���;���?o�����������������G���~���v��������+�*����]��/��u����|�k����������h�?����WZ���~cSb�������k-��Wv��n���V������y?����)��S9a���>��{���L=��� ������@����Q6������^3���������k�P��v{�n7���?�i���v��4{���i��z�N��������\\-j.����muw��e��&��#Z
��$`��R��TN���s�~,�� ���{�;3o,�_�)}(=�'���`,#�*7��]�n?Q���`�*�$�H���c�_�``���@�����I����r��pb ����@<=9?|��t\6��9=�U�LF��7���X��Fzn8�[3������u�������Nx��S�.��������2/���8�G�[[�����!���������������k�N~��]?~�f�3t�9�������N������g���_f�7m\���#Ib#��[��� �ny���V0��!��m.i��1��@oV����������\����[���9]���[��J��u�Z7g����P�iu4�e�)���t6%8B�<�w�^�\i�
��l����N���i;�t�w������h�w�/����6C�v
V��?����P*�Y��4����v�����"�b�����6h����i0r�Q��<��9�p����F���������)�:���"��w�R�u�~�����!�\�b
�6Et�Y���y!��~V��B�pM����1>��P�4Ly%�INR��6_����C�$�JV��F��t��l��;!�r���2����	q�Yk�����_*�[�W
OZ%~�1�/��_kB<����3�9���w�.�i��W��[6!U������##�A$����G���x*�4���/^��b~�nM��Khi���.F��f��z0p�s���?�|�oa�5Y��<���sx��K�Y"�{s�����n|������5aq��>�ez�F
g��,R.���M%��������!�!�[

�5��$�I�t�tB���	_E�?�v�$v)gz������>sn�/ql�*5�d���G�R�?L���4�����z���0k583a2A��<?�"5C/��-���Ck*�[1�#w�I�.����F�1>�^�������r�q�,y�r
�qb��IJ(4�
T	oQ�CB��������H�l�SZ������Ba+[i�9��}8  ���{��^ ���Zb�
6������^M<'��W�asith(�����,��\~N�C/=?����� �z�<��k1�w.#^7��5B�P�%���Wc8���4-�������C�m���vJ�pq�/�x�[�B�0� �"�
�E�%�0����Y���A����Q|	|�<�M�4��!7��35P
�����!*�_7�#C~��3��:pT�am����`�\��L�����w�(�Lo�T�p
K�M8��"�	��h�H-���1)�-�X��p���\��T�����W��E)�����t�$��g�v����	����X��851�.d�v
���=@����R�+Y��o�kj �5�����!����$��J��2�f�\�$]3$�)���e@'$�L�����?ER����Xq
DW������)Z�kJ�a�ZR�)Y-G��z���8�,��=J���2�RG��>	y���T��h��l�mj��( �K{�G�X�t��4zS
K���aBHm�1[D��v��W���bs�l��L��2|x[�,���?b�$F�-�h)1E���0��HJ?����S!
��"��xXOQ����
O~^�p���J� RT)���>(�a!z�9�0-��P#-�M"yo��h#��"��5�Tn1AJe�Y?�^8�2�����08� �G���9��N	�y0Xk��LB/HqL���jL�!��W+'I�_O
��6G�"��U��a��?�����{���]�m)��7���?/�?�R�����?��,�}!����d<9$I(�F�l�B�c��P5�t����������`�
4�4q�	hh��!�/�/_��]�+=�nTu��*�:N�[.�u
���G��9c�#�6q�D<�;;;<8�8x������--g`�/��e���T���������xZ_��o������b���)�lC�9��G��V�������1�_�@��o�& Wv�$����C�������\���Xy���%���C��~�--�ky�CNL`���V��y�)�Ttd����(i+1ZH���Z�I���5XN�@a��x��a2�*V?�/c�}/"&IB���~wCa�u�P��`��Jn��u�|��G�������l���G��5�UjY��`���.���l�W� �-�
5��p|�w�<������X?^��3�go
���L��������O|'������I5��'���Dlnz��em~�~��N!i�����d����)9��,Tg2�z���)�Fh�jY��,z��!}���]cO�
�#�9�0�,������@C��]�y�F���r��p���s�V.�g��`�������|�d'�f�,�Q W���4��D������kg�����o�D$j������qy1��4�d�"�&����h�D.����p��!��_����S������<��H�,��R�R�1�n|�-����A|R8i�)�D��>������[rC�S����j��D������*PTD�,�GR8l�
Z^<��4��V$B���|f�#�g�D�gL�q��hI��EP�Sv
�k163���1�cj��{�'�I!�S��@�M.K���p����F-#
nh�n��7\���b���@_�NLt�3^�m��t����ln�|�YH��,���
�^���Q�D�D��u1��`Z#E���!��jbwa�A�Z���\8gd��;���1H��6��V�E����l7������M��J|j/Boc�?nE����Va\��������rf��G5XH�%|���d�����l������f�J���<6�'�A��/A�GH��H�a�����.���vIu{oj��(���a�w&%��Dlx<����c��]��('mEm��[�$���R����g@��}E����5^?��U�>��`o�&��F����PUd�vj���ipY=:y�$�5��#�q�.��m�7Q������>2����DJ,���������cM�)�:�nf��R���?s0k)�PNS�&��T���*�R���#F��kJZ
�X#�oA�����b�:�
2)��o������17/i�]n��r��W�w{�������m�Xp���k\����&@�b�Mb]�#���"��3�y����Oiq��P������\^)t$�:)s>Ezc$/AV���M�llo�����/1���L#�)��-$�Ca1sBH��+�rO�. }�\d��^�2��=p�HJ;Ik&���d�Z,8D?�8��
(s3��>�:���A�_���M�E$�7|���r/�0W �`��t�����N#�.6��_`��1�0�n�~n!"O��<�����m >����#����e�%��u���R�'�<l�K�H0��"��L.�F�%w�i������k�2��Q�t�O�"��q�O��"��R�'�L��d,
��;�xHj�r�`Bw1U�#�Q���7���}�SY`.|D��Q���i7�V�I��"��0�d�i}w��q�cMC�-R��9R����r�;�(����[l��/0P�f&�5%a&o0
=8���?BOL��]�*)��	@P��h���X$?��j���8G/�����D0L=�{)���]/�b��L�H/���|����So1&�����zM�%W�8����,��"6�L�r
���,�1�D���=�
��#�����
}��2*+�C��:�6k�~����yZi�U���:��DES?-�C�L��3UB���Z�tk'��Fq��l&]�aL93m������*�����JS`|���-�9�1n�"{����v��i���u�8c8+~��O�u����J�R��mmu��%��@����
������x6��5 ���o����G�NTNc�S4�H�������:C�+�Wh����9��O��J�N$�d�8��XO)����L���q�`\�:q���+��Zn>8;;9����=O���8�GZF��;<��).��8�Z�4�8(�	Vu^�&V���-b���[�77Q���^m{ypYemms3��z>B-���(
�P[O�������>�d�N�d]�N���D2�~;���s�����H�x�8�3e,H�>:���;��\��XQ1�qXm�2����
kJg��\������>o2#��9Ev��\�$��l�:��C6V��������u�����\�I�R)���[�#\��DY3Y��H�{�V[��(:���N��f4�U�����O�m?)�#���v�b�E���q�j>�)��O*�����
e�z�t�2c,Q�a�g��>���I���.(#��g����G�RhS6�R[����y]��iW���j�H�5��-���������3�95~���_��|�����PVb�P����nE��Td.�`I+�h���;��D�Oy�-��|�
�����$ON#:�E>`|W��\��I�,�J��y�tj��	,7��'e�3#�d�L��<[M�4�	����8g�k�g}@<�ZR�^�0���K:�d6&_�1
�J#h��9���0p�14Y�o�n6�JM������C8�]`;��L'��l�],�Qn�!�y�O���9��O���������a_@�1UE`>�yJ�nF�>a��R))�6����>��:@������`�MI<�@���n.*��:d�a�����k:n^EZ^.|E���#������La���)�n�)�-{$����)()���V?��W!��c�N���K����I#C���?�����+'���L0����y7�|��6����;�2m����*&n�W��Pd!w<fcfMO��$G6�H�L�~P����F�Fj�#�12i�o�L�������V�����J����6�/3�����^$r�&gj
9�bEq!w,l���R)���i���;r�!�R*x��Y��F\\���gx����S����l����S�@����<.�}��eUq���.F�2�]R���*��-����d�1|�D��i?U�ZM���T���|�C�,�������n���>Vg�n��{��;j4���Dv���h\~;(�6���v�������V/`;G�
c9�����_1��i���
�!���T{H�*���a;XX�&��CN�)Z�����`J��A����=��'4�N���������d���+�?%�B��PY����f���?��[kk�o��j��������}C2���(�>��)���������o�.j���88:9��'��~��K���� �W��A���G����-$���Q������o����O.������7�GdG�g��=bG��,�/����0q��k=C���UMf|F��?:<8����N���j�/3�s��K	�:%P�1������,s��<��h���/��'�;���q��-��^���{����w�R� ������|��Kp�}�����8�����/uI�A�CIy�@$��� �=2c��2gn��E a��-'f�8�F|�]	��i�a��!xS�hR�M7�Ul�u�FK�E�Q�����JZ�'�o��E�[�k,^F���!���Q�e
61+P��c)����K�L���_��������*�J�]iI�f�oU�l��
�r�:�^��V�.1*��Z/��P|�C�����IZ����������@�� ���0�n�+
x�}��"��l_�I�{W�q��,F(��k��/i��C�"������K�L��k�����kZ�7�����.!�2K;������������9��i4����6^�$/����H�)2�v���~���V�g�8���s���IzM5Y&�/>��[����X�]� �g�)�v�,����u�*gZ�^����z$/����~�Q����������$�py����=���������?���m�{�~�5tz�����q@,���{�V���w��w:��W������v[������S�!3�
-Qy)A��	$�T��:F�����
�zd�x~6������A����9*�f��c	h�|�j�����C��>��Eg�e,����;���``����4+1���>~������B�q21W��]��}L]���t�Mp����]��~�����O>x��8�������i�}�V6w��/��7o�}�~s��zxvw�@��������'~�~.��%��B�x��3g�Z:sL.K�EOP�%����3�zJ�����
D~�]��U��;D���b�s�QR��]+�:�?�}V�n������f����B�;�}Lw[�gW�-IY�Em����;��{u�Z��6]��%yu�d������&sg��LJZ���C�����tL�hy"��s:Z��![��`0r�x��hHg����Q�S B�/o-k���N��i)��8���I���>*�&�^��c�)c�
�hR�����zc
��r/If���aOOT�2��Ba���9z1>K�h.lWN�X���I'������� �������x�"��Gy�����6�����f��^��q�>;��~�����2�\��`X��Kv������W�/p�p��	�F���E����j�7�5��0|(^�'4`��M�D���fe ���|f�P����\O��~�|5����^���.5��@1��L_��Z0���� 3��=Vk����3����' �C��_2r`u���;�>�0>�0��b9�}�{�ek�[�9]S��l\��"{�k�G<2G�4��	��o��WN��4�M���~6w�4�aJ9�q��&�2F����@3��!���\�' ���&�"��2�}�x~�VI�/yf�R��e�%?��BNt��6E���2
)��v#g���'8�b�Oc�uy%1�#��T�fO��bm2�Qj�Td1��� U���
�v���������������/����&�>Jg�f�Y�y���H���7c"H'd���y���u�X�����23P�@}�hH����0g^���R��`�h��
��S�.�
fH�����^��A>^�S���z�T�T�j,|���M|��?������g�0r��(z�FA����������&��������������v�m��=B]�C�W�[k�[�#�&��d�I/��Sp����g{(Y��+a�;�I�P���2���
z�nbT�aR�su�xM�&~B$�T�B��C�s5��sr�w�_��r���
N�}��5ni`�I`}��8��K�zb���9���n6t��Y�p��t�mtd��;#9��Z�Fo�t�n�7W<L����#O��&~l����{����S5�$���Dl69Kd��G��@�����cP��72�
\`F3��S�|�oD%^�X��v�p!6a�/M��mP��$�yc���
�AvL�(9��'	z���eg�L��� 8��X�\�b�l���{���I����XbB^r�R
�qn7�w���	�Fw�O���_R�e	�;�t�1E��v�`���GK����z��h�lr��\6�
�0����x����#�O��P���e��&
��6��N�x+8��qH��?���~Uq�1|)"��-9����P�:��x�������3h�����IoU!:�R�m��j����������-K0��o����X:�"��@PV<�}�hc�E-M�=\u%.���/.���q��hh��i
�cHU3�2����6tQi����2^�b�"��X?F�Q�w�W�B����w�G���9�Jg�uz;���}�=���Z=:U�K�X���x����c�:K��?=���y��I��`�i����u�;������h4����=������&w��I���eFD���(^���{��i)s-������9%�8#���G�:!_��|���B��A����������������eXI-�Q���9��'�Fc�q�t�wW=
��s����X������Sh�;�����k�>������JW8
+��d'����C^������+ z`��7�?����p����
?eG6�r<iaE�Ih�v{��8�Q�yH�X�|��]g��L��I�,"��x�{��W#����K`���/�d]P�-E������T��E~ -!L05��`��Ky�1��e:Z�(	_�������6��b�F����]��n��6{����@���q_
jO��'�(�W���
]�(�i���)�qWON MS�'��s�K��(M
��l�&E�:�������o�N_������q��8Hs�n�6�������E\r�x�m�Gv�����!���Y1����G����O����z:�3�wX||���j�����������_������z�ly����uz�q{��4��V���&�v{$W��4�;P.��hMoo�]��v�=��7�[��:�=�n�J�o�n��ll�������������R�l���4��������&���h�{`��n�m���Y�w�a�9Q�%Ths�c���P�a�5�����W���4�.�R)j�8xr|����kE�J����)�������I�m:�
e6�U�&kt�n!���]Q6�BC^��b��(�f����x?,)��/u%�k)��
�]���$}��W��>QA����GR(g�)o���t��u]U2����)XV;'��%�ji�v�����W6RvF-�9mwW<#	�;NH�P��>��B���C��
�w���[��Qd3\�s��3��!�`Z]�<<�\Zp�����
����@��Z�]�B���h{*T���k�������;�UQ�a����J�����!34���
��$8������L�%�h-B��[3����M��3�CO^������+����F�Ur��������������|1y�Me=�{�������s���P��<�|����?`��������[#9jno����X�����c�����8�~g��}����fg��S���K�-�B�uL��h)-+Ll}��,Dx5�s!/&x�D��U0�4 �a�b1G3y Z��W��4	��>4�8�0)��yUGV���0|{���P�J?��B���D`����	�]\�v���2��%X*G�-�W2�������J��}Z{��4�2�n�����]��o:/�i?�9��|�{�^��v�W�_�U�����������|���t�|%O���M�yW����w+Y�_D��$�B�Y���y��yk;{���[��y�xF+4����NS:�I������u�N��3.0��.�E-iHamd������S�1�V�#;s�����������(�l���[�T��c|.T�4��Sra�u��~�A��8{{�Wzt�C<���qLi�(Ax0��*�:~�p~�?��\/�o����S��I4>U-��X����}S���b�k��)���t]U��}��O���:��nr�]�t�\��K��[�Wxuy�^��J�}�\����a�HET��~��f�|@��2e�+P�+u��L��[,��U���O�`�V�.����,���5��?���r����4�L"IU����0�;������.���.�y�q��@-���+���J������87>9��u$��9�������t����I#@��r�b��{Z8{L��2�S�E�E{*N����p��[Pn5�����T�e�B�:4���m5!Fw]%�g��B�U�A�3W)�}*
�p����N�@8��E���;���^�b1�?'b��� ���
p/��Zrt���8��L����\S�'����20��D�W�����wZ8�������IN�_�����"<!d��|��~x���������������`%������y�Q��oz�m�|��_���H��O���M�o�����fK����	��h��,������v�a�>h����>M�����'�U�+S����<���lu�"��\�A���Rg�&R3y&���~f�\n�������"��U`�>�qn
�%�-e��r1��^�]k���u�n[{�Bz�9W'��5�d��:?�[�cu0�3B�{�,X�2��)�sK�bd%;�����JR�Z�=@�y��f
�����u�S�Wh�%Im�Z����*uR>�+�P������q�K<�0�G�5��/)�<}i����iA�f��hx��-��&�S_������fsfT�z���S�+>
9���$=��Q"�2���h����h�������������J��2�����t&���l�6�|���b"�Z��Ak���W�s�:fV
#��$�!��p�����������#r����~�?��c2���������|����l����t:]�3
�A�sd���l;�mg�uzN����?/C�&���H2�4wv{�]�����zd��+)��J�`%�b��`n��V���a}�\cg.�����k$S�2|�����3%���
p�)���,!���������	���rcsj�K����(��Y%�����9�\1��7����������f�����;n��/:W�/�����N����������[f�����7��5�<��Kq��Yt����: e�)0���~����E��d�������"��X5�_�C��X�a���xkw�N��I�M0�����z�/�/_��j%<��k�+�_H
0����}������������QJ6�,�������^���6����2�@1�:f��zX�����c��������/���N2�HJ_Q1���?&������
/��Y5��h�
U�N������`!.�2A]���3�T��V��jJ.W��A_vBW
Jc`����W.�]��y�{?/d�������C!���82_�xn|E�qTB��y��� e�` :�=���j���V����)p@����H�O�@�������F%)#�pSS"�U?%�rM#]���j�U~�&�~��&�W?��6����S=�3*���ln���h���I�K5��T=U�x���A
�\)7v> w��SU���2��Dc���ye���@;.a4�Js�D%<+�	6�/��Wjg��~����j�C�2�zx��?���>��d��v�����]'gB}$��U����?L��d�?���?�������`4j���N�������n��G��M��+G�N�=���S������6K��}�&h���-�AK��ThY1��@����W�&��b���g�n�@������T�����
��uv�rM���?��\c�7�J���t��<Hb2D+�����Q�=��}�]�Ak�Jn�*�T��'.��ZQ��K�8����(���������k�a
=��7��{��rl���� �5V������uo0���OU�jF��u���2�!����1�,�!)S�TgP�Qk����|�)n[�p�^�pT��z0��2u����D���n��x���a��_Up���zH^2E�/���� ��\K=Q��S������c�BD����KT��f@e��d;:�� �{�8��`�oh�C�h����*����ay�	��-��'��>V�k��:|{y���t��������/������������nd��q�������7wV)� ����r"��e���&�:��JnT\�J��V��T���+&a��	�*\t���f�n��v�<����1����:������s0J����:��^A������j-���A�6E�h�UW������-?��p�'�l����So�����1���b{�p�0'p��"yX���x^1>�%A#fI�AR�r�O���)h
0�� ���Q<������Wh��(�������9u���e�2������9�����V%piY��TU�M�8h�8[�t=�8���v�����E3�DX�
_��J��~SNCc�/qO`#���C����-���9� e��"��8�s����8������O,�.���0���*")
��\b�R.�������:�c��ip��>K��/�1C�u������V��5,sI!�����5[�bYm��q�S{�TE;��Mv^��k�Nn����#�&�����2�����Fg��(���>R��8��:��"����I�U���j{��9x�qI���2c��&%��9R��0�Y�!l���
��+�=h/4E�X�M2�$Pm�B�hC�.���!Y�h�,��
�:�1��7�x������A�p�B��`�C ,cZvJ4�X�K������sa��3���������D����V!�7{�O_����G�!PA��Rq���i���M���{���i64+�3��o�U�.�D�L^"et$����'�TS��(p�������!VI�;C[����|�|�����U�/kw��R([g'���-�$�k�U���Ks�`R�����]��Y5�4*����M�'�������S�.�Az�1bb�����M��(���f�!�#G��6UX���G����� �#�@-?������b�\S�E���a��a5Nlt>i�����a���MJ��A�]�:m���U{���M+D�x9��$��}_ ��~�������6=�TR�d;j��!�����E���"�w�K�;�Zg����v��������Euo�

q'��%*�p ����O�����m�b�"��}Oj����&��L�b`Sm��QZ��.g�����{o�..������O_�|ut�|�(���"��!P�I��eXYM<�k;[�D�j�������)�
_��(�w�A�t4����o� *�k#�5�������/N�������)������_�}�r��`�F@�����>�7�oy�����-aZ
X�@��#[� A���5�(�Z�&��|�6�q�Vyx~p�=<����{��zI����A����7]���r��d���&�HT�-��v�����}I����("%�[d��S�(0�%{�a%��o����6�������J<j|����
�)uN6Jm���d�B+���[D*T~����������XHv��nK����)_M}?aC���K��-�)���w�}�ipY=8;;9�����5_��c��N������f����� q��EIm���@���x�)j��}@*�?����,*������5Q�����[m�����8N���D���K"�����J1Ne�=��\3��Q�t/&��T��43�B�a�`�cu5vf,��|;��-[��`���V%3?�_��su����'^��t��),f�{M�h���kA��_���`p���dX�2`F�����{���^�oB��l����Jl�oC9���4�Z��O�j�M
��]%�:�xG"�*�vA[^��7����]F�
^h�.KN+*���C��7D1�6���fpO� ��E���(@ h����L��^)���h�R%kb�b�2]_������!�(��Mb�E�D��%�8����_�)0.�#��>:�8:y��\EC��	�v���$���C��[�kB~xzE,�0 �l����4�ij��af*��&�<wC>A2:���F �Mo5h��)/8v������wc ���h�+w�Z��W�CP�q_�t���w+�����'�����2�ac���C�/%���iO��
-���9_��r]��U��j���O��K�Z�i|�:���e����(������M%
	p��w�	�{6�O����^�l#����[0����������!o0]���;��go����v�<c�j������t��g�!�� �9��~;KeSD���2'!cMys{c6�X����*
���X(=yD�x��z$NA�$����go^�5������ 3�1v�91�A��s�Cd���Yf������6E��4�/���=6�
i�K�#��m8��-9zhI`�<�s�x>3����y�)eZ8�f�e�Vv�d�n���on�F��]���b��a��[��~%�)����/s�@�Ob����%�p�1S���Pb�%R���V���8��l�,�NZBL�_�\�
#,�I	lc�r���@���������-�Z*�F�@���.�Z#
��O3��8\f
j�FM����|�Z�`�U 'kZ�
�J�����KIAo���6����-��J�v���.���$a 0�>1
����y}�w7,m�Li=F�Wb{�����EH���0^�%M�(��)H�}�����5� .����g/��cmV��-�����V��H?,��a���T�s��rC�%'sx��%�
�K��M��)���Y2�R����uH��B��h#kl�3��R��R��'�����t��2Wg�+Z�����dH\VRPl����nnZ���;h���J���O�3[�r=P560u��\�������@O���`�bx|r1<<F�������^�/�.6��:����e�p�M������(=���$�[�>��H��WF�s�7�`�a�����b2"^F�?Al_A$-9��5Y)����R-`����F�������o�
���K�;���6+�!�Jp��5 ������e�=����+6i���=HG����DB0f�pX�I�{�$���Y��t!YOQ�AI�!�<&=R�L����HiUj���C�)[���]���r@��e��(���X'c6�z�al�V�%����U�N5g9*o����(��'���� �#��f��|s��|�.�����"��f�o|3��,���;���l���^����B�c�W�lW�F��F[���Y����L����Q�H�q���"1�c���������l�g�70eQ#01L��o����WuC(�����n)��p�1>���Sh9����*�RH�Y��=
MK�,-#B�&��������NtX�/���1��������?�n�< ���9�c�,%���|(q\�u�YT7@���q��T�D�+Q�m��#����t�)���
*��"NL�X=yb4�-8(3�M����s�� �����I��J���+<M���L*1���x�T��F�5}Z������Ik`��<����:�:����\a���3��*M�����r��X��v�%���lm���������T,����d
��"��f3�r&�����GR����1
����}��z<�Z�=:J�<���A�Y{��rF�LsZ��L������/o}��)GGR�1`�*�%�h�mc�zN:F�WP����8�I\,��I�����"{7������r�ky�h*��ma��Wf�����m�8_7O�6):�&��nNqA<��$� ������@��������dO����*z��!��dB3�FONM�1\��8�]&J&��7�5Tq��d�.��G��m7�yl�@�~�=���z)���{'�m�P"����~9q"e����Z�D�D�l\�.uJ�^2��@	I�f��J�yd�d����K��w<����X��d����T���5�.|�?��~+��/g�_���'��O%�M��Ta�,��+�
DdE"�	�#��pY�:7����s8��/��\�x��L�^Wi����n�l������k��oC���^U��
���v��L����(���o[\��Tp�1W�-�B�p�M<Z\�x3�D��U3�o<�����S��)��hy�U9���7����$����Bls����wjv��w�"��29S�<M����g�Duu3E����q��7�0�d�&�-]|��N
;�*��gO�������sc���d�_r���;����5�����n����:G�����	��)�
������6�����S������31UG��z���a���Fs�=8��B7�<��jJo!^-�E��T���C�%�����i��j����Eqv�Z���N$H�[��D��u$�k|*�A��@�=u	?a�J��]����v,�pX�^rg�T��Hqj��F�TG��mP5P�O�[^D��|������)����
,�)���,���
SS6�LQ�D�"pfj� Kva��Q�����1-�Ff�	
�PJ�O�m%h������(���(��4���NZF�0��&��=�o`�_/< U����\���L3�Nx�;��k5�����1�De,� U;�R����~��o�����o��I%�6t�6D\���s�J��"};$�Y���s+-gYP����fk�W��G,X���C�'N\��:m��\9qSJ�O��|_�����o8�� :��2+�a�U���H/�uvtt��Ma�Q��*!#�����
/(���l�����jK|�`I���*A�N�:9�����/,����- �9���������p�b�tr~��l���Xr#`�Z:�$��������6��p�"���?���V�����5�}b���}����~@D�{v����T5�n��p��Z���EON��x�0ZZ��o���26<�_��������*����Yz� S��Q��.H����"9�ea� c���t� ;��Nr[]��4��v�O�ybS���f�TW���k����d9	�_&�*izS�����
7>�6)V��;�������Gr5���k�8Iq�������ouU��o,�m����n��t�i������������^�]�4�*	�����Q�x�l����R��k��K���5k�Q��rG�J��:KiVd,
�F|S5O���`�p���Zt�mI��3 _���c�SR�K`��u��������s6��('�D�k����p�
.��d%:�()�R�����qWes�[<a:��x�A4�P��"���,<�/�e{@���~`��W2VC�Q�G��l�������Xl�A5���=V���E@�������f����'	����-�2#���,�
-����������:�U�5HE�b~�����vu�"1~�A�ax�!�����'�T�_�n���D��uU��E��R��4���gA�
�+��1|���E�n�3�'��l$�)����c�4�x�)c*�G�F
_�lqJ�����9�RYZV�LXG������tk:�m���y�zM!j:�:g�53�����R	R!r&���g����:��L��S�QV_"����d�al����eQO^3I\���96V�d�Fic�A�{P�3F��^3�i�r���6�fB��;���FR�l����C���H������SmO��kfg�,���h�����������#h�1_�!�.n%[�:���
c��$�h��yt�!.:��� '�	G��0�R����x`��(8��2�U��X�dff!�yaq�r�b��L2r�$���[�+�a�Xi�T�xvpK�]�l�C_2�h1*��R����������{o4�
3�;����'�n��A �r�QbIr����E����>
e*'q�n~�����g�T�l��]3{��}��4�1p�8d��A����7dE�(�#n�������,"Zt���0,_�9�|=ZQ�[4�+%�� bG���V�������h	����m�2`��+OY��E����G,�S����t�R2.�<��bV��v���L��3�T{�j�N��%�Q�R�in�)�
��%��.q���%��<�D���Z�Z�.���J*[�
��H*��Z���J��������r�m�9QM��W	L����y�
r#=Q���'�����o$Qd��j*������^�p�?6�]���);�������	�������q��&a�
9���R���NK_��8�1]�>c Ru�R>%$�^-������> P���>,�ikB��8�~g���*��C?�����)�	&a 4%�>@�z�����?a{�EJ����p������"��@Z
%e��K�I�)l�6F��0��^���	��0�B����J��Vm�CBD�]��M)��1��fI�e���p+�r
���PSAJ�`�0�����n���������`��m�<�� ��S��b�C�����$�%|E3�G��WN�g/���_`�g{������}�������1��_'�Aj��o��f�u&j%�I�,�nI��i��0����F]3W<P]6�#Y)���C�����n��m>����xk���pX�'>����!�.=�v%G�7���������?O����h�UMeV�X�)`.�����0�%�Q�r��T#������`OM%�jk�*mb�%��d-/\�����	l�f�[����:'�G�gi�
�@�
��Wp��	�I�Z�8���%��g���T@����i��M\�d��������Q����{��U�L�g-�kk�<��.j�Db8/�rPm[,�6�,��=��)����:V��u*������gzf���Tl�b�a�x�X������U7���`Y�m�u�,YA���|��hE�_��AZ/�*��m�>�	)��Z����9���p�T1!��n��n���.�Q�n�hB�J�b��-����l�����U����O�=r��]?��BV�6�N���X�� �F���	��3$�,�Fy���F�hW�2	���J��k�g���Y���I��W�b�V_������P���A
M���Pe��g(���m��4�$5��`������"�R*#����F]w1�eI ��0%�K��E����y\�����]�9����-���+�����1Qd��z�������]l���b.w�d��h��{rL�"_�
��Qb��^'��4XP��|����^.RP|)�����Rb�kU�\��=��a|C���a�1b��������MFp��L��7�d�[�b�n�7B�9\��?)*�V�)�i9y�����m�e(/�'P	�������o7�q���f�Wxqx��� J<��&�M�Vr]�����SE�,�RS�8���L�P�U����� [j�B�7��n����fP�y��;i����I��p�,��c���j�������H�MH�\B@J�A3%�����e�Ll��*��DJ�,(q�{��Kg����M���H��=�}���*p��x0h��=���|J4�=���*#�1�?	|�1�w��nd����z��38�T�:��!���BVB�\m,}���`m0��O��L�L�X}�1U��r����W����Y�s�x�f���d�29�*&=l/�KUu�4������h�o���������tK�2e��2�A�������i���iXl�g�q%+K�I�>z�E:������:�{��(�L	��-E<�zd��-���
�x���J���_D�EF���w��q����,k��<��]�-��e����+����aMY�i���zK�s+�:s������D(���B�� ���i_��T5���J�	[.�����i'��d����2
`u9�n���v����lo��T{"�p�gb�1�~I#���x6g���+�CN��c,�4��T���5FFVyG��l��*��?�s�'��%�H����77[0�tJ7^�+3cU)u���?�{,��m<�teX������#��C�K%6��V��%	�����H�����J6�� �
g����{Y��C��5�l)<	�P���x���/�����Y�9���x�v�u6U������6��>/�Kjd"�u�0�E�Q�
<&&�
���P�$ ���b�x�?9.�a�)�'A(��s@>e���8��e����f�4���E������^��
Q���z��s�5%���w����Vk���u���x<������������#Y~'�b�nZy[D�Pq,���}&�).e;T%����G��SLE��SI��;'D'1�K�@]8L�����'+^x1nn�-��t]�����L����2&��{�m��*�
e4�s-6H���%��-�\J�����
���E��l�\�n������2���f��Ol�M��
-^@���O����@��>t���sV�p��=�-MT����?<F�{��-��i�}�m�3��0������&���|�6��m������}Z��@f�w�5AQ��@����4-������G�Is{g�����N��m�F�<�(�����6���T��u:t����������H"�����U���T�T�	s������p=:��i\�|=X�����j!{q�ea{�*X����JW�P�]<e9�8I'������V*���!�����d��]a
~�M<��j&vn�)3�'v:�\no;��*�ID��������G�PI1��E��"��K�H����*��J:���M
0�Z�y"�8y�v�Sj�@W��������u�uI�0^�K��5����BTA�ER�|�f��mV(�m�o%��R�P�P����e�TU�w�����8�L�sATbO���<�Tu��������U����?5~������������Z6��L�~����}2�:��lD&$F4'�y�p��7&59Q���m������nhrI��K�!��@_����y�����*ihv)�9��{���XG �!=G���<���*t�v���
�:`��R����9T��W�h_����	3��I�Y<�%iA����<|�oPJ��m�O����+		T���c�#%�����~g���.vH��h�@��zLs)8�mu2�
��3�i��M���]v,l��?&M�d��"2*0\��L�$����DZ1�lJ$^��;9	��xW�%��F�T��!Y��Bp����a!w1�I�J�~I� �!��NW1n���	��b����J*���@�;�x-���^�����H<��FT�'0�Y�^Y�%�d`
����$3Q����: a�^��7~�e( �I
�&����%[��[��b6*{T2�E^u��[�j/>8�h�!�U�R����6'��f�.B�:T&��������
:��
P<�9[0u����!y�����I���P�d�7�&}��:TDa��F�� a*�`�������?O��l����)h5�,l���GX��*�g�������V��V���T��k�`.��TN����~�n7�N<���n5��A}�sus�k]gNP�s���uX��&�N���~�>�m�~��^�%Z����l�{��h���~��D�k,����Pf?���f����<a>'?/1�������N{�tz;����ooz#9�[�m�7���h��-��"y.��5��.�'��f��`v��U0��
.��.�����hS��p�Q���
t�+^�^M�z�����n�����R���9_}�?��]����V����4����6ZV*���I�����o���������'ND������)?j�����
�Hc06�<�eq5=
��T�,��
���k��_�������-���D2t�DX���ee��P(����JrVM���������'���A�1 Y��Q5�
�[^<Y�]����T�[�~��=���kO�=x~�+N%^G{%��+y��5�G�C�b����8@}��k��*/�h��"�^v�U�����y��aCF[����9�=m�{:>�������������u��[��q~������E���������_f j5.�_�J�Q�rn:���7�>���~��[b3��=X�h�X��C����f��ce�p��_Dw���?/�@�_��Oj���B��W�KoG��zJ�W	�j�����]�Q(YV77�t����F�����-N��j��� (6';�Ao�h�:��H��v{��D�ih��������l������wX*��aUVC*b�������������������4����oN^��VB E�p~��A������9p���oj�Q<�=";��y�wvx{
�����tf|&�(��jn������%)�����Z�3����on.IBYt���o�
?:����K�7�������=������U��K��K���e��&�����|8�B��&H�[�r��).*��������<�{u.v���a���*���K?�8����������l�����g�N�:�l�;f�����l�<�T��q��Xig�^����V�EZ2b��eh&?�F�1rG:�Q��~���[��	.���7�n7kx�����/A����~L0a����5���|8s%G%�5�E6��{0����6�.�bQo���N��"�wB�*��~$^���~s2<�
���H�����!�[��z�XE��w����$
���������{hnP3��I��((_t(�}��by�{��\4v�
�q�YO���i1��qg��w��`�S���{��Ge����!���n��ZLVy(
<8l��0�N����a�����L���K�h����_������U�`�[�@Ng=�\R6��q�!{��M��bN�+���q�7uTA��Q�MP�
������'�����LN������,Y$��������T��>*S��=?��o����[`����rBS;6N����v���V7$R;�d�\J�KNj�������`���(���[��d#$T�N�VN��r������{\f�����9�]���KJu�V������d[R�������� �
������9x����j�=);7�����Td��F��*�6N,��l����7��P����'����Ks���W�i0��gW����r����a��}���D���:"��Q.�O
5�a�dW��n/�r(_��@O�����j��(wK�3�O���_��Z������0��Yb�R�������	�cG���a�.k���Ma]B�kWd[HE,���z��K������8U-��r�T��"8�h(m_��XEk�[�M���}u-�IP8�*���h`I��M'B845���-����2��7�:-9K�@R�=������!$.T{�
;CUh�Tg����B����9[���8�����-G��8��A9��/�Hpzk�DR��"XE[X`5-iA\~�_����y��'9���Rp[8�=r�:r�^=������!o�I~�JJ�\Q���7T��k�$�>��#4b����W�6����WI�
���}���T��~CwynSE4t�H�=@����%�r�|��<��|��aN���$������� ���yG#��aA�B}�I���h�4��ei�*�-���nm��b�L��;R��,�or,�<))�s7�L��oH\|�&6�V��,%�Q
�m�F��B�����1x�����R�x-�I�����V>9Gn��0����T�-�*t�5���*�N�S���c�o,Xe�J�������u��%
SM���R	R�f(!��o���dr?�%�9+7�F9S���+CS�����F��S����b�(�|=Rk�bc�Q"�`=�f�[�<)2Y�����u��J�[�?O\7���]�xt�-���LgX$�C_dC��R%���J�8����du�+"Z��Mw�<�Qd�t���% p}�(qB���)��X��9U���?!�����x?K��H^b2+D=���aJR��	#K7#A��R2�Di�/�%N�.����������k�m�P�B)$�����T�,Q	|�6�����0��#/�(>����������������������+O|��������
�����,2��=�0������N�>a�Py�����n$^�i���@m/eL��n�������^+�:��9�"U�����j��2sZ��9Y � ~�l��J8�*]2<��.!#���q������u} ����^X������=����T�����!�x3�w�j�&*(2:����h���	Nj�fX�l�{��d�) -I>
��{���!!���$o#��Ld�(9�SuD��3RI�K3<x�|�/CJ��)Ky_�4���@E�2K�N2}����o(�I�Es�x}x����*���J�Ay����������"�*o4/�E=U�z���n�L5������)Y�d�Y#�a!y��e�l�U��O~���T)Ym��c�(y)�G�g����JpL�����RR0��Tp���.P���z KOf�"G7��^'��3���0a�eW9��]#m�!���)}�EVR��&i�9��#��V�����`�Te8
x���a��<]GYZ�2� ��,��'O����I��b�(�e4��@U�h�2H�����"����3��q�#�=%r�T����S��_:��E#����%AJI7��f��X�O��0�
��zA�)*�1�N~t��^K2�+���-���������/�r]y�T2,��*���'�8!��o.�-��@S� L$�����oon����[���bK� ��cO)�lkbK^I����)H�b���eM6���y�
����[wj7���F��������-���P`j��B��]�(�
���8P�A4.�R�+�-VsN7B�25����p�S7DG��;{��Q�2"&������sz,����t�l���M������h$%Dp�H@���z���}?|b������!�7Xd��#���j
�#��Z�9�=����6���`���t���x��"�b�/���f<I?c:bh�&���4���V6��M�u�[H�b�Q�h,��Q�������LD_?#u�`�!����H�Ja�J&�od�a�}��}�8�x��dU��?s��P�
_�����<c��om�v������G�F�kRk��&�6,^�FX.r���?��@�	�!���%|�6g[a!t=��<���U��U��<���W%C�����~�K�T�@�U� tU�\�a1�B������8���C).�C�2�k+c �Y�K���;����@��'a�]f�u�����F���i�S(��CYJ��6V��BU�=s��&kQs5���UjQ��Q��e��Y��$y�����.�H�����!�\�����w�������!bEC�|��WY����j���1��Vv��~�D_�G�>'2�m�l�"=�*@���[���	� �w��"���Q
Bp6���T����B���=]P���O��h�S�5�;
��?"�4����kma��R\{���<)���
P�z0���c��
���?�'#����[:�b0��S���+t(E�~�&���vjW;�����I���� �v�a���=L{��p�$��R��~d��l�+�<q:Q��������,����C`ca��:���������G��$(�?L����9�x�	�\V�X�`	����00��,�6�-��)�a��Y�_j\<@��y&�k��dl����Qh2xz@1J�O����eyJ��!*,��4��~����U�����Q0w���(��h��lr
C\���i�s�c����f�����R����b��"�~z3�p��f+���,��:
�&�g�=uh���]tc��V��zk�����������i�+�����TW_�	)��?I/yi�"����aa�R�zn���`����%�[�D�c"�����@P��72d6{���u���pFz������DB�%v�(��C�Doq�h<����A%��	�{�py$���z�a*.�/��s/��$��hG
��1?�~���KuD0�9d���]�m�3��,� p����Qv�"���GM���H�����h{`���_� !��HM�)]q��[��x�t�����������V)w��f��Ha�v-qIO�_��}���(8r�;hDU���������>���(i�r��V
�SbkK����0k���$��1����&�k�q7��/��E�-6j�7���7C�oxMc�x^SX&wf�`��N;o!I���Kb	[��c���{����6zy���l_�+���^�h�a�b6��]/�/�K`^���W}8*��������G��]h���*mB���J�

��h��T1�T"���IT�Y=T/B�
�aFG��a�*H��<}#�@C��=P�x�r1��{h��Sw�sM��-�p��P~,�B�#��:��F	p�L!K����1����7U#z�}�1����Q�6������|;�F����������~�������t�����~ e�t�rD;�an��X���K���W��fC��@]a�p����M8�:j\�_\Zh
�������V�������5DW]�ON_����,G�]�}����*����jV;��O����N��-M���2+�}���K}�-��������Z�o��.��D��\�������KV/Mk}17�<�en���a5���F;)���1���C�}����D8��L8��(�x���������p/W���e�x[_��0O��:������������DTH����U�C�Q�=B�\42?8������=,����e�Eb�h�k���s/|/
�c49��
��q�W��xP����;���f+` +�wZ����������������!�ck���9��5�d������w��f+��������V��6ig�[�?������;�]�<�?�di�qo,/�Ko�M�[qh�x*;���s����O��=��B���>��D�d#�~�eg�M�	q��w,�����I
��Xi��~��f�r�	
RO=�R|�F�[�_������$.T�� -	
��+9��]6�$Rm������r�%�S�8����G2<HLwB�I�"/��RJRp��3;���Qq���W�!����a8gu�V�?�z�/`��r�9N|���z��C{�; ��[E��� <:���k	���:L��$0�@	�������&�V�r�;�Y����*7Vh 2��g�[�S���d5��4Sw��yc)���P���q���l���SE�2��U�C�m��
�V0�!�{�"�����e���=��L�e��65W����T#��N		��F�(q��N'_����nBc��&����}BcfV��`�|y�B����^Q�����@T
��l��Z[�� �&�m��0���K���m')}��LYlP�%����J�,[+i!^��Pr�u��&l��*����B��v��1=�c	#�
4�v2�'�U�e(w;�F��h�YNG��cu��R�g�G1q�cMxq��c��o�t+�	E�X8�@�D,JB��)Am��a/r�6�m&;X�!��Y����7�-1bX[��w�-	$c
�,{C�!LN��{��)1C����,���t�&0<})"��xQj�Q,���zDj����Oc�o�����LU����[�{�H�[]��4���Nz��>L�|�U|;�3P�k�b{����������w�����������~Z'�{��.���������wA��^�E{[~3lR���=
�Q{w���7TH�*�����������	�E�L������n?u;�[�m����(��}�_s��U�s�#3��3��c������+W�>�����0����S?u���U�yn��A��0�W2�����#!p��n������:n�UuT�}��~���+Rb�L�^��~aq�p����o����(���_(�t�M�<<�z����ts�f�-���0�7���i�7r���P���Ro�Jo��k��#+S���L�'��y�����S%*����������P���4r*�di��k�����(��SX�T }���e��������U��7UW�ZL���Z�xx7 �����RX�1����������A���_�3������!�[{[�|��Q��"|���)}�����V����=�76Z��V��my��A��U��T�m��L5�Ie����+�q�������y��	HMC�b���h6l�P�����U����������i2|�,�|�'����&�����n����O�Z'�m���=��L��Rj���'!���+3�Q�|�
����Pc�>�V~��++K������U=n(?E?�'J�W�1��t�%��zY���k6��QSq�i��LD�h���4r*D���A�%U��A\K-���K-�]Gk��h�Ql��~���BWd4c��<ca���i~,a�0�>I}b<x���d����6����j-HUkBms���n���<����5kC�66{�	����.��LF�������q����������ww�}�Z�{�^����v*X�?P���M�����
�?�O"T�/l�onG����g{k�p~�mEz������;�O���?(=q"�+F��KLi���U�F��l��)�Fw��dw�������9��(v���J����cC�z�����������t���ng�^v7��YVTg��n(k��n��nL�|qh��28����l���S��71�	���;M?C<��uo�AO[��U���7�u�|��������% �n�F�M.)��[IC�OR�������6H:���PfY�6�/���8P��Z!�}���c��2�\x��nV:}���	&g@��F_v�?����A����-=�)�U�:��[��-��x�"�a~���M��iaI������UoT���.�� ����������fjuOE����r��o���|�] ��oIQ��\&����L�<q���{f�O��O9���b�&�<��7��?��|��_&lZ��b�~���������W�m��(E U�UZ�N��S���|���3�n�#��K}� �������|Qi��2GY�N+6+$q�y�luv�����^kkk��[Q��l�b�����aq�	d`X�L��� �)����$#<�������|�
�Z�Vg!��@k%��O�|�3X�y3�F�4��U�Oi�+��z{kw��D����z�E�a�����s���_������e=OV�v'42�G\1�]u����^JRxs��j��2�z�6N�v\�a!���A����To�W�!k+�-pv.LQ0t����f�f&�?��"=��l2�YE�@�,@@�I�5CiJ��LG��N�z?�[{h��n���]8�z�������������o�$�m�"��7�>oVe{�s�p���p���������w\p��O���,kn��������)������M������cH���*������;xM�9�q�������L��.�	~��*�*�i�EMY�UI,��J�f�*��{���q��6{�vg�UWV2����LK���]�]��PHW\
���M��m�����O&7���V���9,3Y����q��xT�=�W��e�R ��*v�daMq�7�z�;�n����������f�������X[i�"���:_�U�0U*�-8��E��r9D�����Xi%���6��+���7���x��7)����$U�K/��nDN�8���A�O�A��o��]VxX�]���r�����{�PdT�~�'�aS��|8�A|��&��;[��;����]�RX��b���|6dk���k��t8,��y#��[1��?�\T���Hs��*�M_}J��e�d	@N&�!g;v?�����t|G;C{A�h'Q�S�V��W��}�����`Z��sGy�K�K�1S�TXRUPg��n�i��.�
����Z����<n#�XI��{�S�{���	���v��*�cM��S�k����N�?���+��dE&�Q6wP���c�rG���Ug��,������M�-z~+�������7�������x����{����|6Ip@�\}0������0�eRkh�3��og�O������?g��?^.m�=>_T_�_���=�K��cY��K���2����x���������v��JE�j��(zn��QXQ�]c[���XlMu	��5�����GA>D�9j����E`4w:��k!{��x�0�R��
���3���{�|jF8C� :��gU�Y
�T-���������~e6.��FD?j"Y5��|!�zZ�Ry��1F�.�m"�(1�aP
h���1d�A _��i,���j6&G}N���#X+�u�P�>�A���9�hXpF�H�\���z�
N�OP(�����*���q��
�;�^���B�`@2��p�V���+�Gb6�D-�jxMN�sF��:��Eh:�B��:I������,��&�h������
�9A�<���:�&5L�c��Wq��k�>����*�j'b"���|��UKZ�~W��aE�s�����R�}��Sr��I�`�������p�Hq���Z%]��@JY��Xx��������^��)�
g:B�I�D�������<DH�����h�e8�i4w�-��OP��y�k/#��;����#m�=>58�����CB�L�A�������-�������op��k)p��4:����s�]~��||N��A(<,\����$�v��l�L(������J�!�� ��B\�%�d-�N��J�2x�[��_
���<W���K��Z���-�Dyp�=�[���8x���Ft���B_�}�1/�����������u��_���8�]���"5�8�+��r-�Z��E��H7Z��t�**���m��w]���DL"�#$�s1�3���F��_�x��s+E�3L��
~���q����r�"�O(	~4�����d�����H����b�=D5M�U�B�s���a�s}?��I6#�D�����Z*T^Q���>���L�T
��U��f��)"�un�����xp�M����H�������c`���������U`#\��Y�����h5z�2������>��;i�{wq|t}v�I8�K����o*b�"�F��P%�4�@�3�J�T�f�J"�LG:��-�a�����s����d���^mg���� �0�c�@�$!*��h���)���-}*N�t�Vf��:���F=�r���rtv�>����u���������������������8NB�q�"��'���e�]������z:�Q/����eAV�r�i��BJ������p��4����~��N��'�H�a�@��.�o��|���rg}�����-���l��3�x��<<L:&=(
�n!)[�Wk��p��cpJF=����d���D�(�����_�?���U����S9oGz��7����%�F�qI��`��N����{�#�1>�������Zs����#��������iu^'����1����S#��~V�D]��?��miiE+�����|6�@��aW��������e�����1�Gf�&������E�n\Bx;z��$����O����\>�	YS����.�`x�IR�~^�������>�h>�k��{�T�7�&{���@�LY>��������n
�v���'|l�r�(��P��M%zEH�����
�#E����������c�X�ek5�gS�Nq+H��o�B�-�6�|�E�j(��E�����?��^����I���9��(����3�XHo�.d��[�.����Zvn���������!�bE�>��&(�p�C1��#P�9����rF����@�X�i9��&��1�m6�����
��%�L���j*��Q��VM k�}&����m	R<k� ����<�g&��*��)�[�Kg�'V�r$x�mp��}rv��'�E�e��Q>�cELS���%���7�$���0s�*��E
N9c>�"d��}����T���)R�2d4{���O^�Av�j[���m�i%0����2GY��_�Q�qb�k��B��J�t�����k)��#�Q�����;%M92.b��X 7�jV\����)V�2KL>�8?E�6;�J��*+�n�����'�|��/>^�^r���s��0=����X>���j4[��Nk�������Wb�}��[��zHMC���D`o#c���5���%�b�F���8�ZcE��A�J����G�l��	�|�1���[���6�������zi	0���B�8ey��/�ZOQ4!�jg������_���bx���-��H��*�C`�~������)N-�X�T36)�c�I�'Xb�Y}8���mkK#2���F��o@�+,I�'�7����fY��p�uG�n��4"����7�mX����	m^�lmre:�!<�6

z���� �\����c]Y�4��u�c��I��xv�q�����,G&���Cx�GG�g��Mo�!�t��{�:K��E�;��&���	�K��UK�!����,�Ku;��y�5�Wo���:L�|�S9���)��}�W����^@P	��p+[�]&X�@$��i�NN3�������/v�h,e|+���:������.��)�b�W�������!��������b���mjI���
�����Y�=(���3�Z���S`����E�'{�#(Q���]F_�C�B�(9��zd�	�AR�Pw���u�<JA
B��������Wg��T�-<�O����|����M^��=�CU��5�n����6!=��W�(B+y�wZ��[����ZC{}��>fd��n�F�����Z�������N8A��L��B2c���^@��n�i	n���xQ�m�,q���"��}�[����#�{
���������k]�B�x��I�P�$Q �z]�|�K�������O��?�^+=?���odfA�����M�0���t^F���uR�[M�
�[������c�x������PT��_ON��F}��{������n�������u�����
��mbX�����)���	B���E�%����
�N�����_�)�KB9h����*sg�K��i�
�G��� ����h}��M�ZpY*��L�X;U��I����2E��j�@��^Ry��eNS�*�`(O	r_L���/q���(����K!E��S�Un��5�1>���d���6Z����H
KK�=t��8��|e��(�����R<���m��:��s����F�@���9���y q\�����<A���
�1s��������0Y�,7�Jm�����,|��U"�T����HW��;wEqX���m]k�7�/�1I�� �ZxW�OC�"�b�[9!S��N4�����B.8/��z}^�n�����E��K�kV�Zr�I���:B�X7�@=�l���:5"��R
u��H����?� ���UB�P2c���?M�"�^�O�q��P�I�`�~J�83�`�w��������7d{���������d#�-��	�),>�+��F�D)���I�3�4���	� wn7�&j5cY�ks����^?�b�uv��=��9������N�s\Qf�S7��-�a%�����2b �F���0��R|~6b���AS��wUC�b���Z���V�Z~tB��YdF���y�]eX��*EP��VLlJ4���(E�`VG"���BFa��Y�%2�-*�|D�����V���|@���{[������2c?4=-�����!?�����&��
�"n{���L���
)R6��	Q���H.d&�g����4�Ht^��K���z}�������B���� �;�z��GY�Z�L��y�BAU�R�ci���<O2�G����C�n��/��V0��UV�X8O_Z��L��R2	��0����Sn~w���������XH�����L����P�^(_)U�0h�{�X%RW.{uU���W�;a���R�x]����/�\����8|L]��Q����?���7JU'�7
j��Xma��J�����.;'M��/��\�N?����a;5t}�6V~'>^������& ���x=�v"Vaa���}����E��m�v�����q���n��@=����@)AGQ�	Ir���6�k#�tN!�L�r(�H`�x�dV�mU����9H��e�����������$��/\s��\���3l��I�$22a�p���"���"��k�K���F�<�
AF�y��#.����1b?�Y�UG	��0�bn&>��������'����<��*����~�O����4��H�3��>��F�eWk~goOK;F~���h���������
%�H�H�xq�x��^��\����h8V�[j&�w��?�=,[�v��8��y���v����T�����a���9�s��wyz��|�l��P4�,Ej�J:���KF��p�Ir3�3(�v��L9��K�l��B�E�/%e9t����f!��~h_>�'r��x�o#
��9���w���%������i��CsH(	��p(�o�1�4�W�-�'�$_Fk{��f��v�x�$�/M�����[i}��S��� z-�J��NIz�R�"
��C�����f��LM�KA�Gl2�.��H��1�S�
M��
�����79g����'�n,�P�6Kf��51���������K��������%6w#JCu��A]��A��Q��l6��ix������������`k>����?�V�2��hShF�z'�z�.�����lUr��v����^����z�\&���������{��9��v6����XG��
�+a������P�h�r�7E��uSa���l?���2��}������%�K'f``r����Do�J��a��@O]�9
&�NLedQ��/	K��������s	n2K�K�y2
� ���6�t4�'�-J���w�^�����A�yQ`��2*�E"��2�h|!o:��d�[13��9|��;�2���^��8�z��z)��������H(�i���!l~�Y��PQ}�=�/���P��lm�H���W��% ���~
���R����+�6���&�
�J�"Z���,� 	���7���4�
��u�j�F�crF���ZVA��[��o
�u_��������et�0�v������?(�{m�j���g,oh���i��D�+R�f#r�@�b X�\p�uv?��-���OOO��/.O%{p#���X�5@�y�5(�/���e�zi�>;
�OJ��	��d��������mvGc)�@��SW����zTm����Wq�u�)�n�9�^��H���sI�
�������?���$<yQ��L������4-�rz�iZ��W'Q+����
�I��������d_R8��Cm�K�KNf�����Xna��:��KHb�C�M�V������z3V���Z(3yM�Wx�"Y�@!���FL"
��?����b�,����5[�n���d��~N��h����%p�A�F�!&�$�?����$�4�b!���dU�O��h8��(�����b���,��-��&��R�/r 4�R~n7�y<1�X6�d�n���d�dN���)���euw3M�
�/3oE�_�r�,HUX%+���plL�C��
+����$b�qCw��-s���o��k�W �{\����*g�3�]��o�3!Z�c"������h���e�/hd����Ru.�<�18�\5����
�v�t�+�Y
V����E�%c�P�i{������Kn^u�$��sO�dB�+Gl���lOW�f���.��MNOqW
?W'�F�� ��$�S&���,=YwV��}��lw����%��L�f�ar�E����^����f&:�<���S*8����i+�V��Y��BD�WH���������f�$��wvp��r���Pd����3I����0�H	�I�>�CC)_Q>��w��[f��%i��B�n�Y��#t��h,`�5�o�A���!�x����'�{�\�j�gEf[��*�DzU@�J������_�1���^���yi�/�J�SEoUko|�j��{�5Nn��@f^6d�v\����������k��oD�����d!�`(�/�L�"[�._40���o-_H�^�����v��A�Q�,�}���/�:���`�~"�W��5"��Q���B�c�Q�s(^D��B��!Tw�Cd��{SI��ZD�] t�z����J`V+�n�O�OG}JgP��"89G����b�4�>a�}h�$��:�t}x|q=���4�\��``��B\X�T���e���Y��5�Ax�9j 4(v2
�E`.$���*Q����-�jN���G��g�g��wy�A��pJ�pFt����o
R���VY[L!��cB��1�����������
��t��w5m��������`V
�	3V���3+�1�z�����a�nx����	��W^�`\����7Y��6��|�U�����VF���)dZ�;����N����������������e���w"~�X2��e���5Q`���K��DB��
����0��U���X �d� ���43��������O0V3�HJP���T<_��D4L���$	����\N��=�Q/��T"���X�!��L%Wt.>����[���Q`O���/^ �i�}TTV_��4;&����n��F����|�O}�����6�����;�A�0153����"%$L���h��w��l�^Fg�+�?��R[2���a���Kvu�*���1kt7�8�%�2���=d?5�T�ML����V���{i������=is�?�U�3�<='�����w�t]$���P���d���T�|��\5I._/����V�������~#�	�/Dk����gd�14�) i����3�_�=[�29g�
L�"��B��Ol�VW����w��"A����{��=O��Y�����(�Bp8�T���#=��xHw�^c�.
9>��
����"5(�bM9���_�?~xw�~v�Y�Cs/��^4�������[��p~�������6�n����3�����Sy��(~��E�?�+QZ��l�C����X
@E��K>�������R���V������>�
�EOM�2g0k9��?������@���G[a@���9\d6^u��;{s�"$x������G'�h���� 9���^}�Q���!���nx�������W}�������X�3�*0�4H��
�%�lr�l���B��Z�Xg�P/��I��D������X|�!���+���P;�_.?8L���������/.������?\_����� ������8mscc���UY���������MY���E4��9F����"0)?��Z9q#j��@���DMy���v<I��W����VJ^�*�_i.�!������Uv(�/8���p~������"�*�;j�|zl�b}B#J���Q*0�/�m
�/��X����`iO���I.\����/��������U�VH�&�x�=��uW�(_ {�$b~q��`�^)��C�'�tXr��H�1XgbP"�i|{�_�$1�@���Do	a�������ED#�p��������e������������2b�']y"��]�u������r]M3���SXW<����"
�J����F��_�h�������@�"������H��c����X�M����/�Y�k�����>v���_�e�OG����W
(����/�����xu��S\�(9�e8�/s����4H�Lb�d����Y����x����PW���>�y�[�������de%K�������K������Y�~�&
��rIC�H��V_���,v*e�T].��8�����G
�������Z�\������.��2�%S���Si3�*�����I��q&��Wu�PQ�S����R!�:l�)��B#ZI^��p)��X�	P���`����&;s������@�<�]��*�,kW��N��$�|)&HY��U��U��.5���q(��\XU
����0�Bi
��C5�ND)K����Y�_���d��8U�R�kg*U���7/����yJ���/�V�s�y��^�E����Z��:����>f8�6�����$8���7�c
+�������h����bnQ�\0�����yu����"wz5B�zun	8�b����F�rzB %(�[E1�q�����/u�7�z�&q�/J
�V���9�����[���B
k5�����V���[3r�H��>�Wj
���(����j2��eY��`G6*DH������bl�����o��yD��R����Z��y�K@���I��.O���N/��_&)�HtK�%#D�{����f��v�S��=�c5������J�����2�p"d,�>����q�O=�)�D�j��a�W� �"�u��\ym����;��%G#!i�ec���0��@�8l\a<t��~^c��h�
�<�T|����CT��Z�zf5,�Q���c\C��^2A���b�W
`"o# _G�Q�0�R��=@o[Hm{���J�(;��Y+�2g�2�T���qw�]����Fs��-Vc�	���*`|j��� �����	�1B�~�&��3=�%��4�kc���Ykq��q3��~�?C��U_G�����~��?\��P����x$�����_X���|����=`��]����S��|�z��Y�R������P�
:����0�5��w;S�����v����sBw,�� ��~���M����S,��0����X#�:	����fZx"��T{��=���bF���$�B5��X9\?%_�UH�D��9���D��[P�u��c����^Q��$��=��w��n�Pxh�l���8.��GW��n����iJ�fe��v}��e]���v��pH���7����[!
����J�vl�e�
�g��yA
1k�(K�i4�
F��tss���Y��������9,l=���fa_�d�ZA2�f�n�A`t���q��X����L���)��|����]�PLG����
{$���#�u�b v�_�A Eh�������p��|t�F��v����;\mv��ml�>�M>�f�a
�?'����!�E������,��l@����<���l>���pi�!G�Q���1���G��;	�/0��WoU��Q���o��V}/�[�xQ?�}0���t�q�������=�}Tr�)�R���a	��as���R
���j�M��������Sh��T��������|������4P��+�x�Pl����������2�!k�Yc�jk�&W���y�9;����N�X{�,;��
7l�P�=�?*��U]z��X*|�g ���C��f
�-�g��JFg��}{1�f������;{��Cmr��x��f���`����o��[_g���!^���P���m=O����H_J|�����j`c�.��&� N���T!�9���|����kk/�d�?�9����G\
��H(������ ]���p��y�����Z�r��Vk��|� PP�`+�"H�W�XJ3���H�i�:;>H�|��W��/����U��������=ll���(��7��:�^�!
>Hp�jt,�H����q�Zx?X��UR�0-?���/XX��k�zj���9C��_��������S�Yx<hS�hS�[�e4_oSX���L8�
)z9"��a<JZ�k�����~�:���Z�x��<l���	�������Wk{���KU�-�5n��#�
����q�v�:�����J\[�GrY�������<hY�lbf���R�4���]�����R�8������:��5�q�~�����uae����������/�3,��������&h��v�[f�5�S�@�y�D��$�6�4)<P��Xw�q��j�U�p�_���z�(��)F�Q�����������:�Q�Jg����C��[��m�����(�Hqu��1B�M�fF[h�N��D��	H��{{g?$��c	�-mg���`�F)�?1z����>L�������@N�6b�E��`�7#����9�<�����/��s>���bF��-�L("�
������ en>���bI��:,hoB	�
"�4��i�������ypn�;�$l��s@��pkqF:�p�9���
���{E�M!��bat}'��1���L��r��4�#����A�b?�B�f�?PW�m
�0���/c)g,��i��E���v%l!����P
��h�k�,.�����v�e
�2����G�{
A�"hm�>��5�P"uM�r�M���q��i:H���/����#���P/��&���n:�x�^o,��d^��-#�m:.���gQ<��n��!�M��J4�$�
���/f�(h|��
���M����EkA70g5<��(y�!nqOor�B�x�Y��%:!�w�������yh��# U��9��C�`4p[/?\�NnX����V�
o�^��<�z{���[p �0���:Z��s�W(���9�����)Y#j��h.����ch��aU4��8���Hb;8���>3�2����?�T<Z�U{d
5�����0����\�~��c�/9s�4�������� 76�r�B����9�M�RG�������I��b�nal?����A+�v��`�����!y���Cd���dJK�`����0;`����X�9���������W��5���F7BW�|��	]_���_@k$c���
�6U#$���x�����������������/�:���2��0�*o�>����
���H����Q�+��$�<j���A�eD�� �����m�`S�E#��,j�,��<�s�D�j�j�j�^-�.g�lQ|������^��,��pC����������Q3zE"m�����bK-��?F+6*���T��gq�H�M-�*���a3$����6Tq�z"Vu��9:��[s9���NGXg�@1J��$I�������
s0�E��z-��W���n����L
�{������9����u(i��\a���-�p�o���`�h��0�~6���.*e��86����P4a���1�,���3�LZ���+���G<wY�����d�YA����_���J)c�(��]K�t���M,�*�UL���3�G�6j�Vr�3
WY@ecQE�\��F�v_�-���m�iD+$~�'0�� K�:�����C7\E"���T_;1�rO��n���0��l��r�V���sO�$
���9��h�����e��T�.#>���
�?�Y��c�5g?r�����^�	�/�����(:e��0�����D�	�t����X�;K��6L�Q�}�g��q���5�����kZX��l����]
��e�
�1�.������9�
�������\����wv`+L<YZ���J�������[�t�w
JRl��z�.��`	�%�l�D���)�Yjp�d���b�Bd���b4h��N��
����V��v�'����g��
}��0I��sS�].�A�<jp���?\^�9�t��	���)�N���4���	��t�4������������F��]��}���h�B`:OS���5$\��.�'�����}����F����q!rq� A�|�"��QdDlP ��\�Y����"�����]*����kP�E(�������������:�����B����Y���"vv�2�5\�������>u��A,S|���g���pUp4T9�4!GRY�r�>�2�h���?(��\��`�>*T*����B�����+kt.�t���dI?ai�M6����������x}����(�%l��q����h��|�	�����g��as��la�,��y��-c���~���-^���G�~������_�o����d�����JY�fq���>aQ����a������N��R�E�/��Ie�S'�TL��q�����s�����k\��k���R5�
�d�x��{���I����xz���@�qt���Q��S��^"��aFo���9S���^���&�����r�[_����&,/�m��A.����	��6�tv���8�N�5�����+��Q`�1�[&���CEC9��2~��UU�D����P�NsDS�Bv)���	D#A\����������!���F���!)I}o	�]�z��<O�4o|�k���X�<m�)/B+p����;��X0rd�:}+�X����c{�?�<C�8G
�{x!e��-�;O��q�����2����)�3O�N�)��N����f�1��?5�QB�c~�<I�y��}t��?|��
�y���(��.��������\�tL�(>���	��eS��g���-H�Q��w����P��a]���0�$>�C���0��-��
������II�@�� ��Y���Pd�k��N'����?�A���.;�����@��i�[��>��7	AG����f@���:&4����f�.H����`��5��T�^H�t%��p�J>�����
�||����l�������R�Us��Cn�p������������+����Q����v<�}��P�"�	|<g�b�����~
��{�D��?`��x�qZ����{lDhOU����W����~�����(#��m�IzL�'K��3|�����)Jw���jnN�
��P��v���c/���S��I�/�������	:	7����&����qru�Se��@�e�4`:!%
�
,c}`7Jr7���HqU�LV����!���4��X�.��(�@�����83�>-�������~��
��g���	>��f��h~���L�9LX���|C����J���^�+|w��F�� �i����m�i%0�������&&�*tq:%�D?BF�SB%�7i��J�p>���&���"����(&��2G���<O���F�bNK������IJj��A*��S�@^�"�[oKw
�#���p|�<��j����:UY�
ST�X���rII�j����};U�����l���Q�Lb����gE�R��cX���s�U�Yb���	���6�`$���s�9�N���6�>���X��GxZ�i� ���\�!a�G�����>F+	�$A����Y�#`�L����BV��;WE�����I�z�Q�h���f�H�7�)�����:O���$�&h�aF#��b���}|
"��B,�d������!;�;'����	��L�_�K�����j�����h�����t+�����{RLq�JoQ�'d�J�I<V�6�������
'�C�(z?yt��
���`��f���S6h�gS�m�(����;#d�1;��32!�&S?@�6
���p��N���
q���%���aS����P��Q���?�9�g������T|k�+s:��e���LIR���D�r�$1��NI��`:�7�@x�J'�������#f�w�=��o���W�Ud!F��� c�e�>��.����U�:�l���'�����>�
\F����`�|i��_�/I��s��?��8�X�������^'�(��!�*s��98��zo�"�_��P��%Ttf=H�L�<���|tL���/`��j!P�A��WV$�-��W#y+�Ws��e���*a�G���Bb�-��i��3��1��E�z��-�-�;�xe��p"[Q���A���$&(��W�Mb�����oO���pAt�s<{�=6�&��Z�����������lZm-�]
�2��*���-�v��R�@k�4b��
�\�G���� �E��7j���|��d��L�����ds��Xp�o���N�e��w�{��n����������Vg���F����������zo]^[[��f��n4�5�A�����������3(�wq��������h�.���;}wv�^�mo%!�l3��0sI[����lz����R�] ���x����� ��f��
w����0�������'��(�C3����&L=�� ��LJ6+��'�~�4;h��d��8
����V���!|��p}�#t���]�"p�$�������O�j8�R���L.%g����g!�=�N����|�� uL��������f�q!��x�].AGQj����z�rh��$��D�HJP�����$V�����������`v��P6�@;zD�{��%�����s�&�~����O��P�g{b���k��L(��
�k��m�d5�!|kas:����T�
��;�Lz~��0:�:W�U{�s�3�(�u��G���B�<d=9���
?4Fx��}��S��a�Hi�X2����<d�������G���'�0�M�H��`�*j�zF�)���8�o��3��B��
KFr�XQ�$7@&x���n(.�h�J��(���H�)�U��;�`�5�P!�)��VX5wR�������
�~^cd�Zp������S�i	wg��"L�O�=�M5�Vx(��;�p=!@Ef<CoX}C6�j~�
��0K)���2��Yiq�����Y"%������
C2H�H0I>m�2�P	��Z$���W�16�
5C����D��+��Fh�QP0���$�M�0E|�W+<�d���	)�)V)�A�]?J�gG�����=�/�n����
����c��=6����`b�=;���9an������2T�Y�N�/'��C��YB�����pH��=�u;�/ �!����[E�P�������m��M
��Y�:���)�,��6�zt��PG��yv[��)W�'�VN6T��9�z*����Q��(d9�n�<���<UR��H�0�Z������p�R�� V�v�||
~�Wq��7
UW|3mH�V�Y��p�@2G���q*��
���d���MM�.�b+��`��.�� �DYts�1��^��j����?���
��d�����'��)�-r�u�?��.�����"	�}"N�� ����X��Y�p�Tn8%Ky�q��x��Iw6�'6�����'YZ�6W���Phc��p���=%��K����YI��dK2��'���@�X��JLR��c)����P'��]L��B����rS%����z������R%w�����!��3��a���������!�����6�Yo���t�������S���|)*�RUz�b@7�/�8�XD]C���@Mx6��D�7"���V���3�n�E�vO�=Ev�H�=�����E8"#�*f�����^&"�x��D���L�y���m�����K�`���|��zl/�?�Cp��7�ja�����&��`C����']�XR�-�B2Gy,Za(L�������d�������G>63J@NP�,���t,8��|9��b�2
dX�5H��U�X	�����K<��V���1�� k�X��B�����MF�`l-P�/&��&��F`��Rl����6DpK�O�s�AV��K��f"6{V�����{`���\��h��E����#I��4|B�I0�$�|����HQ]PG�@�k���l�J�F��p�?���tS
QPm����\��[����g
�������5�S�����2�i�M�����>:�
���w�I�9�E0.�^V�:X:5����g���1+I4Y�O%��_��;���z<QF=_���`������DO	OAQ��}�>����[�.m��N�f��B�@���6��_���������d�=�1L��������7@:^cE��|S�Q�<\���cb$&���m�)B���&u)��9��T������n����jv�c�E0buM{�}�PP�����M����Z
*l���*�-Y�:��P�������e�3�5�Y6�IhFu�EM*{�����������PQ� �3��T&����!:������
>u�d����/r�!��
�A���D�
�J�!G_�d�
�}����9{��S�kS����i���G��t��� '�=x��1YZ��c�=�@�������r��R���}���)�n��X�H������uRs�����C�Y�d��L���� ��FxS����L�b�d��&2G�'f��+��D�a�
�,t�'����M��OW1���R�G9E��	G��wq��U�t�)����&�������F�������w��� iC��������+*\�������"��~�oP�i6����S������1��@��R=���U'`o������35�j���,���J+�����L���9B5z�;��f(��`�%aXa������!�e�x����yk�R�R]g�
\�*�"��YAF=_-n�)��~��/�{�/���Mr��*?Dh\������s	m	3���y�(;/z]O�fm�!���%����a�X�X:�33`}K�H��kTH|(�����7�$����`��������H�w��D�����N�+
�V�^\-�a�>���V����!�Z���il�j�_=�0s0�$T��43���H}3F%���B|�y��D,s]��Q����,���K��
��S<�!�d>N�x�w� �����]��2l�N��1���Vg1.n���V)�Rz�������K�#~�,�W���������N��qM��#�������Zl'G����C���O���'pP�b���Vf��%+�l�"b�q�t�^��������B3�����w��������>���k�Jm��>���1%���39R<�%����;oS�n���x�2g�L�K���������V�I�%:_�3��9��s*N�����{�7>�7|�j~�����%����������#��@,@�.4;r�����i�>P�\�[���9X�D��6	���D#Ni��A��R��n�4�l�����t�E)X��������nl/L��	~}��v��v�1;9*����EF<�`��u ����wPn��6��k���7��UH��XO�������k�[�d�Y�{\sQ$6��2,B�fO
�'j��`Q�8��wA�����B�1k�P�M����v	�����x�]��
@��_�N�����j�nh�Q��')���+�As��W�b���� T�,3������:KW/*��XJ��� ��G��?5W����e,k�&~7���J8DO5���"N{)����a'S�d�Z�`7Y$nk�^i@�����B��"cN�����b��.�����@�&z�D���|s��t�i���/���be�e�C���+�����tc����R�2i��*�~����S�����QH\���7�$HF�%�B�b����}������_��4�;4q�;bu�����1CT����M]�����mrF
���!!���6��(r����.�������EK�S(]Lh(T�?Uo�4�Le�g}�5������+
������_yNGp�,�������,T���w��A���`��w�����=�{��k�}s�4Akq���&L/X�8T�� pJ�\������6��-u}4[+�<���<(��.N�9�� B���5<0����q ��|����l��v�VJ���Y���������-h
���`�)}
���E��F4�30��}@n�.�]#[�Bc�,�o�B.SOV8P��)�d�Y�
�"T���KHw0%�;���
���Yr�x���sZh�W����b���k�Tp��/��L8|�c�&S���x��E*�4�?�	c�.i �2a:#'>�,��;/#�nH-��M��xh��[+Uf2u;L'i��
u�G�{�a��
F�[���R��T!)���{�v�aqi�Bh�=�V^�+mpc��r�2�����j�4uQ�:������2`-k'�]I�o<��.�p8��9�v6����0�xD�;��R�L��D��g�<�����kh5<�%lI2��F�5?&���!��<���������u��6!y?S�/:�����i����>�
ZpsJ��L��Q��s\�5�+`"���9%��V%
���s�^��b�a�0�.Tb�����RWv���}Yf�"�/(1�B#.VPf����Ux{iaB
_�]2������8���XJ�����F��X��=�_w�j����(RO����m����'V���n1k�������t@��������!��:�P<u]����
�<0j��
LW�
lx^Q��u�%��Q�:�T[����aJ�3�q%�Shg�b`���JO/D�K��{�x=�XH��� A���	������0�p���r�;u�@���i��0.��+����?)�����Dz�_2����u�HU�����{�oW���S�$=���|�5�:����T��d����^^^\6�'�Q�e�t��Z'��@}.aV���t?z��6B��?���@1^��)��w����`<G��������v��~���v�Jko�g�P����J� ��[4W+��X����S�&�
�P�f��l�ZE�������M{������-�O���0���'���J�P��(V���^��(�'���������&�����^�q��`�������[�Wh=�T_
�)Sp��
Rl%����J��a��l�g�C?]�";l�}�	�D�k��Z�����5���0�}#lq������LPV���_��oM��k����!]s�����k���[���5"����M
uxg�q��nG[7�Tf~b�j��O&$+%�$�������>z�^i�n�jhA��M���M{m��-�}��!r��- 
qO��k����o=D�c�r�����w����e���-r���=��mA�" ����D�V���'\��P�)FjV|�)�zr���C�
�
,�����g�H��[��|���{%�kG>IsRc���h�h�����3�%5
��@B�dr��
���Se�#!0��j�>�����v@�l��5����m�5���KA�-d�K�������^�WV�:b1���r�Q����a��1��4�$}��i:�|s����R�a�|�jS4=���Ls*	��!��U�ie3��ELh��f�g�4�f���Ke�]�
xA$?�������4�0��VS��ySg��g�E9������e��N��:x,��0DX�_��)�_�������J�hr�~��<:?~ux	"�,3�8;�9����Cd������t�m�:�G�Wz]�������e�����o�0�#��;�����GG�9}N�'����'4n�a����/�m�������r�/�I��p��(�V�a9�3�TGo��s�9��������p�r"��2�W��\r��VF�H5Bz*��+��f�Lt7��<�b��,�4���{m�����1K�Z�qC���C+\�pXD4Pi�L0J���?��b��,�)�N�=S��z4�!����w-�N�0�u���E�b��{�e
��T�'��V:d���
��
���[%�.YJ5���$c�%���:<6v���]��<���X���o3�ZDa������YN�� ��&�����H�A�V�W/h$4����6L�v"�W�
��_������.t�O���x�A= K$�+TV
���^�]j�t.7r�5���%�n�<C \����
��{����Tm��h=�+lk4�;��D�?�/�djp���n��^�p��7��SJ��y��B��)����l������;�m����p��`cco�p/�m�v����������65��C\��}0��	��
0�)1���7��{|����W�G��.���N�����������`6U48�&��"��-���.�x�]|����Dsw!�M�6�<�=�������Z����F���9����[;����l�&h�oa�A�����_M��l����I#��Cl��\�]��/�N��AO����[A���xW���������N4�'�ol���r��� 7��pW���\�D��7�;q���q���Hw�����G*��
����[����@���TKwyztR����t��i������3�c�VV�I�%�d�	����
�������O�[��x�s�Cso�p����:���.���m�A������������0��$h�G��\����s�������6���y,���M���{G�M��	��������w���V�r�Zp����cc����c>����O2����X5��.��w�WIA����z9
�0��]�X	
��r���Zv��	��~�����u�
����t���8z��_%6���Qh3�Y�UX#uS��`X�6@��<��F���l�����p�m�c����
~�[�����i�V������-/ �l4��eVMj���dt3%W?Yu�hp|�72$�c�h��q�����a2�l����?��e�(�
q����d2g�������`*�m!�<����G%���e�������C���#��G=��t�����#E���p O�_Fg�h���hQ�Q`38��T��H|!��JGx
+�mE��1LI/+�M�S[�U���;�<��C����+�X�v��������1l��MVLkNj�L������:��_���-� �8��^��[�����h���������y ���'�U��{.�=��r��]Y1mq$LG`!���K|�'����"���5���aB����mI�������3Tm>�%���Wwix�4�.\8N�m/�C1�H�J[@�0��b���m�F%~��03���7z?2����������]mo5[$N<���K�\L��n������V��a_kv��)�F�����t���Rl��R��j�B��D@�@����%"���[��A��0 ���}WW��D#���6���^^��)�K�[3 �H���.���^����H�a���T�����S�6X5
��p��DH�h�q,���������������j��{�Uv���S2].T�/�>�����tA�OL�Lb�.<B����i\!�Y�������f�j^B��4/��5���Vk��;�3Dk{�w��L:����5�\��j�����A���X�nI����#�TF��R0HvT���P�/�BM�.7!�!M����`��D�^�.�:�Jc)3i,:�~NCc��8r��B����V`&Jz�p�47ln��~7m
Z}J��?Y}��j����a���jg7Ql�4W����������;�_��)�}f�@|K���%����%��="�Cb��\.�X\<�P�6��~�h8�\�HM
�H��,�%<�i��G��~����BX��sGD���*��.a
i��@�v�q�.yJ���:�����h<���$���+���]����p}:�6$}����n/���W���bW���kld<�X.s��G��7�X�)��(3���XS�`�_��X��!:A
;�A�vp�8�=�>�E��~Tj_	�e��gU��*�JQ��H�@����v��1|g������H��8� D*w�P�x����d�v��&��Gf�J�r����yt~����$f��E�+��-t�/e���^=pu���9�j��EP$��7���{����Z�iR�u�
Q�������J��G735m��C�U��Y2RL"��\3�'�N�^9�	�M��j����_�go�^����=E�d��1@`4x�*	%��k�7��X�L�$�#w�����\	��t
r�F�{.�*ZLm=�WO	H_f�M��/�
l��{�q�18�g,�wja y�i���AO������u�0�r���&I���GH��/�A���A>g��$RC���2_�� W����~<e�,��(q����v��Iq�3��i�Fe31��T�W�#��t�	�M���e�cu��Y��S*oD7�
]l����o�M���^2�(]c����/e����*�}6��q<���Pe��fY�/��8e�`��l�_�Y��p���0������2n�~�3S,�J�k�J)��t�i��������g�5�oC��t�j� )�>2�mH�P��dHu2�z�����b��������eR�-�O1s��J|}��
����~Yur�I|[������r
����E��z��d��v��T�����Q�U��m�dr,�[a��[���`���kv������A��s��wj-
cV�-
��(E�E+�����`a�X� ��������&���������;���|�����pX�E��t���Dt�(F*v4��Q�!�=�Ju�+�)��
YA�X��y9��Px�qu�Xk/��)�������mf��Z-������v�9��d�a2;��	��,_I4^�{�s�U���!g���k�g���9l&��N|������n��&�[�u��?b����r�������E�WT]�!Gz��
1�Y�O�2U
'��TBm63��3:�3�oX���^#�M���-���%$z�0A����n-�'�=�R��Uz�(=m&�gQ��K~���D([��X��X��?�%lS�P �s�f]�^�">�.��J��������\���8p�B�����r,��N�U��/Q�.1)�.��;h�Q�����,YJp�������EO�Q����
�i��kK[�!����]<����f1W�
zt��z��d�� ����+��V��Vp����
�-e�\=��)�
T/�g���{��:;��������a���o�;;�f9K�Ud��V���= �4���*���[�E����+��s
a�g��;T1Q��puN�~z����������? =b�@�U�=}�9�ob����Fi������R���|uc�7�K��OH��5���V5�)���v��JG~GXW5��t��
��K�d-�|�%)���3>���GH0��/���h��QR���=o����o��]'��g��d��$����*X���{��p��Z/z������-)^�u����]�������#�)�+\^�M��b��Z��L(�����������������O���Bw����tW��.��-Rz#C�H|�l��l���
wr�
��{��ng� ���76Z���x7��I�q!<Z�^�C��.8���,���p�
�A�R�C	�*K�����8s@)�����2/���G�K"��.o�OH�Iz��X��U�	�?+]�+\74���7�naC�����o���V�����v���;��vEtJa��6�P��%��p�1�)�T�~z��5?���&�����'��?~%[7�3��	����!&������O��7�����]�^7���xz�_�������O��^��v �JUh��l���2�s
#�n�gR0��;X0
�(�x�p�89�y�>
M�`l���ek�������M����[����E����)����l�FC@�v�����w�ilubu�����^��u���EK%#WWI�����;��Y���>��(���J�����5�7��������j�h=b54O=��L�F�
8����@�U��?��/�A+�����n'V,ukwoo���Pt��2�]���J��~�<���{I������ngJ����o~d�'����?�.�����������;���.����e�����Hvd<�3���r�)�k{{�^�%��P
.a�X_*{=��/�3�����C�\�W;E=�}�~���J�~Y�f#��������pR"��jup��aQ����Ll]���^b�'�!�����@N������!/#���G?zw��\]��QR�a(�Ca�����J��g ��=��>��N��_�H}�����M�@�4�
z��������!��{?������#��� ��1I`��g��~#�M�4��W�sLdZ�d2�����X
�[�\�}�~W-U��~w��������./��K���:��?�L�zb�8v{���0��H�����~]
��&z���_d3������?��������
8R<��P�voO�NN/�y����<�f����x` \!0��������N�V�9D����U���B5g\��_�OR��S9jT{��:���8���Y���%@�0�w��8b��=�%�S�_�7t����z�Z�#��A����y�/V-uy�8��j�{�#�my����)[rA���x���%���-�
�<�E��x�����N����M�m�l��a���1��C����d�k2�ZT�D���Ah� Q���+���0#���5V�0�t�P�N��3��>p�
���$���m<����,o��\j�B2����B~L��#L)z���*!�Q�z�N2�������)g���d�q�3'����#(
��&h!$`�U�,�����Ja�����D++��!��P[/�J	v�t����I��o�����apV�jT���/�������Z@�MwfQ��y������m,=���9M��Ro��t��i�;^�=��n�%R��F���L��/� H�����5ON$��%�����~�Qh�|J���JP0��G
z��6�*��^0�i��D�e�iX�'&������&��x�O���rCR��e�QK��n��0�j�l��/�-Js��8^��y�6�B������#B���
��0x�5��0�1�4�]�0��a8i{M��.1����D�Ug�y*��@dr����0~1�����b�3Az�*�Es���GMz�F�mC�_v#�g@�����P3�CY�1G�oT�� 8�������
e21��"�
����'m����M����&~�a����j���i;�8����W�\��G��}��M����z�<���>qa�����D��Ptr�!���8�R�+I^,!�8���#>��R@9uU�|���I�1H��a�/2�����W��x�!����y���C���&`���P���fQ!�e���B������JD������j����54��@U�1��+�"c���"����w�o�u0]2�����Tz�@����3^�����\��L�/5����I3\�'�;����2����:����D�2����fY������$�z?I<l��*�JL�W��;�z�tT�h9�0$��O9>�d2�����'�������\y�{75m�
�?�
Q��'������$|��^m6a
�4�9�x/�
�'�h�Y4�)����'�7�	-W�5�U�Rd�x���hjb$f.B��Gpw+�������qo�8�L�l�+s�c#dM���z��Z���P�{�=�jrS���K�
��&�����Pv�Re�
����8xeb
�����cK�v��-E,�C&j�%��
��iK0d8Wm����W�3C���|:���� [�QI���?{���8�{�rj���r*s�z��9��b1m:�.�5�#U����T�s�.D#'�
mI��=`�$3�������J���h��'NV��T.}`�������)���D|FF��"F2�5��T�X�T=4��!s��py�GXG�?'�.&�ZDs�<��-��n6������n3���x/�t��j��]�Mi��j��w����q��X��OoE& �|��6rm5�mo=�@:5�]<_�J�2-z�����sI�R ���v�����F�}�"�&9!�5�(3�_eL4xP��j�3�0�d��R���M��	�!�1,7s<�
�����j��y�-�N�8@o���#0S
@]��D�`���>M��J!`��R���9���+�9@Sp��-��'��w����gh�e�
�z�E�;=D�!�'O5LF��P�<�������1��
�9�df�dZ����fyJo����i�
�m%Km�[��S����������A��;�v��q:�^w��m���d�D-�P�;p��"?�M���
Q@���O��r����������<~��:B���cN0S`��@�H����O���B#7��38J
�&W^��A�-�~�(��+��Y����[�r �;���8X����fN\&�hv�8eu<+��j��c�D�`���2_�`�H[��g#}�G�o|��SU��b�Z3�n�U�����C�'��T�k�Q��"���5P����A)TE�^�j��buw�W#
����f!,�V����J������rxa�(��>;��%3�����b�=�Q?���d��EA���n���x�+��w�a��$�9�^{�u��`9������Bo�$ %o��ER�����c���=���~��o��G/�S�\������������������n`9=�M^����� �xG�%-��?k�����c�
��9O�,K!�����D��#owz��;2C��'���)v��4�s)�����2��;�:������*������N��������f��s:�z��y�3��3-)�%�i�3�!������s
��o5�N�=\Y����b�{%�D�`�^M��(����M��(|�*���z��t�����[M:���
e����1f�"<�`>
�=$���	���g5����T��u��e��zH�C�.&�Lj��\+��A�.r�(����L���K_m��2M��g��+����\�%]��LC��'������=����I�
����D55m�G�6�x����|1�a����By��B��~��R�^`{��#b�`�dh����14=*"�i���!����x�m6����>��������Zn����]�4����%[R�
1� (_}X���+��h�^���^D��'Mb
��������U��W�ZPkra�����D�Z�*y�	�v~�h}@�g8��P4���m���9��G��h
�[��B��� �2���(v��������ns�r&7
��.�Mi[�h��|�B=�)(���$��[���Y���${
c�gh�{t
!j�Sa�kw'�d(�KE�a���7�5�O�9SH8�f/G��@��v<��kX�F�A��FS�%j!!J����P�~�q<��IN� ���[����o�`�k��1��A���N&o+��������hH��N����������T<��a}4W�I���<�d�k��XN�ES�D��'��E���'�����xm�gy�+;�l	m
!�z	m��V�N�94�1��EI����3�y��)�����TW��H���@��@S�=+J�,~?���o6�p���^�v�]'g�`����dt��}m'���J�#�%��+��$�R�����������N�^o���[8���&��m�o452%�G�5�&q�%2����������������������v��q�F9�Nvh����R�	�<�ir_������}l���)[�@��)�1*[��W�5�%;a"8dkB#�ckZ3�1mK�l����,��wG�)�=���<>?=zc9���R�Ds��%o��Ww8�C������;����0����u*	N=]D���n�wZ�A{%�A��k?�9���G���eM�z%;�����rtO��_��S����w���_���L���i��e���f�-��u�0^^��__\�<4�������/�%li����r�@����;2��PqX ���������9�v���s�n��r�^�!����sKBh�6�l�V�OlV�b�X���25�����/�K%X�&r"���2�\��.��	����@Yq��s���,��JS_�T4)��*{*�N�?��4�9�a<���r*'���H��#@F�^@���%���f$�z�����N��~TE��#�}Qra��'|���Q�W
�{e�z��G��+�W{%�Z�<T~��� *�L��0����HW}/��	A�|Fiw d���8�Mx���?�e��J"����b1�o*���-_d{�����v��\�j�"���-�Q��%��������p��;���P����QA�!���R-hW��sb�M���������s(��q���?q;Qd8��@8������b�����{�N���M���wbG��[lVQ7�#4l2��
L�_�iv�w_�q��]��V�K5��j��<�����yz�:�A3
�� c���>����(B����)S�&[�m���n��8A��D�8L/n�
K���:n�����dL�]��~u��$�BX��!�[��1L�b�
��p��h�3
\���U��h���P���/n�e��k�?�������>n��<���`W�;����.#��jz��6���kzS69����9���%Di2���D��c��b�cFh�	�u�qG�98�4t��L�Q�6��n��\6-�Y0��@N8��;��o+l�Uh������l��������N��#��|��Pxt�dc���`��n�B���<&l�����Q4��p+������9�"Bw��7��x��JP����f:5C�(l�(�W�h��!��5��n����Ipn�"0*�6�w�M��[�z���~����%����m[�����")'|4���U
��M����!���O'��������<�H����d\� �r*�&3�\B�,�)����!C����:I�!����8��+�B�R�R�JnS�������&>����e
���_�y���!�R�_��p��!Bn���-V�)@s���6���%�5�%��$�-"�Z
�7�nQ��S�i�_l:f�W�%n/{����@d���������gK���X�%�s,��d����w����'w����ojc����w}�N&9=:��v��+<���i`Q��oL�x�i�3t�w����]a�H��G[��O?���kr�U��~y��_q��	D�������E���o��V�������j�U{�h���Z������.�7t*��J�5��X~�����������������A�g���O�A���t:M�6�n�i4���p�n����p>Gl��\fe���9�d��8�>F��/�z&h�[�N�������7�������a�=��=��i��;'������
�9��r�A����^��P�@U�����:V����\���?���g�/�up ��~U4�����0n���8h6�6�q�/6� ~<�e�����O����j���wX-����7��������l&W����� �����^(J�&\,V�}:)��]�L0
n�Y�������7Fpp�H4���v��s�����7Cd8�������
��_�V�����G]�xT�#&�i���'������f�d:��7��f����I�'*����L8V�cC�B�]m(X���Uwm�N��K�c����6��}�QL�r�����6�0�:��"zs�@no��1��+x�����}P"�Akz}�����d���z�+Y��3z�IM�2	�mdY�;
8V��f����2��BO �����P�����^�������#��"D��T��k�9�f2[E������7�a�����RV���@�f�\87��]#]�45��`v��<,U�U�r�/��C�����gi����8n�u:��v<��iu�q�������J��Y���N��v�l���v��S��t������)�
�Z ��c��QV������	�HN��W�
"<���z����:R��6n��O��$�C�p���R+PB���@|�?�
����g
DOJkQ�L!�>�~|�XM�`W�g�2�"�|
���$+)a�2���cbT�ANVu�U��$��u�W��+"�~�2�-�<�����N�!���c1���ZWmgLIk���C��_�r���vB����
p�5��	r�Q[CS�8![�����
��E� 	��
'Om���8�������|e�����Qr�[����!!���!�O����\�G���7���>����Q�F��
�����������Bp�g8�S��W�Px�M�eJ�=������|)l	#Y�������S_�����9�^�,����v�ZN��UI�,��h�����~��72{%��<z���R��9qq^�6s^$�����NYT�r�3����}��U��O?l�~���)��u�(����{�z�]j�����I-�u=��@)KMsQ�`"Zn�',��*��4���D`��']�M����_�7����iQ�gF*�J������E���W=�����_�)����D�E���*�G�?��B������H�)�)���S��|� ,y	Nh@�&1�G���A.������-�]�cP���D��S��t24] � k�H`|�%������������.����kF��j[^�mn"�������g`����g7��N���Y���L�1�e��Z�2a����V:�;Y���*\���p�
k�u0���5���$1��	��F�W�;O(�q]�tQ+�X2%U�^v
=�}�C��e�Oc�+�y����&E�#�#Q����H[d&������(&���fN�B���F�L�E���=p��&�I�C1����T&����"����M(�����ISr������w��p�+YX��~|����P{�����_~���ic����V�3UL� ��`�Y1��|A��{�M��b�2IGd�-�����6��<����Y$��N�����Xx�����V�&����E����Z���7�����b>�� *��7��v>U����P��+�8�����F����;7h���N�MH��v.���M�jS6�!�f�s�F�+#Z��{6�u�������FC��Z���R�od3��t������B�2��S�q}��P^30��t��J�����b�4������������W*���Y6���)d���u'mt��<sy�����D�s��#K?���*Dl3�����1e�|b����[��[��0�1������6)����L�R(j1E����/����b��B���+���(����Ss�'�k,�������a���l������.��Z'1���J��Q��!��Z�Y�9��7@2������ER�Q�m�
����f���t�F��u��DY���DY)q��b_{)(��f���z�-��%��'�c)_l��b����\�O��JP%#��e����9i^C�n�1&Z���*!VK6������������<����z��I'���(��Z�K�op����E$����8�{�g�����@��P��v�o�@:^'����+��(W�������J�z{��>�,��P	�O���
�a�������h�fq������������N��;5�\G��-��YV3����,��k�n/U�=�����u��FVF����lFvi�7�F��g���i���Y0-oK�eF���L~��i�@rE����Lz��������e0��'#�1&�� ��l2F�����)���U��
C�8�H�r�O�?��_��$�����d(�(�a��4&���j7��D�z�Q�#x��	�&K���we����:�+�����Qiw�t{�KI�������?@^Gr�D�~p�M�%c\���az,��s�����k ���>���V
�|�r�hDan|����]�!U��\f��O�%�>-5�2&�Q
��v�N0\�3n���n��<�i��$�d�LE�Tx�EJ��� �����d,d}��H��:�W\�K�y��������j�7��$�rW��~"g�J��Q��P�� ��'�>k�hi��%�9��l��%�s� �,_�
���Gl�@�	����|i?������wa����W��p��?YP��u'��=p�q
Q�@�)��A(h��!���d1Ev)U+��j1B�����r<����F�r{��!��A$�������<����{�TH�H�#��r�P:���lpv|���R�Gx��S���v��&���D���UN73v|��|F���U#UZ�\�V\+U�`����%$����9DI=&7���N�������bcr�t��@��J�����pn����	Fjb��8G�9"�T�83���@N�8�%<]�`i���]p�>����^��5f��f���K�l���OpX��-���x��2k�.��E��('W�� ��	�p~��M�t�.������/��q�j�Z!
�Zl����8#t?����/L�h�����L!X|.2'*�d�eTV^��2��u���pZ��g.K�����)AWY�K���21����vH3l4���6���m���Fr�	Bw���oESWy*8i��xl3[�US�O���OkXE_R�rU��w��r�$l@"��)�����y*f���� ���b [�A=��>=�U��]��s�6h����OB�g��b4P,WDl.{(f�]�t�X��5���i�W_�T����x��K��Z�MN]C��F��WOE�]0KWU�)blz�����Y0:����p�T��I�F�f��q��zF���~LT�3So��~��	��El�d�����R��mU`8��Y�P~����Q������;e�"R��O���r%�f�*F�{��e�@��2������[kx�V��2�������2�v�nF�R��>���t�������|Zi�
�R��y���3���z��l��\D;����,����S���@L�3���,���M��^����s�b s��$�7���i��a�\!k�.�9��l�G~��:�=��J��{~R�_�\'I��G/�����k�����h��G	��GiWI�A9kb����"1��K;i���W��rj��\���d���V�r�=kA?�����f���oO�X7�������r�9�&s9�LjK|VO�h_�0���(X��R�/���
�@��C9����'ri�F	�?�S�����m�����Sp^���T��f��85W�g�aq������}��t������z����������9��6YH�5	(Q�����(�N��<F���o����$�7s����>Q���&m�|��t��������C��n��<��XOR���^,� �d0OV�drh�!��L| .�0���.���uH�8��L��P����IK�5	�6X_�$�!�^�$9���v��.A2��4�*k�ia����'\���N#M�%;�����m���h�{�^�m�?����w��3��
�,���V����;���[�^�R<��Ap��(��w�7c�)I��'#f�������o�}h.�}��8�K���n)��w,���j�0�J'&�N+���SG�������.�A�:��8Y>���36.��x��FM�{S��
G7 =���$x�*�I�i�3�X�����=�E��7�{�������������������I����S�3)��Q�5%l�b��!L9������>^d�Y�D�`B_�L�b}�Ry���*�I
�:��%0�R�1������b����$�_���u�_���u�_���u�_���u�_���u�_���u�?�|�X`
#26Rushabh Lathia
rushabh.lathia@gmail.com
In reply to: Thomas Munro (#25)
Re: Parallel Hash take II

While re-basing the parallel-B-tree-index-build patch on top v22 patch
sets, found cosmetic review:

1) BufFileSetEstimate is removed but it's still into buffile.h

+extern size_t BufFileSetEstimate(int stripes);

2) BufFileSetCreate is renamed with BufFileSetInit, but used at below
place in comments:

* Attach to a set of named BufFiles that was created with BufFileSetCreate.

Thanks,

On Wed, Oct 25, 2017 at 11:33 AM, Thomas Munro <
thomas.munro@enterprisedb.com> wrote:

On Tue, Oct 24, 2017 at 10:10 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

Here is an updated patch set that does that ^.

It's a bit hard to understand what's going on with the v21 patch set I
posted yesterday because EXPLAIN ANALYZE doesn't tell you anything
interesting. Also, if you apply the multiplex_gather patch[1] I
posted recently and set multiplex_gather to off then it doesn't tell
you anything at all, because the leader has no hash table (I suppose
that could happen with unpatched master given sufficiently bad
timing). Here's a new version with an extra patch that adds some
basic information about load balancing to EXPLAIN ANALYZE, inspired by
what commit bf11e7ee did for Sort.

Example output:

enable_parallel_hash = on, multiplex_gather = on:

-> Parallel Hash (actual rows=1000000 loops=3)
Buckets: 131072 Batches: 16
Leader: Shared Memory Usage: 3552kB Hashed: 396120 Batches
Probed: 7
Worker 0: Shared Memory Usage: 3552kB Hashed: 276640 Batches
Probed: 6
Worker 1: Shared Memory Usage: 3552kB Hashed: 327240 Batches
Probed: 6
-> Parallel Seq Scan on simple s (actual rows=333333 loops=3)

-> Parallel Hash (actual rows=10000000 loops=8)
Buckets: 131072 Batches: 256
Leader: Shared Memory Usage: 2688kB Hashed: 1347720
Batches Probed: 36
Worker 0: Shared Memory Usage: 2688kB Hashed: 1131360
Batches Probed: 33
Worker 1: Shared Memory Usage: 2688kB Hashed: 1123560
Batches Probed: 38
Worker 2: Shared Memory Usage: 2688kB Hashed: 1231920
Batches Probed: 38
Worker 3: Shared Memory Usage: 2688kB Hashed: 1272720
Batches Probed: 34
Worker 4: Shared Memory Usage: 2688kB Hashed: 1234800
Batches Probed: 33
Worker 5: Shared Memory Usage: 2688kB Hashed: 1294680
Batches Probed: 37
Worker 6: Shared Memory Usage: 2688kB Hashed: 1363240
Batches Probed: 35
-> Parallel Seq Scan on big s2 (actual rows=1250000 loops=8)

enable_parallel_hash = on, multiplex_gather = off (ie no leader
participation):

-> Parallel Hash (actual rows=1000000 loops=2)
Buckets: 131072 Batches: 16
Worker 0: Shared Memory Usage: 3520kB Hashed: 475920 Batches
Probed: 9
Worker 1: Shared Memory Usage: 3520kB Hashed: 524080 Batches
Probed: 8
-> Parallel Seq Scan on simple s (actual rows=500000 loops=2)

enable_parallel_hash = off, multiplex_gather = on:

-> Hash (actual rows=1000000 loops=3)
Buckets: 131072 Batches: 16
Leader: Memory Usage: 3227kB
Worker 0: Memory Usage: 3227kB
Worker 1: Memory Usage: 3227kB
-> Seq Scan on simple s (actual rows=1000000 loops=3)

enable_parallel_hash = off, multiplex_gather = off:

-> Hash (actual rows=1000000 loops=2)
Buckets: 131072 Batches: 16
Worker 0: Memory Usage: 3227kB
Worker 1: Memory Usage: 3227kB
-> Seq Scan on simple s (actual rows=1000000 loops=2)

parallelism disabled (traditional single-line output, unchanged):

-> Hash (actual rows=1000000 loops=1)
Buckets: 131072 Batches: 16 Memory Usage: 3227kB
-> Seq Scan on simple s (actual rows=1000000 loops=1)

(It actually says "Tuples Hashed", not "Hashed" but I edited the above
to fit on a standard punchcard.) Thoughts?

[1] /messages/by-id/CAEepm=2U+%25
2BLp3bNTv2Bv_kkr5NE2pOyHhxU%3DG0YTa4ZhSYhHiw%40mail.gmail.com

--
Thomas Munro
http://www.enterprisedb.com

--
Rushabh Lathia

#27Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Rushabh Lathia (#26)
1 attachment(s)
Re: Parallel Hash take II

On Fri, Oct 27, 2017 at 12:24 AM, Rushabh Lathia
<rushabh.lathia@gmail.com> wrote:

While re-basing the parallel-B-tree-index-build patch on top v22 patch
sets, found cosmetic review:

Thanks!

1) BufFileSetEstimate is removed but it's still into buffile.h

+extern size_t BufFileSetEstimate(int stripes);

Fixed.

2) BufFileSetCreate is renamed with BufFileSetInit, but used at below
place in comments:

* Attach to a set of named BufFiles that was created with BufFileSetCreate.

Fixed.

Other minor tweaks: I fixed a harmless warning from Visual C++ and
added a CHECK_FOR_INTERRUPTS() to
ExecParallelHashJoinPartitionOuter()'s loop.

Two other changes:

1. Improved concurrency for sharedtuplestore.c.

Last week I investigated some test failures on AppVeyor CI and
discovered a small problem with sharedtuplestore.c on Windows: it
could occasionally get ERROR_ACCESS_DENIED errors when attempting to
open files that were concurrently being unlinked (unlinking is not
atomic on Windows, see pgsql-bugs #14243 for another manifestation).
That code was a bit sloppy (though not incorrect), and was easy to fix
by doing some checks in a different order, but...

While hacking on that I realised that sharedtuplestore.c's parallel
scan efficiency was pretty terrible anyway, so I made an improvement
that I'd earlier threatened to make in a comment. Instead of holding
a per-file lock while reading individual tuples, it now works in
page-multiple-sized chunks. Readers only acquire a spinlock when they
need a new chunk, don't hold any locks while doing IO, and never read
overlapping pages. From a user perspective, this means that EXPLAIN
(BUFFERS) temporary buffer read counts are approximately the same as
for the equivalent non-parallel hash join, because each worker reads a
disjoint set of temporary file pages instead of reading interleaved
tuples from the same pages, and there is no more LWLock "shared
tuplestore" that can show up in wait_event when backends pile up on
the same batch. It writes very slightly more than it reads because of
unread pages at the end of the final chunk (because it reads back in
tuple-at-a-time and thus page-at-a-time, not whole chunk at a time --
I considered reading whole chunks and then returning pointer to
MinimalTuples in the chunk, but that required MAXALIGNing data in the
files on disk which made the files noticeably bigger).

So, to summarise, there are now three layers of contention avoidance
strategy being used by Parallel Hash Join for scanning batches in
parallel:

i) Participants in a Parallel Hash Join try to work on different
batches so they avoid scanning the same SharedTuplestore completely.
That's visible with EXPLAIN ANALYZE as "Batches Probed" (that shows
the number of outer batches scanned; it doesn't seem worth the pixels
to show "Batches Loaded" for the number of inner batches scanned which
may be lower).

ii) When that's not possible, they start out reading from different
backing files by starting with the one they wrote themselves and then
go around the full set. That means they don't contend on the per-file
read-head lock until a reader drains a whole file and then choses
another one that's still being read by someone else.

iii) [New in this version] Since they might still finish up reading
from the same file (and often do towards the end of a join), the
tuples are chopped into multi-page chunks and participants are
allocated chunks using a spinlock-protected counter. This is quite a
lot like Parallel Sequential Scan, with some extra complications due
to variable sized chunks.

2. Improved oversized tuple handling.

I added a new regression test case to exercise the oversized tuple
support in ExecParallelHashLoadTuple() and sts_puttuple(), as
requested by Andres a while back. (Thanks to Andrew Gierth for a
pointer on how to get detoasted tuples into a hash join table without
disabling parallelism.) While testing that I realised that my
defences against some degenerate behaviour with very small work_mem
weren't quite good enough. Previously, I adjusted space_allowed to
make sure every backend could allocate at least one memory chunk
without triggering an increase in the number of batches. Now, I leave
space_allowed alone but explicitly allow every backend to allocate at
least one chunk ignoring the memory budget (whether regular chunk size
or oversized tuple size), to avoid futile repeated batch increases
when a single monster tuple is never going to fit in work_mem.

A couple of stupid things outstanding:

1. EXPLAIN ANALYZE for Parallel Hash "actual" shows the complete row
count, which is interesting to know (or not? maybe I should show it
somewhere else?), but the costing shows the partial row estimate used
for costing purposes.
2. The BufFileSet's temporary directory gets created even if you
don't need it for batches. Duh.
3. I don't have a good query rescan regression query yet. I wish I
could write my own query plans to test the executor.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

parallel-hash-v23.patchset.tgzapplication/x-gzip; name=parallel-hash-v23.patchset.tgzDownload
#28Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#2)
Re: Parallel Hash take II

On Tue, Aug 1, 2017 at 9:28 AM, Andres Freund <andres@anarazel.de> wrote:

On 2017-07-26 20:12:56 +1200, Thomas Munro wrote:

I'll report on performance separately.

Looking forward to that ;)

Here are some experimental results from a Xeon E5-2695 v3 with a ton
of RAM and spinning disks (EDB lab server "scylla"). I used TPC-H
dbgen scale 10 with the additional indexes suggested by Tomas
Vondra[1]https://github.com/tvondra/pg_tpch/blob/master/dss/tpch-index.sql. 10GB of source data (= 23GB pgdata dir) is obviously quite
small as these things go, and I hope we'll run some of these with a
much larger scale soon (it's a big job), but it's enough to runs
queries for tens of seconds to minutes so it's definitely in parallel
query territory and shows some pretty interesting effects IMHO.

First, here's a stupid self-join as a warm-up. The query is SELECT
COUNT(*) FROM lineitem r JOIN lineitem s USING (l_orderkey,
l_linenumber), where lineitem is ~60 million rows.

(1) With work_mem set sky-high so no batching is required, how much
speed-up does each worker contribute with this feature off (= same as
unpatched master) and on? In this table, each cell shows the speed-up
compared to w=0 (no workers):

parallel_hash | w=0 | w=1 | w=2 | w=3 | w=4 | w=5 | w=6
---------------+-------+-------+-------+-------+-------+-------+-------
off | 1.00x | 1.42x | 1.66x | 1.76x | 1.66x | 1.68x | 1.69x
on | 1.00x | 1.76x | 2.54x | 3.21x | 3.80x | 4.30x | 4.79x

(2) With work_mem set to 128MB this query needs 32 batches. Again,
how much speed-up does each worker contribute with the feature off and
on?

parallel_hash | w=0 | w=1 | w=2 | w=3 | w=4 | w=5 | w=6
---------------+-------+-------+-------+-------+-------+-------+-------
off | 1.00x | 1.37x | 1.49x | 1.32x | 1.67x | 1.63x | 1.64x
on | 1.00x | 1.94x | 2.72x | 2.82x | 3.02x | 4.64x | 5.98x

I haven't tried to grok the shape of that curve yet. Interestingly
(not shown here) the 32 batch parallel hash actually starts to beat
the single-batch parallel hash somewhere around 5-6 workers, and at 15
workers it achieves 9.53x speed-up compared to w=0 and is still
gaining as you add more workers, whereas the single-batch version tops
out around 8 workers. This may be in part due to the trade-offs
discussed in "Design and Evaluation of Main Memory Hash Join
Algorithms for Multi-core CPUs" (short version: partitioning up front
can pay off by reducing cache misses at various levels and some
research databases would consider that), but I would think we're
probably pretty far away from that frontier and there other probably
other more basic problems. Investigation/profiling required.

Next, here are some numbers from the TPC-H queries. I included only
queries where a Parallel Hash was selected by the planner. I stopped
at w=6 because that's the highest number of workers the planner would
pick by default at that scale. (If I'm getting the maths right, TPC-H
scale 300's lineitem table should inspire about 10 workers to get out
of bed; you get an extra worker each time a table triples in size.)

(3) What is the speed-up with enable_parallel_hash = on vs
enable_parallel_hash = off? Here is the result table for various
numbers of workers, with work_mem set to 1GB.

query | w=0 | w=1 | w=2 | w=3 | w=4 | w=5 | w=6
-------+-------+-------+-------+-------+-------+-------+-------
3 | 1.02x | 1.16x | 1.37x | 1.79x | 1.95x | 2.29x | 2.44x
5 | 1.03x | 1.15x | 1.20x | 1.44x | 1.95x | 2.05x | 1.34x
7 | 1.02x | 1.26x | 1.54x | 2.18x | 2.57x | 1.25x | 1.32x
8 | 1.00x | 1.56x | 1.49x | 1.47x | 1.40x | 0.55x | 0.55x
9 | 1.02x | 1.24x | 1.35x | 1.50x | 1.59x | 1.82x | 1.82x
10 | 1.02x | 1.16x | 1.19x | 1.44x | 1.51x | 1.75x | 1.83x
12 | 1.01x | 1.22x | 1.53x | 0.72x | 0.74x | 0.73x | 0.99x
14 | 1.00x | 1.08x | 1.18x | 1.33x | 1.41x | 1.54x | 1.52x
16 | 1.01x | 1.22x | 1.10x | 1.10x | 1.10x | 1.11x | 1.10x
18 | 0.99x | 1.07x | 1.05x | 0.99x | 0.99x | 0.99x | 1.03x
21 | 1.00x | 1.28x | 1.24x | 1.34x | 0.18x | 0.19x | 0.23x

Some commentary on the cases where the performance was apparently hurt
by the feature: for Q21 with w=3 workers and above with
enable_parallel_hash = off the planner switched from a hash join to a
nested loop and that turned out to be better, but with
enable_parallel_hash = on it didn't give up on hash join until there
were 6 workers. Something similar happened with Q8 around 5 workers.
Q21 has some major cardinality estimation problems as discussed
elsewhere, and on this run I didn't think to apply the patch that
fixes (?) that. In other words, as far as I can tell, all of those
are cases where there is possibly room for general planner improvement
outside this project: the point at which we flip from one plan type to
another moves around, not necessarily indicating a problem with
Parallel Hash as an executor node. That isn't to say I'm not
interested in understanding the causes better and trying to fix them
if I can.

(4) The same comparison, with work_mem set to 128MB resulting in more batching:

query | w=0 | w=1 | w=2 | w=3 | w=4 | w=5 | w=6
-------+-------+-------+-------+-------+-------+-------+-------
3 | 1.03x | 1.23x | 1.44x | 1.76x | 1.97x | 2.23x | 2.55x
5 | 1.01x | 1.07x | 1.25x | 1.44x | 1.79x | 2.05x | 1.31x
7 | 1.02x | 1.42x | 1.73x | 1.26x | 1.20x | 1.28x | 1.33x
8 | 1.01x | 1.57x | 1.51x | 1.49x | 1.41x | 0.55x | 0.52x
9 | 0.99x | 1.14x | 1.43x | 1.56x | 1.82x | 1.96x | 2.06x
10 | 1.02x | 1.08x | 1.24x | 1.38x | 1.51x | 1.54x | 1.65x
12 | 1.02x | 1.02x | 0.71x | 0.73x | 0.73x | 0.99x | 0.99x
14 | 1.03x | 1.06x | 1.19x | 1.37x | 1.59x | 1.58x | 1.59x
16 | 1.00x | 1.21x | 1.10x | 1.09x | 1.13x | 1.12x | 1.12x
18 | 0.98x | 1.22x | 1.28x | 1.21x | 1.10x | 0.98x | 0.95x
21 | 0.96x | 1.25x | 1.56x | 0.41x | 0.41x | 0.87x | 1.18x

Similar, with the inflection points moving around a bit.

(5) Another way to look at the data is to see how much speed-up each
new worker gives you with and without this feature, as I did for the
self-join above. In this table, there are two lines for each query.
The first line shows the speed-up as we add more workers with
enable_parallel_hash = off (should be same as unpatched master), and
the second line shows the speed-up as we add more workers, with
enable_parallel_hash = on.

query | w=0 | w=1 | w=2 | w=3 | w=4 | w=5 | w=6
-------+-------+-------+-------+-------+-------+-------+-------
3 | 1.00x | 1.60x | 2.00x | 2.07x | 2.27x | 2.23x | 2.39x
3 | 1.00x | 1.83x | 2.68x | 3.64x | 4.35x | 5.02x | 5.72x
-------+-------+-------+-------+-------+-------+-------+-------
5 | 1.00x | 1.58x | 2.14x | 2.36x | 2.30x | 2.57x | 8.68x
5 | 1.00x | 1.75x | 2.49x | 3.29x | 4.34x | 5.09x | 11.28x
-------+-------+-------+-------+-------+-------+-------+-------
7 | 1.00x | 1.44x | 1.75x | 1.61x | 1.67x | 4.02x | 4.35x
7 | 1.00x | 1.78x | 2.66x | 3.44x | 4.24x | 4.93x | 5.64x
-------+-------+-------+-------+-------+-------+-------+-------
8 | 1.00x | 1.19x | 1.28x | 1.31x | 1.36x | 3.30x | 3.34x
8 | 1.00x | 1.85x | 1.90x | 1.93x | 1.91x | 1.82x | 1.85x
-------+-------+-------+-------+-------+-------+-------+-------
9 | 1.00x | 1.59x | 2.19x | 2.52x | 2.81x | 2.76x | 2.74x
9 | 1.00x | 1.94x | 2.88x | 3.69x | 4.38x | 4.92x | 4.89x
-------+-------+-------+-------+-------+-------+-------+-------
10 | 1.00x | 1.45x | 1.92x | 2.19x | 2.36x | 2.28x | 2.49x
10 | 1.00x | 1.65x | 2.25x | 3.10x | 3.48x | 3.91x | 4.48x
-------+-------+-------+-------+-------+-------+-------+-------
12 | 1.00x | 1.50x | 1.76x | 4.71x | 5.66x | 6.61x | 7.61x
12 | 1.00x | 1.81x | 2.65x | 3.36x | 4.14x | 4.78x | 7.43x
-------+-------+-------+-------+-------+-------+-------+-------
14 | 1.00x | 1.40x | 1.68x | 1.86x | 1.97x | 1.95x | 1.95x
14 | 1.00x | 1.50x | 1.98x | 2.47x | 2.76x | 2.98x | 2.95x
-------+-------+-------+-------+-------+-------+-------+-------
16 | 1.00x | 1.01x | 1.25x | 1.31x | 1.35x | 1.38x | 1.39x
16 | 1.00x | 1.22x | 1.36x | 1.43x | 1.47x | 1.53x | 1.53x
-------+-------+-------+-------+-------+-------+-------+-------
18 | 1.00x | 0.86x | 0.93x | 1.08x | 1.11x | 1.22x | 1.15x
18 | 1.00x | 0.93x | 0.98x | 1.08x | 1.11x | 1.22x | 1.20x
-------+-------+-------+-------+-------+-------+-------+-------
21 | 1.00x | 1.12x | 0.49x | 0.59x | 5.10x | 5.67x | 5.18x
21 | 1.00x | 1.44x | 0.62x | 0.80x | 0.95x | 1.08x | 1.22x

[1]: https://github.com/tvondra/pg_tpch/blob/master/dss/tpch-index.sql

--
Thomas Munro
http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#29Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Thomas Munro (#27)
1 attachment(s)
Re: Parallel Hash take II

On Mon, Oct 30, 2017 at 1:49 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

A couple of stupid things outstanding:

1. EXPLAIN ANALYZE for Parallel Hash "actual" shows the complete row
count, which is interesting to know (or not? maybe I should show it
somewhere else?), but the costing shows the partial row estimate used
for costing purposes.

Fixed.

2. The BufFileSet's temporary directory gets created even if you
don't need it for batches. Duh.

Fixed.

I also refactored shared temporary files a bit while looking into
this. The shared file ownership mechanism is now promoted to its own
translation unit sharedfileset.c and it now works with fd.c files.
buffile.c can still make use of it. That seems like a better division
of labour.

3. I don't have a good query rescan regression query yet. I wish I
could write my own query plans to test the executor.

I found a query that rescans a parallel-aware hash join and added a
couple of variants to the regression tests.

I also decluttered the EXPLAIN ANALYZE output for enable_parallel_hash
= off a bit.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

parallel-hash-v24.patchset.tgzapplication/x-gzip; name=parallel-hash-v24.patchset.tgzDownload
#30Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#29)
Re: Parallel Hash take II

Hi,

Here's a review of v24

+set min_parallel_table_scan_size = 0;
+set parallel_setup_cost = 0;
+-- Make a simple relation with well distributed keys and correctly
+-- estimated size.
+create table simple as
+  select generate_series(1, 20000) AS id, 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa';
+alter table simple set (parallel_workers = 2);
+analyze simple;
+-- Make a relation whose size we will under-estimate.  We want stats
+-- to say 1000 rows, but actually there are 20,000 rows.
+create table bigger_than_it_looks as
+  select generate_series(1, 20000) as id, 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa';
+alter table bigger_than_it_looks set (autovacuum_enabled = 'false');
+alter table bigger_than_it_looks set (parallel_workers = 2);
+delete from bigger_than_it_looks where id <= 19000;
+vacuum bigger_than_it_looks;
+analyze bigger_than_it_looks;
+insert into bigger_than_it_looks
+  select generate_series(1, 19000) as id, 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa';

It seems kinda easier to just manipulate ndistinct and reltuples...

+set max_parallel_workers_per_gather = 0;
+set work_mem = '4MB';

I hope there's a fair amount of slop here - with different archs you're
going to see quite some size differences.

+-- The "good" case: batches required, but we plan the right number; we
+-- plan for 16 batches, and we stick to that number, and peak memory
+-- usage says within our work_mem budget
+-- non-parallel
+set max_parallel_workers_per_gather = 0;
+set work_mem = '128kB';

So how do we know that's actually the case we're testing rather than
something arbitrarily different? There's IIRC tests somewhere that just
filter the json explain output to the right parts...

+/*
+ * Build the name for a given segment of a given BufFile.
+ */
+static void
+MakeSharedSegmentName(char *name, const char *buffile_name, int segment)
+{
+	snprintf(name, MAXPGPATH, "%s.%d", buffile_name, segment);
+}

Not a fan of this name - you're not "making" a filename here (as in
allocating or such). I think I'd just remove the Make prefix.

+/*
+ * Open a file that was previously created in another backend with
+ * BufFileCreateShared in the same SharedFileSet using the same name.  The
+ * backend that created the file must have called BufFileClose() or
+ * BufFileExport() to make sure that it is ready to be opened by other
+ * backends and render it read-only.
+ */

Is it actually guaranteed that it's another backend / do we rely on
that?

+BufFile *
+BufFileOpenShared(SharedFileSet *fileset, const char *name)
+{
+	/*
+	 * If we didn't find any files at all, then no BufFile exists with this
+	 * tag.
+	 */
+	if (nfiles == 0)
+		return NULL;

s/taag/name/?

+/*
+ * Delete a BufFile that was created by BufFileCreateShared in the given
+ * SharedFileSet using the given name.
+ *
+ * It is not necessary to delete files explicitly with this function.  It is
+ * provided only as a way to delete files proactively, rather than waiting for
+ * the SharedFileSet to be cleaned up.
+ *
+ * Only one backend should attempt to delete a given name, and should know
+ * that it exists and has been exported or closed.
+ */
+void
+BufFileDeleteShared(SharedFileSet *fileset, const char *name)
+{
+	char		segment_name[MAXPGPATH];
+	int			segment = 0;
+	bool		found = false;
+
+	/*
+	 * We don't know how many segments the file has.  We'll keep deleting
+	 * until we run out.  If we don't manage to find even an initial segment,
+	 * raise an error.
+	 */
+	for (;;)
+	{
+		MakeSharedSegmentName(segment_name, name, segment);
+		if (!SharedFileSetDelete(fileset, segment_name, true))
+			break;
+		found = true;
+		++segment;
+	}

Hm. Do we properly delete all the files via the resowner mechanism if
this fails midway? I.e. if there are no leading segments? Also wonder if
this doesn't need a CFI check.

+void
+PathNameCreateTemporaryDir(const char *basedir, const char *directory)
+{
+	if (mkdir(directory, S_IRWXU) < 0)
+	{
+		if (errno == EEXIST)
+			return;
+
+		/*
+		 * Failed.  Try to create basedir first in case it's missing. Tolerate
+		 * ENOENT to close a race against another process following the same
+		 * algorithm.
+		 */
+		if (mkdir(basedir, S_IRWXU) < 0 && errno != ENOENT)
+			elog(ERROR, "cannot create temporary directory \"%s\": %m",
+				 basedir);

ENOENT or EEXIST?

+File
+PathNameCreateTemporaryFile(const char *path, bool error_on_failure)
+{
+	File		file;
+
+	/*
+	 * Open the file.  Note: we don't use O_EXCL, in case there is an orphaned
+	 * temp file that can be reused.
+	 */
+	file = PathNameOpenFile(path, O_RDWR | O_CREAT | O_TRUNC | PG_BINARY);
+	if (file <= 0)
+	{
+		if (error_on_failure)
+			elog(ERROR, "could not create temporary file \"%s\": %m", path);
+		else
+			return file;
+	}
+
+	/* Mark it for temp_file_limit accounting. */
+	VfdCache[file].fdstate |= FD_TEMP_FILE_LIMIT;
+
+	/*
+	 * We don't set FD_DELETE_AT_CLOSE for files opened this way, but we still
+	 * want to make sure they get closed at end of xact.
+	 */
+	ResourceOwnerEnlargeFiles(CurrentResourceOwner);
+	ResourceOwnerRememberFile(CurrentResourceOwner, file);
+	VfdCache[file].resowner = CurrentResourceOwner;

So maybe I'm being pedantic here, but wouldn't the right order be to do
ResourceOwnerEnlargeFiles() *before* creating the file? It's a memory
allocating operation, so it can fail, which'd leak the file.

+/*
+ * Open a file that was created with PathNameCreateTemporaryFile, possibly in
+ * another backend.  Files opened this way don't count agains the

s/agains/against/

+ * temp_file_limit of the caller, are read-only and are automatically closed
+ * at the end of the transaction but are not deleted on close.
+ */
+File
+PathNameOpenTemporaryFile(const char *path)
+{
+	File		file;
+
+	/* We open the file read-only. */
+	file = PathNameOpenFile(path, O_RDONLY | PG_BINARY);
+
+	/* If no such file, then we don't raise an error. */
+	if (file <= 0 && errno != ENOENT)
+		elog(ERROR, "could not open temporary file \"%s\": %m", path);
+
+	if (file > 0)
+	{
+		/*
+		 * We don't set FD_DELETE_AT_CLOSE for files opened this way, but we
+		 * still want to make sure they get closed at end of xact.
+		 */
+		ResourceOwnerEnlargeFiles(CurrentResourceOwner);
+		ResourceOwnerRememberFile(CurrentResourceOwner, file);
+		VfdCache[file].resowner = CurrentResourceOwner;

Same complaint as above, ResourceOwnerEnlargeFiles() should be done
earlier.

+/*
+ * Delete a file by pathname.  Return true if the file existed, false if
+ * didn't.
+ */
+bool
+PathNameDeleteTemporaryFile(const char *path, bool error_on_failure)
+{
+	struct stat filestats;
+	int			stat_errno;
+
+	/* Get the final size for pgstat reporting. */
+	if (stat(path, &filestats) != 0)
+		stat_errno = errno;
+	else
+		stat_errno = 0;
+
+	/*
+	 * Unlike FileClose's automatic file deletion code, we tolerate
+	 * non-existence to support BufFileDeleteShared which doesn't know how
+	 * many segments it has to delete until it runs out.
+	 */
+	if (stat_errno == ENOENT)
+		return false;
+
+	if (unlink(path) < 0)
+	{
+		if (errno != ENOENT)
+			elog(error_on_failure ? ERROR : LOG,
+				 "cannot unlink temporary file \"%s\": %m", path);
+		return false;
+	}
+
+	if (stat_errno == 0)
+		ReportTemporaryFileUsage(path, filestats.st_size);
+	else
+	{
+		errno = stat_errno;
+		elog(LOG, "could not stat file \"%s\": %m", path);
+	}

All these messages are "not expected to ever happen" ones, right?

+ return true;
+}
+
/*
* close a file when done with it
*/
@@ -1537,10 +1747,17 @@ FileClose(File file)
Delete(file);
}

+	if (vfdP->fdstate & FD_TEMP_FILE_LIMIT)
+	{
+		/* Subtract its size from current usage (do first in case of error) */
+		temporary_files_size -= vfdP->fileSize;
+		vfdP->fileSize = 0;
+	}

So, is it right to do so unconditionally and without regard for errors?
If the file isn't deleted, it shouldn't be subtracted from fileSize. I
guess you're managing that through the flag, but that's not entirely
obvious.

diff --git a/src/backend/storage/file/sharedfileset.c b/src/backend/storage/file/sharedfileset.c
new file mode 100644
index 00000000000..6da80838b37
--- /dev/null
+++ b/src/backend/storage/file/sharedfileset.c
@@ -0,0 +1,240 @@
+/*-------------------------------------------------------------------------
+ *
+ * sharedfileset.c
+ *	  Shared temporary file management.
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *	  src/backend/storage/file/sharedfileset.c
+ *
+ *-------------------------------------------------------------------------
+ */

A slightly bigger comment wouldn't hurt.

+/*
+ * Attach to a set of directories that was created with SharedFileSetInit.
+ */
+void
+SharedFileSetAttach(SharedFileSet *fileset, dsm_segment *seg)
+{
+	bool		success;
+
+	SpinLockAcquire(&fileset->mutex);
+	if (fileset->refcnt == 0)
+		success = false;

I've not read finished reading through this, but is this safe? If the
segment's gone, is the spinlock guaranteed to still be a spinlock? I
suspect this isn't a problem because just the underlying data is
removed, but the SharedFileSet stays alive?

+static void
+GetSharedFileSetPath(char *path, SharedFileSet *fileset, Oid tablespace)
+{
+	char		tempdirpath[MAXPGPATH];
+	
+	GetTempTablespacePath(tempdirpath, tablespace);
+	snprintf(path, MAXPGPATH, "%s/%s%d.%d.sharedfileset" PG_TEMP_SUBDIR_SUFFIX,
+			 tempdirpath, PG_TEMP_FILE_PREFIX,
+			 fileset->creator_pid, fileset->number);
+}
+/*
+ * Sorting hat to determine which tablespace a given shared temporary file
+ * belongs in.
+ */
+static Oid
+ChooseTablespace(const SharedFileSet *fileset, const char *name)
+{
+	uint32		hash = hash_any((const unsigned char *) name, strlen(name));
+
+	return fileset->tablespaces[hash % fileset->ntablespaces];
+}

Hm. I wonder if just round-robin through these isn't a better approach.

+/*
+ * Compute the full path of a file in a SharedFileSet.
+ */
+static void
+GetSharedFilePath(char *path, SharedFileSet *fileset, const char *name)
+{
+	char		dirpath[MAXPGPATH];
+
+	GetSharedFileSetPath(dirpath, fileset, ChooseTablespace(fileset, name));
+	snprintf(path, MAXPGPATH, "%s/" PG_TEMP_FILE_PREFIX ".%s", dirpath, name);
+}
diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c
index 4c35ccf65eb..8b91d5a6ebe 100644
--- a/src/backend/utils/resowner/resowner.c
+++ b/src/backend/utils/resowner/resowner.c
@@ -528,16 +528,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner,
 				PrintRelCacheLeakWarning(res);
 			RelationClose(res);
 		}
-
-		/* Ditto for dynamic shared memory segments */
-		while (ResourceArrayGetAny(&(owner->dsmarr), &foundres))
-		{
-			dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres);
-
-			if (isCommit)
-				PrintDSMLeakWarning(res);
-			dsm_detach(res);
-		}
 	}
 	else if (phase == RESOURCE_RELEASE_LOCKS)
 	{
@@ -654,6 +644,16 @@ ResourceOwnerReleaseInternal(ResourceOwner owner,
 				PrintFileLeakWarning(res);
 			FileClose(res);
 		}
+
+		/* Ditto for dynamic shared memory segments */
+		while (ResourceArrayGetAny(&(owner->dsmarr), &foundres))
+		{
+			dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres);
+
+			if (isCommit)
+				PrintDSMLeakWarning(res);
+			dsm_detach(res);
+		}
 	}

Is that entirely unproblematic? Are there any DSM callbacks that rely on
locks still being held? Please split this part into a separate commit
with such analysis.

+/* The initial size of chunks in pages. */
+#define STS_MIN_CHUNK_PAGES 4

Could use quick description at how you've arrived at that specific
value.

+/* Chunk written to disk. */
+typedef struct SharedTuplestoreChunk
+{
+	int			npages;			/* Size of this chunk in BLCKSZ pages. */
+	int			ntuples;		/* Number of tuples in this chunk. */
+	char		data[FLEXIBLE_ARRAY_MEMBER];
+} SharedTuplestoreChunk;
+
+/* Per-participant shared state. */
+typedef struct SharedTuplestoreParticipant
+{
+	slock_t		mutex;
+	BlockNumber	read_page;		/* Page number for next read. */
+	BlockNumber	npages;			/* Number of pages written. */
+	bool		writing;		/* Used only for assertions. */
+
+	/*
+	 * We need variable sized chunks, because we might be asked to store
+	 * gigantic tuples.  To avoid the locking contention that would come from
+	 * reading chunk sizes from disk, we store the chunk size for ranges of
+	 * the file in a compact format in memory.  chunk_pages starts out at
+	 * STS_MIN_CHUNK_PAGES and then doubles each time we reach a page listed
+	 * in chunk_expansion_log.
+	 */
+	BlockNumber	chunk_expansion_log[sizeof(BlockNumber) * CHAR_BIT];
+	int			chunk_expansions;
+	int			chunk_expansion;
+	int			chunk_pages;

This needs more explanation.

+/*
+ * Initialize a SharedTuplestore in existing shared memory.  There must be
+ * space for sts_estimate(participants) bytes.  If flags is set to the value
+ * SHARED_TUPLESTORE_SINGLE_PASS, the files may in future be removed more
+ * eagerly (but this isn't yet implemented).

s/iset set to the value/includes the value/ - otherwise it's not really
a flags argument.

+ * Tuples that are stored may optionally carry a piece of fixed sized
+ * meta-data which will be retrieved along with the tuple.  This is useful for
+ * the hash codes used for multi-batch hash joins, but could have other
+ * applications.

"hash codes"?

+/*
+ * Prepare to rescan.  Only participant should call this.  After it returns,
+ * all participants should call sts_begin_parallel_scan() and then loop over
+ * sts_parallel_scan_next().
+ */

s/should/may/? Also maybe document what happens with in-progress reads
(or rather them not being allowed to exist)?

+/*
+ * Write a tuple.  If a meta-data size was provided to sts_initialize, then a
+ * pointer to meta data of that size must be provided.
+ */
+void
+sts_puttuple(SharedTuplestoreAccessor *accessor, void *meta_data,
+			 MinimalTuple tuple)
+{
+	/* Do we have space? */
+	size = accessor->sts->meta_data_size + tuple->t_len;
+	if (accessor->write_pointer + size >= accessor->write_end)
+	{
+		/* Try flushing to see if that creates enough space. */
+		if (accessor->write_chunk != NULL)
+			sts_flush_chunk(accessor);
+
+		/*
+		 * It may still not be enough in the case of a gigantic tuple, or if
+		 * we haven't created a chunk buffer at all yet.
+		 */
+		if (accessor->write_pointer + size >= accessor->write_end)
+		{
+			SharedTuplestoreParticipant *participant;
+			size_t	space_needed;
+			int		pages_needed;
+
+			/* How many pages to hold this data and the chunk header? */
+			space_needed = offsetof(SharedTuplestoreChunk, data) + size;
+			pages_needed = (space_needed + (BLCKSZ - 1)) / BLCKSZ;
+			pages_needed = Max(pages_needed, STS_MIN_CHUNK_PAGES);
+
+			/*
+			 * Double the chunk size until it's big enough, and record that
+			 * fact in the shared expansion log so that readers know about it.
+			 */
+			participant = &accessor->sts->participants[accessor->participant];
+			while (accessor->write_pages < pages_needed)
+			{
+				accessor->write_pages *= 2;
+				participant->chunk_expansion_log[participant->chunk_expansions++] =
+					accessor->write_page;
+			}

Hm. Isn't that going to be pretty unfunny if you have one large and a
lot of small tuples?

+			/* Create the output buffer. */
+			if (accessor->write_chunk != NULL)
+				pfree(accessor->write_chunk);
+			accessor->write_chunk = (SharedTuplestoreChunk *)
+				palloc0(accessor->write_pages * BLCKSZ);

Are we guaranteed to be in a long-lived memory context here?

+/*
+ * Get the next tuple in the current parallel scan.
+ */
+MinimalTuple
+sts_parallel_scan_next(SharedTuplestoreAccessor *accessor, void *meta_data)
+{
+	SharedTuplestoreParticipant *p;
+	BlockNumber	read_page;
+	int			chunk_pages;
+	bool		eof;
+
+	for (;;)
+	{
+		/* Can we read more tuples from the current chunk? */
+		if (likely(accessor->read_ntuples < accessor->read_ntuples_available))
+			return sts_read_tuple(accessor, meta_data);

I'm not convinced this is a good use of likely/unlikely (not biased and
not performance critical enough).

+		/* Find the location of a new chunk to read. */
+		p = &accessor->sts->participants[accessor->read_participant];
+
+		SpinLockAcquire(&p->mutex);
+		eof = p->read_page >= p->npages;
+		if (!eof)
+		{
+			/*
+			 * Figure out how big this chunk is.  It will almost always be the
+			 * same as the last chunk loaded, but if there is one or more
+			 * entry in the chunk expansion log for this page then we know
+			 * that it doubled that number of times.  This avoids the need to
+			 * do IO to adjust the read head, so we don't need to hold up
+			 * concurrent readers.  (An alternative to this extremely rarely
+			 * run loop would be to use more space storing the new size in the
+			 * log so we'd have 'if' instead of 'while'.)
+			 */
+			read_page = p->read_page;
+			while (p->chunk_expansion < p->chunk_expansions &&
+				   p->chunk_expansion_log[p->chunk_expansion] == p->read_page)
+			{
+				p->chunk_pages *= 2;
+				p->chunk_expansion++;
+			}
+			chunk_pages = p->chunk_pages;
+
+			/* The next reader will start after this chunk. */
+			p->read_page += chunk_pages;
+		}
+		SpinLockRelease(&p->mutex);

This looks more like the job of an lwlock rather than a spinlock.

+/*
+ * Create the name used for our shared BufFiles.
+ */
+static void
+make_name(char *name, SharedTuplestoreAccessor *accessor, int participant)
+{
+	snprintf(name, MAXPGPATH, "%s.p%d", accessor->sts->name, participant);
+}

Name's a bit generic. And it's still not really making ;)

Going to buy some groceries and then look at the next patches.

- Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

In reply to: Andres Freund (#30)
Re: Parallel Hash take II

On Tue, Nov 7, 2017 at 1:01 PM, Andres Freund <andres@anarazel.de> wrote:

+/*
+ * Build the name for a given segment of a given BufFile.
+ */
+static void
+MakeSharedSegmentName(char *name, const char *buffile_name, int segment)
+{
+       snprintf(name, MAXPGPATH, "%s.%d", buffile_name, segment);
+}

Not a fan of this name - you're not "making" a filename here (as in
allocating or such). I think I'd just remove the Make prefix.

+1

Can we document the theory behind file naming here, if that isn't in
the latest version? This is a routine private to parallel hash join
(or shared tuplestore), not Buffile. Maybe Buffile should have some
opinion on this, though. Just as a matter of style.

+/*
+ * Delete a BufFile that was created by BufFileCreateShared in the given
+ * SharedFileSet using the given name.
+ *
+ * It is not necessary to delete files explicitly with this function.  It is
+ * provided only as a way to delete files proactively, rather than waiting for
+ * the SharedFileSet to be cleaned up.
+ *
+ * Only one backend should attempt to delete a given name, and should know
+ * that it exists and has been exported or closed.
+ */

This part is new to me. We now want one backend to delete a given
filename. What changed? Please provide a Message-Id reference if that
will help me to understand.

For now, I'm going to guess that this development had something to do
with the need to deal with virtual FDs that do a close() on an FD to
keep under backend limits. Do I have that right?

+       /*
+        * We don't set FD_DELETE_AT_CLOSE for files opened this way, but we still
+        * want to make sure they get closed at end of xact.
+        */
+       ResourceOwnerEnlargeFiles(CurrentResourceOwner);
+       ResourceOwnerRememberFile(CurrentResourceOwner, file);
+       VfdCache[file].resowner = CurrentResourceOwner;

So maybe I'm being pedantic here, but wouldn't the right order be to do
ResourceOwnerEnlargeFiles() *before* creating the file? It's a memory
allocating operation, so it can fail, which'd leak the file.

I remember going to pains to get this right with my own unifiable
BufFile concept. I'm going to wait for an my question about file
deletion + resowners before commenting further, though.

+       if (vfdP->fdstate & FD_TEMP_FILE_LIMIT)
+       {
+               /* Subtract its size from current usage (do first in case of error) */
+               temporary_files_size -= vfdP->fileSize;
+               vfdP->fileSize = 0;
+       }

So, is it right to do so unconditionally and without regard for errors?
If the file isn't deleted, it shouldn't be subtracted from fileSize. I
guess you're managing that through the flag, but that's not entirely
obvious.

I think that the problem here is that the accounting is expected to
always work. It's not like there is a resowner style error path in
which temporary_files_size gets reset to 0.

Is that entirely unproblematic? Are there any DSM callbacks that rely on
locks still being held? Please split this part into a separate commit
with such analysis.

I was always confused about the proper use of DSM callbacks myself.
They seemed generally underdocumented.

+                       /* Create the output buffer. */
+                       if (accessor->write_chunk != NULL)
+                               pfree(accessor->write_chunk);
+                       accessor->write_chunk = (SharedTuplestoreChunk *)
+                               palloc0(accessor->write_pages * BLCKSZ);

Are we guaranteed to be in a long-lived memory context here?

I imagine that Thomas looked to tuplestore_begin_heap() + interXact as
a kind of precedent here. See comments above that function.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#32Robert Haas
robertmhaas@gmail.com
In reply to: Andres Freund (#30)
Re: Parallel Hash take II

On Tue, Nov 7, 2017 at 4:01 PM, Andres Freund <andres@anarazel.de> wrote:

+       ResourceOwnerEnlargeFiles(CurrentResourceOwner);
+       ResourceOwnerRememberFile(CurrentResourceOwner, file);
+       VfdCache[file].resowner = CurrentResourceOwner;

So maybe I'm being pedantic here, but wouldn't the right order be to do
ResourceOwnerEnlargeFiles() *before* creating the file? It's a memory
allocating operation, so it can fail, which'd leak the file.

That's not pedantic ... that's a very sound criticism. IMHO, anyway.

diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c
index 4c35ccf65eb..8b91d5a6ebe 100644
--- a/src/backend/utils/resowner/resowner.c
+++ b/src/backend/utils/resowner/resowner.c
@@ -528,16 +528,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner,
PrintRelCacheLeakWarning(res);
RelationClose(res);
}
-
-               /* Ditto for dynamic shared memory segments */
-               while (ResourceArrayGetAny(&(owner->dsmarr), &foundres))
-               {
-                       dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres);
-
-                       if (isCommit)
-                               PrintDSMLeakWarning(res);
-                       dsm_detach(res);
-               }
}
else if (phase == RESOURCE_RELEASE_LOCKS)
{
@@ -654,6 +644,16 @@ ResourceOwnerReleaseInternal(ResourceOwner owner,
PrintFileLeakWarning(res);
FileClose(res);
}
+
+               /* Ditto for dynamic shared memory segments */
+               while (ResourceArrayGetAny(&(owner->dsmarr), &foundres))
+               {
+                       dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres);
+
+                       if (isCommit)
+                               PrintDSMLeakWarning(res);
+                       dsm_detach(res);
+               }
}

Is that entirely unproblematic? Are there any DSM callbacks that rely on
locks still being held? Please split this part into a separate commit
with such analysis.

FWIW, I think this change is a really good idea (I recommended it to
Thomas at some stage, I think). The current positioning was decided
by me at a very early stage of parallel query development where I
reasoned as follows (1) the first thing we're going to implement is
going to be parallel quicksort, (2) that's going to allocate a huge
amount of DSM, (3) therefore we should try to free it as early as
possible. However, I now thing that was wrongheaded, and not just
because parallel quicksort didn't turn out to be the first thing we
developed. Memory is the very last resource we should release when
aborting a transaction, because any other resource we have is tracked
using data structures that are stored in memory. Throwing the memory
away before the end therefore makes life very difficult. That's why,
for backend-private memory, we clean up most everything else in
AbortTransaction() and then only destroy memory contexts in
CleanupTransaction(). This change doesn't go as far, but it's in the
same general direction, and I think rightly so. My error was in
thinking that the primary use of memory would be for storing data, but
really it's about where you put your control structures.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#33Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Robert Haas (#32)
Re: Parallel Hash take II

On Wed, Nov 8, 2017 at 10:32 AM, Robert Haas <robertmhaas@gmail.com> wrote:

On Tue, Nov 7, 2017 at 4:01 PM, Andres Freund <andres@anarazel.de> wrote:

diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c
index 4c35ccf65eb..8b91d5a6ebe 100644
--- a/src/backend/utils/resowner/resowner.c
+++ b/src/backend/utils/resowner/resowner.c
@@ -528,16 +528,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner,
PrintRelCacheLeakWarning(res);
RelationClose(res);
}
-
-               /* Ditto for dynamic shared memory segments */
-               while (ResourceArrayGetAny(&(owner->dsmarr), &foundres))
-               {
-                       dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres);
-
-                       if (isCommit)
-                               PrintDSMLeakWarning(res);
-                       dsm_detach(res);
-               }
}
else if (phase == RESOURCE_RELEASE_LOCKS)
{
@@ -654,6 +644,16 @@ ResourceOwnerReleaseInternal(ResourceOwner owner,
PrintFileLeakWarning(res);
FileClose(res);
}
+
+               /* Ditto for dynamic shared memory segments */
+               while (ResourceArrayGetAny(&(owner->dsmarr), &foundres))
+               {
+                       dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres);
+
+                       if (isCommit)
+                               PrintDSMLeakWarning(res);
+                       dsm_detach(res);
+               }
}

Is that entirely unproblematic? Are there any DSM callbacks that rely on
locks still being held? Please split this part into a separate commit
with such analysis.

FWIW, I think this change is a really good idea (I recommended it to
Thomas at some stage, I think).

Yeah, it was Robert's suggestion; I thought I needed *something* like
this but was hesitant for the niggling reason that Andres mentions:
what if someone somewhere (including code outside our source tree)
depends on this ordering because of unlocking etc?

At that time I thought that my clean-up logic wasn't going to work on
Windows without this reordering, because we were potentially closing
file handles after unlinking the files, and I was under the impression
that Windows wouldn't like that. Since then I've learned that Windows
does actually allow it, but only if all file handles were opened with
the FILE_SHARE_DELETE flag. We always do that (see src/port/open.c),
so in fact this change is probably not needed for my patch set (theory
not tested). I will put it in a separate patch as requested by
Andres, because it's generally a good idea anyway for the reasons that
Robert explained (ie you probably always want to clean up memory last,
since it might contain the meta-data/locks/control objects/whatever
you'll need to clean up anything else).

--
Thomas Munro
http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#34Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Peter Geoghegan (#31)
Re: Parallel Hash take II

Hi Peter,

See responses to a couple of points below. I'll respond to the other
points separately (ie with code/comment changes).

On Wed, Nov 8, 2017 at 10:32 AM, Peter Geoghegan <pg@bowt.ie> wrote:

On Tue, Nov 7, 2017 at 1:01 PM, Andres Freund <andres@anarazel.de> wrote:

+/*
+ * Delete a BufFile that was created by BufFileCreateShared in the given
+ * SharedFileSet using the given name.
+ *
+ * It is not necessary to delete files explicitly with this function.  It is
+ * provided only as a way to delete files proactively, rather than waiting for
+ * the SharedFileSet to be cleaned up.
+ *
+ * Only one backend should attempt to delete a given name, and should know
+ * that it exists and has been exported or closed.
+ */

This part is new to me. We now want one backend to delete a given
filename. What changed? Please provide a Message-Id reference if that
will help me to understand.

For now, I'm going to guess that this development had something to do
with the need to deal with virtual FDs that do a close() on an FD to
keep under backend limits. Do I have that right?

No -- this is simply an option available to client code that wants to
clean up individual temporary files earlier. Such client code is
responsible for meeting the synchronisation requirements described in
the comment, but it's entirely optional. For example:
SharedTuplestore is backed by multiple files and it could take the
opportunity to delete individual files after they've been scanned, if
you told it you were going to scan only once
(SHARED_TUPLESTORE_SINGLE_PASS) and if it could prove that no other
backend could still need to read from it, as mentioned in a comment in
sts_end_parallel_scan().

However, since I changed to the page/chunk based model (spinlock while
advancing block counter, mimicking Parallel Seq Scan) instead of the
earlier Fisher Price version (LWLock while reading each tuple with a
shared read head maintained with tell/seek), I don't actually do that.
Hitting the end of the file no longer means that no one else is
reading from the file (someone else might still be reading from an
earlier chunk even though you've finished reading the final chunk in
the file, and the vfd systems means that they must be free to close
and reopen the file at any time). In the current patch version the
files are cleaned up wholesale at two times: SharedFileSet cleanup
triggered by DSM destruction, and SharedFileSet reset triggered by
rescan. Practically, it's always the former case. It's vanishingly
rare that you'd actually want to be rescanning a Parallel Hash that
spills to disk but in that case we delete the old files and recreate,
and that case is tested in the regression tests.

If it bothers you that I have an API there that I'm not actually using
yet, I will remove it.

+       if (vfdP->fdstate & FD_TEMP_FILE_LIMIT)
+       {
+               /* Subtract its size from current usage (do first in case of error) */
+               temporary_files_size -= vfdP->fileSize;
+               vfdP->fileSize = 0;
+       }

So, is it right to do so unconditionally and without regard for errors?
If the file isn't deleted, it shouldn't be subtracted from fileSize. I
guess you're managing that through the flag, but that's not entirely
obvious.

I think that the problem here is that the accounting is expected to
always work. It's not like there is a resowner style error path in
which temporary_files_size gets reset to 0.

But there is a resowner error path in which File handles get
automatically closed and temporary_files_size gets adjusted. The
counter goes up when you create, and goes down when you close or
resowner closes for you. Eventually you either close and the
bookkeeping is consistent or you crash and it doesn't matter. And
some kind of freak multiple close attempt is guarded against by
setting the files size to 0 so we can't double-subtract. Do you see a
bug?

None of this has any impact on whether files are leaked: either
SharedFileSet removes the files, or you crash (or take a filesystem
snapshot, etc) and RemovePgTempFiles() mops them up at the next clean
startup.

--
Thomas Munro
http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#35Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#29)
Re: Parallel Hash take II

Hi,

* avoids wasting memory on duplicated hash tables
* avoids wasting disk space on duplicated batch files
* avoids wasting CPU executing duplicate subplans

What's the last one referring to?

+static void
+MultiExecParallelHash(HashState *node)
+{
+	switch (BarrierPhase(build_barrier))
+	{
+		case PHJ_BUILD_ALLOCATING:
+
+			/*
+			 * Either I just allocated the initial hash table in
+			 * ExecHashTableCreate(), or someone else is doing that.  Either
+			 * way, wait for everyone to arrive here so we can proceed, and
+			 * then fall through.
+			 */
+			BarrierArriveAndWait(build_barrier, WAIT_EVENT_HASH_BUILD_ALLOCATING);

Can you add a /* fallthrough */ comment here? Gcc is warning if you
don't. While we currently have lotsa other places not having the
annotation, it seem reasonable to have it in new code.

+		case PHJ_BUILD_HASHING_INNER:
+
+			/*
+			 * It's time to begin hashing, or if we just arrived here then
+			 * hashing is already underway, so join in that effort.  While
+			 * hashing we have to be prepared to help increase the number of
+			 * batches or buckets at any time, and if we arrived here when
+			 * that was already underway we'll have to help complete that work
+			 * immediately so that it's safe to access batches and buckets
+			 * below.
+			 */
+			if (PHJ_GROW_BATCHES_PHASE(BarrierAttach(&pstate->grow_batches_barrier)) !=
+				PHJ_GROW_BATCHES_ELECTING)
+				ExecParallelHashIncreaseNumBatches(hashtable);
+			if (PHJ_GROW_BUCKETS_PHASE(BarrierAttach(&pstate->grow_buckets_barrier)) !=
+				PHJ_GROW_BUCKETS_ELECTING)
+				ExecParallelHashIncreaseNumBuckets(hashtable);
+			ExecParallelHashEnsureBatchAccessors(hashtable);

"accessors" sounds a bit weird for a bunch of pointers, but maybe that's
just my ESL senses tingling wrongly.

/* ----------------------------------------------------------------
@@ -240,12 +427,15 @@ ExecEndHash(HashState *node)
  * ----------------------------------------------------------------
  */
 HashJoinTable
-ExecHashTableCreate(Hash *node, List *hashOperators, bool keepNulls)
+ExecHashTableCreate(HashState *state, List *hashOperators, bool keepNulls)
+	/*
+	 * Parallel Hash tries to use the combined work_mem of all workers to
+	 * avoid the need to batch.  If that won't work, it falls back to work_mem
+	 * per worker and tries to process batches in parallel.
+	 */

One day we're going to need a better approach to this. I have no idea
how, but this per-node, and now per_node * max_parallelism, approach has
only implementation simplicity as its benefit.

+static HashJoinTuple
+ExecParallelHashLoadTuple(HashJoinTable hashtable, MinimalTuple tuple,
+						  dsa_pointer *shared)
+{
+static void
+ExecParallelHashJoinSetUpBatches(HashJoinTable hashtable, int nbatch)
+{
+/*
+ * Get the first tuple in a given bucket identified by number.
+ */
+static HashJoinTuple
+ExecHashFirstTupleInBucket(HashJoinTable hashtable, int bucketno)
+{
+	if (hashtable->parallel_state)
+	{
+		dsa_pointer p =
+		dsa_pointer_atomic_read(&hashtable->buckets.shared[bucketno]);

Can you make this, and possibly a few other places, more readable by
introducing a temporary variable?

+/*
+ * Insert a tuple at the front of a chain of tuples in DSA memory atomically.
+ */
+static void
+ExecParallelHashPushTuple(dsa_pointer_atomic *head,
+						  HashJoinTuple tuple,
+						  dsa_pointer tuple_shared)
+{
+	do
+	{
+		tuple->next.shared = dsa_pointer_atomic_read(head);
+	} while (!dsa_pointer_atomic_compare_exchange(head,
+												  &tuple->next.shared,
+												  tuple_shared));
+}

This is hard to read.

+ * While in the phase PHJ_BUILD_HASHING_INNER a separate pair of barriers may
+ * be used repeatedly as required to coordinate expansions in the number of
+ * batches or buckets.  Their phases are as follows:
+ *
+ *   PHJ_GROW_BATCHES_ELECTING       -- initial state
+ *   PHJ_GROW_BATCHES_ALLOCATING     -- one allocates new batches
+ *   PHJ_GROW_BATCHES_REPARTITIONING -- all rep
s/rep/repartition/?

#include "access/htup_details.h"
+#include "access/parallel.h"
#include "executor/executor.h"
#include "executor/hashjoin.h"
#include "executor/nodeHash.h"
#include "executor/nodeHashjoin.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "utils/memutils.h"
-
+#include "utils/sharedtuplestore.h"

deletes a separator newline.

 /* ----------------------------------------------------------------
@@ -138,6 +236,18 @@ ExecHashJoin(PlanState *pstate)
 					/* no chance to not build the hash table */
 					node->hj_FirstOuterTupleSlot = NULL;
 				}
+				else if (hashNode->parallel_state != NULL)
+				{
+					/*
+					 * The empty-outer optimization is not implemented for
+					 * shared hash tables, because no one participant can
+					 * determine that there are no outer tuples, and it's not
+					 * yet clear that it's worth the synchronization overhead
+					 * of reaching consensus to figure that out.  So we have
+					 * to build the hash table.
+					 */
+					node->hj_FirstOuterTupleSlot = NULL;
+				}

Hm. Isn't MultiExecParallelHash already doing so?

-				node->hj_JoinState = HJ_NEED_NEW_OUTER;
+				if (hashtable->parallel_state)
+				{
+					Barrier    *build_barrier;
+
+					build_barrier = &hashtable->parallel_state->build_barrier;
+					if (BarrierPhase(build_barrier) == PHJ_BUILD_HASHING_OUTER)
+					{
+						/*
+						 * If multi-batch, we need to hash the outer relation
+						 * up front.
+						 */
+						if (hashtable->nbatch > 1)
+							ExecParallelHashJoinPartitionOuter(node);
+						BarrierArriveAndWait(build_barrier,
+											 WAIT_EVENT_HASH_BUILD_HASHING_OUTER);
+					}
+					Assert(BarrierPhase(build_barrier) == PHJ_BUILD_DONE);
+
+					/* Each backend should now select a batch to work on. */
+					hashtable->curbatch = -1;
+					node->hj_JoinState = HJ_NEED_NEW_BATCH;
+
+					continue;
+				}
+				else
+					node->hj_JoinState = HJ_NEED_NEW_OUTER;

You know what I'm going to say about all these branches, and sigh.

If we don't split this into two versions, we at least should store
hashNode->parallel_state in a local var, so the compiler doesn't have to
pull that out of memory after every external function call (of which
there are a lot). In common cases it'll end up in a callee saved
registers, and most of the called functions won't be too register
starved (on x86-64).

+/*
+ * Choose a batch to work on, and attach to it.  Returns true if successful,
+ * false if there are no more batches.
+ */
+static bool
+ExecParallelHashJoinNewBatch(HashJoinState *hjstate)
+{
+					/*
+					 * This batch is ready to probe.  Return control to
+					 * caller. We stay attached to batch_barrier so that the
+					 * hash table stays alive until everyone's finish probing

*finished?

+				case PHJ_BATCH_DONE:
+
+					/*
+					 * Already done.  Detach and go around again (if any
+					 * remain).
+					 */
+					BarrierDetach(batch_barrier);
+
+					/*
+					 * We didn't work on this batch, but we need to observe
+					 * its size for EXPLAIN.
+					 */
+					ExecParallelHashUpdateSpacePeak(hashtable, batchno);
+					hashtable->batches[batchno].done = true;
+					hashtable->curbatch = -1;
+					break;

Hm, maybe I'm missing something, but why is it guaranteed that "we
didn't work on this batch"?

+void
+ExecShutdownHashJoin(HashJoinState *node)
+{
+	/*
+	 * By the time ExecEndHashJoin runs in a worker, shared memory has been
+	 * destroyed.  So this is our last chance to do any shared memory cleanup.
+	 */

This comment doesn't really make much sense to me.

+void
+ExecHashJoinReInitializeDSM(HashJoinState *state, ParallelContext *cxt)
+{

could use a header comment.

a) The executor side is starting to look good.
b) This is a lot of code.
c) I'm tired, planner has to wait till tomorrow.

- Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#36Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#29)
Re: Parallel Hash take II

Hi,

@@ -747,7 +747,7 @@ try_hashjoin_path(PlannerInfo *root,
 	 * never have any output pathkeys, per comments in create_hashjoin_path.
 	 */
 	initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
-						  outer_path, inner_path, extra);
+						  outer_path, inner_path, extra, false);
 	if (add_path_precheck(joinrel,
 						  workspace.startup_cost, workspace.total_cost,
@@ -761,6 +761,7 @@ try_hashjoin_path(PlannerInfo *root,
 									  extra,
 									  outer_path,
 									  inner_path,
+									  false,	/* parallel_hash */
 									  extra->restrictlist,
 									  required_outer,
 									  hashclauses));
@@ -776,6 +777,10 @@ try_hashjoin_path(PlannerInfo *root,
  * try_partial_hashjoin_path
  *	  Consider a partial hashjoin join path; if it appears useful, push it into
  *	  the joinrel's partial_pathlist via add_partial_path().
+ *	  The outer side is partial.  If parallel_hash is true, then the inner path
+ *	  must be partial and will be run in parallel to create one or more shared
+ *	  hash tables; otherwise the inner path must be complete and a copy of it
+ *	  is run in every process to create separate identical private hash tables.
  */

When do we have "or more shared hash tables" rather than one? Are you
thinking about subordinate nodes?

@@ -1839,6 +1846,10 @@ hash_inner_and_outer(PlannerInfo *root,
 		 * able to properly guarantee uniqueness.  Similarly, we can't handle
 		 * JOIN_FULL and JOIN_RIGHT, because they can produce false null
 		 * extended rows.  Also, the resulting path must not be parameterized.
+		 * We should be able to support JOIN_FULL and JOIN_RIGHT for Parallel
+		 * Hash, since in that case we're back to a single hash table with a
+		 * single set of match bits for each batch, but that will require
+		 * figuring out a deadlock-free way to wait for the probe to finish.
 		 */

s/should be able/would be able/?

index 6a45b68e5df..2d38a5efae8 100644
--- a/src/backend/storage/ipc/barrier.c
+++ b/src/backend/storage/ipc/barrier.c
@@ -451,7 +451,6 @@ BarrierDetachImpl(Barrier *barrier, bool arrive)
 		release = true;
 		barrier->arrived = 0;
 		++barrier->phase;
-		Assert(barrier->selected);
 		barrier->selected = false;
 	}

Uh, what?

diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index 35523bd8065..40a076d976f 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -5821,6 +5821,9 @@ analyze extremely_skewed;
 insert into extremely_skewed
   select 42 as id, 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'
   from generate_series(1, 19000);
+-- Make a relation with a couple of enormous tuples.
+create table wide as select generate_series(1, 2) as id, rpad('', 320000, 'x') as t;
+alter table wide set (parallel_workers = 2);

I'm doubtful this is actually going to be a wide tuple - this'll get
compressed down quite a bit, no?

postgres[26465][1]=# SELECT octet_length(t), pg_column_size(t) FROM wide ;
┌──────────────┬────────────────┐
│ octet_length │ pg_column_size │
├──────────────┼────────────────┤
│ 320000 │ 3671 │
│ 320000 │ 3671 │
└──────────────┴────────────────┘
(2 rows)

(and yes, it's ridiculous that a compressed datum of that size still
takes up 3kb)

+-- parallel with parallel-aware hash join
+set max_parallel_workers_per_gather = 2;
+set work_mem = '128kB';
+set enable_parallel_hash = on;

I think it'd be better if we structured the file so we just sat guc's
with SET LOCAL inside a transaction.

+-- parallel with parallel-aware hash join
+set max_parallel_workers_per_gather = 2;
+set work_mem = '64kB';
+set enable_parallel_hash = on;
+explain (costs off)
+  select count(*) from simple r join extremely_skewed s using (id);
+                              QUERY PLAN                               
+-----------------------------------------------------------------------
+ Finalize Aggregate
+   ->  Gather
+         Workers Planned: 2
+         ->  Partial Aggregate
+               ->  Parallel Hash Join
+                     Hash Cond: (r.id = s.id)
+                     ->  Parallel Seq Scan on simple r
+                     ->  Parallel Hash
+                           ->  Parallel Seq Scan on extremely_skewed s
+(9 rows)
+
+select count(*) from simple r join extremely_skewed s using (id);
+ count 
+-------
+ 20000
+(1 row)

As written before, I think it'd be good if we extracted the number of
batches from json output to be sure things are going sensibly.

All-in-all this part looks fairly boring.

- Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#37Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#36)
1 attachment(s)
Re: Parallel Hash take II

Hi Andres and Peter,

Please see below for inline responses to your feedback. New patch attached.

On Wed, Nov 8, 2017 at 10:01 AM, Andres Freund <andres@anarazel.de> wrote:

+set min_parallel_table_scan_size = 0;
+set parallel_setup_cost = 0;
+-- Make a simple relation with well distributed keys and correctly
+-- estimated size.
+create table simple as
+  select generate_series(1, 20000) AS id, 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa';
+alter table simple set (parallel_workers = 2);
+analyze simple;
+-- Make a relation whose size we will under-estimate.  We want stats
+-- to say 1000 rows, but actually there are 20,000 rows.
+create table bigger_than_it_looks as
+  select generate_series(1, 20000) as id, 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa';
+alter table bigger_than_it_looks set (autovacuum_enabled = 'false');
+alter table bigger_than_it_looks set (parallel_workers = 2);
+delete from bigger_than_it_looks where id <= 19000;
+vacuum bigger_than_it_looks;
+analyze bigger_than_it_looks;
+insert into bigger_than_it_looks
+  select generate_series(1, 19000) as id, 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa';

It seems kinda easier to just manipulate ndistinct and reltuples...

Done.

+set max_parallel_workers_per_gather = 0;
+set work_mem = '4MB';

I hope there's a fair amount of slop here - with different archs you're
going to see quite some size differences.

Yeah, this is a problem I wrestled with. See next.

+-- The "good" case: batches required, but we plan the right number; we
+-- plan for 16 batches, and we stick to that number, and peak memory
+-- usage says within our work_mem budget
+-- non-parallel
+set max_parallel_workers_per_gather = 0;
+set work_mem = '128kB';

So how do we know that's actually the case we're testing rather than
something arbitrarily different? There's IIRC tests somewhere that just
filter the json explain output to the right parts...

Yeah, good idea. My earlier attempts to dump out the hash join
dimensions ran into problems with architecture sensitivity and then
some run-to-run non-determinism in the parallel case (due to varying
fragmentation depending on how many workers get involved in time).
The attached version tells you about batch growth without reporting
the exact numbers, except in the "ugly" case where we know that there
is only one possible outcome because the extreme skew detector is
guaranteed to go off after the first nbatch increase (I got rid of all
other tuples except ones with the same key to make this true).

This exercise did reveal a bug in
0008-Show-hash-join-per-worker-information-in-EXPLAIN-ANA.patch
though: it is capturing shared instrumentation too soon in the
non-Parallel Hash case so the nbatch reported by EXPLAIN ANALYZE might
be too low if we grew while probing. Oops. Will post a fix for that.

+/*
+ * Build the name for a given segment of a given BufFile.
+ */
+static void
+MakeSharedSegmentName(char *name, const char *buffile_name, int segment)
+{
+       snprintf(name, MAXPGPATH, "%s.%d", buffile_name, segment);
+}

Not a fan of this name - you're not "making" a filename here (as in
allocating or such). I think I'd just remove the Make prefix.

Done. I also changed some similar code where I'd used GetXXX when
building paths.

+/*
+ * Open a file that was previously created in another backend with
+ * BufFileCreateShared in the same SharedFileSet using the same name.  The
+ * backend that created the file must have called BufFileClose() or
+ * BufFileExport() to make sure that it is ready to be opened by other
+ * backends and render it read-only.
+ */

Is it actually guaranteed that it's another backend / do we rely on
that?

No, it could be any backend that is attached to the SharedFileSet,
including the current one. Wording improved.

+BufFile *
+BufFileOpenShared(SharedFileSet *fileset, const char *name)
+{
+       /*
+        * If we didn't find any files at all, then no BufFile exists with this
+        * tag.
+        */
+       if (nfiles == 0)
+               return NULL;

s/taag/name/?

Fixed.

+/*
+ * Delete a BufFile that was created by BufFileCreateShared in the given
+ * SharedFileSet using the given name.
+ *
+ * It is not necessary to delete files explicitly with this function.  It is
+ * provided only as a way to delete files proactively, rather than waiting for
+ * the SharedFileSet to be cleaned up.
+ *
+ * Only one backend should attempt to delete a given name, and should know
+ * that it exists and has been exported or closed.
+ */
+void
+BufFileDeleteShared(SharedFileSet *fileset, const char *name)
+{
+       char            segment_name[MAXPGPATH];
+       int                     segment = 0;
+       bool            found = false;
+
+       /*
+        * We don't know how many segments the file has.  We'll keep deleting
+        * until we run out.  If we don't manage to find even an initial segment,
+        * raise an error.
+        */
+       for (;;)
+       {
+               MakeSharedSegmentName(segment_name, name, segment);
+               if (!SharedFileSetDelete(fileset, segment_name, true))
+                       break;
+               found = true;
+               ++segment;
+       }

Hm. Do we properly delete all the files via the resowner mechanism if
this fails midway? I.e. if there are no leading segments? Also wonder if
this doesn't need a CFI check.

The resowner mechanism recursively deletes everything, so order
doesn't matter here. CFI added.

+void
+PathNameCreateTemporaryDir(const char *basedir, const char *directory)
+{
+       if (mkdir(directory, S_IRWXU) < 0)
+       {
+               if (errno == EEXIST)
+                       return;
+
+               /*
+                * Failed.  Try to create basedir first in case it's missing. Tolerate
+                * ENOENT to close a race against another process following the same
+                * algorithm.
+                */
+               if (mkdir(basedir, S_IRWXU) < 0 && errno != ENOENT)
+                       elog(ERROR, "cannot create temporary directory \"%s\": %m",
+                                basedir);

ENOENT or EEXIST?

Oops, right, fixed.

+File
+PathNameCreateTemporaryFile(const char *path, bool error_on_failure)
+{
+       File            file;
+
+       /*
+        * Open the file.  Note: we don't use O_EXCL, in case there is an orphaned
+        * temp file that can be reused.
+        */
+       file = PathNameOpenFile(path, O_RDWR | O_CREAT | O_TRUNC | PG_BINARY);
+       if (file <= 0)
+       {
+               if (error_on_failure)
+                       elog(ERROR, "could not create temporary file \"%s\": %m", path);
+               else
+                       return file;
+       }
+
+       /* Mark it for temp_file_limit accounting. */
+       VfdCache[file].fdstate |= FD_TEMP_FILE_LIMIT;
+
+       /*
+        * We don't set FD_DELETE_AT_CLOSE for files opened this way, but we still
+        * want to make sure they get closed at end of xact.
+        */
+       ResourceOwnerEnlargeFiles(CurrentResourceOwner);
+       ResourceOwnerRememberFile(CurrentResourceOwner, file);
+       VfdCache[file].resowner = CurrentResourceOwner;

So maybe I'm being pedantic here, but wouldn't the right order be to do
ResourceOwnerEnlargeFiles() *before* creating the file? It's a memory
allocating operation, so it can fail, which'd leak the file.

Fixed. See also commit c5269472.

+/*
+ * Open a file that was created with PathNameCreateTemporaryFile, possibly in
+ * another backend.  Files opened this way don't count agains the

s/agains/against/

Fixed.

+ * temp_file_limit of the caller, are read-only and are automatically closed
+ * at the end of the transaction but are not deleted on close.
+ */
+File
+PathNameOpenTemporaryFile(const char *path)
+{
+       File            file;
+
+       /* We open the file read-only. */
+       file = PathNameOpenFile(path, O_RDONLY | PG_BINARY);
+
+       /* If no such file, then we don't raise an error. */
+       if (file <= 0 && errno != ENOENT)
+               elog(ERROR, "could not open temporary file \"%s\": %m", path);
+
+       if (file > 0)
+       {
+               /*
+                * We don't set FD_DELETE_AT_CLOSE for files opened this way, but we
+                * still want to make sure they get closed at end of xact.
+                */
+               ResourceOwnerEnlargeFiles(CurrentResourceOwner);
+               ResourceOwnerRememberFile(CurrentResourceOwner, file);
+               VfdCache[file].resowner = CurrentResourceOwner;

Same complaint as above, ResourceOwnerEnlargeFiles() should be done
earlier.

Fixed.

+/*
+ * Delete a file by pathname.  Return true if the file existed, false if
+ * didn't.
+ */
+bool
+PathNameDeleteTemporaryFile(const char *path, bool error_on_failure)
+{
+       struct stat filestats;
+       int                     stat_errno;
+
+       /* Get the final size for pgstat reporting. */
+       if (stat(path, &filestats) != 0)
+               stat_errno = errno;
+       else
+               stat_errno = 0;
+
+       /*
+        * Unlike FileClose's automatic file deletion code, we tolerate
+        * non-existence to support BufFileDeleteShared which doesn't know how
+        * many segments it has to delete until it runs out.
+        */
+       if (stat_errno == ENOENT)
+               return false;
+
+       if (unlink(path) < 0)
+       {
+               if (errno != ENOENT)
+                       elog(error_on_failure ? ERROR : LOG,
+                                "cannot unlink temporary file \"%s\": %m", path);
+               return false;
+       }
+
+       if (stat_errno == 0)
+               ReportTemporaryFileUsage(path, filestats.st_size);
+       else
+       {
+               errno = stat_errno;
+               elog(LOG, "could not stat file \"%s\": %m", path);
+       }

All these messages are "not expected to ever happen" ones, right?

You'd have to suffer a nasty filesystem failure, remount read-only or
manually with permissions or something. Not sure where the line is,
but I've changed all of these new elog calls to ereport.

+ return true;
+}
+
/*
* close a file when done with it
*/
@@ -1537,10 +1747,17 @@ FileClose(File file)
Delete(file);
}

+       if (vfdP->fdstate & FD_TEMP_FILE_LIMIT)
+       {
+               /* Subtract its size from current usage (do first in case of error) */
+               temporary_files_size -= vfdP->fileSize;
+               vfdP->fileSize = 0;
+       }

So, is it right to do so unconditionally and without regard for errors?
If the file isn't deleted, it shouldn't be subtracted from fileSize. I
guess you're managing that through the flag, but that's not entirely
obvious.

I think it is. Reasoning: The existing behaviour of fd.c is that if
we don't manage to delete temporary files, we'll LOG something and
forget about them (they'll be cleaned up eventually by a clean restart
or human intervention). If you have a filesystem that lets you create
and write files but not unlink them then Postgres will eventually eat
all your disk. The alternative would be not to adjust
temporary_files_size so that we prevent this backend from creating
more temporary files: that might make some kind of sense but it
wouldn't solve any real operational problems: you could still eat all
the disk space by disconnecting and reconnecting to get a new session
with a new temp filespace allowance. If we want true space limit
we'll need to design that, but that seems out of scope for this
discussion.

diff --git a/src/backend/storage/file/sharedfileset.c b/src/backend/storage/file/sharedfileset.c
new file mode 100644
index 00000000000..6da80838b37
--- /dev/null
+++ b/src/backend/storage/file/sharedfileset.c
@@ -0,0 +1,240 @@
+/*-------------------------------------------------------------------------
+ *
+ * sharedfileset.c
+ *       Shared temporary file management.
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ *       src/backend/storage/file/sharedfileset.c
+ *
+ *-------------------------------------------------------------------------
+ */

A slightly bigger comment wouldn't hurt.

Done.

+/*
+ * Attach to a set of directories that was created with SharedFileSetInit.
+ */
+void
+SharedFileSetAttach(SharedFileSet *fileset, dsm_segment *seg)
+{
+       bool            success;
+
+       SpinLockAcquire(&fileset->mutex);
+       if (fileset->refcnt == 0)
+               success = false;

I've not read finished reading through this, but is this safe? If the
segment's gone, is the spinlock guaranteed to still be a spinlock? I
suspect this isn't a problem because just the underlying data is
removed, but the SharedFileSet stays alive?

If you have a dsm_segment object, you are attached to it and it exists
(ie is mapped in your backend). The spinlock is guaranteed to be a
spinlock. The race being guarded against here is: a worker starts up
and attaches to the DSM segment and then tries to attach to the
SharedFileSet but finds that the other backends have all detached and
the refcount is 0. That's actually quite unlikely, because they
detach from the SharedFileSet as part of the process of detaching from
the DSM segment, so there is a pretty narrow window of time in which
SharedFileSet's refcount is 0 but the DSM segment still exists (they
haven't detached from that yet) and you managed to attach to the DSM
segment.

This is a variant of the problem SQL Smith found and I fixed in commit
fddf45b38097d14301d249fbeebca32e40233bd2. I think in general anything
that has its own reference count or other kind of state indicating "I
have been destroyed, you can't use me" and lives in a DSM segment
needs protection against this race. In general, I think parallel
workers that start up late (say in an error case where the leader gave
up or something) are bound to show racy error messages ranging from
"can't attach to this invalid DSM handle" to "I attached to the DSM
segment, but I can't seem to attach to <thing> inside it", which are
all just versions of "hey, where'd everyone go?" with different
timing.

+/*
+ * Sorting hat to determine which tablespace a given shared temporary file
+ * belongs in.
+ */
+static Oid
+ChooseTablespace(const SharedFileSet *fileset, const char *name)
+{
+       uint32          hash = hash_any((const unsigned char *) name, strlen(name));
+
+       return fileset->tablespaces[hash % fileset->ntablespaces];
+}

Hm. I wonder if just round-robin through these isn't a better approach.

The problem is that all backends opening that file by name need to
agree on which tablespace it lives in, and I don't have per-file shmem
space to remember them. In an earlier version I required all callers
to tell me which "stripe number" the file lived in and to promise that
they would use the same stripe number for the same filename every time
in every backend, so then maybe the calling code could somehow figure
out how to round-robin the stripe numbers... but that seemed at best
clumsy. This way seemed much tidier, and has only one downside I
could think of: when you have only 2 files in the set there is a 50%
chance that they both finish up in the same tablespace. With any more
than that it should quickly approach even load balancing. It didn't
seem worth trying very hard to solve that problem, since people don't
really use temp_tablespaces for serious load balancing anyway AFAIK.

+/*
+ * Compute the full path of a file in a SharedFileSet.
+ */
+static void
+GetSharedFilePath(char *path, SharedFileSet *fileset, const char *name)
+{
+       char            dirpath[MAXPGPATH];
+
+       GetSharedFileSetPath(dirpath, fileset, ChooseTablespace(fileset, name));
+       snprintf(path, MAXPGPATH, "%s/" PG_TEMP_FILE_PREFIX ".%s", dirpath, name);
+}
diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c
index 4c35ccf65eb..8b91d5a6ebe 100644
--- a/src/backend/utils/resowner/resowner.c
+++ b/src/backend/utils/resowner/resowner.c
@@ -528,16 +528,6 @@ ResourceOwnerReleaseInternal(ResourceOwner owner,
PrintRelCacheLeakWarning(res);
RelationClose(res);
}
-
-               /* Ditto for dynamic shared memory segments */
-               while (ResourceArrayGetAny(&(owner->dsmarr), &foundres))
-               {
-                       dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres);
-
-                       if (isCommit)
-                               PrintDSMLeakWarning(res);
-                       dsm_detach(res);
-               }
}
else if (phase == RESOURCE_RELEASE_LOCKS)
{
@@ -654,6 +644,16 @@ ResourceOwnerReleaseInternal(ResourceOwner owner,
PrintFileLeakWarning(res);
FileClose(res);
}
+
+               /* Ditto for dynamic shared memory segments */
+               while (ResourceArrayGetAny(&(owner->dsmarr), &foundres))
+               {
+                       dsm_segment *res = (dsm_segment *) DatumGetPointer(foundres);
+
+                       if (isCommit)
+                               PrintDSMLeakWarning(res);
+                       dsm_detach(res);
+               }
}

Is that entirely unproblematic? Are there any DSM callbacks that rely on
locks still being held? Please split this part into a separate commit
with such analysis.

I've removed this change. As far as I know I was wrong about Windows
needing this change for my patch set (due to FILE_SHARED_DELETE).

+/* The initial size of chunks in pages. */
+#define STS_MIN_CHUNK_PAGES 4

Could use quick description at how you've arrived at that specific
value.

Done. The new comment is:

/*
* The initial size of chunks, in pages. This is somewhat arbitrarily set to
* match the size of HASH_CHUNK, so that Parallel Hash obtains new chunks of
* tuples at approximately the same rate as it allocates new chunks of memory
* to insert them into.
*/

+/* Chunk written to disk. */
+typedef struct SharedTuplestoreChunk
+{
+       int                     npages;                 /* Size of this chunk in BLCKSZ pages. */
+       int                     ntuples;                /* Number of tuples in this chunk. */
+       char            data[FLEXIBLE_ARRAY_MEMBER];
+} SharedTuplestoreChunk;
+
+/* Per-participant shared state. */
+typedef struct SharedTuplestoreParticipant
+{
+       slock_t         mutex;
+       BlockNumber     read_page;              /* Page number for next read. */
+       BlockNumber     npages;                 /* Number of pages written. */
+       bool            writing;                /* Used only for assertions. */
+
+       /*
+        * We need variable sized chunks, because we might be asked to store
+        * gigantic tuples.  To avoid the locking contention that would come from
+        * reading chunk sizes from disk, we store the chunk size for ranges of
+        * the file in a compact format in memory.  chunk_pages starts out at
+        * STS_MIN_CHUNK_PAGES and then doubles each time we reach a page listed
+        * in chunk_expansion_log.
+        */
+       BlockNumber     chunk_expansion_log[sizeof(BlockNumber) * CHAR_BIT];
+       int                     chunk_expansions;
+       int                     chunk_expansion;
+       int                     chunk_pages;

This needs more explanation.

I hope this is now explained a bit better -- see also a few points further down.

+/*
+ * Initialize a SharedTuplestore in existing shared memory.  There must be
+ * space for sts_estimate(participants) bytes.  If flags is set to the value
+ * SHARED_TUPLESTORE_SINGLE_PASS, the files may in future be removed more
+ * eagerly (but this isn't yet implemented).

s/iset set to the value/includes the value/ - otherwise it's not really
a flags argument.

Fixed.

+ * Tuples that are stored may optionally carry a piece of fixed sized
+ * meta-data which will be retrieved along with the tuple.  This is useful for
+ * the hash codes used for multi-batch hash joins, but could have other
+ * applications.

"hash codes"?

Fixed.

+/*
+ * Prepare to rescan.  Only participant should call this.  After it returns,
+ * all participants should call sts_begin_parallel_scan() and then loop over
+ * sts_parallel_scan_next().
+ */

s/should/may/? Also maybe document what happens with in-progress reads
(or rather them not being allowed to exist)?

Done. Note that this interface is not currently used, but it could
obviously be used for rescans. In an earlier version this needed to
be called even for the first scan, but I got rid of that requirement.

+/*
+ * Write a tuple.  If a meta-data size was provided to sts_initialize, then a
+ * pointer to meta data of that size must be provided.
+ */
+void
+sts_puttuple(SharedTuplestoreAccessor *accessor, void *meta_data,
+                        MinimalTuple tuple)
+{
+       /* Do we have space? */
+       size = accessor->sts->meta_data_size + tuple->t_len;
+       if (accessor->write_pointer + size >= accessor->write_end)
+       {
+               /* Try flushing to see if that creates enough space. */
+               if (accessor->write_chunk != NULL)
+                       sts_flush_chunk(accessor);
+
+               /*
+                * It may still not be enough in the case of a gigantic tuple, or if
+                * we haven't created a chunk buffer at all yet.
+                */
+               if (accessor->write_pointer + size >= accessor->write_end)
+               {
+                       SharedTuplestoreParticipant *participant;
+                       size_t  space_needed;
+                       int             pages_needed;
+
+                       /* How many pages to hold this data and the chunk header? */
+                       space_needed = offsetof(SharedTuplestoreChunk, data) + size;
+                       pages_needed = (space_needed + (BLCKSZ - 1)) / BLCKSZ;
+                       pages_needed = Max(pages_needed, STS_MIN_CHUNK_PAGES);
+
+                       /*
+                        * Double the chunk size until it's big enough, and record that
+                        * fact in the shared expansion log so that readers know about it.
+                        */
+                       participant = &accessor->sts->participants[accessor->participant];
+                       while (accessor->write_pages < pages_needed)
+                       {
+                               accessor->write_pages *= 2;
+                               participant->chunk_expansion_log[participant->chunk_expansions++] =
+                                       accessor->write_page;
+                       }

Hm. Isn't that going to be pretty unfunny if you have one large and a
lot of small tuples?

It will increase the parallel scan grain size, and then keep that size
for the rest of the contents of one backend's output file. I am aware
of two downsides to using a large parallel grain:

1. It determines the amount of unfairness when we run out of data:
it's the maximum amount of extra data that the unlucky last worker can
finish up with after all the others have finished. I think this
effect is reduced by higher level factors: when a reader runs out of
data in one backend's file, it'll start reading another backend's
file. If it's hit the end of all backends' files and this is an outer
batch, Parallel Hash will just go and work on another batch
immediately.

2. It affects the read-ahead heuristics. On Linux, if the parallel
scan grain size is larger than the read-ahead window and some other
backend advances the block counter between your reads then the scan
looks like random IO. Suppose you have the default read-ahead window
size of 512kB. You need to hit a tuple over 256kB in size *and* be
unlucky enough to have multiple backends reading the same file
concurrently (which I try to avoid as discussed elsewhere) to befuddle
the read-ahead heuristics. If this is ever a problem we could
consider explicit read-ahead advice.

A couple of alternatives seemed like bad ideas: we could read the
chunk size from the chunk headers, but then each read would be
dependent on (ie have to wait for) the preceding read, like Parallel
(btree) Index Scan and unlike Parallel Seq Scan; or we could track
changes in chunk size in shared memory in a more sophisticated way
that allows decreasing, but that would no longer be small, simple and
fixed size as I have it.

Better ideas?

BTW this code is covered by the regression test.

+                       /* Create the output buffer. */
+                       if (accessor->write_chunk != NULL)
+                               pfree(accessor->write_chunk);
+                       accessor->write_chunk = (SharedTuplestoreChunk *)
+                               palloc0(accessor->write_pages * BLCKSZ);

Are we guaranteed to be in a long-lived memory context here?

I changed it so that it captures CurrentMemoryContext when you
intialise or attach and uses that for allocating buffers.

+/*
+ * Get the next tuple in the current parallel scan.
+ */
+MinimalTuple
+sts_parallel_scan_next(SharedTuplestoreAccessor *accessor, void *meta_data)
+{
+       SharedTuplestoreParticipant *p;
+       BlockNumber     read_page;
+       int                     chunk_pages;
+       bool            eof;
+
+       for (;;)
+       {
+               /* Can we read more tuples from the current chunk? */
+               if (likely(accessor->read_ntuples < accessor->read_ntuples_available))
+                       return sts_read_tuple(accessor, meta_data);

I'm not convinced this is a good use of likely/unlikely (not biased and
not performance critical enough).

Removed.

+               /* Find the location of a new chunk to read. */
+               p = &accessor->sts->participants[accessor->read_participant];
+
+               SpinLockAcquire(&p->mutex);
+               eof = p->read_page >= p->npages;
+               if (!eof)
+               {
+                       /*
+                        * Figure out how big this chunk is.  It will almost always be the
+                        * same as the last chunk loaded, but if there is one or more
+                        * entry in the chunk expansion log for this page then we know
+                        * that it doubled that number of times.  This avoids the need to
+                        * do IO to adjust the read head, so we don't need to hold up
+                        * concurrent readers.  (An alternative to this extremely rarely
+                        * run loop would be to use more space storing the new size in the
+                        * log so we'd have 'if' instead of 'while'.)
+                        */
+                       read_page = p->read_page;
+                       while (p->chunk_expansion < p->chunk_expansions &&
+                                  p->chunk_expansion_log[p->chunk_expansion] == p->read_page)
+                       {
+                               p->chunk_pages *= 2;
+                               p->chunk_expansion++;
+                       }
+                       chunk_pages = p->chunk_pages;
+
+                       /* The next reader will start after this chunk. */
+                       p->read_page += chunk_pages;
+               }
+               SpinLockRelease(&p->mutex);

This looks more like the job of an lwlock rather than a spinlock.

I switched to the alternative algorithm mentioned in parentheses in
the comment. It uses a bit more space, but that loop is gone. In my
mind this is much like Parallel Seq Scan: we acquire a spinlock just
to advance the block pointer. The added complication is that we also
check if the chunk size has changed, which clang renders as this many
instructions:

postgres[0x10047eee0] <+176>: movslq 0x144(%r15,%rbx), %rcx
postgres[0x10047eee8] <+184>: cmpl 0x140(%r15,%rbx), %ecx
postgres[0x10047eef0] <+192>: jge 0x10047ef16 ;
<+230> at sharedtuplestore.c:489
postgres[0x10047eef2] <+194>: leaq (%r15,%rbx), %rdx
postgres[0x10047eef6] <+198>: cmpl %r12d, 0x40(%rdx,%rcx,8)
postgres[0x10047eefb] <+203>: jne 0x10047ef16 ;
<+230> at sharedtuplestore.c:489
postgres[0x10047eefd] <+205>: leaq 0x144(%r15,%rbx), %rsi
postgres[0x10047ef05] <+213>: leal 0x1(%rcx), %edi
postgres[0x10047ef08] <+216>: movl %edi, (%rsi)
postgres[0x10047ef0a] <+218>: movl 0x44(%rdx,%rcx,8), %ecx
postgres[0x10047ef0e] <+222>: movl %ecx, 0x148(%r15,%rbx)
postgres[0x10047ef16] <+230>: movl 0x148(%r15,%rbx), %r15d

That should be OK, right?

+/*
+ * Create the name used for our shared BufFiles.
+ */
+static void
+make_name(char *name, SharedTuplestoreAccessor *accessor, int participant)
+{
+       snprintf(name, MAXPGPATH, "%s.p%d", accessor->sts->name, participant);
+}

Name's a bit generic. And it's still not really making ;)

Renamed to sts_filename.

On Wed, Nov 8, 2017 at 10:32 AM, Peter Geoghegan <pg@bowt.ie> wrote:

On Tue, Nov 7, 2017 at 1:01 PM, Andres Freund <andres@anarazel.de> wrote:

+/*
+ * Build the name for a given segment of a given BufFile.
+ */
+static void
+MakeSharedSegmentName(char *name, const char *buffile_name, int segment)
+{
+       snprintf(name, MAXPGPATH, "%s.%d", buffile_name, segment);
+}

Not a fan of this name - you're not "making" a filename here (as in
allocating or such). I think I'd just remove the Make prefix.

+1

Can we document the theory behind file naming here, if that isn't in
the latest version? This is a routine private to parallel hash join
(or shared tuplestore), not Buffile. Maybe Buffile should have some
opinion on this, though. Just as a matter of style.

I have added some text about naming to BufFileCreateShared().

+                       /* Create the output buffer. */
+                       if (accessor->write_chunk != NULL)
+                               pfree(accessor->write_chunk);
+                       accessor->write_chunk = (SharedTuplestoreChunk *)
+                               palloc0(accessor->write_pages * BLCKSZ);

Are we guaranteed to be in a long-lived memory context here?

I imagine that Thomas looked to tuplestore_begin_heap() + interXact as
a kind of precedent here. See comments above that function.

Yeah, but I had missed something important: I now capture the current
memory context when you initialise or attach to a SharedTuplestore.
That's the point at which the SharedTuplestoreAccessor is allocated,
and the caller needs to make sure that the right memory context is
active, but now that I hold onto it I can use the same one for future
buffer allocations. Like the code highlighted above. In nodeHash.c I
now make sure that always happens hashtable->hashCxt.

On Wed, Nov 8, 2017 at 4:40 PM, Andres Freund <andres@anarazel.de> wrote:

Hi,

* avoids wasting memory on duplicated hash tables
* avoids wasting disk space on duplicated batch files
* avoids wasting CPU executing duplicate subplans

What's the last one referring to?

Erm, I was saying the same thing two different ways (with the
following line). Fixed.

+static void
+MultiExecParallelHash(HashState *node)
+{
+       switch (BarrierPhase(build_barrier))
+       {
+               case PHJ_BUILD_ALLOCATING:
+
+                       /*
+                        * Either I just allocated the initial hash table in
+                        * ExecHashTableCreate(), or someone else is doing that.  Either
+                        * way, wait for everyone to arrive here so we can proceed, and
+                        * then fall through.
+                        */
+                       BarrierArriveAndWait(build_barrier, WAIT_EVENT_HASH_BUILD_ALLOCATING);

Can you add a /* fallthrough */ comment here? Gcc is warning if you
don't. While we currently have lotsa other places not having the
annotation, it seem reasonable to have it in new code.

Fixed all warnings from GCC 7.2 -Wimplicit-fallthrough=3.

+               case PHJ_BUILD_HASHING_INNER:
+
+                       /*
+                        * It's time to begin hashing, or if we just arrived here then
+                        * hashing is already underway, so join in that effort.  While
+                        * hashing we have to be prepared to help increase the number of
+                        * batches or buckets at any time, and if we arrived here when
+                        * that was already underway we'll have to help complete that work
+                        * immediately so that it's safe to access batches and buckets
+                        * below.
+                        */
+                       if (PHJ_GROW_BATCHES_PHASE(BarrierAttach(&pstate->grow_batches_barrier)) !=
+                               PHJ_GROW_BATCHES_ELECTING)
+                               ExecParallelHashIncreaseNumBatches(hashtable);
+                       if (PHJ_GROW_BUCKETS_PHASE(BarrierAttach(&pstate->grow_buckets_barrier)) !=
+                               PHJ_GROW_BUCKETS_ELECTING)
+                               ExecParallelHashIncreaseNumBuckets(hashtable);
+                       ExecParallelHashEnsureBatchAccessors(hashtable);

"accessors" sounds a bit weird for a bunch of pointers, but maybe that's
just my ESL senses tingling wrongly.

Pointers to ParallelHashJoinBatchAccessor objects. That's where we
keep this backend's pointers into ParallelHashJoinBatch objects and
some other backend-local state. It's a pattern: you have some shared
object of type Foo, and every backend interacting with it will also
need an object of type FooAccessor if a simple pointer to the shared
object isn't enough. If you have a better name than 'Accessor' for
this then I'm all ears.

/* ----------------------------------------------------------------
@@ -240,12 +427,15 @@ ExecEndHash(HashState *node)
* ----------------------------------------------------------------
*/
HashJoinTable
-ExecHashTableCreate(Hash *node, List *hashOperators, bool keepNulls)
+ExecHashTableCreate(HashState *state, List *hashOperators, bool keepNulls)
+       /*
+        * Parallel Hash tries to use the combined work_mem of all workers to
+        * avoid the need to batch.  If that won't work, it falls back to work_mem
+        * per worker and tries to process batches in parallel.
+        */

One day we're going to need a better approach to this. I have no idea
how, but this per-node, and now per_node * max_parallelism, approach has
only implementation simplicity as its benefit.

I agree, and I am interested in that subject. In the meantime, I
think it'd be pretty unfair if parallel-oblivious hash join and
sort-merge join and every other parallel plan get to use work_mem * p
(and in some cases waste it with duplicate data), but Parallel Hash
isn't allowed to do the same (and put it to good use).

+static HashJoinTuple
+ExecParallelHashLoadTuple(HashJoinTable hashtable, MinimalTuple tuple,
+                                                 dsa_pointer *shared)
+{
+static void
+ExecParallelHashJoinSetUpBatches(HashJoinTable hashtable, int nbatch)
+{
+/*
+ * Get the first tuple in a given bucket identified by number.
+ */
+static HashJoinTuple
+ExecHashFirstTupleInBucket(HashJoinTable hashtable, int bucketno)
+{
+       if (hashtable->parallel_state)
+       {
+               dsa_pointer p =
+               dsa_pointer_atomic_read(&hashtable->buckets.shared[bucketno]);

Can you make this, and possibly a few other places, more readable by
introducing a temporary variable?

Done.

+/*
+ * Insert a tuple at the front of a chain of tuples in DSA memory atomically.
+ */
+static void
+ExecParallelHashPushTuple(dsa_pointer_atomic *head,
+                                                 HashJoinTuple tuple,
+                                                 dsa_pointer tuple_shared)
+{
+       do
+       {
+               tuple->next.shared = dsa_pointer_atomic_read(head);
+       } while (!dsa_pointer_atomic_compare_exchange(head,
+                                                                                                 &tuple->next.shared,
+                                                                                                 tuple_shared));
+}

This is hard to read.

Change to a for loop with a break.

+ * While in the phase PHJ_BUILD_HASHING_INNER a separate pair of barriers may
+ * be used repeatedly as required to coordinate expansions in the number of
+ * batches or buckets.  Their phases are as follows:
+ *
+ *   PHJ_GROW_BATCHES_ELECTING       -- initial state
+ *   PHJ_GROW_BATCHES_ALLOCATING     -- one allocates new batches
+ *   PHJ_GROW_BATCHES_REPARTITIONING -- all rep
s/rep/repartition/?

Fixed.

-
+#include "utils/sharedtuplestore.h"

deletes a separator newline.

Fixed.

/* ----------------------------------------------------------------
@@ -138,6 +236,18 @@ ExecHashJoin(PlanState *pstate)
/* no chance to not build the hash table */
node->hj_FirstOuterTupleSlot = NULL;
}
+                               else if (hashNode->parallel_state != NULL)
+                               {
+                                       /*
+                                        * The empty-outer optimization is not implemented for
+                                        * shared hash tables, because no one participant can
+                                        * determine that there are no outer tuples, and it's not
+                                        * yet clear that it's worth the synchronization overhead
+                                        * of reaching consensus to figure that out.  So we have
+                                        * to build the hash table.
+                                        */
+                                       node->hj_FirstOuterTupleSlot = NULL;
+                               }

Hm. Isn't MultiExecParallelHash already doing so?

I do support the empty-inner (empty table) optimisation.
MultiExecParallelHash makes sure that all workers agree on the total
number of inner tuples. Therefore they agree on whether to give up
early and not bother scanning the outer relation. The synchronisation
point required to implement this was unavoidable anyway (it's the
"main" synchronisation point in this algorithm: everyone has to be
finished building the hash table before anyone can probe it).

I don't support the empty-outer optimisation. I could support that
like this: all workers try to pull one tuple from the outer relation,
before even building the hash table. Then they would synchronise with
each other and see if anybody managed to get a tuple. If none of them
did, then we can give up even earlier and skip building the hash
table. The question is: is it worth introducing an extra
synchronisation point to reach consensus? It seems unlikely that the
outer relation is empty, so my answer is "no". There may be other
complications I haven't thought of.

-                               node->hj_JoinState = HJ_NEED_NEW_OUTER;
+                               if (hashtable->parallel_state)
+                               {
+                                       Barrier    *build_barrier;
+
+                                       build_barrier = &hashtable->parallel_state->build_barrier;
+                                       if (BarrierPhase(build_barrier) == PHJ_BUILD_HASHING_OUTER)
+                                       {
+                                               /*
+                                                * If multi-batch, we need to hash the outer relation
+                                                * up front.
+                                                */
+                                               if (hashtable->nbatch > 1)
+                                                       ExecParallelHashJoinPartitionOuter(node);
+                                               BarrierArriveAndWait(build_barrier,
+                                                                                        WAIT_EVENT_HASH_BUILD_HASHING_OUTER);
+                                       }
+                                       Assert(BarrierPhase(build_barrier) == PHJ_BUILD_DONE);
+
+                                       /* Each backend should now select a batch to work on. */
+                                       hashtable->curbatch = -1;
+                                       node->hj_JoinState = HJ_NEED_NEW_BATCH;
+
+                                       continue;
+                               }
+                               else
+                                       node->hj_JoinState = HJ_NEED_NEW_OUTER;

You know what I'm going to say about all these branches, and sigh.

BTW this is not per-tuple code -- it runs once at the end of hashing.
Not sure what you're looking for here.

If we don't split this into two versions, we at least should store
hashNode->parallel_state in a local var, so the compiler doesn't have to
pull that out of memory after every external function call (of which
there are a lot). In common cases it'll end up in a callee saved
registers, and most of the called functions won't be too register
starved (on x86-64).

Hmm. Well I did that already in v24 -- in many places there is now a
local variable called pstate.

+/*
+ * Choose a batch to work on, and attach to it.  Returns true if successful,
+ * false if there are no more batches.
+ */
+static bool
+ExecParallelHashJoinNewBatch(HashJoinState *hjstate)
+{
+                                       /*
+                                        * This batch is ready to probe.  Return control to
+                                        * caller. We stay attached to batch_barrier so that the
+                                        * hash table stays alive until everyone's finish probing

*finished?

Fixed.

+                               case PHJ_BATCH_DONE:
+
+                                       /*
+                                        * Already done.  Detach and go around again (if any
+                                        * remain).
+                                        */
+                                       BarrierDetach(batch_barrier);
+
+                                       /*
+                                        * We didn't work on this batch, but we need to observe
+                                        * its size for EXPLAIN.
+                                        */
+                                       ExecParallelHashUpdateSpacePeak(hashtable, batchno);
+                                       hashtable->batches[batchno].done = true;
+                                       hashtable->curbatch = -1;
+                                       break;

Hm, maybe I'm missing something, but why is it guaranteed that "we
didn't work on this batch"?

We just attached to it (see switch(BarrierAttach(batch_barrier))) and
jumped to this label. If it's PHJ_BATCH_DONE, then we know that some
other backend(s) did all the work and brought it to that state,
because otherwise our own backend-local flag
hashtable->batches[batchno].done would have been true and we'd have
skipped it.

In any case -- sorry about this -- the following patch that adds
EXPLAIN ANALYZE support actually removes that little bit of code
there. In the version you quote I wanted to make sure that every
backend had a peak at every batch's peak size. The follow-on patch
introduces extra shm state to instrument stuff better so that we get
per worker data into explain.c.

+void
+ExecShutdownHashJoin(HashJoinState *node)
+{
+       /*
+        * By the time ExecEndHashJoin runs in a worker, shared memory has been
+        * destroyed.  So this is our last chance to do any shared memory cleanup.
+        */

This comment doesn't really make much sense to me.

Tried again. The point is that ExecShutdownXXX has to leave things in
a state where ExecEndXXX won't try to follow any pointers into DSM
segments, because it'll be unmapped.

+void
+ExecHashJoinReInitializeDSM(HashJoinState *state, ParallelContext *cxt)
+{

could use a header comment.

Standard boiler-plate copied.

a) The executor side is starting to look good.

Thanks!

b) This is a lot of code.

Yeah.

On Thu, Nov 9, 2017 at 8:12 AM, Andres Freund <andres@anarazel.de> wrote:

@@ -747,7 +747,7 @@ try_hashjoin_path(PlannerInfo *root,
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
-                                                 outer_path, inner_path, extra);
+                                                 outer_path, inner_path, extra, false);
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
@@ -761,6 +761,7 @@ try_hashjoin_path(PlannerInfo *root,
extra,
outer_path,
inner_path,
+                                                                         false,        /* parallel_hash */
extra->restrictlist,
required_outer,
hashclauses));
@@ -776,6 +777,10 @@ try_hashjoin_path(PlannerInfo *root,
* try_partial_hashjoin_path
*       Consider a partial hashjoin join path; if it appears useful, push it into
*       the joinrel's partial_pathlist via add_partial_path().
+ *       The outer side is partial.  If parallel_hash is true, then the inner path
+ *       must be partial and will be run in parallel to create one or more shared
+ *       hash tables; otherwise the inner path must be complete and a copy of it
+ *       is run in every process to create separate identical private hash tables.
*/

When do we have "or more shared hash tables" rather than one? Are you
thinking about subordinate nodes?

If there are multiple batches, we'll load several hash tables into
memory at the same time.

@@ -1839,6 +1846,10 @@ hash_inner_and_outer(PlannerInfo *root,
* able to properly guarantee uniqueness.  Similarly, we can't handle
* JOIN_FULL and JOIN_RIGHT, because they can produce false null
* extended rows.  Also, the resulting path must not be parameterized.
+                * We should be able to support JOIN_FULL and JOIN_RIGHT for Parallel
+                * Hash, since in that case we're back to a single hash table with a
+                * single set of match bits for each batch, but that will require
+                * figuring out a deadlock-free way to wait for the probe to finish.
*/

s/should be able/would be able/?

Done. It would definitely work (and worked in the earlier
deadlock-prone design).

index 6a45b68e5df..2d38a5efae8 100644
--- a/src/backend/storage/ipc/barrier.c
+++ b/src/backend/storage/ipc/barrier.c
@@ -451,7 +451,6 @@ BarrierDetachImpl(Barrier *barrier, bool arrive)
release = true;
barrier->arrived = 0;
++barrier->phase;
-               Assert(barrier->selected);
barrier->selected = false;
}

Uh, what?

A fixup squashed into the wrong patch. Fixed.

As for the change itself: when I introduced BarrierArriveAndDetach()
it became possible for the barrier to advanced through phases without
anyone being selected, so the assertion wasn't valid.

diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index 35523bd8065..40a076d976f 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -5821,6 +5821,9 @@ analyze extremely_skewed;
insert into extremely_skewed
select 42 as id, 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'
from generate_series(1, 19000);
+-- Make a relation with a couple of enormous tuples.
+create table wide as select generate_series(1, 2) as id, rpad('', 320000, 'x') as t;
+alter table wide set (parallel_workers = 2);

I'm doubtful this is actually going to be a wide tuple - this'll get
compressed down quite a bit, no?

postgres[26465][1]=# SELECT octet_length(t), pg_column_size(t) FROM wide ;
┌──────────────┬────────────────┐
│ octet_length │ pg_column_size │
├──────────────┼────────────────┤
│ 320000 │ 3671 │
│ 320000 │ 3671 │
└──────────────┴────────────────┘
(2 rows)

(and yes, it's ridiculous that a compressed datum of that size still
takes up 3kb)

The tuple is small on disk, but it allows me to create a large tuple
in the hash table with the following incantation (hat-tip to Andrew
Gierth for this trick):

select length(max(s.t))
from wide left join (select id, coalesce(t, '') || '' as t from
wide) s using (id);

A non-strict expression and a left join result in a projected
detoasted decompressed monster tuple which you can confirm by sticking
an elog into ExecParallelHashLoadTuple() like so:

2017-11-10 13:57:07.193 NZDT [7337] LOG: tuple size = 320040

+-- parallel with parallel-aware hash join
+set max_parallel_workers_per_gather = 2;
+set work_mem = '128kB';
+set enable_parallel_hash = on;

I think it'd be better if we structured the file so we just sat guc's
with SET LOCAL inside a transaction.

I wrapped the whole region of join.sql concerned with hash joins in a
transaction that rolls back, so I don't have to write LOCAL. That's
just as good, right?

All-in-all this part looks fairly boring.

Thank you.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

parallel-hash-v25.patchset.tgzapplication/x-gzip; name=parallel-hash-v25.patchset.tgzDownload
#38Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#37)
Re: [HACKERS] Parallel Hash take II

Hi,

On 2017-11-14 01:30:30 +1300, Thomas Munro wrote:

+-- The "good" case: batches required, but we plan the right number; we
+-- plan for 16 batches, and we stick to that number, and peak memory
+-- usage says within our work_mem budget
+-- non-parallel
+set max_parallel_workers_per_gather = 0;
+set work_mem = '128kB';

So how do we know that's actually the case we're testing rather than
something arbitrarily different? There's IIRC tests somewhere that just
filter the json explain output to the right parts...

Yeah, good idea. My earlier attempts to dump out the hash join
dimensions ran into problems with architecture sensitivity and then
some run-to-run non-determinism in the parallel case (due to varying
fragmentation depending on how many workers get involved in time).
The attached version tells you about batch growth without reporting
the exact numbers, except in the "ugly" case where we know that there
is only one possible outcome because the extreme skew detector is
guaranteed to go off after the first nbatch increase (I got rid of all
other tuples except ones with the same key to make this true).

Hm. The way you access this doesn't quite seem right:
+--
+-- exercises for the hash join code
+--
+begin;
+set min_parallel_table_scan_size = 0;
+set parallel_setup_cost = 0;
+-- Extract bucket and batch counts from an explain analyze plan.  In
+-- general we can't make assertions about how many batches (or
+-- buckets) will be required because it can vary, but we can in some
+-- special cases and we can check for growth.
+create or replace function hash_join_batches(query text)
+returns table (original int, final int) language plpgsql
+as
+$$
+declare
+  line text;
+  matches text[];
+begin
+  for line in
+    execute 'explain analyze ' || query
+  loop
+    matches := (regexp_matches(line, '  Batches: ([0-9]+) \(originally ([0-9]+)\)'));
+    if matches is not null then
+      original := matches[2]::int;
+      final := matches[1]::int;
+      return next;
+    else
+      matches := regexp_matches(line, '  Batches: ([0-9]+)');
+      if matches is not null then
+        original := matches[1]::int;
+        final := original;
+        return next;
+      end if;
+    end if;
+  end loop;
+end;
+$$;

Why not use format json and access the output that way? Then you can be
sure you access the right part of the tree and such?

+       else
+       {
+               errno = stat_errno;
+               elog(LOG, "could not stat file \"%s\": %m", path);
+       }

All these messages are "not expected to ever happen" ones, right?

You'd have to suffer a nasty filesystem failure, remount read-only or
manually with permissions or something. Not sure where the line is,
but I've changed all of these new elog calls to ereport.

Oh, I'd been fine keeping them as elogs. The one exception would have
been out-of-space cases which'll occur in practice.

+       if (vfdP->fdstate & FD_TEMP_FILE_LIMIT)
+       {
+               /* Subtract its size from current usage (do first in case of error) */
+               temporary_files_size -= vfdP->fileSize;
+               vfdP->fileSize = 0;
+       }

So, is it right to do so unconditionally and without regard for errors?
If the file isn't deleted, it shouldn't be subtracted from fileSize. I
guess you're managing that through the flag, but that's not entirely
obvious.

I think it is. Reasoning: The existing behaviour of fd.c is that if
we don't manage to delete temporary files, we'll LOG something and
forget about them (they'll be cleaned up eventually by a clean restart
or human intervention).

IOW: Never ;)

+/*
+ * Write a tuple.  If a meta-data size was provided to sts_initialize, then a
+ * pointer to meta data of that size must be provided.
+ */
+void
+sts_puttuple(SharedTuplestoreAccessor *accessor, void *meta_data,
+                        MinimalTuple tuple)
+{
+       /* Do we have space? */
+       size = accessor->sts->meta_data_size + tuple->t_len;
+       if (accessor->write_pointer + size >= accessor->write_end)
+       {
+               /* Try flushing to see if that creates enough space. */
+               if (accessor->write_chunk != NULL)
+                       sts_flush_chunk(accessor);
+
+               /*
+                * It may still not be enough in the case of a gigantic tuple, or if
+                * we haven't created a chunk buffer at all yet.
+                */
+               if (accessor->write_pointer + size >= accessor->write_end)
+               {
+                       SharedTuplestoreParticipant *participant;
+                       size_t  space_needed;
+                       int             pages_needed;
+
+                       /* How many pages to hold this data and the chunk header? */
+                       space_needed = offsetof(SharedTuplestoreChunk, data) + size;
+                       pages_needed = (space_needed + (BLCKSZ - 1)) / BLCKSZ;
+                       pages_needed = Max(pages_needed, STS_MIN_CHUNK_PAGES);
+
+                       /*
+                        * Double the chunk size until it's big enough, and record that
+                        * fact in the shared expansion log so that readers know about it.
+                        */
+                       participant = &accessor->sts->participants[accessor->participant];
+                       while (accessor->write_pages < pages_needed)
+                       {
+                               accessor->write_pages *= 2;
+                               participant->chunk_expansion_log[participant->chunk_expansions++] =
+                                       accessor->write_page;
+                       }

Hm. Isn't that going to be pretty unfunny if you have one large and a
lot of small tuples?

It will increase the parallel scan grain size, and then keep that size
for the rest of the contents of one backend's output file. I am aware
of two downsides to using a large parallel grain:

1. It determines the amount of unfairness when we run out of data:
it's the maximum amount of extra data that the unlucky last worker can
finish up with after all the others have finished. I think this
effect is reduced by higher level factors: when a reader runs out of
data in one backend's file, it'll start reading another backend's
file. If it's hit the end of all backends' files and this is an outer
batch, Parallel Hash will just go and work on another batch
immediately.

Consider e.g. what happens if there's the occasional 500MB datum, and
the rest's very small...

Better ideas?

Not really. I'm more than a bit suspicous of this solution, but I don't
really have a great suggestion otherwise. One way to combat extreme
size skew would be to put very large datums into different files.

But I think we probably can go with your approach for now, ignoring my
failure prone spidey senses ;)

+               /* Find the location of a new chunk to read. */
+               p = &accessor->sts->participants[accessor->read_participant];
+
+               SpinLockAcquire(&p->mutex);
+               eof = p->read_page >= p->npages;
+               if (!eof)
+               {
+                       /*
+                        * Figure out how big this chunk is.  It will almost always be the
+                        * same as the last chunk loaded, but if there is one or more
+                        * entry in the chunk expansion log for this page then we know
+                        * that it doubled that number of times.  This avoids the need to
+                        * do IO to adjust the read head, so we don't need to hold up
+                        * concurrent readers.  (An alternative to this extremely rarely
+                        * run loop would be to use more space storing the new size in the
+                        * log so we'd have 'if' instead of 'while'.)
+                        */
+                       read_page = p->read_page;
+                       while (p->chunk_expansion < p->chunk_expansions &&
+                                  p->chunk_expansion_log[p->chunk_expansion] == p->read_page)
+                       {
+                               p->chunk_pages *= 2;
+                               p->chunk_expansion++;
+                       }
+                       chunk_pages = p->chunk_pages;
+
+                       /* The next reader will start after this chunk. */
+                       p->read_page += chunk_pages;
+               }
+               SpinLockRelease(&p->mutex);

This looks more like the job of an lwlock rather than a spinlock.

I switched to the alternative algorithm mentioned in parentheses in
the comment. It uses a bit more space, but that loop is gone. In my
mind this is much like Parallel Seq Scan: we acquire a spinlock just
to advance the block pointer. The added complication is that we also
check if the chunk size has changed, which clang renders as this many
instructions:

postgres[0x10047eee0] <+176>: movslq 0x144(%r15,%rbx), %rcx
postgres[0x10047eee8] <+184>: cmpl 0x140(%r15,%rbx), %ecx
postgres[0x10047eef0] <+192>: jge 0x10047ef16 ;
<+230> at sharedtuplestore.c:489
postgres[0x10047eef2] <+194>: leaq (%r15,%rbx), %rdx
postgres[0x10047eef6] <+198>: cmpl %r12d, 0x40(%rdx,%rcx,8)
postgres[0x10047eefb] <+203>: jne 0x10047ef16 ;
<+230> at sharedtuplestore.c:489
postgres[0x10047eefd] <+205>: leaq 0x144(%r15,%rbx), %rsi
postgres[0x10047ef05] <+213>: leal 0x1(%rcx), %edi
postgres[0x10047ef08] <+216>: movl %edi, (%rsi)
postgres[0x10047ef0a] <+218>: movl 0x44(%rdx,%rcx,8), %ecx
postgres[0x10047ef0e] <+222>: movl %ecx, 0x148(%r15,%rbx)
postgres[0x10047ef16] <+230>: movl 0x148(%r15,%rbx), %r15d

That should be OK, right?

It's not too bad. Personally I'm of the opinion though that pretty much
no new spinlocks should be added - their worst case performance
characteristics are bad enough for that to be only worth the
experimentation in case swhere each cycle really matters and where
contention is unlikely.

One day we're going to need a better approach to this. I have no idea
how, but this per-node, and now per_node * max_parallelism, approach has
only implementation simplicity as its benefit.

I agree, and I am interested in that subject. In the meantime, I
think it'd be pretty unfair if parallel-oblivious hash join and
sort-merge join and every other parallel plan get to use work_mem * p
(and in some cases waste it with duplicate data), but Parallel Hash
isn't allowed to do the same (and put it to good use).

I'm not sure I care about fairness between pieces of code ;)

-                               node->hj_JoinState = HJ_NEED_NEW_OUTER;
+                               if (hashtable->parallel_state)
+                               {
+                                       Barrier    *build_barrier;
+
+                                       build_barrier = &hashtable->parallel_state->build_barrier;
+                                       if (BarrierPhase(build_barrier) == PHJ_BUILD_HASHING_OUTER)
+                                       {
+                                               /*
+                                                * If multi-batch, we need to hash the outer relation
+                                                * up front.
+                                                */
+                                               if (hashtable->nbatch > 1)
+                                                       ExecParallelHashJoinPartitionOuter(node);
+                                               BarrierArriveAndWait(build_barrier,
+                                                                                        WAIT_EVENT_HASH_BUILD_HASHING_OUTER);
+                                       }
+                                       Assert(BarrierPhase(build_barrier) == PHJ_BUILD_DONE);
+
+                                       /* Each backend should now select a batch to work on. */
+                                       hashtable->curbatch = -1;
+                                       node->hj_JoinState = HJ_NEED_NEW_BATCH;
+
+                                       continue;
+                               }
+                               else
+                                       node->hj_JoinState = HJ_NEED_NEW_OUTER;

You know what I'm going to say about all these branches, and sigh.

BTW this is not per-tuple code -- it runs once at the end of hashing.
Not sure what you're looking for here.

It was more a general statement about all the branches in nodeHashjoin,
than about these specific branches. Should've made that clearer. There's
definitely branches in very common parts:
case HJ_NEED_NEW_OUTER:

/*
* We don't have an outer tuple, try to get the next one
*/
if (hashtable->parallel_state)
outerTupleSlot =
ExecParallelHashJoinOuterGetTuple(outerNode, node,
&hashvalue);
else
outerTupleSlot =
ExecHashJoinOuterGetTuple(outerNode, node, &hashvalue);

I don't think you should do so now, but I think a reasonable approach
here would be to move the HJ_BUILD_HASHTABLE code into a separate
function (it really can't be hot). Then have specialized ExecHashJoin()
versions for parallel/non-parallel and potentially for outer/inner/anti.

If we don't split this into two versions, we at least should store
hashNode->parallel_state in a local var, so the compiler doesn't have to
pull that out of memory after every external function call (of which
there are a lot). In common cases it'll end up in a callee saved
registers, and most of the called functions won't be too register
starved (on x86-64).

Hmm. Well I did that already in v24 -- in many places there is now a
local variable called pstate.

See above piece of code, and a few others, in nodeHash.

I think it'd be better if we structured the file so we just sat guc's
with SET LOCAL inside a transaction.

I wrapped the whole region of join.sql concerned with hash joins in a
transaction that rolls back, so I don't have to write LOCAL. That's
just as good, right?

Not really imo. Being able to read a test without going through all
previous ones is a lot better.

Greetings,

Andres Freund

#39Robert Haas
robertmhaas@gmail.com
In reply to: Andres Freund (#38)
Re: [HACKERS] Parallel Hash take II

On Tue, Nov 14, 2017 at 4:24 PM, Andres Freund <andres@anarazel.de> wrote:

I agree, and I am interested in that subject. In the meantime, I
think it'd be pretty unfair if parallel-oblivious hash join and
sort-merge join and every other parallel plan get to use work_mem * p
(and in some cases waste it with duplicate data), but Parallel Hash
isn't allowed to do the same (and put it to good use).

I'm not sure I care about fairness between pieces of code ;)

I realize you're sort of joking here, but I think it's necessary to
care about fairness between pieces of code.

I mean, the very first version of this patch that Thomas submitted was
benchmarked by Rafia and had phenomenally good performance
characteristics. That turned out to be because it wasn't respecting
work_mem; you can often do a lot better with more memory, and
generally you can't do nearly as well with less. To make comparisons
meaningful, they have to be comparisons between algorithms that use
the same amount of memory. And it's not just about testing. If we
add an algorithm that will run twice as fast with equal memory but
only allow it half as much, it will probably never get picked and the
whole patch is a waste of time.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#40Andres Freund
andres@anarazel.de
In reply to: Robert Haas (#39)
Re: [HACKERS] Parallel Hash take II

On 2017-11-15 08:37:11 -0500, Robert Haas wrote:

On Tue, Nov 14, 2017 at 4:24 PM, Andres Freund <andres@anarazel.de> wrote:

I agree, and I am interested in that subject. In the meantime, I
think it'd be pretty unfair if parallel-oblivious hash join and
sort-merge join and every other parallel plan get to use work_mem * p
(and in some cases waste it with duplicate data), but Parallel Hash
isn't allowed to do the same (and put it to good use).

I'm not sure I care about fairness between pieces of code ;)

I realize you're sort of joking here, but I think it's necessary to
care about fairness between pieces of code.

Indeed I kinda was.

I mean, the very first version of this patch that Thomas submitted was
benchmarked by Rafia and had phenomenally good performance
characteristics. That turned out to be because it wasn't respecting
work_mem; you can often do a lot better with more memory, and
generally you can't do nearly as well with less. To make comparisons
meaningful, they have to be comparisons between algorithms that use
the same amount of memory. And it's not just about testing. If we
add an algorithm that will run twice as fast with equal memory but
only allow it half as much, it will probably never get picked and the
whole patch is a waste of time.

But this does bug me, and I think it's what made me pause here to make a
bad joke. The way that parallelism treats work_mem makes it even more
useless of a config knob than it was before. Parallelism, especially
after this patch, shouldn't compete / be benchmarked against a
single-process run with the same work_mem. To make it "fair" you need to
compare parallelism against a single threaded run with work_mem *
max_parallelism.

Thomas argues that this makes hashjoins be treated faily vis-a-vi
parallel-oblivious hash join etc. And I think he has somewhat of a
point. But I don't think it's quite right either: In several of these
cases the planner will not prefer the multi-process plan because it uses
more work_mem, it's a cost to be paid. Whereas this'll optimize towards
using work_mem * max_parallel_workers_per_gather amount of memory.

This makes it pretty much impossible to afterwards tune work_mem on a
server in a reasonable manner. Previously you'd tune it to something
like free_server_memory - (max_connections * work_mem *
80%_most_complex_query). Which you can't really do anymore now, you'd
also need to multiply by max_parallel_workers_per_gather. Which means
that you might end up "forcing" paralellism on a bunch of plans that'd
normally execute in too short a time to make parallelism worth it.

I don't really have a good answer to "but what should we otherwise do",
but I'm doubtful this is quite the right answer.

Greetings,

Andres Freund

In reply to: Andres Freund (#40)
Re: [HACKERS] Parallel Hash take II

On Wed, Nov 15, 2017 at 10:35 AM, Andres Freund <andres@anarazel.de> wrote:

I realize you're sort of joking here, but I think it's necessary to
care about fairness between pieces of code.

Indeed I kinda was.

When I posted v1 of parallel CREATE INDEX, it followed the hash join
model of giving workMem (maintenance_work_mem) to every worker. Robert
suggested that my comparison with a serial case was therefore not
representative, since I was using much more memory. I actually changed
the patch to use a single maintenance_work_mem share for the entire
operation afterwards, which seemed to work better. And, it made very
little difference to performance for my original benchmark in the end,
so I was arguably wasting memory in v1.

I mean, the very first version of this patch that Thomas submitted was
benchmarked by Rafia and had phenomenally good performance
characteristics. That turned out to be because it wasn't respecting
work_mem; you can often do a lot better with more memory, and
generally you can't do nearly as well with less. To make comparisons
meaningful, they have to be comparisons between algorithms that use
the same amount of memory. And it's not just about testing. If we
add an algorithm that will run twice as fast with equal memory but
only allow it half as much, it will probably never get picked and the
whole patch is a waste of time.

The contrast with the situation with Thomas and his hash join patch is
interesting. Hash join is *much* more sensitive to the availability of
memory than a sort operation is.

I don't really have a good answer to "but what should we otherwise do",
but I'm doubtful this is quite the right answer.

I think that the work_mem model should be replaced by something that
centrally budgets memory. It would make sense to be less generous with
sorts and more generous with hash joins when memory is in short
supply, for example, and a model like this can make that possible. The
work_mem model has always forced users to be far too conservative.
Workloads are very complicated, and always having users target the
worst case leaves a lot to be desired.

--
Peter Geoghegan

#42Andres Freund
andres@anarazel.de
In reply to: Peter Geoghegan (#41)
Re: [HACKERS] Parallel Hash take II

Hi,

On 2017-11-15 10:57:35 -0800, Peter Geoghegan wrote:

I don't really have a good answer to "but what should we otherwise do",
but I'm doubtful this is quite the right answer.

I think that the work_mem model should be replaced by something that
centrally budgets memory. It would make sense to be less generous with
sorts and more generous with hash joins when memory is in short
supply, for example, and a model like this can make that possible. The
work_mem model has always forced users to be far too conservative.
Workloads are very complicated, and always having users target the
worst case leaves a lot to be desired.

Obviously that's nice and worthwhile goal, but it seems more than a bit
out of reach for this patchset.

Greetings,

Andres Freund

#43Robert Haas
robertmhaas@gmail.com
In reply to: Andres Freund (#40)
Re: [HACKERS] Parallel Hash take II

On Wed, Nov 15, 2017 at 1:35 PM, Andres Freund <andres@anarazel.de> wrote:

But this does bug me, and I think it's what made me pause here to make a
bad joke. The way that parallelism treats work_mem makes it even more
useless of a config knob than it was before. Parallelism, especially
after this patch, shouldn't compete / be benchmarked against a
single-process run with the same work_mem. To make it "fair" you need to
compare parallelism against a single threaded run with work_mem *
max_parallelism.

I don't really know how to do a fair comparison between a parallel
plan and a non-parallel plan. Even if the parallel plan contains zero
nodes that use work_mem, it might still use more memory than the
non-parallel plan, because a new backend uses a bunch of memory. If
you really want a comparison that is fair on the basis of memory
usage, you have to take that into account somehow.

But even then, the parallel plan is also almost certainly consuming
more CPU cycles to produce the same results. Parallelism is all about
trading away efficiency for execution time. Not just because of
current planner and executor limitations, but intrinsically, parallel
plans are less efficient. The globally optimal solution on a system
that is short on either memory or CPU cycles is to turn parallelism
off.

Thomas argues that this makes hashjoins be treated faily vis-a-vi
parallel-oblivious hash join etc. And I think he has somewhat of a
point. But I don't think it's quite right either: In several of these
cases the planner will not prefer the multi-process plan because it uses
more work_mem, it's a cost to be paid. Whereas this'll optimize towards
using work_mem * max_parallel_workers_per_gather amount of memory.

In general, we have no framework for evaluating the global impact on
the system of our decisions. Not just with parallelism, but in
general, plans that use memory are going to typically beat plans that
don't, because using more memory is a good way to make things run
faster, so the cost goes down, and the cost is what matters.
Everything optimizes for eating as many resources as possible, even if
there is only an extremely marginal gain over a more costly plan that
uses dramatically fewer resources.

A good example of this is a parallel bitmap heap scan vs. a parallel
sequential scan. With enough workers, we switch from having one
worker scan the index and then having all workers do a joint scan of
the heap -- to just performing a parallel sequential scan. Because of
Moore's law, as you add workers, waiting for the non-parallel index
scan to build the bitmap eventually looks less desirable than
accepting that you're going to uselessly scan a lot of pages -- you
stop caring, because you have enough raw power to just blast through
it.

The best non-parallel example I can think of off-hand is sorts.
Sometimes, reducing the amount of memory available for a sort really
doesn't cost very much in terms of execution time, but we always run a
sort with the full allotted work_mem, even if that's gigantic. We
won't use it all if the data is smaller than work_mem, but if there's
batching going on then we will, even if it doesn't really help.

This makes it pretty much impossible to afterwards tune work_mem on a
server in a reasonable manner. Previously you'd tune it to something
like free_server_memory - (max_connections * work_mem *
80%_most_complex_query). Which you can't really do anymore now, you'd
also need to multiply by max_parallel_workers_per_gather. Which means
that you might end up "forcing" paralellism on a bunch of plans that'd
normally execute in too short a time to make parallelism worth it.

I think you just need to use max_connections +
Min(max_parallel_workers, max_worker_processes) instead of
max_connections. You can't use parallel query for every query at the
same time with reasonable settings...

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#44Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#40)
Re: [HACKERS] Parallel Hash take II

On Thu, Nov 16, 2017 at 7:35 AM, Andres Freund <andres@anarazel.de> wrote:

On 2017-11-15 08:37:11 -0500, Robert Haas wrote:

I mean, the very first version of this patch that Thomas submitted was
benchmarked by Rafia and had phenomenally good performance
characteristics. That turned out to be because it wasn't respecting
work_mem; you can often do a lot better with more memory, and
generally you can't do nearly as well with less. To make comparisons
meaningful, they have to be comparisons between algorithms that use
the same amount of memory. And it's not just about testing. If we
add an algorithm that will run twice as fast with equal memory but
only allow it half as much, it will probably never get picked and the
whole patch is a waste of time.

But this does bug me, and I think it's what made me pause here to make a
bad joke. The way that parallelism treats work_mem makes it even more
useless of a config knob than it was before. Parallelism, especially
after this patch, shouldn't compete / be benchmarked against a
single-process run with the same work_mem. To make it "fair" you need to
compare parallelism against a single threaded run with work_mem *
max_parallelism.

Thomas argues that this makes hashjoins be treated faily vis-a-vi
parallel-oblivious hash join etc. And I think he has somewhat of a
point. But I don't think it's quite right either: In several of these
cases the planner will not prefer the multi-process plan because it uses
more work_mem, it's a cost to be paid. Whereas this'll optimize towards
using work_mem * max_parallel_workers_per_gather amount of memory.

This makes it pretty much impossible to afterwards tune work_mem on a
server in a reasonable manner. Previously you'd tune it to something
like free_server_memory - (max_connections * work_mem *
80%_most_complex_query). Which you can't really do anymore now, you'd
also need to multiply by max_parallel_workers_per_gather. Which means
that you might end up "forcing" paralellism on a bunch of plans that'd
normally execute in too short a time to make parallelism worth it.

Currently our way of choosing the number of workers is 'rule based':
we use a simple formula that takes relation sizes and some GUCs and
per-relation options as inputs. The comparison against non-parallel
plans is cost based of course, but we won't consider any other number
of workers.

Suppose we had 'cost based' worker number selection instead: simply
try planning with various different worker counts and pick the winner.
Then I think we'd see the moral hazard in this scheme more clearly:
the planner effectively has a free memory printing press. It will
think that it's a good idea to use a huge number of workers to get
more and more work_mem-sized hash tables or in-memory sorts into
memory (whether that's with partition-wise join, Parallel Hash, or
something else).

We could switch to a model where work_mem is divided by the number of
workers. Parallel Hash would be able to use a full work_mem by
combining them, and parallel-oblivious Hash would be able to use only
work_mem / participants. That'd be the other way to give Parallel
Hash a fair amount of memory compared to the competition, but I didn't
propose that because it'd be a change to the already-released
behaviour. Then I'd have been saying "hey, look at this new plan I
wrote, it performs really well if you first tie the other plans' shoe
laces together". It may actually be better though, even without
Parallel Hash in the picture.

--
Thomas Munro
http://www.enterprisedb.com

#45Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Peter Geoghegan (#41)
Re: [HACKERS] Parallel Hash take II

On Thu, Nov 16, 2017 at 7:57 AM, Peter Geoghegan <pg@bowt.ie> wrote:

The contrast with the situation with Thomas and his hash join patch is
interesting. Hash join is *much* more sensitive to the availability of
memory than a sort operation is.

I don't really have a good answer to "but what should we otherwise do",
but I'm doubtful this is quite the right answer.

I think that the work_mem model should be replaced by something that
centrally budgets memory. It would make sense to be less generous with
sorts and more generous with hash joins when memory is in short
supply, for example, and a model like this can make that possible. The
work_mem model has always forced users to be far too conservative.
Workloads are very complicated, and always having users target the
worst case leaves a lot to be desired.

In the old days, Oracle had only simple per-operation memory limits
too, and that applied to every operation in every thread just like our
work_mem. It's interesting that they had separate knobs for sort and
hash though, and defaulted to giving hash twice as much.

With a whole-plan memory target, our planner would probably begin to
plan join order differently to minimise the number of hash tables in
memory at once, like other RDBMSs. Not sure how the plan-wide target
should work though -- try many plans, giving different portions of
budget to different subplans? That should work fine if you like
O(please-melt-my-computer), especially if combined with a similar
approach to choosing worker numbers. Some kind of feedback system?
Seems like a different kind of planner, but I have no clue. If you
have ideas/papers/references, it'd be great to see a new thread on
that subject.

--
Thomas Munro
http://www.enterprisedb.com

#46Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Robert Haas (#43)
Re: [HACKERS] Parallel Hash take II

On Thu, Nov 16, 2017 at 8:09 AM, Robert Haas <robertmhaas@gmail.com> wrote:

On Wed, Nov 15, 2017 at 1:35 PM, Andres Freund <andres@anarazel.de> wrote:

But this does bug me, and I think it's what made me pause here to make a
bad joke. The way that parallelism treats work_mem makes it even more
useless of a config knob than it was before. Parallelism, especially
after this patch, shouldn't compete / be benchmarked against a
single-process run with the same work_mem. To make it "fair" you need to
compare parallelism against a single threaded run with work_mem *
max_parallelism.

I don't really know how to do a fair comparison between a parallel
plan and a non-parallel plan. Even if the parallel plan contains zero
nodes that use work_mem, it might still use more memory than the
non-parallel plan, because a new backend uses a bunch of memory. If
you really want a comparison that is fair on the basis of memory
usage, you have to take that into account somehow.

But even then, the parallel plan is also almost certainly consuming
more CPU cycles to produce the same results. Parallelism is all about
trading away efficiency for execution time. Not just because of
current planner and executor limitations, but intrinsically, parallel
plans are less efficient. The globally optimal solution on a system
that is short on either memory or CPU cycles is to turn parallelism
off.

The guys who worked on the first attempt at Parallel Query for
Berkeley POSTGRES (and then ripped that out, moving to another project
called XPRS which I have found no trace of, perhaps it finished up in
some commercial RDBMS) wrote this[1]:

"The objective function that XPRS uses for query optimization is a
combination of resource consumption and response time as follows:

cost = resource consumption + w * response time

Here w is a system-specifc weighting factor. A small w mostly
optimizes resource consumption, while a large w mostly optimizes
response time. Resource consumption is measured by the number of disk
pages accessed and number of tuples processed, while response time is
the elapsed time for executing the query."

http://db.cs.berkeley.edu/papers/ERL-M93-28.pdf

--
Thomas Munro
http://www.enterprisedb.com

#47Andres Freund
andres@anarazel.de
In reply to: Robert Haas (#43)
Re: [HACKERS] Parallel Hash take II

Hi,

On 2017-11-15 14:09:13 -0500, Robert Haas wrote:

On Wed, Nov 15, 2017 at 1:35 PM, Andres Freund <andres@anarazel.de> wrote:

But this does bug me, and I think it's what made me pause here to make a
bad joke. The way that parallelism treats work_mem makes it even more
useless of a config knob than it was before. Parallelism, especially
after this patch, shouldn't compete / be benchmarked against a
single-process run with the same work_mem. To make it "fair" you need to
compare parallelism against a single threaded run with work_mem *
max_parallelism.

I don't really know how to do a fair comparison between a parallel
plan and a non-parallel plan. Even if the parallel plan contains zero
nodes that use work_mem, it might still use more memory than the
non-parallel plan, because a new backend uses a bunch of memory. If
you really want a comparison that is fair on the basis of memory
usage, you have to take that into account somehow.

That's not quite what I'm concerned about. Consider something
(completely artifical) like:

tpch_5[18786][1]=# SET work_mem = '50MB';
tpch_5[18786][1]=# EXPLAIN SELECT c_name, count(*) FROM orders JOIN customer ON (o_custkey = c_custkey) WHERE o_orderdate BETWEEN '1995-01-01' AND '1995-01-05' GROUP BY 1 ORDER BY count(*) DESC LIMIT 10;
┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ QUERY PLAN │
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Limit (cost=77344.16..77344.19 rows=10 width=27) │
│ -> Sort (cost=77344.16..77379.68 rows=14206 width=27) │
│ Sort Key: (count(*)) DESC │
│ -> HashAggregate (cost=76895.12..77037.18 rows=14206 width=27) │
│ Group Key: customer.c_name │
│ -> Hash Join (cost=35347.04..76824.09 rows=14206 width=19) │
│ Hash Cond: (orders.o_custkey = customer.c_custkey) │
│ -> Bitmap Heap Scan on orders (cost=302.04..41599.74 rows=14206 width=4) │
│ Recheck Cond: ((o_orderdate >= '1995-01-01'::date) AND (o_orderdate <= '1995-01-05'::date)) │
│ -> Bitmap Index Scan on i_o_orderdate (cost=0.00..298.49 rows=14206 width=0) │
│ Index Cond: ((o_orderdate >= '1995-01-01'::date) AND (o_orderdate <= '1995-01-05'::date)) │
│ -> Hash (cost=25670.00..25670.00 rows=750000 width=23) │
│ -> Seq Scan on customer (cost=0.00..25670.00 rows=750000 width=23) │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
(13 rows)

tpch_5[18786][1]=# SET work_mem = '10MB';
tpch_5[18786][1]=# EXPLAIN SELECT c_name, count(*) FROM orders JOIN customer ON (o_custkey = c_custkey) WHERE o_orderdate BETWEEN '1995-01-01' AND '1995-01-05' GROUP BY 1 ORDER BY count(*) DESC LIMIT 10;
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ QUERY PLAN │
├─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Limit (cost=82847.92..82847.94 rows=10 width=27) │
│ -> Sort (cost=82847.92..82883.43 rows=14206 width=27) │
│ Sort Key: (count(*)) DESC │
│ -> HashAggregate (cost=82398.87..82540.93 rows=14206 width=27) │
│ Group Key: customer.c_name │
│ -> Merge Join (cost=42580.44..82327.84 rows=14206 width=19) │
│ Merge Cond: (customer.c_custkey = orders.o_custkey) │
│ -> Index Scan using i_c_custkey on customer (cost=0.42..37663.43 rows=750000 width=23) │
│ -> Sort (cost=42579.54..42615.05 rows=14206 width=4) │
│ Sort Key: orders.o_custkey │
│ -> Bitmap Heap Scan on orders (cost=302.04..41599.74 rows=14206 width=4) │
│ Recheck Cond: ((o_orderdate >= '1995-01-01'::date) AND (o_orderdate <= '1995-01-05'::date)) │
│ -> Bitmap Index Scan on i_o_orderdate (cost=0.00..298.49 rows=14206 width=0) │
│ Index Cond: ((o_orderdate >= '1995-01-01'::date) AND (o_orderdate <= '1995-01-05'::date)) │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
(14 rows)

Note how the plan switched from a hashjoin to a mergejoin solely on the
basis of different work_mem settings, and that there's obviously
different costs associated between the two plans.

What I'm basically worried about is that the *only* reason for some
plans to choose to use parallelism is that essentially the effective
amount of work_mem between the plans is that the parallel one uses
(max_parallel_workers_per_gather + 1) * work_mem. Which might push
queries to use parallelism even if it's not actually beneficial in
reducing runtime.

Thomas' earlier comparison of this behaviour with e.g. parallel
oblivious hash nodes does *NOT* seem apt to me. There's currently
effectively no cost pressure for/against parallelism for those (even if
there potentially should). Which means they do not favor parallel
queries solely because they're allowed to use more memory, and thus it's
far less likely that every of those nodes uses the maximum alloted
work_mem.

I think it's wrong to just multiply the amount of work_mem that way, and
it'll bite use. Introducing a separate guc, perhaps inheriting from
work_mem if set to -1, that limits the amount of memory inside a
parallel node seems saner. That value then would not be multiplied with
the chosen worker number.

But even then, the parallel plan is also almost certainly consuming
more CPU cycles to produce the same results. Parallelism is all about
trading away efficiency for execution time. Not just because of
current planner and executor limitations, but intrinsically, parallel
plans are less efficient. The globally optimal solution on a system
that is short on either memory or CPU cycles is to turn parallelism
off.

I do think that our parallelism isn't properly tunable on that front. I
think we really need something like a 'parallel_resource_efficiency'
[0..1] GUC.

As far as I understand it we currently cost a gather's startup cost
once, not per worker. That startup cost effectively include redundant
work like materializing a table, building parallel oblivious hashtables,
resorting the same table for a mergejoin etc...

That's fine if your goal is solely to return a single query as fast as
possible, even if doubling the resources will only give you a minimal
cost advantage. If instead your goal is to optimize both for individual
query performance and overall system throughput that's obviously not
good.

I think we should cost the startup cost of paralell nodes more like
startup_cost * (max_parallel_workers_per_gather * parallel_resource_efficiency + 1)

which'd allow to tune query performance for using parallelism even for
tiny benefits (parallel_resource_efficiency = 0), and conversely tune it
so it only gets used if the overall loss of efficiency is small
(parallel_resource_efficiency = 0.9 or such).

(skipping over a lot of details for such a proposal)

Now, currently that'd have the weakness that we sometimes would end up
not using parallelism, because at the determined amount of parallelism
it's not beneficial to use it, even though it'd be still worthwhile to
use a lower level of parallelism. But given we currently don't plan for
multiple degrees of parallelism that seems the right thing to do if you
care about efficiency / overall throughput.

Greetings,

Andres Freund

#48Robert Haas
robertmhaas@gmail.com
In reply to: Andres Freund (#47)
Re: [HACKERS] Parallel Hash take II

On Thu, Nov 16, 2017 at 6:42 PM, Andres Freund <andres@anarazel.de> wrote:

What I'm basically worried about is that the *only* reason for some
plans to choose to use parallelism is that essentially the effective
amount of work_mem between the plans is that the parallel one uses
(max_parallel_workers_per_gather + 1) * work_mem. Which might push
queries to use parallelism even if it's not actually beneficial in
reducing runtime.

I don't get it. If we switch to using parallelism and the runtime
doesn't go down, that just means the costing is wrong. The costing is
supposed to reflect the runtime.

It's true that adding parallel hash may tend to increase the amount of
memory that gets used in practice. But it seems to me that any plan
that uses memory and is sometimes chosen over a non-memory-using plan
does that.

Thomas' earlier comparison of this behaviour with e.g. parallel
oblivious hash nodes does *NOT* seem apt to me. There's currently
effectively no cost pressure for/against parallelism for those (even if
there potentially should). Which means they do not favor parallel
queries solely because they're allowed to use more memory, and thus it's
far less likely that every of those nodes uses the maximum alloted
work_mem.

I agree that there is no cost pressure for or against parallelism.
The design which I have been pursuing with parallel query up to this
point is that cost represents execution time, so minimizing cost means
minimizing execution time, and that's the goal. If we want, we can
put a tax on parallel query plans, so that they're only chosen when
they are *substantially* cheaper than non-parallel plans, or even that
a plan with a few workers is to be preferred over a plan with more
workers unless the gains are sufficiently substantial. But I don't
think the right way to do that is to prevent a parallel hash from
using as much memory as a non-parallel hash in the same spot would
use.

Rather, what we could do is, for example, have a knob that multiplies
the cost of a partial path by a floating-point value when we insert a
Gather/Gather Merge node. If you want to always pick the cheapest
path regardless of whether it uses parallelism, set the GUC to 1.0.
If you only want to pick a parallel query path if it's twice as cheap
as the best non-parallel path, set the GUC to 2.0.

I think it's wrong to just multiply the amount of work_mem that way, and
it'll bite use. Introducing a separate guc, perhaps inheriting from
work_mem if set to -1, that limits the amount of memory inside a
parallel node seems saner. That value then would not be multiplied with
the chosen worker number.

I don't mind having an option to override the amount of memory that
parallel hash is allowed to used, but I'm also not yet convinced that
we have a real problem that needs solving.

As far as I understand it we currently cost a gather's startup cost
once, not per worker.

Yes - because cost is supposed to measure execution time, and the
workers start all at once, not sequentially. The startup time for the
workers as a whole doesn't get much longer as the number of workers
rises. It might be that the formula should be 1000 + 50/worker or
something rather than a constant, but I doubt it matters very much.

That startup cost effectively include redundant
work like materializing a table, building parallel oblivious hashtables,
resorting the same table for a mergejoin etc...

Right, because that work all contributes to total execution time.
Actually, when the same worker is going to be redone in multiple
workers, we should really inflate the cost, because if two workers
each sort the same table, it takes each of them longer than it would
take a single worker to sort the table for itself. That's one of the
biggest problems with parallel costing right now, from what I've seen:
we're too willing to do the same work over in each of 4-6
participants, not realizing that they're going to content for buffer
locks and I/O bandwidth.

That's fine if your goal is solely to return a single query as fast as
possible, even if doubling the resources will only give you a minimal
cost advantage. If instead your goal is to optimize both for individual
query performance and overall system throughput that's obviously not
good.

I agree.

I think we should cost the startup cost of paralell nodes more like
startup_cost * (max_parallel_workers_per_gather * parallel_resource_efficiency + 1)

which'd allow to tune query performance for using parallelism even for
tiny benefits (parallel_resource_efficiency = 0), and conversely tune it
so it only gets used if the overall loss of efficiency is small
(parallel_resource_efficiency = 0.9 or such).

No, I think that's wrong. I think you need to apply the adjustment at
the end of the (parallel) planning process, not incrementally to each
node. Otherwise, the costs assigned to the individual nodes becomes
some unholy amalgam of execution time and total resource expenditure.
I think that will cause strange and stupid plan choices. See also
notes above.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#49Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#37)
Re: [HACKERS] Parallel Hash take II

Hi,

On 2017-11-14 01:30:30 +1300, Thomas Munro wrote:

New patch attached.

(I've commit some of the preliminary work)

Looking at 0005-Add-infrastructure-for-sharing-temporary-files-betwe.patch:
- The created path/filenames seem really redundant:
base/pgsql_tmp/pgsql_tmp11160.9.sharedfileset.d/pgsql_tmp.o3of8.p0.0

Including pgsql_tmp no less than three times seems a bit absurd.

I'm quite inclined to just remove all but the first.

- There seems to be a moment where could leak temporary file
directories:
File
SharedFileSetCreate(SharedFileSet *fileset, const char *name)
{
char path[MAXPGPATH];
File file;

SharedFilePath(path, fileset, name);
file = PathNameCreateTemporaryFile(path, false);

/* If we failed, see if we need to create the directory on demand. */
if (file <= 0)
{
char tempdirpath[MAXPGPATH];
char filesetpath[MAXPGPATH];
Oid tablespace = ChooseTablespace(fileset, name);

TempTablespacePath(tempdirpath, tablespace);
SharedFileSetPath(filesetpath, fileset, tablespace);
PathNameCreateTemporaryDir(tempdirpath, filesetpath);
file = PathNameCreateTemporaryFile(path, true);
}

return file;
}

The resowner handling is done in PathNameCreateTemporaryFile(). But if
we fail after creating the directory we'll not have a resowner for
that. That's probably not too bad.

- related to the last point, I'm kinda wondering why we need sub-fileset
resowners? Given we're dealing with error paths in resowners I'm not
quite seeing the point - we're not going to want to roll back
sub-parts of of a fileset, no?

- If we want to keep these resowners, shouldn't we unregister them in
PathNameDeleteTemporaryFile?

- PathNameCreateTemporaryFile() and OpenTemporaryFile() now overlap
quite a bit. Can't we rejigger things to base the second on the first?
At the very least the comments need to work out the difference more
closely.

- It's not clear to me why it's correct to have the vfdP->fdstate & FD_TEMPORARY
handling in FileClose() be independent of the file being deleted. At
the very least there needs to be a comment explaining why we chose
that behaviour.

- I think we need to document somehwere that the temp_file_limit in a
shared file set applies independently for each participant that's
writing something. We also should discuss whether that's actually
sane behaviour.

Greetings,

Andres Freund

In reply to: Andres Freund (#49)
Re: [HACKERS] Parallel Hash take II

On Fri, Nov 17, 2017 at 1:55 PM, Andres Freund <andres@anarazel.de> wrote:

Looking at 0005-Add-infrastructure-for-sharing-temporary-files-betwe.patch:
- The created path/filenames seem really redundant:
base/pgsql_tmp/pgsql_tmp11160.9.sharedfileset.d/pgsql_tmp.o3of8.p0.0

Including pgsql_tmp no less than three times seems a bit absurd.

I'm quite inclined to just remove all but the first.

+1

- It's not clear to me why it's correct to have the vfdP->fdstate & FD_TEMPORARY
handling in FileClose() be independent of the file being deleted. At
the very least there needs to be a comment explaining why we chose
that behaviour.

Isn't that just because only one backend is supposed to delete the
file, but they all must close it and do temp_file_limit accounting?
Sorry if I missed something (my explanation seems too obvious to be
correct).

- I think we need to document somehwere that the temp_file_limit in a
shared file set applies independently for each participant that's
writing something. We also should discuss whether that's actually
sane behaviour.

This is already the documented behavior of temp_file_limit, fwiw.

Another question for Thomas: Is it okay that routines like
BufFileOpenShared() introduce new palloc()s (not repalloc()s) to
buffile.c, given that struct BufFile already contains this?:

/*
* resowner is the ResourceOwner to use for underlying temp files. (We
* don't need to remember the memory context we're using explicitly,
* because after creation we only repalloc our arrays larger.)
*/
ResourceOwner resowner;

Maybe we need to remember the original caller's memory context, too?
Either that, or the contract/comments for memory contexts need to be
revised.

--
Peter Geoghegan

#51Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#37)
Re: [HACKERS] Parallel Hash take II

Hi,

On 2017-11-14 01:30:30 +1300, Thomas Munro wrote:

New patch attached.

0002-Add-a-barrier-primitive-for-synchronizing-backends.patch
- Intro starts with:
+ *
+ * This implementation of barriers allows for static sets of participants
+ * known up front, or dynamic sets of participants which processes can join or
  that's a bit low-level, no? Should explain what they're for, not
  everyone's going to be familiar with the concept.
- The comment for BarrierInit() reads a bit weird:
+ * Initialize this barrier, setting a static number of participants that we
+ * will wait for at each computation phase.  To use a dynamic number of
+ * participants, this number should be zero, and BarrierAttach and
  Set a static number! Except maybe not?

- I'm not entirely convinced that we want the barrier debug stuff
merged, what are your feelings about that? It's like half the code,
and adds some complexity to the non debug code... If we indeed want to
keep it, it probably should be documented in config.sgml? And get an
entry in pg_config_manual.h?

- Can we add assertions ensuring nobody attaches to a static barrier?

- If I understand correctly currently the first participant to arrive at
a barrier is going to be selected, and the last wakes everyone
up. Wouldn't it be better to do have the last arrived participant be
selected? It already got a scheduler timeslice, it's memory is most
likely to be in cache etc? Given that in cases where selection plays a
role it's going to be blocking everyone else, using the process most
likely to finish first seems like a good idea. I guess the
BarrierDetach() implementation would be slightly more complex?

Regards,

Andres

#52Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#51)
1 attachment(s)
Re: [HACKERS] Parallel Hash take II

Hi Andres and Peter,

Here's a new patch set with responses to the last batch of review comments.

On Wed, Nov 15, 2017 at 10:24 AM, Andres Freund <andres@anarazel.de> wrote:

Hm. The way you access this doesn't quite seem right:
...
+ matches := regexp_matches(line, ' Batches: ([0-9]+)');
...

Why not use format json and access the output that way? Then you can be
sure you access the right part of the tree and such?

Okay, done.

1. It determines the amount of unfairness when we run out of data:
it's the maximum amount of extra data that the unlucky last worker can
finish up with after all the others have finished. I think this
effect is reduced by higher level factors: when a reader runs out of
data in one backend's file, it'll start reading another backend's
file. If it's hit the end of all backends' files and this is an outer
batch, Parallel Hash will just go and work on another batch
immediately.

Consider e.g. what happens if there's the occasional 500MB datum, and
the rest's very small...

Better ideas?

Not really. I'm more than a bit suspicous of this solution, but I don't
really have a great suggestion otherwise. One way to combat extreme
size skew would be to put very large datums into different files.

Right. I considered opening a separate file for each chunk size (4
page, 8 page, 16 page, ...). Then each file has uniform chunk size,
so you're not stuck with a large chunk size after one monster tuple
comes down the pipe. I didn't propose that because it leads to even
more file descriptors being used. I'd like to move towards fewer file
descriptors, because it's a cap on the number of partitions you can
reasonably use. Perhaps in future we could develop some kind of
general purpose file space manager that would let us allocate extents
within a file, and then SharedTuplestore could allocate extents for
each chunk size. Not today though.

But I think we probably can go with your approach for now, ignoring my
failure prone spidey senses ;)

Cool.

This looks more like the job of an lwlock rather than a spinlock.

... assembler ...

That should be OK, right?

It's not too bad. Personally I'm of the opinion though that pretty much
no new spinlocks should be added - their worst case performance
characteristics are bad enough for that to be only worth the
experimentation in case swhere each cycle really matters and where
contention is unlikely.

Changed to LWLock.

One day we're going to need a better approach to this. I have no idea
how, but this per-node, and now per_node * max_parallelism, approach has
only implementation simplicity as its benefit.

I agree, and I am interested in that subject. In the meantime, I
think it'd be pretty unfair if parallel-oblivious hash join and
sort-merge join and every other parallel plan get to use work_mem * p
(and in some cases waste it with duplicate data), but Parallel Hash
isn't allowed to do the same (and put it to good use).

I'm not sure I care about fairness between pieces of code ;)

I'll leave that discussion to run in a separate subthread...

BTW this is not per-tuple code -- it runs once at the end of hashing.
Not sure what you're looking for here.

It was more a general statement about all the branches in nodeHashjoin,
than about these specific branches. Should've made that clearer. There's
definitely branches in very common parts:

...

I don't think you should do so now, but I think a reasonable approach
here would be to move the HJ_BUILD_HASHTABLE code into a separate
function (it really can't be hot). Then have specialized ExecHashJoin()
versions for parallel/non-parallel and potentially for outer/inner/anti.

Okay, here's my proposal for how to get new branches out of the
per-tuple path. I have separated ExecHashJoin() and
ExecParallelHashJoin() functions. They call an inline function
ExecHashJoinImpl() with a constant parameter, so that I effectively
instantiate two variants with the unwanted branches removed by
constant folding. Then the appropriate variant is installed as the
ExecProcNode function pointer.

Just "inline" wasn't enough though. I defined
pg_attribute_always_inline to force that on GCC/Clang et al and MSVC.

I also created a separate function ExecParallelScanHashBucket(). I
guess I could have extended the above trick into ExecScanHashBucket()
and further too, but it's called from a different translation unit,
and I also don't want to get too carried away with this technique. I
chose to supply different functions.

So -- that's introducing a couple of new techniques into the tree.
Treating ExecProcNode as a configuration point for a single node type,
and the function instantiation trick. Thoughts?

If we don't split this into two versions, we at least should store
hashNode->parallel_state in a local var, so the compiler doesn't have to
pull that out of memory after every external function call (of which
there are a lot). In common cases it'll end up in a callee saved
registers, and most of the called functions won't be too register
starved (on x86-64).

Hmm. Well I did that already in v24 -- in many places there is now a
local variable called pstate.

See above piece of code, and a few others, in nodeHash.

I tackled some more of these.

I think it'd be better if we structured the file so we just sat guc's
with SET LOCAL inside a transaction.

I wrapped the whole region of join.sql concerned with hash joins in a
transaction that rolls back, so I don't have to write LOCAL. That's
just as good, right?

Not really imo. Being able to read a test without going through all
previous ones is a lot better.

Added savepoint/rollback to savepoint around each individual test.
You still need to do the setup at the top of the section (create
tables etc, set a couple of gucs).

On Sat, Nov 18, 2017 at 10:55 AM, Andres Freund <andres@anarazel.de> wrote:

(I've commit some of the preliminary work)

Thanks!

Looking at 0005-Add-infrastructure-for-sharing-temporary-files-betwe.patch:

[For things quoted and commented on by Peter, see further down]

- There seems to be a moment where could leak temporary file
directories:
[...]
PathNameCreateTemporaryDir(tempdirpath, filesetpath);
file = PathNameCreateTemporaryFile(path, true);
[...]

The resowner handling is done in PathNameCreateTemporaryFile(). But if
we fail after creating the directory we'll not have a resowner for
that. That's probably not too bad.

It doesn't matter because the resowner handling in
PathNameCreateTemporaryFile() is only concerned with closing file
handles, not deleting stuff. In this world cleanup is done by
SharedFileSet via DSM detach hooks, and that directory is already
covered.

- related to the last point, I'm kinda wondering why we need sub-fileset
resowners? Given we're dealing with error paths in resowners I'm not
quite seeing the point - we're not going to want to roll back
sub-parts of of a fileset, no?

It's just to make sure the individual file handles don't leak. Even
though this system takes care of deleting all the files and
directories, it's still a programming error to forget to close the
BufFile objects, and we want the resowner machinery to complain about
leaked File objects if you forget (unless error path, in which case it
just silently closes them). That's not a change.

- If we want to keep these resowners, shouldn't we unregister them in
PathNameDeleteTemporaryFile?

Unlinking the file is entirely separate from having it open. See above.

- PathNameCreateTemporaryFile() and OpenTemporaryFile() now overlap
quite a bit. Can't we rejigger things to base the second on the first?
At the very least the comments need to work out the difference more
closely.

Ok, I have refactored this. There is now a static function
RegisterTemporaryFile() which contains the common part called by
OpenTemporaryFile(), PathNameCreateTemporaryFile() and
PathNameOpenTemporaryFile(). That ensures the file handle is closed
automatically via *two* mechanisms: resowner and AtEOXact_Files.

Those three functions put different flags into fdstate though:

OpenTemporaryFile(): FD_DELETE_AT_CLOSE | FD_TEMP_FILE_LIMIT
PathNameCreateTemporaryFile(): FD_TEMP_FILE_LIMIT
PathNameOpenTemporaryFile(): 0

Explanation: The file handles get closed automatically in all three
cases (unless disabled with interXact). Only the anonymous private
temporary files get deleted at close. Named (shared) temporary files
are the caller's problem, and in this case SharedFileSet will unlink
them when it deletes the tree they live it. Only files you created
count against your temporary file limit. If you open a file someone
else created it we don't double-count it.

On Sat, Nov 18, 2017 at 11:22 AM, Peter Geoghegan <pg@bowt.ie> wrote:

On Fri, Nov 17, 2017 at 1:55 PM, Andres Freund <andres@anarazel.de> wrote:

Looking at 0005-Add-infrastructure-for-sharing-temporary-files-betwe.patch:
- The created path/filenames seem really redundant:
base/pgsql_tmp/pgsql_tmp11160.9.sharedfileset.d/pgsql_tmp.o3of8.p0.0

Including pgsql_tmp no less than three times seems a bit absurd.

I'm quite inclined to just remove all but the first.

+1

Yeah. That was because I wanted to apply the same function
recursively, and it only deletes things beginning with the prefix. In
this version I pass down a flag to say that it should delete
everything after the first level. That means that I removed the last
one from your example but kept the first two. I had only added the
third. The first two are prior art.

Now it looks like this:

base/pgsql_tmp/pgsql_tmp84528.0.sharedfileset/o999of1024.p1.0

If you create stuff in there that doesn't start with pgsql_tmp then at
startup if refuses to delete it. Someone might value that ancient
feature, so it should probably be discussed somewhere more visible
than this and handled in a different commit if you want to change
that, no? Example:

2017-11-22 19:23:31.055 NZDT [84606] LOG: unexpected file found in
temporary-files directory: "base/pgsql_tmp/random_name"

- It's not clear to me why it's correct to have the vfdP->fdstate & FD_TEMPORARY
handling in FileClose() be independent of the file being deleted. At
the very least there needs to be a comment explaining why we chose
that behaviour.

Isn't that just because only one backend is supposed to delete the
file, but they all must close it and do temp_file_limit accounting?
Sorry if I missed something (my explanation seems too obvious to be
correct).

Stepping back: the goal of this exercise is to split up or replace
different behaviours associated with temporary files so they could be
enabled individually. Deleting files: now done at DSM segment
destruction. Closing file handles: done in every backend. Tracking
temp file space usage: only done in backends that created a file
(private or shared), but not in backends that merely opened a file
created by someone else. Does this make sense?

- I think we need to document somehwere that the temp_file_limit in a
shared file set applies independently for each participant that's
writing something. We also should discuss whether that's actually
sane behaviour.

This is already the documented behavior of temp_file_limit, fwiw.

We could definitely come up with something better. My proposal is
that the each backend gets to create temp_file_limit worth of
temporary stuff and hold a file descriptor open. When the file
descriptor is closed, it stops counting against the creator's file
limit, but it still exists. It exists until someone deletes it, which
happens on rescan or end of join in my current patch. It needs to
keep existing so that other people can open it. A better system would
be one that applies temp_file_limit to your *session*, that is, your
leader process + any workers it might create. That would require a
lot more book-keeping and communication than I have: writing to a file
would have to consume from a shared-memory counter or something. Do
you think we need to do that?

Another question for Thomas: Is it okay that routines like
BufFileOpenShared() introduce new palloc()s (not repalloc()s) to
buffile.c, given that struct BufFile already contains this?:

/*
* resowner is the ResourceOwner to use for underlying temp files. (We
* don't need to remember the memory context we're using explicitly,
* because after creation we only repalloc our arrays larger.)
*/
ResourceOwner resowner;

Maybe we need to remember the original caller's memory context, too?
Either that, or the contract/comments for memory contexts need to be
revised.

BufFileCreateShared (new), BufFileOpenShared (new), BufFileCreateTemp
(old) are all constructors of BufFile objects. They all use palloc
and thus capture the caller's CurrentMemoryContext implicitly. I
continue that tradition in the new functions. The new kinds of
BufFile don't ever allocate any more memory, but if they ever did
they'd need to either use repalloc or start capturing the memory
context during construction explicitly.

On Sat, Nov 18, 2017 at 12:20 PM, Andres Freund <andres@anarazel.de> wrote:

0002-Add-a-barrier-primitive-for-synchronizing-backends.patch
- Intro starts with:
+ *
+ * This implementation of barriers allows for static sets of participants
+ * known up front, or dynamic sets of participants which processes can join or
that's a bit low-level, no? Should explain what they're for, not
everyone's going to be familiar with the concept.

I have added a one sentence description from the Wikipedia article
(with attribution). We reference Wikipedia in a couple of other
places.

- The comment for BarrierInit() reads a bit weird:
+ * Initialize this barrier, setting a static number of participants that we
+ * will wait for at each computation phase.  To use a dynamic number of
+ * participants, this number should be zero, and BarrierAttach and
Set a static number! Except maybe not?

Fixed.

- I'm not entirely convinced that we want the barrier debug stuff
merged, what are your feelings about that? It's like half the code,
and adds some complexity to the non debug code... If we indeed want to
keep it, it probably should be documented in config.sgml? And get an
entry in pg_config_manual.h?

I found it useful during development, but I've ripped it out of this version.

- Can we add assertions ensuring nobody attaches to a static barrier?

Done.

- If I understand correctly currently the first participant to arrive at
a barrier is going to be selected, and the last wakes everyone
up. Wouldn't it be better to do have the last arrived participant be
selected? It already got a scheduler timeslice, it's memory is most
likely to be in cache etc? Given that in cases where selection plays a
role it's going to be blocking everyone else, using the process most
likely to finish first seems like a good idea. I guess the
BarrierDetach() implementation would be slightly more complex?

Good idea. Now done that way. BarrierDetach doesn't get more
complicated, but BarrierArriveAndWait() needs to be prepared to be
elected when it's woken up, just in case the barrier only advanced
because of someone detaching. I changed to a model where instead of
setting and clearing a flag, I record the phase number when someone is
elected. If that doesn't match the current phase then no one has been
elected yet. Also, I standardised on 'elect' as the word for this.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

parallel-hash-v26.patchset.tgzapplication/x-gzip; name=parallel-hash-v26.patchset.tgzDownload
#53Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Thomas Munro (#52)
1 attachment(s)
Re: [HACKERS] Parallel Hash take II

On Thu, Nov 23, 2017 at 12:36 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

Here's a new patch set with responses to the last batch of review comments.

Rebased on top of the recent SGML->XML change. Also:

1. The final patch in the v26 patchset extended EXPLAIN ANALYZE
output to show per-worker information. I'm withdrawing that patch for
now. If you want to see how many tuples each backend hashed you can
already do that with (ANALYZE, VERBOSE). It's a pre-existing bug that
you don't get batch/bucket/size info when Hash Join doesn't run in the
leader, and it's a pre-existing bug that EXPLAIN doesn't show
information for the leader separately. I decided that it's not this
patchset's job to fix that stuff, and it's not entirely clear what the
best approach is anyway. Let's discuss the way that information is
captured and displayed separately from the Parallel Hash feature.

2. I found a way to crash v26 by starting a worker very late. Fixed.

Unfortunately I saw a one-off case of an assertion failure in
ExecParallelHashRepartitionRest()/sts_begin_parallel_scan() on Travis
CI that I can't explain. I haven't been able to reproduce it there or
on any other machine since. I am still looking into it.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

parallel-hash-v27.patchset.tgzapplication/x-gzip; name=parallel-hash-v27.patchset.tgzDownload
���Z��y{��8|�?�_SIq���e�Vj[�$_����g���I3CJf����=�f!)yi�Zm,���rpp�3�"o4����_Wo�;��7\�r��?���^��t:�~w�w�����G4Z�zw���i�E��m���#:�lK~����`(��$
���f����j����gZ��0�F����q8��H^E2��pR��xWg���*?�h����#&�A� �w�n�]��x����%���N��?��%��|��G�X�;�N}���V���<�;���e����6Z�a_����xN������������	hRB0���:{�x�K)���_5Z�?��LF�P�����I��7�����UD�+�W	�6����~�-��t1��Cf����������������"AJAH))�FJ�H))� �e�T:���W���Xx��(���  ���{���y.�@�+��/����88�|##l ��<~0�f+a����b�M`���Z������p�=J�,�6W�A�x
3
&W�6����r�#���#wiK��&����>]����]��H�'~�Z��D
�q��j}��-�m�r-���/���[�����1(� �����r��DS�qp�M��_;������v..om�J�n�Z�
f��^c��5���/?�a.��~c�n�j�k����z�)D�p��zg	&��{��gQ���v*�"�������P�3@T-�r�DC���U�FKD�m�Y%�G�T���H����l,�g�;&�A�s����U0�N@��8��4���^(���J�X�U3���O{�0��������0��|��z8$H����	L��hk1�`$��-<�5�	q2!(Wr"�%�L<��y���������o���+`��0����xS�������yDxb�������v�E�
4���CB�AP��0�����SQ��]����W���Zik�v&O��N�|2��NO��}y�.�����V��L�O���{W��+8*�-/.m����-_F@xJ[B|���qp�|�'m!~������7����T>��y�^�Q��	����N�b�OpN��_����[���1�����yx�nr�QN��u��5q�y���@?� ���d;����^��C�'���:p�`x.�-|8�5<X�98������2�r$�f�.���
`�%�	�FC�qs�
���H!�';I�7xa�&n��#P�z0��<�'���P{�u#>DPn����_�nZ`�d��:�D-�!�]���W���C�w����J��E�;���S^����0�K^K��0(��j������������t��}�����k8��a>��(��4��\�q�Q>Z0��g�4@���i�{�U���djd���r��LN��)/D��Co�N���5�����l(-��2��,7q�5����
$S�cI3A�Et�}U�4������@b�2E�-�������aN<���%d����n�^�~pu%�pX�^0��~���]7�\��n�o�U����������`[i����psm(��"�'<�O}\��U�@��`���DF�1�4_�����M�0��u���X�0�81�0�O���{8��Z���-7���L�1%:��e'q�A�"��6��ID&$�)�	Lc
�x�h��Q�������qj�2�����V�s�<bf[X�n���
@f���d��_�"X��N�n�����b����kc[�6��|c�3����6�A8E�= ~i?�8y���\W �������5����!�U��/�W�������/�
a��T��/\@��W��#����3�+9��'���7Y�FN������,k&���K�@o
���C���`�X������NF���p�i!���G�����b�\�}��=@��]2?}{|�7q���M�Np��yx���Y������a`�`��Q8��E9�x�b��i5���gq�<qh��� ��T
�
��J���e���Q�d]�:4���m���y"��I0������,Ph9TMLkj�$,��f������
~.���4����P�J!N�}
p�"��1tV,
G���g�:7p���aBo��a��<N���:{���[u�~�{�o��}A+`������!�H8����;�"D����;���d 
�L��
��7��e7�
h��W�����5\���Y=��1���������L{4A�^~���W���B���dY�����p�����|�W���L#fG#��|��:}���cv�r_9��"�A�E�3��?�p��;'��	R��S�5am��Pg@L�\���X {a�g1�3�j����C;n0���r-��k(s�o��x~~�:���OO�(P����7/>������V9���Z���w�7������9��
�`���h�;�w�S;�����SH������"�t�	��iqD���B�Dj@8T�d�&��������Z���[f�|-N�Vd�	Bn�:�J������g��Uz�b��90T�d ��=F���D�s�w���#d���M�QxK@X�4�����!������^��~{�X�f
����rj}*�[����sg����{�	����(e�/����j�U�4��4�+��������Gg��;7S�������7Y���u&w����>���""��8�C�|B�!�1gj�M�+���)8C����FC����
:MFrF>zv�i���pO�la6�h7���[F���~����\�u��[$���'<
�8@G��[����������h��5ib����������*��o��f`0�h=��?��O�~E+�E��7�����#�_���sb�r����*���m���V��j�j{����k�����u���:�
��5�Zm�����Nw�x:�!w�Fm>	~����N4����a���}�(��qv_��4��g�5�����}��s��k��r���h���v��h��G���6��c;��\��o��n+�����[�����]x_|E��*}���_����������=��?-�����.�Z�e�����Q���vW�������s��GE�����@|9'��\b���[�����}������8 ���� ��/���t��=`b���'�_����EMe`yK�Z�S��J�����gy��&����}/�U�Q0��FV������X�I�OX�*.���q:������V��M�����_���������f��{�f�-;����o���^�S�����������y=� �$�c���o���o����PH)R���FJa��T:�����0�N�hF����tmA�,|����Drp���Q����[/�.pZ�'*�?�����%�4�� 1s���;�?��`�E�<�b,15>Rr.��]2�����P(���n��N*���y�����>l���3Z�Y����^��w����,��~�E���8��2Z��@�Y�Tya_\�f�x{���UT���x���XN�����N���8����������^���?Dk���������~���y������2P��
�m|�v0l��
��i��%m�����h�W�@�=az4����a]�M�nw�n�J�9huv�:������.W��,�ke����u:����K�V���������Y���������=s��#���#�������%��i�?F>������KB���c�n�I����]���7*�R�������=��k!~��=�o�Fx%���8�&��3L^y���J[
����c�
��<�5�����s9���7�Y}���
�G�������t@�������7��)M�-��#�RH��di���r��w����#����6��@��M�+�J�{����\��~��;�:����?�����)��\�0��o~6��f ���g�(�.X?Vl���^��VZ ��}%^Z�g ���)
��E"�
H��8�W��/�����8�
�y�>	<3,����������O����$����qN�������T�5�$+�~Pc	t��	qh�������)���0�nAr�|�7W8��K}�'�k[�c�^��������Or)�& �<j.}l0����"j���&Hz� K@��E����IJ]�]��	�/���
��7A�>�~�N� �Se1y}@�
�30�D�gQ����$H`���%
�M��9
u���x��4��K��I
�K�D�"�y�����FWa���meco�0&����n�����
��4�Q�4=A��*a@�x?��@Nx���bK��Z���g�AL�# r�48��*�I�����{jM���G��0�+o@(0J%
q�F�(���B��@{����e
��!���	W'&�1�����sy�	�&��?S������������z6t&t�(	��Q�����pt#�c�������aE<%�8R:as�qit���SI�����D~P�A�C��I2^���A
mb������vV�BC�`j'H���V����bM���y8��!�zS�MD�����/��F��U��D�����V������G�����0�����"�K��O���0";q����%���u�r�����q~����(8#6�-���<X&W 
5���w�"���z�*���o`i�!�
�{,!~���%����!%�K��'����T$���GA?�X<g�:h+����3M;����	��"�@��[853$]x���o�A����W&�}�2����T���Y�5��GC��a��������/��t}!)�L�
%`i:!y�N�
���BAIrBL��7��@t�S�I�'ph��)}���H�<��{@�A�Q
Wf3*�����=����:2p��H�E��B�D����MM�cy`O���dN��.2���~������7�Ij���������2����xV>�V��V�Vi�]�3�Rl��zbr��$�RN�)/D�
�{|$9�Z�z������g���8��)!E5��(�`����Dr
����j�y T�$o�-0�Bc�r���L0�g�	�"���,2��O�������|�H���+l2�;kM�!I:����I�C�������$�����j��o���2u�4_��|�+�����q�p�|[H��L�,�O���?��|�����k��>�9kM�.�OqJ��q!���X����=�.�1����| ����_5�Pa6Q'���xzP ���n��X��m�k[��Wf�>� CF�;���<y~rtxyr�FI���}���
��(��?�&z�ZG`���_gv$�3LF��@G���8�e��xd�z�-�d+��L�fZ]��z�t#as�6�@���D;�q����*I�3���tW�C��q�����[(�����s����4D��.��&x��7��:��g��r�!��N�5��u�WAL������Zh
�hQS(p�>�!g�P�`�t��q1
&���{��Oq��0�n�����c���Q�W�f��"���d��i����}p�&�p[�n�����a@Pj��w���~�P���(^�Z9�d�
�FK�.�!a���?$�"u���G�5b��8������F�$�X��U������1`4oQ��-z���T� ��K�H��>b�|��^�d�4�������>�7�h.��L��N��7
@�~�x��8|6������v����Csg�������,��m��$,��C�x��S��V��U���|�(�����W�l*����J�S���]B)y���?q�F�.�>�H:�9u/������m5���-B����H���u80���/�����=�M�%**�w-8P�L.9�%&u��ymV�-L5�����&4��(g	�
��\���uN!��!g�y�m���@�����VH\6X��Pi�@�������:n%���3���J���I���B�t�I}���.W��P�����P�R=�V���`;Cm�F�������^�a���V���m��������Y��5���h���	%�jJf>gj�ex1�r�rR8����Y ����$�������Q����E%��`����#�a8	�GU�A�lo�X
��d�1R��&�9NT�K���8L^�h8�]�'��E2�O���)(�y��u������uQ"���7�����y����������!��)���K�~�Ed${��Z��-5�N���[��<p�c�/�`8�����7����}1�#��iu;i6S����c)Q9�~��?g�1PHYi)�,z�����?�8:{��{�^�l���-�@VP[j[�$ivP��W����J��`,�������
�NZ��B5k=���{ {�mX�\=��X����H�Wx)&K8�����Q%�"��*�Wt� aV��0���hz�m���gESw+�n�����������f0��\���|����4���3%���B�K\=4���e����[WS���"8�g�bNRJ(��"�0av���Z��zW-X'f�,�OQ�?�
^����_�������)�������F*�c� �E�H������W+D�0-�gXE�|i�m��9����'@56
�E�����������q����29r�+����mEC51����(WA?}Wj^`8���S�6���z��-'�1R:r��M���:����X��2sc����CK����J����!)>i:,nS�L�s��,������O�)����Q`�R[���,e�P:�������W4��<��d�$�#g����!�)_�L��H�4�n�������!3K%Y�"�&6
��&!�y���&a�����F�l{���L;X��9
���I��:� -���+ M����iQbO�\��U������E����~�+)���t���Z-�AW�GY����W]�W��Z3,��|������l����T�[��RDG9P{��!�6[g��Y�"YI����^4Y�-�R[cs���K�)��[Z��Q�����4��q�E����uZ-v��/�������fo��S��Z��Q����~����~���3��_0t�����'�����k������\�����I$E�����0pQ<H��1����E `p��7�� �FlA�%�Uv;P+�� �i,����x4�����<�f���?c8�������,1��L��P1��=`r�$$�r&E���z.���$4�a�%\�����Z+���/)7PWJ�ANPR��	�5�0z�����Z�5l��4|2�wA�$o2u�F���yz�� ���*�j��_b���������6W����X���t��]!u���N�$���K�wZ���j0T�k�n�Q�g�������D�y�B?�=L�R��1P>/ZT)�����[��]��h���T�g�����E~(��������n�����������Fswg��m�7���-��������^A�gK��H��#�0H���)��dG�^\�k�sx~!g:$��u��a��P����?#)C~����R�T9���Z��/�0;0&bR���w�HA���%KV�����\�xv�Z����}(��������EO�6���!��
��R���E��$1vg�N�8����!��W������<����b����/=�|>���_�WHp�����N��0���<��Y��V����/����w�?�^��~v�;�D��~��f��?'���^��x������_�����O'����������oO_<���];@���P:VtI�*uI�Jwi�w3��� ��[?���nwI�+s���b:&t��h�@����U�v�$��(�p��[������t���G�����m���r�-����nr~���FMi6�v� ��d�6��^]��;~gul��8��mG�S��.�Jwe|.�������E!��z�P)�~������l)������`(fHz�pb���6~���&�q�*�FZ���C������i��f���������M���������
����o7V�*��s��D%?P�%�	��Z�T�#��@��x�M%�/P�z��I�4WRLejb��G���P��	���x��sM���*�DW0�1W�����f^�T��:6>�Z�8��n�����1n��0q�O_�-�aRL���#Kv1G�=������f>��`,�X�	��Q
Fj�V�(�>��
����d;x�a���hNL�������cVZ� ����}k�>0�:(;u��;;�F��R�0"�H�F �O�_���H!@�HxT�Q�Mb�*�����~@tP�@B&f�$2FqK�C����V@ ^�Z)}�������l*�����~Rj�
	)�HC��T��3�
�0@��0�)��f����S��PYM�Aq��bm?�(�1o'��#?�~J_����B�5F����������k�=U��Dl�!�	
�_
���DS!�i��
=l��V�g��tEs�:���#�h3
����������=��Mp{��G��&U%+	XT��;|g�U��S	/={�0�xF��M���'�yt:"�<�W�{8�lf�_L���������Hh;��V����W���U=��A)��)|�+����� ��^��=,�H��|�����
�6����Q�i0.���S:D��)��KM�����:���)75i� �D�
M�nt�\�`DP+:?�k�VB[�����d������~�Cg��K(������t�s�
�%�����*/E=j
C���2�r��3�7���T�'I��?���ZO� ���en�����g��/+��7q��AE� �����yI8s�C/�:W�{L��'F�����F�7����G3�������FB����y)�|&�in(�s�
�o��R�^��u����X��������9�.G8����?�V�e����6���}���A�]���\�v���91��������X3�pq��t�j�oD��r���Im�p%l�%����Q=%�A��H)�]���l��U���D.�t*=�6G)`'Tto����J���dX��*����O��
 b��h���.���Z��x�^�8Ui$�
,vv��o`P��Z����%�h�����+ob����#�.A�kH��8KP��(X��I����C�9j�#cw*�sK.�!��C����������M���(�Qf
����f��9��:�}����9�}�U�R�p��}�����_@u�GHL�>�f�w����.����$Xwb�� ~9���'�x����y��Qw�t���4�u/.�`��2d��


C�#����;�����X,��o8M�I�z^���w�dD�/z=#x�$���G��mG�0��/{L:p��b��<��y>�M�B�@�d�j��G9{�^|�q�?�_�X�}�n�^��}�66aax�k�s�:�(��\J��6��5��~_2���,�wR�c��1�g�����O�x��1�*�
���{����N$X�x"q!I��*��S�����M)���d{*���F[�!�$�J�I�v<��fa0������e�Q0�KJ�%F g"�m2*'������;�������Oi�
��l���W��f�Zlm�S�v����/������7����=��(�P�R:�O���]DD����$�6	
���r��2����O#��"�J���	�R�a���K���b8�d>���(�v��%U��Eau>��F>���G�zTEI>����,��s�-�a���Yz->CKK�������.��H�����.����T4��z�f�� �3�`��v����������Q�Y�����%���	B���
�t��F;1&A�&3���m�*Pd��g� <k%T�1n�4]���hu^='�V[�)���&Be_P�3z�����7���V4_�L ��0�O�����]|��\�8/^^��e+ 
C$x`�I ~�|BW���������m�<*I��_WaX����h��0*���9l�����=��1T`X�r��i\���T�5�AY�0P�9��s�+9+���Da�
��T$�Z��!*cUZ�����B�"�~�y�,/�\L�!�U�*�G���AZA?
9:�B��MP��R����J���H���{}$(�?������sJC�G�[&u`��%�
����Y����}Fq�	������G��^��k���M��m�r��t1;70������'0�:?�RN��q�_����spK7:���W���,M.^�O�����r���5��d�\b������y��l��z�V�6��������6_����Kmx�q���F[zY�HH!YN�nJ�������<D�,�b�~0��`a'�<FgT�O�����c�u&���t�:q�������p�����d_a��i4��^��,�6I3���0Q�QFY�J_d�/�~P��e������zV[�w�!F���<4T�8��r���l[���:b��|B�b�d1�����Y��9�j�K��D�(���[���	��:�Z��)�[��m$��<&�K���L�9S�	��u
h#���t���s��5L�9��p�T!g�T^�`i8a'�Q���)�rx�C�6Bm(j-#������[�bR����S������Bk�X.BHTv,t5S�`F||���H(N��~��,����ko��N�S���4�,
V�����
}d�[Mb�}9���Rr���PuH��,�O��|%��`m�2���H�eE�v��1������"<��
��t�O���
�z����N�����Qn��,���:�/d
U' �����Kh��'IH�`$���:�<�^��^�^�9�C��8���h��{~����������4������!���f3w*l�f������Mm��f��W������.,8�$�_���\<��L�<1|�X�`A}i�mK���,
��Y���r�R�V�3o6�S]���G|���r���������8��gW���_��:O�h-���;�@I$���= w������}�sp��`�����{������h�Te����GX�+����\���V�K7u+������S�2����������
��b�%m`����CS4�
�	��i�k�����}��UX���U����$�E��b�����y�@��u����f�I�����+-���<����=0�eA):�)������a�,���A��w��!�l�����Z�����z�]gQI���9n]�����!$�A�lE����3d0f�-�R�,1����6������2����_����P�2������]�}L�}��>��7���
c�p6��D]w��n��6�����y������'��b�B�V�u\:��7"S�<���e$!}	[J��LR:�+Q��
������^%��:o�F���2ox�TBk��C�Y����j��k,�>Non��C�"����F��\�JM�H�$�<���q`�v=��3[�����X�a)T�PAb��R�����Lb*)k��j����I;�t���F�����dL�+������C���+�1�������=i�{��~����C���_���9����,E�S�FX�@��U@G,W\�����"D�(���*���|��C9�HU�L���V��7�x�U�G�,D�����cTI���pu1z?%R��P����
:o��1]����E����osm7�Q �>~,��8��db���T`&��s�����Ym�;�Q��F��8�la5q���B'v�w%��'"��������9��6�*8&g��.��Y.{����'�w���������m�
�M����n!4�W������U�5h�2�� ��[V6Z�ZfNK���M��'��"A���X?�����h���P����|�aP�GD���Y�����O>4����*f0K��5b-��a�����U�6�,��S������e�su�G:�OEs��3�z�������$m�&;���Mr�1`�k��m���7��?���������g���C��E�h����8�\�3�G���RM��s��Es2�-����{��*��&%��>�F�^=�H��N��%�~�3�*1VEK����*'���U��A�|�����*P�>�L��f������jT:t�f�d�����xv�����������/���{�'�*n��={Q�E���J����-���c9?b4J���%r�(��Y��IiA�/9���V/4?���������>�? ����U����G��uJ��+YE�rO��^s�r����m�6����@���b�=3�yi��<7uv��:����w U�%��y?
����7���T��jk��d�#AH��)8"�hl���N�l��@�f�~2�4=�� ����x:�OJ��$�P�IrS_/!�vF"�%Z�x�aUe�������c������4��jW��\r�c�pa��$Y���Y�
��'&�Y�k��n0��q�b�"T�O���KT�����,^n��J�������������v�|n�+�~m2����46�n#_���qy����+Y�mV0iM,����5��<{�������M}hhz��a�',N�$`Rp�����������5����o�;d�����Mf]��Z�w�����&�#����c%5xS������E��5��=>�89}�#>>�<=�[�T�/C�[���f�+^���������k����$�����^�CI�ZE��!9��A3��'(���K�����,E�URZ��������Xu��m�*�f7����2��&�Ca�����\,���JJ*0G���x/s�
�W���� '����������
�Ig�����d�R*�1�R���r<$d��:���\#��{-I"����4����C�!)���1v2�B�rF�?#~�����j�����z���d��i���F����h&V�1e��~�<��p��bT�e��F�k�]c`&����������3T@��iM�"����g���u>��^�����o�����'o���`��<�����tD��P(��VJ,���4���x�96�1fElF�Po���<9�^�_�#B��s��4��q��Q��t*fB��'�+*r��WRq���mp����FT�=��
9��J9D*M�j�!+C����=��8�����X����y���qT.n��5�.8�i�����d�q2���W&��������]������[{�O,B��iZq���_�����_�I��9)����5�n���+`�~7+{�����}$�>���/�&ik
���35�	�����h�S�K�g4�<�o�;X1��y	`�~�!����{�e�-�7�F>�=�X �7<�c��i���1� �Ty�Ii����q���C�6T�0ke����*�$�����SsL�bG��%�khm_jl��DZ�"s�{�QL������)�E�,|7}<����kjz����"�(�vvF.�[�x��t�G�b��h��u�a�>T��!g����C4r�{���M�H�������G�H;��J�j�t������ @�	b��8W��P+|3���O4�m����pgO����sNv���llX���EVSJB�&Q},�`�;���]q��1�^F�Sd��P��Py��J���������!4�`h[�T�&���,^���x;PW����W���.�����m�+w��
*���v���y�$���(���'����	k����k��$����Q�r�Y!�NZ&Q,�R���&�MjQ��/���o�Z96"|*|���o�||J��/A�!�.^h���On��?�"Xw�����n��.�;{��%�V���}��~5����,N
����
�d���J����C��.G�����Kh��>����ho-��$�����&iY1l��bG�����������O>���
�)���;{;�8V�`A�J���Os���r�D��M���8�KI���RT�O6cba�[>:'���L�|�yMk�u]�y�S���F>%�=Q2�r����]{�����g�uEo�B�����
�z��7��'���b_QA�[b=�*�/��|��C	+
�t�e�(Hx��[`mJ��5���l������-���=�������M0�XR�����9Vw�qu
�g����c�Ui5���OH�MH�|B@�%���"��D���R��
���b�
�)���g�����.�@��XUS��E�[�8W���B�rs���41�H���0���2�-eqy2��la�)�l�6��R��������OL��'�LM	������������S�����i@*HF��N"`W��N�T��f��M�~�����|�Wv3����/��r�\=�/��V�iLl���&��]�KG����a���xs4�tl��{�3�+�a�C�6�%4�[�����0u��*����x9�J�Q	By)��5~�)��M]��:O�/z�,ll�,	5U��c9��\��S+�&A��S��Uo�0�\k����-e�n$�_�8�����p�Ax]��W�_���#���9y��x���%��`|��S
�UKc
<�L��+������r����C��OMr%vc�/�������C��o*�B�f�^�(������0	�R�.o�4��04��������iog������-�tlL����%�m������n
'_�5V�.������T�Hs8�=���7:�Fg���`�F����i�1A����Qo�����F����Y���	���,����8 �<��U�(���p.����^r�*�����L�qD7��������8 C%��w:<f3�be��
����+�a�m���(����|��`U��w���H��
�Cg+���Ay�}�1E��a��H*t�m�"�Uo�U����|��9���)RoR�M���SQ��H�q*�B4���T���P��d���|����wv~����z���������P�o=3
���
kj'iT�� �T���#��8��?�
C{"gqJ-�����
t���T?f��k��������K�6�v�;�l)���F�X���G\DN��til����������Y�%����!�r����|"?L�21��$��q\a(9��&W����P��V���4[������S1�:V����T��C�������L�X��c}����i��"��(�ziA����c��W�H.�'Z1���(�2�/3B���v��tR[(�U�CO�
$�&Eb�zYB�	���H5]��e�S��w�:��WX�r�Z-�����h�~-����M��>(k�o����38N'�O�/��S�Z{�5Z2�^`j:���Kq?�;[F��}�����0�����}���$1H��|@�b��$�G7�
�M�&�<�Q*��g��i����H8�Rmc��T���|�[���O<USm�1�3���-�"9E5�x\Tn�-F����r��X0�i=);����8Q%�e=����N)�>��o:6����/J����/��6���$���:��r\��w�f�N��W�gU��M����sj��WA�6=N�/��+��%��YK'(Z�:n,;��$i�!l-��N���r*��k	G;���2�t8�^)s>�K�������\���IJ������9GT'rV��[��8��O6JJ5e'/�$,��B��B�!
��8���-);��V����Hg=��j�O�qO��"L�[���g�o$����
����6������)F�������i������W����?n����F�����>���Jy��\���/H��#QzJ��*��svb��(S�Lk��)Nldy��Y��<Y$������o�� �]��Ax�k�������v�-x�(�R9p��`y�~�8���H����Q�Z8�!N���T��|:����z��S�L�9<4$����e����8�9K���{�#��%����'�p��F�{8�M��x�14;��-5lm�z��t��_K9U��cQ�����u�<:}X���������������������q�����x���gmfN�.t�Q���.�������P0���E�t������+�9)V�r�s*���/s(/����I�V� �
�-����{�c�d�y-B�*6I5Ps�k���`���2K�WF19���F���Fe-w������,hI�����(�VyF�6�����a��OY�[x�� �k�t�\��ub5T��M7�y��y��y^*�u�U������|�UNo�v�{gQ����;����\���p4*Xg����}C����tM����k���A �<S��8C�#�n�kC�s��'%F��{(�]���W�/�����47q��A��������4E�$`lv2���o9�Wt�pN��#u_[�KLz�>��7��D����,qN�2�$��u8��� �����q�b�DS\W&I*J�$�0�@�93��Fd$p��Q,�N6B'}
KM�����N���b�bb����G�qeI=���Ml�6k�<(��Dy,�d�_�8������&[I�mN��*��E�Z�GgL�ks�n��"L���y�0`p�NSc�8X��(�wFe�t�����AR\,�:
�]xA�9"�@wP���ls�`;��6���|B55�<H���e�+X.��>r�J�����4���X������_Z#�����$) ��7~
��(?u�?l�]���)K,Z����f]p��HBV�1k�,>����3D(�:p�%���S����V���+���������7Y�4�:�7u��X�u-��e�*�M�m�SZ�����k�S�h�B�Y����q������J�\�+�����AE�I!��MPi��3���]���2�u�����Nc����-��6��� [9�Rb,n�>����%N,�=�`R��o��Y|��#��Z��A��M�h��z��Sg�Y�`3����?s��p>��f�����nk��7;������j�9;SBO��F�����
.��I��=�'��������n�p�\x���y$5��#�����~C�~BL,��k�}TA�����j'��O������l�1�"p�4�"��<�X-����#}8����n���e=d�ov�a�__}8	H����T���y��b6��:/y��>���
�r�*iFihs����IS�O�����%��06^��TN�n+�=x���u��X/y�q��iR8F��
\_o0��w�K�����3�����(�l�����$��m������?�����c�����8:|�D����Q��Z9KjCg)��$�d`
)v����*r��:���]g�=hS���at�[����	B�%3�f!�{�$�Beon�F��g����Q��(jr1�[�����c�8/��_��0�*%>�5Z-ggp;�y���)�������x��t&�
�`$3nr s�=� ��Mn������ds�������5.9�'����L�W�v���d����\�`?�o�!���k,���6�Vk������^��5��~g��e���[_�����g�e�����K�������������/A.0�u2��,��P��T�^%6qm	_�����-�x>���t����RrEJa��4�1��RC�����3�J'��!F�y�:���8`���*�8�i ��x����p�1}F�3�^�L������WYoRq�U��#(�k��q��Gg+��~N�eH�uL[�w: �B�l�w��][���|���%�����b8?�����,�)f�S��
-b�Vs����U�/�%Qj��Z�T���?��,�:�Q�����Nm�QY��������_e�Z����"��17Z�ub2���v�w���O��h�P�j��;��0
�����������9&������(\��
�K��d���;�y�v�z��������N����A�������m;��3��{��:���4��^����f��(!�}qy��X�������-��%��QK�_��I	�r�/�GAE4:-�l�7��=`r��S����/�^����O��(�-�,��J]��X��v�}��GE�CU�����d�D���R(��[\��2�SS�m��^�#�@G�k����#{2�q���n�iY0�)~��[���	�b!#�r�lf����^�p������H������	�� 6��]�$
�9�v��4
�;��fE����M o��?]��34��8^]�+oR�������6>b&^z^\z��y�/�g�i������5o�����t�������}����������U��oe}���2:~�zg�������g7���?����]��01�HJ����U�=�E�6����)��.������{/I>�.��>��	���~�.�Y�S�e�������ch�T5����M�n]��::H���VD������%���&=���\�,�{r�7?��i�dK���nw���������:;����"�.���'�����<��.��+��whGE�V���1�A,	w{p���g�w
����&a'(�zwy~����q������/�v���Y���+9�A�;���2	Z���g j]��[��z	2�7����+���?N2$��J������W_v���Z(�h�����@���V�w�]o���V1^���bE^+.TC*�6k������,��02��0K�����D�9Dh�::;{������,�'5���e��}�~����O����l�Me-8&����M�?�iK[���#�6,��%����1�h��������#w�+�m����3�vw����ve���7Z�������'��J���*��-*��P���9��
��a��'�[�[���N����Nq�p@ci�}���{�fozwf�Q�:�K]�����������=��?�"-]+���U(�@��`� �VX����'���^^���|��/Ii�3�_���!l1E����}���u�i~���O4\6#ff��L)8p���M�B����!1�Z�g�����'ox��������%�&	3t�
�'�+��zZ��6&��J%w�V�<�h���O_����z��=��o��U0���~�U��y?>u��	0�������^�~z|N�>���c'�����udHh�}�-����G��=�B��u\QH����[��H�D��!��a`#�B�,��x(�0�kO�������z�,�`�JM��x���,>���s9)gJA��Ms�����B���*���������I���B �*-�j����qQUR(cc���V�������
��1/0�
���no�7����S:@��|pjG���=8R�����& u��$��tID��W�����ss����bp_���]���X�Uk���V�j�)�u�Q���j��z�T��5��Y����[�AO�bB���3� W���0�>�Q��������H�n�8��f���p�����j���z���]�t�@�#��M���hcU��^���g�����E�g����mfoc�75]�t�zw�5���W��cZ��^Y�f��}��D��!.5��>"�H��*�\�K��D{t�F[��mR�^r�;DG�^��#���L����������V��E�����39�5��&���N:�Js?��?�
r���p^�[Xx��

�9m���n,�gq��UE�b�iWr�����"w0Zg#�������Z/��|����HF����x�0��pE�&����\��S3�5M��L���7�G�;�K�:������;z�L�kpi�C�����n*�@LDF�p���4�r�[�P�O����3�I�Rq�����/�[���:R��[&'����O�1��e<����
�B�OD��{0����N��K9s�2��F�h"/��d.{�����gM7��d�<������C��r���'����7��av�Z��-CTa"�6�D���0�	 <�SLw���L���5r���B���oO�vd+"$���>�.N��ub�%VfbN�N
|���� �~�d^�<��X�I��c�	5q�q��UN�Q�.V����\������b��-!��
:w�|�dm���0X�\��F\�"�����I�g������&'CA����q��o4g`������t�:���X�?hQ����|���a��OiOT��+?XF���i�q���
"��H3���('M
�
i���p��$��"�/����?>�������*T�U�3���.��#�s������N@��L��kI�*�L��oN��j���ppuL'IDx�vE����(��������\�yEQ���^�=��������Z%��x�Vx%E�hA$t
��Y�
1X�R�(�J��XD��i���X���
�)�"|Bs2�8�J���*n���xa_�=����r�N:����� tg5����������\��=`)�OF\<N�n��Z��g��)�c^ji�\�':�)�������D��;D<���{
C
�`K@L	(;r?��L�	cf�7���o	��g?�H8N,EK�I�����0R~3K���9
�U]-iq��d��r&�e��6���(����9�B|��J�CL�3�%_��CTL��X3��5����N��ur�2Y�� �#����}4-$��$�T.�
_���Y����&"�8@f}�$i����&g�2��]���.g5���El�0���$&x4��Y)�f��:}���eU+����L��a["F^�j{ke�c*�5����?�-��3S��y
�����'�����g2��U�p�o�-��_��n
6�	!'�������������d8���A������jjD[��hT6�z��CM�3An*y�Tv�:E�n�%���S�����D[����B��Rz�t24&~�F�>�#&���]��_��r���C������y���t��s�
r	Jnk����=����Y%d���a�L����C�o�l�t0��P��e:���QH����D�p�i�{���o+o&^|�0���_r��i�CGyS��I6A2��+��UIR�G.x,E`2��(���bR����Y����cU� ����Q0[d4�Kd�b�K�sNw�,QS��~M�8�)���)�t8�@�eP��lixTA�1<B�Sbb� �6�q"�~02	�{X���c��,N��B�I1���s����sSg�)�)!�Dv(Ch�x�fp ���
j������������s3��6:�c����Q� �gr~���*��S�Ow`6������������U�\U��� ��K��!Qu"]�Yq���&��'D����Cb�
���L���*
oQ��q(�Y�?��"EGSK������lmn0�?��x������-F%:1oa�zu�zq�2��c����(�f�j6�)'�N���6d{��L��W�/:���s$�)�N�j�����=��^���n���g��
W��s����!U��{hD'�
/��������&h�DG�������N�e��9
8��*��2
��Hy;��Y
��!�s��u�������f���$zI�<�_y���0��������V��?V���Ek�4�,�L�3�XRa��#�6R���x�y#��S�����>��U�����9^����6���R;����(u&m�o:��Z8�i8��=sd�BNl�����D0�K"��U�9yYM2R����j���O���X��<\��
��}{w�� �'��0%]�
�������Ad/Rz����X���1�O�����y��-F���YoD�z.V�����IV�5�K�c��Ueo�L��#h�c!'���Z�D��Py/g�����L�����8����	�g�������g��xE*�51*8j}���R�{�V�b��,H����d>k/�*������0�����#}~@� !R�uI�2vrF4m}�sD$VqN���ti*�]��l����upS���f�
+:�DY9L��}Sl[J�L������U%��h���������@�M������r���� ��'�F���BgB{0R�C<+�E����v�;a��)�U�)�H���d>���H�N{�]I�(g����[�cN��Yu�
i������ ���6����+"����5/�~Q�L����k}�v������ zL3d���_�|5���1�y4_c�6N�;�)�60~������y>J����L�j��(���V@j��O�x�;�iJ#�w`Y�7)u������z�eC��lU�����9XwUk�;�[X����j��vK{�?Z2�_m�C>,��(9����9x�bE��D���1Z0��"��X�|h
��0�� *��.����er���csmm-��������S!x��-���-pl:kh"X��Y����b��a�y��9j������R�4#��y��I����U�'a�����<p�ptO��U&'(��qrTXe��|�&��?�`S����Oh���z��8R�����Z���A��f���b�����Q.{�,�B�. ����x@[������n�M�3�i����C�$�'�l!0I��h�f���f�����jC�S��������X�����A����s<B��ei{����4~� ��R���(�0@^R[�b�F"(�g����!e�I9��)Wh��Q���o��V{���J�&��<&&d��X^��=��Rg}9.����{S���S��"^���p������*)Xft1��Hr�j��@/���%�'��s��G��k���� H���s������Ke� 0�h��I�~��>�(���%W]�tFfj�5���hF�����~��[�!>S�f,pR��l��j�]�Kw�=���\9g�c�!���nE�$=V�����b�m������;Y�	%�%���*�g]���Z"��ow�p�Q��X�5[[���hN���k�t�ciy�O�Jc��<��g�Kit����"<<�eB�����%�C�p��Y�d��0`AZi����<�Z�8a�j3�����p����?�-$j��)����7��U�	5��q��	0)���
Kq���IDD*���9+��f����(P}r5�\1`
9�S	3sd@�8{ :tH�8)���'@�omN���4�F�Z��op%�B���1���Ezp-�����s��@��z�Pf������W^{����Wz�����`���J�M1v�y�7�7�#DZ +"��-��7�h���'�8�p����"����a�5�o=;�*K��}�m=���[fE�C��WX����h�E(��G�H��_����������'���Q����
| n�����
N1�9�����L�}�F��)�:G{�CI����X>&�-�����b
r+���xF22�2uE�QE��B�O2���A������A�����u)�����tW�i0��Z�&TA���*:1�NY '�1����e���)W:1����J������s+c���b[Ks��0��������Kt	;�=;~~���YI�J�!5����*LB�_�����d{��j���w�+��������DP&y�*��I����|OnZr�++���}v����L��V�/���2S�j��|/aeP�7 �>E_�S!��y=�Y�g�-
3�$�{U�;��rKr�u��M����K���D�C�aeW���������Z��%e��
�W�?��se�����)�"�]��s���Q�pJ��]�
y��v9���<���:wv�w%�n?���x��&����,�������+�+Q����:�*|�^������r�N��n������K�P��v��~c����j����{��V����������{�~�3���g�l�7�
�o4�;���o������K��T�@?�`,)�d��}^� '��Y�a��P0���j���kFc�~"���-�Jb�Z4~��9�)���D��-��8���2G���$CecT���
/�J�}���"
P��/�u@
���I�v���������b��T�`����N���|�6�~�%�y�F����ny��Z�1�}O���E�A��>P�]�My��W���M�� 	l�c�����*}���K�M���������PY���Y�&�D��������^����G��f��c��\v����7z�'��`��{��E�Fz��o}q�����u�8���B������U,	4k .�mP�8�����R>d[a��U�a��a��!�&=����@ 
��������VJ���B�\��+W��g�����_��n������g�20K��vg�SoZ���Nkg����%~���������nC�=_��������Z;�������w�f{G~��/�=��K}w��.�����w��J=W�	����Y�J���S��%,�8{ux�FLGXT&��Y�(��������$�5!�����65o���E����5�y.��G@��]4���	�y������<��zFWfG�J��L<��xY_N��7�_��7q=pm��*�#K���s�����FF�!#d#,/U-��$�(r��F�}h7��|dJ�'�j#&G����M!>	��E7������"K�����-�>R�r`�}���eS�or���E
a�����$VV�0ci�� �!��G*��iP=����%^���L�:
���D�k��r.�$mMq^IJ��`���f*�%��a�y�X���f^^a���9��]D�R+x�T����|kf�7K)15w�C�n����rxs�>����K�����R��2�k^ 6������$����Z��'t�OTJ:��'��*2q	��4q�=��v+����	�M����Ce*��#e���2��
q��h�d�#9�f�Q8e�3��I��!�C)_��.�4���h�Z�ZJg�@�&�2&�v6W-8:��P������.�MHn*�>B�-�2�)��g���H��Z�,m���F8X%�6��l*�g�}o�

��4�_\�J�+��'�_�����Tv_K;�+��} ����HSB����GBN�������1��9y\'������P����#4	'k�ZR��p@�:�� f\x���"�"��S����_�����h��������������_
/^�n�����<�v����'�����[oosk����d��j<��W5�,r�W+p9��pRf!��
�b�n�z�� s�H�Ug�)SU��Jen?�U�b����~�����kt��^�O��Z���8:{-/������
rqf4j��s?8�E���CF�n��<����2�F�t��!:��
�u2�����R[�e��ost1~4]	���
���R�A3���P���
�GrPC�v'�h����
�boI?]��g<��a-��>�n�������v�}_c��U-�i�?��h&���c��4�nz��m��>e^������e�Fc������_3�/�D��~�����Mx�S�0;P�`9/��5:�~��#f�7����2mk�>:�5��w���
ad��/b�Fc��&�������>��7�F{G�+A�M�fl����36Z�<c���Xx����{�n���}������NK�������-�������mu[-T��o�����7�EH����������cy�O&�qy����vOD9�_I��w�#d�gr������"���������������*�bUG�@|��DK9z� �W{�D�����O4����5���SQ�}:�`���m�_�m�_���0�0�u_�[1��Q7����d���}���3Y�d<1�S�hb�����)a�%�D���d���(�������b��� D(���r��	,���*�5
}Pl3r�0%��7zNs�s-���3?��r�kK#�
��Q�H�6�%
���Y������E�����4Dk��)K���/}�(����6��vg�V�Z�_coo�\Ba��r�L�	Y��]�����+�`��;u`GE�VW�����.~�i�?i|�����~�
�I��A�Wz��W�7���iPW�e
`�Qt�}��)��������g5����	P����'EO��5���{��+{g&c�?�zG������o'�t�+Ah�����9&�"@��`�/��N��x���e�����X>��C�������=%)|bp�|���dm�Pbs����H�Y�Bf��s����������>������8�;%���#\�l�"���(<K
W���c�q�Yw�1��>�W\0�����������!�R�5�>�,P��E�hfU-�l����E��aP'�Y}N���T�6m.F��,:��>����g�*�O>���)k�����~m ��y&��Dm���#P2�BJd<�d���2��K>�.p"���Q_~K��}8?#`;���O;�����}�c���g��0�G�Yf�%�g��T������$��)��v���;����
��%�b1<�f�]��w�����Q���;�%�an:��������=e2Nf�q	]e����JG{��f����V�&��w����0�:�
^�N��%�~�Q��)J"	s�O���������	]?(ly4�g�xY��p����U�,�O�:��[5"j��D�^��uZ�/M�^i����Z�c�u6�&\
����c%���v�Hn���`�da
�r�����	x��q�B�����F1	'X�3.{����Ep���1�����JWr:P��8D
 �254+�e2��d`��"�fjP��o$F��b�R^����{&�1p��X���lwvq7��v�n&���]�����-s��i�66�/�m�A��w�W���������w�pw��[������sy�MLw��6
�Ev13�5�q���afHE�hma��������s������s#CW�����U���&������l�YO�U�V�A��*%?��J�J���u�)��k���p���V��\�UJ��b�����;��.�����-
o�����m��V���������`�(���q�	��h���s�IAu�F������'���
6i��s�3�w��V{�Y�5���.�7�m����%[i�b����N��d�s�fu�G�0��k�xz�*N�����v��� �x �lO�z���A�I8{?�f�Y�Q���_��W�Mm���jUv�1��&T��p������ ��k��K 
�p�� N�a')1���QTrLr�b���R��M�H[��$��������o�c��1�I��qq�s�
�K l�!�{��cY��1���T���-�Q�����d�6T)7�e�6.���	�	�
��;IRV�]ia��������J���px������/���?{���Fr�
��?E�9�,��Od��
p&��3K�%�@c�V��1g2���}�[W�Z�����������������AKN�R]��`����1���>4|R<���$X�Y�a��(���W��?+������
��]�g^�O��p��B��gdF�q��G`��|��E�O��
���yvK_��C�/Z3��E�)�=|�	o�?������fB�gc4r/��=8R�P@4��,�7n��s�T�Z0�:�6/���?��y9�,<\|��_1g6�.<�+CI���G�	r�IpyPB<�"y���
�o;pH�@|5�U���le��f}K��vl55
�%��h���VA.%U)g�k���uT0�C&�%A^�>���m������?�i��P3S���j��
cf�Z<\� /�V��,�I�$�����1$������I)�~���p�������7C��^���K�c���8?����`H\�8e����t$�
�Z�=$���S��Tc&
�C�C�H���9�"��:K�O�������d�p����V}��
�b>Z���Wk~�F��:�Hr��4|&���6�2�]�r�J�]� �e�$��A��i����*��)O�����~�8{i��c]�tB}�����^�9�l�U;m���d:>�h%�Y���
3��0�(���J�S���S�����Q���6p7��^)�81F���}��\��9����6s�7���FE��8���&�q���x�R� ���-[����Z���:$TY��UP/����F��I�	�-���\{���]�2�i�f��lF�R�E�9Xo�������L��C�����������+�I�}���%v/����K7�I2��Y�Wfe�I9!�[`���z�}���a����f-S������X_�.�N�J���v���)�2D���^�Ou�o<���T����"6��]�+�|�>)�]���5��6����ep��jA�%�#�h�B���F�T�L?T�n�u,Jt�u�C,�#0y1LFH�����,3���Oh1!���&����V8c�[�+V����].$/W�!�-��R������L���R�I�:�rCP�f�I;�i?.�����n�	rOe2S,��*�#�P���>lp��CMf��k4����c�
����n��8���D���(�������H������\�<��y(*���9���5f50���hj
]��c�����O:G���]���E���pK����B�?$��7����!�=�FX�Gt�~A�3���a�x��.���!�A* 5j#�{�"@���3�p i�MSS��$D��^�.����)�+r�o����L=�������s�������]x��@`k0R���B�����Y�2���|6�Kp�F���Z.8
���A�i�L���G�
�����WL-2�!C�� hE~����L08�+0%��[����I��N�����E��8?}��7���[S�!�?.�,���J�(�1
�I7����?Td��)M��j�d�nc�F��x�z2�(���\�}}{y�C�;��j_u.�������>�$������+��:k�o#�G���ch��������Jc�����;[b��l��X����%����1����K����YE�t�������
��KJb<����Q
^�b;�7yE���&�/��FH�C0�7a�o���g�(�!KW�����F
b�%]�S�L`x^�Z`�C.{��!����:��H�Z_�+�0��$�p���9��NS������w(C�*C�x14�LWZ�8^�u(j��BD6xQj_��\	j�|�a�{{;�kp��y�]\t�)����7�d� b���Id��zl�����]��$y��f����M5cuA�����A����bw����]�;�B�4�D�Wq��hN7����Rz�9\���C�}�\��{�i��q����f�(����|�B�CD3f�e�C��/��� �6l���]����<�T3��>�M������A���z��Lq���AE�j�����a����n�:�O�������bD���t�}J�W7���+��UZ��p]\����^A���"~���/)[���\
1���hd��f�$k�W��h$rQ}iY�������R���:��0���u���lLcm^G2y��k9JoZ���Ttl����,�Ze}���_������*�>�z�@�(L+�NIO�L����/�x��������X�HebU+������6��[i�Z9VL���;4���R�����e���?�z����Al�/O���@��F�����[{���8���~�=�)�s��s�b�6B
�X(x%�����j��N�[-q(�Y�O�#+����;����?Ph�#��u,��V�.X�1���l�3���7�z���U�++�`=��:�c~v�����0������V���~�����9��9��/��C�����+P��ltSG�:V:�����D�����
�4Y�p�2��W�dD��F��mBp+,I��&f6�#��#8������~���;�#����~����-�v�����u�d-�P��{1��{#C��!2J�+[2�~L����1s;a_qo�f^l�t2Lt�-{x����vBx�FEG�g��C_�&8����t�%��|�?���2���&�P�U+��!����,�Cy;�lc�5����03x!�����������#}{v�"!�neKu����l��s��z@L������ud3��#����Z]{�l��������9��V�g�$�c�w�
l�IA'��Z�}A����������C���D�Q��f���q��Hl�0���)�B^c�/�X�����Q|cku��A_��8����
�L���<�X@�Cl6Z�>�x�s�wh@�Z4(����\����������/��|h�\�/�����Hz�g,�[@�\��C��M<&�:
25�%u�b�i��K�B�u�Y��A�h�����|����w��Yq�������7��?��w�zdq������9 o�4Q���	
�~�b�H�+�g\���=�#	/��5A�w�^�(�����
�����w�
�k��c
�+� ��h������3&��/w�}�U KI*Y�6w�Pt�����"�Z���2!E���5J����k(���n�3��v��R���U��B��rA��k��cWxv��*��P(
�����`u��D�n@�2��@B:����D$OD��p�wp��77y�V����.x�-��1g��[�	`���&��|kCc����
sZS{/L�=
�3X����Y&a��y����m�����/�0M^;J��D	PW�[,+�h�Q���_6�����u�TLc%�o��]�y��	�u�@,�9�/E��-j��}K|�d�A� �)T�H�h��M<����
3������ ����,��0(�JMl��'�E����G�	:���o�-���1#A�	x�:������%[O]��otN��v|��5��z`�T��u!X�,�T�x�����t�7����S�C�]�K��Z�X��"�r����Y�A��:�\KQ���m��U

���(s�W7��-����(|��}��2b����lu�U�)������V�L��=�F:�����H�AP�#N�[�OK0�}�����0�k�+�P�����:�M��P�x�oH�(��l�-#=!e$��c�9HEwlS��bXl'o�0�2�@�U�A�caf�����CG^���v�����{��vl�������.=S�*�����R'�x^�#���^f�6��b<��1�N.H�0����]���,#x���a�6���@%fv~�0��2�O�)�����c��	��c�������
C� Gc�F�\)&P����Fm��p����:<�L���B�9�K�e�B�M#/��,'����+% ��%/�����
]����P�NXw"�[$j�6��B��������O�wb�rTd%�@4&]����2J��J�&�����Li�������	�'����������a��k`TfCf�w�����
�Lb��U���R�d"V7a�;�}���My;Pd�]~��0�n�c�u/P�L�@���<P��Q�ZB���������l����b�F��|�����	�#k<~2*Au�j�i4m���$�U�2���N�����:Jk�����y�:�D�<�\��)9W7�'����������e4��x$�Qh�*�U?�M��9��l~����o���v����)�����JX6F,s3���5?#s>���?Q�xd�j��Y�r$��l&~���������n���j�*|��^��������p\��G�/(�`�}AG��+�
�?��;��:���	�=��c�|s7Q�[j$�����#�n�Z��D���]�>�������1�����
�t�{Xb7��Uwyz	�|��,��P4�,E���
5���Q����4��������x|���[��/��j�e2K�_I�F������9������\6�����t-1��5� �0��cK�z�����R�r�qGP!p
1�P��-�P���b�j��J��d4��>�n;Z����7����&��Ca���rMt��3%�J���E�''r�i8�G�c�^Qt
�G�7�8�^j,�5�jQ���`UX+RI�dc���>��M�	��c��,�w�m�%�_*�P�E6�"���b�/6E����o������G����U���:u]a;�7G���|�O
��2st�\i������xe\&#L�_�5Q��f&���ug��@������;�����G�6B��%�<�����x���|��"79����y�{�&���tE�E�[��P�`�^�\�����p�%�o��vnw��n��e[~�����00�DAOSR7��d�0�lG�W�9u&I�ec)��	�1�x��Ng�n��7�%�%wY2� ���6��7�'�-K�2�t�_]�����Q��1`��R��E"��2�hw!�7��dv����&eC���L���-5?��]��^J��K)7���r�&(�����g��W�Q*a����1@��FY��n}�+X�
PCo�P������o�����l"z�Y��>��"T�A{,� ���h�\�`���F�xH/�5Q=�>93���m'+!w��~�Iq[���.�������3�Nbjc{�����������_a"�:J�o�jw�����-�b"�j>&GT�z�nP�w[Q��H�������>�|8�lK�f(�+��*$�/���x0\���~���nJ�p��1�r>
?)�_'$k����B���p��mzO})�@|����1��G&���e�!�����S��Tc����$��gzN(�'��
�/��j��/�2��)jOa�M��Jsq���N6N1�q�����4k)���kCr�(H[����%OZ��g#$�~���p�k�b���<N�j��)���1��sR<��@[���nc����3��d���=�7$
3q����%s�L�
��a�G��(�V���;�_e|d�5_����k�;=��,�!BU��������;����
+������Y|��(��X�f7��DR��ev�+Q��[������������i��dL����}0�Q����ms2tu���t��3��(��|�H
L�c��U����*�j�<gJ�R����_���f���k��ukY��H�"��m������#
��m��c����d�R���+��{+-��)4�\�i��R����e:k;���?j�f��`iO9�eH���&��d}O�/��s�dm����Y���}O����;��4�=�Fv4��v9G�U�Y�$��M���{�8�~��PNH�����v��Y���H���,A<x68=}'6<X�����h�w��#�x������Ga�,���"��8+r��^����&=����?^@�4c���D_[?�$�����"���n���y�V���g^#����S�lQ���������	��k��G���o����H�>�"N�C�8U�h`�
��Z��`�jA�c�qB�I�(��N����w���:��:J]{!����z^��
E��Gy��x1 � c:E�xP�-��S�M))��T�B3��
��b5���y�M �@��9����w��AW.�{ ����X���O8i�M�GoJ:��C5Fj<~�Z��EPT.�@�&�� ��P'�3v�W��z0���E����5������"P�� ��j��
�#0���/�G������g��<��27e�x��v_Q����h�
����`[g�1� ��a�/�&��}	@���d�8R:i���Q���+�
�v��T3W�>����M�� �L�e5M�H�p���^�d4�E���fY��^�|�U�����V�
GbZ���������N��$��Aj�a�-���=����D�)��tN�H�D�{WLf	]7�H,Y�n�o���C$&�[Lj>�J��
�*d��C��2p@uu�������!��H�]"������D������H��Zp`!V��R�VW�P
�eLX�I�"Y|�������eF���z��^N�c����6uh�U������/:�������wB����e�*B��e!V��~�.�����>����=`.��ut������(��%��.�����GvU��9c�>+�n21,�e�&�+���,i��X1
X������!���#E�7�g���Y I
��?���c��s�G/�%����/u>9G
�8�m�J"����=���E��2�
��`j��Dz��La
��5���$������
��p{�TA�S���]�X����
/����������/`,vzB/�I@y���}a��r����������Wv'����>~OO8�����r.�d5�qXi{=R��Q�W�Ro�����������3�KMw�o��|��F�.*��������?i��](F�^{=$J�����%��)�Q�-/1���!��
z�V��kx�~���x�u|�{��U��c�2tx@s��
q����d���pU��Ar:������s��iji�DM�1^�b+��W�f!���l�,�K������o�jB��n_�k�(���<f�Q2v�����.����Ox_��������nN������l�M���� ��bg�x�
���c���z��a{�%Z7��4M���Q�.Y|=��`m���h9��/�+��CT�����7�����������Z��}"�%5�����������Wi��'cq|�a��2��52�h���$��w��@��(�]o���|��<�g�9�&TI���
{�������Z��dK��V� �>��E��YP�����[�]dYs�Cd�f��V5k
�h(�OD�����u(�+�|��F������=�>#�p=jz���e�Hr$��5�{"_0���\�x���]^�@���
�C~W
;�(Q8
,�w�S=�KM<J�>
�
<�~�������%��n�2�������@
s�0[L������v�� ��[I�����T����W_w�V�����Q��j�r�9 .��������iX�;>-��HPT�j�K�v��������4g������<�x�j;���

���J�f�*�O�����i��Q�o����iT��T����i�������?�q�B����W�Z|-�Gn�Ae��'���H��r&�f�34��^��y�>P^c�Y/���5XY����t�.�J��(�m�B�������iE.i�� ��������N��t:T��*^vU��������sV��~�Z��P�0�eAq7�8n�1���-"����3���mI�;��C_���M�k}J��XY��R�g"-����\�G+�(27N(��#Z�{�\K��,;XJ���f\k��0M��+�f�'��e
�8���i������������E����������w-be�3��m��in�6��g�EV#J��l�~�����L��i2��_9K���m�9�B����Tj�~�Z��1����%f3kqh��p#�{��vro��ll?~/%��r���.�R\)4e������0�j�Q���%�{�H;f��8A?�$���ZE"QD�>f�]�@C7)Y,u>8�!,�Z��K8(@���!�!2�JaB�&7�c�����u�y8���vG�!��`�'rPJC��-�C���?�@�>�r��(j��;v��,F����#�p���#�t�'u���p ��.��IH6�|B,�1^������fw���hL�A��Q�J�8�P4`��%X,-��t��t�P��`:�W�0��+@��Jgr�W!�/�����f]�����'����lUg����_�^�D���MJ����V���8�t����M���y���kMi5�P��v�DD�R��u�\U!;��!���.�>�l#�I��H�J������#{E�Q\1��"V��>�#X9��Q�����/<:����l��]�/�Y]&"^��GN����~��~5�B]��O��3{o|�y.zT'SMq���}<���s��su����t�X����^�m������f������+��+�
��z�)8������=�n��6�P5J�	�k=�h$�>6.12��4
d���w6���?���%F���zB%f�i����Tb��V�Y_��P�8��m|��M�@�w��������?,�!R�,�'���/�d�"�dl%p���[A�!E��v�c�M��C�G|���<�����
�wj��k[����
�����w��Bkl��l������\��K����[���L���+lM�4ja�{Y����)���R�d~
���0�$j������J�V%�����4�M@|�������Pz*�LHd�K�dD��A�"z
&�F��
w ��L)��������;,+��M��|��V8�vZ�OX��p��dEBR�l/�bI^���BX������
�F��5b�!���n�@8�����)�`�TV&`u�W����������m����vO��8��%�|�t���C2����n��9Na�Q0�X��@U�Dl��@���.[��>��4��e1�:I���i��� ,iK*X;�����T�r���a�=%�!\j��+�	<��@��XW&K�x�Nl~�M�j�=�r�{QE��T�w@�%������t�W@����\�[>�Q�u?Z����~,�.����
�����Q[�p
�@�u�����%�fco��7������A��6U3{�n�"���'�~�dVv��e�d�|�	Tg���qr�<�<'$	��3�S�x�j����W��Y
>=����N:XT�4������?�{g���������L�����x����/R4J:y��qk����3�	y�p#�[}>���C;'�z�/3����5�#g)R������z��r������G���t%���J���N]9j�����C�@?�kahyD	����6�}�Q|�1�S&�	��!�R��$G�/�������"$_&S��%t�V?p!��@��`�d���F����z����zd�;Rl&��.&a�K��vM�3�.��
��>�����*������\�TO�91�8�eb}A��=Q���@����������vc�M���8��h��ji���n���U*��XA
��������
�`����]��d�fX8���]��"�_��)�(1Z�FB
<���E�C =h�9��'��������K��YWx#�o�/Q��oo��V���Nr�Z�L�t��_�����/�Q4kb�tI�>@���1%�R����+�x�xt�6('.yht��n0�������+w[����������Qr�A����,�0��/���D���y��&��hD���6�[����&�~����������SR&�X�Z8��"U�L��� 2���P���@����<]_?d�)��#:�S5��JW�y��j�(�)�������x��&�J>2�y���3o6[�|1�c�"�WG�P��s�m^�
:U�m9��y�_W�h������RU#�md3��cr���-��n�y������Q|S���R)�@l��ma�W��777��T���R)\6_�����|q��fi=3����������I��@�N��R���,<�)��)�{E4_mS���S��y6��������Dk�U�����\�YR�@+�������T;A�&�;�pn���z��T=`_��\h����9>N��aW��L-+���A�sqO.Z���-s��E�fV����qTPD�K�pF:����gz������������+���������]P�������g�U�a����K���
�1+�..���z���C���gXv�\q�=Z�},�����wo�e�hA���Ua&:�TK�(�<�,JY�(W�i��Tj�~\'|��Xx���'F��L����w5�~� `Y�6�����9����#���������Q?��)��?F�L.�(�����$@m>6)K�T'!to���|,i����3/��QJ�>��+�S�)0��`x'��jkg�8�#��oN#W�t�yv�V��/;�S�&����!`�)sO�[j=v������V���
qoI�5Y�
�d��K�)��6�Hp0��M^Z�����C���.���f��zD��7�|NaScU�/�[���L��$�'��'~;cSz
R��7��/��0�A����`�RW��&s�%_&R���m���Yx��sp����kM��b�!�i�~�/�u����Y>,*u���c�.h������?Ut�~���q��4��=��
.G���y�vaq3c��`�$	$
f���w.����1�<�[�c���&���g�=A��n�!�����h�I6���Y|���_�L������lh����<���X����$��~�T�b��b�8{����R��fX�)�6a�L�1�>1(0�<�0Lw�����dG/^H��R��x��w���w��O�����_���
;0X$��gNy���dH�����x--r��
�<,����=�-N"����������O��^'k�z��+ C��k����s*)u���,��9p��
�|D�3��26���R� ��9�M�RGQ�������(c��;�OF*Gxe���/Q�}��-��s��.��4t"�F�K�����wg���1�Qt�����j-'����W��3�8�|F�	CW�Luj����1����H��`X�!�;T�H���������r(���Ys���s;�>�k�UG��S=p~DL�d,�J_��%��I����>���G��hx74�t(;����M?Q^
w��:��o$����j�t��,���W�J�������;���]67�H�����.��������1pX�^G��'�n�J�d�0�Q�N1�@�^0�"���J�?zN����VQH������\aR�R��yV�D�%�A��Q3��Cn�.��r�q	�[�>��@�N@OEj ���=U65��:��V���ILI���I��9��@��Mo����3�]�������+�X�n(l(I���^�t��%%Z�����|^��������5�Pd�g9e�tY��trh*������DG}U��8}%j`��*��)�p�N�/0KZ�5j`SO=bkU��[H=���Dso�LF�Z�����+��2��F4&K�Z���D��U24��;��!���D�T^�)���	U+���:A��D��,���P�ZV*V��^��W�L��d[������\�p�������h�=�( 4�p �sN^�����3s������TmAj���K�Zu�����b�C%��N�q��-�R
�2�����������DFbLIC���G[��i��d*1��l��a_a�"��(j3\c��(:#kR���`G����2����L���qX�O��U�qF{�bD��X�$�k����P���gM6�e@�,5��.�u�+�d6v����Ls��u����������
U/f	�B���c�+��C�}����q|��U	�o/���	�����l�"�. �������'��&0�O�����!��������}g�O����az`�XAZ�����	�j7�cyC\�X��!�|x�s.p��=�_P;��3hh��������EF�|���I�p�F3;5r��������X�u�>��@��Pi�"s�c�[w� ��'S��5���D��?���|��~:�]����z����q��q� a��`$�(20O6t��`!P�j�3��t���wG�JQC�����+EJN�8����H��3\���+�.������J��k5�����a�����]p:T�K��P9\�QdF�.�~��II~�@�C�f�uPJs��%/*���8%44
��������Ne������^�-�T��\��e�Y����A��8z�k��b��]7��Rh��O6f�F�-Z���������,����PN@�
b~�>W���3/�B��e�<,~�7��������p���_FH3�H0;q,�$K��t�T�b5k�9/���wr���N���:�������o�E,���X���>xi�������U�32BE�l4%��"��+���sd�n��n2{p�l(� �QZ��2��t_nkD��-d_$aS�F71TX��W���,(����W~���]���E���T[%w!�r���b����}<���(��t-�R�t^������%���C0cU����P�L�q]�w��&>����-��]���/�5�U~k�vU�{L�p��a�t#dB����j�j9��K������_!_�N|1��O[eR�i
�2^��C|��.f��"!}�������H����>HhC`S"���!$v�1L���T��@R!p�����c�Ru����K?c@�`�e�1�P5��^�,I��(�ArG�%��K���9��2w#@���@��ur~&�^�<%Qv��q�	]�&3�N�?c�J�\L�qw{�V)�B��4����%!�
�w#j�x1@z�s�G���d\��2,\���
,��Y�_4�%���������+p�\$�';'�Ai��R�j�e��>��7�A7�p��&%��`u��!"S����=�p5������0�B�{)}�����k	Rf�Q��G�f��6�u��>`v\D+�D�<���yh�������z@��rn�E��;~��������[��T�7�e��#�S*���D��`.����O��2O�}^��G�U���s��j��Z�~eR��5zd�����j?��@��I���x�MhW���b�R8A'�^������R��u���U�����W��9w��u*�?��������Cz��5�^��qru����2@(/qX�5��!�
�W,�C`��������6��px����
��:��W� -G��I�PV�:��"�I��3l��]��Ont!��R*����65��1��8T�����&���&�����ZF�U���������C�f�����z�{��O�@��w�r+�������X��p�~���\�������r���Rm_�4DP�����U�Fy����TH�I���M5:�����bI��\�JP��Heb|
������x?��R�L��Sx�~�����x������_n�TE3���[SN���rL$�UR*�����+�!Z��$&+���V0e~8F}/G���"�?j���&��I��D��3�fS@��M0���
��d�^n�{H�� �1&K��m�����*;�'�a	��dP�dN`\�p�'�M�	����@������i�l��l
c��k�����|$��	���H���=��A�)_��E�
�4�(�4�)N
�NLY�A%]2��}�K�^����� a��aq�*���70h�J��"����t]�R��x��R:���{��1�"}C	$�(�����3�n����9��?��p8{�_D{?���f�ND�f����9Or�R��Sw����7H�%�
���-iB��6l�]�u,�������Vy<vX~��o�,zN�7��S��i"��Pj)�Q����������B��F�Z����]bA�W��e��I����P�6�����]��<��������~f`a.�A�]K�����g�W�3�\���
3Q��o������_�J�/��r*)����\��(FlQ�5�;�6[���O��$�W�P!�����	-��;�C��p�=��������*�+�"5[�)v<�7��\���6+�����xo�xl���SY��j�kK��< [r$�CA#�`���2qJ�[�B�����h>g��U�m�/_�L������8l�#al�70��X�7:�<�������4<���������3�&�Y�����,a��*?�P(UI��K$4wO�6{Gw`������3��������l�{���
J	�g���8�'�����*��Q�����j�z�������~����w��f�����lcc��W����W�2���7�&���������
�'����7�Py��k{DQ�#�.�.���o�?����w+�a"��q3)�HTJW�*���3���@����1Jl�����0�gV4�)WG�e^�4�8����=��8�t�IglX����bP�NYG�*�u0�200��P�&C�`>���V0��#t��>�\
��:��?�������Qt8�R�32M��L�w��|*^'t�AF_6I�����k��a����Bz���sI#�.����J��;��������()����g�J�Q"��P�?5�y�N�����=b����H��+�4�3��>
��m���D�;�����D_��fJ�SR�n��xt�N���`�e�����c�R��M�F���I��Ce���6��������L��a�Scs�` ���,��l8��.�����k/iM����F�a��n�^so���Q�����#��w��ch�N�#����EB����X���!�������������9V9��
��������P����WI�-����C�[4���+�vW�F}�d��=V��K!��^*�#��G(���
F���k�Q�]6da�=���a�X����2�jh��Pb��,{B(=.���������X��O���6�Rv���������A�F\E{�(-�H�e-�+nqk��R;��O��:��@M���������o����B���p�h��u%C��C����W���i��qj����h���d�H�}�glW�l�oC����O��4E��{�T��^��f`���qmY�
������A��;�cG��?���C���v�����l�A0�5�]���������;$��>n�:�]�aK��������"b(�������x���t��ld]�V�����MSf��K���:��$.��U�&2��^L6T���>�zJ��y��&Q�2�'C�,�������S�8�&K�-z����)n�C�@D�A��]�����s��e���U�;mLQV�u��p32p���rb�������[��M�.�@��`[��|�`�Dat��y����j�c��$,�U�`e�7��'����(�R����	A������q�^��R���s�0�L�t�Ln:�X%�{����V�����[�__E�����b�O�f�*��h��pu�w_�y��R�b��HVR�<����L���
�>0bt��+�@���ph��Q'�2��q�t!����i]�C�<�{��������?�/��/7�����L��m�X�B����);�6�Yp�����M�c�����d���)*�SRY����@/�/�8�YD^G���pkx4��D��/"��xk���>3Ko���vO�=Dv�H^��7������|�Do���W�/������&\��3��7�������X��4�6��a�g�����cx]����]��iB*	6�����$��dI��h
��r�Hh�!7�C%O����������Q��v�����$Zf�O���m�e
�ghT4��X�(a6cTJbi$X�u�Z�~g�)cG���{DO#�C;�'F���`����cZ0�(��jDS��#p��U)6����D��I+"D��'�b� +6���ib�g�l�+�	��Y��+p�A������x$I���������2����q0��4��t�
U��hfJ��qo���z
'�}��W����jh�:pfFQ��u�,�0v�tyFv��m�y�u�����mj6����b��������%��I���" �O/+| ]�$�A9����|�
�aVq������y�^�Y�'�������:{�~K�YI�\�������J�yu�������Y���QU��y�M������bv-x.��E�}�
��������On�t����G�+uF�/������$�E���`��n��;�:�k5��~x���I��+�`��$��	_(���t�����&��*��b>_�U�Z��: 4
�_e4:��x<V��������b?��������]�D�������
/T�]�!�=�����mu�d��~�_��]�}��B�I�q�:�3A���6���P,�^`M�/�J�F�	VPG�`>��!��"���Cw��%��������(������d����j��d�}��?����u���WVL1�ot0�����9��n"�P�Ht����}�t(��t��,��|�uZRa���"(6 &zT�w�%��O��-~A5i���]����dX���\at��Z(|-W�~�|/+��(P�J�"��5`��X	[N�r����Ni�����:%�$j5���UM���B�����������O�������b����}��+e���� /9�}��Jl�:� �B�u%�%��\������im�[�����1�-�C)�&���Z\�keE��6�;D��o�)���� C��;��IN2e%CX�1l�9#�(��?l���N���>���9�#3J��$�!'�+�w���tx�]e��C�05�?�D4iq�D+������JR���~: ��I�8�T����������J��b^�(�k���+�[��[�����S�������zE�"��j�����]Q����+Y��^������`�Y:�G���#$�F������$VW�"�|-(wP����-5V$��+�z����B��q�)���G;�b4������)�l7�
GY���.9 ����F1q(�]�f;�3���DN�L��-�J�������7��E�{Z�[q�����8�f^#!44r�q���<��B�GJ��v�����^��R�Y�{l<Wv`K���5���{3��������������Z@�����0(�P;M{(R���l�*���?��~[����v���>�&��
d�fFHx��������i�����W��9�Z�|��:q_G/��.�=|+*Q����)
�8ZT��X�I?���Dz��%�B� ��Pwj9�P�N�^_���M.���o���WA�b����:��_F?�^w�n�]���Gz���[��������p�$d��b���L�M�n���`$*�4i����8K![��!=�O�7GJq�~w���Y�%�������UoV�(��H\4��`DH<�����;5\�p�M���x�����l�8�&j��/�N'��Br�������!?k#�>\��^���0G��4;��F5�#���?
�����3~�B�O�.%48���A���h�@���.�)b9c��	�
3�b��M'O����
��+p��`S5���{�\���w?�[��������y��hs��7�jm������(oj�h���=���v:F�l:���4�Hf:�a����x�hi���V�p�w��^�v�Qom=�E\�w5(d���1 ���C
.(F��P�OE��=��8���9]�;�L��HR���|��g����)�)$},n"�(��F�"p�Cn��r3a+��c��>�_}���?�_
����	��M�)�(!p�v[`X�i�����9�\���X`���PRCNrY���������L��b1����,�.\|�0������*|�Qv���t���&������,���1�����
&�����VE�/P�;��������7I�����dJ��O��W�v��E���������Z`��(�������;�	�Tu�a�L^6��:�:��;U���ir�~��a���r�1���k��>�>�G,��4����G��SC���W���m}�_$�
a;zK�=��h�If��
4���v����y=0��%�Q|�ca�������\#?�y�;�����*��f�q���
��m��46A,a?`:������/���2�\n�e�d�j>�^2RCJ�|t5����E'���~}7�W�<p�W�/�8�3"
<T47����3m�2L�?�}�v�GBrs�0�>�X��|�����\�N�h��e���r�FXK�u��S_V
�[��)Z�x�G6�5bQ���1C�l�����Qhc4���s��h�'#0nL�`��L|Y�o|����:�y��`�KH\���g�kUz��� ����'��
�l��{������)	Dp�����@>b��hd���z��|G�l�M��O8��1	�����~���"��	��]�p�u�Z��������u�M�e.iFX��Yb�a`'�KVAl5s���.���B�iE[B�vZ��m1�
Zi�O����Y���I�T���*8��#�WG����9�ZA������;4�0���.��V��M������
D�@1��6����]�����4z[�fsE�Gu	���jhA��0NN�Y}���^�������72s� B��/��H�uR��.	�����>��s2\W�D�j� .����(��y7����Y6$�����B��h�<�udcB�����(��+����
���|���zRc��S��k��:s1�W��yK�Jx��M.��aN
�(��f@C�R<f�!��s X���3~�Xepq0�%g�p��tF}A�RF�a:��8b���0D����:�g�����[���9�G�8�E�I@�:�����S��
)f.���8���Sx���v���n�����I`�5l�u	9~chc����@��/�(��]Wi��9NS.���B�4�ol�]������Z4
U� ��=�E�Sg8��V^���������M������\�;��y���A���?3�����fn��p�5�+8dQM��0/������e�*����4w���i^u�^k����`��������}q�.(-�c������0��/���X������4�D���$��_"��K	����C�M�/���#���7	�q�D�/:���-,��%3���e
��6��rTL�e��(���Tg�Q��9�V	&�V|'K�42t��F��1���FWM�%�%�� �fs��#pIy�0������,I:�7�D'�%���|X�?��jk�x�jA��	����-��+ ��Y�;u=�8JxU���������2�;�08
k��@0���H�g�[��,�����p�0e����*)t�o�"	���W�&	�=�����h&w^2Joj�����z�|���	9����������89W`�/�b?Fx�1�(s*��������o<�������C���`mC��H;����������l��{:��2G�c���� �����bx�3�JX�
��G���8��*��m��������/�."����^���n2������:���:q�[n�WPX������=#����A<��f����d:GP,}�
���\�����R���1?���\H.�G��^�����h��c�-�ZX�s�FIa
��`����Gm{=��NBe���%�%j�#���+��^�������l��4�W��/6���G�	���:����'�ma����n��\^����
��aM�b}���8L�[�B:p��/�����lF���b���^X,����6���w���;o�/;�g����xq}Us-�V54� J�S/ rq��i����(?���w/��o3������,|L��	�e�����,�u���|k/��F�{QA%m:i����>)*��,����lu]<b:D��\:��;�>��\?9%AI����a+(Ss��k�=f����n�b��uG�&�s�#��%7�=W�j!��$�k+�@��$��k��P=1d�*@�����
�$
�$�C.�"��me��R'8�Z�Z��J������'�Y���8B3V��/��a�i��e��p��w�]�2u�V�/L�C����7
��&��Z���	W��%G��e�q����f���Ke��T�<�y�?�q*�f)�WTjjXr�B$?��n(&���]�����c��R�Z����0��J!�[S��TT�����J^9Q���p}yt�.��@$������g8"V�N�hu����=�VRvc�h[������J�`�p��Ds���%w�!P]��~���#�s�'�*������YC�^���b�a�s��(g�h#C�����i?}��p�����L�e��-4�N�����6�����k�g������uR,�=J�O�I��yk����Og\�
�h,*@��������cD�^S	x�	;�	�A�+����x��d/�1�ACb6j@1����]���X�����'�*���S+���Q�Ul��>��JmN��[���*yC�N���|J�	�f�Z"[�	;<K"�~H^�	�Z�hq����
�P.��2����_j�Lg�?��{���g���1�3�yE[��j���j�0��={cI��M�B���+��Od}�C��B�{A�
h�v�������������Z���Q��O��)%������I>�Rl��<�.����PQ�o�0�:�&jmT�{���t����T�v����1��r�6{���}���>�O��ku xue)�U��=�����`?�Kv����n���o�5w���9�=�+s��PM��]�S"FN����������OC����w����]v�������XNz���t4��48�&�!���[l%a�2���9�|�.��Y�&������$im����������n����[�VGE[`5A+7���W�q��f��y:@�����!V�.O�������f������V�������5x����:�����6���%(�_�l�e�a=���K��{�^c�������n�[�II��|OE�a���V���H�'m	4�=Ap�t�����e�����OK�����[�W[-'}�2��+���
`s�R�[��v������y�l5���V���o�v�I����������Z�R�^@��J2����%��� ��t������������������%�>�)]x����E��I�Op�����!��b~�����������1cR�bk�BI��>V�@p��v�^�*	���VG!Fa�*V�Eo��<�P�������_(���O�<m�M��xGk	�UBr�[h+��n��f�;� G�
&@��t>v~��s��p�� z8�����o�'�����6�H����t,�w:�����������H-}g��of���'��;����g);}����>�(u�3���	��1�� dY�.UD����|��3�������v��C������`l=)���U��[o2���	E?�E���E=��t"�g\*�/���?�Z�<������>�E
F9��*e���4F�)�Ti�S�Y%�[0��TQ��x�v=��Z�.h+��������N�$���"r_{
+��ao��fZ3D����Mf�I�~-�l"C�[�a;�|hu� �L�y�p OFnj���_���Pmx&T���":���T���f�B�X�{rQC��Ea=�����Bs��{L7��]�^oK�������6��a�9��k�����<^�i70'5A���P6R����FT�^���~���F5~��03��7z?r��G��lK	���D�y2���P�������n3@u���J;;�`�n��?���~9]P&�-Z��Z
4�-^�1�H�V�_�k��!��P�*t��g��%k�g�^������a��b}��5�K�t�\�
��f�A ���#u4�b(rK��c��F����� @�S&B�E#��1��U��V�l~�`X�C���n���C�VuJ���j��w�]4�~�y8�����J3aa*Z�v�4��|V�c�����Y������"���c���R�n���?Pzo����T:Xo�_Y��z\�yYmz{���v!:�����Gu%t���D����:c��+�Q������ ��X4�����9z�����*���D�����
��3�(K�rB
�L�p��H��b���an���n�n�m����
,��&���,�'kb���jgHQ|�4W��^���J��������/��(��}f�@|K���N����������|�}��
���T��&�?�� ,N4WpUC���h2�n	n�r�Qx�_f�%~!�<�#�����nT�l�q`L��p;�u�|H���:��ar�&t=3,�?;��M�����?�7�;��wH��l�~ZTW�T`]��jj�e2�G��af �����A����~�����Qlkc����)lY��A}_�r�P��"�Q�}%l�����T*XT�L*y!V3 ���o��m}��m��K�ial��H�1V�x���'���K������{����?�yv�����{B���6^�������e�����z����_(���B[�!&=����:�������U��X�37W�TB��!���a+����'c�T�^�5�x
u��������(�� ����a��.O����c��?M�spf"b�x>qjU ��F(	>���x�-�=��:	���;(E�8��t��L�C�G��S�����||���DW<�Jn�Q��bs?����u���S+������y��0\������)�H���h&�H o��e
8�|�6�� m��Z�R��!�Z�N���� ��%�6�6�w���%)q�{�u�YO��a��:���~F��@o���8F<Q�Hu����Ft���2�m�x��
�i���k�{�[l51��d��/����g#��c��f�T,@x�,M�~!m����`��l��������0���=�%�|2n�|�3O��J�g�F)f�t�i��a���Y�G�5�/C��t�j��,�>2)sH*
`�Pi��?'c�K��b�����������-���e1o�:���`�u������v���=���9��w)*���&c72G�/�"#�,��t3�gD8��T��u{E"9���X,���-������;H��h�6�V��U�f����j�k����4���Md�9��)|�?4��tL8����u=(���g
��0��r1D��>*��Q�T�hR@3��"A`)�y�[���j�;bqX����\aV�k��ca���>������mf'[-������v�9��dF�4��'�Fx�|
�gPx���
T�D3F�������~�4;
��i��'�V�]�U�c����R�K�A�.�F�>.:�����9�K�n���,�O�ui����tn�z��(/��a]:��,7�K�����������V8W�p�s�:a��Z��W����t�^b|3�����`(���	�R�&����$X�6���JF�Y�����������-����z��*�%n��q�5���B�C����K�,�y��*/������-���g?];O
��Q����*���,��V�Q�����������)��5��ZW���/�o�>f��!yp�u��@��R�D1�W[�Gl;�771���=C���8���@��}�h�,y�����@��`s���h����A�D�	��g��V�����P�s��y�-@Q�'����!����8�G5|������Rv;o����/�Az�$��8��=}�i�oc����FiR�����V��v|��
�bh/��?'��R��YY�m`�Kv�`k�����*v��U�����I^X���K����y���b4�	f�o����7���7 ���}Ue#�*a�;r�.���N�����d�$�
�S�����og9p�����7�����}�q_�l_]^�^���U��e��p���������)������V��/=f��;������e�O]&�z��Z�y�����}�����;'G�G�;�}zN���q�B�)��A�^�
����y/w��E=���{[��dw{s�������������8�Q�"�5�l�ecH8+;)x��������1�}>M���d�����#YU�o#�o��`���2������y=�x{���6��}]����}�����p\:����OH�Vb��u�i:�=o�)�R�
�~����]���F��s�=��(	��Eh`���������]�8�����7������m���D���D�%I�m�	����l.��p{������$��������A���lu�h���R�*x��Y9z5�g�oy�3�������i���Z4�_�	
�`�l���	��� ��X���%��3�m�"��J2�'���Jt��[�zI^��V������n���������]���/��,vY+*����r5�9���n���w��M�����7����s���)x�^��������g���(������wG���W��<��~����L�.��na��]���'���b��	6V��Fk��z�
��t	2����v�D�&Z������,�m`>&��u�,�0�[d~�E��S1�<�"��0^�U�G�	�+�&�9��d���?!~�����/G�O��E]�;�DX�
��=;��Xb?)�=�'0��KhWt�]�e���,0H�l$*���t���t����~�E�mcK�w�$-9�&��!ohLL�;�)��������4Y���UM&���Knp��8
����z��o���r���v�������������8U^0���a �������j"lA������M����3��c������;W����X�B>�G��/ ��]���}�����{�d�j�/����c<����������'+5o��h{��D���B��*�q���D'�:��5��}����6���O(D���Wa�"����|����p�M����Odu��j��7��� ��	s�4vGx�3��j���VP=<H�o�&g��lI:�3���JZ�����O��#wQ*��T)g��2`�rb/N�oxn3f3�+r��9P���?=�&�\����&:��`�v�uA����t3��z��d��Z��
���;SG��z�bJ�0���x���Y~�?��F�
��R��)�����b����<DuX+�7Ma��M���%La��`t$/�T.�ra�Dc!��`g�� \"�mQ�������SC��r+��j�l�j��!��;P[/t��
?��x��y���W�'�r����U��?����[�Y�������,�5�i�\L���|$UY��jUV�������hf��Y�+�f�NL+H
(��nf/4��%���f��yCS�������]D�YbV��������z���j�uL�F�!n�����s$����
��p��� bgx��(��w\���dG�8�R,�y��'c��.p�4G�u��<aza�(��	���&��� }r�l��`���I_Z��YtU���w�y7������K�V]5w��������}���g�N:f�W�&����Di��� D� =�J���p5*�q�M
��p���y���qX����=,�%���ux~@i���V��d@�	��K���>���������t������q"��ET6%�Du�6�J. ��4d�A��H{p�����E	�P�p�7��Ss>}���D��Y���������WTx��3��l����s >��}{������ZP?]>��e��b����m��[�����X�Z~�?d	~�"
���H��d4�bhjq�����!�Gz��g��NN���&{��/m�M��>��<�)��Y$����$;��l�"�G���n�9��nSp��^�Pf�KS���d`���n�G�2�j��'�HPVkd��������7l�-�iks{����@@������l|^s����j�� �	+��-,��Xl�[��O)&�h_v�����S^o,�3��3���B�M?�����6�_�*KF3Y��#�>��=�7pao]�`.�l���P�?��n<��=�
�y�yQ��yD��
T�+
"���'���a_����w\�EO���0���50~Z��j��,�F��Fs�m�~�����J�';�Ef��6����g	����	���[Y	L�g]	�S�>*��09���R�i��;?�`�Bn������D�����[�p�.Ia�EE��[5������U<��io>R���B�����uu�B��z����u(X�|��OXN��:l�C��!b�k����@
���Q�ER}
�[[`o����[vq����Qz����;����i��'�b?���G�]n��:c��Q��F����5z��*s��i�
�[,�@j��6����v`�	��p<��������8N^�l�KJ4P+!E!�?e�|=j%�}0,Gm)}�
�t��0�	���9c#D*��.vo�
��W�&�,b�� u^��vD1��	Z������k#��V��v!�^�5�|:��u�A,�������)� z�eV���f�B7�
����C�/J������|�L��
��������j������G�y:����x� ���)�8��8��
 m�N�������&������V����J�����:J���,��ES��!�$�bN���Ke���>p�b2i8R�=��<�M-�HnC5�����j&�"����H\��0���H��dggw����ln�S���w���H�)��$��j���id����-�(���l>�W�K����1��Dd|8���S���Gmn�p�@�K�� b$w���k �\��m�kD6�4�.nW��):j�l�����`�{n&]��Qn�����)C��=�������v��
��;n�)'�[>��	���E]��e�������K��[�����Qk�.
��/.GZ���[�R�R���!]�j�3��uLP�%Z�YY������>�u�}8�J�E�O�"�!
\��Q��d o�(6r������.��}�\0>�k� ��{��)�����'j���Ov�+�����4�h�x�N?�����.�k�>������%li��`ho�F\��`*	������� *������?AOu����L���N�&���w�T~w�O��`���n+���=����Foss_������KYr��� �Y�������UOhHU��&���+_^rz�,7������j+����Q�7K�{|��NY���X���8E_��kN�/r�z�5���`�����?�M�iN��I�9�����5�;-�$�%
u2�;��@�����G2��6�mIb��eA�fI<��zQ��MC���p����j4MF�����"�������x�`�Q��Z��^���2�� 2������w9PX��l���{	�7�:a�P���g�?��\��	�	b�C:(�1�f�2Z��6�R�Z
-��hC�R��wQmMO>�f>��B�0wn��?�&8�m��<����2��X�Zp��E��E@��z����."�b1����A6#��M2��L�p$�_Z��2b�;R�?u0|E���!'�
Y���lM���b��� Ad���b=gr����oow�Fks���k�4���E<������mq���<@j+�f����WLY����P�,o�0�m�Y�'����Qk�� ��}�:@����A����$�����
���rB�/����>-g�9g�m-��F: g7��-�W�w�d������)�w�<��Ov����^owK] �;�$I��$=]�Q��[M��o�P���h�/fE�HV~(y}�s���������������v1;�#{�1��������r���>^��5 k��'�U�����)���� �Z�J����L
��~��{�~���_I�p�>M��%��Vg�?�:����o�4w��V��&s�i2�3��"w�m��/=���^���S��T�A ��3!��;M_��X/n��v�/j�p_��,k��p�M`�!Z����r9�4)���slq�j�-������I%�z�,J�)w������R�~W�<*�*��U|�LQ�~s7�o�������d�1�k���.���f5#�"�o���3�h���*�W
��R~�b����V�l�on�������f����{�(8z���V��� ZW����������C6������������
����.h,�"��6)S�����
�P��6�����-n�k�f����
u����n��������*n�E7-lh.-lk����I�sE��0����������g����}�B�����5�1�m%qoo�o%;����V�w0P�y�-!�Q�X�����D(z�c�?�`3���=Uj�mr��@������Lm�Z�7o�a�ptq��l����^=9S���}rq�>Q�{��Y��]��k��m���yN�"'WG��������.����'�kxl�L�u��rX�H~�9��e����7����Z]@�Tj�~kT�n������\�6ug�
|��\^<���br���U���Es��F���2��DtJ��Q ���U�N�`E���(r�-T{�w�r5z�7Z���� �o�4��O�9 ]30=C�q5�L�wO��|��D�����������>L�Z��V�9�0�N�~���z��j@������p���0�t
�Q�=9�, ��C�s�|�����B�l�����CH��,P���	��C ����1aC*:b�R���y��,K.��n��4���(����J��?w��O��H
p�����O5��
��_:�"t&Jl"���[��������/�\K3����C-P
����U�D�L�����R<T6D ����@���6�����C�]�?:+h�kE������6���a���;���Z~+/,��FcB�vA�
B�Z���W���G�i��-X��,q�]��X�:���
(j��xt���
���7��"���F��i�0��@��~o\p���������
V��c���tSi���l*���;}_%������
U�CKj��,�J�l�v�f-L~<]P�T������{>D��t:��}�����������;zc�9�&�5���[��s�1��eE������}�#{X�N���P
�;������X+V���~�I��\N2QQ	[W�$,d�y�t��m������tG�]`�]��m��h�Co�fk����]��h��jm{w���(����J�����?��n������,quc}qF5}��g�5���l���)%��k�Y:��Sp%�^�m�XQ��U�T����������a3u�;�Y�J/V����*\Q_Q{�4k��WR�f�:WT3tE�~�+������%v��������������W�����V����/�*��(a�e�����A�=� U��N�����I&x/�����	�c?PB��4?���6���8��M
��mm��o��|�r�K��6��>���U�X����������/~���Z��}Xp�U����j���������/}�-��%���'��������M�s_�� ��4�6���OU��D�I���B���?~�;CL�4�#��K0���{��������c���Gy�J�*$�^C�������	
��C�p��O����+�(��>E�L}�m:d�k	��I�V��e��s<�q��N=���-W�t�w�'�|�d����4���z����'G�d�������)&c�0����{Z�A&�UL����V�jH�QOFkG��X����"=�5Dt��:�,^A����Z�;~���������k'xu�g	,�Bu���iM7�yWQ�_�=�@Ys�	��������W�S�![-�w:=5��~���[����Ey����>��9�������������K�N#�9i��x
�g�IE��w�5xW�kp������;����H��s+!)��W��J������6i2U�X,��h��&�`�����G��7'x�~��������uU�:,���;�����#���o���J��$�M�A-Vg��'+*\S:!~rt�_�b#&%��9���}�`��d�	�qbw0���& R|��}K�dWf`��C�bf�E��H���BJ5aH���'~>��� ;�E�|����4_�ks��?�.��2|�����Rp@o�oK���tlJh�L��_5=o	qj��[�%[�����;HR��%�p�d���3����
����k��A�mk�:�wgd �I��k�"J��DL�~A[S�_Y~e������7������w�r��;}^���^���pM�v8	]�<��A0+p]��5�P�>��Bat+�
>��!/"�������.{X|��H�P�A��������n��$/��t���������v�A9|��Q��%�pS$e�����Q1��Kd�awJu��L�n{��F;��D������!c�H�m����t�����j����j��6��a���y��`�B$�����e���AHh���5,���h�`O#�n�_���|>�_0�8�o��qh�s/��;_�D	���bD�P4*�|����v��F;c������8�G�s�72�s>5�Sh���s��:�d����~<���_{!���%��o�#��&JK���4��tKOlkL�������	y��j:�C����X��!D����N5�m.���S[6y?�����^oG���R�f�QY�����rlx\��k\z����[fzU(�9���c���e��)����N��o�9��s���n�����:���gLvP��M�6��N]8�=2���J�(Q�D�%J�(Q�D�%J�tV�@a
#54Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Thomas Munro (#53)
1 attachment(s)
Re: [HACKERS] Parallel Hash take II

On Mon, Nov 27, 2017 at 10:25 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Thu, Nov 23, 2017 at 12:36 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

Here's a new patch set with responses to the last batch of review comments.

Rebased on top of the recent SGML->XML change.

Andres asked me off-list how I tested the barrier.c case where a
backend detaches, releasing other waiters. There are special cases in
BarrierArriveAndWait() and BarrierDetach() for that to make sure that
the phase advances and waiters are released if they were only waiting
for this one backend to arrive, and that exactly one of them is
"elected" for any serial work. Normally the last to arrive is elected
(see earlier discussion about why that's a good idea), but if the one
that would be last detaches instead of arriving then we'll have to
elect someone else. Here is a throw-away test harness that can be
used to exercise that case. CREATE EXTENSION test_barrier; SELECT
test_barrier_detach_releases(1);. Adding a sleep before BarrierDetach
can be used to influence the race, and adding elog messages to
barrier.c can be used to see when the special path is taken.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

test-barrier.txttext/plain; charset=US-ASCII; name=test-barrier.txtDownload
From 4ba3fe2e318dd1a7b54b3b9ae8853531ed3a6e12 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Thu, 30 Nov 2017 14:00:10 +1300
Subject: [PATCH] A simple test module for barrier.c.

Some things to try:

CREATE EXTENSION test_barrier;
SELECT test_barrier_detach_releases(1);
SELECT test_barrier_reattach_random(4, 1000);
SELECT test_barrier_alternate(4, 1000);
---
 src/test/modules/test_barrier/Makefile             |  18 ++
 .../modules/test_barrier/test_barrier--1.0.sql     |  18 ++
 src/test/modules/test_barrier/test_barrier.c       | 270 +++++++++++++++++++++
 src/test/modules/test_barrier/test_barrier.control |   4 +
 4 files changed, 310 insertions(+)
 create mode 100644 src/test/modules/test_barrier/Makefile
 create mode 100644 src/test/modules/test_barrier/test_barrier--1.0.sql
 create mode 100644 src/test/modules/test_barrier/test_barrier.c
 create mode 100644 src/test/modules/test_barrier/test_barrier.control

diff --git a/src/test/modules/test_barrier/Makefile b/src/test/modules/test_barrier/Makefile
new file mode 100644
index 00000000000..9f330b6a45d
--- /dev/null
+++ b/src/test/modules/test_barrier/Makefile
@@ -0,0 +1,18 @@
+# src/test/modules/test_barrier/Makefile
+
+MODULES = test_barrier
+
+EXTENSION = test_barrier
+DATA = test_barrier--1.0.sql
+PGFILEDESC = "test_barrier -- some tests for barrier.c"
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/test_barrier
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/test_barrier/test_barrier--1.0.sql b/src/test/modules/test_barrier/test_barrier--1.0.sql
new file mode 100644
index 00000000000..7077f3f6567
--- /dev/null
+++ b/src/test/modules/test_barrier/test_barrier--1.0.sql
@@ -0,0 +1,18 @@
+\echo Use "CREATE EXTENSION test_barrier" to load this file. \quit
+
+CREATE FUNCTION test_barrier_alternate(workers int, loops int)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+CREATE FUNCTION test_barrier_reattach_random(workers int, end_phase int)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+CREATE FUNCTION test_barrier_detach_releases(workers int)
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+
diff --git a/src/test/modules/test_barrier/test_barrier.c b/src/test/modules/test_barrier/test_barrier.c
new file mode 100644
index 00000000000..98adefa809d
--- /dev/null
+++ b/src/test/modules/test_barrier/test_barrier.c
@@ -0,0 +1,270 @@
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "funcapi.h"
+#include "miscadmin.h"
+#include "pgstat.h"
+#include "postmaster/bgworker.h"
+#include "storage/barrier.h"
+#include "storage/dsm.h"
+#include "storage/proc.h"
+#include "utils/builtins.h"
+#include "utils/resowner.h"
+
+#include <stdlib.h>
+#include <unistd.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(test_barrier_alternate);
+PG_FUNCTION_INFO_V1(test_barrier_reattach_random);
+PG_FUNCTION_INFO_V1(test_barrier_detach_releases);
+
+extern Datum test_barrier_main(Datum);
+
+typedef enum test_mode
+{
+	TEST_MODE_ALTERNATE,
+	TEST_MODE_REATTACH_RANDOM,
+	TEST_MODE_DETACH_RELEASES
+}			test_mode;
+
+typedef struct test_barrier_alternate_state
+{
+	Barrier		barrier1;
+	Barrier		barrier2;
+	int			loops;
+}			test_barrier_alternate_state;
+
+typedef struct test_barrier_reattach_random_state
+{
+	Barrier		barrier;
+	int			end_phase;
+}			test_barrier_reattach_random_state;
+
+typedef struct test_barrier_detach_releases_state
+{
+	Barrier		barrier;
+}			test_barrier_detach_releases_state;
+
+typedef struct test_barrier_state
+{
+	test_mode	mode;
+	union
+	{
+		test_barrier_alternate_state alternate_state;
+		test_barrier_reattach_random_state reattach_random_state;
+		test_barrier_detach_releases_state detach_releases_state;
+	};
+}			test_barrier_state;
+
+/*
+ * Wait at barrier1 and then barrier2, state->loops times.
+ */
+static void
+do_test_barrier_alternate(test_barrier_alternate_state * state)
+{
+	int			i;
+
+	for (i = 0; i < state->loops; ++i)
+	{
+		BarrierArriveAndWait(&state->barrier1, PG_WAIT_EXTENSION);
+		BarrierArriveAndWait(&state->barrier2, PG_WAIT_EXTENSION);
+	}
+}
+
+/*
+ * Attach, wait a random numer of times, then detach, repeatedly until
+ * state->end_phase is reached.
+ */
+static void
+do_test_barrier_reattach_random(test_barrier_reattach_random_state * state)
+{
+	bool		done = false;
+	int			expected_phase;
+	int			i;
+	int			nwaits;
+
+	/* Make sure each backend uses a different pseudo-random sequence. */
+	srand48(getpid());
+
+	while (!done)
+	{
+		expected_phase = BarrierAttach(&state->barrier);
+		nwaits = (int) (lrand48() % 8);
+		for (i = 0; i < nwaits; ++i)
+		{
+			if (expected_phase == state->end_phase)
+			{
+				done = true;
+				break;
+			}
+
+			BarrierArriveAndWait(&state->barrier, PG_WAIT_EXTENSION);
+			++expected_phase;
+			Assert(BarrierPhase(&state->barrier) == expected_phase);
+		}
+		BarrierDetach(&state->barrier);
+	}
+}
+
+/*
+ * Wait twice.  The first time gets us in sync with the leader
+ * process, and the second time we'll be released when the leader detaches.
+ * (In this case we're using a dynamic barrier, but the leader attached for
+ * the workers already which is a bit weird but necessary for this test.)
+ */
+static void
+do_test_barrier_detach_releases(test_barrier_detach_releases_state *state)
+{
+	/* Make sure the leader is here. */
+	BarrierArriveAndWait(&state->barrier, PG_WAIT_EXTENSION);
+	/* Wait for the leader to detach. */
+	BarrierArriveAndWait(&state->barrier, PG_WAIT_EXTENSION);
+}
+
+Datum
+test_barrier_main(Datum arg)
+{
+	dsm_segment *segment;
+	test_barrier_state *state;
+
+	BackgroundWorkerUnblockSignals();
+
+	CurrentResourceOwner = ResourceOwnerCreate(NULL, "test_barrier_main toplevel");
+
+	segment = dsm_attach(DatumGetInt32(arg));
+	state = (test_barrier_state *) dsm_segment_address(segment);
+	switch (state->mode)
+	{
+		case TEST_MODE_ALTERNATE:
+			do_test_barrier_alternate(&state->alternate_state);
+			break;
+		case TEST_MODE_REATTACH_RANDOM:
+			do_test_barrier_reattach_random(&state->reattach_random_state);
+			break;
+		case TEST_MODE_DETACH_RELEASES:
+			do_test_barrier_detach_releases(&state->detach_releases_state);
+			break;
+		default:
+			Assert(0);
+	}
+	dsm_detach(segment);
+
+	return (Datum) 0;
+}
+
+static void
+launch_test(test_mode mode, int workers, int n)
+{
+	BackgroundWorkerHandle **handles;
+	test_barrier_state *state;
+	dsm_segment *segment;
+	int			i;
+
+	handles = palloc(sizeof(BackgroundWorkerHandle *) * workers);
+
+	segment = dsm_create(sizeof(test_barrier_state), 0);
+	state = (test_barrier_state *) dsm_segment_address(segment);
+
+	/* Initialize state. */
+	state->mode = mode;
+	switch (mode)
+	{
+		case TEST_MODE_ALTERNATE:
+			/* Initialize a static barrier for 'workers' workers. */
+			state->alternate_state.loops = n;
+			BarrierInit(&state->alternate_state.barrier1, workers);
+			BarrierInit(&state->alternate_state.barrier2, workers);
+			break;
+		case TEST_MODE_REATTACH_RANDOM:
+			/* Initialize a dynamic barrier.  They'll attach and detach. */
+			state->reattach_random_state.end_phase = n;
+			BarrierInit(&state->reattach_random_state.barrier, 0);
+			break;
+		case TEST_MODE_DETACH_RELEASES:
+			/* Initialize a dynamic barrier. */
+			BarrierInit(&state->detach_releases_state.barrier, 0);
+			/* Attach on behalf of all participants... */
+			for (i = 0; i < 1 + workers; ++i)
+				BarrierAttach(&state->detach_releases_state.barrier);
+			break;
+		default:
+			Assert(0);
+	}
+
+	/* Start workers. */
+	for (i = 0; i < workers; ++i)
+	{
+		BackgroundWorker bgw;
+
+		snprintf(bgw.bgw_name, sizeof(bgw.bgw_name), "worker%d", i);
+		bgw.bgw_flags = BGWORKER_SHMEM_ACCESS;
+		bgw.bgw_start_time = BgWorkerStart_ConsistentState;
+		bgw.bgw_restart_time = BGW_NEVER_RESTART;
+		snprintf(bgw.bgw_library_name, sizeof(bgw.bgw_library_name),
+				 "test_barrier");
+		snprintf(bgw.bgw_function_name, sizeof(bgw.bgw_function_name),
+				 "test_barrier_main");
+		bgw.bgw_main_arg = Int32GetDatum(dsm_segment_handle(segment));
+		bgw.bgw_notify_pid = MyProcPid;
+
+		if (!RegisterDynamicBackgroundWorker(&bgw, &handles[i]))
+			elog(ERROR, "Can't start worker");
+	}
+
+	/* Some tests require the leader to do something. */
+	switch (mode)
+	{
+		case TEST_MODE_DETACH_RELEASES:
+			/* Workers are waiting for us... */
+			BarrierArriveAndWait(&state->detach_releases_state.barrier,
+								 PG_WAIT_EXTENSION);
+			/*
+			 * Workers are still waiting for us, and we'll release them by
+			 * detaching.
+			 */
+			BarrierDetach(&state->detach_releases_state.barrier);
+			break;
+		default:
+			;
+	}
+
+	/* Wait for workers to complete. */
+	for (i = 0; i < workers; ++i)
+		WaitForBackgroundWorkerShutdown(handles[i]);
+
+	dsm_detach(segment);
+}
+
+Datum
+test_barrier_reattach_random(PG_FUNCTION_ARGS)
+{
+	int			workers = PG_GETARG_INT32(0);
+	int			end_phase = PG_GETARG_INT32(1);
+
+	launch_test(TEST_MODE_REATTACH_RANDOM, workers, end_phase);
+
+	return (Datum) 0;
+}
+
+Datum
+test_barrier_alternate(PG_FUNCTION_ARGS)
+{
+	int			workers = PG_GETARG_INT32(0);
+	int			loops = PG_GETARG_INT32(1);
+
+	launch_test(TEST_MODE_ALTERNATE, workers, loops);
+
+	return (Datum) 0;
+}
+
+Datum
+test_barrier_detach_releases(PG_FUNCTION_ARGS)
+{
+	int			workers = PG_GETARG_INT32(0);
+
+	launch_test(TEST_MODE_DETACH_RELEASES, workers, 0);
+
+	return (Datum) 0;
+}
diff --git a/src/test/modules/test_barrier/test_barrier.control b/src/test/modules/test_barrier/test_barrier.control
new file mode 100644
index 00000000000..1cfe3bc8af3
--- /dev/null
+++ b/src/test/modules/test_barrier/test_barrier.control
@@ -0,0 +1,4 @@
+comment = 'test_barrier'
+default_version = '1.0'
+module_pathname = '$libdir/test_barrier'
+relocatable = true
-- 
2.15.0

#55Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#54)
Re: [HACKERS] Parallel Hash take II

On 2017-11-30 14:17:51 +1300, Thomas Munro wrote:

On Mon, Nov 27, 2017 at 10:25 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Thu, Nov 23, 2017 at 12:36 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

Here's a new patch set with responses to the last batch of review comments.

Rebased on top of the recent SGML->XML change.

Andres asked me off-list how I tested the barrier.c case where a
backend detaches, releasing other waiters. There are special cases in
BarrierArriveAndWait() and BarrierDetach() for that to make sure that
the phase advances and waiters are released if they were only waiting
for this one backend to arrive, and that exactly one of them is
"elected" for any serial work. Normally the last to arrive is elected
(see earlier discussion about why that's a good idea), but if the one
that would be last detaches instead of arriving then we'll have to
elect someone else. Here is a throw-away test harness that can be
used to exercise that case. CREATE EXTENSION test_barrier; SELECT
test_barrier_detach_releases(1);. Adding a sleep before BarrierDetach
can be used to influence the race, and adding elog messages to
barrier.c can be used to see when the special path is taken.

Played some with this, and confirmed that that case
works. Pushed. Interestingly my first few experiments with delaying the
master using pg_usleep() were *not* successful because the leader gets
signalled by the workers successfully having started, interrupting the
sleep...

I've not made any meaningful changes, there was a * missing in an empty
comment line, and I think I've added a single comma.

I do wonder if it's maybe worth adding a single graf about error
handling to the intro? It may not be obvious that this requires
installing a hook calling detach.

- Andres

#56Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#53)
Re: [HACKERS] Parallel Hash take II

Hi,

On 2017-11-27 22:25:12 +1300, Thomas Munro wrote:
Pushed 0003-Add-infrastructure-for-sharing-temporary-files-betwe.patch
after minor adjustments (some including conversing with you).

Changes:
- Changed an impossible elog() into an Assert().
- changed SharedFileSet->counter, and the backend static variable, to
uint32. Not impossible that a 32bit int overflows over the course of a
few weeks, and we a) imo shouldn't unnecessarily rely on signed
overflow being defined b) a negative number would look weird, even if
well defined (-fwrapv et al).
- Added a small comment about arbitrary-ness of the 8 in Oid
tablespaces[8].
- pgindent'ed

Questions:
- Right now RemovePgTempFilesInDir() will recurse into appropriately
named directories, and when it recurses it doesn't require the same
name pattern checks. I think that's good, but I think it'd be prudent
to be a bit more paranoid and prevent recursing into symlinked
subdirectories.
- As we don't clean temp files after crash-restarts it isn't impossible
to have a bunch of crash-restarts and end up with pids *and* per-pid
shared file set counters reused. Which'd lead to conflicts. Do we care?

Greetings,

Andres Freund

#57Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#53)
Re: [HACKERS] Parallel Hash take II

On 2017-11-27 22:25:12 +1300, Thomas Munro wrote:

On Thu, Nov 23, 2017 at 12:36 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

Here's a new patch set with responses to the last batch of review comments.

Looking at 0004-Add-shared-tuplestores.patch

Comments:
- I'd rename mutex to lock. Seems quite possible that we end up with shared
lockers too.
- Expand "Simple mechanism for sharing tuples between backends." intro
comment - that doesn't really seem like a meaningful description of
the mechanism. Should probably mention that it's similar to
tuplestores etc...
- I'm still concerned about the chunking mechanism. How about this
sketch of an alternative solution?

Chunks are always the same length. To avoid having to read the length
from disk while holding a lock, introduce continuation chunks which
store the amount of space needed by the overlarge tuple at the
start. The reading process stays largely the same, except that if a
backend reads a chunk that's a continuation, it reads the length of
the continuation and skips ahead. That could mean that more than one
backend read continuation chunks, but the window is small and there's
normally not goign to be that many huge tuples (otherwise things are
slow anyway).
- why are we using a separate hardcoded 32 for sts names? Why not just
go for NAMEDATALEN or such?
- I'd replace most of the "should's" in comments with "need".

Greetings,

Andres Freund

#58Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#56)
Re: [HACKERS] Parallel Hash take II

On Sat, Dec 2, 2017 at 1:55 PM, Andres Freund <andres@anarazel.de> wrote:

Pushed 0003-Add-infrastructure-for-sharing-temporary-files-betwe.patch
after minor adjustments (some including conversing with you).

Thank you!

Questions:
- Right now RemovePgTempFilesInDir() will recurse into appropriately
named directories, and when it recurses it doesn't require the same
name pattern checks. I think that's good, but I think it'd be prudent
to be a bit more paranoid and prevent recursing into symlinked
subdirectories.

That's why it uses lstat(), so that it sees symlinks rather than what
they point to. It only recurses if S_ISDIR(), and it unlinks anything
else. That means that it unlinks any symlinks rather than (say)
following them and deleting stuff outside the temporary directory
tree. Example:

$ mkdir /tmp/foo
$ touch /tmp/foo/canary
$ mkdir -p $PGDATA/base/pgsql_tmp/pgsql_tmpXXX/bar
$ ln -s /tmp/foo $PGDATA/base/pgsql_tmp/pgsql_tmpXXX/foo
$ ls $PGDATA/base/pgsql_tmp/pgsql_tmpXXX
bar foo
$ postgres/bin/postgres -D $PGDATA
... ^C ...
$ ls $PGDATA/base/pgsql_tmp
$ ls /tmp/foo
canary

Make sense?

- As we don't clean temp files after crash-restarts it isn't impossible
to have a bunch of crash-restarts and end up with pids *and* per-pid
shared file set counters reused. Which'd lead to conflicts. Do we care?

We don't care. PathNameCreateTemporaryDir() on a path that already
exists will return silently. PathNameCreateTemporaryFile() on a path
that already exists, as mentioned in a comment and following an
existing convention, will open and truncate it. So either it was
really an orphan and that is a continuation of our traditional
recycling behaviour, or the calling code has a bug and used the same
path twice and it's not going to end well.

Another type of collision we could have in theory is like this: One
backend creates a SharedFileSet, and various other backends attach to
it. The creator detaches and exits. Later another process is created
with the same PID, and creates a new SharedFileSet, and happens to
have the same counter number, but the original SharedFileSet is still
in existence because there are still running backends attached to it.
Then things get ugly. But this seems improbable, at least as long as
the usage pattern is that leader processes are the only ones creating
SharedFileSet objects and outlive their parallel workers in non-crash
outcomes, but if you think that's too flimsy I suppose it could be
fixed by replacing the per-backend counter variable with a
cluster-wide atomic counter. Then the PID component would be
redundant but I'd suggest keeping it anyway for its diagnostic value.
Thoughts?

Just a reminder: a couple of problems have come up recently in the
Parallel Hash Join patch itself, so please don't consider that one
ready for commit quite yet. They are: (1) Handling the case where
there is no DSA area because we're running a parallel-aware plan in
non-parallel mode due to lack of resources; (2) Investigating a rare
assertion failure. For (1), that may depend on another patch that
I'll post shortly to kill "es_query_dsa" and, come to think of it, for
(2) it's possible that the problem is in either one of the remaining
patches -- SharedTuplestore or Parallel Hash Join -- so please hold
off on committing either of those until I've got to the bottom of
that.

--
Thomas Munro
http://www.enterprisedb.com

#59Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Thomas Munro (#58)
Re: [HACKERS] Parallel Hash take II

On Sat, Dec 2, 2017 at 3:54 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Sat, Dec 2, 2017 at 1:55 PM, Andres Freund <andres@anarazel.de> wrote:

- As we don't clean temp files after crash-restarts it isn't impossible
to have a bunch of crash-restarts and end up with pids *and* per-pid
shared file set counters reused. Which'd lead to conflicts. Do we care?

We don't care. PathNameCreateTemporaryDir() on a path that already
exists will return silently. PathNameCreateTemporaryFile() on a path
that already exists, as mentioned in a comment and following an
existing convention, will open and truncate it. So either it was
really an orphan and that is a continuation of our traditional
recycling behaviour, or the calling code has a bug and used the same
path twice and it's not going to end well.

On further reflection, there are problems with that higher up. (1)
Even though PathNameCreateTemporaryFile() will happily truncate and
reuse an orphaned file when BufFileCreateShared() calls it,
BufFileOpenShared() could get confused by the orphaned files. It
could believe that XXX.1 is a continuation of XXX.0, when in fact it
is junk left over from a crash restart. Perhaps BufFileCreateShared()
needs to delete XXX.{N+1} if it exists, whenever it creates XXX.{N}.
That will create a gap in the series of existing files that will cause
BufFileOpenShared()'s search to terminate. (2) BufFileOpenShared()
thinks that the absence of an XXX.0 file means there is no BufFile by
this name, when it could mistakenly open pre-crash junk due to a
colliding name. I use that condition as information, but I think I
can fix that easily by using SharedTuplestoreParticipant::npage == 0
to detect that there should be no file, rather than trying to open it,
and then I can define this problem away by declaring that
BufFileOpenShared() on a name that you didn't call
BufFileCreateShared() on invokes undefined behaviour. I will make it
so.

--
Thomas Munro
http://www.enterprisedb.com

#60Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#58)
Re: [HACKERS] Parallel Hash take II

On 2017-12-02 15:54:29 +1300, Thomas Munro wrote:

On Sat, Dec 2, 2017 at 1:55 PM, Andres Freund <andres@anarazel.de> wrote:

- Right now RemovePgTempFilesInDir() will recurse into appropriately
named directories, and when it recurses it doesn't require the same
name pattern checks. I think that's good, but I think it'd be prudent
to be a bit more paranoid and prevent recursing into symlinked
subdirectories.

That's why it uses lstat(), so that it sees symlinks rather than what
they point to. It only recurses if S_ISDIR(), and it unlinks anything
else.

Right. I'd somehow confused myself by thinking one'd need an explicit
S_ISLINK check...

Just a reminder: a couple of problems have come up recently in the
Parallel Hash Join patch itself, so please don't consider that one
ready for commit quite yet. They are: (1) Handling the case where
there is no DSA area because we're running a parallel-aware plan in
non-parallel mode due to lack of resources; (2) Investigating a rare
assertion failure. For (1), that may depend on another patch that
I'll post shortly to kill "es_query_dsa" and, come to think of it, for
(2) it's possible that the problem is in either one of the remaining
patches -- SharedTuplestore or Parallel Hash Join -- so please hold
off on committing either of those until I've got to the bottom of
that.

I'm a bit tempted to press ahead regardless of these issues. With your
consent obviously. ISTM we're pretty close to the point where this needs
to be exposed more widely and that'll surely bring more issues to light.

Greetings,

Andres Freund

#61Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Thomas Munro (#59)
1 attachment(s)
Re: [HACKERS] Parallel Hash take II

On Sat, Dec 2, 2017 at 4:46 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Sat, Dec 2, 2017 at 3:54 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

On Sat, Dec 2, 2017 at 1:55 PM, Andres Freund <andres@anarazel.de> wrote:

- As we don't clean temp files after crash-restarts it isn't impossible
to have a bunch of crash-restarts and end up with pids *and* per-pid
shared file set counters reused. Which'd lead to conflicts. Do we care?

We don't care. PathNameCreateTemporaryDir() on a path that already
exists will return silently. PathNameCreateTemporaryFile() on a path
that already exists, as mentioned in a comment and following an
existing convention, will open and truncate it. So either it was
really an orphan and that is a continuation of our traditional
recycling behaviour, or the calling code has a bug and used the same
path twice and it's not going to end well.

On further reflection, there are problems with that higher up. (1)
Even though PathNameCreateTemporaryFile() will happily truncate and
reuse an orphaned file when BufFileCreateShared() calls it,
BufFileOpenShared() could get confused by the orphaned files. It
could believe that XXX.1 is a continuation of XXX.0, when in fact it
is junk left over from a crash restart. Perhaps BufFileCreateShared()
needs to delete XXX.{N+1} if it exists, whenever it creates XXX.{N}.
That will create a gap in the series of existing files that will cause
BufFileOpenShared()'s search to terminate. (2) BufFileOpenShared()
thinks that the absence of an XXX.0 file means there is no BufFile by
this name, when it could mistakenly open pre-crash junk due to a
colliding name. I use that condition as information, but I think I
can fix that easily by using SharedTuplestoreParticipant::npage == 0
to detect that there should be no file, rather than trying to open it,
and then I can define this problem away by declaring that
BufFileOpenShared() on a name that you didn't call
BufFileCreateShared() on invokes undefined behaviour. I will make it
so.

Here is a patch to deal with that problem. Thoughts?

I suppose if we wanted to make this type of problem go away, but still
keep files for forensic purposes, we could introduce a "restart
number", and stick it into the top level temporary directory's name.
That way you'd never be able to collide with files created before a
crash-restart, and we could add O_EXCL so it'd become an error to try
to create the same filename again.

I'll post a new SharedTuplestore and Parallel Hash patch set shortly.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

0001-Add-defenses-against-pre-crash-files-to-BufFileOpenS.patchapplication/octet-stream; name=0001-Add-defenses-against-pre-crash-files-to-BufFileOpenS.patchDownload
From c673efa07f290f9781af3b0ba1399534da762b66 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Tue, 5 Dec 2017 17:14:51 +1300
Subject: [PATCH 1/5] Add defenses against pre-crash files to
 BufFileOpenShared().

Crash restarts currently don't clean up temporary files, as a debugging aid.
If a left-over file happens to have the same name as a segment file we're
trying to create, we'll just truncate and reuse it, but there is a problem:
BufFileOpenShared() determines how many segment files exist by trying to open
.0, .1, .2, ... until it finds no more files.  It might be confused by a junk
file that has the next segment number.  To defend against that, make sure we
always create a gap after the end file by unlinking the following name if it
exists.  Also make it an error to try to open a BufFile that doesn't exist
(has no segment 0), so as not to encourage the development of client code
that depends on an interface that we can't reliably provide.

Author: Thomas Munro
Reviewed-By: Andres Freund
Discussion: https://postgr.es/m/CAEepm%3D2jhCbC_GFQJaaDhWxLB4EXtT3vVd5czuRNaqF5CWSTog%40mail.gmail.com
---
 src/backend/storage/file/buffile.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index fa9940da9b3..c6b210d1379 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -211,6 +211,16 @@ MakeNewSharedSegment(BufFile *buffile, int segment)
 	char		name[MAXPGPATH];
 	File		file;
 
+	/*
+	 * It is possible that there are files left over from before a crash
+	 * restart with the same name.  In order for BufFileOpenShared()
+	 * not to get confused about how many segments there are, we'll unlink
+	 * the next segment number if it already exists.
+	 */
+	SharedSegmentName(name, buffile->name, segment + 1);
+	SharedFileSetDelete(buffile->fileset, name, true);
+
+	/* Create the new segment. */
 	SharedSegmentName(name, buffile->name, segment);
 	file = SharedFileSetCreate(buffile->fileset, name);
 
@@ -303,7 +313,9 @@ BufFileOpenShared(SharedFileSet *fileset, const char *name)
 	 * name.
 	 */
 	if (nfiles == 0)
-		return NULL;
+		ereport(ERROR,
+				(errcode_for_file_access(),
+				 errmsg("could not open BufFile \"%s\"", name)));
 
 	file->numFiles = nfiles;
 	file->files = files;
-- 
2.15.0

#62Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Thomas Munro (#61)
Re: [HACKERS] Parallel Hash take II

On Fri, Dec 8, 2017 at 12:07 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:

I suppose if we wanted to make this type of problem go away, but still
keep files for forensic purposes, we could introduce a "restart
number", and stick it into the top level temporary directory's name.
That way you'd never be able to collide with files created before a
crash-restart, and we could add O_EXCL so it'd become an error to try
to create the same filename again.

Or we could teach crash-restart to move the top level directory (in
each tablespace) to pgsql_tmp.old, so we'd keep the temporary files
from one previous lifetime only. That'd prevent unlimited space
eating in multiple crash scenarios, and we could more comfortably say
that it's entirely safe to delete that directory manually in cases
like this:

/messages/by-id/4f3c89a224ff4660baa62a2b79fb0f1d@ITUPW-EXMBOX3B.UniNet.unisa.edu.au

--
Thomas Munro
http://www.enterprisedb.com

#63Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#61)
Re: [HACKERS] Parallel Hash take II

On 2017-12-08 12:07:04 +1300, Thomas Munro wrote:

I suppose if we wanted to make this type of problem go away, but still
keep files for forensic purposes, we could introduce a "restart
number", and stick it into the top level temporary directory's name.
That way you'd never be able to collide with files created before a
crash-restart, and we could add O_EXCL so it'd become an error to try
to create the same filename again.

I'm deeply unconvinced by the "forensic" argument to not do temp file
cleanup after crash restarts. That causes problems like the one we're
debating upthread in the first place, so I'm wholly unconvinced we
should add to that further by adding another layer of complexity.

My personal opinion is that we should just do temp file cleanup after
crash restarts, and document restart_after_crash = false as the solution
for investigating crashes. I don't want to hold up this patch with a
discussion of that however, so I'm ok with your fix.

Greetings,

Andres Freund

#64Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#63)
1 attachment(s)
Re: [HACKERS] Parallel Hash take II

On Sat, Dec 2, 2017 at 3:46 PM, Andres Freund <andres@anarazel.de> wrote:

Looking at 0004-Add-shared-tuplestores.patch

Comments:
- I'd rename mutex to lock. Seems quite possible that we end up with shared
lockers too.

Done.

- Expand "Simple mechanism for sharing tuples between backends." intro
comment - that doesn't really seem like a meaningful description of
the mechanism. Should probably mention that it's similar to
tuplestores etc...

Done.

- I'm still concerned about the chunking mechanism. How about this
sketch of an alternative solution?

Chunks are always the same length. To avoid having to read the length
from disk while holding a lock, introduce continuation chunks which
store the amount of space needed by the overlarge tuple at the
start. The reading process stays largely the same, except that if a
backend reads a chunk that's a continuation, it reads the length of
the continuation and skips ahead. That could mean that more than one
backend read continuation chunks, but the window is small and there's
normally not goign to be that many huge tuples (otherwise things are
slow anyway).

Done.

I've also included a simple test harness that can be used to drive
SharedTuplestore independently of Parallel Hash, but that patch is not
for commit. See example of usage in the commit message.
(Incidentally I noticed that ParallelWorkerNumber is not marked
PGDLLIMPORT so that fails to build on Windows CI.)

- why are we using a separate hardcoded 32 for sts names? Why not just
go for NAMEDATALEN or such?

Done.

- I'd replace most of the "should's" in comments with "need".

Done.

Another problem I discovered is that v27's way of installing a
different function into ExecProcNode in ExecInitHashJoin() was broken:
it didn't allow for the possibility that there is no DSA area
available due to lack of resources. ExecInitNode() is too soon to
decide. My solution is to provide a way for executor nodes to change
their ExecProcNode functions at any later time, which requires a way
for execProcnode.c to redo any wrapper functions.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

parallel-hash-v28.patchset.tgzapplication/x-gzip; name=parallel-hash-v28.patchset.tgzDownload
#65Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#64)
Re: [HACKERS] Parallel Hash take II

Hi,

Looking at the latest version of the tuplestore patch:

diff --git a/src/backend/utils/sort/sharedtuplestore.c b/src/backend/utils/sort/sharedtuplestore.c
new file mode 100644
index 00000000000..d1233221a58
--- /dev/null
+++ b/src/backend/utils/sort/sharedtuplestore.c
@@ -0,0 +1,583 @@
+/*-------------------------------------------------------------------------
+ *
+ * sharedtuplestore.c
+ *	  Simple mechanism for sharing tuples between backends.
+ *
+ * This module provides a shared temporary tuple storage mechanism, providing
+ * a parallel-aware subset of the features of tuplestore.c.  Multiple backends
+ * can write to a SharedTuplestore, and then multiple backends can later scan
+ * the stored tuples.  Currently, the only scan type supported is a parallel
+ * scan where each backend reads an arbitrary subset of the tuples that were
+ * written.

Cool.

+/* Chunk written to disk. */
+typedef struct SharedTuplestoreChunk
+{
+	int			ntuples;		/* Number of tuples in this chunk. */
+	bool		overflow;		/* Continuation of previous chunk? */
+	char		data[FLEXIBLE_ARRAY_MEMBER];
+} SharedTuplestoreChunk;

Ah. I was thinking we could have the 'overflow' variable be an int,
indicating the remaining length of the oversized tuple. That'd allow us
to skip ahead to the end of the oversized tuple in concurrent processes
after hitting it.

+/*
+ * Write a tuple.  If a meta-data size was provided to sts_initialize, then a
+ * pointer to meta data of that size must be provided.
+ */
+void
+sts_puttuple(SharedTuplestoreAccessor *accessor, void *meta_data,
+			 MinimalTuple tuple)
+{
+	size_t		size;
+
+	/* Do we have our own file yet? */
+	if (accessor->write_file == NULL)
+	{
+		SharedTuplestoreParticipant *participant;
+		char		name[MAXPGPATH];
+
+		/* Create one.  Only this backend will write into it. */
+		sts_filename(name, accessor, accessor->participant);
+		accessor->write_file = BufFileCreateShared(accessor->fileset, name);
+
+		/* Set up the shared state for this backend's file. */
+		participant = &accessor->sts->participants[accessor->participant];
+		participant->writing = true;	/* for assertions only */
+	}
+
+	/* Do we have space? */
+	size = accessor->sts->meta_data_size + tuple->t_len;
+	if (accessor->write_pointer + size >= accessor->write_end)
+	{
+		if (accessor->write_chunk == NULL)
+		{
+			/* First time through.  Allocate chunk. */
+			accessor->write_chunk = (SharedTuplestoreChunk *)
+				MemoryContextAllocZero(accessor->context,
+									   STS_CHUNK_PAGES * BLCKSZ);
+			accessor->write_chunk->ntuples = 0;
+			accessor->write_pointer = &accessor->write_chunk->data[0];
+			accessor->write_end = (char *)
+				accessor->write_chunk + STS_CHUNK_PAGES * BLCKSZ;
+		}
+		else
+		{
+			/* See if flushing helps. */
+			sts_flush_chunk(accessor);
+		}
+
+		/* It may still not be enough in the case of a gigantic tuple. */
+		if (accessor->write_pointer + size >= accessor->write_end)
+		{
+			size_t		written;
+
+			/*
+			 * We'll write the beginning of the oversized tuple, and then
+			 * write the rest in some number of 'overflow' chunks.
+			 */
+			if (accessor->write_pointer + accessor->sts->meta_data_size >=
+				accessor->write_end)
+				elog(ERROR, "meta-data too long");

That seems more like an Assert than a proper elog? Given that we're
calculating size just a few lines above...

+			if (accessor->sts->meta_data_size > 0)
+				memcpy(accessor->write_pointer, meta_data,
+					   accessor->sts->meta_data_size);
+			written = accessor->write_end - accessor->write_pointer -
+				accessor->sts->meta_data_size;
+			memcpy(accessor->write_pointer + accessor->sts->meta_data_size,
+				   tuple, written);

Also, shouldn't the same Assert() be here as well if you have it above?

+static MinimalTuple
+sts_read_tuple(SharedTuplestoreAccessor *accessor, void *meta_data)
+{
+	MinimalTuple tuple;
+	uint32		size;
+	size_t		remaining_size;
+	size_t		this_chunk_size;
+	char	   *destination;
+
+	/*
+	 * We'll keep track of bytes read from this chunk so that we can detect an
+	 * overflowing tuples and switch to reading overflow pages.
+	 */
+	if (accessor->sts->meta_data_size > 0)
+	{
+		if (BufFileRead(accessor->read_file,
+						meta_data,
+						accessor->sts->meta_data_size) !=
+			accessor->sts->meta_data_size)
+			ereport(ERROR,
+					(errcode_for_file_access(),
+					 errmsg("could not read from shared tuplestore temporary file"),
+					 errdetail("Short read while reading meta-data")));

The errdetail doesn't follow the style guide (not a sentence ending with
.), and seems internal-ish. I'm ok with keeping it, but perhaps we
should change all these to be errdetail_internal()? Seems pointless to
translate all of them.

+MinimalTuple
+sts_parallel_scan_next(SharedTuplestoreAccessor *accessor, void *meta_data)
+{
+	SharedTuplestoreParticipant *p;
+	BlockNumber	read_page;
+	bool		eof;
+
+	for (;;)
+	{
+		/* Can we read more tuples from the current chunk? */
+		if (accessor->read_ntuples < accessor->read_ntuples_available)
+			return sts_read_tuple(accessor, meta_data);
+
+		/* Find the location of a new chunk to read. */
+		p = &accessor->sts->participants[accessor->read_participant];
+
+		LWLockAcquire(&p->lock, LW_EXCLUSIVE);
+		eof = p->read_page >= p->npages;
+		if (!eof)
+		{
+			read_page = p->read_page;
+			p->read_page += STS_CHUNK_PAGES;
+		}
+		LWLockRelease(&p->lock);

So if we went to the world I'm suggesting, with overflow containing the
length till the end of the tuple, this'd probably would have to look a
bit different.

+		if (!eof)
+		{
+			SharedTuplestoreChunk chunk_header;
+
+			/* Make sure we have the file open. */
+			if (accessor->read_file == NULL)
+			{
+				char		name[MAXPGPATH];
+
+				sts_filename(name, accessor, accessor->read_participant);
+				accessor->read_file =
+					BufFileOpenShared(accessor->fileset, name);
+				if (accessor->read_file == NULL)
+					elog(ERROR, "could not open temporary file %s", name);

Isn't this more an Assert or just not anything? There's now way
BufFileOpenShared should ever return NULL, no?

+
+			/* Seek and load the chunk header. */
+			if (BufFileSeekBlock(accessor->read_file, read_page) != 0)
+				ereport(ERROR,
+						(errcode_for_file_access(),
+						 errmsg("could not read from shared tuplestore temporary file"),
+						 errdetail("Could not seek to next block")));
+			if (BufFileRead(accessor->read_file, &chunk_header,
+							offsetof(SharedTuplestoreChunk, data)) !=
+				offsetof(SharedTuplestoreChunk, data))
+				ereport(ERROR,
+						(errcode_for_file_access(),
+						 errmsg("could not read from shared tuplestore temporary file"),
+						 errdetail("Short read while reading chunk header")));
+
+			/* If this is an overflow chunk, we skip it. */
+			if (chunk_header.overflow)
+				continue;
+
+			accessor->read_ntuples = 0;
+			accessor->read_ntuples_available = chunk_header.ntuples;
+			accessor->read_bytes = offsetof(SharedTuplestoreChunk, data);

Perhaps somewhere around here comment that we'll just loop around and
call sts_read_tuple() in the next loop iteration?

Greetings,

Andres Freund

#66Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#65)
1 attachment(s)
Re: [HACKERS] Parallel Hash take II

On Thu, Dec 14, 2017 at 11:45 AM, Andres Freund <andres@anarazel.de> wrote:

+       bool            overflow;               /* Continuation of previous chunk? */
+       char            data[FLEXIBLE_ARRAY_MEMBER];
+} SharedTuplestoreChunk;

Ah. I was thinking we could have the 'overflow' variable be an int,
indicating the remaining length of the oversized tuple. That'd allow us
to skip ahead to the end of the oversized tuple in concurrent processes
after hitting it.

Right, that is a bit better as it avoids extra read-skip cycles for
multi-overflow-chunk cases. Done that way.

+                       if (accessor->write_pointer + accessor->sts->meta_data_size >=
+                               accessor->write_end)
+                               elog(ERROR, "meta-data too long");

That seems more like an Assert than a proper elog? Given that we're
calculating size just a few lines above...

It's an error because the logic is not smart enough to split the
optional meta-data and tuple size over multiple chunks. I have added
comments there to explain. That error can be reached by CALL
test_sharedtuplestore(1, 1, 1, 32756, 1), but 32755 is OK. My goal
here is to support arbitrarily large tuples, not arbitrarily large
per-tuple meta-data, since for my use case I only need 4 bytes (a hash
value). This could be improved if required by later features
(probably anyone wanting more general meta-data would want variable
sized meta-data anyway, whereas this is fixed, and it would also be
nice if oversized tuples didn't have to start at the beginning of a
new chunk).

I fixed two nearby fencepost bugs: I made the limit that triggers that
error smaller by size(uint32) and fixed a problem when small tuples
appear after an oversize tuple in a final overflow chunk (found by
hacking the test module to create mixtures of different sized tuples).

+                       if (accessor->sts->meta_data_size > 0)
+                               memcpy(accessor->write_pointer, meta_data,
+                                          accessor->sts->meta_data_size);
+                       written = accessor->write_end - accessor->write_pointer -
+                               accessor->sts->meta_data_size;
+                       memcpy(accessor->write_pointer + accessor->sts->meta_data_size,
+                                  tuple, written);

Also, shouldn't the same Assert() be here as well if you have it above?

No, when it comes to the tuple we just write as much of it as will
fit, and write the rest in the loop below. Comments improved to make
that clear.

+                       ereport(ERROR,
+                                       (errcode_for_file_access(),
+                                        errmsg("could not read from shared tuplestore temporary file"),
+                                        errdetail("Short read while reading meta-data")));

The errdetail doesn't follow the style guide (not a sentence ending with
.), and seems internal-ish. I'm ok with keeping it, but perhaps we
should change all these to be errdetail_internal()? Seems pointless to
translate all of them.

Done.

+               LWLockAcquire(&p->lock, LW_EXCLUSIVE);
+               eof = p->read_page >= p->npages;
+               if (!eof)
+               {
+                       read_page = p->read_page;
+                       p->read_page += STS_CHUNK_PAGES;
+               }
+               LWLockRelease(&p->lock);

So if we went to the world I'm suggesting, with overflow containing the
length till the end of the tuple, this'd probably would have to look a
bit different.

Yeah. I almost wanted to change it back to a spinlock but now it's
grown bigger again...

+               if (!eof)
+               {
+                       SharedTuplestoreChunk chunk_header;
+
+                       /* Make sure we have the file open. */
+                       if (accessor->read_file == NULL)
+                       {
+                               char            name[MAXPGPATH];
+
+                               sts_filename(name, accessor, accessor->read_participant);
+                               accessor->read_file =
+                                       BufFileOpenShared(accessor->fileset, name);
+                               if (accessor->read_file == NULL)
+                                       elog(ERROR, "could not open temporary file %s", name);

Isn't this more an Assert or just not anything? There's now way
BufFileOpenShared should ever return NULL, no?

Right. As of commit 923e8dee this can no longer return NULL (instead
it would raise an error), so I removed this redundant check.

+                       /* If this is an overflow chunk, we skip it. */
+                       if (chunk_header.overflow)
+                               continue;
+
+                       accessor->read_ntuples = 0;
+                       accessor->read_ntuples_available = chunk_header.ntuples;
+                       accessor->read_bytes = offsetof(SharedTuplestoreChunk, data);

Perhaps somewhere around here comment that we'll just loop around and
call sts_read_tuple() in the next loop iteration?

Done.

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

parallel-hash-v29.patchset.tgzapplication/x-gzip; name=parallel-hash-v29.patchset.tgzDownload
�.2Z��{[��8|�E���dma$��co0`����&9��iF0�����������K_gzt�xO��O������TW���A��yd��O�{�� ��fQ��_���j�v��������7�_�#�������V{S������%�mS~&Y�0��d�&S�A�~�{9�����z�a��a6�� ��f>�,O�(�F�~monV��v���c����������o9q������N�!,���N�����o����V�������v7\���v�����d$���h��V��O����q}��L�������*-��Q��4�����K�/k�A����!�[u�p�7^��_l���6@�]M���z�����������������F�Z��^�gp;���p2�cx(�����Q��<wi�G"��q.���F"����I���>�>.��GB��U!N��-��R�����=}&����#{��b���Ww�}&&4������Nr��GS�����xt#���
qh���1��H�"��L��HD�1�"��avr�b��n��O0�Z�O�Fw8��J�
�]F������� ��M���F7��!G!���&��!.�n���md��8�M�,NF/�m���kk�$�o��([��D����������>�n��_��E3|�!j��\�'������8}����S����� ���ab�f�YY�[���F�nwp�
�7���n��>������]��]&y<���$���#�!�]��}
����%��W��0�x�L���G|+�C[��!�~��l�}g�K�n��M{������>��d��e8y"�I��zs�V�~_4�7��k���;�M
�5�,6��[= �;���jt�v�p���Mw��<_�������(�[����X�v<��n��������u�z�QV����rM,��O�X"�v� %i���� z��-��#���x}yxv���suruuz~������������w'
��~<L,�+�!�=�<�.��\]�_��P�cpa��^�z�O��p����������^\��CWE;�x;A`�1�w���ZH���iu���(��������n�[��6�����;�V�9����?�_�S<��z��;�I<�8]&H���� �X�|��4��G����/D�t���:���K���5�_��J�Ar��h58�l2�?�o��;���R�X�s���[�^�_��*8P����&V�x�mm��p<"��������j�;��Q��[��_�Ok��`0}�=_��n5�`�H���������c��V�s�yWz��U���FH��l(��'`k����w0%rF�M(���q�k5�d�@�E�Y�h{��HHJi}l�&��>E���4�;� �I�d5��Gr�z�3��b��	�'���*E��a��a�?��r �C�p�Q1}���I�g9�o��d4��H,���:��5;��S�{�_&.
�-��8��s�AnJ~���4�f���\$|���d|��7�����E{oo��lnZ C]��;��h�8�>E�d<�	�7i2�����+-BNV���8�4��{|rb@�Q�a���]��>=:���J�`��T	�Q��Zm���{Eb�0�{{��}�zp{�������M'���B�!��A8�G���u@vi���b��/y�����B�*F�����x��p����
��3�m��NFA< ������=�/K��"�B��p�%tsp$���Wo;Go?�����/���=�I���b��9�L&�sc��c�D$��
�[��@2,�Y�3��	��@13��$��J��}��(W�W<X�>��\�M���'��'��������0��_/�#AC�A,��^2�z�����wG���h�?���&����l�J�Aj���Nzy�R�������*S7���_Z�g�!�VfrDZU�����G^�x��AC���9F������$���B���+����N~:��sxyy�������N.F����r	.��iI��"�r���V��t��`�ti	� 2���^�%$�<<���F�Xx����9�s�Dv��$��.0=����u�d����`��K2��5�� S�(��5+�Rx���N��8�p�Y��;,����X3ql��~qr��?�/h����F-�k�]i�X�QC:?���E���C��D�>��J��1`s"���X��n��g��O�$�;9��F(�9-3a�EF��A�LP������LG��)��,��cP�����N����q0��������Z��yoU�}����,������y
��\��uf���o��j&!��iF������_�g������=�����xd������k@g�@g1��b{BYu�'��R���`�����=<��AH��	>�����C��$�B]�*�,{��"`,���1#0��\]
=��N�^
�y����A�F
Y�}��A��f���@�wpD�\t����`�8����������V`7_����uh��"��L\6.p����D�5���Dh�H��<}����I�ZqZ)���������hT����}>B�vF��@IF�j:&_H��@���V�m� ����J�$��7�D	d��C��oq���#M0L&#�����IL�]B���a�j	�
P��u�l��8H�S�d����c�����<�J���v��t<m��d��\�������b4��X���vD�����
 �A�����8��
9�u�M��[HW��ke�1������/����7�Pxu�r6�����d#R�'i"���A
lRM9-�g���[�(�F������kRA����J��d�`wI���DG=b-��g��`1C�\H��c���b�64�<�#c0H`i�|������������@��D�C�;).�]���?��JSN/�%f�
	O�� ���EK��e����{���m����������)0��r�����
�`c��E0��!y������++JLF�/�^�d
s%��v���%@g��,V�>��>	��D�K�H\2�~�:�{�3�0]�-��-u��a�O���~�7�RE!%	�M&�5i\:$������%9��������A��p#w������vLx	h�}Q�.�:�N������YS��q=�ArS?��<�l�'��#���D�Ac+@���
�5�WH��1��/������J�_���������a���W��6F�)�����0i\��${��2Z]"�Kj�i�P!��W\
_�i4���)�E��2B��q�<�h�C����Z���
q�y�>yKZ�l�f�y�Z�����P�(�G�.pq��N[0�6o�|�I��J5��*1�H�$g�����&�AVfcfP�J��}��']���0��Ky���Iv��}�4���P���w�o�����c���)}���r���xu3S#B4D�i�����eE^���s��i��
IL�j��4�%!�NR�a��e�@@�avS��#cm1���B|7|���\$lVi�r�-k��&J&9O���/Iw����?��E��WY''Vpz�-)P��@C��R�W���&�b��� /�,K�}r7��N}�B1Z��nt��p)��X35=�Y��a~�D<�a"���w��eWD
�n#��G�$�c$7�����r�+�C�6��z�����K���8(�>�u����tMF�o8�6]����x��)L�w
6p�'*��2mtG�4�����B9H�1i��z�NSRu��
�'���<4<��2����Qe�����g���m�����@�*I���1�k,.<c���U)�x��eM�)<v5���*n���d_2zD��oI��za��)-�8��.�2(9&�L��
I����d��
�
t�#E-5F�!�d��:,��N�F@�6������
�"�Y@�Q5�9
�}�b�_!��>����&2��>��������(.�t>\��������%P���wU�+NP ��6z2��4��m�Z(��;0�DYxshU�9$S6+*%&7Lb�J��t�u��������I$����G "��@3$Vc<��x�o��U�_�-������D��T?2j��>�E2����t/<����WzM��$��r(��'���^�[�@F��O0
��Dpf�f�E�;,�����{���[��G�s�
I��)����5�j&��g���~��Q����.�j��$�b�Ep��P�u'���g!��n���l!Z��W�Q�>�R��a�a�
X~'q���}�����L��;��_��Stw��3J��.Gbv���T��C����B�,C
+�Tp�}��(���bg%V�j����Vl*�@��H��
� *�etOr�X�`�s�a����x���S�&L{�qC��p���n���>��m���vh�u�y�������?]��8�~�3�����C�K�^�@NT����7��cO`U�Y0/	�s�s��px��b(��&���|>��J�mJ�<�e��j�.�{�8o�_���af7O'l
s=�����xp�������>��
�\�e�����uTV�\�<(	��:�|��`P���Q>N������-1���X����x����S ��f
Rj!�P�x+�}FA�`����Y�
�:�����U�r��uY���@B�l9�qExG��x
�����r�}�Ns���?��/��h����=�����&���{�����
�t���%�����z�)�hb��Q4!7�*���*�o�Fk����2F�g�����n��Q����/�_��:�j����w$��"#k����^LF��$
�����y���P�G��c��)�!�Il3�!�����zEhMqc���1J�V�KF��[;�`1�Ke�:m�/O�������59���7�Zfr}+�!w�q�-��a4DcF������
�8�UY�[��" �o[KI��*��S�+�"���G5}���Wi,�:nr�r�++�i��>��9��s�!=��;y�qqQ�U�X����O�l������]\�� �E�:�cH������tV�0�7�G�!P����h"�be�_F��D�1���(4�C��$���<W�%9�#��z�M���/�����/%�L��B!!-y�i�WL�N�G��n*R[�9N��o��ZQ(c������p���o��B�ru���^�O����F�(�'��A��"�>�b�[���E����HZR[1�E��S�+lw�&s��"e�tJ�����H�R ,K�86����4��'�S|c�D���b!��H�\�5���1�@NJ1�H��A�l��k�
�=�1@�)�`�.7s�=����4Th!���H�dx	�}����n�������A�K���&>��l���|g����t-zO0��!�����[��K-�&��X������q�~�6�<����".+S�~P{D���t�%�!���ZL{z���������+ ��Qt������*����+[����O��>�\g�[=Z8��~����y*��q]��1�f���tS���J��@Gq�������k��$�7K���@���������������zT�����������Yg�s/���mdo���?8~�����J�]S2��=��d���Q�}����e~�'2N���"=�`F�o()��U��qh�EVf��xjwlT�����*��?�u�U�3M�S�K���^�U#_()�k�g2�>��]C��2{���(�<�Zc�[)y]Q��h�T�k5������ y(��$�?
�]��]������L�H�#��,���ydT3o&�����@@e)�*S	��`��c�0�.k>.���M|4tN���b���u �� ���
�U{)�
-���?�!m���C�IJ��9Ffe-K����K��M�8W�oME�{�S'/��}��<��V7��!q�E�B`�^�����������^������0�`L��c�?��2���y�8h������2���a����O����2/���9��������H"����P�h��=G���<��GdK�n2�m��X��y)�� ��q�A6a����J��K�[�x�����A��pM��m�Z�|�4=e#�}�4��s.�����(S�����}��>.[\a�ss�|�<*��B�h����u�������|�����E�.�d�A�kP���0��NK��)��l7��X&6�g/��
� �����N���J^6a>��?�p�af�@��)���	�4�V)5T����]�J��',H����q�e���g�H�LU�R�fy��E-Dl�)�k�2G��5�&YM��d�C��U>�h�n��5<�����o�����e���Nxt-�w%���sE<Ns������n����t�N��ueQS��y=�����������"v�#j�a����FZ���v���'�q��ZF�S.���t����7ok<�Sn���-3�:�%���yu�tG��1�$�(�B%/`'g
o�����P�R<��K1���q4N�M_^��Jo�'�e����'���4�,��_�+��v���9G[����F���ll�D�f�c6u��J@&�`e� ����L��e�TY���d[������aF�6�9C����'G��V2�Y�FW���
�`P�A1��������5�\v�O^���WoXe���\��Nm��o��v[;��� g~�����R@��S@�����7M48���|����#<}���;�[+����/�_?�;�_e>���i�|]f���
V��) L7eR~
��I���	��B��YB�!��t"z�l�2�4\��Sb�>��b��Ki��N���V�	�q�zN�0=������f�+��4�7�N9(�������G��
-���h���#�*��8�	3���;�������J\��C��5��R�_{���x��~�����V[���v�����?����7A8�;���yv~�|}~�<:���zV5���_�7��_�y�����?T�ek7��Z{ao�E���
rRkk+h������n�������oP��z�C{x��io���m;�������_�H)) �`���0b�YF�o)��Hl�<��n�f����~�
���D�P�$.5n��Y�j8��m5�BzzHy.�hZ9�Iy�]��'��q��
3��E��R`�^��H�)��?���������8����EB%F���o4�v�+�i`�a@�a!O�N^q��a>-�������\�2��8l}D�MJ��dG�s�J.�kKx�[��V����Z�:��j��mn�������P�Z��%x�@m���7�=K��-9������A�I��lC����D�Z��~��
i��L6;���DC���@���s��8i/FP
"���	���
"2�2�h��dd������i
�K`+�����N����+btPm�a9����~#h��P����{��+dk���
����#�������}��^m�f�D�+�l�i}�O������z�_uh
�U�~�;�M�f������[@�S1h�U�.�cn�P���m����J��"��tD�]��~QAF������1���h"^��g��j��\�y}������?����4��o3�yr��tq
�@(������>}���� S?���?Q��?���.�?�u�e����3f]5u�=<`�H6��1������tTW���1�jO��,��F'$����zu�����a�5g���������oUkP�~4���";��nL�/��?^?AN��Q�~��9b��qqy~tr�������w�8���?��H��pK�k��	>���qx%��9���s�?���;<{�����8�<����`nD���6��V���ztDp-�mEt��/�]d�<����Q����L{�M��5 l�b-��o�������9���o�&W|k���Z��Y��r5A���o�lX�f�&�%����i3��%�������9�����J��<N����K�q���(���f�0�<�?|sz�/���pv�U�:�g��;i����
��"���C�C��Z�����%u����	���	�z�;�$wx�V,�`���C�H8�9�9�������/K8�i�-���zX�'5�������v���������zQ����^�u��N��Y�9����s�o[���o��P������yQ���
��0h 5���XjZZU|��?���	I���q�O�25���V���o�7<��W������U�AB)\��b�A�|���%�5���0dJ�lF�+��v��O5_����hoC�I#u0�m��5T�G,�.��K�
��o�t&� �a�8�^�u{�1������Q���~��)�/�vC<a���!N��7�O��	�[��y�h������M.����N���*��5Z����g���V0Mr6�����7X��]V�|�x�K�!qd/9�%i:����N��N�a�@�M�6Fno��?�H1������]G���(����{<�Y��'��@����}�����t��>eS�9������;1�g�g�r��;�����G��oO1��M����+%����r��.m�r����Sh3�@�IU�u2����Q0F��ag��\z�kla�
y�	�11EL!�B���i��o������ID~�2tP�5�"���Uy�����mXf������&�~-�/ �����Qi���
8��"�Dy�9/��R�J���J���@s!k41���hdYgd����kJ����q�*���!���:&S��S=����)#d�j�5������(�Y�Z3S�i��N�`��KK�A��a-��a�����Q�c�u5q��40�����-���@�a�]T��3��O���%�6�Pl���A:t�4���}	���l�'����2���U��uE6*������Y^�\�����J�0��Y ��9���0������\[�
�9.���^��L��i�
�4���%�G�`O|��`����j�Pe�P�k�q��<[x57VzR��1���U�!��wUO�_*�jE�?=;}��=Q:��__�\���"�<���#��qX����A_:�:����y��B�O;u���zK���f�izjy��a���[�R�kn�(�k���Fy]� ����E.\�k�(�2������c#(,]�p�^�^\�iM��@��w��\��`�J����
_P9{���_/�#�sx�W�Zo�/��@hbE��!W�
�Q$�W��(�)��-7u)K�J�usu6du�
*�|l��h���3e�+����J�-++����+HW�+rIXx�}��UK�Ug[��������2�@�%�=�������+5���!������^w@��eff?�ZYn��m�-Wl���X\X���ne�*����X��d�m�Sw'6S+��>>���������O*E���� �U�}��Q�����_aTu�������6B���\�e5H��.7e�|��p�)�d!$�y�I�����{�����|)�&v���{�^�j�Q�����g(�q��Z����ul����P���t�	�q�n���G���=�T�?xE t�-�s�qOj��[K��9|4�>��z�(���k�
q�������i�Cy�Yt�����%q�Z?��e(��1%�O�������]��d�a�HB��0$����V
Y?�y��N
�}�w��M�	��������Z�����$-�:�4�f���Q���u�E�X��BzOJ%�������#����C�#�B;7`E��c�D~i�`<y;���q(��~$��b��I�^���q^"�5$��������5����-3�Joscg��l����&c��6���K��?��=��V��Ly�B;(5���j^�@�O�o)$
^�a���?
��FD�e4&�=�?���������0l�6��3��B����T�����������)Z�[������[����F����{����N����Yoo��{�������
�����n����Nc`(�*���o�h��,���n��/ �p�����~�H�I/�n%#h��(HQ�x������31#�cT�Q���}����E���tt`V@�
��j:��/��
N�Nw�8�>��$�r����>3O����������(6��l���r@�t��>B_�0B-9�\�~(�@��dj
��	'f[��'B��E�8���8�R��yD���z���*���@��}������9�s�NG	�a����|��D5�t��E��F�EC4��\j���LW�>���}���RY2X���CQ�gv���n��Bc��K�	�hP���hj�[F��(�,wWD�v��6I��8�)NST�Z�����Z
l	���u�2NT���f���M�����V�%DV�0 ��*mQi�!���W����X-�o�y��q�Q.w�q5m�"��}��/y�)�1��x����j�#2
�RWq��BJqm,@��HV������*G\C��V6�N���E����11��1�����h�B]�loj��XE�i��d�u��e�$pU�^�y�Y&\S&]��RY�^�j��\����xDILe�?)��%{I�'Fw�u��|"[�O0��|�����.�t�%��������t��P�X��vg#�2�jf+�U.�v����F�+xd6������9�7��N�����U��J�k�X��R�|�,�$�����)e
e�iJ($��p$��1�O=�5��F��c��|���!$���M���M���o�.��.
���_��H_��U��I7Js �AV�&�?���q ��n|�i���\���q��&�/�m���kk���
��p���$�<������w��K���/���Qk/��:=y�~��O�������:��r6X������;o�|����?����t��z�~#�~�|D�������Qt}�������'��?Lz$`e7��{-���e7x�
�&�|��N
�9X(���m��I ��k���7������������������c��������_��p�����?x`e�Y5~�Rq[{�^�?v�`����0��F#�Q�?���[�mj?d��^2��3J�P�]#j��=���Vi�qy��BGw�t*���_���J�8e��NI_-���<R��T���QC�����k=[�n&���h�J�:*w�_�\���Y@0��P�#4��*|�-��U_,�����)��1��X�,
u�8TF7�M��>7�]�����F����dO7��^�g�1���fia\�&D��t,
T9<�?d�ko�����X����]S�����o��o�%������3�G3����MOG�i�^E�W{sG�'A�-Wm�����*�mq�Qz�,�[U^f��WR!�������l�j����{{�����+*+_��qc{s�"����/��*�O���"��{� �� 4��=y9�;��~�f�{)��~5����
�?�����z�)H�!K���~n?qx��X�y���
{">�Qd�hp����=y)a��a�����/}�i<�V��Gg�������k����%���!������-���o�$e^]���k
���Z��0(�O�����xB�����@
l,=Af�B�� �+�"c5�@oI�TXj���"���s+�z��L�*&�V�3r�0%�b0x��y��UkQ���������w3�+�\���8�h�X��:��C�^jT���b���A�j�R�x�S_K"���������nkuu7���b����" �)6Ab�^��he���#�c��wZ{B�V[�?���6>�����+l�n����I��@�0U��+����C��Q_221����)��;�����7��,�,��[{��K���GG5�����3{�&����a���$���<��J�L���
�G��NTT0����wqg��w���F=����7*�v���O��|�{�� ��?*��<%6�,~����+dn���L�e]{���5Yp�;��S�A�������M�7���X����G�f'�F�k�1�}��J�����J�r��������l��,P���b�&����M����o2�u����D�[��1	-p~�	�O�o;!�,�:!�~�Y��V�X��� *v��D�b2�Q/����{xBJ� ��g�w��7�����|�?�[b�o�%��}8�!`[��������1��c/�Z��$1�G�Mf�[�o8��H*��-[�?b�I��7sr��\���%7����~����E��w��0P��4&9�<�ORv�j�i�][����������9�����T+�[;��nF�����how��o�6�y�g�3���Z�F�M��6)j�d�p�G�,O(GU�������
mMi�'eF�a��).��Q#t!���*�E5�������~{��&����f'���b��8��4u�k��H�����l�*���C�4��yv�������9hC���Rz���d\wp������!Q��_�x�Z��@F6�3@dah+fhf`+f`�������=�:�\���Rnl�g��������0�!z�g�	�xsg�xs���b���}����m��\Z
��,r�
d�-���������{tt�Z��{=pg��[!~��-�r��]��7�;N��Z��n�����9������8��0&��?4�HK���`����V�w���sm���rds2]�}h��[I�����v����gu�	{�{����y�.o�eZ"F������=���6q��B
c���]������R���
_{�IBu�FZ"_�K���i�\khZ����G[��^�����t[��N�l���9��]�Z[�����l2H-���[4�����'������8�b�d8Fa��.��t\���/a���(.wc������4�_1���{��3��Z�E��
p��*����H��hH�x_2d�{����&��M5�����	C��O-��&C����O9;�o,��YX*�a�A�S�enH3������>FwDe*:7X{@DK����J�Sv�|�*�#h�v���U_��$���fck[}<o��h�|���E@8�d�V���<����]�TW���ew��r_w
o��z6����~��2����3���@R�����MH�O@�����Rek�S:�XA)�1��q���Yg�h�j�Q��Gp����E��r?�=�����`-~�)�x~����>S�{�������������"�H3>����@�1��D�����?��h.0��=��p����%cz���h2Ef.JbY����d��YG�r�������/2ky�G�Mt�Lp��+��:��#�����	��H�K+�n���&�c���c{�<��x^3��K�8�
���}�iB�z�$��"<O'�fpc��!���U��D�S��������q���$D��K��	�f�Um^/-W���,�q�$�j$�ul������Vd>�0��){B!F.�e~�n��3�j��>�0Mt�Y@qH\�3�R���8(k������)�U���H��0 0�?*.�;clc?k44|%w�XO�,(a�����������g>��,��,��EK�`�oEo _'c��?���f�G��.����)���/�F�;>��R^z��~�Q�B�^�S%2��<q�2MC��C*�Ws�0������u@G�l�[Ra�G�N���\,RM�
��!F�9�f f�|�D;T=��`j"��l50�����
��Uh��(������������"��|��oV����
8�i����s����~�>�t�%i�6v�2�J{o��A�,���*$������"
>�j���z�|���F0��P�C(����;��7<��E�9���;�2��=����l[A�2:>Dwz\r$��9P���������	�����8L����(1�6�BY���c�G>V�*�0����}��9Y��|�����������+�?��_�I(���|���Xx�1/��c]���/�Yd�Y���4�`�?�]���5�@����ex�`�0J�#�	o4g�����Ta�~	�n�u,Lt�u��2"����H4"d�����zdV�6Z����Z��������@4�F���8��=�TW�!N�X_����d�,���� ��fP2a��(d@�Q���Q�O!��
-�2�<��G����L�L8a%������7(0%V���q�[4P~N�>�,��1b;%��\pn�*����&I��vh������^F�����~�@<�4E2"��JRs��;����q��O�WN�w��;?:�>={�	��b�%���)��������$&g�S������
�h���:���Z$�(E�5J����6���=�.����0P��e�=���2�3
q!,
H�������e��LCp����]w�^��/1�_S%N�@���YQ��:�gg'��E=EZ����3���#��M(���A�<O��B��2I�dF)�,YB� ��JsTq'ss��1yT�]p
��&��\�T@T�b=���L�{��>��
g*w�Th����8��l0�����z.�`p��Q����%*�
,t����&O	�LYO&$w.�P"z��7��?v^a��������:i�\0@l�������'~w�����w'G�b��=�$�(�Jc�p���������:F	l�1JcKq�s�Qgub>.��&)22��tR�Wd���}-'�_B����'�5k ��J�x��M�:vY�S���K�&o��z��T�������|���_�����@�������$��Xw��(��

�j�U��|�:d
G�_��r�U2��
7�b��h����pl��
L�yN�6�W��wgW���H��9-W�(����N�X�_(����M
]*PLk�
[
���VS9;L�J(�q�����0���[O�����b!9����.�M5au�E�q~�rlqv�u�:]y���5$|����6r/P��`�7����@���h���u*�s�Z^v���q.��.p�Rh�%e�{�����
!$a�8���k����v��!�n
B����3X�8�00���2'[w��C�H�S���8.P���u��m�q	�
^���K��x�����y

��ct������X�O,Hy���uq��s|zu�����!_
z�g�Oi���1���qy��&�"�1f���,S��#�$�#
N�w���u8�0���@4��H�6BM^=*�$7��������`EE�����CO�bs�����c��5St�d�C^shZ�v�=9<�5n.�B��!�Q��n�4S,2IQ+������R\c^�$G-�")f����9$���}����R��_��O���qx������w�H��j������N���v'��B���96�9Z�Z�'�����Wj�X�vNZ������Vs@1���IHg��`Q�K�K����T����@m��,��DkrA�#������j���
]�d���0���������y�m���a�o��css���n�f�j��x�����t�De���9���*G��T���"XG CM��0��<Ncb7P�d���R���4b%�����3�j,�\��M�����rt�$Yd�y�O�;�pw�!�O��0�mX�T�j�J�-Yz�N����YI[�,�L6��};�&�����4�_��IrF����V�K�,�����cN������Y&J��������?�w���_|�����R��S�Y9w���i�����������!��k�����az;�<�8��h��[�S��<U��!����_�s����B��*�)�+�
[W����f���]�D5�t�=lB�X��a�)<G�PU��@�4�>�R��_�NF��V�g�$�������3P!�{=-X���B{���h�����LQ�-v����
_�"N3d���^B�����f�2[��|I��A,��:�3��g���U@zw�'���l�@����"�Z�.o���3�jP7��������?��������X.�sqryb�XCO�d*�T��=�Aq�&����Y�V���j������Cy�:a���h��D��>VB����W��������m��n�B+�GJ��.l��|���"��6�6[��<�!C�ar�/���
�+�g\'����q��r/���m���'���j�r��U&��_�
�$���BT7W��pg<5 ;`n��?"���:�{DN� &���6��U�E,�T4�cJ�d���{���r
��%�4�@�_M������*��5|�*��v�2BmcH�`��Fcg�I�����7������9��]Q�l,[*�Pb=i��,���
�(�w��0�<o���HZ�����e�'�Ux�J}��<ii-�d���5�����������9��������9��uq����-�S3f�:��I�;w�f�q���V�P�b5�4z������^�%���[�Gi�Q���&�0����1J��c�����"-i�[7���3b���K��������+Bdm�l��H1�N\H���LB��2t,X{@�m��
(��Wef_��S�qj��(W��UO�%��� f$�1A�U���������4�JQ�&�������HSKU(u���NW`������.R���i����J��t`sa/IRX�@�{L9���H�9��~�5���$���������03������!J'�&
����}�2����`�����} ��OF���L���=h���M �H�BP�G�o-;-�7-4�2��x�]#*���r�?�et�����PXm�-%=+!�HT�K�-�T4`,�l=��������j��6f��}�y���e��n�S�����o9�\�������d���2��mlb��k
V�Cu!K4x��������A��X��#�C@\RQi�����f52�����0k���Th��]v�t}���4���H����D�K�&rB�[�,.e%������K_0*��}U�<2�P[��+H���Wg�'[Y�
�����2�0]�Q��V�c�\����C��g������K
]����P�NXw"�[�����������no�7�S�$�eQ�e������x7kF�d0Y�}��9H��3��+��&E�����z)#���u���9*��"���in���!I��A�	���V���+ ��������t�sRI��@�R���R�aL����q�P�<�@U��h�*K�j���4������������6���y��3�1V����t���T*h����f�5�x|�H�����@+,S9���p������_�Iq�:�f^"."6���|���q/�'��^F}*����&(D��
�^J�����e�!�8n��j��;[�Fo��g�A]`��%�����6?cu>�>����|�X����� ����4>�xW���(�,���R�B���Vv����Q�~������P�} ��F���#k�c�h�zq���*V�Y���:���N|���t��Jm�V����n[����ui��F�4HF7(����*���ya�.�!+a��p���y")���-P�]U$+�����d�({���jj���|}����,Q�JB�5v ��u�
�s��}��O�e�<-�A���9f��
v���ulyU�dZQZ�;UU��2�$ ,-���/Dr�/T�L���a��*T}<�d����
,U����h��y��ye��Er�w�b�s`M��2�E�9������c@��]����M.�J]r�J��E,���dUTd8HV�I�D����t��<�Xh5K&�;�m�E*�TB���t��WV�f�Z�;[�\��������[�Zi
���|��5��ZnNh=�
��<{jt����grS�V���T���q
(@����_W�����w�M�h��j��l��[;�]����v���,�\?���_�U���2������G������6��G�oe)�������V~[7U�z��������&B����|��;�?��)
0u���\7���������bs���&�t��?F�iE�)*w�4��1~�Xl]4�����L���JhfO���X+�d�l�%{�t#�<��0D�Z+��Z$���*�;@z6z�j��Q�����T3�'SFs��'�oi����������_�+w�����,X){YJ�a~f���j���n�~�ZBl����X4�6��-��^q2�}�X���+����'��`��_�r�C���U�������&A���!�o�z�)�n�`R���<���/���_y��Z�gs�����������+k�?G�:�?�D%��l��o��ma�U�Z��-�+�!�m]J��	��w��g''�����'*�l��������e�A����9r��l���:�S�d4�?�������dD&
S�s�g�����t�����#��q�5fY6F�O���D�����u5�!�;=���@���r0�7��������B��LL�����2�f��������h_�8�`SBN��<O������c'm�Km3����@��O�I`�H�=��&K�+�'�Z?��{���������V��b���=}BG��,���������8SFS8A8�@E��)�����#X���1Q���W5>V����5�ewP����8��D�
';Z����*=��4�x"<��9���� Ho,u�ue<)��4���(Ui���Fz�y=M����@����������P����r�����Z�#��3ig�"�eJ�J�|,�94����s}V�s<�������*A��������
kYQ���"�u������H��mk �^_��b]E�r��NW�ewVX��S����i��RE=��@m�4�O|��X�P����r��aSq���d}OH^��#����Y���g�z/�==�K�
�I��w������>RY&*�'4At�c�^q.����)���
���[�����B,R��Kp:2<�l'vz�������/��!�#� ���fr�h�0W��|h��BcE�z����w�
��8��p�N�27����&B������~E��_�`���y@�eQ��B#�J�~� *[���;35�l�F7���W��k_`�����G�������NU/*v}�����X-�C�{�'��L�|��#(��;��
��q~t��?U�i�s�����<}��<�Z��"�Q����>4������f**NM_0�.p6����K����7VI�3XX~�~f��]j����pN�B�=��U�/��c����5%I�4�^c,��k����t�J'�K�����}(�F���e������`f�*r1'	w�	H��Mt� N��l��K�J����`r�_�\^^�^����s+��9������B�]������f�Y#q`d��6$��� �*��5�b2�Y��$�7B#
�J�'G���,iU�J���t������ZR%��d��4��i�RLS�HB'q��AV�h0.y�~k5�ZW���� �|����N��L<�:�ii���>�;�;��P��}��v���^��xJ>�8�
��G���i��������Tf9�m 7P�d!�������P���81���g8�V7����1����3e�����\^����e��%l��?��t�t"�(�{:��>�}%X+����u*J�eUX���"Y0�T�[�����DY����ivLIy��f��Y%X2�>�U�R���Rf��9)(Zq%K%(��Lp+	��\E�BB���{�p������Q��<�@���2������g����nP%c�s��&�R�Z:h6�J��%�:U���"d~e�Oo��@V����)�mfp�*��oq��1I�s���|�cSy����1jT�Y�����(��KI]��6���*������t���D!��i�5�����-�������N����c��<���+��)����R��o������y�������P�����B5�����e.+��ed����]-Na�����Hg|�w���aL��X�eQ��<������+ZA����p�������~�k���������H�D�����;�p3������|����C��}��^R<U����_��������XNo��r�BB?q�K^��:�������������\����4�5q.\%��"H0k9�(>�`6M-������D����t�����P�C��B^N.����N���
��������1 m���;�h�0�����.�����O����Z�+W��\-L�9����Xv�M���S/��bg{����&#%K-�4VV��@����n�i�2���IOY|=�)���+z�2;�_�W8DU���?w^�_v��'��.����k�-���l��@i����E/�����������o�|���9�$)��O��y��A�������{LR�(�Tj�qq%�i�@���W|~Ym�TMsoY�'����6��EB��YPWt���[��D�*�b������Vuk

+�.NDc���u����|���.��K�U{�}�|��h28��%/�HG����N���/b�p�C����]�� ���")7�.��*���DEQp�(�}��1�T�,5�(�����H���t=��o8��-��w�M���Km>	�`��Cf��I�x�=���6�����j����TV��_}�m;��U�4.^�j���Dz���c�E\�W����B
�~����3J*��|���vL�h�
6����4e}���K��!x����I����L���|r�\��|U0�����������/�8�|az��������}�:��	-�/����7���Z0����zQT"��}J�:��:�f�3�U���*���c�Bc�Y���5X��*������Ke���w���rg'��D
V���:"����r�	.���(I�.'���cR�4���Wg8�Q�;�������{��-3���A�ewk�����iG�|Q:#
)������?�e/������Msi���bz$������A�����q|�SRc���7��[0�f��r��4�j�\_�4��_��o&|R)��aHGe#�i��J�������_E���Q';�&G<{�j��E���61z�������}�fi�8\�e����E��2�O��X��Q2�Z���}�d}}��\M��[��)���Q}�3����]�G�@������������wR���2�c2+��l����o�_\K9��W���R�T�R�D�[������!�X1�-B�j���P�G=����[i����%/����\�z�&SK�+�����bv3��]����%NW�a���V�Ou��*�s�_.�����BM������p�X�F�P�����J.�s�^N�g���|�
n��dI���E������� ;t%a�N��(�_���]�\:���b*��2�rW
��������e?	u��3L������>+�b�; �
�������A&���su����@����G2�N1�������fs��~k�W0��+�����t���oH����:>��3U�$K~�����G��h�����Y4���DI�B'��E���G��v{�Q��.�JV���xcZC�%�R��OY�e������m���+�3�j,�=@�[H�{)��B���ujk��2[��O�e
G�{\v#[qw�8I[{{���l�'s���H����������#��(�qJ4�e�m���R��#��:0��|�a���TU�es�U!��N�{�u�D@v�&['�N�B(_i���x�?�4T4�|h����(�%`U�TJ
�}����+�q��L; �4��:��bn'�.�6�����x���T��Q��GJ���#R�V�wG��]��.H�V����oKY����j��B����0(��N���Ub"NNw+��4��2~�L����w�U5m���6$�y:�����k��1��-���`S�6KV��YQm��D�I
�c^��m�<��v8�-�������~�y�OQ�����wnW������<L
Eq9�O�<n9���$veR��s���=�9K.n-	����;��BJ5��b�E%�#�T�w��KL_Y�[��./��b[?d9_j�W����n���C�w���R���lE���Z��K��$N����D�+������c.-���Q�����l���t��{���	krR:�\d9�0���E�^�[6���_���6�S�7���r��.�
���	���
'���/_V��\f84L����[9��}�������v�EX�+��e��a�oy�Qc��
��Q������R�A%���P��(q��D����^�7f�H��h��z��5f�b
�|$�	)����U����*Xq��G���s4I��QQL	�~��H*����o
�K&����(�?yN��70���6wU5�f9�I���?�<Neu�\
S^\���a�� �2x�`�|�\X.�qz����)w\��ZO�Q��,����5����J���+���-h�S����&S�\a�?)��tb3E'r�wb�]�TPV��1�E�N�ty��7����8��6�t�����P�Y��Y"��	�}I(l���W�N�Y��O��2�q�
��sv����K�������d����(* �4;�����]�_K���8��f��#����R!��,�Yr,D4��x��{����.���J���"1�a-��A�o%=*����}��b��S�_9�J�����5H�������;�w������%�!*��w�,�V�M}wN�_]�������K�K�&�z�	��b����'�}��T�������P�����Yy�:�v������3�(w-3��T�����rKz/��znGA�6t;6�������~�
t�Z�F���iS}�6��^Wk3��Tq�G���W�~X��#|���\��S��*n�����/E����<_x��7U�>��$kC���H�f��9���>�m.��`\,�n
�4<U��B��n�[�L:�,�������_Lu�����w����X��%V�[�T��0t�o�����)rX5�J��d���U���g�������C���F���;�E���@Q������}�2��t��;U�J��k����[�~4(p(�y��hK�\�x����6���^o�K�77��u�����P��eAm�@U3��tj��l��8�s��3��==�B5-�[h�L32-=����M��nl{|hyIA�z�Lu��B�}���L����|����2��+�!��E*�C^}����%���U���#q�3@��_���N`�C!G�\o��m�)�;U8?��H��?����������
��M�J.�?��;����y��<�c��6mo����f}c�P��jN���q��ii��]���@�+������XE���������e���H�����(����=
�X�iI�}��y���a�+�I1.}��h�X��"C-c�_�����n�O�|�XM;��5�u�~&��,�����
�_��por_\:<�e��W��#��.��Y���uo�E�hF���efa���]s���<����T��qJ������/�Qf^�0��	a3��`V7�����ID��z���F���n��X�����F���M�F-�
#����_I�g�Q�)����7�~d�|�uF"w�a��|�H�U��w�t�|#���mW��^�gl���N����"m+q��`�7a�Q�ju�iv[�9'���gL�H��@F�p��������Nk������!�3}CyU�@�d9
2���C�:�)�d�}�\��
46R�7�Rs�R��*��Y����`���o>���1X�H	z���<>V���K��,�c��0�1�MtBZ?������IW�q?����/�+�6������XUQ��p���v1�`TJ��[�.>�pp��Lg���z�;���eo�D�8+�c�P�G����_�Kp�0�'pM�2�S*k�.������]�����X9�T�Q�r!��s��}���L%���r�a�v#z�����|H�B�]���S����\�&��� 7^N���@g�Se	�h#�ec����x�Ge���\,�=�5���A~������\+���t�C)�
-K�!�B\z�h��k0�t���25�����;z��jvb������Dx+���������w�����a��iG����r��U�UY�0eH�(1A��@�L����dw�b��9%Z��A`|��d?�[�3�R�����d��uj^�e{��X�����d�K")�\�>wV��9p:+����sv6:�R^��r�X'��6���'Tp4{'��4�z=#��������1>�48o���tA�c��<KG5#s��-;����{�`;o��6l*������hZi>�jN��x�"��4Q�n5��0	^����?��	�aCdv7r���Vs�u�\�h��]AwlW���\�������_q]�#u���{�����5+B���`��9dZy��c�E��06��t���3�w�Pmt���`�,,�	��a�tQ�,�s���������������k�%s9��d�����L��h�� ��~�'O8��s�������t������3RU"`i.U<zVVTg�m����],I�����y�:	��6���� A���-^J������P�M������n��w����&���i�M-��v�2�U���,2d
�P��'�0B��\�r�W����st����m�e4U��U�
�d��9Z����������)�>���(������Z�u��erl�"V�R�.G�����d��hQl�2��9�u?I*�UqP����FT��-�GR[���Vb��z-���g&�E��J>�W���R<����cSK.�DU��dP^pZ�X&���]�	uA�Or����Sf�iS8T-LV6�x�V��&1��D���P� gA�v_!2%&[3x>�iF	�9g��N�5�[�X&�����*9��8�1s/w���HVh���H�K`zXI����T ��:-���P�������T���R��J-U�+�O?��iZ�%���n$F�;�V>�:BYS�C'@V��Ryp<���Jd$��L�O�m�6�vn�_��+������R����L���#1�Q����t>�G�z���5,���:i�<��5Y��U�]�������L��e����8�}\�<��O�:��hm���?n�]��Sr��L�.*3�wU��X���tii��#z������J��o2,�	�����p��P��Fi�4�Y!���:O���t{��R��,����u�\`8V��E	�x���	3����$��L�VF@.OTJ_�����s�3�����<	�(�u..�_a�&�\5���:G3Gi$�S�!���p��������Q=Ubbd*��w�w�d��������)��~1��<��������I���	`c�P����h�0N��`��B�\rv~H�����F�HF��o��^��F,��5�kW���\^]�
�u
�w�R��W2Yh%)�c��<�nu������z��?w��P�/I�C�y�uq:��#��r:�O���o�������:��9����SMt�Z-R
�~�9C������`{���b���z����r
,��yz���"�����b��l�����bWi5�i�'�f�4C��h@M�����e6�x%��������s�!�%���t/[����~���[����������
��:V�(�v���,��I��TIb5i�9������J��%N��N2�FR^KeV�?X���4�o��KJo^�h��`�d�.95����d���f
�4�{��1/����
�����f���77 �������O�5�DF�[��H��`����=#���r�\;�W��r�pU�g��g'��g'��C#�	e�L�=�z�$�E�S������N"32'�iI��������tI�h��`�+A��
�L��|� ��Hp	5��4=5L]q�.W�b@��U��DBv��9�R��h�0B�a2��s6$i���5�C�%,2�3Yc�����
�"�-��[e��y
�"V^�����&���EiP�;
G�D�`��hG���{� '�C��9.��*Zs�Q�`HGR�������Y��Pu�y��O����P0�����V�,�����>y����4}	�/�������.��w1���������)�*��4��r�q]�*3CN�� a��O��M���`oUEx���4M�7������/��Y��
������U0.'�F4�\E{��9�b��;aZ�z�O�2�0F!�
�!Q����w�rh)k��2 \���Y�RQ�f[2%��`u��a$��N&iYd��x0���_�w/$���Ch��R�*�� ��U3_���:����Kr\�+.G��<���w�+$�-����8�Q�����d��u���������1|�>��|).�L1>���!�nq?F���)h���#O�}Y���Ld��<�TS����K�*�p�P�EV\�-�G8WnQ�����	�?���oW���b�Rd�N&���g���r+<���y����n^x���s���E�\t�(dR��?:�;������K��8�:�AW�����/��L�>�
�W,��g�+�����7�m��a__U����8c_�������>sV�:�<-���6S�.|���]��d	W^JF���w�8���X\�w�,q��IHEU�Ch�J�Wk��,�f���!V3'}-@�w�Q��������&�o�{7N[����@:����E4+���+��hA��i�J�Q8C�0S: �!�5,�B�|�,���m��\�Q�.P��*=�vg�T�Qu��-R�B��r�/5Z����9E�*��7�^�J��(�=�y+�t3���������p'�s9�O$��Rh�+�������h��D1���-j�s����/�H::b����O:5���&b�xM�����S�rw�O�s,�<�S�uk�a0&;q��xD�0YR��l[U��<�iDx�hn#fX���0Z������v8�WJp��qz^/r��w���x�R�I�F�%�+�P��x8����H��(��=F�0�T^�����R�(���i�J��N����J����Q��V�K�p����������I
F�P�u�"E����e]$��#v��XsU��@h��Z��c	���OT��P� g.n�g!��[.%�Pl8�=�/�����[.����vF�E0@���j'��]	<@z�M��~t���H����Z���=����n��H�"(�]�����sV��v�/c��E/���sq��$�T�<�%LD|TGN������
�����Q�VL
���1�0hl2��*n��x)S���|vT�C U���8���3������X��^�V��=���\�5�\����*#Q�	��q���*��M?��s��JS��+WF?*%�i;2lvj�c9HF�I��JB���X2�E�`B���NB�������:k��1���bp�f+o���\���X���6�����e�������A�iZ��Pp��n��r@6���R�dL><=��Q��
V69y����#�c�V�m	�._�&Z���S6����&_+��v'�2��4��{a�.r������3d���|���h�Y@YQ(�^��P��UH�]���!�#��k�w|�^��_j�;�f��<X������5,�0��tm��.z�1O���
�����^��������^�����Vk{s��l6��jmeee�/sN�FK��?��2����]��>����g��_�y 
��]^�{w�����f�*_��	5���4�c@��j�P����"�@�����R��t��8�d�7`7�;�y���q���;R��(�8S��bU�c<P�:�2"��n��U�S���D���������rF�"�_{������{�������@B�;(:�,E��&����I�}�,V
"��`E�������uWC#+�=++�tu�k�j�g#t��:�����3��?#�����>J��{��:Lv������T<Oxm}�,�BT����5G�0�z�q"c���s&=`���UU�s�)�h,���@J�=���g�����,�2�o��4N��w���qXX�0p������
[D�+o�a�!F*�4�&w �$�O�F036�����9�c�I��3=l��0��>M`��66Gj�O���aA{m�`;���1�?T���2����Of�E�����hF��eVe�Bf���!�����5��GL��L���p+���)�0
��B��;��E�%%��C<��T�:��	)z�\�|AP��G8�XEpfPh��Xx�T9V9������1����� Z�U���Y�x(s��!��J��C�Q�8��a����Bp���(!��_�0D#�K���y85�zXbM�:�'a������z@C����,� ��!���O�7��)�m~XNf��K���K��H,�j�U4���i+��e���Z$���[:���Vv��	�Af���.��������@�B���p�h��y%C��C��<���%qn�)��1��n8�0��!9RE_������SH�Y!>p��K�5������/����d�c\Y�`����Y�<fep��*�9gj�z�����~n"����$`�k0�D�������GT������!a�E�C��0�0y�h3��V�[��|��#�n��zG6�.�I������Hmc�1.!��:��,�DuE������oP���=+���Zl����jYDI{��iU\���x{'T����z�^|y~y�`mYt��M�N�]������Ac���k��N�V�u����}-���s72p��s�#����E�-�py���
1Pdl+W	���vq0�kp���
q���e����,�&�U������9]J:k�����OtN�i����+�Z�l)�b�9-�k�8����=V���~,)ofo4�-���$����&�1������� U(���^�������������@��I1��S2���T�����A�������=� 6�>
���@D���p_�����i���PP���������?����.���ZT	���o�R.�'�tn/��L��NG6�cf[9u��g�G����T�>;z�8����D!vw�"/���	��{���%��"���<)��,�!]�����Du��ad\C�moc����%z��y��ss��Ml�z�.��������E��kW'g�4�4M]�0���Q�>^c���YMt2��E��T,H�	%��Y����Jo�)$#���"��D���d<�W�l���,
��rtk�kq�B�ea6z?�I�m�}���
df�5��h��H	��.X����q
����4������s���3X�=���&����"cB�hn�q$�� U�� ��8�����V�=�b�k\E���1���l��	��Q��+P��������1�H���O�S
	z�HH��8�@���Z:V�*�k43�sT��6��H=������9�@�j����j���qfA^��tH,�n������p�;_�8������I����.�b��������_o�~�r�����R��oM.��c�
������/�;����y���w�TN�����������P��w��xsx'�����1��O�P^UP�~�L�����w����\�f�=w�1	�����F�gC�:��b���CqF��������$�I���`��U
�����*%4B�=<�b��$�:��`�� �W;�T���Q����<����3�* �������7�����,�Q����1��z�e^D'n���������%�=�X5!��x���E�it\(<�_�����jw��o�j|��W���U�[�MR�s�"8�
����t�r#������\�2	��^�Sr��-r@�_?t�bX��j�s�I�C���r�I9�����Z�d�7�sl��'	�A������T�T��2Q0�h��<t3
5B��|$����-@���SO��������@S*I8gdvl&pz�{�e
������r�4�H�,���ai�����[y@�5:��k�'~�x/��;�Q)���i���a*�-v'D�^����ECjU�IM��M�[�����u!�Q���������LT������N1�&�u_�+�FOI-A^�r��V�����A�4�B�9
`p��:�������Z6����1���-�C)�&�2�jq���5G�tem��E)NT{�M��s���I��dpk<�2g�3!����&
��T�-�3�.���9"���A�<���~7�KH���I���q�������&-/�w����k}����BR���~8 ��I�4�T����k
���&��(�u���VT%��L��O��Y�'$���d����n����Y�]����Q)�lO���������p�����ZH������h����e6D��*��B�(�N��o���&Z��e���W�!�����T]����b4�M���w!�lCtZ�
GQ����l�. ��
��F��+�$@���Q�m�V���������C����������
�{J�����_��q8�b��FC#m4��������k�Hi�yT����������k��������y���Z�������V�XBO�����\@���y�
�Q���=)�s��9/��Av�j�Y��2�0�5���G� S�B�W�X|�(�lG����
�!���g���U�;�%�W��xpW�=|�*�%�����(���	H�#���y��i&�CA�C��id�����ey�P��I��3�5'��O��E3~�	,kM��}wr~�i��}y�s��Y���\j7������# x 
����K{���,�y3�,#����IKw$��T&B1�����S��{y"��������T��%t��j�Z(g�(��H�h�AA�j
ZI&�fh��b�,���tC�aO9��>��������B���r��Gw����B��B�j8[���iza���yt��sv#y^������LM�����O(��r��A��$�����d���f�d���
�"s��]�"�V/�Y�3��"���j0e��m�W���g��A-�c�
��������/Q&�L�_Kd������;��m7`�S>�z�a%/�p&#��2&H�Z�6����
5�a�����|��r�P�>�D���jPHF�v��xW�7�p�����c�	�����F�'�3 5|���q��r�d�Zj�Xh��+
:���hO����&���W�V�|��c�\�C��L�1��Gv����~!>�_Sx���~~����8�F��J�Pn<��rNK��
�Br����gW%i' a�%�7��<��������3�5����b.�V����G��]Xb����:�	
3�	L�f
Lt�Z�P�h��z{]�������A���1�dH��O��a��|g��~����g:��������7�h�h>�� �:	�A�e�Q��<a�������63&�tf��7S��/����0�O���#�XO��'������k�w���b���>���'��v�����Z���Q���YR]��Q�k�|�a�ca��:����[&����#|��C-�{y��d���>7��&�)U�8�n��(���9aB�"����C���P:`����gO����
>��'#Hd���/*�^��2�#�4�i1�S]���5xk���������JhIa���b�I�:�2)g��]���j�����b�$;8]��r�s�|9���G��N�lt��E<��2=�iK����R]FL�����b��x��:PIcQ���uC�p��IK���I��)�
�����A�k�Oz�j_���z?-:��D��`)Lnq�����CL�UY�i��v�Xx���!�J�-���T-�C��Jz&�m�}�R!1��,�d���z&yB�m|���p>1����y]��}Y�T����i�X�k�Z����sj;�03�2

��E\D6.iz����b�*�q(�%#���p�c�?/��ac���2������Xb�#�M��.�������M���`JFZ�X��f)�����:2cA�����!�'t�(U��$�I�Z�s�u�9��r��-P.<�!�#i�.�r{�
�-��`FK�M,�������l>n��L��8�����%��br�8l1v�rNM��S���c�G�jQA����t[U��8m<c����&1��c���21zc���!��r?N�_@X&�lp��G~^ho�;P�� ��O���Orl�y�2$�#��-�3�8o�����JM� `�w�U��,
�$5��z�"R���0R���A��O�;t^�;����a�/�._��>�O�E�h8���=��������Z��z:���+�:t��.q�:5��u���!�y��]8/q��Kl=t&�~(���f���vV��tNk��A`1&d]$C}c��W�ci�Q�E)J0iWi(��U�����8Q+��K������cj��	J4r����T���R��h���,8��'A]��8^����
���9��D_�w�SA��j�	5�
��2r��G��������!�r���1��(v73�	�w�L�p��������v��+-#������q�i�I&-���qq��)-���I�Gs��C������2C��������	�}9*��A��� ���0z���A��@��b�h8&��I��E`�p�hI66�i�����C�)�yn:sD~7Rb�~S�X�b$���F%0+���|���%~�
:x�Q#�U@��^��y������p$����C�mR���d�`���B�o�"��@�y%�D�j�H���ZsnL�{W��(f���Q2+�N�G�X	�
W��O���=�����^�����]������s��"���[[�^+2���j��dJ�;���$�=����x!y^0������u�{�|bF��T
��C�gI�b�s��D-��?ZxUN�(s:R���������bR�
�g����y����<�4��)��	(�G�Z�
�k�f��Tw��������f�D]i�F��P�={a��S�&RW�|+���}���n����k:��
%=s�8�f
$��@?9�y#c�!�P�C2W�?��,��_9���<�$�{D�F���|M�F������|�(Y�T�j�����iee#��m�Xuf�@�@�v:�b�=� �{K������4R2S�Q���_ydI��qk{��tO�B� ���9�9k��#���5�1�9m��y@���iJ�dI_oH'�W�TB%��-�yY�f!���M�]���@1���L�29�A'!ph�7�����?���)��������^
����Zk���������W����[��������d#=�N��^���}<\���7�����g1� �}�Qw�$�o�%S�eYx��n�Y����%���(��|�^��Z�N�b���b�;j*��}��phq�����d>�/�)*J��T�kX

�\�nl�Fi>��7o��phD��1C�I��=�H�D���
Fr����f1�
��P�p�e�diD�Z�72�#�BF���s�
�0��q�����,rtM�4
M=�U�S����/�1�� �Px{�n�G����>T(�4~��&��`��w�=c���@<�0u��
j$qah ��<���8A��`��z(��������7
=>U�r���;!��e�S��b!Fkd*j�k�Zr�_}k�p��q�44�H��\�a��)��35: ��V��TB�1KS��L�NC!�rM�b���x������`O��LF��]�_b�X�:��������0�r56x��%��Is�S���zP
pvzQf�7���E-v��G��)�����h/:TJ�g
��%I�eJ$b�&=-�k	\���a�H���J*tX�O�RP�KMC������?Xv���0��c����?�K�I�bP�q�_�ri�[Y�
�}����HA[����Npv ���U�C�N!��b!1�(������cV0�t���!��'����[�9��E)��|��b�^�c�������7�����2����X��<����U��y3��2["K�	��K�u}��	�ZZ���K�4�B�XO��:���ou!��`�������x�=��,	l,Zz��!��&�#����].�b�������u���	�reA�����9?���8^N7��V���`�M/�tB~���P���\�p"c�-�=�hz1�07��!�U3���I���6������ii��F��3��w���4���k����o�'w��?�o���"q��CH��Q"eH�9'����z��
��b�U�7��r��o�'I��L�,C	B�M�V����"0�����=��`�>�z��������u��y��,���N�=?
���tE���]0��^� �vP�&�.�Z������B�\�Z�X�A���W�G~�R,�F����j���50*J[��0Tp���_<���Y���t�F��g.��a�������:?s.==��u/=����"e%��������J��)�����(�����Y�,V�������/7��u�-���r�Z�5K~���[����0�`@l����MKEpM�CL�u��,u��a��������s��K������,��CHs>`�H�zj9��V���4��n]�^����~5����^cr��"��!}��Z���N0���(�[y�i?��a7	��]B�p;1���$J�(1H335R��K!�R�>����(/{@��h�r�!����2#%��!��1 d
�� {��Y���B |�����sK���Q��^�G#APP(�����Go���U���/����p,*�O�?-���������NA�%�����Q3���%LQ���rj���bt;�A@s�`�_�����zX���E��O+M��z�# )o��t�iE�b�;�`:\��	?�[���f�o8m�
4�3�}�^�$l��ch�SLKA`��N\(6e��������0*k����3��.n`�������n�=���U}�����a���=/>�{^l?���|���/���v��o�Th�����n�
6�{�(���]�c$��rM��b
5��I�z����6���C�����&�6��zy	8�����~�v��
`_4����yN�f�
���*X$�:��,f3�4��#�X	o�f	Am��*Z>�v���������h&�x����<�� �p�����k��%�	���M9
�vPPW84V�������]`/�-I�u�EYzS$&���S�����G}��������$h�%[;�Y�H`��y7^1��z�
�Z�~a�0���6�~��
���:2rZ�R��j�R"�������`�1�?��zZj��X)uab2�}�T��$��.�u	|Q�4h*j,��!m�D�#��do�n���Z6.��E�P!�����+��Gc�m}NL�r�&���\�!�8����
^�V�p��F�hN|�brG�$��Q����0� @�S����E=�����HW�_�h9a�LE"f[����������P��p2�P����Fs��}�1�A��.�Oc��O�8���[�������N�����[��R��4�����	�������/�����Q�k`���/��v��pI;����PN����K������%'a�E�i���4Xk9���H�#S�1��4S�����z��	qV�Cg�c�r�!�����E���5�n�f�p	�k����U�f�?�~7U;�E�<�������3Z�+�{���~�B����g�
�\��D��l6�.��=�>���/p��Sy���*C��8�������k7������-B�(
/�n$��a�#������x���	4.Gx����(��PG8I1�����n�F����K	�S+��6���H�Wg\S��V��V��D����#S�/�b��GDr:3���j�� �����])r�������}�;L�S m��BKn���C����G���Y��|����������b�(X�R��U�i����xdZ(���%)E!z�����)���|>����I�C�����9I�����<� PN�^�]!��J�Q������TZ/�����t
/J �F�2F���N6�V���Ju�S�t�H1�
tU6D!D���Rt[(�R6�qLQ~!���9���[c�����W�T��z�u}�����e<PB�y�_��(b&L��1�!f��h��,��(,HOp���Hb�����@�3�g	�h"�����=E��N�������Q�r�I���y�����r�Q����~w����b�1+}���E���<W������;�l@
d��S{�HI �H����@>f_KGe���	 ��#��I�l��<���6�����k�}Zn��
���;q�;�:����0t���R_��Q?�CO8�JE���c��:6C4)�1\3{G�l��xA,�n���!�����1�_
�����<i}&���?�To�L����.����M�����C%���\�H�
�rg��1����\w2v�3ML�Jz����pbaJ+���{u����
������p@E�!�{:hw���3$�������`��YyT,��lE"������
���v1m4*�$� ��&�����8�GM��69��W�[�	M�,����Y�z��`��rgJYO��Q�z_,���Dr�s��"^�m�F�U��j�&����~�o����L6�D�+�����X>�����e$�	�
:w(.��#o�	?��]���(�V)�����)0�X����(�����#QG�s!�f4��3q�y��R�"y�?3� ����|��R��'�\���Q���6�����9����1��.f�2%-���\�z���K-2hjm"&�zx��:������3�H��f4�ZK?5�r|f��r�1�v~�X��AP�7�~�����k\ub�e���I����<.�>�����i�#��4�>�so53���p����U�]4o�B�������ZK��m�v-"���9�����	��r�]����������>���k~��E([�f��)6X�����1/��D����5�"���Wxy����G����T2W�@�c�~��8
�YN���_�h�bQL�]�4U2jT����M���G?������7=�E| y� �
��}�����]�!��}�7<���#z�V�r^����M���xxL78���b ��P��b�X[����8/�?8�<C6��-@V��q��(!Ir�Ro-�;(��R�W�v��+H���$1v����z�.����������(wF�^`���p����G5|N��}�T(��W'����_�~�@������>���W>p���Q���x}N�-����z���.&���$�f��R[Uy���U[�^�5_�W��1B!�������Xk*�pJ��[��oM��'�$Oq�l�Z_�������PZ��M��x%��o��v����:k����NfKgAo�}WY��]}����������s\�L
�D���Z�n�=��=�=�����u��u7l�������;����3��l+�-=f�������M�:sS���'Zs0���W��'��^�v�NnO\<�"�%����B��ER92( ��d���^��K7�A�?j�[�^�X���R�_/5V$*J2�D������E��|�H����F������r\���?������7��A�������@j�??w�)����N;�����o��������}����}����i!@�Yg������B8%:� ��xy=7���~���s]��$
����O������>����������p� ^��~Eb�{�`��c�;���es��5����"�Y1��~:}���(�Z�7�6Je�/G�ZiP�uK+B����r?���bb	�9�e����W�E_y�T���R� �����p0x��EJ,�.*�<��<�EFw��\�P`XY��Ci�:��V�������������V�+����U��JC����_U���U�(}5���t;)���v0�&����j��H��_���b�S��;$���#�����!��S�������w��p���>a���Q6�k)� wdVh���~�An��8�����C���N�So`i�a����J�F�WU|�3��1{���r�BXj�S2!{�G~���<���bv���z%q����+
1��W���b!�L����M�����g�:����O.�_]z]w��[�
���=���1�]�"�w:��
���P
��
��?����u�
������/�e,<;wR��!&��6?���@���:������@�3^	����F�/{ �[�,{d�W�d��f������f������x�_^��?����9��>����o�i_���&������g������R�
P�����;`}o/�#��$�����.�����O[t0�������Q�u���}�����=Q�9G�<0p������u������N.�������/-�Jc�����y2�G?�G�R�@]a�0'�1QRA�0#�
:e�S	��c�!T�>�����0�����^����b��YH�`d���:I^p������G��@�� �z��<e�p��-���c}0��P�$�
*�}�.Bk��}�Cd��N��J��������p6xv	w����w���n�����"Z�*��V�����j%|�U��zw#�+�`Y�-�K�3�TcN�Q�@�t�/�R#����n#$�[�)��PN�G;E���A�8 O��0�ooB�����W�;f���������O��5rVFP�J8�������,���b.������'X����~�OW �^2�\
�6�G���8���7'�����Z���8>���������������[����*e������ �.���`�%��tr`42����Vb�'���
(�,�RF/�n��/
��_.�����b������]���@����Y'^��M��#�j;�)|�7A8y��*�t����pT`~o�^�X�v�E0��
8�\lv��w1��B���Oo��4F����"�C�RD	����f��� 59�l��z�kRKp�u�R��G�H�n�������`5r�8�p5Y�q�~���S��j���
C�&f�W�@T����!+-��B�_��@�}�HSPu���� �+K�`���\R�?6�w(m�����!���n`�Qs\�'@6��g7��x�N\�G�x:)���`pBr��7���A��*���=<u�;��G�.B��xP�q>y1�>Q1��G��~Z3�
��)�1�����.��HA7���|�,d��~�*�n��w'&4a���r��b
�7W�mi�7��0���00P��o��V6D�.��}�"�^��gp�C2%�"��s���f4vv~s"4�����E0��m��1����[�F(��N�!��)="�9t��u�p{���T�2X�){N:N�����c':� ,����coa�(��S�a��|�j�������6��|�	7�Ra��ld�����8������ z���R:���������Q�DMfG�;;`;���\@�y���8Rh�����*k"
���d4/w����]����6��
[��
�b^�S�@������'u�g1�>�c�>����RT��������a����wl���N�����*��vE������ 
UVB7Q��*�[(���c���9Z�iz��-��c.8���.;�����;�q��Mi���E��{S)�0]�;9Z��Cb�)���p1�*+�J?��L��#pJR�����}+'�������L����X�R-pM�EKS��&
i�sFH���{D��&]��*���j�<��!2-�yN��E�a4=`1��H�V�0^��a������x�]nt+�r/���1q�a%�7@��������]�c2jU�
$��]���WW����h���K.�c��x]X@;0'2��pb��,������uqr�����_f�X��!r0{TJ:��h|�0��5�����Q��T��cq���y����U�=c�����CLn��K�3�[��p���Er;IT�on�����[��{�a���X���
b�
���-��a/
��z7��^S��/��;��(�:N`���<��B�
��/���L,k@�`�|�G�y:���3}�Y� w�M)�V�u�x��A��	H�1�q���+%�$U��X����H��$)+�S�x�|��jfz�V�J�*&$z7��T��-�']&��%������g_S�U�kC1�����b������p�O��T7]��H��~��,
�F����z�Wn6����Z�Fb���
9�}�U���r����C9�1��?�y��P��K��` �|�{����~�[U.xm.pJ��!%W�H�{o �]B����A�K���/KQ�E�U���1"���p�����+�D�����U8+���T<b7�C4��o���2����,:?Zb�T�g^>�U����.B�?�U2�o�~bJ�k����{j�}~��������s����E��$�����_(��kV���a��QMc"�u
{���=����d<D��kf�s��t��!��M�K�~�����:'3����=��0�{��!K�G
�d-�j���O��G������`�v��p��D���R���f�����Q��YA������8�&u��WIX����h����UivSU������@-���dg�	��{����~�����-����jW_����j����;h�K�������&r�z��XI�Ek�6iIT��4�*�Ym������d��YnZ�V��gJ������������)=WT��d���y�P�IkIe(�Y��H=
	6���Y�������~���:�	�"vH��`��j�W��}�X����^����*�3YS�H&��aD�@�.�rDwQ��{w1�����(�~��)^	�
���b>��~?��4X�k���-���|����_F��'�j����>�\aM����|0�l�ExA����]����-���"��	��}H��� L�����x��b�����(#�P��/����\N{y�hf�
�fs������GR�S,so���f�R"lk�}D���]�UI���O�q���0$��Vs�X�6'	]T
���P{w#�����[�Wd8Y�YHM����NZo�_���S�A����
�x��6�ViP,���j�o����:�`��F�2��B&
_i6W�$'���owt�6��4��&m�����+��!�*;���<��b�
���(,��o�<���=�R� 2��B�.����=%:L9DD�h
D���B��m�����X�c�_%\p��~,yB���6�R�� 6��_-U{A-X��FEikoAsZ��������o-fG���� ���{�f�S��+�2��I`R�n���H�Z(�%k90E�M�}�,�&��o��
��R-g���YH���V�fj���j�'e�<�bP��D�Ns�u���f��{5h���d��D���B}8j���b����J�.�����{2Q������L�7��*�'3T�8�4S���b�WD������}�g'�I�)��+��d+��J�� �]�U��d!��s���PB-��L�*�����U1�����8P�<_+���l),������;:4��ZmP����n�R�v��1��U��(f\���[KO����ue@��Z�����:�
��],�[��Q��(���� �f����:�W�l2(p�#20&�y�����x{r}��m!�������=y��=���`��!����1�%���Y ��^fe3I����K�U��T���roZ[���Z[|��8���\��4����5;j�����b'��y���2�k������/O.��,���cG-����_�5���o
rY<j5���v�+D�DE�c�(�h���e[�D|�pZ�=��]p��@������R,����/�as��-��~&��[��R�k���������x�}����X��-+�����	�����~~���i4��u�����s^���[�.���SI��K�����$].��G�~���\�~�W��tyMuz9��E��Z�[�7�!�+G���L����g��V��Y8��b	F5�������3�0s��#�9���+��{�^�|T����$�`��di!���x���pY5��4XMC���|>�����}�4"��������t��GA�+�J�C������vw^�R�����.>\���c���t��W40- �����sv|���_���T�`�0���!�]�@�+c%�_|�G#���(
������tZFb9����T0��&\�E��Q�	���
�
��z��2�������+��1�,��e����OEg&�'�`V���/b�ko������i���)�!p3�����������<�EE#Db[��P�s�>,���D�{{qr�R�g�^���j����?c��;�x�g^���,pO5lz�������������U��pU�?rU���&e�����2���f���q�����Ap��P��!�����<��4z��{�B�_��zo��pP�.���b�P)���O��h)7/� MuT�����V�7��\,B��\�,���5}Q��=���#��/(	�Za�RjQ=+]{8
a���2x�M�2�7&�.�JcIr�U�^0v�#�y4��'�g���Gmu$���	���J��v�,�M%D����|���a��LvC����+�&��T�O�����0�n��J|l�cz{�>�R��V�J�����%��q�1~�H�&�
��`]�k����S�X��h�����pl�
CRG�+�����0���V3t3���R���@���������>����MG��ig��P��L��>Y��dc�R�&�U��v->56VNgcG����eX�U�s�5���8����ex��F���e�O�33�2�?F�[l��������!	���t�3O����A�h������������p&����5���������c����1f����2MeKhu���,�|*�����5+��ff�O������������o�����
X��p�u�a���/�B�
j��?���2r��~��c����Rtj�7z����?���0 �/����-���8f�������?����
��S��E������$�R0Sf:�f�P��V-�u$�<�i�:X���C�R����;���.�[���Fh_��FBs�fA� #�&*$����Pa`��!A���N���q=����8,��2'M��Q?X�������{O�b�M�7`^�.2���XJ�-`���Ou�\P��A�����C�� =��$���d��b""�9x�j�2���]N�!���6��X�E���Es/������������_~_x��_�o��2����A����&4#�i�#
u���D��o��p�P��'��~��E��D�VMeVS9�jNv]�w;7��U�>	`s�F��`�C
����:]F�|Qa����\Qv����_bD���J�@)������|u�u�����V���\�����*2��\����7��F��tHT`��N<E��1�����SR��1i��0[�kK�$y
�@�
��[��[f�]1v�8����!��+~��}8�����k����t?�-Oi��J������0�9�F_���,!�f!�S	� ���������G�(�Y&�V���&x }VrB{�D���~%�1����ola�m	$��F��	}�m���O�j���&��S;�O�%�3�`K���nK���K����S=�(&���#p��=DF�}�6��,�A�n����������^kP/��#Wk�*�RX�1�R;z�L#"�2T�\���#��C0�a<zx��L��vC�t RX~�����Ttq$/�D,RX��w���:�c����H��ig�;��jc��L]�(q�+�$������G��$�����b�!���!;�mD1��~�;>D�!���Z&�<8�cH�l[�QQl0DbZj%�e���|+CP������k6{����p��=f>��p[�F���V[�bqP���j�8��kJ��V�Pi1�J�r�~�?�g�����TJu�C��
��g���n���'����e3Ec�T<�7�8zr����H��U���R�;G:�lO��I��{��:Ke?*���g)�S��V�����~����{*������i��Y#A����|"�9P=]�����~p~/v��
����S)5
580�j�U# +G����,�P~�f[W�O�����<�L=�VQ����k]���B����'x���}W���l�/>c���I�DN�q�-���J�\/��=�?�����l?�����l?�����l?�����l?�����l?�����l?�����l?�����l?�����l?�����
�� 
#67Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#64)
Re: [HACKERS] Parallel Hash take II

Hi,

Looking at the main patch (v28).

First off: This looks pretty good, the code's quite readable now
(especially compared to earlier versions), the comments are good. Really
like the nodeHash split, and the always inline hackery in nodeHashjoin.
Think we're getting really really close.

  *     ExecHashJoinImpl
  *
- *     This function implements the Hybrid Hashjoin algorithm.
+ *     This function implements the Hybrid Hashjoin algorithm.  By forcing it
+ *     to be always inline many compilers are able to specialize it for
+ *     parallel = true/false without repeating the code.
  *

what about adding the above explanation for the always inline?

+		/*
+		 * So far we have no idea whether there are any other participants,
+		 * and if so, what phase they are working on.  The only thing we care
+		 * about at this point is whether someone has already created the
+		 * SharedHashJoinBatch objects, the main hash table for batch 0 and
+		 * (if necessary) the skew hash table yet.  One backend will be
+		 * elected to do that now if necessary.
+		 */

The 'yet' sounds a bit weird in combination with the 'already'.

+ static void
+ ExecParallelHashIncreaseNumBatches(HashJoinTable hashtable)
+ ...
+ 		case PHJ_GROW_BATCHES_ELECTING:
+ 			/* Elect one participant to prepare the operation. */

That's a good chunk of code. I'm ever so slightly inclined to put that
into a separate function. But I'm not sure it'd look good. Feel entirely
free to disregard.

+ static HashJoinTuple
+ ExecParallelHashLoadTuple(HashJoinTable hashtable, MinimalTuple tuple,
+ 						  dsa_pointer *shared)

Not really happy with the name. ExecParallelHashTableInsert() calling
ExecParallelHashLoadTuple() to insert a tuple into the hashtable doesn't
quite strike me as right; the naming similarity to
ExecParallelHashTableLoad seems problematic too.
ExecParallelHashAllocTuple() or such?

One could argue it'd not be a bad idea to keep a similar split as
dense_alloc() and memcpy() have, but I'm not really convinced by
myself. Hm.

+		case PHJ_GROW_BATCHES_ELECTING:
+			/* Elect one participant to prepare the operation. */

Imo that comment could use a one-line summary of what preparing means.

+					/*
+					 * We probably also need a smaller bucket array.  How many
+					 * tuples do we expect per batch, assuming we have only
+					 * half of them so far?

Makes sense, but did cost me a minute of thinking. Maybe add a short
explanation why.

+		case PHJ_GROW_BATCHES_ALLOCATING:
+			/* Wait for the above to be finished. */
+			BarrierArriveAndWait(&pstate->grow_batches_barrier,
+								 WAIT_EVENT_HASH_GROW_BATCHES_ALLOCATING);
+			/* Fall through. */

Just to make sure I understand: The "empty" phase is to protect the
process of the electee doing the sizing calculations etc? And the
reason that's not possible to do just by waiting for
PHJ_GROW_BATCHES_REPARTITIONING is that somebody could dynamically
arrive, i.e. it'd not be needed in a statically sized barrier? Pretty
tired here, sorry ;)

+			/* reset temp memory each time to avoid leaks from qual expr */
+			ResetExprContext(econtext);
+
+			if (ExecQual(hjclauses, econtext))

I personally think it's better to avoid this pattern and store the
result of the ExecQual() in a variable, ResetExprContext() and then act
on the result. No need to keep memory around for longer, and for bigger
contexts you're more likely to have all the metadata in cache.

I'd previously thought about introducing ExecQualAndReset() or such...

* IDENTIFICATION
* src/backend/executor/nodeHashjoin.c
*
+ * PARALLELISM
+ *

This is a pretty good explanation. How about adding a reference to it
from nodeHash.c's header?

+static TupleTableSlot *			/* return: a tuple or NULL */
+ExecHashJoin(PlanState *pstate)
+{
+	/*
+	 * On sufficiently smart compilers this should be inlined with the
+	 * parallel-aware branches removed.
+	 */
+	return ExecHashJoinImpl(pstate, false);

Ah, the explanation I desired above is here. Still seems good to have a
comment at the somewhat suspicious use of always_inline.

+
+	/*
+	 * If we started up so late that the shared batches have been freed
+	 * already by ExecHashTableDetach(), then we are finished.
+	 */
+	if (!DsaPointerIsValid(hashtable->parallel_state->batches))
+		return false;

This is really the only place that weird condition is detected? And why
is that possible in the first place? And if possible, shouldn't we have
detected earlier? Also, if possible, what prevents this to occur in a
way that test fails, because pstate->batches is freed, but not yet
reset?

+					while ((tuple = sts_parallel_scan_next(hashtable->batches[batchno].inner_tuples,
+														   &hashvalue)))

That's a fairly long line. Couldn't this whole block made more readable by
using a temp variable for inner_tuples?

Ok, eye's are closing against my will. This looks pretty damn close.

- Andres

#68Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#66)
Re: [HACKERS] Parallel Hash take II

On 2017-12-14 21:06:34 +1300, Thomas Munro wrote:

On Thu, Dec 14, 2017 at 11:45 AM, Andres Freund <andres@anarazel.de> wrote:

+                       if (accessor->write_pointer + accessor->sts->meta_data_size >=
+                               accessor->write_end)
+                               elog(ERROR, "meta-data too long");

That seems more like an Assert than a proper elog? Given that we're
calculating size just a few lines above...

It's an error because the logic is not smart enough to split the
optional meta-data and tuple size over multiple chunks. I have added
comments there to explain.

I don't see how that requires it to be an elog rather than an assert. As
far as I can tell this is only reachable if
meta_data_size + sizeof(uint32) >= STS_CHUNK_DATA_SIZE. For anything
smaller the sts_flush_chunk() a few lines above would have started a new
chunk.

IOW, we should detect this at sts_initialize() time, elog there, and
Assert() out in puttuple.

I've changed the code like that, fixed my own < vs >= confusion during
the conversion, fixed a typo, and pushed. If you're not ok with that
change, we can easily whack this around.

I've not changed this, but I wonder whether we should rename
sts_puttuple() to sts_put_tuple(), for consistency with
sts_read_tuple(). Or the other way round. Or...

Greetings,

Andres Freund

#69Thomas Munro
thomas.munro@enterprisedb.com
In reply to: Andres Freund (#67)
1 attachment(s)
Re: [HACKERS] Parallel Hash take II

On Thu, Dec 14, 2017 at 10:12 PM, Andres Freund <andres@anarazel.de> wrote:

Looking at the main patch (v28).

First off: This looks pretty good, the code's quite readable now
(especially compared to earlier versions), the comments are good. Really
like the nodeHash split, and the always inline hackery in nodeHashjoin.
Think we're getting really really close.

Thanks!

*     ExecHashJoinImpl
*
- *     This function implements the Hybrid Hashjoin algorithm.
+ *     This function implements the Hybrid Hashjoin algorithm.  By forcing it
+ *     to be always inline many compilers are able to specialize it for
+ *     parallel = true/false without repeating the code.
*

what about adding the above explanation for the always inline?

Ok, I added something like that.

+               /*
+                * So far we have no idea whether there are any other participants,
+                * and if so, what phase they are working on.  The only thing we care
+                * about at this point is whether someone has already created the
+                * SharedHashJoinBatch objects, the main hash table for batch 0 and
+                * (if necessary) the skew hash table yet.  One backend will be
+                * elected to do that now if necessary.
+                */

The 'yet' sounds a bit weird in combination with the 'already'.

Fixed. Also removed an outdated mention of skew hash table.

+ static void
+ ExecParallelHashIncreaseNumBatches(HashJoinTable hashtable)
+ ...
+               case PHJ_GROW_BATCHES_ELECTING:
+                       /* Elect one participant to prepare the operation. */

That's a good chunk of code. I'm ever so slightly inclined to put that
into a separate function. But I'm not sure it'd look good. Feel entirely
free to disregard.

Disregarding for now...

+ static HashJoinTuple
+ ExecParallelHashLoadTuple(HashJoinTable hashtable, MinimalTuple tuple,
+                                                 dsa_pointer *shared)

Not really happy with the name. ExecParallelHashTableInsert() calling
ExecParallelHashLoadTuple() to insert a tuple into the hashtable doesn't
quite strike me as right; the naming similarity to
ExecParallelHashTableLoad seems problematic too.
ExecParallelHashAllocTuple() or such?

One could argue it'd not be a bad idea to keep a similar split as
dense_alloc() and memcpy() have, but I'm not really convinced by
myself. Hm.

Yeah, the names are confusing. So:

1. I changed ExecParallelHashTableLoad() to
ExecParallelHashTableInsertCurrentBatch(). It's just like
ExecParallelHashTableInsert() except that it's not prepared to send
tuples to a different batch or to run out of memory (it's used only
for loading batches from disk where the total size was already checked
by ExecParallelHashTablePrealloc(), and in the parallel case there is
no 'send forward to future batch' requirement so we might as well skip
the conditional stuff).

2. I changed ExecParallelHashLoadTuple() to
ExecParallelHashTupleAlloc(), and made it so that the caller must call
memcpy so that it's more like dense_alloc(). I have now been working
on PostgreSQL long enough that arbitrary differences in case
conventions don't even hurt my eyes anymore.

I also renamed ExecParallelHashTableAllocate() to
ExecParallelHashTableAlloc(), and ExecParallelPreallocate() to
ExecParallelHashTuplePrealloc(), to conform with the that style.

+               case PHJ_GROW_BATCHES_ELECTING:
+                       /* Elect one participant to prepare the operation. */

Imo that comment could use a one-line summary of what preparing means.

Done.

+                                       /*
+                                        * We probably also need a smaller bucket array.  How many
+                                        * tuples do we expect per batch, assuming we have only
+                                        * half of them so far?

Makes sense, but did cost me a minute of thinking. Maybe add a short
explanation why.

Done.

+               case PHJ_GROW_BATCHES_ALLOCATING:
+                       /* Wait for the above to be finished. */
+                       BarrierArriveAndWait(&pstate->grow_batches_barrier,
+                                                                WAIT_EVENT_HASH_GROW_BATCHES_ALLOCATING);
+                       /* Fall through. */

Just to make sure I understand: The "empty" phase is to protect the
process of the electee doing the sizing calculations etc? And the
reason that's not possible to do just by waiting for
PHJ_GROW_BATCHES_REPARTITIONING is that somebody could dynamically
arrive, i.e. it'd not be needed in a statically sized barrier?

Yeah, it's where you wait for the serial phase above to be finished.
It needs an entry point of its own for the reason you said: someone
might attach while the allocation is taking place, and therefore needs
to wait for the allocation phase to finish. By doing it this way
instead of (say) putting a wait at the start of the
PHJ_GROW_BATCHES_REPARTITIONING case, we allow someone to start
running at PHJ_GROW_BATCHES_REPARTITIONING phase and do useful work
immediately. Does that make sense?

+                       /* reset temp memory each time to avoid leaks from qual expr */
+                       ResetExprContext(econtext);
+
+                       if (ExecQual(hjclauses, econtext))

I personally think it's better to avoid this pattern and store the
result of the ExecQual() in a variable, ResetExprContext() and then act
on the result. No need to keep memory around for longer, and for bigger
contexts you're more likely to have all the metadata in cache.

I'd previously thought about introducing ExecQualAndReset() or such...

Makes sense, but this is code that is identical in
ExecScanHashBucket() so I think we should keep it this way for now,
and explore expression context lifetime improvements in a separate
patch? Looks like the same change could be made in other nodes too.

* IDENTIFICATION
* src/backend/executor/nodeHashjoin.c
*
+ * PARALLELISM
+ *

This is a pretty good explanation. How about adding a reference to it
from nodeHash.c's header?

Done.

+static TupleTableSlot *                        /* return: a tuple or NULL */
+ExecHashJoin(PlanState *pstate)
+{
+       /*
+        * On sufficiently smart compilers this should be inlined with the
+        * parallel-aware branches removed.
+        */
+       return ExecHashJoinImpl(pstate, false);

Ah, the explanation I desired above is here. Still seems good to have a
comment at the somewhat suspicious use of always_inline.

Done.

+
+       /*
+        * If we started up so late that the shared batches have been freed
+        * already by ExecHashTableDetach(), then we are finished.
+        */
+       if (!DsaPointerIsValid(hashtable->parallel_state->batches))
+               return false;

This is really the only place that weird condition is detected? And why
is that possible in the first place? And if possible, shouldn't we have
detected earlier? Also, if possible, what prevents this to occur in a
way that test fails, because pstate->batches is freed, but not yet
reset?

ExecParallelHashJoinNewBatch(), where this code appears, is generally
the place that we discover that we're finished. Normally we discover
that we're finished by seeing that there are no batches left to chew
on, by inspecting the per-batch state in shmem. This weird condition
arises when a worker starts up so late that the join is finished and
the shmem space used to tracks batches has already been freed. I
agree that that was badly explained and there was in fact something a
bit kooky about that coding. I have now changed it so that
ExecParallelHashEnsureBatchAccessors() detects this case and has a
better comment to explain it, and ExecParallelHashJoinNewBatch() now
just looks out for hashtable->batches == NULL with a comment referring
to the other place.

You can hit this case by hacking the code thus:

--- a/src/backend/access/transam/parallel.c
+++ b/src/backend/access/transam/parallel.c
@@ -986,6 +986,10 @@ ParallelWorkerMain(Datum main_arg)
        Assert(ParallelWorkerNumber == -1);
        memcpy(&ParallelWorkerNumber, MyBgworkerEntry->bgw_extra, sizeof(int));
+       elog(LOG, "delaying startup to stagger workers ...");
+       pg_usleep(ParallelWorkerNumber * 2000000);
+       elog(LOG, "... continuing");
+

... and then:

create table foo as
select generate_series(1, 1000000)::int a,
'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'::text b;
analyze foo;
set parallel_leader_participation = off;
explain analyze select count(*) from foo r join foo s using (a);

The point is that worker 0 completes the entire multi-batch join and
cleans up, freeing the parallel batch info, and then worker 1 arrives
2 seconds later and it can't even look up the batch info to figure out
which batches are finished. It has to be like that, because worker 0
doesn't even know about worker 1, and as far as it's concerned the
last one out turns out the lights.

+                                       while ((tuple = sts_parallel_scan_next(hashtable->batches[batchno].inner_tuples,
+                                                                                                                  &hashvalue)))

That's a fairly long line. Couldn't this whole block made more readable by
using a temp variable for inner_tuples?

Done.

While running through every test I could think of I discovered that
the multi-batch parallel-aware hash join with rescan regression test
failed to be multi-batch on 32 bit x86, so I needed to tweak the
number of rows in the table called "bar".

--
Thomas Munro
http://www.enterprisedb.com

Attachments:

parallel-hash-v30.patchapplication/octet-stream; name=parallel-hash-v30.patchDownload
From 79a5d4bbdfccf9622f4dff54f16a77f526603db1 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Fri, 29 Sep 2017 23:08:14 +1300
Subject: [PATCH] Add parallel-aware hash joins.

Introduce parallel-aware hash joins that appear in EXPLAIN plans as Parallel
Hash Join with Parallel Hash.  While hash joins could already appear in
parallel queries, they were previously always parallel-oblivious and had a
partial subplan only on the outer side, meaning that the work of the inner
subplan was duplicated in every worker.

After this commit, the planner will consider using a partial subplan on the
inner side too, using the Parallel Hash node to divide the work over the
available CPU cores and combine its results in shared memory.  If the join
needs to be split into multiple batches in order to respect work_mem, then
workers process different batches as much as possible and then work together
on the remaining batches.

The advantages of a parallel-aware hash join over a parallel-oblivious hash
join used in a parallel query are that it:

 * avoids wasting memory on duplicated hash tables
 * avoids wasting disk space on duplicated batch files
 * divides the work of building the hash table over the CPUs

One disadvantage is that there is some communication between the participating
CPUs which might outweigh the benefits of parallelism in the case of small
hash tables.  This is avoided by the planner's existing reluctance to supply
partial plans for small scans, but it may be necessary to estimate
synchronization costs in future if that situation changes.  Another is that
outer batch 0 must be written to disk if multiple batches are required.

A potential future advantage of parallel-aware hash joins is that right and
full outer joins could be supported, since there is a single set of matched
bits for each hashtable, but that is not yet implemented.

A new GUC enable_parallel_hash is defined to control the feature, defaulting
to on.

Author: Thomas Munro
Reviewed-By: Andres Freund, Robert Haas
Tested-By: Rafia Sabih, Prabhat Sahu
Discussion: https://postgr.es/m/CAEepm=2W=cOkiZxcg6qiFQP-dHUe09aqTrEMM7yJDrHMhDv_RA@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=37HKyJ4U6XOLi=JgfSHM3o6B-GaeO-6hkOmneTDkH+Uw@mail.gmail.com
---
 doc/src/sgml/config.sgml                      |   15 +
 doc/src/sgml/monitoring.sgml                  |   62 +-
 src/backend/executor/execParallel.c           |   21 +
 src/backend/executor/execProcnode.c           |    3 +
 src/backend/executor/nodeHash.c               | 1660 +++++++++++++++++++++++--
 src/backend/executor/nodeHashjoin.c           |  608 ++++++++-
 src/backend/nodes/copyfuncs.c                 |    1 +
 src/backend/nodes/outfuncs.c                  |    1 +
 src/backend/nodes/readfuncs.c                 |    1 +
 src/backend/optimizer/path/costsize.c         |   25 +-
 src/backend/optimizer/path/joinpath.c         |   36 +-
 src/backend/optimizer/plan/createplan.c       |   11 +
 src/backend/optimizer/util/pathnode.c         |    5 +-
 src/backend/postmaster/pgstat.c               |   45 +
 src/backend/utils/misc/guc.c                  |   10 +-
 src/backend/utils/misc/postgresql.conf.sample |    1 +
 src/include/executor/hashjoin.h               |  175 ++-
 src/include/executor/nodeHash.h               |   22 +-
 src/include/executor/nodeHashjoin.h           |    6 +
 src/include/nodes/execnodes.h                 |    6 +
 src/include/nodes/plannodes.h                 |    1 +
 src/include/nodes/relation.h                  |    2 +
 src/include/optimizer/cost.h                  |    4 +-
 src/include/optimizer/pathnode.h              |    1 +
 src/include/pgstat.h                          |   15 +
 src/include/storage/lwlock.h                  |    1 +
 src/test/regress/expected/join.out            |  304 ++++-
 src/test/regress/expected/sysviews.out        |    3 +-
 src/test/regress/sql/join.sql                 |  148 ++-
 29 files changed, 3080 insertions(+), 113 deletions(-)

diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 533faf060de..e4a01699e46 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -3647,6 +3647,21 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
       </listitem>
      </varlistentry>
 
+     <varlistentry id="guc-enable-parallel-hash" xreflabel="enable_parallel_hash">
+      <term><varname>enable_parallel_hash</varname> (<type>boolean</type>)
+       <indexterm>
+        <primary><varname>enable_parallel_hash</varname> configuration parameter</primary>
+       </indexterm>
+      </term>
+      <listitem>
+       <para>
+        Enables or disables the query planner's use of hash-join plan
+        types with parallel hash. Has no effect if hash-join plans are not
+        also enabled. The default is <literal>on</literal>.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry id="guc-enable-partition-wise-join" xreflabel="enable_partition_wise_join">
       <term><varname>enable_partition_wise_join</varname> (<type>boolean</type>)
       <indexterm>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index b6f80d97080..8a9793644fa 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -1263,7 +1263,7 @@ postgres   27093  0.0  0.0  30096  2752 ?        Ss   11:34   0:00 postgres: ser
          <entry>Waiting in an extension.</entry>
         </row>
         <row>
-         <entry morerows="17"><literal>IPC</literal></entry>
+         <entry morerows="32"><literal>IPC</literal></entry>
          <entry><literal>BgWorkerShutdown</literal></entry>
          <entry>Waiting for background worker to shut down.</entry>
         </row>
@@ -1279,6 +1279,66 @@ postgres   27093  0.0  0.0  30096  2752 ?        Ss   11:34   0:00 postgres: ser
          <entry><literal>ExecuteGather</literal></entry>
          <entry>Waiting for activity from child process when executing <literal>Gather</literal> node.</entry>
         </row>
+        <row>
+          <entry><literal>Hash/Batch/Allocating</literal></entry>
+          <entry>Waiting for an elected Parallel Hash participant to allocate a hash table.</entry>
+        </row>
+        <row>
+          <entry><literal>Hash/Batch/Electing</literal></entry>
+          <entry>Electing a Parallel Hash participant to allocate a hash table.</entry>
+        </row>
+        <row>
+          <entry><literal>Hash/Batch/Loading</literal></entry>
+          <entry>Waiting for other Parallel Hash participants to finish loading a hash table.</entry>
+        </row>
+        <row>
+          <entry><literal>Hash/Build/Allocating</literal></entry>
+          <entry>Waiting for an elected Parallel Hash participant to allocate the initial hash table.</entry>
+        </row>
+        <row>
+          <entry><literal>Hash/Build/Electing</literal></entry>
+          <entry>Electing a Parallel Hash participant to allocate the initial hash table.</entry>
+        </row>
+        <row>
+          <entry><literal>Hash/Build/HashingInner</literal></entry>
+          <entry>Waiting for other Parallel Hash participants to finish hashing the inner relation.</entry>
+        </row>
+        <row>
+          <entry><literal>Hash/Build/HashingOuter</literal></entry>
+          <entry>Waiting for other Parallel Hash participants to finish partitioning the outer relation.</entry>
+        </row>
+        <row>
+          <entry><literal>Hash/GrowBatches/Allocating</literal></entry>
+          <entry>Waiting for an elected Parallel Hash participant to allocate more batches.</entry>
+        </row>
+        <row>
+          <entry><literal>Hash/GrowBatches/Deciding</literal></entry>
+          <entry>Electing a Parallel Hash participant to decide on future batch growth.</entry>
+        </row>
+        <row>
+          <entry><literal>Hash/GrowBatches/Electing</literal></entry>
+          <entry>Electing a Parallel Hash participant to allocate more batches.</entry>
+        </row>
+        <row>
+          <entry><literal>Hash/GrowBatches/Finishing</literal></entry>
+          <entry>Waiting for an elected Parallel Hash participant to decide on future batch growth.</entry>
+        </row>
+        <row>
+          <entry><literal>Hash/GrowBatches/Repartitioning</literal></entry>
+          <entry>Waiting for other Parallel Hash participants to finishing repartitioning.</entry>
+        </row>
+        <row>
+          <entry><literal>Hash/GrowBuckets/Allocating</literal></entry>
+          <entry>Waiting for an elected Parallel Hash participant to finish allocating more buckets.</entry>
+        </row>
+        <row>
+          <entry><literal>Hash/GrowBuckets/Electing</literal></entry>
+          <entry>Electing a Parallel Hash participant to allocate more buckets.</entry>
+        </row>
+        <row>
+          <entry><literal>Hash/GrowBuckets/Reinserting</literal></entry>
+          <entry>Waiting for other Parallel Hash participants to finish inserting tuples into new buckets.</entry>
+        </row>
         <row>
          <entry><literal>LogicalSyncData</literal></entry>
          <entry>Waiting for logical replication remote server to send data for initial table synchronization.</entry>
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 02b5aa517b5..128caced7a9 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -31,6 +31,7 @@
 #include "executor/nodeCustom.h"
 #include "executor/nodeForeignscan.h"
 #include "executor/nodeHash.h"
+#include "executor/nodeHashjoin.h"
 #include "executor/nodeIndexscan.h"
 #include "executor/nodeIndexonlyscan.h"
 #include "executor/nodeSeqscan.h"
@@ -266,6 +267,11 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
 				ExecBitmapHeapEstimate((BitmapHeapScanState *) planstate,
 									   e->pcxt);
 			break;
+		case T_HashJoinState:
+			if (planstate->plan->parallel_aware)
+				ExecHashJoinEstimate((HashJoinState *) planstate,
+									 e->pcxt);
+			break;
 		case T_HashState:
 			/* even when not parallel-aware, for EXPLAIN ANALYZE */
 			ExecHashEstimate((HashState *) planstate, e->pcxt);
@@ -474,6 +480,11 @@ ExecParallelInitializeDSM(PlanState *planstate,
 				ExecBitmapHeapInitializeDSM((BitmapHeapScanState *) planstate,
 											d->pcxt);
 			break;
+		case T_HashJoinState:
+			if (planstate->plan->parallel_aware)
+				ExecHashJoinInitializeDSM((HashJoinState *) planstate,
+										  d->pcxt);
+			break;
 		case T_HashState:
 			/* even when not parallel-aware, for EXPLAIN ANALYZE */
 			ExecHashInitializeDSM((HashState *) planstate, d->pcxt);
@@ -898,6 +909,11 @@ ExecParallelReInitializeDSM(PlanState *planstate,
 				ExecBitmapHeapReInitializeDSM((BitmapHeapScanState *) planstate,
 											  pcxt);
 			break;
+		case T_HashJoinState:
+			if (planstate->plan->parallel_aware)
+				ExecHashJoinReInitializeDSM((HashJoinState *) planstate,
+											pcxt);
+			break;
 		case T_HashState:
 		case T_SortState:
 			/* these nodes have DSM state, but no reinitialization is required */
@@ -1196,6 +1212,11 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
 				ExecBitmapHeapInitializeWorker((BitmapHeapScanState *) planstate,
 											   pwcxt);
 			break;
+		case T_HashJoinState:
+			if (planstate->plan->parallel_aware)
+				ExecHashJoinInitializeWorker((HashJoinState *) planstate,
+											 pwcxt);
+			break;
 		case T_HashState:
 			/* even when not parallel-aware, for EXPLAIN ANALYZE */
 			ExecHashInitializeWorker((HashState *) planstate, pwcxt);
diff --git a/src/backend/executor/execProcnode.c b/src/backend/executor/execProcnode.c
index fcb8b569999..699dc691792 100644
--- a/src/backend/executor/execProcnode.c
+++ b/src/backend/executor/execProcnode.c
@@ -770,6 +770,9 @@ ExecShutdownNode(PlanState *node)
 		case T_HashState:
 			ExecShutdownHash((HashState *) node);
 			break;
+		case T_HashJoinState:
+			ExecShutdownHashJoin((HashJoinState *) node);
+			break;
 		default:
 			break;
 	}
diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor/nodeHash.c
index afd7384e945..60c3afef09b 100644
--- a/src/backend/executor/nodeHash.c
+++ b/src/backend/executor/nodeHash.c
@@ -10,6 +10,8 @@
  * IDENTIFICATION
  *	  src/backend/executor/nodeHash.c
  *
+ * See note on parallelism in nodeHashjoin.c.
+ *
  *-------------------------------------------------------------------------
  */
 /*
@@ -25,6 +27,7 @@
 #include <limits.h>
 
 #include "access/htup_details.h"
+#include "access/parallel.h"
 #include "catalog/pg_statistic.h"
 #include "commands/tablespace.h"
 #include "executor/execdebug.h"
@@ -32,6 +35,8 @@
 #include "executor/nodeHash.h"
 #include "executor/nodeHashjoin.h"
 #include "miscadmin.h"
+#include "pgstat.h"
+#include "port/atomics.h"
 #include "utils/dynahash.h"
 #include "utils/memutils.h"
 #include "utils/lsyscache.h"
@@ -40,6 +45,8 @@
 
 static void ExecHashIncreaseNumBatches(HashJoinTable hashtable);
 static void ExecHashIncreaseNumBuckets(HashJoinTable hashtable);
+static void ExecParallelHashIncreaseNumBatches(HashJoinTable hashtable);
+static void ExecParallelHashIncreaseNumBuckets(HashJoinTable hashtable);
 static void ExecHashBuildSkewHash(HashJoinTable hashtable, Hash *node,
 					  int mcvsToUse);
 static void ExecHashSkewTableInsert(HashJoinTable hashtable,
@@ -49,6 +56,30 @@ static void ExecHashSkewTableInsert(HashJoinTable hashtable,
 static void ExecHashRemoveNextSkewBucket(HashJoinTable hashtable);
 
 static void *dense_alloc(HashJoinTable hashtable, Size size);
+static HashJoinTuple ExecParallelHashTupleAlloc(HashJoinTable hashtable,
+						  size_t size,
+						  dsa_pointer *shared);
+static void MultiExecPrivateHash(HashState *node);
+static void MultiExecParallelHash(HashState *node);
+static inline HashJoinTuple ExecParallelHashFirstTuple(HashJoinTable table,
+						  int bucketno);
+static inline HashJoinTuple ExecParallelHashNextTuple(HashJoinTable table,
+						  HashJoinTuple tuple);
+static inline void ExecParallelHashPushTuple(dsa_pointer_atomic *head,
+						  HashJoinTuple tuple,
+						  dsa_pointer tuple_shared);
+static void ExecParallelHashJoinSetUpBatches(HashJoinTable hashtable, int nbatch);
+static void ExecParallelHashEnsureBatchAccessors(HashJoinTable hashtable);
+static void ExecParallelHashRepartitionFirst(HashJoinTable hashtable);
+static void ExecParallelHashRepartitionRest(HashJoinTable hashtable);
+static HashMemoryChunk ExecParallelHashPopChunkQueue(HashJoinTable table,
+							  dsa_pointer *shared);
+static bool ExecParallelHashTuplePrealloc(HashJoinTable hashtable,
+							int batchno,
+							size_t size);
+static void ExecParallelHashMergeCounters(HashJoinTable hashtable);
+static void ExecParallelHashCloseBatchAccessors(HashJoinTable hashtable);
+
 
 /* ----------------------------------------------------------------
  *		ExecHash
@@ -72,6 +103,39 @@ ExecHash(PlanState *pstate)
  */
 Node *
 MultiExecHash(HashState *node)
+{
+	/* must provide our own instrumentation support */
+	if (node->ps.instrument)
+		InstrStartNode(node->ps.instrument);
+
+	if (node->parallel_state != NULL)
+		MultiExecParallelHash(node);
+	else
+		MultiExecPrivateHash(node);
+
+	/* must provide our own instrumentation support */
+	if (node->ps.instrument)
+		InstrStopNode(node->ps.instrument, node->hashtable->partialTuples);
+
+	/*
+	 * We do not return the hash table directly because it's not a subtype of
+	 * Node, and so would violate the MultiExecProcNode API.  Instead, our
+	 * parent Hashjoin node is expected to know how to fish it out of our node
+	 * state.  Ugly but not really worth cleaning up, since Hashjoin knows
+	 * quite a bit more about Hash besides that.
+	 */
+	return NULL;
+}
+
+/* ----------------------------------------------------------------
+ *		MultiExecPrivateHash
+ *
+ *		parallel-oblivious version, building a backend-private
+ *		hash table and (if necessary) batch files.
+ * ----------------------------------------------------------------
+ */
+static void
+MultiExecPrivateHash(HashState *node)
 {
 	PlanState  *outerNode;
 	List	   *hashkeys;
@@ -80,10 +144,6 @@ MultiExecHash(HashState *node)
 	ExprContext *econtext;
 	uint32		hashvalue;
 
-	/* must provide our own instrumentation support */
-	if (node->ps.instrument)
-		InstrStartNode(node->ps.instrument);
-
 	/*
 	 * get state info from node
 	 */
@@ -138,18 +198,147 @@ MultiExecHash(HashState *node)
 	if (hashtable->spaceUsed > hashtable->spacePeak)
 		hashtable->spacePeak = hashtable->spaceUsed;
 
-	/* must provide our own instrumentation support */
-	if (node->ps.instrument)
-		InstrStopNode(node->ps.instrument, hashtable->totalTuples);
+	hashtable->partialTuples = hashtable->totalTuples;
+}
+
+/* ----------------------------------------------------------------
+ *		MultiExecParallelHash
+ *
+ *		parallel-aware version, building a shared hash table and
+ *		(if necessary) batch files using the combined effort of
+ *		a set of co-operating backends.
+ * ----------------------------------------------------------------
+ */
+static void
+MultiExecParallelHash(HashState *node)
+{
+	ParallelHashJoinState *pstate;
+	PlanState  *outerNode;
+	List	   *hashkeys;
+	HashJoinTable hashtable;
+	TupleTableSlot *slot;
+	ExprContext *econtext;
+	uint32		hashvalue;
+	Barrier    *build_barrier;
+	int			i;
 
 	/*
-	 * We do not return the hash table directly because it's not a subtype of
-	 * Node, and so would violate the MultiExecProcNode API.  Instead, our
-	 * parent Hashjoin node is expected to know how to fish it out of our node
-	 * state.  Ugly but not really worth cleaning up, since Hashjoin knows
-	 * quite a bit more about Hash besides that.
+	 * get state info from node
 	 */
-	return NULL;
+	outerNode = outerPlanState(node);
+	hashtable = node->hashtable;
+
+	/*
+	 * set expression context
+	 */
+	hashkeys = node->hashkeys;
+	econtext = node->ps.ps_ExprContext;
+
+	/*
+	 * Synchronize the parallel hash table build.  At this stage we know that
+	 * the shared hash table has been or is being set up by
+	 * ExecHashTableCreate(), but we don't know if our peers have returned
+	 * from there or are here in MultiExecParallelHash(), and if so how far
+	 * through they are.  To find out, we check the build_barrier phase then
+	 * and jump to the right step in the build algorithm.
+	 */
+	pstate = hashtable->parallel_state;
+	build_barrier = &pstate->build_barrier;
+	Assert(BarrierPhase(build_barrier) >= PHJ_BUILD_ALLOCATING);
+	switch (BarrierPhase(build_barrier))
+	{
+		case PHJ_BUILD_ALLOCATING:
+
+			/*
+			 * Either I just allocated the initial hash table in
+			 * ExecHashTableCreate(), or someone else is doing that.  Either
+			 * way, wait for everyone to arrive here so we can proceed.
+			 */
+			BarrierArriveAndWait(build_barrier, WAIT_EVENT_HASH_BUILD_ALLOCATING);
+			/* Fall through. */
+
+		case PHJ_BUILD_HASHING_INNER:
+
+			/*
+			 * It's time to begin hashing, or if we just arrived here then
+			 * hashing is already underway, so join in that effort.  While
+			 * hashing we have to be prepared to help increase the number of
+			 * batches or buckets at any time, and if we arrived here when
+			 * that was already underway we'll have to help complete that work
+			 * immediately so that it's safe to access batches and buckets
+			 * below.
+			 */
+			if (PHJ_GROW_BATCHES_PHASE(BarrierAttach(&pstate->grow_batches_barrier)) !=
+				PHJ_GROW_BATCHES_ELECTING)
+				ExecParallelHashIncreaseNumBatches(hashtable);
+			if (PHJ_GROW_BUCKETS_PHASE(BarrierAttach(&pstate->grow_buckets_barrier)) !=
+				PHJ_GROW_BUCKETS_ELECTING)
+				ExecParallelHashIncreaseNumBuckets(hashtable);
+			ExecParallelHashEnsureBatchAccessors(hashtable);
+			ExecParallelHashTableSetCurrentBatch(hashtable, 0);
+			for (;;)
+			{
+				slot = ExecProcNode(outerNode);
+				if (TupIsNull(slot))
+					break;
+				econtext->ecxt_innertuple = slot;
+				if (ExecHashGetHashValue(hashtable, econtext, hashkeys,
+										 false, hashtable->keepNulls,
+										 &hashvalue))
+					ExecParallelHashTableInsert(hashtable, slot, hashvalue);
+				hashtable->partialTuples++;
+			}
+			BarrierDetach(&pstate->grow_buckets_barrier);
+			BarrierDetach(&pstate->grow_batches_barrier);
+
+			/*
+			 * Make sure that any tuples we wrote to disk are visible to
+			 * others before anyone tries to load them.
+			 */
+			for (i = 0; i < hashtable->nbatch; ++i)
+				sts_end_write(hashtable->batches[i].inner_tuples);
+
+			/*
+			 * Update shared counters.  We need an accurate total tuple count
+			 * to control the empty table optimization.
+			 */
+			ExecParallelHashMergeCounters(hashtable);
+
+			/*
+			 * Wait for everyone to finish building and flushing files and
+			 * counters.
+			 */
+			if (BarrierArriveAndWait(build_barrier,
+									 WAIT_EVENT_HASH_BUILD_HASHING_INNER))
+			{
+				/*
+				 * Elect one backend to disable any further growth.  Batches
+				 * are now fixed.  While building them we made sure they'd fit
+				 * in our memory budget when we load them back in later (or we
+				 * tried to do that and gave up because we detected extreme
+				 * skew).
+				 */
+				pstate->growth = PHJ_GROWTH_DISABLED;
+			}
+	}
+
+	/*
+	 * We're not yet attached to a batch.  We all agree on the dimensions and
+	 * number of inner tuples (for the empty table optimization).
+	 */
+	hashtable->curbatch = -1;
+	hashtable->nbuckets = pstate->nbuckets;
+	hashtable->log2_nbuckets = my_log2(hashtable->nbuckets);
+	hashtable->totalTuples = pstate->total_tuples;
+	ExecParallelHashEnsureBatchAccessors(hashtable);
+
+	/*
+	 * The next synchronization point is in ExecHashJoin's HJ_BUILD_HASHTABLE
+	 * case, which will bring the build phase to PHJ_BUILD_DONE (if it isn't
+	 * there already).
+	 */
+	Assert(BarrierPhase(build_barrier) == PHJ_BUILD_HASHING_OUTER ||
+		   BarrierPhase(build_barrier) == PHJ_BUILD_DONE);
 }
 
 /* ----------------------------------------------------------------
@@ -240,12 +429,15 @@ ExecEndHash(HashState *node)
  * ----------------------------------------------------------------
  */
 HashJoinTable
-ExecHashTableCreate(Hash *node, List *hashOperators, bool keepNulls)
+ExecHashTableCreate(HashState *state, List *hashOperators, bool keepNulls)
 {
+	Hash	   *node;
 	HashJoinTable hashtable;
 	Plan	   *outerNode;
+	size_t		space_allowed;
 	int			nbuckets;
 	int			nbatch;
+	double		rows;
 	int			num_skew_mcvs;
 	int			log2_nbuckets;
 	int			nkeys;
@@ -258,10 +450,22 @@ ExecHashTableCreate(Hash *node, List *hashOperators, bool keepNulls)
 	 * "outer" subtree of this node, but the inner relation of the hashjoin).
 	 * Compute the appropriate size of the hash table.
 	 */
+	node = (Hash *) state->ps.plan;
 	outerNode = outerPlan(node);
 
-	ExecChooseHashTableSize(outerNode->plan_rows, outerNode->plan_width,
+	/*
+	 * If this is shared hash table with a partial plan, then we can't use
+	 * outerNode->plan_rows to estimate its size.  We need an estimate of the
+	 * total number of rows across all copies of the partial plan.
+	 */
+	rows = node->plan.parallel_aware ? node->rows_total : outerNode->plan_rows;
+
+	ExecChooseHashTableSize(rows, outerNode->plan_width,
 							OidIsValid(node->skewTable),
+							state->parallel_state != NULL,
+							state->parallel_state != NULL ?
+							state->parallel_state->nparticipants - 1 : 0,
+							&space_allowed,
 							&nbuckets, &nbatch, &num_skew_mcvs);
 
 	/* nbuckets must be a power of 2 */
@@ -280,7 +484,7 @@ ExecHashTableCreate(Hash *node, List *hashOperators, bool keepNulls)
 	hashtable->nbuckets_optimal = nbuckets;
 	hashtable->log2_nbuckets = log2_nbuckets;
 	hashtable->log2_nbuckets_optimal = log2_nbuckets;
-	hashtable->buckets = NULL;
+	hashtable->buckets.unshared = NULL;
 	hashtable->keepNulls = keepNulls;
 	hashtable->skewEnabled = false;
 	hashtable->skewBucket = NULL;
@@ -293,16 +497,21 @@ ExecHashTableCreate(Hash *node, List *hashOperators, bool keepNulls)
 	hashtable->nbatch_outstart = nbatch;
 	hashtable->growEnabled = true;
 	hashtable->totalTuples = 0;
+	hashtable->partialTuples = 0;
 	hashtable->skewTuples = 0;
 	hashtable->innerBatchFile = NULL;
 	hashtable->outerBatchFile = NULL;
 	hashtable->spaceUsed = 0;
 	hashtable->spacePeak = 0;
-	hashtable->spaceAllowed = work_mem * 1024L;
+	hashtable->spaceAllowed = space_allowed;
 	hashtable->spaceUsedSkew = 0;
 	hashtable->spaceAllowedSkew =
 		hashtable->spaceAllowed * SKEW_WORK_MEM_PERCENT / 100;
 	hashtable->chunks = NULL;
+	hashtable->current_chunk = NULL;
+	hashtable->parallel_state = state->parallel_state;
+	hashtable->area = state->ps.state->es_query_dsa;
+	hashtable->batches = NULL;
 
 #ifdef HJDEBUG
 	printf("Hashjoin %p: initial nbatch = %d, nbuckets = %d\n",
@@ -351,10 +560,11 @@ ExecHashTableCreate(Hash *node, List *hashOperators, bool keepNulls)
 
 	oldcxt = MemoryContextSwitchTo(hashtable->hashCxt);
 
-	if (nbatch > 1)
+	if (nbatch > 1 && hashtable->parallel_state == NULL)
 	{
 		/*
-		 * allocate and initialize the file arrays in hashCxt
+		 * allocate and initialize the file arrays in hashCxt (not needed for
+		 * parallel case which uses shared tuplestores instead of raw files)
 		 */
 		hashtable->innerBatchFile = (BufFile **)
 			palloc0(nbatch * sizeof(BufFile *));
@@ -365,23 +575,77 @@ ExecHashTableCreate(Hash *node, List *hashOperators, bool keepNulls)
 		PrepareTempTablespaces();
 	}
 
-	/*
-	 * Prepare context for the first-scan space allocations; allocate the
-	 * hashbucket array therein, and set each bucket "empty".
-	 */
-	MemoryContextSwitchTo(hashtable->batchCxt);
+	MemoryContextSwitchTo(oldcxt);
 
-	hashtable->buckets = (HashJoinTuple *)
-		palloc0(nbuckets * sizeof(HashJoinTuple));
+	if (hashtable->parallel_state)
+	{
+		ParallelHashJoinState *pstate = hashtable->parallel_state;
+		Barrier    *build_barrier;
 
-	/*
-	 * Set up for skew optimization, if possible and there's a need for more
-	 * than one batch.  (In a one-batch join, there's no point in it.)
-	 */
-	if (nbatch > 1)
-		ExecHashBuildSkewHash(hashtable, node, num_skew_mcvs);
+		/*
+		 * Attach to the build barrier.  The corresponding detach operation is
+		 * in ExecHashTableDetach.  Note that we won't attach to the
+		 * batch_barrier for batch 0 yet.  We'll attach later and start it out
+		 * in PHJ_BATCH_PROBING phase, because batch 0 is allocated up front
+		 * and then loaded while hashing (the standard hybrid hash join
+		 * algorithm), and we'll coordinate that using build_barrier.
+		 */
+		build_barrier = &pstate->build_barrier;
+		BarrierAttach(build_barrier);
 
-	MemoryContextSwitchTo(oldcxt);
+		/*
+		 * So far we have no idea whether there are any other participants,
+		 * and if so, what phase they are working on.  The only thing we care
+		 * about at this point is whether someone has already created the
+		 * SharedHashJoinBatch objects and the hash table for batch 0.  One
+		 * backend will be elected to do that now if necessary.
+		 */
+		if (BarrierPhase(build_barrier) == PHJ_BUILD_ELECTING &&
+			BarrierArriveAndWait(build_barrier, WAIT_EVENT_HASH_BUILD_ELECTING))
+		{
+			pstate->nbatch = nbatch;
+			pstate->space_allowed = space_allowed;
+			pstate->growth = PHJ_GROWTH_OK;
+
+			/* Set up the shared state for coordinating batches. */
+			ExecParallelHashJoinSetUpBatches(hashtable, nbatch);
+
+			/*
+			 * Allocate batch 0's hash table up front so we can load it
+			 * directly while hashing.
+			 */
+			pstate->nbuckets = nbuckets;
+			ExecParallelHashTableAlloc(hashtable, 0);
+		}
+
+		/*
+		 * The next Parallel Hash synchronization point is in
+		 * MultiExecParallelHash(), which will progress it all the way to
+		 * PHJ_BUILD_DONE.  The caller must not return control from this
+		 * executor node between now and then.
+		 */
+	}
+	else
+	{
+		/*
+		 * Prepare context for the first-scan space allocations; allocate the
+		 * hashbucket array therein, and set each bucket "empty".
+		 */
+		MemoryContextSwitchTo(hashtable->batchCxt);
+
+		hashtable->buckets.unshared = (HashJoinTuple *)
+			palloc0(nbuckets * sizeof(HashJoinTuple));
+
+		/*
+		 * Set up for skew optimization, if possible and there's a need for
+		 * more than one batch.  (In a one-batch join, there's no point in
+		 * it.)
+		 */
+		if (nbatch > 1)
+			ExecHashBuildSkewHash(hashtable, node, num_skew_mcvs);
+
+		MemoryContextSwitchTo(oldcxt);
+	}
 
 	return hashtable;
 }
@@ -399,6 +663,9 @@ ExecHashTableCreate(Hash *node, List *hashOperators, bool keepNulls)
 
 void
 ExecChooseHashTableSize(double ntuples, int tupwidth, bool useskew,
+						bool try_combined_work_mem,
+						int parallel_workers,
+						size_t *space_allowed,
 						int *numbuckets,
 						int *numbatches,
 						int *num_skew_mcvs)
@@ -433,6 +700,16 @@ ExecChooseHashTableSize(double ntuples, int tupwidth, bool useskew,
 	 */
 	hash_table_bytes = work_mem * 1024L;
 
+	/*
+	 * Parallel Hash tries to use the combined work_mem of all workers to
+	 * avoid the need to batch.  If that won't work, it falls back to work_mem
+	 * per worker and tries to process batches in parallel.
+	 */
+	if (try_combined_work_mem)
+		hash_table_bytes += hash_table_bytes * parallel_workers;
+
+	*space_allowed = hash_table_bytes;
+
 	/*
 	 * If skew optimization is possible, estimate the number of skew buckets
 	 * that will fit in the memory allowed, and decrement the assumed space
@@ -478,7 +755,7 @@ ExecChooseHashTableSize(double ntuples, int tupwidth, bool useskew,
 	 * Note that both nbuckets and nbatch must be powers of 2 to make
 	 * ExecHashGetBucketAndBatch fast.
 	 */
-	max_pointers = (work_mem * 1024L) / sizeof(HashJoinTuple);
+	max_pointers = *space_allowed / sizeof(HashJoinTuple);
 	max_pointers = Min(max_pointers, MaxAllocSize / sizeof(HashJoinTuple));
 	/* If max_pointers isn't a power of 2, must round it down to one */
 	mppow2 = 1L << my_log2(max_pointers);
@@ -510,6 +787,21 @@ ExecChooseHashTableSize(double ntuples, int tupwidth, bool useskew,
 		int			minbatch;
 		long		bucket_size;
 
+		/*
+		 * If Parallel Hash with combined work_mem would still need multiple
+		 * batches, we'll have to fall back to regular work_mem budget.
+		 */
+		if (try_combined_work_mem)
+		{
+			ExecChooseHashTableSize(ntuples, tupwidth, useskew,
+									false, parallel_workers,
+									space_allowed,
+									numbuckets,
+									numbatches,
+									num_skew_mcvs);
+			return;
+		}
+
 		/*
 		 * Estimate the number of buckets we'll want to have when work_mem is
 		 * entirely full.  Each bucket will contain a bucket pointer plus
@@ -564,14 +856,17 @@ ExecHashTableDestroy(HashJoinTable hashtable)
 	/*
 	 * Make sure all the temp files are closed.  We skip batch 0, since it
 	 * can't have any temp files (and the arrays might not even exist if
-	 * nbatch is only 1).
+	 * nbatch is only 1).  Parallel hash joins don't use these files.
 	 */
-	for (i = 1; i < hashtable->nbatch; i++)
+	if (hashtable->innerBatchFile != NULL)
 	{
-		if (hashtable->innerBatchFile[i])
-			BufFileClose(hashtable->innerBatchFile[i]);
-		if (hashtable->outerBatchFile[i])
-			BufFileClose(hashtable->outerBatchFile[i]);
+		for (i = 1; i < hashtable->nbatch; i++)
+		{
+			if (hashtable->innerBatchFile[i])
+				BufFileClose(hashtable->innerBatchFile[i]);
+			if (hashtable->outerBatchFile[i])
+				BufFileClose(hashtable->outerBatchFile[i]);
+		}
 	}
 
 	/* Release working memory (batchCxt is a child, so it goes away too) */
@@ -657,8 +952,9 @@ ExecHashIncreaseNumBatches(HashJoinTable hashtable)
 		hashtable->nbuckets = hashtable->nbuckets_optimal;
 		hashtable->log2_nbuckets = hashtable->log2_nbuckets_optimal;
 
-		hashtable->buckets = repalloc(hashtable->buckets,
-									  sizeof(HashJoinTuple) * hashtable->nbuckets);
+		hashtable->buckets.unshared =
+			repalloc(hashtable->buckets.unshared,
+					 sizeof(HashJoinTuple) * hashtable->nbuckets);
 	}
 
 	/*
@@ -666,14 +962,15 @@ ExecHashIncreaseNumBatches(HashJoinTable hashtable)
 	 * buckets now and not have to keep track which tuples in the buckets have
 	 * already been processed. We will free the old chunks as we go.
 	 */
-	memset(hashtable->buckets, 0, sizeof(HashJoinTuple) * hashtable->nbuckets);
+	memset(hashtable->buckets.unshared, 0,
+		   sizeof(HashJoinTuple) * hashtable->nbuckets);
 	oldchunks = hashtable->chunks;
 	hashtable->chunks = NULL;
 
 	/* so, let's scan through the old chunks, and all tuples in each chunk */
 	while (oldchunks != NULL)
 	{
-		HashMemoryChunk nextchunk = oldchunks->next;
+		HashMemoryChunk nextchunk = oldchunks->next.unshared;
 
 		/* position within the buffer (up to oldchunks->used) */
 		size_t		idx = 0;
@@ -700,8 +997,8 @@ ExecHashIncreaseNumBatches(HashJoinTable hashtable)
 				memcpy(copyTuple, hashTuple, hashTupleSize);
 
 				/* and add it back to the appropriate bucket */
-				copyTuple->next = hashtable->buckets[bucketno];
-				hashtable->buckets[bucketno] = copyTuple;
+				copyTuple->next.unshared = hashtable->buckets.unshared[bucketno];
+				hashtable->buckets.unshared[bucketno] = copyTuple;
 			}
 			else
 			{
@@ -750,6 +1047,379 @@ ExecHashIncreaseNumBatches(HashJoinTable hashtable)
 	}
 }
 
+/*
+ * ExecParallelHashIncreaseNumBatches
+ *		Every participant attached to grow_barrier must run this function
+ *		when it observes growth == PHJ_GROWTH_NEED_MORE_BATCHES.
+ */
+static void
+ExecParallelHashIncreaseNumBatches(HashJoinTable hashtable)
+{
+	ParallelHashJoinState *pstate = hashtable->parallel_state;
+	int			i;
+
+	Assert(BarrierPhase(&pstate->build_barrier) == PHJ_BUILD_HASHING_INNER);
+
+	/*
+	 * It's unlikely, but we need to be prepared for new participants to show
+	 * up while we're in the middle of this operation so we need to switch on
+	 * barrier phase here.
+	 */
+	switch (PHJ_GROW_BATCHES_PHASE(BarrierPhase(&pstate->grow_batches_barrier)))
+	{
+		case PHJ_GROW_BATCHES_ELECTING:
+			/*
+			 * Elect one participant to prepare to grow the number of batches.
+			 * This involves reallocating or resetting the buckets of batch 0
+			 * in preparation for all participants to begin repartitioning the
+			 * tuples.
+			 */
+			if (BarrierArriveAndWait(&pstate->grow_batches_barrier,
+									 WAIT_EVENT_HASH_GROW_BATCHES_ELECTING))
+			{
+				dsa_pointer_atomic *buckets;
+				ParallelHashJoinBatch *old_batch0;
+				int			new_nbatch;
+				int			i;
+
+				/* Move the old batch out of the way. */
+				old_batch0 = hashtable->batches[0].shared;
+				pstate->old_batches = pstate->batches;
+				pstate->old_nbatch = hashtable->nbatch;
+				pstate->batches = InvalidDsaPointer;
+
+				/* Free this backend's old accessors. */
+				ExecParallelHashCloseBatchAccessors(hashtable);
+
+				/* Figure out how many batches to use. */
+				if (hashtable->nbatch == 1)
+				{
+					/*
+					 * We are going from single-batch to multi-batch.  We need
+					 * to switch from one large combined memory budget to the
+					 * regular work_mem budget.
+					 */
+					pstate->space_allowed = work_mem * 1024L;
+
+					/*
+					 * The combined work_mem of all participants wasn't
+					 * enough. Therefore one batch per participant would be
+					 * approximately equivalent and would probably also be
+					 * insufficient.  So try two batches per particiant,
+					 * rounded up to a power of two.
+					 */
+					new_nbatch = 1 << my_log2(pstate->nparticipants * 2);
+				}
+				else
+				{
+					/*
+					 * We were already multi-batched.  Try doubling the number
+					 * of batches.
+					 */
+					new_nbatch = hashtable->nbatch * 2;
+				}
+
+				/* Allocate new larger generation of batches. */
+				Assert(hashtable->nbatch == pstate->nbatch);
+				ExecParallelHashJoinSetUpBatches(hashtable, new_nbatch);
+				Assert(hashtable->nbatch == pstate->nbatch);
+
+				/* Replace or recycle batch 0's bucket array. */
+				if (pstate->old_nbatch == 1)
+				{
+					double		dtuples;
+					double		dbuckets;
+					int			new_nbuckets;
+
+					/*
+					 * We probably also need a smaller bucket array.  How many
+					 * tuples do we expect per batch, assuming we have only
+					 * half of them so far?  Normally we don't need to change
+					 * the bucket array's size, because the size of each batch
+					 * stays the same as we add more batches, but in this
+					 * special case we move from a large batch to many smaller
+					 * batches and it would be wasteful to keep the large
+					 * array.
+					 */
+					dtuples = (old_batch0->ntuples * 2.0) / new_nbatch;
+					dbuckets = ceil(dtuples / NTUP_PER_BUCKET);
+					dbuckets = Min(dbuckets,
+								   MaxAllocSize / sizeof(dsa_pointer_atomic));
+					new_nbuckets = (int) dbuckets;
+					new_nbuckets = Max(new_nbuckets, 1024);
+					new_nbuckets = 1 << my_log2(new_nbuckets);
+					dsa_free(hashtable->area, old_batch0->buckets);
+					hashtable->batches[0].shared->buckets =
+						dsa_allocate(hashtable->area,
+									 sizeof(dsa_pointer_atomic) * new_nbuckets);
+					buckets = (dsa_pointer_atomic *)
+						dsa_get_address(hashtable->area,
+										hashtable->batches[0].shared->buckets);
+					for (i = 0; i < new_nbuckets; ++i)
+						dsa_pointer_atomic_init(&buckets[i], InvalidDsaPointer);
+					pstate->nbuckets = new_nbuckets;
+				}
+				else
+				{
+					/* Recycle the existing bucket array. */
+					hashtable->batches[0].shared->buckets = old_batch0->buckets;
+					buckets = (dsa_pointer_atomic *)
+						dsa_get_address(hashtable->area, old_batch0->buckets);
+					for (i = 0; i < hashtable->nbuckets; ++i)
+						dsa_pointer_atomic_write(&buckets[i], InvalidDsaPointer);
+				}
+
+				/* Move all chunks to the work queue for parallel processing. */
+				pstate->chunk_work_queue = old_batch0->chunks;
+
+				/* Disable further growth temporarily while we're growing. */
+				pstate->growth = PHJ_GROWTH_DISABLED;
+			}
+			else
+			{
+				/* All other participants just flush their tuples to disk. */
+				ExecParallelHashCloseBatchAccessors(hashtable);
+			}
+			/* Fall through. */
+
+		case PHJ_GROW_BATCHES_ALLOCATING:
+			/* Wait for the above to be finished. */
+			BarrierArriveAndWait(&pstate->grow_batches_barrier,
+								 WAIT_EVENT_HASH_GROW_BATCHES_ALLOCATING);
+			/* Fall through. */
+
+		case PHJ_GROW_BATCHES_REPARTITIONING:
+			/* Make sure that we have the current dimensions and buckets. */
+			ExecParallelHashEnsureBatchAccessors(hashtable);
+			ExecParallelHashTableSetCurrentBatch(hashtable, 0);
+			/* Then partition, flush counters. */
+			ExecParallelHashRepartitionFirst(hashtable);
+			ExecParallelHashRepartitionRest(hashtable);
+			ExecParallelHashMergeCounters(hashtable);
+			/* Wait for the above to be finished. */
+			BarrierArriveAndWait(&pstate->grow_batches_barrier,
+								 WAIT_EVENT_HASH_GROW_BATCHES_REPARTITIONING);
+			/* Fall through. */
+
+		case PHJ_GROW_BATCHES_DECIDING:
+
+			/*
+			 * Elect one participant to clean up and decide whether further
+			 * repartitioning is needed, or should be disabled because it's
+			 * not helping.
+			 */
+			if (BarrierArriveAndWait(&pstate->grow_batches_barrier,
+									 WAIT_EVENT_HASH_GROW_BATCHES_DECIDING))
+			{
+				bool		space_exhausted = false;
+				bool		extreme_skew_detected = false;
+
+				/* Make sure that we have the current dimensions and buckets. */
+				ExecParallelHashEnsureBatchAccessors(hashtable);
+				ExecParallelHashTableSetCurrentBatch(hashtable, 0);
+
+				/* Are any of the new generation of batches exhausted? */
+				for (i = 0; i < hashtable->nbatch; ++i)
+				{
+					ParallelHashJoinBatch *batch = hashtable->batches[i].shared;
+
+					if (batch->space_exhausted ||
+						batch->estimated_size > pstate->space_allowed)
+					{
+						int			parent;
+
+						space_exhausted = true;
+
+						/*
+						 * Did this batch receive ALL of the tuples from its
+						 * parent batch?  That would indicate that further
+						 * repartitioning isn't going to help (the hash values
+						 * are probably all the same).
+						 */
+						parent = i % pstate->old_nbatch;
+						if (batch->ntuples == hashtable->batches[parent].shared->old_ntuples)
+							extreme_skew_detected = true;
+					}
+				}
+
+				/* Don't keep growing if it's not helping or we'd overflow. */
+				if (extreme_skew_detected || hashtable->nbatch >= INT_MAX / 2)
+					pstate->growth = PHJ_GROWTH_DISABLED;
+				else if (space_exhausted)
+					pstate->growth = PHJ_GROWTH_NEED_MORE_BATCHES;
+				else
+					pstate->growth = PHJ_GROWTH_OK;
+
+				/* Free the old batches in shared memory. */
+				dsa_free(hashtable->area, pstate->old_batches);
+				pstate->old_batches = InvalidDsaPointer;
+			}
+			/* Fall through. */
+
+		case PHJ_GROW_BATCHES_FINISHING:
+			/* Wait for the above to complete. */
+			BarrierArriveAndWait(&pstate->grow_batches_barrier,
+								 WAIT_EVENT_HASH_GROW_BATCHES_FINISHING);
+	}
+}
+
+/*
+ * Repartition the tuples currently loaded into memory for inner batch 0
+ * because the number of batches has been increased.  Some tuples are retained
+ * in memory and some are written out to a later batch.
+ */
+static void
+ExecParallelHashRepartitionFirst(HashJoinTable hashtable)
+{
+	dsa_pointer chunk_shared;
+	HashMemoryChunk chunk;
+
+	Assert(hashtable->nbatch = hashtable->parallel_state->nbatch);
+
+	while ((chunk = ExecParallelHashPopChunkQueue(hashtable, &chunk_shared)))
+	{
+		size_t		idx = 0;
+
+		/* Repartition all tuples in this chunk. */
+		while (idx < chunk->used)
+		{
+			HashJoinTuple hashTuple = (HashJoinTuple) (chunk->data + idx);
+			MinimalTuple tuple = HJTUPLE_MINTUPLE(hashTuple);
+			HashJoinTuple copyTuple;
+			dsa_pointer shared;
+			int			bucketno;
+			int			batchno;
+
+			ExecHashGetBucketAndBatch(hashtable, hashTuple->hashvalue,
+									  &bucketno, &batchno);
+
+			Assert(batchno < hashtable->nbatch);
+			if (batchno == 0)
+			{
+				/* It still belongs in batch 0.  Copy to a new chunk. */
+				copyTuple =
+					ExecParallelHashTupleAlloc(hashtable,
+											   HJTUPLE_OVERHEAD + tuple->t_len,
+											   &shared);
+				copyTuple->hashvalue = hashTuple->hashvalue;
+				memcpy(HJTUPLE_MINTUPLE(copyTuple), tuple, tuple->t_len);
+				ExecParallelHashPushTuple(&hashtable->buckets.shared[bucketno],
+										  copyTuple, shared);
+			}
+			else
+			{
+				size_t		tuple_size =
+				MAXALIGN(HJTUPLE_OVERHEAD + tuple->t_len);
+
+				/* It belongs in a later batch. */
+				hashtable->batches[batchno].estimated_size += tuple_size;
+				sts_puttuple(hashtable->batches[batchno].inner_tuples,
+							 &hashTuple->hashvalue, tuple);
+			}
+
+			/* Count this tuple. */
+			++hashtable->batches[0].old_ntuples;
+			++hashtable->batches[batchno].ntuples;
+
+			idx += MAXALIGN(HJTUPLE_OVERHEAD +
+							HJTUPLE_MINTUPLE(hashTuple)->t_len);
+		}
+
+		/* Free this chunk. */
+		dsa_free(hashtable->area, chunk_shared);
+
+		CHECK_FOR_INTERRUPTS();
+	}
+}
+
+/*
+ * Help repartition inner batches 1..n.
+ */
+static void
+ExecParallelHashRepartitionRest(HashJoinTable hashtable)
+{
+	ParallelHashJoinState *pstate = hashtable->parallel_state;
+	int			old_nbatch = pstate->old_nbatch;
+	SharedTuplestoreAccessor **old_inner_tuples;
+	ParallelHashJoinBatch *old_batches;
+	int			i;
+
+	/* Get our hands on the previous generation of batches. */
+	old_batches = (ParallelHashJoinBatch *)
+		dsa_get_address(hashtable->area, pstate->old_batches);
+	old_inner_tuples = palloc0(sizeof(SharedTuplestoreAccessor *) * old_nbatch);
+	for (i = 1; i < old_nbatch; ++i)
+	{
+		ParallelHashJoinBatch *shared =
+		NthParallelHashJoinBatch(old_batches, i);
+
+		old_inner_tuples[i] = sts_attach(ParallelHashJoinBatchInner(shared),
+										 ParallelWorkerNumber + 1,
+										 &pstate->fileset);
+	}
+
+	/* Join in the effort to repartition them. */
+	for (i = 1; i < old_nbatch; ++i)
+	{
+		MinimalTuple tuple;
+		uint32		hashvalue;
+
+		/* Scan one partition from the previous generation. */
+		sts_begin_parallel_scan(old_inner_tuples[i]);
+		while ((tuple = sts_parallel_scan_next(old_inner_tuples[i], &hashvalue)))
+		{
+			size_t		tuple_size = MAXALIGN(HJTUPLE_OVERHEAD + tuple->t_len);
+			int			bucketno;
+			int			batchno;
+
+			/* Decide which partition it goes to in the new generation. */
+			ExecHashGetBucketAndBatch(hashtable, hashvalue, &bucketno,
+									  &batchno);
+
+			hashtable->batches[batchno].estimated_size += tuple_size;
+			++hashtable->batches[batchno].ntuples;
+			++hashtable->batches[i].old_ntuples;
+
+			/* Store the tuple its new batch. */
+			sts_puttuple(hashtable->batches[batchno].inner_tuples,
+						 &hashvalue, tuple);
+
+			CHECK_FOR_INTERRUPTS();
+		}
+		sts_end_parallel_scan(old_inner_tuples[i]);
+	}
+
+	pfree(old_inner_tuples);
+}
+
+/*
+ * Transfer the backend-local per-batch counters to the shared totals.
+ */
+static void
+ExecParallelHashMergeCounters(HashJoinTable hashtable)
+{
+	ParallelHashJoinState *pstate = hashtable->parallel_state;
+	int			i;
+
+	LWLockAcquire(&pstate->lock, LW_EXCLUSIVE);
+	pstate->total_tuples = 0;
+	for (i = 0; i < hashtable->nbatch; ++i)
+	{
+		ParallelHashJoinBatchAccessor *batch = &hashtable->batches[i];
+
+		batch->shared->size += batch->size;
+		batch->shared->estimated_size += batch->estimated_size;
+		batch->shared->ntuples += batch->ntuples;
+		batch->shared->old_ntuples += batch->old_ntuples;
+		batch->size = 0;
+		batch->estimated_size = 0;
+		batch->ntuples = 0;
+		batch->old_ntuples = 0;
+		pstate->total_tuples += batch->shared->ntuples;
+	}
+	LWLockRelease(&pstate->lock);
+}
+
 /*
  * ExecHashIncreaseNumBuckets
  *		increase the original number of buckets in order to reduce
@@ -782,14 +1452,15 @@ ExecHashIncreaseNumBuckets(HashJoinTable hashtable)
 	 * ExecHashIncreaseNumBatches, but without all the copying into new
 	 * chunks)
 	 */
-	hashtable->buckets =
-		(HashJoinTuple *) repalloc(hashtable->buckets,
+	hashtable->buckets.unshared =
+		(HashJoinTuple *) repalloc(hashtable->buckets.unshared,
 								   hashtable->nbuckets * sizeof(HashJoinTuple));
 
-	memset(hashtable->buckets, 0, hashtable->nbuckets * sizeof(HashJoinTuple));
+	memset(hashtable->buckets.unshared, 0,
+		   hashtable->nbuckets * sizeof(HashJoinTuple));
 
 	/* scan through all tuples in all chunks to rebuild the hash table */
-	for (chunk = hashtable->chunks; chunk != NULL; chunk = chunk->next)
+	for (chunk = hashtable->chunks; chunk != NULL; chunk = chunk->next.unshared)
 	{
 		/* process all tuples stored in this chunk */
 		size_t		idx = 0;
@@ -804,8 +1475,8 @@ ExecHashIncreaseNumBuckets(HashJoinTable hashtable)
 									  &bucketno, &batchno);
 
 			/* add the tuple to the proper bucket */
-			hashTuple->next = hashtable->buckets[bucketno];
-			hashtable->buckets[bucketno] = hashTuple;
+			hashTuple->next.unshared = hashtable->buckets.unshared[bucketno];
+			hashtable->buckets.unshared[bucketno] = hashTuple;
 
 			/* advance index past the tuple */
 			idx += MAXALIGN(HJTUPLE_OVERHEAD +
@@ -817,14 +1488,101 @@ ExecHashIncreaseNumBuckets(HashJoinTable hashtable)
 	}
 }
 
+static void
+ExecParallelHashIncreaseNumBuckets(HashJoinTable hashtable)
+{
+	ParallelHashJoinState *pstate = hashtable->parallel_state;
+	int			i;
+	HashMemoryChunk chunk;
+	dsa_pointer chunk_s;
 
-/*
- * ExecHashTableInsert
- *		insert a tuple into the hash table depending on the hash value
- *		it may just go to a temp file for later batches
- *
- * Note: the passed TupleTableSlot may contain a regular, minimal, or virtual
- * tuple; the minimal case in particular is certain to happen while reloading
+	Assert(BarrierPhase(&pstate->build_barrier) == PHJ_BUILD_HASHING_INNER);
+
+	/*
+	 * It's unlikely, but we need to be prepared for new participants to show
+	 * up while we're in the middle of this operation so we need to switch on
+	 * barrier phase here.
+	 */
+	switch (PHJ_GROW_BUCKETS_PHASE(BarrierPhase(&pstate->grow_buckets_barrier)))
+	{
+		case PHJ_GROW_BUCKETS_ELECTING:
+			/* Elect one participant to prepare to increase nbuckets. */
+			if (BarrierArriveAndWait(&pstate->grow_buckets_barrier,
+									 WAIT_EVENT_HASH_GROW_BUCKETS_ELECTING))
+			{
+				size_t		size;
+				dsa_pointer_atomic *buckets;
+
+				/* Double the size of the bucket array. */
+				pstate->nbuckets *= 2;
+				size = pstate->nbuckets * sizeof(dsa_pointer_atomic);
+				hashtable->batches[0].shared->size += size / 2;
+				dsa_free(hashtable->area, hashtable->batches[0].shared->buckets);
+				hashtable->batches[0].shared->buckets =
+					dsa_allocate(hashtable->area, size);
+				buckets = (dsa_pointer_atomic *)
+					dsa_get_address(hashtable->area,
+									hashtable->batches[0].shared->buckets);
+				for (i = 0; i < pstate->nbuckets; ++i)
+					dsa_pointer_atomic_init(&buckets[i], InvalidDsaPointer);
+
+				/* Put the chunk list onto the work queue. */
+				pstate->chunk_work_queue = hashtable->batches[0].shared->chunks;
+
+				/* Clear the flag. */
+				pstate->growth = PHJ_GROWTH_OK;
+			}
+			/* Fall through. */
+
+		case PHJ_GROW_BUCKETS_ALLOCATING:
+			/* Wait for the above to complete. */
+			BarrierArriveAndWait(&pstate->grow_buckets_barrier,
+								 WAIT_EVENT_HASH_GROW_BUCKETS_ALLOCATING);
+			/* Fall through. */
+
+		case PHJ_GROW_BUCKETS_REINSERTING:
+			/* Reinsert all tuples into the hash table. */
+			ExecParallelHashEnsureBatchAccessors(hashtable);
+			ExecParallelHashTableSetCurrentBatch(hashtable, 0);
+			while ((chunk = ExecParallelHashPopChunkQueue(hashtable, &chunk_s)))
+			{
+				size_t		idx = 0;
+
+				while (idx < chunk->used)
+				{
+					HashJoinTuple hashTuple = (HashJoinTuple) (chunk->data + idx);
+					dsa_pointer shared = chunk_s + HASH_CHUNK_HEADER_SIZE + idx;
+					int			bucketno;
+					int			batchno;
+
+					ExecHashGetBucketAndBatch(hashtable, hashTuple->hashvalue,
+											  &bucketno, &batchno);
+					Assert(batchno == 0);
+
+					/* add the tuple to the proper bucket */
+					ExecParallelHashPushTuple(&hashtable->buckets.shared[bucketno],
+											  hashTuple, shared);
+
+					/* advance index past the tuple */
+					idx += MAXALIGN(HJTUPLE_OVERHEAD +
+									HJTUPLE_MINTUPLE(hashTuple)->t_len);
+				}
+
+				/* allow this loop to be cancellable */
+				CHECK_FOR_INTERRUPTS();
+			}
+			BarrierArriveAndWait(&pstate->grow_buckets_barrier,
+								 WAIT_EVENT_HASH_GROW_BUCKETS_REINSERTING);
+	}
+}
+
+/*
+ * ExecHashTableInsert
+ *		insert a tuple into the hash table depending on the hash value
+ *		it may just go to a temp file for later batches
+ *
+ * Note: the passed TupleTableSlot may contain a regular, minimal, or virtual
+ * tuple; the minimal case in particular is certain to happen while reloading
  * tuples from batch files.  We could save some cycles in the regular-tuple
  * case by not forcing the slot contents into minimal form; not clear if it's
  * worth the messiness required.
@@ -869,8 +1627,8 @@ ExecHashTableInsert(HashJoinTable hashtable,
 		HeapTupleHeaderClearMatch(HJTUPLE_MINTUPLE(hashTuple));
 
 		/* Push it onto the front of the bucket's list */
-		hashTuple->next = hashtable->buckets[bucketno];
-		hashtable->buckets[bucketno] = hashTuple;
+		hashTuple->next.unshared = hashtable->buckets.unshared[bucketno];
+		hashtable->buckets.unshared[bucketno] = hashTuple;
 
 		/*
 		 * Increase the (optimal) number of buckets if we just exceeded the
@@ -910,6 +1668,94 @@ ExecHashTableInsert(HashJoinTable hashtable,
 	}
 }
 
+/*
+ * ExecHashTableParallelInsert
+ *		insert a tuple into a shared hash table or shared batch tuplestore
+ */
+void
+ExecParallelHashTableInsert(HashJoinTable hashtable,
+							TupleTableSlot *slot,
+							uint32 hashvalue)
+{
+	MinimalTuple tuple = ExecFetchSlotMinimalTuple(slot);
+	dsa_pointer shared;
+	int			bucketno;
+	int			batchno;
+
+retry:
+	ExecHashGetBucketAndBatch(hashtable, hashvalue, &bucketno, &batchno);
+
+	if (batchno == 0)
+	{
+		HashJoinTuple hashTuple;
+
+		/* Try to load it into memory. */
+		Assert(BarrierPhase(&hashtable->parallel_state->build_barrier) ==
+			   PHJ_BUILD_HASHING_INNER);
+		hashTuple = ExecParallelHashTupleAlloc(hashtable,
+											   HJTUPLE_OVERHEAD + tuple->t_len,
+											   &shared);
+		if (hashTuple == NULL)
+			goto retry;
+
+		/* Store the hash value in the HashJoinTuple header. */
+		hashTuple->hashvalue = hashvalue;
+		memcpy(HJTUPLE_MINTUPLE(hashTuple), tuple, tuple->t_len);
+
+		/* Push it onto the front of the bucket's list */
+		ExecParallelHashPushTuple(&hashtable->buckets.shared[bucketno],
+								  hashTuple, shared);
+	}
+	else
+	{
+		size_t		tuple_size = MAXALIGN(HJTUPLE_OVERHEAD + tuple->t_len);
+
+		Assert(batchno > 0);
+
+		/* Try to preallocate space in the batch if necessary. */
+		if (hashtable->batches[batchno].preallocated < tuple_size)
+		{
+			if (!ExecParallelHashTuplePrealloc(hashtable, batchno, tuple_size))
+				goto retry;
+		}
+
+		Assert(hashtable->batches[batchno].preallocated >= tuple_size);
+		hashtable->batches[batchno].preallocated -= tuple_size;
+		sts_puttuple(hashtable->batches[batchno].inner_tuples, &hashvalue,
+					 tuple);
+	}
+	++hashtable->batches[batchno].ntuples;
+}
+
+/*
+ * Insert a tuple into the current hash table.  Unlike
+ * ExecParallelHashTableInsert, this version is not prepared to send the tuple
+ * to other batches or to run out of memory, and should only be called with
+ * tuples that belong in the current batch once growth has been disabled.
+ */
+void
+ExecParallelHashTableInsertCurrentBatch(HashJoinTable hashtable,
+										TupleTableSlot *slot,
+										uint32 hashvalue)
+{
+	MinimalTuple tuple = ExecFetchSlotMinimalTuple(slot);
+	HashJoinTuple hashTuple;
+	dsa_pointer shared;
+	int			batchno;
+	int			bucketno;
+
+	ExecHashGetBucketAndBatch(hashtable, hashvalue, &bucketno, &batchno);
+	Assert(batchno == hashtable->curbatch);
+	hashTuple = ExecParallelHashTupleAlloc(hashtable,
+										   HJTUPLE_OVERHEAD + tuple->t_len,
+										   &shared);
+	hashTuple->hashvalue = hashvalue;
+	memcpy(HJTUPLE_MINTUPLE(hashTuple), tuple, tuple->t_len);
+	HeapTupleHeaderClearMatch(HJTUPLE_MINTUPLE(hashTuple));
+	ExecParallelHashPushTuple(&hashtable->buckets.shared[bucketno],
+							  hashTuple, shared);
+}
+
 /*
  * ExecHashGetHashValue
  *		Compute the hash value for a tuple
@@ -1076,11 +1922,11 @@ ExecScanHashBucket(HashJoinState *hjstate,
 	 * otherwise scan the standard hashtable bucket.
 	 */
 	if (hashTuple != NULL)
-		hashTuple = hashTuple->next;
+		hashTuple = hashTuple->next.unshared;
 	else if (hjstate->hj_CurSkewBucketNo != INVALID_SKEW_BUCKET_NO)
 		hashTuple = hashtable->skewBucket[hjstate->hj_CurSkewBucketNo]->tuples;
 	else
-		hashTuple = hashtable->buckets[hjstate->hj_CurBucketNo];
+		hashTuple = hashtable->buckets.unshared[hjstate->hj_CurBucketNo];
 
 	while (hashTuple != NULL)
 	{
@@ -1104,7 +1950,67 @@ ExecScanHashBucket(HashJoinState *hjstate,
 			}
 		}
 
-		hashTuple = hashTuple->next;
+		hashTuple = hashTuple->next.unshared;
+	}
+
+	/*
+	 * no match
+	 */
+	return false;
+}
+
+/*
+ * ExecParallelScanHashBucket
+ *		scan a hash bucket for matches to the current outer tuple
+ *
+ * The current outer tuple must be stored in econtext->ecxt_outertuple.
+ *
+ * On success, the inner tuple is stored into hjstate->hj_CurTuple and
+ * econtext->ecxt_innertuple, using hjstate->hj_HashTupleSlot as the slot
+ * for the latter.
+ */
+bool
+ExecParallelScanHashBucket(HashJoinState *hjstate,
+				   ExprContext *econtext)
+{
+	ExprState  *hjclauses = hjstate->hashclauses;
+	HashJoinTable hashtable = hjstate->hj_HashTable;
+	HashJoinTuple hashTuple = hjstate->hj_CurTuple;
+	uint32		hashvalue = hjstate->hj_CurHashValue;
+
+	/*
+	 * hj_CurTuple is the address of the tuple last returned from the current
+	 * bucket, or NULL if it's time to start scanning a new bucket.
+	 */
+	if (hashTuple != NULL)
+		hashTuple = ExecParallelHashNextTuple(hashtable, hashTuple);
+	else
+		hashTuple = ExecParallelHashFirstTuple(hashtable,
+											   hjstate->hj_CurBucketNo);
+
+	while (hashTuple != NULL)
+	{
+		if (hashTuple->hashvalue == hashvalue)
+		{
+			TupleTableSlot *inntuple;
+
+			/* insert hashtable's tuple into exec slot so ExecQual sees it */
+			inntuple = ExecStoreMinimalTuple(HJTUPLE_MINTUPLE(hashTuple),
+											 hjstate->hj_HashTupleSlot,
+											 false);	/* do not pfree */
+			econtext->ecxt_innertuple = inntuple;
+
+			/* reset temp memory each time to avoid leaks from qual expr */
+			ResetExprContext(econtext);
+
+			if (ExecQual(hjclauses, econtext))
+			{
+				hjstate->hj_CurTuple = hashTuple;
+				return true;
+			}
+		}
+
+		hashTuple = ExecParallelHashNextTuple(hashtable, hashTuple);
 	}
 
 	/*
@@ -1155,10 +2061,10 @@ ExecScanHashTableForUnmatched(HashJoinState *hjstate, ExprContext *econtext)
 		 * bucket.
 		 */
 		if (hashTuple != NULL)
-			hashTuple = hashTuple->next;
+			hashTuple = hashTuple->next.unshared;
 		else if (hjstate->hj_CurBucketNo < hashtable->nbuckets)
 		{
-			hashTuple = hashtable->buckets[hjstate->hj_CurBucketNo];
+			hashTuple = hashtable->buckets.unshared[hjstate->hj_CurBucketNo];
 			hjstate->hj_CurBucketNo++;
 		}
 		else if (hjstate->hj_CurSkewBucketNo < hashtable->nSkewBuckets)
@@ -1194,7 +2100,7 @@ ExecScanHashTableForUnmatched(HashJoinState *hjstate, ExprContext *econtext)
 				return true;
 			}
 
-			hashTuple = hashTuple->next;
+			hashTuple = hashTuple->next.unshared;
 		}
 
 		/* allow this loop to be cancellable */
@@ -1226,7 +2132,7 @@ ExecHashTableReset(HashJoinTable hashtable)
 	oldcxt = MemoryContextSwitchTo(hashtable->batchCxt);
 
 	/* Reallocate and reinitialize the hash bucket headers. */
-	hashtable->buckets = (HashJoinTuple *)
+	hashtable->buckets.unshared = (HashJoinTuple *)
 		palloc0(nbuckets * sizeof(HashJoinTuple));
 
 	hashtable->spaceUsed = 0;
@@ -1250,7 +2156,8 @@ ExecHashTableResetMatchFlags(HashJoinTable hashtable)
 	/* Reset all flags in the main table ... */
 	for (i = 0; i < hashtable->nbuckets; i++)
 	{
-		for (tuple = hashtable->buckets[i]; tuple != NULL; tuple = tuple->next)
+		for (tuple = hashtable->buckets.unshared[i]; tuple != NULL;
+			 tuple = tuple->next.unshared)
 			HeapTupleHeaderClearMatch(HJTUPLE_MINTUPLE(tuple));
 	}
 
@@ -1260,7 +2167,7 @@ ExecHashTableResetMatchFlags(HashJoinTable hashtable)
 		int			j = hashtable->skewBucketNums[i];
 		HashSkewBucket *skewBucket = hashtable->skewBucket[j];
 
-		for (tuple = skewBucket->tuples; tuple != NULL; tuple = tuple->next)
+		for (tuple = skewBucket->tuples; tuple != NULL; tuple = tuple->next.unshared)
 			HeapTupleHeaderClearMatch(HJTUPLE_MINTUPLE(tuple));
 	}
 }
@@ -1505,8 +2412,9 @@ ExecHashSkewTableInsert(HashJoinTable hashtable,
 	HeapTupleHeaderClearMatch(HJTUPLE_MINTUPLE(hashTuple));
 
 	/* Push it onto the front of the skew bucket's list */
-	hashTuple->next = hashtable->skewBucket[bucketNumber]->tuples;
+	hashTuple->next.unshared = hashtable->skewBucket[bucketNumber]->tuples;
 	hashtable->skewBucket[bucketNumber]->tuples = hashTuple;
+	Assert(hashTuple != hashTuple->next.unshared);
 
 	/* Account for space used, and back off if we've used too much */
 	hashtable->spaceUsed += hashTupleSize;
@@ -1554,7 +2462,7 @@ ExecHashRemoveNextSkewBucket(HashJoinTable hashtable)
 	hashTuple = bucket->tuples;
 	while (hashTuple != NULL)
 	{
-		HashJoinTuple nextHashTuple = hashTuple->next;
+		HashJoinTuple nextHashTuple = hashTuple->next.unshared;
 		MinimalTuple tuple;
 		Size		tupleSize;
 
@@ -1580,8 +2488,8 @@ ExecHashRemoveNextSkewBucket(HashJoinTable hashtable)
 			memcpy(copyTuple, hashTuple, tupleSize);
 			pfree(hashTuple);
 
-			copyTuple->next = hashtable->buckets[bucketno];
-			hashtable->buckets[bucketno] = copyTuple;
+			copyTuple->next.unshared = hashtable->buckets.unshared[bucketno];
+			hashtable->buckets.unshared[bucketno] = copyTuple;
 
 			/* We have reduced skew space, but overall space doesn't change */
 			hashtable->spaceUsedSkew -= tupleSize;
@@ -1760,11 +2668,11 @@ dense_alloc(HashJoinTable hashtable, Size size)
 		if (hashtable->chunks != NULL)
 		{
 			newChunk->next = hashtable->chunks->next;
-			hashtable->chunks->next = newChunk;
+			hashtable->chunks->next.unshared = newChunk;
 		}
 		else
 		{
-			newChunk->next = hashtable->chunks;
+			newChunk->next.unshared = hashtable->chunks;
 			hashtable->chunks = newChunk;
 		}
 
@@ -1789,7 +2697,7 @@ dense_alloc(HashJoinTable hashtable, Size size)
 		newChunk->used = size;
 		newChunk->ntuples = 1;
 
-		newChunk->next = hashtable->chunks;
+		newChunk->next.unshared = hashtable->chunks;
 		hashtable->chunks = newChunk;
 
 		return newChunk->data;
@@ -1803,3 +2711,601 @@ dense_alloc(HashJoinTable hashtable, Size size)
 	/* return pointer to the start of the tuple memory */
 	return ptr;
 }
+
+/*
+ * Allocate space for a tuple in shared dense storage.  This is equivalent to
+ * dense_alloc but for Parallel Hash using shared memory.
+ *
+ * While loading a tuple into shared memory, we might run out of memory and
+ * decide to repartition, or determine that the load factor is too high and
+ * decide to expand the bucket array, or discover that another participant has
+ * commanded us to help do that.  Return NULL if number of buckets or batches
+ * has changed, indicating that the caller must retry (considering the
+ * possibility that the tuple no longer belongs in the same batch).
+ */
+static HashJoinTuple
+ExecParallelHashTupleAlloc(HashJoinTable hashtable, size_t size,
+						   dsa_pointer *shared)
+{
+	ParallelHashJoinState *pstate = hashtable->parallel_state;
+	dsa_pointer chunk_shared;
+	HashMemoryChunk chunk;
+	Size		chunk_size;
+	HashJoinTuple result;
+	int			curbatch = hashtable->curbatch;
+
+	size = MAXALIGN(size);
+
+	/*
+	 * Fast path: if there is enough space in this backend's current chunk,
+	 * then we can allocate without any locking.
+	 */
+	chunk = hashtable->current_chunk;
+	if (chunk != NULL &&
+		size < HASH_CHUNK_THRESHOLD &&
+		chunk->maxlen - chunk->used >= size)
+	{
+
+		chunk_shared = hashtable->current_chunk_shared;
+		Assert(chunk == dsa_get_address(hashtable->area, chunk_shared));
+		*shared = chunk_shared + HASH_CHUNK_HEADER_SIZE + chunk->used;
+		result = (HashJoinTuple) (chunk->data + chunk->used);
+		chunk->used += size;
+
+		Assert(chunk->used <= chunk->maxlen);
+		Assert(result == dsa_get_address(hashtable->area, *shared));
+
+		return result;
+	}
+
+	/* Slow path: try to allocate a new chunk. */
+	LWLockAcquire(&pstate->lock, LW_EXCLUSIVE);
+
+	/*
+	 * Check if we need to help increase the number of buckets or batches.
+	 */
+	if (pstate->growth == PHJ_GROWTH_NEED_MORE_BATCHES ||
+		pstate->growth == PHJ_GROWTH_NEED_MORE_BUCKETS)
+	{
+		ParallelHashGrowth growth = pstate->growth;
+
+		hashtable->current_chunk = NULL;
+		LWLockRelease(&pstate->lock);
+
+		/* Another participant has commanded us to help grow. */
+		if (growth == PHJ_GROWTH_NEED_MORE_BATCHES)
+			ExecParallelHashIncreaseNumBatches(hashtable);
+		else if (growth == PHJ_GROWTH_NEED_MORE_BUCKETS)
+			ExecParallelHashIncreaseNumBuckets(hashtable);
+
+		/* The caller must retry. */
+		return NULL;
+	}
+
+	/* Oversized tuples get their own chunk. */
+	if (size > HASH_CHUNK_THRESHOLD)
+		chunk_size = size + HASH_CHUNK_HEADER_SIZE;
+	else
+		chunk_size = HASH_CHUNK_SIZE;
+
+	/* Check if it's time to grow batches or buckets. */
+	if (pstate->growth != PHJ_GROWTH_DISABLED)
+	{
+		Assert(curbatch == 0);
+		Assert(BarrierPhase(&pstate->build_barrier) == PHJ_BUILD_HASHING_INNER);
+
+		/*
+		 * Check if our space limit would be exceeded.  To avoid choking on
+		 * very large tuples or very low work_mem setting, we'll always allow
+		 * each backend to allocate at least one chunk.
+		 */
+		if (hashtable->batches[0].at_least_one_chunk &&
+			hashtable->batches[0].shared->size +
+			chunk_size > pstate->space_allowed)
+		{
+			pstate->growth = PHJ_GROWTH_NEED_MORE_BATCHES;
+			hashtable->batches[0].shared->space_exhausted = true;
+			LWLockRelease(&pstate->lock);
+
+			return NULL;
+		}
+
+		/* Check if our load factor limit would be exceeded. */
+		if (hashtable->nbatch == 1)
+		{
+			hashtable->batches[0].shared->ntuples += hashtable->batches[0].ntuples;
+			hashtable->batches[0].ntuples = 0;
+			if (hashtable->batches[0].shared->ntuples + 1 >
+				hashtable->nbuckets * NTUP_PER_BUCKET &&
+				hashtable->nbuckets < (INT_MAX / 2))
+			{
+				pstate->growth = PHJ_GROWTH_NEED_MORE_BUCKETS;
+				LWLockRelease(&pstate->lock);
+
+				return NULL;
+			}
+		}
+	}
+
+	/* We are cleared to allocate a new chunk. */
+	chunk_shared = dsa_allocate(hashtable->area, chunk_size);
+	hashtable->batches[curbatch].shared->size += chunk_size;
+	hashtable->batches[curbatch].at_least_one_chunk = true;
+
+	/* Set up the chunk. */
+	chunk = (HashMemoryChunk) dsa_get_address(hashtable->area, chunk_shared);
+	*shared = chunk_shared + HASH_CHUNK_HEADER_SIZE;
+	chunk->maxlen = chunk_size - HASH_CHUNK_HEADER_SIZE;
+	chunk->used = size;
+
+	/*
+	 * Push it onto the list of chunks, so that it can be found if we need to
+	 * increase the number of buckets or batches (batch 0 only) and later for
+	 * freeing the memory (all batches).
+	 */
+	chunk->next.shared = hashtable->batches[curbatch].shared->chunks;
+	hashtable->batches[curbatch].shared->chunks = chunk_shared;
+
+	if (size <= HASH_CHUNK_THRESHOLD)
+	{
+		/*
+		 * Make this the current chunk so that we can use the fast path to
+		 * fill the rest of it up in future calls.
+		 */
+		hashtable->current_chunk = chunk;
+		hashtable->current_chunk_shared = chunk_shared;
+	}
+	LWLockRelease(&pstate->lock);
+
+	Assert(chunk->data == dsa_get_address(hashtable->area, *shared));
+	result = (HashJoinTuple) chunk->data;
+
+	return result;
+}
+
+/*
+ * One backend needs to set up the shared batch state including tuplestores.
+ * Other backends will ensure they have correctly configured accessors by
+ * called ExecParallelHashEnsureBatchAccessors().
+ */
+static void
+ExecParallelHashJoinSetUpBatches(HashJoinTable hashtable, int nbatch)
+{
+	ParallelHashJoinState *pstate = hashtable->parallel_state;
+	ParallelHashJoinBatch *batches;
+	MemoryContext oldcxt;
+	int			i;
+
+	Assert(hashtable->batches == NULL);
+
+	/* Allocate space. */
+	pstate->batches =
+		dsa_allocate0(hashtable->area,
+					  EstimateParallelHashJoinBatch(hashtable) * nbatch);
+	pstate->nbatch = nbatch;
+	batches = dsa_get_address(hashtable->area, pstate->batches);
+
+	/* Use hash join memory context. */
+	oldcxt = MemoryContextSwitchTo(hashtable->hashCxt);
+
+	/* Allocate this backend's accessor array. */
+	hashtable->nbatch = nbatch;
+	hashtable->batches = (ParallelHashJoinBatchAccessor *)
+		palloc0(sizeof(ParallelHashJoinBatchAccessor) * hashtable->nbatch);
+
+	/* Set up the shared state, tuplestores and backend-local accessors. */
+	for (i = 0; i < hashtable->nbatch; ++i)
+	{
+		ParallelHashJoinBatchAccessor *accessor = &hashtable->batches[i];
+		ParallelHashJoinBatch *shared = NthParallelHashJoinBatch(batches, i);
+		char		name[MAXPGPATH];
+
+		/*
+		 * All members of shared were zero-initialized.  We just need to set
+		 * up the Barrier.
+		 */
+		BarrierInit(&shared->batch_barrier, 0);
+		if (i == 0)
+		{
+			/* Batch 0 doesn't need to be loaded. */
+			BarrierAttach(&shared->batch_barrier);
+			while (BarrierPhase(&shared->batch_barrier) < PHJ_BATCH_PROBING)
+				BarrierArriveAndWait(&shared->batch_barrier, 0);
+			BarrierDetach(&shared->batch_barrier);
+		}
+
+		/* Initialize accessor state.  All members were zero-initialized. */
+		accessor->shared = shared;
+
+		/* Initialize the shared tuplestores. */
+		snprintf(name, sizeof(name), "i%dof%d", i, hashtable->nbatch);
+		accessor->inner_tuples =
+			sts_initialize(ParallelHashJoinBatchInner(shared),
+						   pstate->nparticipants,
+						   ParallelWorkerNumber + 1,
+						   sizeof(uint32),
+						   SHARED_TUPLESTORE_SINGLE_PASS,
+						   &pstate->fileset,
+						   name);
+		snprintf(name, sizeof(name), "o%dof%d", i, hashtable->nbatch);
+		accessor->outer_tuples =
+			sts_initialize(ParallelHashJoinBatchOuter(shared,
+													  pstate->nparticipants),
+						   pstate->nparticipants,
+						   ParallelWorkerNumber + 1,
+						   sizeof(uint32),
+						   SHARED_TUPLESTORE_SINGLE_PASS,
+						   &pstate->fileset,
+						   name);
+	}
+
+	MemoryContextSwitchTo(oldcxt);
+}
+
+/*
+ * Free the current set of ParallelHashJoinBatchAccessor objects.
+ */
+static void
+ExecParallelHashCloseBatchAccessors(HashJoinTable hashtable)
+{
+	int			i;
+
+	for (i = 0; i < hashtable->nbatch; ++i)
+	{
+		/* Make sure no files are left open. */
+		sts_end_write(hashtable->batches[i].inner_tuples);
+		sts_end_write(hashtable->batches[i].outer_tuples);
+		sts_end_parallel_scan(hashtable->batches[i].inner_tuples);
+		sts_end_parallel_scan(hashtable->batches[i].outer_tuples);
+	}
+	pfree(hashtable->batches);
+	hashtable->batches = NULL;
+}
+
+/*
+ * Make sure this backend has up-to-date accessors for the current set of
+ * batches.
+ */
+static void
+ExecParallelHashEnsureBatchAccessors(HashJoinTable hashtable)
+{
+	ParallelHashJoinState *pstate = hashtable->parallel_state;
+	ParallelHashJoinBatch *batches;
+	MemoryContext oldcxt;
+	int			i;
+
+	if (hashtable->batches != NULL)
+	{
+		if (hashtable->nbatch == pstate->nbatch)
+			return;
+		ExecParallelHashCloseBatchAccessors(hashtable);
+	}
+
+	/*
+	 * It's possible for a backend to start up very late so that the whole
+	 * join is finished and the shm state for tracking batches has already
+	 * been freed by ExecHashTableDetach().  In that case we'll just leave
+	 * hashtable->batches as NULL so that ExecParallelHashJoinNewBatch()
+	 * gives up early.
+	 */
+	if (!DsaPointerIsValid(pstate->batches))
+		return;
+
+	/* Use hash join memory context. */
+	oldcxt = MemoryContextSwitchTo(hashtable->hashCxt);
+
+	/* Allocate this backend's accessor array. */
+	hashtable->nbatch = pstate->nbatch;
+	hashtable->batches = (ParallelHashJoinBatchAccessor *)
+		palloc0(sizeof(ParallelHashJoinBatchAccessor) * hashtable->nbatch);
+
+	/* Find the base of the pseudo-array of ParallelHashJoinBatch objects. */
+	batches = (ParallelHashJoinBatch *)
+		dsa_get_address(hashtable->area, pstate->batches);
+
+	/* Set up the accessor array and attach to the tuplestores. */
+	for (i = 0; i < hashtable->nbatch; ++i)
+	{
+		ParallelHashJoinBatchAccessor *accessor = &hashtable->batches[i];
+		ParallelHashJoinBatch *shared = NthParallelHashJoinBatch(batches, i);
+
+		accessor->shared = shared;
+		accessor->preallocated = 0;
+		accessor->done = false;
+		accessor->inner_tuples =
+			sts_attach(ParallelHashJoinBatchInner(shared),
+					   ParallelWorkerNumber + 1,
+					   &pstate->fileset);
+		accessor->outer_tuples =
+			sts_attach(ParallelHashJoinBatchOuter(shared,
+												  pstate->nparticipants),
+					   ParallelWorkerNumber + 1,
+					   &pstate->fileset);
+	}
+
+	MemoryContextSwitchTo(oldcxt);
+}
+
+/*
+ * Allocate an empty shared memory hash table for a given batch.
+ */
+void
+ExecParallelHashTableAlloc(HashJoinTable hashtable, int batchno)
+{
+	ParallelHashJoinBatch *batch = hashtable->batches[batchno].shared;
+	dsa_pointer_atomic *buckets;
+	int			nbuckets = hashtable->parallel_state->nbuckets;
+	int			i;
+
+	batch->buckets =
+		dsa_allocate(hashtable->area, sizeof(dsa_pointer_atomic) * nbuckets);
+	buckets = (dsa_pointer_atomic *)
+		dsa_get_address(hashtable->area, batch->buckets);
+	for (i = 0; i < nbuckets; ++i)
+		dsa_pointer_atomic_init(&buckets[i], InvalidDsaPointer);
+}
+
+/*
+ * If we are currently attached to a shared hash join batch, detach.  If we
+ * are last to detach, clean up.
+ */
+void
+ExecHashTableDetachBatch(HashJoinTable hashtable)
+{
+	if (hashtable->parallel_state != NULL &&
+		hashtable->curbatch >= 0)
+	{
+		int			curbatch = hashtable->curbatch;
+		ParallelHashJoinBatch *batch = hashtable->batches[curbatch].shared;
+
+		/* Make sure any temporary files are closed. */
+		sts_end_parallel_scan(hashtable->batches[curbatch].inner_tuples);
+		sts_end_parallel_scan(hashtable->batches[curbatch].outer_tuples);
+
+		/* Detach from the batch we were last working on. */
+		if (BarrierArriveAndDetach(&batch->batch_barrier))
+		{
+			/*
+			 * Technically we shouldn't access the barrier because we're no
+			 * longer attached, but since there is no way it's moving after
+			 * this point it seems safe to make the following assertion.
+			 */
+			Assert(BarrierPhase(&batch->batch_barrier) == PHJ_BATCH_DONE);
+
+			/* Free shared chunks and buckets. */
+			while (DsaPointerIsValid(batch->chunks))
+			{
+				HashMemoryChunk chunk =
+				dsa_get_address(hashtable->area, batch->chunks);
+				dsa_pointer next = chunk->next.shared;
+
+				dsa_free(hashtable->area, batch->chunks);
+				batch->chunks = next;
+			}
+			if (DsaPointerIsValid(batch->buckets))
+			{
+				dsa_free(hashtable->area, batch->buckets);
+				batch->buckets = InvalidDsaPointer;
+			}
+		}
+		ExecParallelHashUpdateSpacePeak(hashtable, curbatch);
+		/* Remember that we are not attached to a batch. */
+		hashtable->curbatch = -1;
+	}
+}
+
+/*
+ * Detach from all shared resources.  If we are last to detach, clean up.
+ */
+void
+ExecHashTableDetach(HashJoinTable hashtable)
+{
+	if (hashtable->parallel_state)
+	{
+		ParallelHashJoinState *pstate = hashtable->parallel_state;
+		int			i;
+
+		/* Make sure any temporary files are closed. */
+		if (hashtable->batches)
+		{
+			for (i = 0; i < hashtable->nbatch; ++i)
+			{
+				sts_end_write(hashtable->batches[i].inner_tuples);
+				sts_end_write(hashtable->batches[i].outer_tuples);
+				sts_end_parallel_scan(hashtable->batches[i].inner_tuples);
+				sts_end_parallel_scan(hashtable->batches[i].outer_tuples);
+			}
+		}
+
+		/* If we're last to detach, clean up shared memory. */
+		if (BarrierDetach(&pstate->build_barrier))
+		{
+			if (DsaPointerIsValid(pstate->batches))
+			{
+				dsa_free(hashtable->area, pstate->batches);
+				pstate->batches = InvalidDsaPointer;
+			}
+		}
+
+		hashtable->parallel_state = NULL;
+	}
+}
+
+/*
+ * Get the first tuple in a given bucket identified by number.
+ */
+static inline HashJoinTuple
+ExecParallelHashFirstTuple(HashJoinTable hashtable, int bucketno)
+{
+	HashJoinTuple tuple;
+	dsa_pointer	p;
+
+	Assert(hashtable->parallel_state);
+	p = dsa_pointer_atomic_read(&hashtable->buckets.shared[bucketno]);
+	tuple = (HashJoinTuple) dsa_get_address(hashtable->area, p);
+
+	return tuple;
+}
+
+/*
+ * Get the next tuple in the same bucket as 'tuple'.
+ */
+static inline HashJoinTuple
+ExecParallelHashNextTuple(HashJoinTable hashtable, HashJoinTuple tuple)
+{
+	HashJoinTuple next;
+
+	Assert(hashtable->parallel_state);
+	next = (HashJoinTuple) dsa_get_address(hashtable->area, tuple->next.shared);
+
+	return next;
+}
+
+/*
+ * Insert a tuple at the front of a chain of tuples in DSA memory atomically.
+ */
+static inline void
+ExecParallelHashPushTuple(dsa_pointer_atomic *head,
+						  HashJoinTuple tuple,
+						  dsa_pointer tuple_shared)
+{
+	for (;;)
+	{
+		tuple->next.shared = dsa_pointer_atomic_read(head);
+		if (dsa_pointer_atomic_compare_exchange(head,
+												&tuple->next.shared,
+												tuple_shared))
+			break;
+	}
+}
+
+/*
+ * Prepare to work on a given batch.
+ */
+void
+ExecParallelHashTableSetCurrentBatch(HashJoinTable hashtable, int batchno)
+{
+	Assert(hashtable->batches[batchno].shared->buckets != InvalidDsaPointer);
+
+	hashtable->curbatch = batchno;
+	hashtable->buckets.shared = (dsa_pointer_atomic *)
+		dsa_get_address(hashtable->area,
+						hashtable->batches[batchno].shared->buckets);
+	hashtable->nbuckets = hashtable->parallel_state->nbuckets;
+	hashtable->log2_nbuckets = my_log2(hashtable->nbuckets);
+	hashtable->current_chunk = NULL;
+	hashtable->current_chunk_shared = InvalidDsaPointer;
+	hashtable->batches[batchno].at_least_one_chunk = false;
+}
+
+/*
+ * Take the next available chunk from the queue of chunks being worked on in
+ * parallel.  Return NULL if there are none left.  Otherwise return a pointer
+ * to the chunk, and set *shared to the DSA pointer to the chunk.
+ */
+static HashMemoryChunk
+ExecParallelHashPopChunkQueue(HashJoinTable hashtable, dsa_pointer *shared)
+{
+	ParallelHashJoinState *pstate = hashtable->parallel_state;
+	HashMemoryChunk chunk;
+
+	LWLockAcquire(&pstate->lock, LW_EXCLUSIVE);
+	if (DsaPointerIsValid(pstate->chunk_work_queue))
+	{
+		*shared = pstate->chunk_work_queue;
+		chunk = (HashMemoryChunk)
+			dsa_get_address(hashtable->area, *shared);
+		pstate->chunk_work_queue = chunk->next.shared;
+	}
+	else
+		chunk = NULL;
+	LWLockRelease(&pstate->lock);
+
+	return chunk;
+}
+
+/*
+ * Increase the space preallocated in this backend for a given inner batch by
+ * at least a given amount.  This allows us to track whether a given batch
+ * would fit in memory when loaded back in.  Also increase the number of
+ * batches or buckets if required.
+ *
+ * This maintains a running estimation of how much space will be taken when we
+ * load the batch back into memory by simulating the way chunks will be handed
+ * out to workers.  It's not perfectly accurate because the tuples will be
+ * packed into memory chunks differently by ExecParallelHashTupleAlloc(), but
+ * it should be pretty close.  It tends to overestimate by a fraction of a
+ * chunk per worker since all workers gang up to preallocate during hashing,
+ * but workers tend to reload batches alone if there are enough to go around,
+ * leaving fewer partially filled chunks.  This effect is bounded by
+ * nparticipants.
+ *
+ * Return false if the number of batches or buckets has changed, and the
+ * caller should reconsider which batch a given tuple now belongs in and call
+ * again.
+ */
+static bool
+ExecParallelHashTuplePrealloc(HashJoinTable hashtable, int batchno, size_t size)
+{
+	ParallelHashJoinState *pstate = hashtable->parallel_state;
+	ParallelHashJoinBatchAccessor *batch = &hashtable->batches[batchno];
+	size_t		want = Max(size, HASH_CHUNK_SIZE - HASH_CHUNK_HEADER_SIZE);
+
+	Assert(batchno > 0);
+	Assert(batchno < hashtable->nbatch);
+
+	LWLockAcquire(&pstate->lock, LW_EXCLUSIVE);
+
+	/* Has another participant commanded us to help grow? */
+	if (pstate->growth == PHJ_GROWTH_NEED_MORE_BATCHES ||
+		pstate->growth == PHJ_GROWTH_NEED_MORE_BUCKETS)
+	{
+		ParallelHashGrowth growth = pstate->growth;
+
+		LWLockRelease(&pstate->lock);
+		if (growth == PHJ_GROWTH_NEED_MORE_BATCHES)
+			ExecParallelHashIncreaseNumBatches(hashtable);
+		else if (growth == PHJ_GROWTH_NEED_MORE_BUCKETS)
+			ExecParallelHashIncreaseNumBuckets(hashtable);
+
+		return false;
+	}
+
+	if (pstate->growth != PHJ_GROWTH_DISABLED &&
+		batch->at_least_one_chunk &&
+		(batch->shared->estimated_size + size > pstate->space_allowed))
+	{
+		/*
+		 * We have determined that this batch would exceed the space budget if
+		 * loaded into memory.  Command all participants to help repartition.
+		 */
+		batch->shared->space_exhausted = true;
+		pstate->growth = PHJ_GROWTH_NEED_MORE_BATCHES;
+		LWLockRelease(&pstate->lock);
+
+		return false;
+	}
+
+	batch->at_least_one_chunk = true;
+	batch->shared->estimated_size += want + HASH_CHUNK_HEADER_SIZE;
+	batch->preallocated = want;
+	LWLockRelease(&pstate->lock);
+
+	return true;
+}
+
+/*
+ * Update this backend's copy of hashtable->spacePeak to account for a given
+ * batch.  This is called at the end of hashing for batch 0, and then for each
+ * batch when it is done or discovered to be already done.  The result is used
+ * for EXPLAIN output.
+ */
+void
+ExecParallelHashUpdateSpacePeak(HashJoinTable hashtable, int batchno)
+{
+	size_t		size;
+
+	size = hashtable->batches[batchno].shared->size;
+	size += sizeof(dsa_pointer_atomic) * hashtable->nbuckets;
+	hashtable->spacePeak = Max(hashtable->spacePeak, size);
+}
diff --git a/src/backend/executor/nodeHashjoin.c b/src/backend/executor/nodeHashjoin.c
index ab1632cc13d..7eb9174da29 100644
--- a/src/backend/executor/nodeHashjoin.c
+++ b/src/backend/executor/nodeHashjoin.c
@@ -10,18 +10,112 @@
  * IDENTIFICATION
  *	  src/backend/executor/nodeHashjoin.c
  *
+ * PARALLELISM
+ *
+ * Hash joins can participate in parallel query execution in several ways.  A
+ * parallel-oblivious hash join is one where the node is unaware that it is
+ * part of a parallel plan.  In this case, a copy of the inner plan is used to
+ * build a copy of the hash table in every backend, and the outer plan could
+ * either be built from a partial or complete path, so that the results of the
+ * hash join are correspondingly either partial or complete.  A parallel-aware
+ * hash join is one that behaves differently, coordinating work between
+ * backends, and appears as Parallel Hash Join in EXPLAIN output.  A Parallel
+ * Hash Join always appears with a Parallel Hash node.
+ *
+ * Parallel-aware hash joins use the same per-backend state machine to track
+ * progress through the hash join algorithm as parallel-oblivious hash joins.
+ * In a parallel-aware hash join, there is also a shared state machine that
+ * co-operating backends use to synchronize their local state machines and
+ * program counters.  The shared state machine is managed with a Barrier IPC
+ * primitive.  When all attached participants arrive at a barrier, the phase
+ * advances and all waiting participants are released.
+ *
+ * When a participant begins working on a parallel hash join, it must first
+ * figure out how much progress has already been made, because participants
+ * don't wait for each other to begin.  For this reason there are switch
+ * statements at key points in the code where we have to synchronize our local
+ * state machine with the phase, and then jump to the correct part of the
+ * algorithm so that we can get started.
+ *
+ * One barrier called build_barrier is used to coordinate the hashing phases.
+ * The phase is represented by an integer which begins at zero and increments
+ * one by one, but in the code it is referred to by symbolic names as follows:
+ *
+ *   PHJ_BUILD_ELECTING              -- initial state
+ *   PHJ_BUILD_ALLOCATING            -- one sets up the batches and table 0
+ *   PHJ_BUILD_HASHING_INNER         -- all hash the inner rel
+ *   PHJ_BUILD_HASHING_OUTER         -- (multi-batch only) all hash the outer
+ *   PHJ_BUILD_DONE                  -- building done, probing can begin
+ *
+ * While in the phase PHJ_BUILD_HASHING_INNER a separate pair of barriers may
+ * be used repeatedly as required to coordinate expansions in the number of
+ * batches or buckets.  Their phases are as follows:
+ *
+ *   PHJ_GROW_BATCHES_ELECTING       -- initial state
+ *   PHJ_GROW_BATCHES_ALLOCATING     -- one allocates new batches
+ *   PHJ_GROW_BATCHES_REPARTITIONING -- all repartition
+ *   PHJ_GROW_BATCHES_FINISHING      -- one cleans up, detects skew
+ *
+ *   PHJ_GROW_BUCKETS_ELECTING       -- initial state
+ *   PHJ_GROW_BUCKETS_ALLOCATING     -- one allocates new buckets
+ *   PHJ_GROW_BUCKETS_REINSERTING    -- all insert tuples
+ *
+ * If the planner got the number of batches and buckets right, those won't be
+ * necessary, but on the other hand we might finish up needing to expand the
+ * buckets or batches multiple times while hashing the inner relation to stay
+ * within our memory budget and load factor target.  For that reason it's a
+ * separate pair of barriers using circular phases.
+ *
+ * The PHJ_BUILD_HASHING_OUTER phase is required only for multi-batch joins,
+ * because we need to divide the outer relation into batches up front in order
+ * to be able to process batches entirely independently.  In contrast, the
+ * parallel-oblivious algorithm simply throws tuples 'forward' to 'later'
+ * batches whenever it encounters them while scanning and probing, which it
+ * can do because it processes batches in serial order.
+ *
+ * Once PHJ_BUILD_DONE is reached, backends then split up and process
+ * different batches, or gang up and work together on probing batches if there
+ * aren't enough to go around.  For each batch there is a separate barrier
+ * with the following phases:
+ *
+ *  PHJ_BATCH_ELECTING       -- initial state
+ *  PHJ_BATCH_ALLOCATING     -- one allocates buckets
+ *  PHJ_BATCH_LOADING        -- all load the hash table from disk
+ *  PHJ_BATCH_PROBING        -- all probe
+ *  PHJ_BATCH_DONE           -- end
+ *
+ * Batch 0 is a special case, because it starts out in phase
+ * PHJ_BATCH_PROBING; populating batch 0's hash table is done during
+ * PHJ_BUILD_HASHING_INNER so we can skip loading.
+ *
+ * Initially we try to plan for a single-batch hash join using the combined
+ * work_mem of all participants to create a large shared hash table.  If that
+ * turns out either at planning or execution time to be impossible then we
+ * fall back to regular work_mem sized hash tables.
+ *
+ * To avoid deadlocks, we never wait for any barrier unless it is known that
+ * all other backends attached to it are actively executing the node or have
+ * already arrived.  Practically, that means that we never return a tuple
+ * while attached to a barrier, unless the barrier has reached its final
+ * state.  In the slightly special case of the per-batch barrier, we return
+ * tuples while in PHJ_BATCH_PROBING phase, but that's OK because we use
+ * BarrierArriveAndDetach() to advance it to PHJ_BATCH_DONE without waiting.
+ *
  *-------------------------------------------------------------------------
  */
 
 #include "postgres.h"
 
 #include "access/htup_details.h"
+#include "access/parallel.h"
 #include "executor/executor.h"
 #include "executor/hashjoin.h"
 #include "executor/nodeHash.h"
 #include "executor/nodeHashjoin.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "utils/memutils.h"
+#include "utils/sharedtuplestore.h"
 
 
 /*
@@ -42,24 +136,34 @@
 static TupleTableSlot *ExecHashJoinOuterGetTuple(PlanState *outerNode,
 						  HashJoinState *hjstate,
 						  uint32 *hashvalue);
+static TupleTableSlot *ExecParallelHashJoinOuterGetTuple(PlanState *outerNode,
+								  HashJoinState *hjstate,
+								  uint32 *hashvalue);
 static TupleTableSlot *ExecHashJoinGetSavedTuple(HashJoinState *hjstate,
 						  BufFile *file,
 						  uint32 *hashvalue,
 						  TupleTableSlot *tupleSlot);
 static bool ExecHashJoinNewBatch(HashJoinState *hjstate);
+static bool ExecParallelHashJoinNewBatch(HashJoinState *hjstate);
+static void ExecParallelHashJoinPartitionOuter(HashJoinState *node);
 
 
 /* ----------------------------------------------------------------
- *		ExecHashJoin
+ *		ExecHashJoinImpl
  *
- *		This function implements the Hybrid Hashjoin algorithm.
+ *		This function implements the Hybrid Hashjoin algorithm.  It is marked
+ *		with an always-inline attribute so that ExecHashJoin() and
+ *		ExecParallelHashJoin() can inline it.  Compilers that respect the
+ *		attribute should create versions specialized for parallel == true and
+ *		parallel == false with unnecessary branches removed.
  *
  *		Note: the relation we build hash table on is the "inner"
  *			  the other one is "outer".
  * ----------------------------------------------------------------
  */
-static TupleTableSlot *			/* return: a tuple or NULL */
-ExecHashJoin(PlanState *pstate)
+pg_attribute_always_inline
+static TupleTableSlot *
+ExecHashJoinImpl(PlanState *pstate, bool parallel)
 {
 	HashJoinState *node = castNode(HashJoinState, pstate);
 	PlanState  *outerNode;
@@ -71,6 +175,7 @@ ExecHashJoin(PlanState *pstate)
 	TupleTableSlot *outerTupleSlot;
 	uint32		hashvalue;
 	int			batchno;
+	ParallelHashJoinState *parallel_state;
 
 	/*
 	 * get information from HashJoin node
@@ -81,6 +186,7 @@ ExecHashJoin(PlanState *pstate)
 	outerNode = outerPlanState(node);
 	hashtable = node->hj_HashTable;
 	econtext = node->js.ps.ps_ExprContext;
+	parallel_state = hashNode->parallel_state;
 
 	/*
 	 * Reset per-tuple memory context to free any expression evaluation
@@ -138,6 +244,18 @@ ExecHashJoin(PlanState *pstate)
 					/* no chance to not build the hash table */
 					node->hj_FirstOuterTupleSlot = NULL;
 				}
+				else if (parallel)
+				{
+					/*
+					 * The empty-outer optimization is not implemented for
+					 * shared hash tables, because no one participant can
+					 * determine that there are no outer tuples, and it's not
+					 * yet clear that it's worth the synchronization overhead
+					 * of reaching consensus to figure that out.  So we have
+					 * to build the hash table.
+					 */
+					node->hj_FirstOuterTupleSlot = NULL;
+				}
 				else if (HJ_FILL_OUTER(node) ||
 						 (outerNode->plan->startup_cost < hashNode->ps.plan->total_cost &&
 						  !node->hj_OuterNotEmpty))
@@ -155,15 +273,19 @@ ExecHashJoin(PlanState *pstate)
 					node->hj_FirstOuterTupleSlot = NULL;
 
 				/*
-				 * create the hash table
+				 * Create the hash table.  If using Parallel Hash, then
+				 * whoever gets here first will create the hash table and any
+				 * later arrivals will merely attach to it.
 				 */
-				hashtable = ExecHashTableCreate((Hash *) hashNode->ps.plan,
+				hashtable = ExecHashTableCreate(hashNode,
 												node->hj_HashOperators,
 												HJ_FILL_INNER(node));
 				node->hj_HashTable = hashtable;
 
 				/*
-				 * execute the Hash node, to build the hash table
+				 * Execute the Hash node, to build the hash table.  If using
+				 * Parallel Hash, then we'll try to help hashing unless we
+				 * arrived too late.
 				 */
 				hashNode->hashtable = hashtable;
 				(void) MultiExecProcNode((PlanState *) hashNode);
@@ -189,7 +311,34 @@ ExecHashJoin(PlanState *pstate)
 				 */
 				node->hj_OuterNotEmpty = false;
 
-				node->hj_JoinState = HJ_NEED_NEW_OUTER;
+				if (parallel)
+				{
+					Barrier    *build_barrier;
+
+					build_barrier = &parallel_state->build_barrier;
+					Assert(BarrierPhase(build_barrier) == PHJ_BUILD_HASHING_OUTER ||
+						   BarrierPhase(build_barrier) == PHJ_BUILD_DONE);
+					if (BarrierPhase(build_barrier) == PHJ_BUILD_HASHING_OUTER)
+					{
+						/*
+						 * If multi-batch, we need to hash the outer relation
+						 * up front.
+						 */
+						if (hashtable->nbatch > 1)
+							ExecParallelHashJoinPartitionOuter(node);
+						BarrierArriveAndWait(build_barrier,
+											 WAIT_EVENT_HASH_BUILD_HASHING_OUTER);
+					}
+					Assert(BarrierPhase(build_barrier) == PHJ_BUILD_DONE);
+
+					/* Each backend should now select a batch to work on. */
+					hashtable->curbatch = -1;
+					node->hj_JoinState = HJ_NEED_NEW_BATCH;
+
+					continue;
+				}
+				else
+					node->hj_JoinState = HJ_NEED_NEW_OUTER;
 
 				/* FALL THRU */
 
@@ -198,9 +347,14 @@ ExecHashJoin(PlanState *pstate)
 				/*
 				 * We don't have an outer tuple, try to get the next one
 				 */
-				outerTupleSlot = ExecHashJoinOuterGetTuple(outerNode,
-														   node,
-														   &hashvalue);
+				if (parallel)
+					outerTupleSlot =
+						ExecParallelHashJoinOuterGetTuple(outerNode, node,
+														  &hashvalue);
+				else
+					outerTupleSlot =
+						ExecHashJoinOuterGetTuple(outerNode, node, &hashvalue);
+
 				if (TupIsNull(outerTupleSlot))
 				{
 					/* end of batch, or maybe whole join */
@@ -240,10 +394,12 @@ ExecHashJoin(PlanState *pstate)
 					 * Need to postpone this outer tuple to a later batch.
 					 * Save it in the corresponding outer-batch file.
 					 */
+					Assert(parallel_state == NULL);
 					Assert(batchno > hashtable->curbatch);
 					ExecHashJoinSaveTuple(ExecFetchSlotMinimalTuple(outerTupleSlot),
 										  hashvalue,
 										  &hashtable->outerBatchFile[batchno]);
+
 					/* Loop around, staying in HJ_NEED_NEW_OUTER state */
 					continue;
 				}
@@ -258,11 +414,23 @@ ExecHashJoin(PlanState *pstate)
 				/*
 				 * Scan the selected hash bucket for matches to current outer
 				 */
-				if (!ExecScanHashBucket(node, econtext))
+				if (parallel)
 				{
-					/* out of matches; check for possible outer-join fill */
-					node->hj_JoinState = HJ_FILL_OUTER_TUPLE;
-					continue;
+					if (!ExecParallelScanHashBucket(node, econtext))
+					{
+						/* out of matches; check for possible outer-join fill */
+						node->hj_JoinState = HJ_FILL_OUTER_TUPLE;
+						continue;
+					}
+				}
+				else
+				{
+					if (!ExecScanHashBucket(node, econtext))
+					{
+						/* out of matches; check for possible outer-join fill */
+						node->hj_JoinState = HJ_FILL_OUTER_TUPLE;
+						continue;
+					}
 				}
 
 				/*
@@ -362,8 +530,16 @@ ExecHashJoin(PlanState *pstate)
 				/*
 				 * Try to advance to next batch.  Done if there are no more.
 				 */
-				if (!ExecHashJoinNewBatch(node))
-					return NULL;	/* end of join */
+				if (parallel)
+				{
+					if (!ExecParallelHashJoinNewBatch(node))
+						return NULL;	/* end of parallel-aware join */
+				}
+				else
+				{
+					if (!ExecHashJoinNewBatch(node))
+						return NULL;	/* end of parallel-oblivious join */
+				}
 				node->hj_JoinState = HJ_NEED_NEW_OUTER;
 				break;
 
@@ -374,6 +550,38 @@ ExecHashJoin(PlanState *pstate)
 	}
 }
 
+/* ----------------------------------------------------------------
+ *		ExecHashJoin
+ *
+ *		Parallel-oblivious version.
+ * ----------------------------------------------------------------
+ */
+static TupleTableSlot *			/* return: a tuple or NULL */
+ExecHashJoin(PlanState *pstate)
+{
+	/*
+	 * On sufficiently smart compilers this should be inlined with the
+	 * parallel-aware branches removed.
+	 */
+	return ExecHashJoinImpl(pstate, false);
+}
+
+/* ----------------------------------------------------------------
+ *		ExecParallelHashJoin
+ *
+ *		Parallel-aware version.
+ * ----------------------------------------------------------------
+ */
+static TupleTableSlot *			/* return: a tuple or NULL */
+ExecParallelHashJoin(PlanState *pstate)
+{
+	/*
+	 * On sufficiently smart compilers this should be inlined with the
+	 * parallel-oblivious branches removed.
+	 */
+	return ExecHashJoinImpl(pstate, true);
+}
+
 /* ----------------------------------------------------------------
  *		ExecInitHashJoin
  *
@@ -400,6 +608,12 @@ ExecInitHashJoin(HashJoin *node, EState *estate, int eflags)
 	hjstate = makeNode(HashJoinState);
 	hjstate->js.ps.plan = (Plan *) node;
 	hjstate->js.ps.state = estate;
+
+	/*
+	 * See ExecHashJoinInitializeDSM() and ExecHashJoinInitializeWorker() where
+	 * this function may be replaced with a parallel version, if we managed to
+	 * launch a parallel query.
+	 */
 	hjstate->js.ps.ExecProcNode = ExecHashJoin;
 
 	/*
@@ -661,6 +875,64 @@ ExecHashJoinOuterGetTuple(PlanState *outerNode,
 	return NULL;
 }
 
+static TupleTableSlot *
+ExecParallelHashJoinOuterGetTuple(PlanState *outerNode,
+								  HashJoinState *hjstate,
+								  uint32 *hashvalue)
+{
+	HashJoinTable hashtable = hjstate->hj_HashTable;
+	int			curbatch = hashtable->curbatch;
+	TupleTableSlot *slot;
+
+	/*
+	 * In the Parallel Hash case we only run the outer plan directly for
+	 * single-batch hash joins.  Otherwise we have to go to batch files, even
+	 * for batch 0.
+	 */
+	if (curbatch == 0 && hashtable->nbatch == 1)
+	{
+		slot = ExecProcNode(outerNode);
+
+		while (!TupIsNull(slot))
+		{
+			ExprContext *econtext = hjstate->js.ps.ps_ExprContext;
+
+			econtext->ecxt_outertuple = slot;
+			if (ExecHashGetHashValue(hashtable, econtext,
+									 hjstate->hj_OuterHashKeys,
+									 true,	/* outer tuple */
+									 HJ_FILL_OUTER(hjstate),
+									 hashvalue))
+				return slot;
+
+			/*
+			 * That tuple couldn't match because of a NULL, so discard it and
+			 * continue with the next one.
+			 */
+			slot = ExecProcNode(outerNode);
+		}
+	}
+	else if (curbatch < hashtable->nbatch)
+	{
+		MinimalTuple tuple;
+
+		tuple = sts_parallel_scan_next(hashtable->batches[curbatch].outer_tuples,
+									   hashvalue);
+		if (tuple != NULL)
+		{
+			slot = ExecStoreMinimalTuple(tuple,
+										 hjstate->hj_OuterTupleSlot,
+										 false);
+			return slot;
+		}
+		else
+			ExecClearTuple(hjstate->hj_OuterTupleSlot);
+	}
+
+	/* End of this batch */
+	return NULL;
+}
+
 /*
  * ExecHashJoinNewBatch
  *		switch to a new hashjoin batch
@@ -803,6 +1075,135 @@ ExecHashJoinNewBatch(HashJoinState *hjstate)
 	return true;
 }
 
+/*
+ * Choose a batch to work on, and attach to it.  Returns true if successful,
+ * false if there are no more batches.
+ */
+static bool
+ExecParallelHashJoinNewBatch(HashJoinState *hjstate)
+{
+	HashJoinTable hashtable = hjstate->hj_HashTable;
+	int			start_batchno;
+	int			batchno;
+
+	/*
+	 * If we started up so late that the batch tracking array has been freed
+	 * already by ExecHashTableDetach(), then we are finished.  See also
+	 * ExecParallelHashEnsureBatchAccessors().
+	 */
+	if (hashtable->batches == NULL)
+		return false;
+
+	/*
+	 * If we were already attached to a batch, remember not to bother checking
+	 * it again, and detach from it (possibly freeing the hash table if we are
+	 * last to detach).
+	 */
+	if (hashtable->curbatch >= 0)
+	{
+		hashtable->batches[hashtable->curbatch].done = true;
+		ExecHashTableDetachBatch(hashtable);
+	}
+
+	/*
+	 * Search for a batch that isn't done.  We use an atomic counter to start
+	 * our search at a different batch in every participant when there are
+	 * more batches than participants.
+	 */
+	batchno = start_batchno =
+		pg_atomic_fetch_add_u32(&hashtable->parallel_state->distributor, 1) %
+		hashtable->nbatch;
+	do
+	{
+		uint32		hashvalue;
+		MinimalTuple tuple;
+		TupleTableSlot *slot;
+
+		if (!hashtable->batches[batchno].done)
+		{
+			SharedTuplestoreAccessor *inner_tuples;
+			Barrier    *batch_barrier =
+			&hashtable->batches[batchno].shared->batch_barrier;
+
+			switch (BarrierAttach(batch_barrier))
+			{
+				case PHJ_BATCH_ELECTING:
+
+					/* One backend allocates the hash table. */
+					if (BarrierArriveAndWait(batch_barrier,
+											 WAIT_EVENT_HASH_BATCH_ELECTING))
+						ExecParallelHashTableAlloc(hashtable, batchno);
+					/* Fall through. */
+
+				case PHJ_BATCH_ALLOCATING:
+					/* Wait for allocation to complete. */
+					BarrierArriveAndWait(batch_barrier,
+										 WAIT_EVENT_HASH_BATCH_ALLOCATING);
+					/* Fall through. */
+
+				case PHJ_BATCH_LOADING:
+					/* Start (or join in) loading tuples. */
+					ExecParallelHashTableSetCurrentBatch(hashtable, batchno);
+					inner_tuples = hashtable->batches[batchno].inner_tuples;
+					sts_begin_parallel_scan(inner_tuples);
+					while ((tuple = sts_parallel_scan_next(inner_tuples,
+														   &hashvalue)))
+					{
+						slot = ExecStoreMinimalTuple(tuple,
+													 hjstate->hj_HashTupleSlot,
+													 false);
+						ExecParallelHashTableInsertCurrentBatch(hashtable, slot,
+																hashvalue);
+					}
+					sts_end_parallel_scan(inner_tuples);
+					BarrierArriveAndWait(batch_barrier,
+										 WAIT_EVENT_HASH_BATCH_LOADING);
+					/* Fall through. */
+
+				case PHJ_BATCH_PROBING:
+
+					/*
+					 * This batch is ready to probe.  Return control to
+					 * caller. We stay attached to batch_barrier so that the
+					 * hash table stays alive until everyone's finished
+					 * probing it, but no participant is allowed to wait at
+					 * this barrier again (or else a deadlock could occur).
+					 * All attached participants must eventually call
+					 * BarrierArriveAndDetach() so that the final phase
+					 * PHJ_BATCH_DONE can be reached.
+					 */
+					ExecParallelHashTableSetCurrentBatch(hashtable, batchno);
+					sts_begin_parallel_scan(hashtable->batches[batchno].outer_tuples);
+					return true;
+
+				case PHJ_BATCH_DONE:
+
+					/*
+					 * Already done.  Detach and go around again (if any
+					 * remain).
+					 */
+					BarrierDetach(batch_barrier);
+
+					/*
+					 * We didn't work on this batch, but we need to observe
+					 * its size for EXPLAIN.
+					 */
+					ExecParallelHashUpdateSpacePeak(hashtable, batchno);
+					hashtable->batches[batchno].done = true;
+					hashtable->curbatch = -1;
+					break;
+
+				default:
+					elog(ERROR, "unexpected batch phase %d",
+						 BarrierPhase(batch_barrier));
+			}
+		}
+		batchno = (batchno + 1) % hashtable->nbatch;
+	} while (batchno != start_batchno);
+
+	return false;
+}
+
 /*
  * ExecHashJoinSaveTuple
  *		save a tuple to a batch file.
@@ -964,3 +1365,176 @@ ExecReScanHashJoin(HashJoinState *node)
 	if (node->js.ps.lefttree->chgParam == NULL)
 		ExecReScan(node->js.ps.lefttree);
 }
+
+void
+ExecShutdownHashJoin(HashJoinState *node)
+{
+	if (node->hj_HashTable)
+	{
+		/*
+		 * Detach from shared state before DSM memory goes away.  This makes
+		 * sure that we don't have any pointers into DSM memory by the time
+		 * ExecEndHashJoin runs.
+		 */
+		ExecHashTableDetachBatch(node->hj_HashTable);
+		ExecHashTableDetach(node->hj_HashTable);
+	}
+}
+
+static void
+ExecParallelHashJoinPartitionOuter(HashJoinState *hjstate)
+{
+	PlanState  *outerState = outerPlanState(hjstate);
+	ExprContext *econtext = hjstate->js.ps.ps_ExprContext;
+	HashJoinTable hashtable = hjstate->hj_HashTable;
+	TupleTableSlot *slot;
+	uint32		hashvalue;
+	int			i;
+
+	Assert(hjstate->hj_FirstOuterTupleSlot == NULL);
+
+	/* Execute outer plan, writing all tuples to shared tuplestores. */
+	for (;;)
+	{
+		slot = ExecProcNode(outerState);
+		if (TupIsNull(slot))
+			break;
+		econtext->ecxt_outertuple = slot;
+		if (ExecHashGetHashValue(hashtable, econtext,
+								 hjstate->hj_OuterHashKeys,
+								 true,	/* outer tuple */
+								 false, /* outer join, currently unsupported */
+								 &hashvalue))
+		{
+			int			batchno;
+			int			bucketno;
+
+			ExecHashGetBucketAndBatch(hashtable, hashvalue, &bucketno,
+									  &batchno);
+			sts_puttuple(hashtable->batches[batchno].outer_tuples,
+						 &hashvalue, ExecFetchSlotMinimalTuple(slot));
+		}
+		CHECK_FOR_INTERRUPTS();
+	}
+
+	/* Make sure all outer partitions are readable by any backend. */
+	for (i = 0; i < hashtable->nbatch; ++i)
+		sts_end_write(hashtable->batches[i].outer_tuples);
+}
+
+void
+ExecHashJoinEstimate(HashJoinState *state, ParallelContext *pcxt)
+{
+	shm_toc_estimate_chunk(&pcxt->estimator, sizeof(ParallelHashJoinState));
+	shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+void
+ExecHashJoinInitializeDSM(HashJoinState *state, ParallelContext *pcxt)
+{
+	int			plan_node_id = state->js.ps.plan->plan_node_id;
+	HashState  *hashNode;
+	ParallelHashJoinState *pstate;
+
+	/*
+	 * Disable shared hash table mode if we failed to create a real DSM
+	 * segment, because that means that we don't have a DSA area to work with.
+	 */
+	if (pcxt->seg == NULL)
+		return;
+
+	ExecSetExecProcNode(&state->js.ps, ExecParallelHashJoin);
+
+	/*
+	 * Set up the state needed to coordinate access to the shared hash
+	 * table(s), using the plan node ID as the toc key.
+	 */
+	pstate = shm_toc_allocate(pcxt->toc, sizeof(ParallelHashJoinState));
+	shm_toc_insert(pcxt->toc, plan_node_id, pstate);
+
+	/*
+	 * Set up the shared hash join state with no batches initially.
+	 * ExecHashTableCreate() will prepare at least one later and set nbatch
+	 * and space_allowed.
+	 */
+	pstate->nbatch = 0;
+	pstate->space_allowed = 0;
+	pstate->batches = InvalidDsaPointer;
+	pstate->old_batches = InvalidDsaPointer;
+	pstate->nbuckets = 0;
+	pstate->growth = PHJ_GROWTH_OK;
+	pstate->chunk_work_queue = InvalidDsaPointer;
+	pg_atomic_init_u32(&pstate->distributor, 0);
+	pstate->nparticipants = pcxt->nworkers + 1;
+	pstate->total_tuples = 0;
+	LWLockInitialize(&pstate->lock,
+					 LWTRANCHE_PARALLEL_HASH_JOIN);
+	BarrierInit(&pstate->build_barrier, 0);
+	BarrierInit(&pstate->grow_batches_barrier, 0);
+	BarrierInit(&pstate->grow_buckets_barrier, 0);
+
+	/* Set up the space we'll use for shared temporary files. */
+	SharedFileSetInit(&pstate->fileset, pcxt->seg);
+
+	/* Initialize the shared state in the hash node. */
+	hashNode = (HashState *) innerPlanState(state);
+	hashNode->parallel_state = pstate;
+}
+
+/* ----------------------------------------------------------------
+ *		ExecHashJoinReInitializeDSM
+ *
+ *		Reset shared state before beginning a fresh scan.
+ * ----------------------------------------------------------------
+ */
+void
+ExecHashJoinReInitializeDSM(HashJoinState *state, ParallelContext *cxt)
+{
+	int			plan_node_id = state->js.ps.plan->plan_node_id;
+	ParallelHashJoinState *pstate =
+	shm_toc_lookup(cxt->toc, plan_node_id, false);
+
+	/*
+	 * It would be possible to reuse the shared hash table in single-batch
+	 * cases by resetting and then fast-forwarding build_barrier to
+	 * PHJ_BUILD_DONE and batch 0's batch_barrier to PHJ_BATCH_PROBING, but
+	 * currently shared hash tables are already freed by now (by the last
+	 * participant to detach from the batch).  We could consider keeping it
+	 * around for single-batch joins.  We'd also need to adjust
+	 * finalize_plan() so that it doesn't record a dummy dependency for
+	 * Parallel Hash nodes, preventing the rescan optimization.  For now we
+	 * don't try.
+	 */
+
+	/* Detach, freeing any remaining shared memory. */
+	if (state->hj_HashTable != NULL)
+	{
+		ExecHashTableDetachBatch(state->hj_HashTable);
+		ExecHashTableDetach(state->hj_HashTable);
+	}
+
+	/* Clear any shared batch files. */
+	SharedFileSetDeleteAll(&pstate->fileset);
+
+	/* Reset build_barrier to PHJ_BUILD_ELECTING so we can go around again. */
+	BarrierInit(&pstate->build_barrier, 0);
+}
+
+void
+ExecHashJoinInitializeWorker(HashJoinState *state,
+							 ParallelWorkerContext *pwcxt)
+{
+	HashState  *hashNode;
+	int			plan_node_id = state->js.ps.plan->plan_node_id;
+	ParallelHashJoinState *pstate =
+	shm_toc_lookup(pwcxt->toc, plan_node_id, false);
+
+	/* Attach to the space for shared temporary files. */
+	SharedFileSetAttach(&pstate->fileset, pwcxt->seg);
+
+	/* Attach to the shared state in the hash node. */
+	hashNode = (HashState *) innerPlanState(state);
+	hashNode->parallel_state = pstate;
+
+	ExecSetExecProcNode(&state->js.ps, ExecParallelHashJoin);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index b1515dd8e10..84d717102d7 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -1057,6 +1057,7 @@ _copyHash(const Hash *from)
 	COPY_SCALAR_FIELD(skewTable);
 	COPY_SCALAR_FIELD(skewColumn);
 	COPY_SCALAR_FIELD(skewInherit);
+	COPY_SCALAR_FIELD(rows_total);
 
 	return newnode;
 }
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index b59a5219a72..e468d7cc415 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -927,6 +927,7 @@ _outHash(StringInfo str, const Hash *node)
 	WRITE_OID_FIELD(skewTable);
 	WRITE_INT_FIELD(skewColumn);
 	WRITE_BOOL_FIELD(skewInherit);
+	WRITE_FLOAT_FIELD(rows_total, "%.0f");
 }
 
 static void
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 0d17ae89b0c..1133c70a1ca 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -2213,6 +2213,7 @@ _readHash(void)
 	READ_OID_FIELD(skewTable);
 	READ_INT_FIELD(skewColumn);
 	READ_BOOL_FIELD(skewInherit);
+	READ_FLOAT_FIELD(rows_total);
 
 	READ_DONE();
 }
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 877827dcb52..c3daacd3ea6 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -129,6 +129,7 @@ bool		enable_hashjoin = true;
 bool		enable_gathermerge = true;
 bool		enable_partition_wise_join = false;
 bool		enable_parallel_append = true;
+bool		enable_parallel_hash = true;
 
 typedef struct
 {
@@ -3130,16 +3131,19 @@ initial_cost_hashjoin(PlannerInfo *root, JoinCostWorkspace *workspace,
 					  JoinType jointype,
 					  List *hashclauses,
 					  Path *outer_path, Path *inner_path,
-					  JoinPathExtraData *extra)
+					  JoinPathExtraData *extra,
+					  bool parallel_hash)
 {
 	Cost		startup_cost = 0;
 	Cost		run_cost = 0;
 	double		outer_path_rows = outer_path->rows;
 	double		inner_path_rows = inner_path->rows;
+	double		inner_path_rows_total = inner_path_rows;
 	int			num_hashclauses = list_length(hashclauses);
 	int			numbuckets;
 	int			numbatches;
 	int			num_skew_mcvs;
+	size_t		space_allowed;	/* unused */
 
 	/* cost of source data */
 	startup_cost += outer_path->startup_cost;
@@ -3160,6 +3164,15 @@ initial_cost_hashjoin(PlannerInfo *root, JoinCostWorkspace *workspace,
 		* inner_path_rows;
 	run_cost += cpu_operator_cost * num_hashclauses * outer_path_rows;
 
+	/*
+	 * If this is a parallel hash build, then the value we have for
+	 * inner_rows_total currently refers only to the rows returned by each
+	 * participant.  For shared hash table size estimation, we need the total
+	 * number, so we need to undo the division.
+	 */
+	if (parallel_hash)
+		inner_path_rows_total *= get_parallel_divisor(inner_path);
+
 	/*
 	 * Get hash table size that executor would use for inner relation.
 	 *
@@ -3170,9 +3183,12 @@ initial_cost_hashjoin(PlannerInfo *root, JoinCostWorkspace *workspace,
 	 * XXX at some point it might be interesting to try to account for skew
 	 * optimization in the cost estimate, but for now, we don't.
 	 */
-	ExecChooseHashTableSize(inner_path_rows,
+	ExecChooseHashTableSize(inner_path_rows_total,
 							inner_path->pathtarget->width,
 							true,	/* useskew */
+							parallel_hash,	/* try_combined_work_mem */
+							outer_path->parallel_workers,
+							&space_allowed,
 							&numbuckets,
 							&numbatches,
 							&num_skew_mcvs);
@@ -3204,6 +3220,7 @@ initial_cost_hashjoin(PlannerInfo *root, JoinCostWorkspace *workspace,
 	workspace->run_cost = run_cost;
 	workspace->numbuckets = numbuckets;
 	workspace->numbatches = numbatches;
+	workspace->inner_rows_total = inner_path_rows_total;
 }
 
 /*
@@ -3226,6 +3243,7 @@ final_cost_hashjoin(PlannerInfo *root, HashPath *path,
 	Path	   *inner_path = path->jpath.innerjoinpath;
 	double		outer_path_rows = outer_path->rows;
 	double		inner_path_rows = inner_path->rows;
+	double		inner_path_rows_total = workspace->inner_rows_total;
 	List	   *hashclauses = path->path_hashclauses;
 	Cost		startup_cost = workspace->startup_cost;
 	Cost		run_cost = workspace->run_cost;
@@ -3266,6 +3284,9 @@ final_cost_hashjoin(PlannerInfo *root, HashPath *path,
 	/* mark the path with estimated # of batches */
 	path->num_batches = numbatches;
 
+	/* store the total number of tuples (sum of partial row estimates) */
+	path->inner_rows_total = inner_path_rows_total;
+
 	/* and compute the number of "virtual" buckets in the whole join */
 	virtualbuckets = (double) numbuckets * (double) numbatches;
 
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 02a630278f7..e774130ac8d 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -747,7 +747,7 @@ try_hashjoin_path(PlannerInfo *root,
 	 * never have any output pathkeys, per comments in create_hashjoin_path.
 	 */
 	initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
-						  outer_path, inner_path, extra);
+						  outer_path, inner_path, extra, false);
 
 	if (add_path_precheck(joinrel,
 						  workspace.startup_cost, workspace.total_cost,
@@ -761,6 +761,7 @@ try_hashjoin_path(PlannerInfo *root,
 									  extra,
 									  outer_path,
 									  inner_path,
+									  false,	/* parallel_hash */
 									  extra->restrictlist,
 									  required_outer,
 									  hashclauses));
@@ -776,6 +777,10 @@ try_hashjoin_path(PlannerInfo *root,
  * try_partial_hashjoin_path
  *	  Consider a partial hashjoin join path; if it appears useful, push it into
  *	  the joinrel's partial_pathlist via add_partial_path().
+ *	  The outer side is partial.  If parallel_hash is true, then the inner path
+ *	  must be partial and will be run in parallel to create one or more shared
+ *	  hash tables; otherwise the inner path must be complete and a copy of it
+ *	  is run in every process to create separate identical private hash tables.
  */
 static void
 try_partial_hashjoin_path(PlannerInfo *root,
@@ -784,7 +789,8 @@ try_partial_hashjoin_path(PlannerInfo *root,
 						  Path *inner_path,
 						  List *hashclauses,
 						  JoinType jointype,
-						  JoinPathExtraData *extra)
+						  JoinPathExtraData *extra,
+						  bool parallel_hash)
 {
 	JoinCostWorkspace workspace;
 
@@ -808,7 +814,7 @@ try_partial_hashjoin_path(PlannerInfo *root,
 	 * cost.  Bail out right away if it looks terrible.
 	 */
 	initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
-						  outer_path, inner_path, extra);
+						  outer_path, inner_path, extra, true);
 	if (!add_partial_path_precheck(joinrel, workspace.total_cost, NIL))
 		return;
 
@@ -821,6 +827,7 @@ try_partial_hashjoin_path(PlannerInfo *root,
 										  extra,
 										  outer_path,
 										  inner_path,
+										  parallel_hash,
 										  extra->restrictlist,
 										  NULL,
 										  hashclauses));
@@ -1839,6 +1846,10 @@ hash_inner_and_outer(PlannerInfo *root,
 		 * able to properly guarantee uniqueness.  Similarly, we can't handle
 		 * JOIN_FULL and JOIN_RIGHT, because they can produce false null
 		 * extended rows.  Also, the resulting path must not be parameterized.
+		 * We would be able to support JOIN_FULL and JOIN_RIGHT for Parallel
+		 * Hash, since in that case we're back to a single hash table with a
+		 * single set of match bits for each batch, but that will require
+		 * figuring out a deadlock-free way to wait for the probe to finish.
 		 */
 		if (joinrel->consider_parallel &&
 			save_jointype != JOIN_UNIQUE_OUTER &&
@@ -1848,11 +1859,27 @@ hash_inner_and_outer(PlannerInfo *root,
 			bms_is_empty(joinrel->lateral_relids))
 		{
 			Path	   *cheapest_partial_outer;
+			Path	   *cheapest_partial_inner = NULL;
 			Path	   *cheapest_safe_inner = NULL;
 
 			cheapest_partial_outer =
 				(Path *) linitial(outerrel->partial_pathlist);
 
+			/*
+			 * Can we use a partial inner plan too, so that we can build a
+			 * shared hash table in parallel?
+			 */
+			if (innerrel->partial_pathlist != NIL && enable_parallel_hash)
+			{
+				cheapest_partial_inner =
+					(Path *) linitial(innerrel->partial_pathlist);
+				try_partial_hashjoin_path(root, joinrel,
+										  cheapest_partial_outer,
+										  cheapest_partial_inner,
+										  hashclauses, jointype, extra,
+										  true /* parallel_hash */ );
+			}
+
 			/*
 			 * Normally, given that the joinrel is parallel-safe, the cheapest
 			 * total inner path will also be parallel-safe, but if not, we'll
@@ -1870,7 +1897,8 @@ hash_inner_and_outer(PlannerInfo *root,
 				try_partial_hashjoin_path(root, joinrel,
 										  cheapest_partial_outer,
 										  cheapest_safe_inner,
-										  hashclauses, jointype, extra);
+										  hashclauses, jointype, extra,
+										  false /* parallel_hash */ );
 		}
 	}
 }
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index f6c83d0477c..1a0d3a885f9 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -4192,6 +4192,17 @@ create_hashjoin_plan(PlannerInfo *root,
 	copy_plan_costsize(&hash_plan->plan, inner_plan);
 	hash_plan->plan.startup_cost = hash_plan->plan.total_cost;
 
+	/*
+	 * If parallel-aware, the executor will also need an estimate of the total
+	 * number of rows expected from all participants so that it can size the
+	 * shared hash table.
+	 */
+	if (best_path->jpath.path.parallel_aware)
+	{
+		hash_plan->plan.parallel_aware = true;
+		hash_plan->rows_total = best_path->inner_rows_total;
+	}
+
 	join_plan = make_hashjoin(tlist,
 							  joinclauses,
 							  otherclauses,
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 54126fbb6a5..2aee156ad33 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2278,6 +2278,7 @@ create_mergejoin_path(PlannerInfo *root,
  * 'extra' contains various information about the join
  * 'outer_path' is the cheapest outer path
  * 'inner_path' is the cheapest inner path
+ * 'parallel_hash' to select Parallel Hash of inner path (shared hash table)
  * 'restrict_clauses' are the RestrictInfo nodes to apply at the join
  * 'required_outer' is the set of required outer rels
  * 'hashclauses' are the RestrictInfo nodes to use as hash clauses
@@ -2291,6 +2292,7 @@ create_hashjoin_path(PlannerInfo *root,
 					 JoinPathExtraData *extra,
 					 Path *outer_path,
 					 Path *inner_path,
+					 bool parallel_hash,
 					 List *restrict_clauses,
 					 Relids required_outer,
 					 List *hashclauses)
@@ -2308,7 +2310,8 @@ create_hashjoin_path(PlannerInfo *root,
 								  extra->sjinfo,
 								  required_outer,
 								  &restrict_clauses);
-	pathnode->jpath.path.parallel_aware = false;
+	pathnode->jpath.path.parallel_aware =
+		joinrel->consider_parallel && parallel_hash;
 	pathnode->jpath.path.parallel_safe = joinrel->consider_parallel &&
 		outer_path->parallel_safe && inner_path->parallel_safe;
 	/* This is a foolish way to estimate parallel_workers, but for now... */
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 5c256ff8abf..b502c1bb9b3 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3586,6 +3586,51 @@ pgstat_get_wait_ipc(WaitEventIPC w)
 		case WAIT_EVENT_EXECUTE_GATHER:
 			event_name = "ExecuteGather";
 			break;
+		case WAIT_EVENT_HASH_BATCH_ALLOCATING:
+			event_name = "Hash/Batch/Allocating";
+			break;
+		case WAIT_EVENT_HASH_BATCH_ELECTING:
+			event_name = "Hash/Batch/Electing";
+			break;
+		case WAIT_EVENT_HASH_BATCH_LOADING:
+			event_name = "Hash/Batch/Loading";
+			break;
+		case WAIT_EVENT_HASH_BUILD_ALLOCATING:
+			event_name = "Hash/Build/Allocating";
+			break;
+		case WAIT_EVENT_HASH_BUILD_ELECTING:
+			event_name = "Hash/Build/Electing";
+			break;
+		case WAIT_EVENT_HASH_BUILD_HASHING_INNER:
+			event_name = "Hash/Build/HashingInner";
+			break;
+		case WAIT_EVENT_HASH_BUILD_HASHING_OUTER:
+			event_name = "Hash/Build/HashingOuter";
+			break;
+		case WAIT_EVENT_HASH_GROW_BATCHES_ALLOCATING:
+			event_name = "Hash/GrowBatches/Allocating";
+			break;
+		case WAIT_EVENT_HASH_GROW_BATCHES_DECIDING:
+			event_name = "Hash/GrowBatches/Deciding";
+			break;
+		case WAIT_EVENT_HASH_GROW_BATCHES_ELECTING:
+			event_name = "Hash/GrowBatches/Electing";
+			break;
+		case WAIT_EVENT_HASH_GROW_BATCHES_FINISHING:
+			event_name = "Hash/GrowBatches/Finishing";
+			break;
+		case WAIT_EVENT_HASH_GROW_BATCHES_REPARTITIONING:
+			event_name = "Hash/GrowBatches/Repartitioning";
+			break;
+		case WAIT_EVENT_HASH_GROW_BUCKETS_ALLOCATING:
+			event_name = "Hash/GrowBuckets/Allocating";
+			break;
+		case WAIT_EVENT_HASH_GROW_BUCKETS_ELECTING:
+			event_name = "Hash/GrowBuckets/Electing";
+			break;
+		case WAIT_EVENT_HASH_GROW_BUCKETS_REINSERTING:
+			event_name = "Hash/GrowBuckets/Reinserting";
+			break;
 		case WAIT_EVENT_LOGICAL_SYNC_DATA:
 			event_name = "LogicalSyncData";
 			break;
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 0f7a96d85c3..e32901d506a 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -929,7 +929,15 @@ static struct config_bool ConfigureNamesBool[] =
 		true,
 		NULL, NULL, NULL
 	},
-
+	{
+		{"enable_parallel_hash", PGC_USERSET, QUERY_TUNING_METHOD,
+			gettext_noop("Enables the planner's user of parallel hash plans."),
+			NULL
+		},
+		&enable_parallel_hash,
+		true,
+		NULL, NULL, NULL
+	},
 	{
 		{"geqo", PGC_USERSET, QUERY_TUNING_GEQO,
 			gettext_noop("Enables genetic query optimization."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 842cf3601aa..69f40f04b05 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -301,6 +301,7 @@
 #enable_sort = on
 #enable_tidscan = on
 #enable_partition_wise_join = off
+#enable_parallel_hash = on
 
 # - Planner Cost Constants -
 
diff --git a/src/include/executor/hashjoin.h b/src/include/executor/hashjoin.h
index 82acadf85ba..3bc27b266ff 100644
--- a/src/include/executor/hashjoin.h
+++ b/src/include/executor/hashjoin.h
@@ -15,7 +15,10 @@
 #define HASHJOIN_H
 
 #include "nodes/execnodes.h"
+#include "port/atomics.h"
+#include "storage/barrier.h"
 #include "storage/buffile.h"
+#include "storage/lwlock.h"
 
 /* ----------------------------------------------------------------
  *				hash-join hash table structures
@@ -63,7 +66,12 @@
 
 typedef struct HashJoinTupleData
 {
-	struct HashJoinTupleData *next; /* link to next tuple in same bucket */
+	/* link to next tuple in same bucket */
+	union
+	{
+		struct HashJoinTupleData *unshared;
+		dsa_pointer shared;
+	}			next;
 	uint32		hashvalue;		/* tuple's hash code */
 	/* Tuple data, in MinimalTuple format, follows on a MAXALIGN boundary */
 }			HashJoinTupleData;
@@ -112,8 +120,12 @@ typedef struct HashMemoryChunkData
 	size_t		maxlen;			/* size of the buffer holding the tuples */
 	size_t		used;			/* number of buffer bytes already used */
 
-	struct HashMemoryChunkData *next;	/* pointer to the next chunk (linked
-										 * list) */
+	/* pointer to the next chunk (linked list) */
+	union
+	{
+		struct HashMemoryChunkData *unshared;
+		dsa_pointer shared;
+	}			next;
 
 	char		data[FLEXIBLE_ARRAY_MEMBER];	/* buffer allocated at the end */
 }			HashMemoryChunkData;
@@ -121,8 +133,148 @@ typedef struct HashMemoryChunkData
 typedef struct HashMemoryChunkData *HashMemoryChunk;
 
 #define HASH_CHUNK_SIZE			(32 * 1024L)
+#define HASH_CHUNK_HEADER_SIZE (offsetof(HashMemoryChunkData, data))
 #define HASH_CHUNK_THRESHOLD	(HASH_CHUNK_SIZE / 4)
 
+/*
+ * For each batch of a Parallel Hash Join, we have a ParallelHashJoinBatch
+ * object in shared memory to coordinate access to it.  Since they are
+ * followed by variable-sized objects, they are arranged in contiguous memory
+ * but not accessed directly as an array.
+ */
+typedef struct ParallelHashJoinBatch
+{
+	dsa_pointer buckets;		/* array of hash table buckets */
+	Barrier		batch_barrier;	/* synchronization for joining this batch */
+
+	dsa_pointer chunks;			/* chunks of tuples loaded */
+	size_t		size;			/* size of buckets + chunks in memory */
+	size_t		estimated_size; /* size of buckets + chunks while writing */
+	size_t		ntuples;		/* number of tuples loaded */
+	size_t		old_ntuples;	/* number of tuples before repartitioning */
+	bool		space_exhausted;
+
+	/*
+	 * Variable-sized SharedTuplestore objects follow this struct in memory.
+	 * See the accessor macros below.
+	 */
+} ParallelHashJoinBatch;
+
+/* Accessor for inner batch tuplestore following a ParallelHashJoinBatch. */
+#define ParallelHashJoinBatchInner(batch)							\
+	((SharedTuplestore *)											\
+	 ((char *) (batch) + MAXALIGN(sizeof(ParallelHashJoinBatch))))
+
+/* Accessor for outer batch tuplestore following a ParallelHashJoinBatch. */
+#define ParallelHashJoinBatchOuter(batch, nparticipants) \
+	((SharedTuplestore *)												\
+	 ((char *) ParallelHashJoinBatchInner(batch) +						\
+	  MAXALIGN(sts_estimate(nparticipants))))
+
+/* Total size of a ParallelHashJoinBatch and tuplestores. */
+#define EstimateParallelHashJoinBatch(hashtable)						\
+	(MAXALIGN(sizeof(ParallelHashJoinBatch)) +							\
+	 MAXALIGN(sts_estimate((hashtable)->parallel_state->nparticipants)) * 2)
+
+/* Accessor for the nth ParallelHashJoinBatch given the base. */
+#define NthParallelHashJoinBatch(base, n)								\
+	((ParallelHashJoinBatch *)											\
+	 ((char *) (base) +													\
+	  EstimateParallelHashJoinBatch(hashtable) *  (n)))
+
+/*
+ * Each backend requires a small amount of per-batch state to interact with
+ * each ParalellHashJoinBatch.
+ */
+typedef struct ParallelHashJoinBatchAccessor
+{
+	ParallelHashJoinBatch *shared;	/* pointer to shared state */
+
+	/* Per-backend partial counters to reduce contention. */
+	size_t		preallocated;	/* pre-allocated space for this backend */
+	size_t		ntuples;		/* number of tuples */
+	size_t		size;			/* size of partition in memory */
+	size_t		estimated_size; /* size of partition on disk */
+	size_t		old_ntuples;	/* how many tuples before repartioning? */
+	bool		at_least_one_chunk;	/* has this backend allocated a chunk? */
+
+	bool		done;			/* flag to remember that a batch is done */
+	SharedTuplestoreAccessor *inner_tuples;
+	SharedTuplestoreAccessor *outer_tuples;
+} ParallelHashJoinBatchAccessor;
+
+/*
+ * While hashing the inner relation, any participant might determine that it's
+ * time to increase the number of buckets to reduce the load factor or batches
+ * to reduce the memory size.  This is indicated by setting the growth flag to
+ * these values.
+ */
+typedef enum ParallelHashGrowth
+{
+	/* The current dimensions are sufficient. */
+	PHJ_GROWTH_OK,
+	/* The load factor is too high, so we need to add buckets. */
+	PHJ_GROWTH_NEED_MORE_BUCKETS,
+	/* The memory budget would be exhausted, so we need to repartition. */
+	PHJ_GROWTH_NEED_MORE_BATCHES,
+	/* Repartitioning didn't help last time, so don't try to do that again. */
+	PHJ_GROWTH_DISABLED
+} ParallelHashGrowth;
+
+/*
+ * The shared state used to coordinate a Parallel Hash Join.  This is stored
+ * in the DSM segment.
+ */
+typedef struct ParallelHashJoinState
+{
+	dsa_pointer batches;		/* array of ParallelHashJoinBatch */
+	dsa_pointer old_batches;	/* previous generation during repartition */
+	int			nbatch;			/* number of batches now */
+	int			old_nbatch;		/* previous number of batches */
+	int			nbuckets;		/* number of buckets */
+	ParallelHashGrowth growth;	/* control batch/bucket growth */
+	dsa_pointer chunk_work_queue;	/* chunk work queue */
+	int			nparticipants;
+	size_t		space_allowed;
+	size_t		total_tuples;	/* total number of inner tuples */
+	LWLock		lock;			/* lock protecting the above */
+
+	Barrier		build_barrier;	/* synchronization for the build phases */
+	Barrier		grow_batches_barrier;
+	Barrier		grow_buckets_barrier;
+	pg_atomic_uint32 distributor;	/* counter for load balancing */
+
+	SharedFileSet  fileset;		/* space for shared temporary files */
+} ParallelHashJoinState;
+
+/* The phases for building batches, used by build_barrier. */
+#define PHJ_BUILD_ELECTING				0
+#define PHJ_BUILD_ALLOCATING			1
+#define PHJ_BUILD_HASHING_INNER			2
+#define PHJ_BUILD_HASHING_OUTER			3
+#define PHJ_BUILD_DONE					4
+
+/* The phases for probing each batch, used by for batch_barrier. */
+#define PHJ_BATCH_ELECTING				0
+#define PHJ_BATCH_ALLOCATING			1
+#define PHJ_BATCH_LOADING				2
+#define PHJ_BATCH_PROBING				3
+#define PHJ_BATCH_DONE					4
+
+/* The phases of batch growth while hashing, for grow_batches_barrier. */
+#define PHJ_GROW_BATCHES_ELECTING		0
+#define PHJ_GROW_BATCHES_ALLOCATING		1
+#define PHJ_GROW_BATCHES_REPARTITIONING 2
+#define PHJ_GROW_BATCHES_DECIDING		3
+#define PHJ_GROW_BATCHES_FINISHING		4
+#define PHJ_GROW_BATCHES_PHASE(n)		((n) % 5)	/* circular phases */
+
+/* The phases of bucket growth while hashing, for grow_buckets_barrier. */
+#define PHJ_GROW_BUCKETS_ELECTING		0
+#define PHJ_GROW_BUCKETS_ALLOCATING		1
+#define PHJ_GROW_BUCKETS_REINSERTING	2
+#define PHJ_GROW_BUCKETS_PHASE(n)		((n) % 3)	/* circular phases */
+
 typedef struct HashJoinTableData
 {
 	int			nbuckets;		/* # buckets in the in-memory hash table */
@@ -133,8 +285,13 @@ typedef struct HashJoinTableData
 	int			log2_nbuckets_optimal;	/* log2(nbuckets_optimal) */
 
 	/* buckets[i] is head of list of tuples in i'th in-memory bucket */
-	struct HashJoinTupleData **buckets;
-	/* buckets array is per-batch storage, as are all the tuples */
+	union
+	{
+		/* unshared array is per-batch storage, as are all the tuples */
+		struct HashJoinTupleData **unshared;
+		/* shared array is per-query DSA area, as are all the tuples */
+		dsa_pointer_atomic *shared;
+	}			buckets;
 
 	bool		keepNulls;		/* true to store unmatchable NULL tuples */
 
@@ -153,6 +310,7 @@ typedef struct HashJoinTableData
 	bool		growEnabled;	/* flag to shut off nbatch increases */
 
 	double		totalTuples;	/* # tuples obtained from inner plan */
+	double		partialTuples;	/* # tuples obtained from inner plan by me */
 	double		skewTuples;		/* # tuples inserted into skew tuples */
 
 	/*
@@ -185,6 +343,13 @@ typedef struct HashJoinTableData
 
 	/* used for dense allocation of tuples (into linked chunks) */
 	HashMemoryChunk chunks;		/* one list for the whole batch */
+
+	/* Shared and private state for Parallel Hash. */
+	HashMemoryChunk current_chunk;	/* this backend's current chunk */
+	dsa_area   *area;			/* DSA area to allocate memory from */
+	ParallelHashJoinState *parallel_state;
+	ParallelHashJoinBatchAccessor *batches;
+	dsa_pointer current_chunk_shared;
 }			HashJoinTableData;
 
 #endif							/* HASHJOIN_H */
diff --git a/src/include/executor/nodeHash.h b/src/include/executor/nodeHash.h
index 0974f1edc21..db40fcd17a0 100644
--- a/src/include/executor/nodeHash.h
+++ b/src/include/executor/nodeHash.h
@@ -17,17 +17,33 @@
 #include "access/parallel.h"
 #include "nodes/execnodes.h"
 
+struct SharedHashJoinBatch;
+
 extern HashState *ExecInitHash(Hash *node, EState *estate, int eflags);
 extern Node *MultiExecHash(HashState *node);
 extern void ExecEndHash(HashState *node);
 extern void ExecReScanHash(HashState *node);
 
-extern HashJoinTable ExecHashTableCreate(Hash *node, List *hashOperators,
+extern HashJoinTable ExecHashTableCreate(HashState *state, List *hashOperators,
 					bool keepNulls);
+extern void ExecParallelHashTableAlloc(HashJoinTable hashtable,
+							  int batchno);
 extern void ExecHashTableDestroy(HashJoinTable hashtable);
+extern void ExecHashTableDetach(HashJoinTable hashtable);
+extern void ExecHashTableDetachBatch(HashJoinTable hashtable);
+extern void ExecParallelHashTableSetCurrentBatch(HashJoinTable hashtable,
+									 int batchno);
+void		ExecParallelHashUpdateSpacePeak(HashJoinTable hashtable, int batchno);
+
 extern void ExecHashTableInsert(HashJoinTable hashtable,
 					TupleTableSlot *slot,
 					uint32 hashvalue);
+extern void ExecParallelHashTableInsert(HashJoinTable hashtable,
+							TupleTableSlot *slot,
+							uint32 hashvalue);
+extern void ExecParallelHashTableInsertCurrentBatch(HashJoinTable hashtable,
+						  TupleTableSlot *slot,
+						  uint32 hashvalue);
 extern bool ExecHashGetHashValue(HashJoinTable hashtable,
 					 ExprContext *econtext,
 					 List *hashkeys,
@@ -39,12 +55,16 @@ extern void ExecHashGetBucketAndBatch(HashJoinTable hashtable,
 						  int *bucketno,
 						  int *batchno);
 extern bool ExecScanHashBucket(HashJoinState *hjstate, ExprContext *econtext);
+extern bool ExecParallelScanHashBucket(HashJoinState *hjstate, ExprContext *econtext);
 extern void ExecPrepHashTableForUnmatched(HashJoinState *hjstate);
 extern bool ExecScanHashTableForUnmatched(HashJoinState *hjstate,
 							  ExprContext *econtext);
 extern void ExecHashTableReset(HashJoinTable hashtable);
 extern void ExecHashTableResetMatchFlags(HashJoinTable hashtable);
 extern void ExecChooseHashTableSize(double ntuples, int tupwidth, bool useskew,
+						bool try_combined_work_mem,
+						int parallel_workers,
+						size_t *space_allowed,
 						int *numbuckets,
 						int *numbatches,
 						int *num_skew_mcvs);
diff --git a/src/include/executor/nodeHashjoin.h b/src/include/executor/nodeHashjoin.h
index 7469bfbf60c..8469085d7e3 100644
--- a/src/include/executor/nodeHashjoin.h
+++ b/src/include/executor/nodeHashjoin.h
@@ -20,6 +20,12 @@
 extern HashJoinState *ExecInitHashJoin(HashJoin *node, EState *estate, int eflags);
 extern void ExecEndHashJoin(HashJoinState *node);
 extern void ExecReScanHashJoin(HashJoinState *node);
+extern void ExecShutdownHashJoin(HashJoinState *node);
+extern void ExecHashJoinEstimate(HashJoinState *state, ParallelContext *pcxt);
+extern void ExecHashJoinInitializeDSM(HashJoinState *state, ParallelContext *pcxt);
+extern void ExecHashJoinReInitializeDSM(HashJoinState *state, ParallelContext *pcxt);
+extern void ExecHashJoinInitializeWorker(HashJoinState *state,
+							 ParallelWorkerContext *pwcxt);
 
 extern void ExecHashJoinSaveTuple(MinimalTuple tuple, uint32 hashvalue,
 					  BufFile **fileptr);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 1a35c5c9ada..44d8c47d2c2 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -25,6 +25,7 @@
 #include "utils/hsearch.h"
 #include "utils/queryenvironment.h"
 #include "utils/reltrigger.h"
+#include "utils/sharedtuplestore.h"
 #include "utils/sortsupport.h"
 #include "utils/tuplestore.h"
 #include "utils/tuplesort.h"
@@ -43,6 +44,8 @@ struct ExprState;				/* forward references in this file */
 struct ExprContext;
 struct ExprEvalStep;			/* avoid including execExpr.h everywhere */
 
+struct ParallelHashJoinState;
+
 typedef Datum (*ExprStateEvalFunc) (struct ExprState *expression,
 									struct ExprContext *econtext,
 									bool *isNull);
@@ -2026,6 +2029,9 @@ typedef struct HashState
 
 	SharedHashInfo *shared_info;	/* one entry per worker */
 	HashInstrumentation *hinstrument;	/* this worker's entry */
+
+	/* Parallel hash state. */
+	struct ParallelHashJoinState *parallel_state;
 } HashState;
 
 /* ----------------
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 02fb366680f..d763da647b8 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -880,6 +880,7 @@ typedef struct Hash
 	AttrNumber	skewColumn;		/* outer join key's column #, or zero */
 	bool		skewInherit;	/* is outer join rel an inheritance tree? */
 	/* all other info is in the parent HashJoin node */
+	double		rows_total;		/* estimate total rows if parallel_aware */
 } Hash;
 
 /* ----------------
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
index 1108b6a0ea0..3b9d303ce4e 100644
--- a/src/include/nodes/relation.h
+++ b/src/include/nodes/relation.h
@@ -1464,6 +1464,7 @@ typedef struct HashPath
 	JoinPath	jpath;
 	List	   *path_hashclauses;	/* join clauses used for hashing */
 	int			num_batches;	/* number of batches expected */
+	double		inner_rows_total;	/* total inner rows expected */
 } HashPath;
 
 /*
@@ -2315,6 +2316,7 @@ typedef struct JoinCostWorkspace
 	/* private for cost_hashjoin code */
 	int			numbuckets;
 	int			numbatches;
+	double		inner_rows_total;
 } JoinCostWorkspace;
 
 #endif							/* RELATION_H */
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index 5a1fbf97c38..27afc2eaeb3 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -69,6 +69,7 @@ extern bool enable_hashjoin;
 extern bool enable_gathermerge;
 extern bool enable_partition_wise_join;
 extern bool enable_parallel_append;
+extern bool enable_parallel_hash;
 extern int	constraint_exclusion;
 
 extern double clamp_row_est(double nrows);
@@ -153,7 +154,8 @@ extern void initial_cost_hashjoin(PlannerInfo *root,
 					  JoinType jointype,
 					  List *hashclauses,
 					  Path *outer_path, Path *inner_path,
-					  JoinPathExtraData *extra);
+					  JoinPathExtraData *extra,
+					  bool parallel_hash);
 extern void final_cost_hashjoin(PlannerInfo *root, HashPath *path,
 					JoinCostWorkspace *workspace,
 					JoinPathExtraData *extra);
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 99f65b44f22..3ef12b323bb 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -153,6 +153,7 @@ extern HashPath *create_hashjoin_path(PlannerInfo *root,
 					 JoinPathExtraData *extra,
 					 Path *outer_path,
 					 Path *inner_path,
+					 bool parallel_hash,
 					 List *restrict_clauses,
 					 Relids required_outer,
 					 List *hashclauses);
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 089b7c3a108..58f3a19bc61 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -803,6 +803,21 @@ typedef enum
 	WAIT_EVENT_BGWORKER_STARTUP,
 	WAIT_EVENT_BTREE_PAGE,
 	WAIT_EVENT_EXECUTE_GATHER,
+	WAIT_EVENT_HASH_BATCH_ALLOCATING,
+	WAIT_EVENT_HASH_BATCH_ELECTING,
+	WAIT_EVENT_HASH_BATCH_LOADING,
+	WAIT_EVENT_HASH_BUILD_ALLOCATING,
+	WAIT_EVENT_HASH_BUILD_ELECTING,
+	WAIT_EVENT_HASH_BUILD_HASHING_INNER,
+	WAIT_EVENT_HASH_BUILD_HASHING_OUTER,
+	WAIT_EVENT_HASH_GROW_BATCHES_ELECTING,
+	WAIT_EVENT_HASH_GROW_BATCHES_FINISHING,
+	WAIT_EVENT_HASH_GROW_BATCHES_REPARTITIONING,
+	WAIT_EVENT_HASH_GROW_BATCHES_ALLOCATING,
+	WAIT_EVENT_HASH_GROW_BATCHES_DECIDING,
+	WAIT_EVENT_HASH_GROW_BUCKETS_ELECTING,
+	WAIT_EVENT_HASH_GROW_BUCKETS_REINSERTING,
+	WAIT_EVENT_HASH_GROW_BUCKETS_ALLOCATING,
 	WAIT_EVENT_LOGICAL_SYNC_DATA,
 	WAIT_EVENT_LOGICAL_SYNC_STATE_CHANGE,
 	WAIT_EVENT_MQ_INTERNAL,
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index a347ee4d7de..97e4a0bbbd3 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -211,6 +211,7 @@ typedef enum BuiltinTrancheIds
 	LWTRANCHE_BUFFER_MAPPING,
 	LWTRANCHE_LOCK_MANAGER,
 	LWTRANCHE_PREDICATE_LOCK_MANAGER,
+	LWTRANCHE_PARALLEL_HASH_JOIN,
 	LWTRANCHE_PARALLEL_QUERY_DSA,
 	LWTRANCHE_SESSION_DSA,
 	LWTRANCHE_SESSION_RECORD_TABLE,
diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out
index 9d3abf0ed05..a7cfdf1f443 100644
--- a/src/test/regress/expected/join.out
+++ b/src/test/regress/expected/join.out
@@ -5884,6 +5884,9 @@ insert into extremely_skewed
 update pg_class
   set reltuples = 2, relpages = pg_relation_size('extremely_skewed') / 8192
   where relname = 'extremely_skewed';
+-- Make a relation with a couple of enormous tuples.
+create table wide as select generate_series(1, 2) as id, rpad('', 320000, 'x') as t;
+alter table wide set (parallel_workers = 2);
 -- The "optimal" case: the hash table fits in memory; we plan for 1
 -- batch, we stick to that number, and peak memory usage stays within
 -- our work_mem budget
@@ -5924,6 +5927,7 @@ rollback to settings;
 savepoint settings;
 set local max_parallel_workers_per_gather = 2;
 set local work_mem = '4MB';
+set local enable_parallel_hash = off;
 explain (costs off)
   select count(*) from simple r join simple s using (id);
                       QUERY PLAN                       
@@ -5955,6 +5959,43 @@ $$);
  f                    | f
 (1 row)
 
+rollback to settings;
+-- parallel with parallel-aware hash join
+savepoint settings;
+set local max_parallel_workers_per_gather = 2;
+set local work_mem = '4MB';
+set local enable_parallel_hash = on;
+explain (costs off)
+  select count(*) from simple r join simple s using (id);
+                         QUERY PLAN                          
+-------------------------------------------------------------
+ Finalize Aggregate
+   ->  Gather
+         Workers Planned: 2
+         ->  Partial Aggregate
+               ->  Parallel Hash Join
+                     Hash Cond: (r.id = s.id)
+                     ->  Parallel Seq Scan on simple r
+                     ->  Parallel Hash
+                           ->  Parallel Seq Scan on simple s
+(9 rows)
+
+select count(*) from simple r join simple s using (id);
+ count 
+-------
+ 20000
+(1 row)
+
+select original > 1 as initially_multibatch, final > original as increased_batches
+  from hash_join_batches(
+$$
+  select count(*) from simple r join simple s using (id);
+$$);
+ initially_multibatch | increased_batches 
+----------------------+-------------------
+ f                    | f
+(1 row)
+
 rollback to settings;
 -- The "good" case: batches required, but we plan the right number; we
 -- plan for some number of batches, and we stick to that number, and
@@ -5996,6 +6037,7 @@ rollback to settings;
 savepoint settings;
 set local max_parallel_workers_per_gather = 2;
 set local work_mem = '128kB';
+set local enable_parallel_hash = off;
 explain (costs off)
   select count(*) from simple r join simple s using (id);
                       QUERY PLAN                       
@@ -6027,6 +6069,43 @@ $$);
  t                    | f
 (1 row)
 
+rollback to settings;
+-- parallel with parallel-aware hash join
+savepoint settings;
+set local max_parallel_workers_per_gather = 2;
+set local work_mem = '128kB';
+set local enable_parallel_hash = on;
+explain (costs off)
+  select count(*) from simple r join simple s using (id);
+                         QUERY PLAN                          
+-------------------------------------------------------------
+ Finalize Aggregate
+   ->  Gather
+         Workers Planned: 2
+         ->  Partial Aggregate
+               ->  Parallel Hash Join
+                     Hash Cond: (r.id = s.id)
+                     ->  Parallel Seq Scan on simple r
+                     ->  Parallel Hash
+                           ->  Parallel Seq Scan on simple s
+(9 rows)
+
+select count(*) from simple r join simple s using (id);
+ count 
+-------
+ 20000
+(1 row)
+
+select original > 1 as initially_multibatch, final > original as increased_batches
+  from hash_join_batches(
+$$
+  select count(*) from simple r join simple s using (id);
+$$);
+ initially_multibatch | increased_batches 
+----------------------+-------------------
+ t                    | f
+(1 row)
+
 rollback to settings;
 -- The "bad" case: during execution we need to increase number of
 -- batches; in this case we plan for 1 batch, and increase at least a
@@ -6069,6 +6148,7 @@ rollback to settings;
 savepoint settings;
 set local max_parallel_workers_per_gather = 2;
 set local work_mem = '128kB';
+set local enable_parallel_hash = off;
 explain (costs off)
   select count(*) from simple r join bigger_than_it_looks s using (id);
                             QUERY PLAN                            
@@ -6100,6 +6180,43 @@ $$);
  f                    | t
 (1 row)
 
+rollback to settings;
+-- parallel with parallel-aware hash join
+savepoint settings;
+set local max_parallel_workers_per_gather = 1;
+set local work_mem = '192kB';
+set local enable_parallel_hash = on;
+explain (costs off)
+  select count(*) from simple r join bigger_than_it_looks s using (id);
+                                QUERY PLAN                                 
+---------------------------------------------------------------------------
+ Finalize Aggregate
+   ->  Gather
+         Workers Planned: 1
+         ->  Partial Aggregate
+               ->  Parallel Hash Join
+                     Hash Cond: (r.id = s.id)
+                     ->  Parallel Seq Scan on simple r
+                     ->  Parallel Hash
+                           ->  Parallel Seq Scan on bigger_than_it_looks s
+(9 rows)
+
+select count(*) from simple r join bigger_than_it_looks s using (id);
+ count 
+-------
+ 20000
+(1 row)
+
+select original > 1 as initially_multibatch, final > original as increased_batches
+  from hash_join_batches(
+$$
+  select count(*) from simple r join bigger_than_it_looks s using (id);
+$$);
+ initially_multibatch | increased_batches 
+----------------------+-------------------
+ f                    | t
+(1 row)
+
 rollback to settings;
 -- The "ugly" case: increasing the number of batches during execution
 -- doesn't help, so stop trying to fit in work_mem and hope for the
@@ -6142,6 +6259,7 @@ rollback to settings;
 savepoint settings;
 set local max_parallel_workers_per_gather = 2;
 set local work_mem = '128kB';
+set local enable_parallel_hash = off;
 explain (costs off)
   select count(*) from simple r join extremely_skewed s using (id);
                        QUERY PLAN                       
@@ -6171,6 +6289,42 @@ $$);
         1 |     2
 (1 row)
 
+rollback to settings;
+-- parallel with parallel-aware hash join
+savepoint settings;
+set local max_parallel_workers_per_gather = 1;
+set local work_mem = '128kB';
+set local enable_parallel_hash = on;
+explain (costs off)
+  select count(*) from simple r join extremely_skewed s using (id);
+                              QUERY PLAN                               
+-----------------------------------------------------------------------
+ Finalize Aggregate
+   ->  Gather
+         Workers Planned: 1
+         ->  Partial Aggregate
+               ->  Parallel Hash Join
+                     Hash Cond: (r.id = s.id)
+                     ->  Parallel Seq Scan on simple r
+                     ->  Parallel Hash
+                           ->  Parallel Seq Scan on extremely_skewed s
+(9 rows)
+
+select count(*) from simple r join extremely_skewed s using (id);
+ count 
+-------
+ 20000
+(1 row)
+
+select * from hash_join_batches(
+$$
+  select count(*) from simple r join extremely_skewed s using (id);
+$$);
+ original | final 
+----------+-------
+        1 |     4
+(1 row)
+
 rollback to settings;
 -- A couple of other hash join tests unrelated to work_mem management.
 -- Check that EXPLAIN ANALYZE has data even if the leader doesn't participate
@@ -6192,10 +6346,11 @@ rollback to settings;
 -- that we can check that instrumentation comes back correctly.
 create table foo as select generate_series(1, 3) as id, 'xxxxx'::text as t;
 alter table foo set (parallel_workers = 0);
-create table bar as select generate_series(1, 5000) as id, 'xxxxx'::text as t;
+create table bar as select generate_series(1, 10000) as id, 'xxxxx'::text as t;
 alter table bar set (parallel_workers = 2);
 -- multi-batch with rescan, parallel-oblivious
 savepoint settings;
+set enable_parallel_hash = off;
 set parallel_leader_participation = off;
 set min_parallel_table_scan_size = 0;
 set parallel_setup_cost = 0;
@@ -6246,6 +6401,7 @@ $$);
 rollback to settings;
 -- single-batch with rescan, parallel-oblivious
 savepoint settings;
+set enable_parallel_hash = off;
 set parallel_leader_participation = off;
 set min_parallel_table_scan_size = 0;
 set parallel_setup_cost = 0;
@@ -6293,6 +6449,108 @@ $$);
  f
 (1 row)
 
+rollback to settings;
+-- multi-batch with rescan, parallel-aware
+savepoint settings;
+set enable_parallel_hash = on;
+set parallel_leader_participation = off;
+set min_parallel_table_scan_size = 0;
+set parallel_setup_cost = 0;
+set parallel_tuple_cost = 0;
+set max_parallel_workers_per_gather = 2;
+set enable_material = off;
+set enable_mergejoin = off;
+set work_mem = '64kB';
+explain (costs off)
+  select count(*) from foo
+    left join (select b1.id, b1.t from bar b1 join bar b2 using (id)) ss
+    on foo.id < ss.id + 1 and foo.id > ss.id - 1;
+                                QUERY PLAN                                
+--------------------------------------------------------------------------
+ Aggregate
+   ->  Nested Loop Left Join
+         Join Filter: ((foo.id < (b1.id + 1)) AND (foo.id > (b1.id - 1)))
+         ->  Seq Scan on foo
+         ->  Gather
+               Workers Planned: 2
+               ->  Parallel Hash Join
+                     Hash Cond: (b1.id = b2.id)
+                     ->  Parallel Seq Scan on bar b1
+                     ->  Parallel Hash
+                           ->  Parallel Seq Scan on bar b2
+(11 rows)
+
+select count(*) from foo
+  left join (select b1.id, b1.t from bar b1 join bar b2 using (id)) ss
+  on foo.id < ss.id + 1 and foo.id > ss.id - 1;
+ count 
+-------
+     3
+(1 row)
+
+select final > 1 as multibatch
+  from hash_join_batches(
+$$
+  select count(*) from foo
+    left join (select b1.id, b1.t from bar b1 join bar b2 using (id)) ss
+    on foo.id < ss.id + 1 and foo.id > ss.id - 1;
+$$);
+ multibatch 
+------------
+ t
+(1 row)
+
+rollback to settings;
+-- single-batch with rescan, parallel-aware
+savepoint settings;
+set enable_parallel_hash = on;
+set parallel_leader_participation = off;
+set min_parallel_table_scan_size = 0;
+set parallel_setup_cost = 0;
+set parallel_tuple_cost = 0;
+set max_parallel_workers_per_gather = 2;
+set enable_material = off;
+set enable_mergejoin = off;
+set work_mem = '4MB';
+explain (costs off)
+  select count(*) from foo
+    left join (select b1.id, b1.t from bar b1 join bar b2 using (id)) ss
+    on foo.id < ss.id + 1 and foo.id > ss.id - 1;
+                                QUERY PLAN                                
+--------------------------------------------------------------------------
+ Aggregate
+   ->  Nested Loop Left Join
+         Join Filter: ((foo.id < (b1.id + 1)) AND (foo.id > (b1.id - 1)))
+         ->  Seq Scan on foo
+         ->  Gather
+               Workers Planned: 2
+               ->  Parallel Hash Join
+                     Hash Cond: (b1.id = b2.id)
+                     ->  Parallel Seq Scan on bar b1
+                     ->  Parallel Hash
+                           ->  Parallel Seq Scan on bar b2
+(11 rows)
+
+select count(*) from foo
+  left join (select b1.id, b1.t from bar b1 join bar b2 using (id)) ss
+  on foo.id < ss.id + 1 and foo.id > ss.id - 1;
+ count 
+-------
+     3
+(1 row)
+
+select final > 1 as multibatch
+  from hash_join_batches(
+$$
+  select count(*) from foo
+    left join (select b1.id, b1.t from bar b1 join bar b2 using (id)) ss
+    on foo.id < ss.id + 1 and foo.id > ss.id - 1;
+$$);
+ multibatch 
+------------
+ f
+(1 row)
+
 rollback to settings;
 -- A full outer join where every record is matched.
 -- non-parallel
@@ -6383,5 +6641,49 @@ select  count(*) from simple r full outer join simple s on (r.id = 0 - s.id);
  40000
 (1 row)
 
+rollback to settings;
+-- exercise special code paths for huge tuples (note use of non-strict
+-- expression and left join required to get the detoasted tuple into
+-- the hash table)
+-- parallel with parallel-aware hash join (hits ExecParallelHashLoadTuple and
+-- sts_puttuple oversized tuple cases because it's multi-batch)
+savepoint settings;
+set max_parallel_workers_per_gather = 2;
+set enable_parallel_hash = on;
+set work_mem = '128kB';
+explain (costs off)
+  select length(max(s.t))
+  from wide left join (select id, coalesce(t, '') || '' as t from wide) s using (id);
+                           QUERY PLAN                           
+----------------------------------------------------------------
+ Finalize Aggregate
+   ->  Gather
+         Workers Planned: 2
+         ->  Partial Aggregate
+               ->  Parallel Hash Left Join
+                     Hash Cond: (wide.id = wide_1.id)
+                     ->  Parallel Seq Scan on wide
+                     ->  Parallel Hash
+                           ->  Parallel Seq Scan on wide wide_1
+(9 rows)
+
+select length(max(s.t))
+from wide left join (select id, coalesce(t, '') || '' as t from wide) s using (id);
+ length 
+--------
+ 320000
+(1 row)
+
+select final > 1 as multibatch
+  from hash_join_batches(
+$$
+  select length(max(s.t))
+  from wide left join (select id, coalesce(t, '') || '' as t from wide) s using (id);
+$$);
+ multibatch 
+------------
+ t
+(1 row)
+
 rollback to settings;
 rollback;
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
index 2b738aae7c5..c9c8f51e1c5 100644
--- a/src/test/regress/expected/sysviews.out
+++ b/src/test/regress/expected/sysviews.out
@@ -82,11 +82,12 @@ select name, setting from pg_settings where name like 'enable%';
  enable_mergejoin           | on
  enable_nestloop            | on
  enable_parallel_append     | on
+ enable_parallel_hash       | on
  enable_partition_wise_join | off
  enable_seqscan             | on
  enable_sort                | on
  enable_tidscan             | on
-(14 rows)
+(15 rows)
 
 -- Test that the pg_timezone_names and pg_timezone_abbrevs views are
 -- more-or-less working.  We can't test their contents in any great detail
diff --git a/src/test/regress/sql/join.sql b/src/test/regress/sql/join.sql
index 0e933e00d54..a6a452f9609 100644
--- a/src/test/regress/sql/join.sql
+++ b/src/test/regress/sql/join.sql
@@ -2028,6 +2028,10 @@ update pg_class
   set reltuples = 2, relpages = pg_relation_size('extremely_skewed') / 8192
   where relname = 'extremely_skewed';
 
+-- Make a relation with a couple of enormous tuples.
+create table wide as select generate_series(1, 2) as id, rpad('', 320000, 'x') as t;
+alter table wide set (parallel_workers = 2);
+
 -- The "optimal" case: the hash table fits in memory; we plan for 1
 -- batch, we stick to that number, and peak memory usage stays within
 -- our work_mem budget
@@ -2050,6 +2054,22 @@ rollback to settings;
 savepoint settings;
 set local max_parallel_workers_per_gather = 2;
 set local work_mem = '4MB';
+set local enable_parallel_hash = off;
+explain (costs off)
+  select count(*) from simple r join simple s using (id);
+select count(*) from simple r join simple s using (id);
+select original > 1 as initially_multibatch, final > original as increased_batches
+  from hash_join_batches(
+$$
+  select count(*) from simple r join simple s using (id);
+$$);
+rollback to settings;
+
+-- parallel with parallel-aware hash join
+savepoint settings;
+set local max_parallel_workers_per_gather = 2;
+set local work_mem = '4MB';
+set local enable_parallel_hash = on;
 explain (costs off)
   select count(*) from simple r join simple s using (id);
 select count(*) from simple r join simple s using (id);
@@ -2082,6 +2102,22 @@ rollback to settings;
 savepoint settings;
 set local max_parallel_workers_per_gather = 2;
 set local work_mem = '128kB';
+set local enable_parallel_hash = off;
+explain (costs off)
+  select count(*) from simple r join simple s using (id);
+select count(*) from simple r join simple s using (id);
+select original > 1 as initially_multibatch, final > original as increased_batches
+  from hash_join_batches(
+$$
+  select count(*) from simple r join simple s using (id);
+$$);
+rollback to settings;
+
+-- parallel with parallel-aware hash join
+savepoint settings;
+set local max_parallel_workers_per_gather = 2;
+set local work_mem = '128kB';
+set local enable_parallel_hash = on;
 explain (costs off)
   select count(*) from simple r join simple s using (id);
 select count(*) from simple r join simple s using (id);
@@ -2115,6 +2151,22 @@ rollback to settings;
 savepoint settings;
 set local max_parallel_workers_per_gather = 2;
 set local work_mem = '128kB';
+set local enable_parallel_hash = off;
+explain (costs off)
+  select count(*) from simple r join bigger_than_it_looks s using (id);
+select count(*) from simple r join bigger_than_it_looks s using (id);
+select original > 1 as initially_multibatch, final > original as increased_batches
+  from hash_join_batches(
+$$
+  select count(*) from simple r join bigger_than_it_looks s using (id);
+$$);
+rollback to settings;
+
+-- parallel with parallel-aware hash join
+savepoint settings;
+set local max_parallel_workers_per_gather = 1;
+set local work_mem = '192kB';
+set local enable_parallel_hash = on;
 explain (costs off)
   select count(*) from simple r join bigger_than_it_looks s using (id);
 select count(*) from simple r join bigger_than_it_looks s using (id);
@@ -2148,6 +2200,21 @@ rollback to settings;
 savepoint settings;
 set local max_parallel_workers_per_gather = 2;
 set local work_mem = '128kB';
+set local enable_parallel_hash = off;
+explain (costs off)
+  select count(*) from simple r join extremely_skewed s using (id);
+select count(*) from simple r join extremely_skewed s using (id);
+select * from hash_join_batches(
+$$
+  select count(*) from simple r join extremely_skewed s using (id);
+$$);
+rollback to settings;
+
+-- parallel with parallel-aware hash join
+savepoint settings;
+set local max_parallel_workers_per_gather = 1;
+set local work_mem = '128kB';
+set local enable_parallel_hash = on;
 explain (costs off)
   select count(*) from simple r join extremely_skewed s using (id);
 select count(*) from simple r join extremely_skewed s using (id);
@@ -2175,11 +2242,12 @@ rollback to settings;
 
 create table foo as select generate_series(1, 3) as id, 'xxxxx'::text as t;
 alter table foo set (parallel_workers = 0);
-create table bar as select generate_series(1, 5000) as id, 'xxxxx'::text as t;
+create table bar as select generate_series(1, 10000) as id, 'xxxxx'::text as t;
 alter table bar set (parallel_workers = 2);
 
 -- multi-batch with rescan, parallel-oblivious
 savepoint settings;
+set enable_parallel_hash = off;
 set parallel_leader_participation = off;
 set min_parallel_table_scan_size = 0;
 set parallel_setup_cost = 0;
@@ -2206,6 +2274,61 @@ rollback to settings;
 
 -- single-batch with rescan, parallel-oblivious
 savepoint settings;
+set enable_parallel_hash = off;
+set parallel_leader_participation = off;
+set min_parallel_table_scan_size = 0;
+set parallel_setup_cost = 0;
+set parallel_tuple_cost = 0;
+set max_parallel_workers_per_gather = 2;
+set enable_material = off;
+set enable_mergejoin = off;
+set work_mem = '4MB';
+explain (costs off)
+  select count(*) from foo
+    left join (select b1.id, b1.t from bar b1 join bar b2 using (id)) ss
+    on foo.id < ss.id + 1 and foo.id > ss.id - 1;
+select count(*) from foo
+  left join (select b1.id, b1.t from bar b1 join bar b2 using (id)) ss
+  on foo.id < ss.id + 1 and foo.id > ss.id - 1;
+select final > 1 as multibatch
+  from hash_join_batches(
+$$
+  select count(*) from foo
+    left join (select b1.id, b1.t from bar b1 join bar b2 using (id)) ss
+    on foo.id < ss.id + 1 and foo.id > ss.id - 1;
+$$);
+rollback to settings;
+
+-- multi-batch with rescan, parallel-aware
+savepoint settings;
+set enable_parallel_hash = on;
+set parallel_leader_participation = off;
+set min_parallel_table_scan_size = 0;
+set parallel_setup_cost = 0;
+set parallel_tuple_cost = 0;
+set max_parallel_workers_per_gather = 2;
+set enable_material = off;
+set enable_mergejoin = off;
+set work_mem = '64kB';
+explain (costs off)
+  select count(*) from foo
+    left join (select b1.id, b1.t from bar b1 join bar b2 using (id)) ss
+    on foo.id < ss.id + 1 and foo.id > ss.id - 1;
+select count(*) from foo
+  left join (select b1.id, b1.t from bar b1 join bar b2 using (id)) ss
+  on foo.id < ss.id + 1 and foo.id > ss.id - 1;
+select final > 1 as multibatch
+  from hash_join_batches(
+$$
+  select count(*) from foo
+    left join (select b1.id, b1.t from bar b1 join bar b2 using (id)) ss
+    on foo.id < ss.id + 1 and foo.id > ss.id - 1;
+$$);
+rollback to settings;
+
+-- single-batch with rescan, parallel-aware
+savepoint settings;
+set enable_parallel_hash = on;
 set parallel_leader_participation = off;
 set min_parallel_table_scan_size = 0;
 set parallel_setup_cost = 0;
@@ -2266,4 +2389,27 @@ explain (costs off)
 select  count(*) from simple r full outer join simple s on (r.id = 0 - s.id);
 rollback to settings;
 
+-- exercise special code paths for huge tuples (note use of non-strict
+-- expression and left join required to get the detoasted tuple into
+-- the hash table)
+
+-- parallel with parallel-aware hash join (hits ExecParallelHashLoadTuple and
+-- sts_puttuple oversized tuple cases because it's multi-batch)
+savepoint settings;
+set max_parallel_workers_per_gather = 2;
+set enable_parallel_hash = on;
+set work_mem = '128kB';
+explain (costs off)
+  select length(max(s.t))
+  from wide left join (select id, coalesce(t, '') || '' as t from wide) s using (id);
+select length(max(s.t))
+from wide left join (select id, coalesce(t, '') || '' as t from wide) s using (id);
+select final > 1 as multibatch
+  from hash_join_batches(
+$$
+  select length(max(s.t))
+  from wide left join (select id, coalesce(t, '') || '' as t from wide) s using (id);
+$$);
+rollback to settings;
+
 rollback;
-- 
2.15.0

#70Andres Freund
andres@anarazel.de
In reply to: Thomas Munro (#69)
Re: [HACKERS] Parallel Hash take II

Hi,

Not really happy with the name. ExecParallelHashTableInsert() calling
ExecParallelHashLoadTuple() to insert a tuple into the hashtable doesn't
quite strike me as right; the naming similarity to
ExecParallelHashTableLoad seems problematic too.
ExecParallelHashAllocTuple() or such?

One could argue it'd not be a bad idea to keep a similar split as
dense_alloc() and memcpy() have, but I'm not really convinced by
myself. Hm.

Yeah, the names are confusing. So:

Cool.

Just to make sure I understand: The "empty" phase is to protect the
process of the electee doing the sizing calculations etc? And the
reason that's not possible to do just by waiting for
PHJ_GROW_BATCHES_REPARTITIONING is that somebody could dynamically
arrive, i.e. it'd not be needed in a statically sized barrier?

Yeah, it's where you wait for the serial phase above to be finished.
[ Explanation ] Does that make sense?

Yes.

+                       /* reset temp memory each time to avoid leaks from qual expr */
+                       ResetExprContext(econtext);
+
+                       if (ExecQual(hjclauses, econtext))

I personally think it's better to avoid this pattern and store the
result of the ExecQual() in a variable, ResetExprContext() and then act
on the result. No need to keep memory around for longer, and for bigger
contexts you're more likely to have all the metadata in cache.

I'd previously thought about introducing ExecQualAndReset() or such...

Makes sense, but this is code that is identical in
ExecScanHashBucket() so I think we should keep it this way for now,
and explore expression context lifetime improvements in a separate
patch? Looks like the same change could be made in other nodes too.

Ok.

+
+       /*
+        * If we started up so late that the shared batches have been freed
+        * already by ExecHashTableDetach(), then we are finished.
+        */
+       if (!DsaPointerIsValid(hashtable->parallel_state->batches))
+               return false;

This is really the only place that weird condition is detected? And why
is that possible in the first place? And if possible, shouldn't we have
detected earlier? Also, if possible, what prevents this to occur in a
way that test fails, because pstate->batches is freed, but not yet
reset?

ExecParallelHashJoinNewBatch(), where this code appears, is generally
the place that we discover that we're finished. Normally we discover
that we're finished by seeing that there are no batches left to chew
on, by inspecting the per-batch state in shmem. This weird condition
arises when a worker starts up so late that the join is finished and
the shmem space used to tracks batches has already been freed. I
agree that that was badly explained and there was in fact something a
bit kooky about that coding. I have now changed it so that
ExecParallelHashEnsureBatchAccessors() detects this case and has a
better comment to explain it, and ExecParallelHashJoinNewBatch() now
just looks out for hashtable->batches == NULL with a comment referring
to the other place.

Thanks for updating.

My compiler complained that ExecHashJoinImpl() might not be
inlinable. That's just because you declared it always_inline without
actually making it an inline function...

Pushed. Yeha! Congrats, this has been quite the project.

I suspect we'll find a bunch of problems, both on the planning and
execution side, but I think at this point we're much more likely to find
and resolve these in-tree vs. out of tree.

Btw, I see dsa_get_address() show up pretty prominently in profiles. I
kinda wonder if there's some cases where we could ameliorate the cost by
recognizing that a bunch of lookups are all going to reside in the same
segment.

- Andres