Check that streaming replica received all data after master shutdown

Started by Vladimir Borodinabout 11 years ago7 messages
#1Vladimir Borodin
root@simply.name

Hi all.

I have a simple script for planned switchover of PostgreSQL (9.3 and 9.4) master to one of its replicas. This script checks a lot of things before doing it and one of them is that all data from master has been received by replica that is going to be promoted. Right now the check is done like below:

On the master:

postgres@pgtest03d ~ $ psql -t -A -c 'select pg_current_xlog_location();'
0/33000090
postgres@pgtest03d ~ $ /usr/pgsql-9.3/bin/pg_ctl stop -m fast
waiting for server to shut down.... done
server stopped
postgres@pgtest03d ~ $ /usr/pgsql-9.3/bin/pg_controldata | head
pg_control version number: 937
Catalog version number: 201306121
Database system identifier: 6061800518091528182
Database cluster state: shut down
pg_control last modified: Mon 05 Jan 2015 06:47:57 PM MSK
Latest checkpoint location: 0/34000028
Prior checkpoint location: 0/33000028
Latest checkpoint's REDO location: 0/34000028
Latest checkpoint's REDO WAL file: 0000001B0000000000000034
Latest checkpoint's TimeLineID: 27
postgres@pgtest03d ~ $

On the replica (after shutdown of master):

postgres@pgtest03g ~ $ psql -t -A -c "select pg_xlog_location_diff(pg_last_xlog_replay_location(), '0/34000028');"
104
postgres@pgtest03g ~ $

These 104 bytes seems to be the size of shutdown checkpoint record (as I can understand from pg_xlogdump output).

postgres@pgtest03g ~/9.3/data/pg_xlog $ /usr/pgsql-9.3/bin/pg_xlogdump -s 0/33000090 -t 27
rmgr: XLOG len (rec/tot): 0/ 32, tx: 0, lsn: 0/33000090, prev 0/33000028, bkp: 0000, desc: xlog switch
rmgr: XLOG len (rec/tot): 72/ 104, tx: 0, lsn: 0/34000028, prev 0/33000090, bkp: 0000, desc: checkpoint: redo 0/34000028; tli 27; prev tli 27; fpw true; xid 0/6010; oid 54128; multi 1; offset 0; oldest xid 1799 in DB 1; oldest multi 1 in DB 1; oldest running xid 0; shutdown
pg_xlogdump: FATAL: error in WAL record at 0/34000028: record with zero length at 0/34000090

postgres@pgtest03g ~/9.3/data/pg_xlog $

I’m not sure that these 104 bytes will always be 104 bytes to have a strict equality while checking. Could it change in the future? Or is there a better way to understand that streaming replica received all data after master shutdown? The check that pg_xlog_location_diff returns 104 bytes seems a bit strange.

Thanks.

--
May the force be with you...
http://simply.name

#2Vladimir Borodin
root@simply.name
In reply to: Vladimir Borodin (#1)
Re: Check that streaming replica received all data after master shutdown

05 янв. 2015 г., в 18:15, Vladimir Borodin <root@simply.name> написал(а):

Hi all.

I have a simple script for planned switchover of PostgreSQL (9.3 and 9.4) master to one of its replicas. This script checks a lot of things before doing it and one of them is that all data from master has been received by replica that is going to be promoted. Right now the check is done like below:

On the master:

postgres@pgtest03d ~ $ psql -t -A -c 'select pg_current_xlog_location();'
0/33000090
postgres@pgtest03d ~ $ /usr/pgsql-9.3/bin/pg_ctl stop -m fast
waiting for server to shut down.... done
server stopped
postgres@pgtest03d ~ $ /usr/pgsql-9.3/bin/pg_controldata | head
pg_control version number: 937
Catalog version number: 201306121
Database system identifier: 6061800518091528182
Database cluster state: shut down
pg_control last modified: Mon 05 Jan 2015 06:47:57 PM MSK
Latest checkpoint location: 0/34000028
Prior checkpoint location: 0/33000028
Latest checkpoint's REDO location: 0/34000028
Latest checkpoint's REDO WAL file: 0000001B0000000000000034
Latest checkpoint's TimeLineID: 27
postgres@pgtest03d ~ $

On the replica (after shutdown of master):

postgres@pgtest03g ~ $ psql -t -A -c "select pg_xlog_location_diff(pg_last_xlog_replay_location(), '0/34000028');"
104
postgres@pgtest03g ~ $

These 104 bytes seems to be the size of shutdown checkpoint record (as I can understand from pg_xlogdump output).

postgres@pgtest03g ~/9.3/data/pg_xlog $ /usr/pgsql-9.3/bin/pg_xlogdump -s 0/33000090 -t 27
rmgr: XLOG len (rec/tot): 0/ 32, tx: 0, lsn: 0/33000090, prev 0/33000028, bkp: 0000, desc: xlog switch
rmgr: XLOG len (rec/tot): 72/ 104, tx: 0, lsn: 0/34000028, prev 0/33000090, bkp: 0000, desc: checkpoint: redo 0/34000028; tli 27; prev tli 27; fpw true; xid 0/6010; oid 54128; multi 1; offset 0; oldest xid 1799 in DB 1; oldest multi 1 in DB 1; oldest running xid 0; shutdown
pg_xlogdump: FATAL: error in WAL record at 0/34000028: record with zero length at 0/34000090

postgres@pgtest03g ~/9.3/data/pg_xlog $

I’m not sure that these 104 bytes will always be 104 bytes to have a strict equality while checking. Could it change in the future? Or is there a better way to understand that streaming replica received all data after master shutdown? The check that pg_xlog_location_diff returns 104 bytes seems a bit strange.

+hackers

Could anyone help?

Thanks.

Thanks.

--
May the force be with you...
http://simply.name

--
May the force be with you...
http://simply.name

#3Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Vladimir Borodin (#1)
Re: Check that streaming replica received all data after master shutdown

Vladimir Borodin wrote:

I’m not sure that these 104 bytes will always be 104 bytes to have a
strict equality while checking. Could it change in the future?

There is no promise that WAL record format stays unchanged. Sometimes
we change a WAL record in a minor release.

Or is there a better way to understand that streaming replica received
all data after master shutdown? The check that pg_xlog_location_diff
returns 104 bytes seems a bit strange.

I guess you could pg_xlogdump the difference and verify that it is a
shutdown checkpoint record. As far as I remember there should always be
one at the end of recovery.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#4Heikki Linnakangas
hlinnakangas@vmware.com
In reply to: Vladimir Borodin (#2)
Re: Check that streaming replica received all data after master shutdown

On 01/13/2015 12:11 PM, Vladimir Borodin wrote:

05 пїЅпїЅпїЅ. 2015 пїЅ., пїЅ 18:15, Vladimir Borodin <root@simply.name> пїЅпїЅпїЅпїЅпїЅпїЅпїЅ(пїЅ):

Hi all.

I have a simple script for planned switchover of PostgreSQL (9.3 and 9.4) master to one of its replicas. This script checks a lot of things before doing it and one of them is that all data from master has been received by replica that is going to be promoted. Right now the check is done like below:

On the master:

postgres@pgtest03d ~ $ psql -t -A -c 'select pg_current_xlog_location();'
0/33000090
postgres@pgtest03d ~ $ /usr/pgsql-9.3/bin/pg_ctl stop -m fast
waiting for server to shut down.... done
server stopped
postgres@pgtest03d ~ $ /usr/pgsql-9.3/bin/pg_controldata | head
pg_control version number: 937
Catalog version number: 201306121
Database system identifier: 6061800518091528182
Database cluster state: shut down
pg_control last modified: Mon 05 Jan 2015 06:47:57 PM MSK
Latest checkpoint location: 0/34000028
Prior checkpoint location: 0/33000028
Latest checkpoint's REDO location: 0/34000028
Latest checkpoint's REDO WAL file: 0000001B0000000000000034
Latest checkpoint's TimeLineID: 27
postgres@pgtest03d ~ $

On the replica (after shutdown of master):

postgres@pgtest03g ~ $ psql -t -A -c "select pg_xlog_location_diff(pg_last_xlog_replay_location(), '0/34000028');"
104
postgres@pgtest03g ~ $

These 104 bytes seems to be the size of shutdown checkpoint record (as I can understand from pg_xlogdump output).

postgres@pgtest03g ~/9.3/data/pg_xlog $ /usr/pgsql-9.3/bin/pg_xlogdump -s 0/33000090 -t 27
rmgr: XLOG len (rec/tot): 0/ 32, tx: 0, lsn: 0/33000090, prev 0/33000028, bkp: 0000, desc: xlog switch
rmgr: XLOG len (rec/tot): 72/ 104, tx: 0, lsn: 0/34000028, prev 0/33000090, bkp: 0000, desc: checkpoint: redo 0/34000028; tli 27; prev tli 27; fpw true; xid 0/6010; oid 54128; multi 1; offset 0; oldest xid 1799 in DB 1; oldest multi 1 in DB 1; oldest running xid 0; shutdown
pg_xlogdump: FATAL: error in WAL record at 0/34000028: record with zero length at 0/34000090

postgres@pgtest03g ~/9.3/data/pg_xlog $

IпїЅm not sure that these 104 bytes will always be 104 bytes to have a strict equality while checking. Could it change in the future? Or is there a better way to understand that streaming replica received all data after master shutdown? The check that pg_xlog_location_diff returns 104 bytes seems a bit strange.

Don't rely on it being 104 bytes. It can vary across versions, and
across different architectures.

You could simply check that the standby's pg_last_xlog_replay_location()

master's "Latest checkpoint location", and not care about the exact

difference.

- Heikki

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#5Sameer Kumar
sameer.kumar@ashnik.com
In reply to: Heikki Linnakangas (#4)
Re: Check that streaming replica received all data after master shutdown

On Wed, Jan 14, 2015 at 2:11 AM, Heikki Linnakangas <hlinnakangas@vmware.com

wrote:

On 01/13/2015 12:11 PM, Vladimir Borodin wrote:

05 янв. 2015 г., в 18:15, Vladimir Borodin <root@simply.name> написал(а):

Hi all.

I have a simple script for planned switchover of PostgreSQL (9.3 and
9.4) master to one of its replicas. This script checks a lot of things
before doing it and one of them is that all data from master has been
received by replica that is going to be promoted. Right now the check is
done like below:

On the master:

postgres@pgtest03d ~ $ psql -t -A -c 'select
pg_current_xlog_location();'
0/33000090
postgres@pgtest03d ~ $ /usr/pgsql-9.3/bin/pg_ctl stop -m fast
waiting for server to shut down.... done
server stopped
postgres@pgtest03d ~ $ /usr/pgsql-9.3/bin/pg_controldata | head
pg_control version number: 937
Catalog version number: 201306121
Database system identifier: 6061800518091528182
Database cluster state: shut down
pg_control last modified: Mon 05 Jan 2015 06:47:57 PM MSK
Latest checkpoint location: 0/34000028
Prior checkpoint location: 0/33000028
Latest checkpoint's REDO location: 0/34000028
Latest checkpoint's REDO WAL file: 0000001B0000000000000034
Latest checkpoint's TimeLineID: 27
postgres@pgtest03d ~ $

On the replica (after shutdown of master):

postgres@pgtest03g ~ $ psql -t -A -c "select
pg_xlog_location_diff(pg_last_xlog_replay_location(), '0/34000028');"
104
postgres@pgtest03g ~ $

These 104 bytes seems to be the size of shutdown checkpoint record (as I
can understand from pg_xlogdump output).

postgres@pgtest03g ~/9.3/data/pg_xlog $ /usr/pgsql-9.3/bin/pg_xlogdump
-s 0/33000090 -t 27
rmgr: XLOG len (rec/tot): 0/ 32, tx: 0, lsn:
0/33000090, prev 0/33000028, bkp: 0000, desc: xlog switch
rmgr: XLOG len (rec/tot): 72/ 104, tx: 0, lsn:
0/34000028, prev 0/33000090, bkp: 0000, desc: checkpoint: redo 0/34000028;
tli 27; prev tli 27; fpw true; xid 0/6010; oid 54128; multi 1; offset 0;
oldest xid 1799 in DB 1; oldest multi 1 in DB 1; oldest running xid 0;
shutdown
pg_xlogdump: FATAL: error in WAL record at 0/34000028: record with zero
length at 0/34000090

postgres@pgtest03g ~/9.3/data/pg_xlog $

I’m not sure that these 104 bytes will always be 104 bytes to have a
strict equality while checking. Could it change in the future? Or is there
a better way to understand that streaming replica received all data after
master shutdown? The check that pg_xlog_location_diff returns 104 bytes
seems a bit strange.

Don't rely on it being 104 bytes. It can vary across versions, and across
different architectures.

You could simply check that the standby's pg_last_xlog_replay_location() >
master's "Latest checkpoint location", and not care about the exact
difference.

​I believe there were some changes made in v9.3 which will wait for pending
WALs to be replica​ted before a fast and smart shutdown (of master) can
close the replication connection.

http://git.postgresql.org/pg/commitdiff/985bd7d49726c9f178558491d31a570d47340459

#6Kyotaro HORIGUCHI
horiguchi.kyotaro@lab.ntt.co.jp
In reply to: Sameer Kumar (#5)
Re: [HACKERS] Check that streaming replica received all data after master shutdown

Hi,

On Wed, Jan 14, 2015 at 2:11 AM, Heikki Linnakangas <hlinnakangas@vmware.com

wrote:

On 01/13/2015 12:11 PM, Vladimir Borodin wrote:

05 янв. 2015 г., в 18:15, Vladimir Borodin <root@simply.name> написал(а):

Hi all.

I have a simple script for planned switchover of PostgreSQL (9.3 and
9.4) master to one of its replicas. This script checks a lot of things
before doing it and one of them is that all data from master has been
received by replica that is going to be promoted. Right now the check is
done like below:

On the master:

postgres@pgtest03d ~ $ psql -t -A -c 'select
pg_current_xlog_location();'
0/33000090
postgres@pgtest03d ~ $ /usr/pgsql-9.3/bin/pg_ctl stop -m fast
waiting for server to shut down.... done
server stopped
postgres@pgtest03d ~ $ /usr/pgsql-9.3/bin/pg_controldata | head
pg_control version number: 937
Catalog version number: 201306121
Database system identifier: 6061800518091528182
Database cluster state: shut down
pg_control last modified: Mon 05 Jan 2015 06:47:57 PM MSK
Latest checkpoint location: 0/34000028
Prior checkpoint location: 0/33000028
Latest checkpoint's REDO location: 0/34000028
Latest checkpoint's REDO WAL file: 0000001B0000000000000034
Latest checkpoint's TimeLineID: 27
postgres@pgtest03d ~ $

On the replica (after shutdown of master):

postgres@pgtest03g ~ $ psql -t -A -c "select
pg_xlog_location_diff(pg_last_xlog_replay_location(), '0/34000028');"
104
postgres@pgtest03g ~ $

These 104 bytes seems to be the size of shutdown checkpoint record (as I
can understand from pg_xlogdump output).

postgres@pgtest03g ~/9.3/data/pg_xlog $ /usr/pgsql-9.3/bin/pg_xlogdump
-s 0/33000090 -t 27
rmgr: XLOG len (rec/tot): 0/ 32, tx: 0, lsn:
0/33000090, prev 0/33000028, bkp: 0000, desc: xlog switch
rmgr: XLOG len (rec/tot): 72/ 104, tx: 0, lsn:
0/34000028, prev 0/33000090, bkp: 0000, desc: checkpoint: redo 0/34000028;
tli 27; prev tli 27; fpw true; xid 0/6010; oid 54128; multi 1; offset 0;
oldest xid 1799 in DB 1; oldest multi 1 in DB 1; oldest running xid 0;
shutdown
pg_xlogdump: FATAL: error in WAL record at 0/34000028: record with zero
length at 0/34000090

postgres@pgtest03g ~/9.3/data/pg_xlog $

I’m not sure that these 104 bytes will always be 104 bytes to have a
strict equality while checking. Could it change in the future? Or is there
a better way to understand that streaming replica received all data after
master shutdown? The check that pg_xlog_location_diff returns 104 bytes
seems a bit strange.

Don't rely on it being 104 bytes. It can vary across versions, and across
different architectures.

You could simply check that the standby's pg_last_xlog_replay_location() >
master's "Latest checkpoint location", and not care about the exact
difference.

I believe there were some changes made in v9.3 which will wait for pending
WALs to be replica​ted before a fast and smart shutdown (of master) can
close the replication connection.

http://git.postgresql.org/pg/commitdiff/985bd7d49726c9f178558491d31a570d47340459

I don't understand the relation between it and 104 bytes, it says
that the change is backpatched up to 9.1. Since it assures all
xlog records to be transferred if no trouble happens. Relying on
the mechanism, you don't need to check that if master is known to
have gracefully shut down and had no trouble around the
environment. Judging from that you want this check, I suppose
you're not guaranteed not to have trouble or not trusting the
mechanism itself.

Given the condition, as Alvaro said upthread, verifying that the
last record is a shutdown checkpoint should raise a lot the
chance for the all record being received except for the exteme
case such that the master have upped and downed while replication
connection cannot be made. For the case, I think there's no means
to confirm that by standby alone, you should at least compare the
next LSN to the last xlog record with the old master by any
means. Or doing any sanity check of the database on the standby
utilizing the nature of the data instead?

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#7Sameer Kumar
sameer.kumar@ashnik.com
In reply to: Kyotaro HORIGUCHI (#6)
2 attachment(s)
Re: Check that streaming replica received all data after master shutdown

On Thu, Jan 15, 2015 at 6:19 PM, Kyotaro HORIGUCHI <
horiguchi.kyotaro@lab.ntt.co.jp> wrote:

On Wed, Jan 14, 2015 at 2:11 AM, Heikki Linnakangas <

hlinnakangas@vmware.com

wrote:

On 01/13/2015 12:11 PM, Vladimir Borodin wrote:

05 янв. 2015 г., в 18:15, Vladimir Borodin <root@simply.name>

написал(а):

Hi all.

I have a simple script for planned switchover of PostgreSQL (9.3 and
9.4) master to one of its replicas. This script checks a lot of

things

before doing it and one of them is that all data from master has been
received by replica that is going to be promoted. Right now the

check is

done like below:

On the master:

postgres@pgtest03d ~ $ psql -t -A -c 'select
pg_current_xlog_location();'
0/33000090
postgres@pgtest03d ~ $ /usr/pgsql-9.3/bin/pg_ctl stop -m fast
waiting for server to shut down.... done
server stopped
postgres@pgtest03d ~ $ /usr/pgsql-9.3/bin/pg_controldata | head
pg_control version number: 937
Catalog version number: 201306121
Database system identifier: 6061800518091528182
Database cluster state: shut down
pg_control last modified: Mon 05 Jan 2015 06:47:57 PM MSK
Latest checkpoint location: 0/34000028
Prior checkpoint location: 0/33000028
Latest checkpoint's REDO location: 0/34000028
Latest checkpoint's REDO WAL file: 0000001B0000000000000034
Latest checkpoint's TimeLineID: 27
postgres@pgtest03d ~ $

On the replica (after shutdown of master):

postgres@pgtest03g ~ $ psql -t -A -c "select
pg_xlog_location_diff(pg_last_xlog_replay_location(), '0/34000028');"
104
postgres@pgtest03g ~ $

These 104 bytes seems to be the size of shutdown checkpoint record

(as I

can understand from pg_xlogdump output).

postgres@pgtest03g ~/9.3/data/pg_xlog $

/usr/pgsql-9.3/bin/pg_xlogdump

-s 0/33000090 -t 27
rmgr: XLOG len (rec/tot): 0/ 32, tx: 0, lsn:
0/33000090, prev 0/33000028, bkp: 0000, desc: xlog switch
rmgr: XLOG len (rec/tot): 72/ 104, tx: 0, lsn:
0/34000028, prev 0/33000090, bkp: 0000, desc: checkpoint: redo

0/34000028;

tli 27; prev tli 27; fpw true; xid 0/6010; oid 54128; multi 1;

offset 0;

oldest xid 1799 in DB 1; oldest multi 1 in DB 1; oldest running xid

0;

shutdown
pg_xlogdump: FATAL: error in WAL record at 0/34000028: record with

zero

length at 0/34000090

postgres@pgtest03g ~/9.3/data/pg_xlog $

I’m not sure that these 104 bytes will always be 104 bytes to have a
strict equality while checking. Could it change in the future? Or is

there

a better way to understand that streaming replica received all data

after

master shutdown? The check that pg_xlog_location_diff returns 104

bytes

seems a bit strange.

Don't rely on it being 104 bytes. It can vary across versions, and

across

different architectures.

You could simply check that the standby's

pg_last_xlog_replay_location() >

master's "Latest checkpoint location", and not care about the exact
difference.

I believe there were some changes made in v9.3 which will wait for

pending

WALs to be replica​ted before a fast and smart shutdown (of master) can
close the replication connection.

http://git.postgresql.org/pg/commitdiff/985bd7d49726c9f178558491d31a570d47340459

I don't understand the relation between it and 104 bytes, it says
that the change is backpatched up to 9.1. Since it assures all
xlog records to be transferred if no trouble happens. Relying on
the mechanism, you don't need to check that if master is known to
have gracefully shut down and had no trouble around the
environment. Judging from that you want this check, I suppose
you're not guaranteed not to have trouble or not trusting the
mechanism itself.

​Right! I was coming from the point that if master has shutdown gracefully

then you don't really need to worry about ensuring with such checks on
Standby (it is supposed to get the pending WAL before master goes down.

This obviously (as rightly pointed out by you), would not work if master
has not shutdown gracefully or if there is a connection issue between
master and slave while master is being shutdown (even if it is smart or
fast shutdown)​.

Given the condition, as Alvaro said upthread, verifying that the
last record is a shutdown checkpoint should raise a lot the
chance for the all record being received except for the exteme
case such that the master have upped and downed while replication
connection cannot be made.

​I am not sure if this would cover the cases where the master has gone down
abruptly or has crashed (or the service has been killed).​

For the case, I think there's no means
to confirm that by standby alone, you should at least compare the
next LSN to the last xlog record with the old master by any
means.

​That is the method that occurred to ​me as well while reading the first
part of your comments. :)

Or doing any sanity check of the database on the standby
utilizing the nature of the data instead?

Best Regards,

*Sameer Kumar | Database Consultant*

*ASHNIK PTE. LTD.*

101 Cecil Street, #11-11 Tong Eng Building, Singapore 069533

M: *+65 8110 0350* T: +65 6438 3504 | www.ashnik.com

*[image: icons]*

[image: Email patch] <http://www.ashnik.com/&gt;

This email may contain confidential, privileged or copyright material and
is solely for the use of the intended recipient(s).

Attachments:

image006.jpgimage/jpeg; name=image006.jpgDownload
����JFIF``��C
	
		
$ &%# #"(-90(*6+"#2D26;=@@@&0FKE>J9?@=��C
=)#)==================================================��,
"��	
���}!1AQa"q2���#B��R��$3br�	
%&'()*456789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz���������������������������������������������������������������������������	
���w!1AQaq"2�B����	#3R�br�
$4�%�&'()*56789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz��������������������������������������������������������������������������?��;]�s$��s/V��l�1��4���-������A]s��jJ�S�4S+]=��YcF:.k5�C���*��I>��|O1�C�E�W���E��d���l�[o����
�m����9��k.�����E�����!��pYs���X��t� �B9���i�u�uW@����1}��*���q�5��u0���C*���9���x�N?:��G)��^&�5	�|��"1a��x&�|�����U�
��o�*�|Kdmf�B�4Z6n@�s����^�G*�L��?*<����~T�J9P�����+������&�J�����&�o����|����m��Mo��m�L�gs�z=���5�lF&&A���V��%k#9����	���>s��+�4�k���������q[[ ml��i6�g�+/�T���Gm�	���>s��+�4�k���������q[[���=*������!?7�;=q�SX��o�=����]7�|���W�koL���lV�v�u������{@�p[nU����b+��w����������g9�����������<������y��O���Q��&?����L��?*}r����<������y��O��T<����~Ty1��4���Q��g���O�����y��\����j~0��G!���R���Y���p�`���+_k�H�UK�������F�G�R�K��r�7j9P����������
����	\�+~Uw�������C�O������Zl��<qO����2p2G���}ev���Ef�T����*s�
I�|�](�@t��I�`����J���O��t{e�������i7��py=���������=/���O��O�!�/��7���(qBN�<����~Ty1��4���R�"��KE�h���b��&w���=j�^#��K���A�8�,��W�v�[��Z�~������O��&?����b��wV���An��=����l$����E���h�?d����C���������.����E���$o�1��4���c��i�W7u�;�mF+e6�3$'���������S��%����%�/���$�����}2F~���a{X����O��&?����s�������Skl�B��$��f`��$��[��{�[+9�hD3�tQ��1� �v���;��6v������&?����L��?*��Y�h/�-E�[XX�mP�U���89�M��1���R?��%��<�2�|1��y�F����O��&?����a����������$�#6�����t���/�o����,aH�+*ns!$���#��M�=�F����O��&?����s��������o��sG�,D��p2p��a�����s�H	f')����|�u,�c�t����'G���<������y��J�����t4���3���y��G���O��L�T�&�WTEfc���^�yQ��4��3X����+LG�
X����?�x��tO��0���?��N.�K����2������W]<"�5M��sK?g�\����
_��z��[@�q�����u�Ye�����$!�����Xi:5���h����n�Q�$�GM��3�1\������O+X���AFo��G�����j�nO�}rx3������_��?��?�U�8WL�����O�d^��������;��?��?��?�U�(�%�%��r�����������.�?��o��j����j�6��u%}��s?�����5B{���A���7bV���"=�P�L�rp:���N+S�����i%�Y��ge����]m�����dl�@=��K�h�
� m�\�D0FDd�����Ur��t=Jint�}�W~��A]=r�?�7����uU�J<��V	�P�fW�<��w4i$���Wq�`�r*b[	��cQ��c�3
�(8�oF��]�m;O�����)����zu&��)��m*M�������#�p3���t����;�Zq��z?C��,Me&����N���������*?i�>���kg{w�E`�a��U�r���?��t��*[�����U������s��<�:s������^)�F7���`.6����T9������<x��U����Z�2T<��OC����s����tX���.�9��k������*K�cv�-%-%X�3��x<Mw,Gk�>�>�b�����9\,h��|�<�WI>��J�����VW.���8�)�������&��[~u��5v���p�s�}g�<Nr��|��Q�����i[Y2���Q!l��Q���1�6�����?
��$`�{u�M�G��Q<���%by��i{
����]��^$���C����rNG��6��c)	YR�v�0���][���������d�v���"�t��>X*2|����F��H9%���
O��FUC��q_�n3]����\������B��^A�l��(��N3�Z:$���+�8��Tg8�mB��S�N�B-J������K�=�i����c�������_H�jV��d;D��������u6�U�!�B��,���t�u-�V��K�wIn��� ������f5����(��qN���j���� �h��9�Tq�{�CV��A���Kwq���Q�c��p���kSR�t+�bv��k{�J#�����~c��U�
6�.a��d
o�x�}�����w3�����������4�>V�P�YL�9-�x�}�}}S�������z��s��W,'R�1��a�A���h��kh��t�������������X�,-�C9e�s�������l����62�#s>��]j����\��jHXW�~~��_TE��W}N8�]:lvS�������_�J���	B6������X�^[�u~��O������==H�����J->MFqw*�H��W�zt��Mv�M�_�����\#f	�#;��=��Zt��:�K�<h��� ��0��>��lj������hM����y�6��v1�<Tp\�\�3XC���ps$BO�z{{��Vt�I��4z��r/�v��;8+�3�r8�zKf��o�����Iqp��yr��+63������
}:����h�o��H�$B�����T���k'H�P�
�)��4�����Cc�!�kP|�����8�B�������p���Z�����������i&�A4k��&��Fp�#��?�����p�\��@�<��	���V��s�+(���8�I�F]�M���5���>�������9i�B$����r�:V�2kG�H��)�������$V�@�qq;@�~s�`���������tVpIq�7��+����A�@*�������/�/:��\:6�<m�	���j7W�U�rE�Mr�6s����������G"����
�+���B��&K<�2F�!a��b���	�]
�3�Vq"J��r����XO>�
�[�&{��1���&f�<�g���KX��&��\I�h��$�he1�V$u����)�I�������h����Av�p���'���p�I��&���$�P��qso1�@D�\c �z�jk�"��N,��������C��T
���S������^;��r��n�eC�
F2Ks�������6���i7�l��q�q��D�����
���8#8������sqt#�1���C/sU�q��>cj�nzd������V5���%�H�S)�m��	�&?�A9��)����C�
�Z]�4��o&��o`�1�)�z%����#J������P�b�}�~����yguaioyp&y��1'��@�5kZ�&�����s�:�9}���r���m+�=��u����o�+�<����Y,���
�<3kAm=��S��H��0w�8�J�Mr���e�����)���Y)`�p=s�)�j��AP���������Hs�$�H���P*�'��xv:�>���tiK���h�G����MA�Q��7���x�
Z��l��}������V)9=�R��'dj�����o�\��~�k����^u���z���m�������5�6�qyr��H����=���"��+�5�J�i��>g��]��#��������;[T�<�
?���y}�j�V�&s��������������|����<�\\N�O�����?��M����]��#p7�����+W%{��H�	B<����}�o�7�+8��^�M)#,���=$����Yzb'��W5��K�Ff������kj��M�������"��eu$�u'��>�O��'ys~�G�-���������ja�JI;�dzr��NMZ*���:��mW��e����5VG
�z��Q�}j�������a�����z�fes�g����}[��������`�ZP%E��A�*����^H/�{�i����#@��sYJ���oCHP�v�����29���xWO��me���������8��V�������/m$K����Y�NB1��9�V�E�����;�2,��9�j���D���&��������������R��RaK
�Z���kt2����1��(_M�W*�T����ju����x��l���������_@*k��^X���RMk+0H ������e�0q�����nk)����G��R����.<7k�ys���H�!b�1�3`w���g�W-w<J�g�xM�����������&���	T&VS��}y�b�	����d�g���\v#�����{5������YA$2Oa[����x�����kh�(�wc����T�A^V"z�k���������|A[?�3Rv$��r���1U4�8�i������o��� p=�K�o������5)�tw���EE�Z��-S�����ps�q�C&�2H�7�L,�@!�z��U�����T}����Rw�:�K���S�EH"��;�
�1�y'�4;��,�-������`3��?\
��\���Q���_���Ff����{��$�o���|���L���I�N��������A<g99�3[kE���\���Q�������uU�H����N}s��Z�S�rz/�G���*9��d�����D3��c%[���EB��z+��7 `���Z?kE���\���Vm&.S8Y��]M��r�G9��,_qk[����>O<������=���rz/�E��!r��K;�7�������&�����
�����<�ym������k��*>�'��TY)J�++9�gx�:~��8�\��y������q���qK���_������T�d4�\����OE���\���U����hZ}��-������<p��TM��)�#Z�������U������k��*9�?��p���21������������_���������U?�����k��*9�������D�*I$�����3Y���>�#G������cR
`�n�{�=�Z�kE���[�/�G:���St����Z1pe$�'9# ����=���O��y^` ���ay�������_������Ts�+��x���mc��L���q�}�jU?����T}�OE���@:KV}J��,Q��'qS��������_�������?h����7�%dS$`�b9Pq�}p*/��AQ}�?.,�]�.APH�i�k��*>�'��T{@��=2�O����U�M�YV(fR�uS�~��������k��*=��Wa�zu��i�	*!���C�d~�t�&������BH������rz/�G���_��hB�ivW�][E)���\����JF�,^�^5�&��a^r:��k��*>�'��T{P�@4�%�7�k�<���c?\w�fY�i�a�����
�xc��qM�\���Q��=���*!��{K�V/���[pzc9�qV�����!��$*��2S�T_k��*>�'��T{P�]�:���x�����yfQ�'�v�5����4�IV�W��@?�'����'��U��\�%���4BC��>>�{Bf�"�c/_�X[7D���a�G��_�^}42+��%���9���c��n?�_��C�`C4�����bTvG�Z5+��O�q{i�+�����WW'���$�L�����-s�6|�W�+zu�No�����dY�Z�m�=~��%�Ip�c������9j���g��_��������k�����pUV�%��=�eq���=9�����6D�����O�������_��Ib����Iu9�����@����_��?���N�����c�+9b"f���s��N����G������
�����}�
�U�O������}������
�UE+�_Z'�y���R.�n?�����
��Z6��Q�����(��Cz���D��I�1�(����_��y����J��������
z�P�|�~c�+��C�X�|�p�d�My����=\<��
image005.jpgimage/jpeg; name=image005.jpgDownload
����JFIF``��C
	
		
$ &%# #"(-90(*6+"#2D26;=@@@&0FKE>J9?@=��C
=)#)==================================================���"��	
���}!1AQa"q2���#B��R��$3br�	
%&'()*456789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz���������������������������������������������������������������������������	
���w!1AQaq"2�B����	#3R�br�
$4�%�&'()*56789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz��������������������������������������������������������������������������?�>;�i5�P��v����x���z<Z��x�\�)���������Y�T��t�t�7��o
���k�"�#"c�8_A]r�����Sn�e��X����o�����+3����#�Y-��
��Y`CV<N�2<�-1��r2C�9�<��;��YA.[���n�-�����������@�#f����B�B�3i����+#�X����k1�����f3�������o�-
A��eX�A�w)�����c�T�II4k��Q������~+y���T�U6�o���t-r=o���p+�	?
�
kQ�q|��]\�d��c��0p������C�������2�r�j�T�7;�zt7Z+#�1�(��
�g)m���d�a*�H�'U�>���4G4�Fv���RQ����!'��_%��x�"�c{prO��(?k�����y�!:R�Z�+�^GU�kO��%U%���J�������<N���]G�lefF����#$*���-���tbm����v��u��k9&�Jjr�7f������Y
�vT�u
]��H<��S�Q�O�+���MMo8$������Z�K�e�uV�H����s��v��Y��x�V������(�����E0\Dn"����g���=85���OguOp�Z��o�+�/,1�/z�I�����N�^04���+�RB2��?*�R}�gW�;:�[�xK	'�J�.~��r���y��_}�m���%m����O���:��Z���_������*"�ff�8���zST]����0uU�;��)#�I^RB)<�'��+MH���w��'��\��������3j9����m�����!�d��}Ke�\�e��!�����&5����gt8������=�:�+��W����=jD�w?g2Ag�`8������������S��-����bP\.I86,���QRw��`+N���)sK�\����������	�F��1��G���K��=��R�Bp�wh[}���.D����o|�F=+7]���=Ao����PP�dR�L���)]����f�>$�otI������J��4���8Dv�&�@z�P24;)H2E��FXpz�����l��"$S�*�3�E������_N�B��(�BI��}+/�������s���n������ViY3oI�-tKia��3+y$m���d���}�
"=5����a(
���}���Ei�b��������������~�,cv1��8���s��!��'
�3��rx���9��+$ah��-�
�Xm�>��=��hwf�#�f~Q��W���E]����}>�8���A#�t`����\q�r��IQ-�"8C,jaA }GZ(���"�����K��!ES��ryc���