Pluggable toaster
Hi!
We are working on custom toaster for JSONB [1]/messages/by-id/de83407a-ae3d-a8e1-a788-920eb334f25b@sigaev.ru </messages/by-id/de83407a-ae3d-a8e1-a788-920eb334f25b@sigaev.ru>, because current TOAST is
universal for any data type and because of that it has some disadvantages:
- "one toast fits all" may be not the best solution for particular
type or/and use cases
- it doesn't know the internal structure of data type, so it cannot
choose an optimal toast strategy
- it can't share common parts between different rows and even
versions of rows
Modification of current toaster for all tasks and cases looks too
complex, moreover, it will not works for custom data types. Postgres
is an extensible database, why not to extent its extensibility even
further, to have pluggable TOAST! We propose an idea to separate
toaster from heap using toaster API similar to table AM API etc.
Following patches are applicable over patch in [1]/messages/by-id/de83407a-ae3d-a8e1-a788-920eb334f25b@sigaev.ru </messages/by-id/de83407a-ae3d-a8e1-a788-920eb334f25b@sigaev.ru>
1) 1_toaster_interface_v1.patch.gz
https://github.com/postgrespro/postgres/tree/toaster_interface
Introduces syntax for storage and formal toaster API. Adds column
atttoaster to pg_attribute, by design this column should not be equal to
invalid oid for any toastable datatype, ie it must have correct oid for
any type (not column) with non-plain storage. Since toaster may support
only particular datatype, core should check correctness of toaster set
by toaster validate method. New commands could be found in
src/test/regress/sql/toaster.sql
On-disk toast pointer structure now has one more possible struct -
varatt_custom with fixed header and variable tail which uses as a
storage for custom toasters. Format of built-in toaster is kept to allow
simple pg_upgrade logic.
Since toaster for column could be changed during table's lifetime we had
two options about toaster's drop operation:
- if column's toaster has been changed, then we need to re-toast all
values, which could be extremely expensive. In any case,
functions/operators should be ready to work with values toasted by
different toasters, although any toaster should execute simple
toast/detoast operation, which allows any existing code to
work with the new approach. Tracking dependency of toasters and
rows looks as bad idea.
- disallow drop toaster. We don't believe that there will be many
toasters at the same time (number of AM isn't very high too and
we don't believe that it will be changed significantly in the near
future), so prohibition of dropping of toaster looks reasonable.
In this patch set we choose second option.
Toaster API includes get_vtable method, which is planned to access the
custom toaster features which isn't covered by this API. The idea is,
that toaster returns some structure with some values and/or pointers to
toaster's methods and caller could use it for particular purposes, see
patch 4). Kind of structure identified by magic number, which should be
a first field in this structure.
Also added contrib/dummy_toaster to simplify checking.
psql/pg_dump are modified to support toaster object concept.
2) 2_toaster_default_v1.patch.gz
https://github.com/postgrespro/postgres/tree/toaster_default
Built-in toaster implemented (with some refactoring) uisng toaster API
as generic (or default) toaster. dummy_toaster here is a minimal
workable example, it saves value directly in toast pointer and fails if
value is greater than 1kb.
3) 3_toaster_snapshot_v1.patch.gz
https://github.com/postgrespro/postgres/tree/toaster_snapshot
The patch implements technology to distinguish row's versions in toasted
values to share common parts of toasted values between different
versions of rows
4) 4_bytea_appendable_toaster_v1.patch.gz
https://github.com/postgrespro/postgres/tree/bytea_appendable_toaster
Contrib module implements toaster for non-compressed bytea columns,
which allows fast appending to existing bytea value. Appended tail
stored directly in toaster pointer, if there is enough place to do it.
Note: patch modifies byteacat() to support contrib toaster. Seems, it's
looks ugly and contrib module should create new concatenation function.
We are open for any questions, discussions, objections and advices.
Thank you.
Peoples behind:
Oleg Bartunov
Nikita Gluhov
Nikita Malakhov
Teodor Sigaev
[1]: /messages/by-id/de83407a-ae3d-a8e1-a788-920eb334f25b@sigaev.ru </messages/by-id/de83407a-ae3d-a8e1-a788-920eb334f25b@sigaev.ru>
/messages/by-id/de83407a-ae3d-a8e1-a788-920eb334f25b@sigaev.ru
</messages/by-id/de83407a-ae3d-a8e1-a788-920eb334f25b@sigaev.ru>
--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/
Attachments:
Import Notes
Reply to msg id not found: CAF4Au4zsdZ3tf+YAaq12-SzqfCxfOTc1NanjBcUsw+5L-0iiA@mail.gmail.comReference msg id not found: CAF4Au4zsdZ3tf+YAaq12-SzqfCxfOTc1NanjBcUsw+5L-0iiA@mail.gmail.com
On Thu, 30 Dec 2021 at 16:40, Teodor Sigaev <teodor@sigaev.ru> wrote:
We are working on custom toaster for JSONB [1], because current TOAST is
universal for any data type and because of that it has some disadvantages:
- "one toast fits all" may be not the best solution for particular
type or/and use cases
- it doesn't know the internal structure of data type, so it cannot
choose an optimal toast strategy
- it can't share common parts between different rows and even
versions of rows
Agreed, Oleg has made some very clear analysis of the value of having
a higher degree of control over toasting from within the datatype.
In my understanding, we want to be able to
1. Access data from a toasted object one slice at a time, by using
knowledge of the structure
2. If toasted data is updated, then update a minimum number of
slices(s), without rewriting the existing slices
3. If toasted data is expanded, then allownew slices to be appended to
the object without rewriting the existing slices
Modification of current toaster for all tasks and cases looks too
complex, moreover, it will not works for custom data types. Postgres
is an extensible database, why not to extent its extensibility even
further, to have pluggable TOAST! We propose an idea to separate
toaster from heap using toaster API similar to table AM API etc.
Following patches are applicable over patch in [1]
ISTM that we would want the toast algorithm to be associated with the
datatype, not the column?
Can you explain your thinking?
We already have Expanded toast format, in-memory, which was designed
specifically to allow us to access sub-structure of the datatype
in-memory. So I was expecting to see an Expanded, on-disk, toast
format that roughly matched that concept, since Tom has already shown
us the way. (varatt_expanded). This would be usable by both JSON and
PostGIS.
Some other thoughts:
I imagine the data type might want to keep some kind of dictionary
inside the main toast pointer, so we could make allowance for some
optional datatype-specific private area in the toast pointer itself,
allowing a mix of inline and out-of-line data, and/or a table of
contents to the slices.
I'm thinking could also tackle these things at the same time:
* We want to expand TOAST to 64-bit pointers, so we can have more
pointers in a table
* We want to avoid putting the data length into the toast pointer, so
we can allow the toasted data to be expanded without rewriting
everything (to avoid O(N^2) cost)
--
Simon Riggs http://www.EnterpriseDB.com/
Hi all!
Simon, thank you for your review.
I'll try to give a brief explanation on some topics you've mentioned.
My colleagues would correct me if I miss the point and provide some more
details.
Agreed, Oleg has made some very clear analysis of the value of having
a higher degree of control over toasting from within the datatype.
Currently we see the biggest flaw in TOAST functionality is that it
does not provide any means for extension and modification except
modifying the core code itself. It is not possible to use any other
TOAST strategy except existing in the core, the same issue is with
assigning different TOAST methods to columns and datatypes.
The main point in this patch is actually to provide an open API and
syntax for creation of new Toasters as pluggable extensions, and
to make an existing (default) toaster to work via this API without
affecting its function. Also, the default toaster is strongly cross-tied
with Heap access, with somewhat unclear code relations (headers,
function locations and calls, etc.) that are not quite good logically
structured and ask to be straightened out.
In my understanding, we want to be able to
1. Access data from a toasted object one slice at a time, by using
knowledge of the structure
2. If toasted data is updated, then update a minimum number of
slices(s), without rewriting the existing slices
3. If toasted data is expanded, then allownew slices to be appended to
the object without rewriting the existing slices
There are two main ideas behind Pluggable Toaster patch -
First - to provide an extensible API for all Postgres developers, to
be able to develop and plug in custom toasters as independent
extensions for different data types and columns, to use different
toast strategies, access and compression methods, and so on;
Second - to refactor current Toast functionality, to improve Toast
code structure and make it more logically structured and
understandable, to 'detach' default ('generic', as it is currently
named, or maybe the best naming for it to be 'heap') toaster from DBMS
core code, route it through new API and hide all existing internal
specific Toast functionality behind new API.
All the points you mentioned are made available for development by
this patch (and, actually, some are being developed - in the
bytea_appendable_toaster part of this patch or jSONb toaster by Nikita
Glukhov, he could provide much better explanation on this topic).
Modification of current toaster for all tasks and cases looks too
complex, moreover, it will not works for custom data types. Postgres
is an extensible database, why not to extent its extensibility even
further, to have pluggable TOAST! We propose an idea to separate
toaster from heap using toaster API similar to table AM API etc.
Following patches are applicable over patch in [1]
ISTM that we would want the toast algorithm to be associated with the
datatype, not the column?
Can you explain your thinking?
This possibility is considered for future development.
We already have Expanded toast format, in-memory, which was designed
specifically to allow us to access sub-structure of the datatype
in-memory. So I was expecting to see an Expanded, on-disk, toast
format that roughly matched that concept, since Tom has already shown
us the way. (varatt_expanded). This would be usable by both JSON and
PostGIS.
The main disadvantage is that it does not suppose either usage of any
other toasting strategies, or compressions methods except plgz and
lz4.
Some other thoughts:
I imagine the data type might want to keep some kind of dictionary
inside the main toast pointer, so we could make allowance for some
optional datatype-specific private area in the toast pointer itself,
allowing a mix of inline and out-of-line data, and/or a table of
contents to the slices.
It is partly implemented in jSONb custom Toaster, as I mentioned
above, and also could be considered for future improvement of existing
Toaster as an extension.
I'm thinking could also tackle these things at the same time:
* We want to expand TOAST to 64-bit pointers, so we can have more
pointers in a table
This issue is being discussed but not currently implemented, it was
considered as one of the possible future improvements.
* We want to avoid putting the data length into the toast pointer, so
we can allow the toasted data to be expanded without rewriting
everything (to avoid O(N^2) cost)
May I correct you - actual relation is O(N), not O(N^2).
Currently data length is stored outside customized toaster data, in
the varatt_custom structure that is supposed to be used by all custom
(extended) toasters. Data that is specific to some custom Toasted will
be stored inside va_toasterdata structure.
Looking forward to your thoughts on our work.
--
Best regards,
Nikita A. Malakhov
On Wed, Jan 5, 2022 at 5:46 PM Simon Riggs <simon.riggs@enterprisedb.com>
wrote:
Show quoted text
On Thu, 30 Dec 2021 at 16:40, Teodor Sigaev <teodor@sigaev.ru> wrote:
We are working on custom toaster for JSONB [1], because current TOAST is
universal for any data type and because of that it has somedisadvantages:
- "one toast fits all" may be not the best solution for particular
type or/and use cases
- it doesn't know the internal structure of data type, so it cannot
choose an optimal toast strategy
- it can't share common parts between different rows and even
versions of rowsAgreed, Oleg has made some very clear analysis of the value of having
a higher degree of control over toasting from within the datatype.In my understanding, we want to be able to
1. Access data from a toasted object one slice at a time, by using
knowledge of the structure
2. If toasted data is updated, then update a minimum number of
slices(s), without rewriting the existing slices
3. If toasted data is expanded, then allownew slices to be appended to
the object without rewriting the existing slicesModification of current toaster for all tasks and cases looks too
complex, moreover, it will not works for custom data types. Postgres
is an extensible database, why not to extent its extensibility even
further, to have pluggable TOAST! We propose an idea to separate
toaster from heap using toaster API similar to table AM API etc.
Following patches are applicable over patch in [1]ISTM that we would want the toast algorithm to be associated with the
datatype, not the column?
Can you explain your thinking?We already have Expanded toast format, in-memory, which was designed
specifically to allow us to access sub-structure of the datatype
in-memory. So I was expecting to see an Expanded, on-disk, toast
format that roughly matched that concept, since Tom has already shown
us the way. (varatt_expanded). This would be usable by both JSON and
PostGIS.Some other thoughts:
I imagine the data type might want to keep some kind of dictionary
inside the main toast pointer, so we could make allowance for some
optional datatype-specific private area in the toast pointer itself,
allowing a mix of inline and out-of-line data, and/or a table of
contents to the slices.I'm thinking could also tackle these things at the same time:
* We want to expand TOAST to 64-bit pointers, so we can have more
pointers in a table
* We want to avoid putting the data length into the toast pointer, so
we can allow the toasted data to be expanded without rewriting
everything (to avoid O(N^2) cost)--
Simon Riggs http://www.EnterpriseDB.com/
In my understanding, we want to be able to
1. Access data from a toasted object one slice at a time, by using
knowledge of the structure
2. If toasted data is updated, then update a minimum number of
slices(s), without rewriting the existing slices
3. If toasted data is expanded, then allownew slices to be appended to
the object without rewriting the existing slices
There are more options:
1 share common parts between not only versions of row but between all
rows in a column. Seems strange but examples:
- urls often have a common prefix and so storing in a prefix tree (as
SP-GiST does) allows significantly decrease storage size
- the same for json - it's often use case with common part of its
hierarchical structure
- one more usecase for json. If json use only a few schemes
(structure) it's possible to store in toast storage only values and
don't store keys and structure
2 Current toast storage stores chunks in heap accesses method and to
provide fast access by toast id it makes an index. Ideas:
- store chunks directly in btree tree, pgsql's btree already has an
INCLUDE columns, so, chunks and visibility data will be stored only
in leaf pages. Obviously it reduces number of disk's access for
"untoasting".
- use another access method for chunk storage
ISTM that we would want the toast algorithm to be associated with the
datatype, not the column?
Can you explain your thinking?
Hm. I'll try to explain my motivation.
1) Datatype could have more than one suitable toasters. For different
usecases: fast retrieving, compact storage, fast update etc. As I
told above, for jsonb there are several optimal strategies for
toasting: for values with a few different structures, for close to
hierarchical structures, for values with different parts by access
mode (easy to imagine json with some keys used for search and some
keys only for output to user)
2) Toaster could be designed to work with different data type. Suggested
appendable toaster is designed to work with bytea but could work with
text
Looking on this point I have doubts where to store connection between
toaster and datatype. If we add toasteroid to pg_type how to deal with
several toaster for one datatype? (And we could want to has different
toaster on one table!) If we add typoid to pg_toaster then how it will
work with several datatypes? An idea to add a new many-to-many
connection table seems workable but here there are another questions,
such as will any toaster work with any table access method?
To resolve this bundle of question we propose validate() method of
toaster, which should be called during DDL operation, i.e. toaster is
assigned to column or column's datatype is changed.
More thought:
Now postgres has two options for column: storage and compression and now
we add toaster. For me it seems too redundantly. Seems, storage should
be binary value: inplace (plain as now) and toastable. All other
variation such as toast limit, compression enabling, compression kind
should be an per-column option for toaster (that's why we suggest valid
toaster oid for any column with varlena/toastable datatype). It looks
like a good abstraction but we will have a problem with backward
compatibility and I'm afraid I can't implement it very fast.
We already have Expanded toast format, in-memory, which was designed
specifically to allow us to access sub-structure of the datatype
in-memory. So I was expecting to see an Expanded, on-disk, toast
format that roughly matched that concept, since Tom has already shown
us the way. (varatt_expanded). This would be usable by both JSON and
PostGIS.
Hm, I don't understand. varatt_custom has variable-length tail which
toaster could use it by any way, appandable toaster use it to store
appended tail.
Some other thoughts:
I imagine the data type might want to keep some kind of dictionary
inside the main toast pointer, so we could make allowance for some
optional datatype-specific private area in the toast pointer itself,
allowing a mix of inline and out-of-line data, and/or a table of
contents to the slices.I'm thinking could also tackle these things at the same time:
* We want to expand TOAST to 64-bit pointers, so we can have more
pointers in a table
* We want to avoid putting the data length into the toast pointer, so
we can allow the toasted data to be expanded without rewriting
everything (to avoid O(N^2) cost)
Right
--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/
I'd like to ask your opinion for next small questions
1) May be, we could add toasteroid to pg_type to for default toaster for
datatype. ALTER TYPE type SET DEFAULT TOASTER toaster;
2) The name of default toaster is deftoaster, which was choosen at
night, may be heap_toaster is better? heap because default toaster
stores chunks in heap table.
Thank you!
--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/
On 1/14/22 19:41, Teodor Sigaev wrote:
In my understanding, we want to be able to
1. Access data from a toasted object one slice at a time, by using
knowledge of the structure
2. If toasted data is updated, then update a minimum number of
slices(s), without rewriting the existing slices
3. If toasted data is expanded, then allownew slices to be appended to
the object without rewriting the existing slicesThere are more options:
1 share common parts between not only versions of row but between all
rows in a column. Seems strange but examples:
- urls often have a common prefix and so storing in a prefix tree (as
SP-GiST does) allows significantly decrease storage size
- the same for json - it's often use case with common part of its
hierarchical structure
- one more usecase for json. If json use only a few schemes
(structure) it's possible to store in toast storage only values and
don't store keys and structure
This sounds interesting, but very much like column compression, which
was proposed some time ago. If we haven't made much progrees with that
patch (AFAICS), what's the likelihood we'll succeed here, when it's
combined with yet more complexity?
Maybe doing that kind of compression in TOAST is somehow simpler, but I
don't see it.
2 Current toast storage stores chunks in heap accesses method and to
provide fast access by toast id it makes an index. Ideas:
- store chunks directly in btree tree, pgsql's btree already has an
INCLUDE columns, so, chunks and visibility data will be stored only
in leaf pages. Obviously it reduces number of disk's access for
"untoasting".
- use another access method for chunk storage
Maybe, but that probably requires more thought - e.g. btree requires the
values to be less than 1/3 page, so I wonder how would that play with
toasting of values.
ISTM that we would want the toast algorithm to be associated with the
datatype, not the column?
Can you explain your thinking?Hm. I'll try to explain my motivation.
1) Datatype could have more than one suitable toasters. For different
usecases: fast retrieving, compact storage, fast update etc. As I
told above, for jsonb there are several optimal strategies for
toasting: for values with a few different structures, for close to
hierarchical structures, for values with different parts by access
mode (easy to imagine json with some keys used for search and some
keys only for output to user)
2) Toaster could be designed to work with different data type. Suggested
appendable toaster is designed to work with bytea but could work with
textLooking on this point I have doubts where to store connection between
toaster and datatype. If we add toasteroid to pg_type how to deal with
several toaster for one datatype? (And we could want to has different
toaster on one table!) If we add typoid to pg_toaster then how it will
work with several datatypes? An idea to add a new many-to-many
connection table seems workable but here there are another questions,
such as will any toaster work with any table access method?To resolve this bundle of question we propose validate() method of
toaster, which should be called during DDL operation, i.e. toaster is
assigned to column or column's datatype is changed.
Seems you'd need a mapping table, to allow M:N mapping between types and
toasters, linking it to all "compatible" types. It's not clear to me how
would this work with custom data types, domains etc.
Also, what happens to existing values when you change the toaster? What
if the toasters don't use the same access method to store the chunks
(heap vs. btree)? And so on.
More thought:
Now postgres has two options for column: storage and compression and now
we add toaster. For me it seems too redundantly. Seems, storage should
be binary value: inplace (plain as now) and toastable. All other
variation such as toast limit, compression enabling, compression kind
should be an per-column option for toaster (that's why we suggest valid
toaster oid for any column with varlena/toastable datatype). It looks
like a good abstraction but we will have a problem with backward
compatibility and I'm afraid I can't implement it very fast.
So you suggest we move all of this to toaster? I'd say -1 to that,
because it makes it much harder to e.g. add custom compression method, etc.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Hi,
This sounds interesting, but very much like column compression, which
was proposed some time ago. If we haven't made much progrees with that
patch (AFAICS), what's the likelihood we'll succeed here, when it's
combined with yet more complexity?
The main concern is that this patch provides open API for Toast
functionality
and new Toasters could be written as extensions and plugged in instead of
default one, for any column (or datatype). It was not possible before,
because
Toast functionality was part of the Postgres core code and was not meant to
be modified.
Maybe doing that kind of compression in TOAST is somehow simpler, but I
don't see it.
[Custom ]Toaster extension itself is not restricted to only toast data, it
could be
used for compression, encryption, etc, just name it - any case when data
meant
to be transformed in some complicated way before being stored and
(possibly)
transformed backwards while being selected from the table, along with some
simpler but not so obvious transformations like removing common parts shared
by all data in column before storing it and restoring column value to full
during
selection.
Seems you'd need a mapping table, to allow M:N mapping between types and
toasters, linking it to all "compatible" types. It's not clear to me how
would this work with custom data types, domains etc.
Any suitable [custom] Toaster could be plugged in for any table column,
or [duscussible] for datatype and assigned by default to the according
column
for any table using this datatype.
Also, what happens to existing values when you change the toaster? What
if the toasters don't use the same access method to store the chunks
(heap vs. btree)? And so on.
For newer data there is no problem - it will be toasted by the newly
assigned Toaster.
For detoasting - the Toaster ID is stored in Toast pointer and in principle
all data
could be detoasted by the according toaster if it is available. But this is
the topic
for discussion and we are open for proposals, because there are possible
cases
where older Toaster is not available - the older used Toaster extension is
not installed
at all or was uninstalled, it was upgraded to the newer version. Currently
we see
two ways of handling this case - to restrict changing the toaster, and to
re-toast
all toasted data which could be very heavy if Toaster is assigned to a
widely used
datatype, and we're looking forward to any ideas.
So you suggest we move all of this to toaster? I'd say -1 to that,
because it makes it much harder to e.g. add custom compression method, etc.
Original compression methods, etc. are not affected by this patch.
Regards,
--
Nikita Malakhov
Postgres Professional
https://postgrespro.ru/
On Mon, Jan 17, 2022 at 10:23 PM Tomas Vondra <tomas.vondra@enterprisedb.com>
wrote:
Show quoted text
On 1/14/22 19:41, Teodor Sigaev wrote:
In my understanding, we want to be able to
1. Access data from a toasted object one slice at a time, by using
knowledge of the structure
2. If toasted data is updated, then update a minimum number of
slices(s), without rewriting the existing slices
3. If toasted data is expanded, then allownew slices to be appended to
the object without rewriting the existing slicesThere are more options:
1 share common parts between not only versions of row but between all
rows in a column. Seems strange but examples:
- urls often have a common prefix and so storing in a prefix tree (as
SP-GiST does) allows significantly decrease storage size
- the same for json - it's often use case with common part of its
hierarchical structure
- one more usecase for json. If json use only a few schemes
(structure) it's possible to store in toast storage only values and
don't store keys and structureThis sounds interesting, but very much like column compression, which
was proposed some time ago. If we haven't made much progrees with that
patch (AFAICS), what's the likelihood we'll succeed here, when it's
combined with yet more complexity?Maybe doing that kind of compression in TOAST is somehow simpler, but I
don't see it.2 Current toast storage stores chunks in heap accesses method and to
provide fast access by toast id it makes an index. Ideas:
- store chunks directly in btree tree, pgsql's btree already has an
INCLUDE columns, so, chunks and visibility data will be stored only
in leaf pages. Obviously it reduces number of disk's access for
"untoasting".
- use another access method for chunk storageMaybe, but that probably requires more thought - e.g. btree requires the
values to be less than 1/3 page, so I wonder how would that play with
toasting of values.ISTM that we would want the toast algorithm to be associated with the
datatype, not the column?
Can you explain your thinking?Hm. I'll try to explain my motivation.
1) Datatype could have more than one suitable toasters. For different
usecases: fast retrieving, compact storage, fast update etc. As I
told above, for jsonb there are several optimal strategies for
toasting: for values with a few different structures, for close to
hierarchical structures, for values with different parts by access
mode (easy to imagine json with some keys used for search and some
keys only for output to user)
2) Toaster could be designed to work with different data type. Suggested
appendable toaster is designed to work with bytea but could work with
textLooking on this point I have doubts where to store connection between
toaster and datatype. If we add toasteroid to pg_type how to deal with
several toaster for one datatype? (And we could want to has different
toaster on one table!) If we add typoid to pg_toaster then how it will
work with several datatypes? An idea to add a new many-to-many
connection table seems workable but here there are another questions,
such as will any toaster work with any table access method?To resolve this bundle of question we propose validate() method of
toaster, which should be called during DDL operation, i.e. toaster is
assigned to column or column's datatype is changed.Seems you'd need a mapping table, to allow M:N mapping between types and
toasters, linking it to all "compatible" types. It's not clear to me how
would this work with custom data types, domains etc.Also, what happens to existing values when you change the toaster? What
if the toasters don't use the same access method to store the chunks
(heap vs. btree)? And so on.More thought:
Now postgres has two options for column: storage and compression and now
we add toaster. For me it seems too redundantly. Seems, storage should
be binary value: inplace (plain as now) and toastable. All other
variation such as toast limit, compression enabling, compression kind
should be an per-column option for toaster (that's why we suggest valid
toaster oid for any column with varlena/toastable datatype). It looks
like a good abstraction but we will have a problem with backward
compatibility and I'm afraid I can't implement it very fast.So you suggest we move all of this to toaster? I'd say -1 to that,
because it makes it much harder to e.g. add custom compression method, etc.regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Hi!
Maybe doing that kind of compression in TOAST is somehow simpler, but I
don't see it.
Seems, in ideal world, compression should be inside toaster.
2 Current toast storage stores chunks in heap accesses method and to
provide fast access by toast id it makes an index. Ideas:
- store chunks directly in btree tree, pgsql's btree already has an
INCLUDE columns, so, chunks and visibility data will be stored only
in leaf pages. Obviously it reduces number of disk's access for
"untoasting".
- use another access method for chunk storageMaybe, but that probably requires more thought - e.g. btree requires the
values to be less than 1/3 page, so I wonder how would that play with
toasting of values.
That's ok, because chunk size is 2000 bytes right now and its could be
saved.
Seems you'd need a mapping table, to allow M:N mapping between types and
toasters, linking it to all "compatible" types. It's not clear to me how
would this work with custom data types, domains etc.
If toaster will look into internal structure then it should know type's
binary format. So, new custom types have a little chance to work with
old custom toaster. Default toaster works with any types.
Also, what happens to existing values when you change the toaster? What
if the toasters don't use the same access method to store the chunks
(heap vs. btree)? And so on.
vatatt_custom contains an oid of toaster and toaster is not allowed to
delete (at least, in suggested patches). So, if column's toaster has
been changed then old values will be detoasted by toaster pointed in
varatt_custom structure, not in column definition. This is very similar
to storage attribute works: we we alter storage attribute only new
values will be stored with pointed storage type.
More thought:
Now postgres has two options for column: storage and compression and
now we add toaster. For me it seems too redundantly. Seems, storage
should be binary value: inplace (plain as now) and toastable. All
other variation such as toast limit, compression enabling, compression
kind should be an per-column option for toaster (that's why we suggest
valid toaster oid for any column with varlena/toastable datatype). It
looks like a good abstraction but we will have a problem with backward
compatibility and I'm afraid I can't implement it very fast.So you suggest we move all of this to toaster? I'd say -1 to that,
because it makes it much harder to e.g. add custom compression method, etc.
Hmm, I suggested to leave only toaster at upper level. Compression kind
could be chosen in toaster's options (not implemented yet) or even make
an API interface to compression to make it configurable. Right now,
module developer could not implement a module with new compression
method and it is a disadvantage.
--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/
On 1/18/22 15:56, Teodor Sigaev wrote:
Hi!
Maybe doing that kind of compression in TOAST is somehow simpler, but
I don't see it.Seems, in ideal world, compression should be inside toaster.
I'm not convinced that's universally true. Yes, I'm sure certain TOAST
implementations would benefit from tighter control over compression, but
does that imply compression and toast are redundant? I doubt that,
because we compress non-toasted types too, for example. And layering has
a value too, as makes it easier to replace the pieces.
2 Current toast storage stores chunks in heap accesses method and to
provide fast access by toast id it makes an index. Ideas:
- store chunks directly in btree tree, pgsql's btree already has an
INCLUDE columns, so, chunks and visibility data will be stored only
in leaf pages. Obviously it reduces number of disk's access for
"untoasting".
- use another access method for chunk storageMaybe, but that probably requires more thought - e.g. btree requires
the values to be less than 1/3 page, so I wonder how would that play
with toasting of values.That's ok, because chunk size is 2000 bytes right now and its could be
saved.
Perhaps. My main point is that we should not be making too many radical
changes at once - it makes it much harder to actually get anything done.
So yeah, doing TOAST through IOT might be interesting, but I'd leave
that for a separate patch.
Seems you'd need a mapping table, to allow M:N mapping between types
and toasters, linking it to all "compatible" types. It's not clear to
me how would this work with custom data types, domains etc.If toaster will look into internal structure then it should know type's
binary format. So, new custom types have a little chance to work with
old custom toaster. Default toaster works with any types.
The question is what happens when you combine data type with a toaster
that is not designed for that type. I mean, imagine you have a JSONB
toaster and you set it for a bytea column. Naive implementation will
just crash, because it'll try to process bytea as if it was JSONB.
It seems better to prevent such incompatible combinations and restrict
each toaster to just compatible data types, and the mapping table
(linking toaster and data types) seems a way to do that.
However, it seems toasters are either generic (agnostic to data types,
treating everything as bytea) or specialized. I doubt any specialized
toaster can reasonably support multiple data types, so maybe each
toaster can have just one "compatible type" OID. If it's invalid, it'd
be "generic" and otherwise it's useful for that type and types derived
from it (e.g. domains).
So you'd have the toaster OID in two places:
pg_type.toaster_oid - default toaster for the type
pg_attribute.toaster_oid - current toaster for this column
and then you'd have
pg_toaster.typid - type this toaster handles (or InvalidOid for generic)
Also, what happens to existing values when you change the toaster?
What if the toasters don't use the same access method to store the
chunks (heap vs. btree)? And so on.vatatt_custom contains an oid of toaster and toaster is not allowed to
delete (at least, in suggested patches). So, if column's toaster has
been changed then old values will be detoasted by toaster pointed in
varatt_custom structure, not in column definition. This is very similar
to storage attribute works: we we alter storage attribute only new
values will be stored with pointed storage type.
IIRC we do this for compression methods, right?
More thought:
Now postgres has two options for column: storage and compression and
now we add toaster. For me it seems too redundantly. Seems, storage
should be binary value: inplace (plain as now) and toastable. All
other variation such as toast limit, compression enabling,
compression kind should be an per-column option for toaster (that's
why we suggest valid toaster oid for any column with
varlena/toastable datatype). It looks like a good abstraction but we
will have a problem with backward compatibility and I'm afraid I
can't implement it very fast.So you suggest we move all of this to toaster? I'd say -1 to that,
because it makes it much harder to e.g. add custom compression method,
etc.Hmm, I suggested to leave only toaster at upper level. Compression kind
could be chosen in toaster's options (not implemented yet) or even make
an API interface to compression to make it configurable. Right now,
module developer could not implement a module with new compression
method and it is a disadvantage.
If you have to implement custom toaster to implement custom compression
method, doesn't that make things more complex? You'd have to solve all
the issues for custom compression methods and also all issues for custom
toaster. Also, what if you want to just compress the column, not toast?
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Hi,
I'm not convinced that's universally true. Yes, I'm sure certain TOAST
implementations would benefit from tighter control over compression, but
does that imply compression and toast are redundant? I doubt that,
because we compress non-toasted types too, for example. And layering has
a value too, as makes it easier to replace the pieces.
Not exactly. It is a mean to control TOAST itself without changing the core
each time you want to change Toast strategy or method. Compression is
just an example. And no Toasters are available without the patch proposed,
there is the one and only.
Perhaps. My main point is that we should not be making too many radical
changes at once - it makes it much harder to actually get anything done.
So yeah, doing TOAST through IOT might be interesting, but I'd leave
that for a separate patch.
That's why 4 distinct patches with incremental changes were proposed -
1) just new Toaster API with some necessary core changes required by the
API;
2) default toaster routed via new API (but all it's functions are not
affected
and dummy toaster extension as an example);
3) 1+2+some refactoring and versioning;
4) extension module for bytea columns.
Toast through IOT is a topic for discussion but does not seem to give a
major
advantage over existing storage method, according to tests.
It seems better to prevent such incompatible combinations and restrict
each toaster to just compatible data types, and the mapping table
(linking toaster and data types) seems a way to do that.
To handle this case a validate function (toastervalidate_function) is
proposed
in the TsrRoutine structure.
If you have to implement custom toaster to implement custom compression
method, doesn't that make things more complex? You'd have to solve all
the issues for custom compression methods and also all issues for custom
toaster. Also, what if you want to just compress the column, not toast?
Default compression is restricted to 2 compression methods, all other means
require extensions. Also, the name Toaster is a little bit misleading
because
it intends that data is being sliced, but it is not always the case, to be
toasted
a piece of bread must not necessarily be sliced.
Regards,
--
Nikita Malakhov
Postgres Professional
https://postgrespro.ru/
On Tue, Jan 18, 2022 at 7:06 PM Tomas Vondra <tomas.vondra@enterprisedb.com>
wrote:
Show quoted text
On 1/18/22 15:56, Teodor Sigaev wrote:
Hi!
Maybe doing that kind of compression in TOAST is somehow simpler, but
I don't see it.Seems, in ideal world, compression should be inside toaster.
I'm not convinced that's universally true. Yes, I'm sure certain TOAST
implementations would benefit from tighter control over compression, but
does that imply compression and toast are redundant? I doubt that,
because we compress non-toasted types too, for example. And layering has
a value too, as makes it easier to replace the pieces.2 Current toast storage stores chunks in heap accesses method and to
provide fast access by toast id it makes an index. Ideas:
- store chunks directly in btree tree, pgsql's btree already has an
INCLUDE columns, so, chunks and visibility data will be storedonly
in leaf pages. Obviously it reduces number of disk's access for
"untoasting".
- use another access method for chunk storageMaybe, but that probably requires more thought - e.g. btree requires
the values to be less than 1/3 page, so I wonder how would that play
with toasting of values.That's ok, because chunk size is 2000 bytes right now and its could be
saved.Perhaps. My main point is that we should not be making too many radical
changes at once - it makes it much harder to actually get anything done.
So yeah, doing TOAST through IOT might be interesting, but I'd leave
that for a separate patch.Seems you'd need a mapping table, to allow M:N mapping between types
and toasters, linking it to all "compatible" types. It's not clear to
me how would this work with custom data types, domains etc.If toaster will look into internal structure then it should know type's
binary format. So, new custom types have a little chance to work with
old custom toaster. Default toaster works with any types.The question is what happens when you combine data type with a toaster
that is not designed for that type. I mean, imagine you have a JSONB
toaster and you set it for a bytea column. Naive implementation will
just crash, because it'll try to process bytea as if it was JSONB.It seems better to prevent such incompatible combinations and restrict
each toaster to just compatible data types, and the mapping table
(linking toaster and data types) seems a way to do that.However, it seems toasters are either generic (agnostic to data types,
treating everything as bytea) or specialized. I doubt any specialized
toaster can reasonably support multiple data types, so maybe each
toaster can have just one "compatible type" OID. If it's invalid, it'd
be "generic" and otherwise it's useful for that type and types derived
from it (e.g. domains).So you'd have the toaster OID in two places:
pg_type.toaster_oid - default toaster for the type
pg_attribute.toaster_oid - current toaster for this columnand then you'd have
pg_toaster.typid - type this toaster handles (or InvalidOid for generic)
Also, what happens to existing values when you change the toaster?
What if the toasters don't use the same access method to store the
chunks (heap vs. btree)? And so on.vatatt_custom contains an oid of toaster and toaster is not allowed to
delete (at least, in suggested patches). So, if column's toaster has
been changed then old values will be detoasted by toaster pointed in
varatt_custom structure, not in column definition. This is very similar
to storage attribute works: we we alter storage attribute only new
values will be stored with pointed storage type.IIRC we do this for compression methods, right?
More thought:
Now postgres has two options for column: storage and compression and
now we add toaster. For me it seems too redundantly. Seems, storage
should be binary value: inplace (plain as now) and toastable. All
other variation such as toast limit, compression enabling,
compression kind should be an per-column option for toaster (that's
why we suggest valid toaster oid for any column with
varlena/toastable datatype). It looks like a good abstraction but we
will have a problem with backward compatibility and I'm afraid I
can't implement it very fast.So you suggest we move all of this to toaster? I'd say -1 to that,
because it makes it much harder to e.g. add custom compression method,
etc.Hmm, I suggested to leave only toaster at upper level. Compression kind
could be chosen in toaster's options (not implemented yet) or even make
an API interface to compression to make it configurable. Right now,
module developer could not implement a module with new compression
method and it is a disadvantage.If you have to implement custom toaster to implement custom compression
method, doesn't that make things more complex? You'd have to solve all
the issues for custom compression methods and also all issues for custom
toaster. Also, what if you want to just compress the column, not toast?regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Thu, Dec 30, 2021 at 11:40 AM Teodor Sigaev <teodor@sigaev.ru> wrote:
We are working on custom toaster for JSONB [1], because current TOAST is
universal for any data type and because of that it has some disadvantages:
- "one toast fits all" may be not the best solution for particular
type or/and use cases
- it doesn't know the internal structure of data type, so it cannot
choose an optimal toast strategy
- it can't share common parts between different rows and even
versions of rows
I agree ... but I'm also worried about what happens when we have
multiple table AMs. One can imagine a new table AM that is
specifically optimized for TOAST which can be used with an existing
heap table. One can imagine a new table AM for the main table that
wants to use something different for TOAST. So, I don't think it's
right to imagine that the choice of TOASTer depends solely on the
column data type. I'm not really sure how this should work exactly ...
but it needs careful thought.
--
Robert Haas
EDB: http://www.enterprisedb.com
Hi,
The patch provides assigning toaster to a data column separately. Assigning
to a data type is considered worthy
but is also a topic for further discussion and is not included in patch.
We've been thinking how to integrate AMs and custom toasters, so any
thoughts are welcome.
Regards,
On Thu, Jan 20, 2022 at 7:00 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Dec 30, 2021 at 11:40 AM Teodor Sigaev <teodor@sigaev.ru> wrote:
We are working on custom toaster for JSONB [1], because current TOAST is
universal for any data type and because of that it has somedisadvantages:
- "one toast fits all" may be not the best solution for particular
type or/and use cases
- it doesn't know the internal structure of data type, so it cannot
choose an optimal toast strategy
- it can't share common parts between different rows and even
versions of rowsI agree ... but I'm also worried about what happens when we have
multiple table AMs. One can imagine a new table AM that is
specifically optimized for TOAST which can be used with an existing
heap table. One can imagine a new table AM for the main table that
wants to use something different for TOAST. So, I don't think it's
right to imagine that the choice of TOASTer depends solely on the
column data type. I'm not really sure how this should work exactly ...
but it needs careful thought.--
Robert Haas
EDB: http://www.enterprisedb.com
--
Regards,
--
Nikita Malakhov
Postgres Professional
https://postgrespro.ru/
I agree ... but I'm also worried about what happens when we have
multiple table AMs. One can imagine a new table AM that is
specifically optimized for TOAST which can be used with an existing
heap table. One can imagine a new table AM for the main table that
wants to use something different for TOAST. So, I don't think it's
right to imagine that the choice of TOASTer depends solely on the
column data type. I'm not really sure how this should work exactly ...
but it needs careful thought.
Right. that's why we propose a validate method (may be, it's a wrong
name, but I don't known better one) which accepts several arguments, one
of which is table AM oid. If that method returns false then toaster
isn't useful with current TAM, storage or/and compression kinds, etc.
--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/
Hi Hackers,
In addition to original patch set for Pluggable Toaster, we have two more
patches
(actually, small, but important fixes), authored by Nikita Glukhov:
1) 0001-Fix-toast_tuple_externalize.patch
This patch fixes freeing memory in case of new toasted value is the same as
old one,
this seems incorrect, and in this case the function just returns instead of
freeing old value;
2) 0002-Fix-alignment-of-custom-TOAST-pointers.patch
This patch adds data alignment for new varatt_custom data structure in
building tuples,
since varatt_custom must be aligned for custom toasters (in particular,
this fix is very
important to JSONb Toaster).
These patches must be applied on top of the latter
4_bytea_appendable_toaster_v1.patch.
Please consider them in reviewing Pluggable Toaster.
Regards.
On Wed, Feb 2, 2022 at 10:35 AM Teodor Sigaev <teodor@sigaev.ru> wrote:
I agree ... but I'm also worried about what happens when we have
multiple table AMs. One can imagine a new table AM that is
specifically optimized for TOAST which can be used with an existing
heap table. One can imagine a new table AM for the main table that
wants to use something different for TOAST. So, I don't think it's
right to imagine that the choice of TOASTer depends solely on the
column data type. I'm not really sure how this should work exactly ...
but it needs careful thought.Right. that's why we propose a validate method (may be, it's a wrong
name, but I don't known better one) which accepts several arguments, one
of which is table AM oid. If that method returns false then toaster
isn't useful with current TAM, storage or/and compression kinds, etc.--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/
--
Nikita Malakhov
Postgres Professional
https://postgrespro.ru/
Attachments:
0001-Fix-toast_tuple_externalize.patchapplication/octet-stream; name=0001-Fix-toast_tuple_externalize.patchDownload
From c16114bf17cee5b5649d3dfa462c4a8b594fb2f0 Mon Sep 17 00:00:00 2001
From: Nikita Glukhov <n.gluhov@postgrespro.ru>
Date: Tue, 22 Feb 2022 20:52:45 +0300
Subject: [PATCH 1/2] Fix toast_tuple_externalize()
---
src/backend/access/table/toast_helper.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index bfbe941d6ac..11d870535e1 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -345,6 +345,9 @@ toast_tuple_externalize(ToastTupleContext *ttc, int attribute, int maxDataLen,
maxDataLen, options)
);
+ if (*value == old_value)
+ return;
+
if ((attr->tai_colflags & TOASTCOL_NEEDS_FREE) != 0)
pfree(DatumGetPointer(old_value));
attr->tai_colflags |= TOASTCOL_NEEDS_FREE;
--
2.25.1
0002-Fix-alignment-of-custom-TOAST-pointers.patchapplication/octet-stream; name=0002-Fix-alignment-of-custom-TOAST-pointers.patchDownload
From 6db6630943be39eee31cb0411e2a849ec97bc66c Mon Sep 17 00:00:00 2001
From: Nikita Glukhov <n.gluhov@postgrespro.ru>
Date: Wed, 12 Jan 2022 01:23:18 +0300
Subject: [PATCH 2/2] Fix alignment of custom TOAST pointers
---
src/backend/access/common/heaptuple.c | 5 ++++-
src/include/access/tupmacs.h | 2 +-
2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0b56b0fa5a9..47c808d462f 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -242,7 +242,10 @@ fill_val(Form_pg_attribute att,
else
{
*infomask |= HEAP_HASEXTERNAL;
- /* no alignment, since it's short by definition */
+ if (VARATT_IS_CUSTOM(val))
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ /* else no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
}
diff --git a/src/include/access/tupmacs.h b/src/include/access/tupmacs.h
index 65ac1ef3fc8..ddb164ef2ba 100644
--- a/src/include/access/tupmacs.h
+++ b/src/include/access/tupmacs.h
@@ -104,7 +104,7 @@
*/
#define att_align_datum(cur_offset, attalign, attlen, attdatum) \
( \
- ((attlen) == -1 && VARATT_IS_SHORT(DatumGetPointer(attdatum))) ? \
+ ((attlen) == -1 && (!VARATT_IS_CUSTOM(DatumGetPointer(attdatum)) && VARATT_IS_SHORT(DatumGetPointer(attdatum)))) ? \
(uintptr_t) (cur_offset) : \
att_align_nominal(cur_offset, attalign) \
)
--
2.25.1
Hi Hackers,
Because of 3 months have passed since Pluggable Toaster presentation and a
lot of
commits were pushed into v15 master - we would like to re-introduce
this patch
rebased onto actual master. Last commit being used -
commit 641f3dffcdf1c7378cfb94c98b6642793181d6db (origin/master)
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date: Fri Mar 11 13:47:26 2022 -0500
Updated patch consists of 4 patch files, next version (v2) of original
patch files
(please check original commit message from 30 Dec 2020):
1) 1_toaster_interface_v2.patch.gz
https://github.com/postgrespro/postgres/tree/toaster_interface
Introduces syntax for storage and formal toaster API.
2) 2_toaster_default_v2.patch.gz
https://github.com/postgrespro/postgres/tree/toaster_default
Built-in toaster implemented (with some refactoring) uisng toaster API
as generic (or default) toaster.
3) 3_toaster_snapshot_v2.patch.gz
https://github.com/postgrespro/postgres/tree/toaster_snapshot
The patch implements technology to distinguish row's versions in toasted
values to share common parts of toasted values between different
versions of rows
4) 4_bytea_appendable_toaster_v2.patch.gz
https://github.com/postgrespro/postgres/tree/bytea_appendable_toaster
Contrib module implements toaster for non-compressed bytea columns,
which allows fast appending to existing bytea value.
These patches also include 2 minor fixes made after commit fest presentation
1) Fix for freeing memory in case of new toasted value is the same as old
one,
this seems incorrect, and in this case the function just returns instead of
freeing old value;
2) Fix of data alignment for new varatt_custom data structure in building
tuples,
since varatt_custom must be aligned for custom toasters (in particular,
this fix is very
important to JSONb Toaster).
Thanks!
On Wed, Feb 2, 2022 at 10:35 AM Teodor Sigaev <teodor@sigaev.ru> wrote:
I agree ... but I'm also worried about what happens when we have
multiple table AMs. One can imagine a new table AM that is
specifically optimized for TOAST which can be used with an existing
heap table. One can imagine a new table AM for the main table that
wants to use something different for TOAST. So, I don't think it's
right to imagine that the choice of TOASTer depends solely on the
column data type. I'm not really sure how this should work exactly ...
but it needs careful thought.Right. that's why we propose a validate method (may be, it's a wrong
name, but I don't known better one) which accepts several arguments, one
of which is table AM oid. If that method returns false then toaster
isn't useful with current TAM, storage or/and compression kinds, etc.--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/
--
Regards,
Nikita Malakhov
Postgres Professional
https://postgrespro.ru/
Attachments:
1_toaster_interface_v2.patch.gzapplication/x-gzip; name=1_toaster_interface_v2.patch.gzDownload
�*9b 1_toaster_interface_v2.patch �;ks�H���W�x���y�4��������x�#v6B*��Bb�h�w���eV�������p3�2���U�Y��c��k>;���{!���S}-�����.z����@� ��E�tO�i�.���jI� sIV��l�,��B?�����O.{l:���k����oT��G��
�m �]v�-��Z�V�����R��$��IK:�H� ;�3%�i�V\�:��v��ay6����q��w�r��h��j[�c����'�&%��������v/������;�V��xj4�f���f2<X�ihM5>]�
����0�)���k6�)�e��Z
��=�f��MXJ
�-��
�T\G=_+���sEU�����ng[�����������|4����|=��<���7�������DWO1����i��i<E5��Sby1�vz��F��H-�%��BmNg3�Q4
|�M�7�U`�79�`v{$+P�bin�hBg�ES�����i��Ww���D�n!�L-O��l��Fs�j*kj��V��K�Ay��h��6i����T 4�E����l;C�������E�-5+Tno��s<X�b �1#��C�)�.�
BQV>�^q\OG�5_�8m�g�i���?��&�6-��=��9�^�q#�dbb�Ch�����f{/{�����DhB�l i���0��<#�tTE��s��e_2���3,`������f��a��]$�1�.�{�q{`�^
K5}�[@,����M0��'@����S&��UHF���xYf�P89hA8Na"�������k��"&�]>�<kH�������$��2�,�b�f�h�pE�������g�p�� ���4������"
Y �X�)��s���^���T0s�����f�`����@�Y�T
���s�|�6�_m�ry���d>��X�d`W}
P!����p#��6�p��<������6QJ+�u�.Q�����B��|��g��[�U���#5)hT!Gu(P$h����v�$��@�����@S�vL����8���L?���(f^rz)7+:�V��"�^�����oA���������Q�3��:3Tf� �C�m���X7���P��C����X<�P+�G�Q��'��.����9 ��|���.����LJ%mm�S�����^�=}q_,x��c��8Uo�A�P ��C8��k�����?_\��)����\����# e�>�@��a+��l�����5��d���s�7���
&A����Hu��~��\��X���k�������66 �X�6��/.�Z������N{1YV�s* ����d���gYB���(=�������nr=Y��,Ea��~Qv{���\��-����T�:�
�q9�n?-��<��n�� F��
Z�������� C�*�}���V���������\���o���{y����`�m6���������J��^+f@h0�������pf���b���"�*P��&���u����W�t��1�z�����X����������E�����bX�� {�.qm�Q�� ���N����y�-���B���h5!�e�k�Puk�G������)����V4�S1��_~�
Y�7���*�*���I�
�7�X�YE�_=.fK�)�FK�V���h�q6���-��F���������t����h��,���q4��Ks0
����d�"�k>�)���U+(��-�z��B"z���6{�M�����{J�[m��c�-���g�+��;���g��R�
���u�N�uF��`�������cl���U�nI���� �j��_��- ��~����AbMn���$
��|t�dA7��[3���2>S���G��i��cJ����hz3�V��`9$��A�$��T�B@�>�gQ�v�w�
��C8���v��#������\�Y���M�������r-����F�[�_~����]��zG�d��E�;e���v�f`��$��B6~Tv��@6������^�-W����J<F�@���d��/��-8`����]-"���p�,��(�a>�Ax�������o�yq�p�7�R�D����.�}I�v�3��@O�\� H�x�r���5,*��>�e#%�\����'3g�-D�-
^��|�Z��OY�m�����H�A u@����M�����g8E�9P�����>�z����$����D���rx�~����� �\O��[#�m�x���}/w�\C�g�tI�Ti�������.x�e{xi?S-��tt��,~�1IE�]&�����|������p� ���g�#��*��o����?",���6���8l��6T���$�3�Mqp�)�{���
�s��&�C�r��>J�v�w��D/z�fS����n��lS�B���HT�E����3�v�|����v�T0���a����n�N�b
�
8������lG�P6\R<�s����P��[1���=��r�7n��NR$PG��[7��P/)P�� wt=t���'
Y����b���M��^$Z��KE�0\�d|��[% �����Q���0�b�z�vZ}I�h6u�t��Y��ZL�V����X���Z_}��
;/�9����g��5��� d���}o�Q|���Y�&�W� � �o.�;������d��q`��������*����u��p|�\�@���
�����Y��=RDd�.�������(�\ tw���Z���r��r�������[��_�4��r�o!���,�y=� o�cG��c�����c����zA�� *�Pe���� ��9(������z�����u2P���W'�[C��g�kJ��&�oY�
g�� �wY��ph+ �1���� �\'(Qy��j�RI�?X�� �ye� ,��R1�C�o�*1�H �T+qZU�f�l���0��l��+D�@�|$�A����`�T��k�m--l]�H��D����P�w`���2?����$���8�(�&- Y�_��vS
����'�����D���8n����g��2����Zd>����
��TS������D���za��]#o�����m���eS��Y��u#��6�u�g��W�����X�m�\kJzA\��G��Y�P�������j �!R2�t�oB�J������"��G!`�E�$���s��.������;F�*"3���}�����V�j����h��0uFAWYf��9Y�j01aP(��+fQ���H���u��c��� ���.'�����v�Z������K-�"v8:$}Y��c�� ��7.&O&��<����;�D"�*,AfL�������W&<���*���J��b�������d��g��<����b����r�����6U����s7�0��r�����h�(��
^� 4�f��.=��"vh��j� �,\�\]7 �6_��n�1j�y�w�F�(�5 D�������o�:i��O�>~��b����|{��H�x/�;�IS��x�t�Nv��k��qC (��i�� l���`f�������� UG��:��1�4;��@)�8���{�T\7O��
���z��@��k�*p��L��bj��v����>�8*h4'�������oz�����~m��8�B�9[�)��DRc���z,ES���>F��#R����G�G�n�D�;Z,������%��`"g�A�����N���]��x��c���-`�u���b8��_��wNWx��9G�/����~P�����J�������+)�%r�L�z��`��mca���l��[]��Gw��X�F�5?�U�b8V�`��"���
kE5���A�~��@5����'�����&�������-�:0,��!�}v�sx)�?mc���H�"� {��X��j2���a. ����.:LD#�C�I�k�1����{��W,6R�����@3#�����Z�b��v2����u\�?TR*�|��~x#k�m����~�v��N��@��������p�z�/Wx�D����,.��PQ4m��1~�����T����,~g�Q��4�� �Om �P&(�G���D�@��>B��=���0_
���RqhH��XD]��T������\q�&� d�Rl�r����%�&��^4_p�A��#]�������N��VG��=������zx!���3d�#]����vX�c�M��T}�Y�\���o���=�'��WC�u�K%0BJD����M.����W����Og�K��&